Tutorial setup 2MBWE with rsync over the internet

This documents my project on setting up 2 mybooks that synchronise using rsync over the internet.

First off, why did I want this… easy, I want to make sure there is off-site copy of the contents of my pictures and documents. It's a off-site copy, a sort of backup. It should be a hand-off setup, once setup I should only be worried about it failing.


Some requirements I had for my solution:

  • runs every day, so the remote copy can fall behind maximum one day (under most circumstances)
  • synchronise during the night only, the mybooks must perform normal during the day
  • the setup needs local redudant data as well, meaning raid-1 is a must
  • each local PUBLIC share will be available for the network (and household) to right data to
  • the remote copy will be setup in a read-only BACKUP share (and read/write for a logged in admin or user)
  • only new files should be copied (minimum data) and deleted files should be removed from the remote copy
  • i want syslog rotated once a day, and mailed to me every day (I will read them from time to time to discover if something fails)
  • when there is a lot of data to be synchronised it is acceptable that it takes multiple days to synchronise
  • a synchronisation should not be launched when another one is running (so it should behave like a little daemon)
  • the internet is NOT safe, so data should be synchronised over a encrypted tunnel (IPSEC, VPN, SSL-VPN or SSH tunnel)
  • the internet is NOT safe, open ports should not have userid's and passwords to be bruteforced or scriptkiddies
  • the locations are physical seperated, providers might change over time, assume nothing about the IP setup of the ISP (dynamic of static), meaning assume that are NOT fixed IP's to work with
  • the only network thing I should need to setup in the network router is a portforward

The idea is to setup one disk at my home, and another mybook at another location at a relative (parents, brothers/sisters, work, whatever)…

So my solution is the following setup:

  • two mybooks, both configured in raid-1
  • each mybook has a unique name and uses a dynamic DNS setup (and should update automatically)
  • use of SSH with a sufficient keylength (»1024bits) to secure the data over internet
  • use rsync to efficiently copy data over the internet
  • remote copy should go to the BACKUP share

Let's get setting up, how I did this… Well, I started out buying two new mybooks MBWE 1TB version. They turned out to have the 2.0.15 firmware. First uppack mybook #1, connect it to the router. After connecting, and powering it up. You need to figure out that the IP address is (I personally hate the "setup" disk software that I need to install on my computer) of your new mybook. Just going to then network neighbourhood on my vista laptop worked fine (should work on a XP machine as well).

Initial setup of mybook

Ok, follow the normal procedure, go through the setup wizard on the webGUI (find it here: http://<ip-of-the-mybook/).
Remember to change the default password to something other then 123456 (defaults userid/password are: admin/1234356).

  1. Give the mybook a name like: MYBOOK1
  2. Add a shared folder to the setup: BACKUP (File Sharing - Add a Shared Folder - Begin Wizard)
  3. Add a user to the setup: <YOURFIRSTNAME> (File Share - User Management - Create User [BUTTON])
  4. Make sure that the <YOURFIRSTNAME> user has rights to Read/Write to the BACKUP fileshare
  5. Setup the disksetup of your mybook to: RAID-1 (Drive Management - Change Drive Type - Secure Volume RAID-1)

Setting up RAID-1 will take a while. So leave the disk alone, to check if it is done, you can check the status in the webinterface. You should get Drive Status: OK (top right corner).

Do this for both mybooks. Now you should have created two local mybooks, with unique names, mybook1 and mybook2 (this is what I will call them for the rest of this tutorial).

So far, it's pretty normal setup of a MyBook ;-) Now lets get hacking…

Hack and first steps

Enable SSH and setup Optware package system

Simply follow the instructions first-steps-with-mbwe. Do this for each of the mybooks, now you should be able to logon to the mybooks using a ssh enabled terminal emulator. (Windows users) I use Putty, find it here http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html Use the Installer version, this is the most complete version of Putty, with some usefull tools, like Pageant (PuTTY's authenication agent). This will be required later on, once you have setup the "no" userid/password functionality for SSH.

Also I would suggest you change the password of your root account on each mybook.

Setup Telnet as a backup

Since I am planning to disable the SSH userid/password logon later on, we want to make sure we have a second way into the box. So logon to your hacked mybook using PuTTY. From now on I will assume that you will sudo to root every time you logon, or you directly logon to the root account.

So you are in? Then go and edit inetd.conf to enable the Telnet daemon.

# vi /etc/inetd.conf

nano is simpler for newbies as a texteditor. Use nano -w to prevent automatic line wrapping from interfering with your code.

So change the:

#telnet  stream  tcp     nowait  root    /usr/sbin/telnetd       telnetd

Now remove the # in the front of telnet:

telnet  stream  tcp     nowait  root    /usr/sbin/telnetd       telnetd

Save your changes, now you have a working telnet daemon. I would you NEVER forward to port 23 from the internet router and do not put the mybook box in a DMZ configuration of your network router.

Dynamic DNS setup

To make sure that we can find each mybook over internet first thing I did was to setup a Dynamic DNS account. Personally I have used DynDNS with succes over the years, there are others like no-ip.com. Both services should work, and have FREE accounts available.
Pick one, go and register two hostnames. I will assume that you registered with example.org and I will assume you where able to register the hosts to (surprise suprise) mybook1 and mybook2.

O right, please use a very long password for your account. I suggest you generate one using GRC's Perfect Password page. I like Steve's password mentality, when using static passwords for this, they should be long, you should not be able to remember them… safe and secure. So make sure you save the password in a textfile somewhere.

So you now have a account, with a long passwords, and two host names registred (mybook1.example.com and mybook2.example.com). So next thing it to make sure that the mybooks each report their external IP addresses to the Dynamic DNS service. We will use inadyn for this. So lets go get the package from optware first.

# ipkg install inadyn

Next we need to setup inadyn so that it launces at reboot, so edit inittab to update the DDNS accounts.

# vi /etc/inittab

And find # Logging Junk and add the following line before that line:

::once:/opt/bin/inadyn -u <user account> -p <very long grc pass> --syslog --update_period_sec 3600 --forced_update_period 864000 -a <mybook hostname>.dyndns.org

The parameters mean the following:
Parameter Function
-u <your account name with DDNS provider> The username if applicable
-p <very long password created grc perfect password> The password if applicable
--syslog Forces the inadyn to write it's logmessage to the syslog /var/log/message file
--update_period_sec 3600 Make inadyn rerun every 3600 seconds, meaning once a hour
--forced_update_period 864000 Forces and IP update every 10 days (to prevent timeout on your DDNS account)
-a <hostname>.<url of DDNS provider> 1 A host name alias. This option can appear multiple times, for each domain that has the same IP. Discussion on the inadyn setup.

Make sure to do this on both mybooks. This setup step is complete. So to verify that it works, do the following. Give the reboot command on both mybooks. After reboot reconnect to the mybooks, and do a ps -ef|grep inadyn to verify it's running. Then logon to your DDNS providers page, and go into your account. Now you should see both mybooks reporting the same IP address to the service. This should match your external IP address. Find out your current external IP address as follows: http://checkip.dyndns.com/

Now when the IP address is changed (by your ISP), or your mybook is moved to another location (other network), or, moved to a different provider. It updates the dyndns hostname, so you can always find your mybook on the internet. Just thing what other things you can do with this someone steals the box from you, you could even wipe it remotely or even brick it from a remote location ;-) Or spy on the bastards that steal it… just make sure you change the default root password.

Logrotate setup

Good housekeeping for your syslog logfile. All programs log their message to the syslog daemon. This in turn writes them to the /var/log/message. This file will become very large after a while, instead of letting it grow I setup logrotate to rotate the logfile.

So first lets install the logrotate tool using optware, follow the install instructions here.

Next we have to setup a configuration for logrotate and run it on a daily schedule using crontab.

First the logrotate.conf need to be setup. This configuration will rotate the /var/log/message file once a week, and keep 5 copies on the filesystem. Meaning you can go back into the logfiles for about a month. Then after rotating the file it will kill both the syslog daemon and the klogd daemon. They will be respawned automagically.

So create/open the configuration file using the following command:

vi /opt/etc/logrotate.conf

Next up, the actual configuration file:


/var/log/messages {
        rotate 5
                /bin/busybox killall syslogd
                /bin/busybox killall klogd

include /opt/etc/logrotate.d

You can go to the logrotate manpage for more options. In one configuration file you can rotate lots of logfiles, also you could send the files using a mail command. This is not the option I choose, but it could be done.

Next we need to add a job to the crontab. For this to work you need to have cron installed. On certain mybook versions, cron is not installed by default. In that case see the page on cron before you continue.

Now open the crontab file for the root user.

vi /etc/crontabs/root

And add the following line of in the crontab file:

02 4 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf

This means that every night, at 4:02am the logrotate file is launched, and the configuration is used.

To activate the new crontab configuration you need to stop and start the crontab service:

/etc/init.d/crond.sh stop
/etc/init.d/crond.sh start

Adding SSH keys to automate login

So we want the mybooks to autologin using a long keyfile. Later on we will remove the option to log in using a password for the SSH setup (basically hardening the SSH setup).

Start on mybook1. We need to repeat this procedure on mybook2.

Let's move over to the special .ssh directory

cd ~
mkdir .ssh
cd .ssh

Next generate a SSH keypair:

ssh-keygen -b 4096 -N '' -t rsa -f id_rsa

This command generates a new long keyfile with NO passphrase, filetype RSA and filename id_rsa. This will take a couple of minutes, so please be patient with your mybook, it's a slow little computer you know. When done you should have two new files called id_rsa and id_rsa.pub. This is a SSH keypair, both a public key and a private key.

So now we copy over the public keyfile from the mybook1 to mybook2. Execute the next four commands.

ssh root@mybook2 'mkdir .ssh'
cat ~/.ssh/id_rsa.pub | ssh root@mybook2 'cat >> .ssh/authorized_keys'
ssh root@mybook2 'chmod -R go-rwx .ssh'
ssh root@mybook2 'chmod go-rwx ~'

The first command makes a new directory. The next command then adds the local public key to the authorized_keys file on the other mybook. The final commands adjust directory and file permissions so that nobody except the owner, root, can access or work with files in the .ssh and /root folders. SSH is very fussy about these permissions, and will fail without telling you why if it deems the current settings insufficiently secure.

Every ssh command above will make you log on to the box, just enter the root password you have set.

Now go back to the beginning, log on to mybook2 and repeat this procedure of generating keys and copying the public key over to mybook1. When you have keys generated on both boxes, and copied over the public key into the authorized_keys. Now you should be able to log on using ssh without the need to enter a password, instead the private key is used to authenticate the session.

Synchronizing the files over the net

Next step is to setup the synchronisation of your PUBLIC folder on mybook1 to the BACKUP folder on the mybook2 (and visa versa).

Creating the rsync script

So lets start with a little script I wrote, so that rsync will only run once. Even if this script is triggered multiple time and a rsync job is still running, then it will just die. If there is no job running, it will run. Ok, so lets create a file:

vi ~/sync_files.sh

Then copy and paste the script below into the file. Remember to hit 'i' first when using vi.



start() {
        if [ ! -e "$PIDFILE" ]
                touch $PIDFILE
                echo "Synchronizing files"
                $RSYNC -avz -e "$SSH" --delete $RUSER@$RHOST:$RPATH $LPATH
                rm -f $PIDFILE

stop() {
        if [ -e "$PIDFILE" ]
                killall rsync
                echo "Stopping filesynching"
                rm -f $PIDFILE

restart() {

cleanup() {
        killall rsync
        rm -f $PIDFILE

case "$1" in
                echo $"Usage: $0 {start|stop|restart}"
                exit 1

exit $?

Create this script on both mybook1 and mybook2. Modify the script so that the RHOST reflects the RemoteHOST hostname. At this time we will comment out (using #) the dyndns name. We first want to test the local setup in our local network.

Some information on the parameters used in the script:
1. the trailing slash
LHOST=/shares/internal/BACKUP/ with or without trailing slash.
A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name", but in both cases the attributes of the containing directory are transferred to the containing directory on the destination.
2. compression
Since the mybooks are not the fastest NAS devices you will find that if you setup these boxes are rather slow, using compression on these devices might not seem to help. Actually if you do some speedtests on your local LAN you will find that turning off compression (without the 'z' argument) in the rsync command it will actually be twice as fast. My thinking of leaving compression on is that the SPEED of your upstream datalink will far be lower then your speed hit you receive. The compression might even be worth it over a slow 1Mbit uplink)… I have no numbers at this time to prove this, so… if you do let us know.

Setting the eXecutable script

To finish creating the script, we need to set the eXecutable flag. So execute this command:

chmod +x sync_files.sh

Before setting up crontab you can test the rsync up. Just execute then next command:

~/sync_files.sh start

If all is well, then you will now see rsync be launched. If there is any contents in the PUBLIC folder you will see it being synced over the network.

Adding a cron job

So the next step is setting up a cronjob to make is synchronize during the night. Now of course this up to you, you can change this cronjob timing using the crontab manual.

So open the crontab of the root user:

vi /etc/crontabs/root

And add the following line of in the crontab file:

10 0-7,22-23 * * * /root/sync_files.sh start
0 8 * * 1-5 /root/sync_files.sh stop

So the first line will start rsync every day from 10pm through 7am the next day, at 10 minutes past the hour. And the cleanup is done on weekdays at 8am, by stopping the rsync.

Some thoughts on your sync schedule

Now the you will notice that on weekdays it will run during the night. And stop synchronizing during the daytime. Also it will run all over the weekend, starting at Friday night and keep going until Monday morning. Now, you really want to think about your optimal schedule. Another setup could be to sync on a hourly basis, and stop off all rsync ever other day. Just think and setup.

hardening the SSH configuration

So now our next step is to harden the SSH configuration, we do this to by editing the configuration file ((/etc/ssh_config}}.

#       $OpenBSD: ssh_config,v 1.19 2003/08/13 08:46:31 markus Exp $

# This is the ssh client system-wide configuration file.  See
# ssh_config(5) for more information.  This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.

# Configuration data is parsed as follows:
#  1. command line options
#  2. user-specific file
#  3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.

# Site-wide defaults for various options

 Host *
   ForwardAgent no
   ForwardX11 no
   RhostsRSAAuthentication no
   RSAAuthentication yes
   PasswordAuthentication no
   HostbasedAuthentication yes
   BatchMode no
   CheckHostIP no
   AddressFamily any
   ConnectTimeout 0
   StrictHostKeyChecking no
   IdentityFile ~/.ssh/identity
   IdentityFile ~/.ssh/id_rsa
   IdentityFile ~/.ssh/id_dsa
   Port 22
   Protocol 2
   Cipher 3des
   Ciphers aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc
   EscapeChar ~

Lets now configure the SSH configuration file in such a way that it will no longer accept a password. It should ONLY work with the rsa keyfile we created. These are the options that are used to set it up.
  • PasswordAuthenication set to no, so passwords are no longer used.
  • HostbasedAuthentication set to yes, so the RSA key will now be used to logon to the other computer.
  • StrictHostKeyChecking set to no, this way ssh will not complain when the hostname changes
  • CheckHostIP set to no, this way ssh will not complain when an up address has changed of one of the known hosts.
  • Protocol set to 2, this means that you can only use ssh version 2 protocol. No fallback allowed.

For more information check out the ssh_config manual.

After you changed this, you will need to reboot. The only way to login after this point is to use Telnet…. or by adding an extra rsa key to the authorized_keys file in the home directory. In case you use a Windows box and you use PuTTY, then you start using Pageant. This is the PuTTY agent, that keeps your keys for logging into mybooks.

Sending out the logfiles on a daily basis

You could use ssmtp, however if you are like me and want to use Gmail, then you need to setup msmtp
For some reason there is no smtp mail setup. So after some looking around I decided to setup msmtp2.

So lets use Optware to install msmtp, this works with Gmail and TLS.

ipkg install msmtp

One more package was needed that ipkg didn't solve the dependency for:

ipkg install libiconv

then run:


Now we will set up the smtp server you want to connect to, in this case Gmail.

Create the configuration file and edit:

vi /opt/etc/msmtprc  (optionally you can have user specific config in each home dir)

and add:

account gmail
host smtp.gmail.com
auth on
user *** change to yourgmailuser@gmail.com***
password ***change to password***
port 587
auto_from on
tls on
tls_certcheck off
logfile /var/log/msmtp.log
account default : gmail

Needless to say maybe, but you need to do this on both mybooks.

Setup daily mail with msmtp

Next step is to setup a script to sent the message logfile.

vi ~/logfile-message-sender-en.sh

This next script is based on the raid mail script. Anyway, I modified it a little, so that I can use it to sent out any logfile.


# Param 1: Name of the file to send to the recipiencts
cat >$message <<-EOF
Subject: Hostname `hostname`  Logfile: $1


if [ ! -e $message ]
        # cannot create a temporary file to hold an e-mail message
        exit 0;

    # compose an e-mail
        cat >>$message <<-EOF

        Hostname: `hostname`
        ---[ BOF Filename: $1 ]---
`cat $1`
        ---[ EOF Filename: $1 ]----


# e-mail all the people in the e-mail list file.
for address in `cat $recipients`
        cat >$tmpmessage <<-EOF
        TO: $address
        cat $message >>$tmpmessage

        /opt/bin/msmtp $address <$tmpmessage

if [ -e $message ]
        rm $message

Make sure the script can execute:

chmod +x logfile-message-sender-en.sh

Also setup the recipients list for the messages:

vi /root/mail-recipients

Every line in the fill can be a recipient that will be mailed.

Now we need to setup another cronjob. So open the crontab of the root user:

vi /etc/crontabs/root

And add the following line of in the crontab file:

01 4 * * * /root/logfile-message-sender-en.sh /var/log/messages

So every day at 4:01am the messages logfile. You can repeat this job for other logfiles, anything you want.

To activate the new crontab configuration you need to stop and start the crontab service:

/etc/init.d/crond.sh stop
/etc/init.d/crond.sh start

Again do this on both mybooks.

Rebooting from time to time

I know, linux is stable… it's not windows… blah, blah, blah… as a former Netware admin I know a server can run for more than a year without a problem. Also having experience with so Windows and Unix boxes I know that this is true for some setups, others need their reboot from time to time. Just to be safe I decided that in my setup the boxes will reboot once a week.

4 4 * * 0 reboot

This will reboot the mybook once a week, at 4:04am on sunday.

To activate the new crontab configuration you need to stop and start the crontab service:

/etc/init.d/crond.sh stop
/etc/init.d/crond.sh start

Moving to the final destination


Before moving out the "mybook2" to it's finaly location (whatever this means for you). I suggest you do some decent testing. What do I mean. Well, copy data to either PUBLIC directory. Then let it rest for a night, and check if it moved over to the PUBLIC directory.

Before moving the mybook, copy everything you want to the PUBLIC directory of mybook1. Let it sit, for a day to sync. Check the BACKUP the content of mybook2. In case you have lots of data to sync you do NOT want to delete the files before moving. This prevents an initial sync over internet.


If you testing has concluded succesfully it is time to move the boxes, and yes, to change the sync_files.sh setups on both boxes.
So go into the sync_files.sh and modify the RHOST lines. Comment out the localhost, and uncomment the DynDNS based RHOST line. Again do this on both systems, and shutdown the box you are about to move. Type:



Now before leaving you need to setup you router so it's forward port 22 to your mybook. This will enable ssh connections over internet to your mybook. In you do not know how you can do this, just goto Portforward.com. It explains it well, for most types of routers.

Remote control your PC

If you are moving the box a distance I suggest you setup a LogMeIn Account so that you can log backin to your PC from a distance. This way you can edit your local setup through telnet and even modify your router setup.

Getting in your car

Time to get the second mybook to its final location then. We are prepared now.
driving hunderds of miles to the final destination

Setting up the other side

So once you get to the final location, you setup the mybook. Test to see if it work. Now you need to logon to the box again, use telnet to login on the local network. Also check to see if you can copy files over to the mybook. And see if you can look into the BACKUP.

Again Portforwarding

Now modify the router setup at the destination so it's forward port 22 to the mybook2. This will enable ssh connections over internet to mybook2. In you do not know how you can do this, just goto Portforward.com. It explains it well, for most types of routers. We should be ready for synchronisation.

Some final checks

This should do the trick. To verify everything work, logon to the local mybook, using telnet.

To verify execute the following commands:


If setup correctly, you should see a sync in progress.

Also check to see if you can login using ssh… execute:

ssh root@mybook1.dyndns.org

You should now login to your homebox.

If you need to troubleshoot, you can login using LogMeIn to connect to your PC at home.


Ok, so this should do it. And feedback is welcome… need more information, just pm me. I will try to clearify whatever it is.

Moving SSH to a different port

After a while I got sick of the scriptkiddie hitting the box. So decided to move away from port 22. I should have known better, so what do you need to change. Two files needs modification:
1) changing services file
Change the ssh service (both the UDP and the TCP) to a port of your choice. Example could be: 12345
2) change the ssh_config file
Change the port setting to the same port you picked at step 1.

Now you need to go back to your routers, and modify the portforward settings to the selected port.

Then you are ready to test, just ssh from one box to the other.
This removes all script kiddied from you logs.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License