Duplicity Back Up To Amazon S3

Duplicity Backup to Amazon s3

Assumes: optware installed, SSH access, Amazon s3 account and s3 keys

  • Install python, duplicity, gpg
ipkg install python26  
ipkg install py26-duplicity 
ipkg install py26-boto
ipkg install gnupg

These next steps taken from How To Install Duplicity On QNAP NAS

  • Crucial! Create link to gpg
ln -s /opt/bin/gpg2 /opt/bin/gpg
  • Create GPG key on your main computer (not on the MyBook). Default options. Remember your passphrase
gpg --gen-key
gpg --list-keys
gpg --output mykey_pub.gpg --armor --export <key no>
gpg --output mykey_sec.gpg --armor --export-secret-key <key no>
  • Copy key files to the MyBook (e.g. Filezilla with SFTP connection) and import & trust (full trust)
gpg --import mykey_pub.gpg
gpg --import mykey_sec.gpg
gpg --list-keys
gpg --edit-key <key no>

gpg> list
gpg> trust
  • Make sure to back up GPG key files (including /root/.gpg), as these are needed to restore backups..
  • Many scripts are available for duplicity backup to s3, here is a simple one. You probably need logging and a cron job.

Update: Feb 2013

New script and guide available: http://www.x2q.net/blog/2013/02/24/howto-backup-wd-mybook-live-to-amazon-s3-and-glacier/


# Export some ENV variables so you don't have to type anything

export AWS_ACCESS_KEY_ID=<your access key>
export AWS_SECRET_ACCESS_KEY=<your secret key>
export PASSPHRASE=<your GPG passphrase>
export PATH="/opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin"
export HOME="/root"

GPG_KEY=<your key id>

# The source of your backup

# The destination
# Note that the bucket need not exist
# but does need to be unique amongst all
# Amazon S3 users. So, choose wisely.

DEST=s3+http://<your bucket name>

duplicity --tempdir=/DataVolume/Temp --encrypt-key=${GPG_KEY}  --sign-key=${GPG_KEY} --s3-european-buckets --s3-use-new-style  ${SOURCE} ${DEST}

# Reset the ENV variables. Don't need them sitting around

So far, only tested with small backups - encrytion makes transfer fairly slow, and we might need duplicity throttling options or it may gab too much upstream bandwidth. Need to also test restore.

Update: Oct 2011

Intermittent error on backup
gpg: pkglue.c:41: mpi_from_sexp: Assertion ‘data’ failed.
..looks like a bug fixed in current version of gpg (2.0.18), awaiting optware update (running 2.0.17)

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License