Backup a volume of your Amazon Instance

I have an amazon instance with 2 volumes. The first volume is the OS and the second volume is our database (postgres) and our files. So we decided to do daily backups of the second volume by taking a snapshot every night. I realize that there is some literature out there regarding the atomicity of the snapshot and whether you should freeze the file system while backing up etc. ect. I choose to ignore all that (or rather postpone for later) and do just plain vanilla snapshots on a daily basis.

Having learned that nothing is as simple as it seems, and since there were client sites on the my instance, I decided to create a new AMI and do my experiments on that - which proved to be a wonderful idea.

Creating programatically a snapshot is dead simple. You just run the following:

ec2-create-snapshot vol-123456 -d "my brand new snapshot"

...I wish!

My AMI did not have the ec2-api-tools installed which meant that none of the amazon commands meant anything to it. To cut a long story short, and save you some time there are a few steps you need to do before installing the tools:

Enable multiverse! Got to /etc/apt/sources.list and change the word universe to multiverse. You should end up with something like:

deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid main multiverse
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid main multiverse
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main multiverse
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main multiverse
deb http://security.ubuntu.com/ubuntu lucid-security main multiverse
deb-src http://security.ubuntu.com/ubuntu lucid-security main multiverse

Now run:

sudo apt-get update
sudo apt-get install ec2-api-tools

Catch for t1.micro instances! I did all of the above but my instance just froze as I run one of the above commands. Apparently there is a bug on ubuntu. The only way around is to run a m1.small instance rather than a t1.micro. This is a pain but if you can afford a tiny amount of down time then you are fine. So shut down the instance and upgrade:

ec2-modify-instance-attribute --instance-type m1.small i-123456mmm

Start your beefed up server and run again the update and ec2-api-tools install commands, and then shut it down and reverse the beefing up!

ec2-modify-instance-attribute --instance-type t1.micro i-123456mmm

You should be there now! Here is the juicy part: Create a ruby executable which will perform the backup. The file can look something like that: (make sure that you do a chmod 755 my-backup-script.rb in order to be able to execute it)

# daily-backup.rb
instance = "i-123456mmm"
dbvolume =  "vol-123456"
resp = `ec2-create-snapshot -K /home/ubuntu/.ec2/pk-YOUR_PEM_FILE.pem -C /home/ubuntu/.ec2/cert-YOUR_CERT_FILE.pem '#{dbvolume}' -d "development daily db backup $(date)"`
`logger '#{resp}'`
`logger Snapshot Return Status '#{$?.to_i}'`

You should open the crontab file (crontab -e) and make an entry for running the script.

1 1 * * * /path/to/scripts/daily-backup.rb

Go to bed and tomorrow morning your server will have baked a beautiful little snapshot of your volume!

P.S. You must make sure that you have copied your certificate and private key on your instance. You should also set some environment variables. Very briefly:

In your .profile or /etc/profile.d/mycustomesettings.h add:

export EC2_HOME=~/.ec2
export EC2_PRIVATE_KEY=`ls $EC2_HOME/pk-*.pem`
export EC2_CERT=`ls $EC2_HOME/cert-*.pem`

I prefer to keep the certificates in the ~/.ec2 directory. Notice that although I have set the environment variables in my profile, I still provide in my ruby script the full path for the key and certificate. This is because when the cron job runs the environment variables are not available.

comments powered by Disqus