{"id":51,"date":"2015-01-25T02:07:02","date_gmt":"2015-01-25T07:07:02","guid":{"rendered":"http:\/\/packetlost.com\/blog\/?p=51"},"modified":"2017-02-07T21:40:52","modified_gmt":"2017-02-08T02:40:52","slug":"backup-your-ec2-amazon-linux-wordpress-blog-to-s3","status":"publish","type":"post","link":"https:\/\/packetlost.com\/blog\/2015\/01\/25\/backup-your-ec2-amazon-linux-wordpress-blog-to-s3\/","title":{"rendered":"Backup your EC2 Amazon Linux WordPress Blog to S3"},"content":{"rendered":"<p>So I finally decided to run my own Linux server and utilize the AWS free tier for a year.<\/p>\n<p>It was a great learning experience and I wanted to share the most difficult part of the process, backing up my new blog to S3. Automatically of course.<\/p>\n<p>I had just finished configuring my sever how I wanted. I followed these great guides I found on the net to get me up and running.<\/p>\n<ul>\n<li><a href=\"http:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/install-LAMP.html\" target=\"_blank\">Tutorial: Installing a LAMP Web Server on Amazon Linux<\/a><\/li>\n<li><a href=\"http:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/hosting-wordpress.html\" target=\"_blank\">Tutorial: Hosting a WordPress Blog with Amazon EC2<\/a><\/li>\n<li><a href=\"http:\/\/cafeandrew.com\/archives\/2339\" target=\"_blank\">Setting up an FTP server on your Amazon Linux AMI<\/a><\/li>\n<\/ul>\n<p>After I wrote a few posts and configured some plugins on this here blog it was time to figure out how to automate Linux. Something I have never done before.<\/p>\n<p><strong>Step 1) Generate a script to take backups of my site.<\/strong><\/p>\n<p>This wasn&#8217;t easy, and took a few hours of my time. Over an hour of which was finally tracked down to starting my .sh file on a windows system (using notepad++). Apparently the carriage return character on Windows and Linux is different and there was something in this file that made all my files get generated with &#8216;?&#8217; in the file name. When I tried to download the files being created by the backup script in WinSCP I was greeted with invalid file name syntax errors. It wasn&#8217;t until I ran the bash script with sudo that an prompt appeared upon file deletion showing me &#8216;\\r&#8217; was in the file name and not a question mark.<\/p>\n<p>Once I FINALLY tracked down the <a href=\"http:\/\/www.linuxquestions.org\/questions\/linux-newbie-8\/why-is-%5Cr-appended-to-my-filename-and-does-it-matter-885300\/\" target=\"_blank\">root cause<\/a> of my file creation issues I was off to the races. Thankfully during all this I got the hang of Nano (after admitting temporary defeat learning VIM) and was able to easily create a new shell script file from the ssh window and get my script working. Below is the code I ended up with. Mostly based off <a href=\"http:\/\/lifehacker.com\/5885392\/automatically-back-up-your-web-site-every-night\" target=\"_blank\">this LifeHacker article<\/a>.<\/p>\n<p><strong><em>Actually starting Step1:<\/em><\/strong><\/p>\n<p>So here is what you need to do to configure automatic WordPress backups to S3. My approach is to backup weekly and keep 1 month of backups on the server and 90 days of backups in S3.<\/p>\n<p>I started off by making a \/backups and \/backups\/files directory in my ec2-user home directory. This folder will hold my scripts and backup files going forward. This is the directory you will be in by deault after you SSH into an amazon linux instance as ec2-user.<\/p>\n<pre class=\"lang:sh decode:true \">mkdir backups\r\ncd backups\r\nmkdir files\r\nnano backups.sh\r\n<\/pre>\n<p>With Nano open, copy and paste the below code into nano. Then press Control+X to save the file.<\/p>\n<pre class=\"lang:sh decode:true\" title=\"backups.sh\">#!\/bin\/bash\r\n\r\n#this is a variable to hold the date and time to be used in the files\r\n_nowvar=`date +\"%m_%d_%Y\"`\r\n\r\n#mysql backup command - you can utilize the same database user that you configured wordpress with\r\nmysqldump --user=databaseuser --password=databasepassword databasename | gzip &gt; \"\/home\/ec2-user\/backups\/files\/dbbackup_$_nowvar.sql.gz\"\r\n\r\n#tarballs the entire apache directory\r\ntar czf \/home\/ec2-user\/backups\/files\/sitebackup_SITENAME_$_nowvar.tar.gz \/var\/www\/html\/\r\n\r\n#deletes any files over 28 days old\r\nfind \/home\/ec2-user\/backups\/files* -mtime +28 -exec rm -f {} \\;<\/pre>\n<p>Once the backups.sh file is created, we need to give it execute privileges.<\/p>\n<pre class=\"lang:sh decode:true\">chmod +x backups.sh<\/pre>\n<p>Now we can run it to make sure it works with bash. Or move right onto scheduling it to occur automatically as a cron job.<\/p>\n<p>Checking it with bash:<\/p>\n<pre class=\"lang:sh decode:true\">bash backups.sh<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>Step 2) Configuring the script to run automatically<\/strong><\/p>\n<p>Scheduling it with cron:<\/p>\n<p>First things first for me, scheduling cron jobs is done with crontab. Crontab&#8217;s default editor was VIM which is very confusing to a Linux novice such as myself. Lets change the default crontab editor to nano&#8230;<\/p>\n<pre class=\"lang:sh decode:true\">$ export EDITOR=\"nano\"<\/pre>\n<p>And now lets configure our backup shell script to run Sunday mornings at 12:05 AM EST (0505 UTC).<\/p>\n<p><a href=\"http:\/\/kvz.io\/blog\/2007\/07\/29\/schedule-tasks-on-linux-using-crontab\/\" target=\"_blank\">A great guide is found here<\/a>.<\/p>\n<pre class=\"lang:sh decode:true\">crontab -e\r\n\r\n05 05 * * 0 \/home\/ec2-user\/backups\/backups.sh\r\n<\/pre>\n<p>Don&#8217;t forget to Control-X to have nano save the edited crontab file. It appears as Amazon Linux automatically elevates to sudo to accomplish crontab changes because I configured everything without sudo.<\/p>\n<p>Now our site is backing up automatically. So lets offload these backups to S3.<\/p>\n<p><strong>Step 3) Syncing the automatic weekly backup files to S3<\/strong><\/p>\n<p>H\/T to <a href=\"https:\/\/www.webniraj.com\/2013\/03\/24\/using-amazon-s3-to-backup-your-server\/\" target=\"_blank\">this helpful blog post for guidance<\/a>.<\/p>\n<p>Create an S3 bucket. Then create an IAM user, assign it to a group, and give the group the following policy to restrict it to only having access to the new bucket. Replace the bucketname as needed.<\/p>\n<pre class=\"lang:java decode:true\">{\r\n  \"Version\": \"2012-10-17\",\r\n  \"Statement\": [\r\n    {\r\n      \"Effect\": \"Allow\",\r\n      \"Action\": [\"s3:ListBucket\"],\r\n      \"Resource\": [\"arn:aws:s3:::YOURNEWBUCKET\"]\r\n    },\r\n    {\r\n      \"Effect\": \"Allow\",\r\n      \"Action\": [\r\n        \"s3:PutObject\",\r\n        \"s3:GetObject\",\r\n        \"s3:DeleteObject\"\r\n      ],\r\n      \"Resource\": [\"arn:aws:s3:::YOURNEWBUCKET\/*\"]\r\n    }\r\n  ]\r\n}<\/pre>\n<p>Or you can just use your root IAM credentials, whatever floats your boat.<\/p>\n<p>Next up, install s3cmd onto your Amazon Linux instance. While s3cmd is very useful, its third party developed and not an actual Amazon command line feature, so we have to download it from another repository. We can install s3cmd onto an Amazon Linux instance with the following command.<\/p>\n<pre class=\"lang:sh decode:true \">sudo yum --enablerepo epel install s3cmd<\/pre>\n<p>You will have to accept some certificate prompts during the install.<\/p>\n<p>Once s3cmd is installed we can configure it with our IAM credentials. Don&#8217;t worry, with proper restricted IAM credential setup it will fail the configuration check at the end.<\/p>\n<pre class=\"lang:sh decode:true\">s3cmd --configure<\/pre>\n<p>Create a shell script to sync our backup files to s3. Make sure we are still in the backups directory and use nano to create the script.<\/p>\n<pre class=\"lang:sh decode:true\">nano sendtos3.sh<\/pre>\n<p>Paste the following code into nano and press Control-X to save. Don&#8217;t forget to change the bucket name.<\/p>\n<pre class=\"lang:sh decode:true\">#!\/bin\/bash\r\ns3cmd sync -v \/home\/ec2-user\/backups\/files\/ s3:\/\/YOURNEWBUCKET\/files\/ &gt; \/home\/ec2-user\/backups\/s3_backup.log 2&gt;&amp;1 &amp;\r\n\r\n<\/pre>\n<p>Configure the script to be executed.<\/p>\n<pre class=\"lang:sh decode:true\">chmod +x sendtos3.sh<\/pre>\n<p>Now you can use some of the steps above to execute the script manually to make sure it works or schedule the script to run a few minutes after the backup script via cron.<\/p>\n<p>You can check the logfile with tail for more information.<\/p>\n<pre class=\"lang:sh decode:true \">tail \/home\/ec2-user\/backups\/s3_backup.log\r\n<\/pre>\n<p><strong>Wrapping it up<\/strong><\/p>\n<p>At this point you should have your compressed WordPress database backups and compressed Apache files being created weekly. Then they are being synchronized to S3 shortly after. What if we want to keep the files on S3 longer than the files on the server?<\/p>\n<p>All we need to do is enable versioning on the bucket. Then apply a lifecycle policy to permanently delete previous versions after 60 days. Now we have 90 day backup retention.<\/p>\n<p>Anyway, I hope this helps. I tried to link to all blogs that helped me get up and running.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>So I finally decided to run my own Linux server and utilize the AWS free tier for a year. It was a great learning experience and I wanted to share the most difficult part of the process, backing up my &hellip;<\/p>\n<p class=\"read-more\"><a href=\"https:\/\/packetlost.com\/blog\/2015\/01\/25\/backup-your-ec2-amazon-linux-wordpress-blog-to-s3\/\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,23],"tags":[27,25,24,28,26],"class_list":["post-51","post","type-post","status-publish","format-standard","hentry","category-amazon-web-services","category-linux","tag-backup","tag-linux","tag-s3","tag-shell-scripting","tag-wordpress"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/posts\/51","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/comments?post=51"}],"version-history":[{"count":6,"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/posts\/51\/revisions"}],"predecessor-version":[{"id":57,"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/posts\/51\/revisions\/57"}],"wp:attachment":[{"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/media?parent=51"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/categories?post=51"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/packetlost.com\/blog\/wp-json\/wp\/v2\/tags?post=51"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}