Adam Howitt's Blog

Moving your static assets to Amazon S3

If you need a cheap host for your podcast, pdf files, site images or any of your static site assets, take a look at Amazon S3.  I've been using it since just after it launched and it's a great way to reduce bandwidth to your ColdFusion/ Application server.  For WalkJogRun my average monthly bill is just under $6, not bad for a site that has seen 27 million hits this last year.  I'll give you a quickstart guide to for using it with your site and at a later date I'll describe how to use S3.cfc to integrate S3 into your ColdFusion application to allow you to push files from your site visitors up there too.  You should be up and running in about 10 minutes or less.

To use it for your site you will need to setup an Amazon S3 account (http://www.amazonaws.com).  Once you create an account and join the Amazon S3 service, you will be shown an access key and a secret key.  Copy these down.  Before we get into copying files, you might want to add a CNAME host "s3" to your DNS records to point to S3.amazonaws.com.  This means when you are done copying files the DNS records may have been refreshed and you'll be ready to rock.  Adding a CNAME as described will make it possible for you to refer to the files you put up on Amazon S3 with http://s3.yourdomain.com/myfile.jpg for example.

To start uploading your files in the meantime:

1. Download S3Fox for firefox https://addons.mozilla.org/en-US/firefox/addon/3247
2. Restart firefox when it is done
3. Go to Tools>S3 Organizer
4. Where it says Manage account enter an account name
5. Access Key is the access key you got when you registered
6. Secret Key is the other part from when you registered
7. Click close and you should see an empty directory on the right and your local filesystem on the left
8. Right click on the right panel and click "create directory"
9. The directory should be the CNAME you created in the DNS earlier.  So if your site is walkjogrun.net and the CNAME is s3, you should create a directory called "s3.walkjogrun.net".  Note that at the root of S3 your directory names have to be unique across the whole of Amazon's S3 userbase.  This is how they are able to do the CNAME virtual hosting.  As long as you used your own domain name and not mine, you should see a folder created when you hit OK
10. Double click the new directory to go one level down.  This is the root used for your new subdomain so if I create a directory called images I can access them with http://s3.walkjogrun.net/images

A note on terminology is important here.  I have used the term "directory" because that is the level of abstraction provided by s3fox to make it look like an FTP service.  Amazon calls any directories in the root "buckets", and buckets are the things that have to be unique.  Anything lower than the root in a sub-directory doesn't have to have a unique name.  The technical reason for this is that, again, S3fox abstracts the complexity.  In reality, every file you upload becomes a row in a big database as a key and a value.  The key to each file you upload is the unique bucket name concatenated with a string, so by this construction each filename becomes unique too.  It's important to know this if you are going to go one step further and use S3.cfc but for now it's just a bit of extra info.

To add files to the images directory:

1. Double click on the folder to go into it
2. Browse in the left panel to the files you are adding
3. Highlight the file you are adding and either right click and say upload or select multiple files and folders and hit the right arrow in the middle of the two panels
4. Once you see the files have all uploaded below, go up to a directory to the /s3.yourdomain.com/ directory
5. Right click on the directory and hit "edit acl"
6. Ensure that "everyone" has read permissions and check "apply to subfolders"

Any file uploaded to the images directory would then be available under s3.yourdomain.com/images/myimage.jpg etc.  You can right click any file in the right panel and select "Copy URL to clipboard" then paste it into your browser to see if it worked.  

That's it!  Other uses for the service are that you can backup files off your laptop or use it to host large design proofs that would take forever on your local machine.  Did I tell you how fast it is?  WalkJogRun images load 10X faster from Amazon under load than off my humble 10Mb/sec cable connection!