Adam Howitt's Blog

Virtually Home

This site has been shifted around over the years from host to host and so it has moved again. We're up on Amazon's EC2 servers for much better stability.

I had previously been on a dedicated VPC solution from a small hosting company but kept running into issues with bad VPS nodes, Windows web server attacks replacing operating system files and a general lack of stability. So I'm back to running the boxes myself so if we're down, it's my fault.

New Focus

I'm hoping to spend more time talking about the technical aspects of the work I do in my consulting business Adam Howitt Consulting, specifically our iPhone and iPad development services. I'll be talking about some of the hurdles involved and showcasing some of the projects. I've always been a big believer of sharing the solutions to development challenges I face as I go along. It's not necessarily going to be 100% original Adam Howitt code but the blog posts will typically combine all the research I did to fix something into a consolidated tutorial you won't find anywhere else.

Q&A

I would also like to open the floor to anyone with iOS development questions. If you need help I'd like to incorporate a mailbag blog post theme each week answering someone's questions publicly for all to learn from so feel free to email me through the contact form at AHC if you want me to answer a question.

All that said, enough dreaming and let's get down to business! The next blog post will talk about the recently launched ForestXplorer iPhone app I built for the Forestry Commission of England. I've also just wrapped an ad-hoc iPad app build for Coors Light but I'll need to figure out how much or little I can say about that since it's not live yet...

Run a Linux process on a remote server and disconnect terminal session

I've been writing some nifty python scripts lately as we (WalkJogRun) migrate our route data from XML files to Google Maps encoded polylines. I finally whipped the script into shape and started pulling our route data from Amazon S3, encoding as SQL update statements and then rinse and repeat for nearly 500,000 legacy XML files. Trouble is, 500,000 files on my local machine would take about 20 hours.

Amazon AWS to the rescue! I span up a new Amazon EC2 medium size instance (high CPU) and pushed my script up to the cloud. The beauty of this is that the bandwidth between EC2 and S3 is free whereas my local machine relied on a crappy network connection and incurs the bandwidth charge on our S3 account. Ordinarily S3 bandwidth is pretty cheap but the script running on our EC2 machine is pulling down 5gb of data per minute and processing 1,000 files per minute. In around 8 hours we should have every legacy file processed and converted to a SQL statement we can run against our MySQL database.

But wait. My terminal timed out. Doh. If I login to the server over terminal and start the python script it is connected to my shell so if the connection breaks (on my crappy internet connection) the process is terminated on the server. The batch isn't as smart as it could be so it doesn't gracefully restart where it left off so I called on my friendly linux whiz Scott Frazer for a solution.

Scott directed me to the NOHUP command and after a few minutes of reading, a little typing and restarting my first batch I was able to disconnect my session thanks to this helpful command:

Roughly, nohup says don't hangup when I do but because my script outputs to the terminal as it runs the rest of the output needs to be redirected to files instead of the console.

If you run this command, exit and log back in you'll see the PTS/1 replaced by ? when you run the ps -fe command. I'm able to track the progress of my script because each iteration/file writes a line to the sql file it generates and the handy wc -l command tells me how many lines are in that file.

So thanks to Scott I'll have all my new shiny encoded polylines for our database and we can start to eliminate the need for the legacy XML files. The background there is that Google Maps on the iPhone and in the browser both support polyline encoding such that instead of a large xml file of latitude and longitude coordinates (some as big as 40kb!) I get an encoded string I can insert into the database each no more than 1000 characters.

WalkJogRun iPhone Application v1 is live!

After over 200 hours of development, sweat, tears and late nights after work, Apple finally approved our app. It's available in the App Store today and I'd love to hear your feedback.

Under the hood it connects to WalkJogRun.net built on ColdFusion and MySQL, hosted at Amazon, to pull routes from our database. We've grown to over 400,000 routes in over 200 countries and growing every day.

Learn more here or download a copy here.

Elastic IP addresses for Amazon

Amazon launched elastic IP addresses today for EC2. Elastic IP addresses allow you to assign a static ip address to your Amazon instance which makes it far more suitable for hosting your own sites now.

Prior to the launch of elastic IP addresses, your EC2 instances would be assigned an IP address from the pool that was not guaranteed to remain under your power. So if your instance crashed and you had to bring up a new instance, you get a new IP address and, if you were hosting a site on the old IP would now have to login to your domain name provider and update the A NAME record to the new address. PITA.

I just cut WalkJogRun over to an elastic IP address and it was far less painful than I imagined. The process for using elastic IP addresses is as follows:

• Start an Amazon EC2 instance if you don't already have one
• Request an IP address using the latest version of the EC2 tools and the ec2-allocate-address command
• Assign that IP address to an instance id (i-something)
  ec2-associate-address -i i-b3e019da 75.101.157.145
• Rush over to GoDaddy or wherever your DNS is recorded and change the A NAME to point at this new address.

Read the feature announcement here

PASV mode FTP on Amazon EC2 with VSFTP

I've just spent the better part of an hour bashing my head against an FTP problem with Amazon EC2 so I thought I'd share the answer.

The problem was that active mode FTP kept crashing when I tried to use Beyond Compare to compare my local machine to my Amazon EC2 hosted  site.  I would get

when using active and frequently saw suggestions to use PASV mode instead.

PASV mode on Ubuntu's default VSFTP installation is enabled by default but the default is to use any available pair of ports to handle passive mode.  Since Amazon locks down all ports except those you open manually my requests were being denied. I don't want to open up all ports on the server for the potential that a client tries to connect on any port.

You can force VSFTP to use a set range of ports for passive support by editing your /etc/vsftpd.conf to specify a port range.  I've used 15393-4 here:

Restart VSFTP

Open your Amazon EC2 firewall ports as specified. I do this using the firefox plugin EC2UI to accept the range 15393-4.

Lastly go to beyond compare Tools > Options > FTP > Firewall / Proxy and check the Passive Mode box.

Tada!