Developing Single Serving Sites using Ruby CGI scripts on Dreamhost

1 March, 2008

There's been a lot of hullabaloo lately about Single Serving Sites. Stimulated by the inexplicable runaway success of Barack Obama is Your New Bicylce, these simple sites that provide a small dollop of amusement (isitchristmas.com) or utility (istwitterdown.com) have become all the rage.

Of course, I've been making them for awhile now, e.g. Largehearted Goat and The NY Times Explains the Ratings. At first, creating a new SSS is a satisfying experience. You have a whacky idea and withing a few hours, you've registered a domain, written some simple code, and put something up. But, over time, each one gets to be more and more of a burden. My ideas, at least, have involved sites that need to be constantly updated with new information over time. Since I've always implemented these sites with simple ruby scripts that run on my local machine and then upload the static finished versions of the sites, this has meant keeping an eye on unreliable cron jobs and, sometimes, hand maintenance. And, over the years, I've wondered if there was a better solution.

Today, I took the first steps towards finding one. It turns out that good-ole CGI scripts — so foreign to those of us who's main experience has taken place in the age of sophisticated web frameworks like Rails — make a great basis for SSS development. What follows is a basic introduction to writing and running CGI scripts with Ruby. I'll focus on Dreamhost as a deployment target since it's the service I have access to for this kind of thing and also the issues that arise there are probably not that dissimilar from what comes up with the other shared hosting services that are the natural habitat of SSSes.

Step One: Get Ruby and Rubygems up and running

If you plan on keeping your SSSes extremely spartan simple, you might be able to skip this step. Dreamhost accounts come with an old version of Ruby already installed. If you don't need to install any custom gems for your scripts or control anything else about your ruby environment, you can skip straight down to the step two. However, if you plan on doing anything the least bit more sophisticated — from installing one individual gem all the way up to using Rails itself — you've got a bit of work to do beforey you can get started on the fun part.

Being far from an expert on unix build process and dependencies, I followed Nate Clark's excellent instructions for building Ruby and Rubygems on Dreamhost with just a few discrepancies. My biggest note is that the version of the Ruby source to which he links is out-of-date. I changed his line for download the Ruby source to:

$ wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6-p111.tar.gz

But, don't trust me! The most recent version of Ruby changes all the time and you can always find the most recent version of Ruby at ruby-lang.org. The one other thing worth noting that I discovered was that in any of the compilations make may die by timeout (Dreamhost kills processes that last longer than a given interval, a major pain point we'll be returning to in a moment when we try to install individual gems). It won't always be clear that a timeout was at fault so if you see a 'configure', 'make', or 'make install' command die mysteriously, just go ahead and try again.

Step Two: CGI Hello World with Ruby

Now that we've got Ruby and Rubygems installed, the first thing we want to do is to just run a basic 'hello world' CGI script to make sure that we've got things configured correctly. Here are the steps:

  1. Log into dreamhost over ssh.
  2. Navigate to the desired directory within your web space.
  3. Create a new file, e.g. test.cgi.
  4. Fill it with the following content (if you skipped Step One and are using the build-in DH Ruby, the first 'shebang' line should read: "#!/usr/bin/ruby" instead of what is shown):

    #!/usr/bin/env ruby
    require "cgi"
    cgi = CGI.new("html3")
    cgi.out("text/plain"){
    "Hello World #{Time.now}"
    }
  5. Change the permissions of that file by running:

    $ chmod 775 test.cgi

That should do it. Navigate to that path in your browser and you should see output that looks something like this:

Hello World Sat Mar 01 19:04:53 -0800 2008

Step Three: Install Gems by Hand

Now, if you plan on doing anything more interesting than displaying the date, the odds are you're going to want to take advantage of Ruby's great and growing weatlh of libraries made available as Gems. Unfortunately, the RubyGems server system running at RubyForge has lately become incredibly slow. In order to find packages and detect dependecies, the default 'gem install' command has to communicate with that server and so becomes subject to the ruthless Dreamhost killing of long-running processes. In my experience, trying to install gems on Dreamhost will universally end in frustration:

$ gem install feedalizer
Bulk updating Gem source index for: http://gems.rubyforge.org
Killed

What to do?

Fortunately, in addition to using the RubyForge server to search for gems, it's possible to install them directly from .gem files if you can find them for your desired packages. For example, when building a recent SSS I wanted to use Feedalizer — a great library that scrapes web pages and automatically creates RSS feeds from the resulting content — so I hunted down the .gem file via the RubyForge website, downloaded it, attempted local installation, and then (when that failed) chased down the sources for the dependencies (in this case just Hpricot) to follow the same process. To give you a taste of things here were the gory details:

Installing Feedalizer:

$ wget http://rubyforge.org/frs/download.php/13797/feedalizer-0.1.0.gem
$ gem install feedalizer-0.1.0.gem

which threw an error complaining about the need for the Hpricot dependency, so:

$ wget http://code.whytheluckystiff.net/dist/hpricot-0.5.140.tgz
$ tar xzf hpricot-0.5.140.tgz
$ cd hpricot-0.5.140
$ rake gem
$ gem install pkg/hpricot-0.5.gem

The 'rake gem' step there is necessary because _why only makes the source directory available for direct download rather than an explicit .gem file. After this completes successefully, return to the 'gem install' step for Feedalizer above (be sure to the return to the directory into which you dowmloaded the Feedalizer .gem before running it).

Much bumpier than the smooth ride normally provided by 'gem install', but it'll get the job done. You could clamber your way down similar rutted paths for most any other gem you needed for you script.

And that should be more than enough to get you started. If you run into any troubles, dive into the comments here and on the other blog posts linked for help. Also, if anyone has any experience using lightweight persistence strategies in this context, I'd love to hear about them, espcially if they're file-system only; a lot of my scripts requiring saving records and I tend to use a rickety YAML-based system for them that could stand improving.

Note: Thanks to The Pug Automatic for inspiring me to start down this path in the first place.

Tagged: , , , , ,