Toward Actual Open Source Web Applications

29 May, 2006

In the twinkling buzzword constellation that constitutes the Web 2.0 firmament, "open source" is one of the brightest and most twinlky of stars. In Tim O'Reilly's original article announcing the concept, the phrase appears in no fewer than seven different (highly jargonic) contexts: (1) commodity servers running open source operating systems, (2) open content creation through peer-production methods, (3) open editing of collective knowledge repositories, (4) open architectures that build collective value automatically through users pursuing their own "selfish" interests, (5) users treated as a collective of co-developers, (6) software that remains in perpetual "beta" -- a state of constant development in which the product is never declared finished -- and (7) open access to data via RSS and APIs.

As O'Reilly tells the tale, contemporary web applications have interacted with the open source software community in just about every way possible: they've consumed its products, they've aped its architecture, and they've even co-opted its rhetoric and style. The one thing they haven't done is actually open source their own applications. And I'm beginning to wonder why not.

Why aren't there any websites that are truly open source? Why aren't there any websites that are actually written and maintained by a hierarchy of volunteers checking out and modifying liberally-licensed source code from a public repository?

Open source thrives in the context of commodity software that's needed by a large number of people, especially geeks, and for which branding and marketing provide little competitive advantage: think Linux, Apache, and Firefox. The web certainly has no shortage of applications that fit this bill (search and wikis come immediately to mind). So what are the practical obstacles preventing open source coders -- who've rushed eagerly into just about every other class of software -- from building web applications? What's the hold up?

One obstacle might be infrastructure. Where desktop and server software can be easily compiled and run locally on each individual contributor's computer while they work on it, mainstream web applications (e.g., Google) require extensive server hardware and voluminous bandwidth capacity far beyond what is available to the average web coder.

On the other hand, these need not be barriers to the development of such applications, merely their deployment. Contemporary development platforms, such as Ruby on Rails, come packaged with everything necessary to be built, tested, and run locally including a lightweight development server, a full-stack application framework, and a fully-integrated testing environment. With smart use of good testing practices such as mock objects and fixtures the run-of-the-mill personal computer can become a viable development platform even for large and complex web applications which require enormous quantities of data. And further, if such an application were designed in the emerging contemporary style as a series of independent modules which communicate via an API, the problem would be even further reduced, becoming mostly one of coordination and management like any other open source effort.

And the deployment problem is not necessarily intractable either. I see two potential solutions: the benevolent giant and the swarm of peons. Think of the enormous server and bandwidth resources devoted to open-in-spirit web applications by such non-profit organizations as the Wikimedia Foundation, and the Internet Archive. An entity at that scale could easily step up and provide the necessary infrastructure for an open source web application that was being written by volunteers. Further, such an organization would almost definitely have the clout and community mindshare to create such a project in the first place if they wanted.

The second potential solution is a little more out there. With just a dash of wickedly ingenious infrastructure mojo, the massive computing demands of a start-up open source web application could be distributed amongst its organizers, maintainers, and testers themselves. Using a shared computing architecture along the lines of Berkeley's SETI at home, the big chunks of data processing required by, say, the spidering and indexing capacity of our open source search engine, could be broken up into bite-sized bits and chewed up by a network of volunteer personal computers.

I'm sure each of us could think of a handful of other obstacles, both technical and social, facing anyone wanting to build a true open source web app. But I'm also sure that if the conversation got started, we'd just as quickly start coming up with solutions. That's the advantage of open source. As the old chestnut -- which I'm sure I saw quoted in O'Reilly's essay -- goes, "with enough eyes, all bugs are shallow".

Tagged: , , , , , , , ,