Hype Machine Download Script

24 January, 2006

Update: Soon after writing this post, I received a very nice email from Hype Machine's Anthony Volodkin asking me to take down access to the script I described here. He explained that since the streamed mp3s can't be fed through the Coral server the script represented a danger to his bandwidth bills and since its existence implied that he maintains permanent links to downloadable versions of the mp3s it could get him in legal trouble. As a big fan of the service, I decided to remove access to my script and make a few other alterations to this post. Volodkin also, very kindly, agreed to let me reprint some of his email to me, which I have done below the post itself. I think the whole episode just goes to further illustrate the chilling effect that the mainstream music industry's fear of digital distribution -- and their great willingness to use invasive legal tools to excise the most creative agents of that distribution -- has had on new and vibrant methods for promoting even their own products.

It seems that recently all the traditional avenues for finding out about new music have dissolved into uselessness. I honestly can't remember the last time I listened to commercial radio and even the memories I have of doing so without horror at what I found are starting to get fuzzy; Rolling Stone and its ilk are more likely to feature Keira Knightley or some paleozoic sixties artifact than anything that could conceivably be labeled new; MTV, last I had cable, seemed to have just about abandoned music altogether (even on their myriad alternative channels) in favor of ever more 'lifestyle' programming (i.e. nearly naked attractive twenty-somethings running around exotic locations or going on secret dates in fighter planes while their friends tell their parents where they hide their pornography).

So, what's left? More and more, what's left is: mp3 blogs. Just individuals out there talking about music, bravely posting songs they like despite the potentially quite serious legal consequences of the musicians' and labels' ungratefulness. There are a lot of them and a lot of the best music to make it in the last couple of years first broke on their pages. But their very diversity and number makes them hard to navigate and even harder to keep track of. So, along came Hype Machine, a site that aggregates the feeds of selected mp3 blogs offering a great compound RSS feed of all of the blogged tracks and a searchable archive that's starting to get pretty deep. It's a great way to squeeze a bunch of the frantic mp3 blog activity through one narrow channel so that you can keep your eye on it.

There's one problem though. Once the files drop off of the original blog's hosting, they disappear from Hype Machine too. . .sort of. You see, the governing logic of the mp3 blog ecosystem is the fear of reprisals from representatives of the bands that the blogs are promoting. I know, it doesn't make any sense, but it happens. Bands and labels issue cease and desist letters to the very people who are in the process of becoming the gate keepers of taste for music lovers of the internet generation. Just look at any mp3 blog, you'll always see the "If you are the rights holder for any of these songs and you want them taken down just contact us immediately. Everyone else: please buy music, don't steal" mea culpa.

So, everyone's trying to keep the music out there while avoiding legal trouble. This results in some strange situations. For example, on Hype Machine, even after an mp3 drops off the original blog the blog post will still show up in searches and you can still listen to the songs in an onsite flash player, but you can't download them. If this last statement set off your alarm bells, then you may have anticipated the point I'm about to make:

You see, for Hype Machine to make the songs playable in their flash player, they have to put mp3s of them online. It turns out they make copies of the mp3s as they pass through their site, mirror them on a Coral Distribution Network (basically a system for taking the bandwidth burden off of the original blogger: once somebody listens to the mp3 on Hype Machine, they make a copy of it and then serve it to the next ten thousand visitors themselves rather than sticking the original blogger with an outrageous ISP bill; it's only the polite thing to do), and then put them online at URLs of their own, just like any other page on the internet.

Basically, what it comes down to is that Hype Machine's concession to "Digital Rights Management" -- to the desire of copyright holders to control how their audience uses their content -- is to make the music freely available online, but hidden behind obscure URLs which they never make public. It's the kind of decision that smart people would only make in a field run by fear and secrecy rather than a clear and specific public policy and precedent.

And now, here's where I was faced with a moral dilemma: as soon as I saw Hype Machine's flash player, I knew I could get to all the mp3s. You see, I'd seen the player before. It's the open source XSPF flash music player designed by Fabricio Zuardi. We use it ourselves on Music For Dozens. I know how it works. It uses a standard (XML) format to describe the name, title, and URL for each song in the playlist. It is designed to be machine readable. I knew I could make a pretty nifty little script that would allow me to search for tracks on Hype machine and then download the mp3s that I found whether or not Hype Machine made that feature publicly available.

The dilemma came in the question of whether or not to make the resulting script public. On the pro side, it's a pretty neat little hack, it would be useful to a lot of people, and Hype Machine are making this music available at normal (if hard to find) public urls, so they should expect people to hit them like any other. On the downside, if the script spread widely and everyone started downloading any mp3 they wanted from Hype Machine's archive, it could get them in legal trouble and so they would pretty soon have to do something to prevent this from working which could make their excellent service that I enjoy worse for everyone.

In the end, obviously, I decided to make the script available or I wouldn't be talking about it here. I thought that as a Ruby shell script (and one which might take a little setup to get running) it would only spread to enthusiasts (of either the musical or Rubyist variety) and therefore not put too much stress on Hype Machine itself. Also by releasing it with a relatively strict license I could prevent some other less ethical person from wrapping the same idea to in a quick-buck scheme which might hurt Hype Machine in exactly the way I'm trying to avoid.

Here previously followed a more detailed description of how the script worked along with a link to download it and some explanation of its interface. Instead, I offer (with his permission) some of Hype Machine-proprietor Anthony Volodkin's thoughts on the twin subjects of why he wanted me to remove the script and the dangers faced by creative projects like Hype Machine in this strange world:

I am really glad you understand the issues involved and why I've chosen to implement things in this fashion, preventing people from being able to download the audio. The setup tries to address both the capacity issues involved with serving such a massive amount of downloads (the Flash streaming takes place directly from my server without Coral (it's too slow), but the enclosure links in the RSS feed point to the Coralized URLs of those posted by the bloggers, as you've pointed out) and the massive gray area where mp3 blogs currently reside. I would love for both of these things to be different, but there is little I can do. Ultimately, all that I really want to happen is for people to discover music that they will want to hear on repeat for days, and if they still like it afterwards, buy the CD or the track over iTunes, eMusic and etc.

Tagged: , , , ,