Sifry's Alerts

The Technorati Story: How a New Web-Services Product Grew out of a Research Assignment

When Doc and I started doing research for this feature, I was still something of a blogging neophyte. While I was experienced with all the components of the LAMP platform (Linux, Apache, MySQL, PHP, Perl and Python), had an account with Advogato and had even set up my own blog using Moveable Type, I was mostly a blog software consumer. This project gave me a chance to hack.

After we set up our blog software server (kindly loaned to us by Penguin Computing), I installed Moveable Type and immediately looked for ways to improve it. First I wrote an e-mail-to-blog and a Jabber-to-blog set of interfaces, so I could post using my favorite e-mail and IM clients. (I've always believed the web browser is a lousy web-services client--something we all intuitively know whenever filling out a long form on the web.) Then I began to think about the kinds of web services I'd like to see as a regular blogger.

The result, after three weekends of hacking, is Technorati.com, a new site that provides four services:

1. Link Cosmos shows you what blogs are linking to your blog, or any other blog or any arbitrary URL. Every time another blog saves a post, it sends out an RSS notification that Technorati receives, puts in a database and lists on request in your current cosmos. This is a major advance over referrer logs, which only show followed links.

2. Google Rank shows you the top 100 sites on Google for a given search term. Technorati rechecks Google's rankings daily, so you can see how the rankings change over time.

3. Google Juice tells you where your blog ranks in the top 1,000 for any search term.

4. Watchlists keep and track historical information about a given site, allowing you to see new links to your site quickly as well as historical ranking information not shown in standard views.

Watchlists are how Technorati answers the question ``How can you make money with web services?'' For $5 a year you get a daily e-mail with your latest cosmos listings. For $10 a year you get instant access to live watchlist information through an RSS feed.

As I write this, Technorati is only a few days old, and it's already been to the top of both Blogdex and DayPop. It's also made about $250. (update: that number is up to about $2,000 now)

Technorati is brand new, a work-in-progress and invited into the world by a plethora of open-source tools and open protocols, many of which are products of the blogging development community. I used a LAMP combination with Linux, Apache, MySQL and PHP for the live scripting and Perl for the back-end web robot and other back-end tasks. XML-RPC pings power the activation of the web spider, so Technorati is always full of up-to-date information. It exports RSS feeds for people who want to view output in a more structured format, such as an RSS browser. I set up payment and billing through PayPal's open APIs. These use HTTP POST methods that easily perform credit-card processing and billing.

In programming we stand on the shoulders of giants. So I'd like to run a bunch of people in Technorati's credits: Dave Winer, who wrote XML-RPC and was a driving force behind SOAP and RSS; Ben and Mena Trott, who wrote Movable Type; Rasmus Lerdorf and many others who developed PHP; Larry Wall and many others who developed Perl; the entire Apache team; Monty Widenius, David Axmark and the MySQL development team; Evan Willams and the folks behind the Blogger API; the Google team behind the Google API; and, of course, Linus Torvalds and the Linux development community. Posted by dsifry at March 9, 2003 3:36 PM | TrackBack | View blog reactions