For the past few days, we’ve experienced a bit of a slowdown in the timeliness of our data. To give you an idea, our normal median time between being pinged by a blog and having the data available in our index is under 7 minutes. Recently it’s been running around several hours.
Unfortunately, a good deal of this is attributable to the increase of spam that’s coming at us. The growing number of link farms creates a much greater load on our spiders. Even worse, when spam makes it into our databases, we need to pause our spiders while take explicit steps to purge the spam. This is a time-consuming and complicated process. Also, some of our ancillary systems, like correctly updated link counts, have taken a hit as we work through these issues. I’m sorry if your blog counts haven’t been updating recently, we’re working on it diligently.
We hope to be past this spate of problems in the next few days. We’re continuing to work to defend our systems from spam attacks. Just as important, we’re looking to the blogging community to work together to come up with comprehensive measures to address these issues. At the upcoming Spam Summit, we’re looking forward to working with the best minds in our industry to do just this.
Charlie Rose had Andrew Sullivan, Ana Marie Cox, Joe Trippi, and Glenn Reynolds on his show last week, and I was pretty amazed when I watched the preview clips, and then the actual show. Here’s what was highlighted by Rose (listen to the mp3, thanks, Niall!)
Rose: How do they do that, though – I mean, how do you find out people who are showing a particularly incisive mind and ability…
(Garbled as many people answer)
Reynolds: Start reading blogs, and following links, and after a while you would… I think if I had a big organization, say if I ran the New York Times or some piece of it, I would pay somebody – and you wouldn’t have to pay him a lot, it’s not very hard – to just plug the URL for every New York Times story into Technorati, which will then give you a list of every blog that links to that story, and see what people say. And if you find a bunch of people saying there’s a mistake in it, I’d run a correction just like that, and I’d print it somewhere on the website, the blogs that pointed it out. And you would turn a bunch of adversaries into a bunch of unpaid assistant editors and fact checkers overnight. And I don’t understand why more organizations don’t think that way. Because, uh…
Rose: They might, after this program.
Here’s the audio. Thanks again, Glenn! What a tremendously humbling experience. Thanks very much. I hope we continue to earn your praise.
I expect that he’s making a major ruckus at the Pearly Gates. There’s lots of good tributes out there on the web to him by writers much better than me, here’s how to keep track…
Technorati is coordinating a Web Spam Squashing Summit in Sunnyvale next Thursday, February 24, and I would like to extend an invitation to all tool developers to attend. Many thanks to Yahoo! for hosting the event on their campus.
The summit will focus on web spam — not email spam. Web spam includes comment spam, link spam, TrackBack spam, tag spam, and fake weblogs. We are bringing all of the key players together in one room to discuss current projects seeking to address the common problem and hope to leave the event with a solid set of actions. Key industry players such as AOL, Google, MSN, Six Apart and Yahoo have all confirmed their attendance.
Space is limited so please send a prompt reply to email@example.com if you are interested in attending. If you or your company is playing a role in enriching conversations on the web you are invited to attend the summit as space allows. Include the organization or tool that you’re developing or representing as well, thanks. To make things more productive for everyone coming, please include a short paragraph covering one or more of the following in your email, so we can distribute it to all the folks coming to the summit:
- Problem Statement: describe a form of spam you are dealing with
- Current Solutions: describe a current solution you have implemented and how it works
- In Development: describe a solution you are working on and why it will be better
See you there!
We are also planning a webcast/IRC for folks who can’t make it, more to come as all the details are worked out.
Right now I’m in my office showing a bunch of smart folks what blogging is all about.
The blogosphere is buzzing about the acquisition. Here’s how you can track it in real-time.