Dealing with Blog Spam
2For the past few days, we’ve experienced a bit of a slowdown in the timeliness of our data. To give you an idea, our normal median time between being pinged by a blog and having the data available in our index is under 7 minutes. Recently it’s been running around several hours.
Unfortunately, a good deal of this is attributable to the increase of spam that’s coming at us. The growing number of link farms creates a much greater load on our spiders. Even worse, when spam makes it into our databases, we need to pause our spiders while take explicit steps to purge the spam. This is a time-consuming and complicated process. Also, some of our ancillary systems, like correctly updated link counts, have taken a hit as we work through these issues. I’m sorry if your blog counts haven’t been updating recently, we’re working on it diligently.
We hope to be past this spate of problems in the next few days. We’re continuing to work to defend our systems from spam attacks. Just as important, we’re looking to the blogging community to work together to come up with comprehensive measures to address these issues. At the upcoming Spam Summit, we’re looking forward to working with the best minds in our industry to do just this.
Related posts:
Gresham’s Law and Blogging
Two slightly worrying posts that suggest to me that the linked economy of the blogosphere might be more fragile than we would like. First, Brad DeLong gives us an economist’s take on Technorati’s recent difficulties….
David– I suppose that a trusted association of content providers is inevitable. Seems that’s another thing we neglected to get around to discussing at the Blo/Jo Credibility conference.
Jon