Scaling, performance, and plain old bug fixing

21
Posted by David Sifry on July 14, 2005 at 1:59 am

What a couple of months this has been! First, some stats on what’s going on in the blogosphere. Technorati is now tracking over 13.3 Million blogs, and 1.3 billion links. We are seeing over 900,000 posts per day on average, which means we’re adding about 10 posts per second. We’re also seeing about 80,000 new weblogs created each day. That’s more weblogs created each day than there were total when I started the service in November 2002. And our search traffic has increased by over 40% month on month for each of the last 4 months. The day of the london bombings we saw over 1.2 Million posts, and had an additional 30% increase in traffic as people turned to weblogs, moblogs, and other citizen’s media for instant updates on events in London, survivor accounts, and sharing of deep feelings on the tragedy.

Recently a number of people have had some pretty public complaints about some of Technorati’s services. Thanks for the terrific feedback and comments. I feel your pain.

We sat down, listened hard to what you were saying, and then got to work. And tonight, we rolled out a raft of bug fixes and performance enhancements that should help most, if not all of the Cosmos (URL) searches you do on Technorati. It will also help with the speed of all searches across the site.

Give it a go – Here’s the results for this blog, and here’s how you can give it a try.

These improvements don’t fix everything – some searches are still slow, and while we pride ourselves on completeness and fast index times, there’s still a long way to go. Performance and scalability improvements are our number 1 priority until this is fixed.

For those of you who have been having problems, we’re working our butts off to win back your trust. I hope that the fixes we rolled out tonight and will roll out over the coming days and weeks will be of service to you. Because, in the end, that’s what this is all about to us – to be of service to all of you.

Thanks again for the great feedback and comments. Keep us on our toes.

Share

Related posts:

  1. Performance and Scalability improvement progress report #2
  2. Performance and Scalability improvement progress report #1
  3. Technorati Performance Improvement Update
  4. Technorati site redesign and features launched!
  5. Technorati Growing Pains

Both comments and pings are currently closed.

Posted in: Uncategorized

This article has 21 comments

  1. Hans on Experience 07/14/2005, 3:38 am:

    Hi David, my trackback did not show up here, so here is the comment: http://hmestrum.blogs.com/my_weblog/2005/07/strange_behavio.html
    “Strange behaviour of Technorati. David are you listening?
    Strange! My postings do not show up anymore in Technorati. Also the postings of Maarten do not. I saw this some days ago when I posted the first Blogstorm post. Tagged with the blogstorm tag. It does not show up…”

  2. Jason Dowdell 07/14/2005, 5:09 am:

    Dave, I’ll be quite honest with you… I was one of the ones blogging about this that Doc called out. But I wanted to personally say thanks for giving a great lesson on damage control and doing whatever it takes to keep your customer [users]. I put my thoughts this am.
    Good job and good luck!
    ~jason dowdell

  3. The Newest Industry 07/14/2005, 5:12 am:

    Technorati: Performance Fixes?

    Dave Sifry from Technorati very publicly announce a mea culpa, and said that he and his team rolled out some performance enhancements that should resolve the problems.[here]
    Dave, they’re still there. Go to GrabPERF and select any of the Technorati …

  4. Robert Scoble 07/14/2005, 5:45 am:

    Dave, I’m worried you’re being out-executed by your competitors. Take a look: http://radio.weblogs.com/0001011/2005/07/14.html#a10642

  5. Hans on Experience 07/14/2005, 5:48 am:

    Thanks David. You are really listening! Great. I hope the problem is solved soon.

  6. Susan Kitchens 07/14/2005, 10:24 am:

    Thanks for the update. It’s a vast improvement for me when I do my technorati bruised-ego surfing. Now instead of a plain page with the URL/Squid error message, I consistently get a nicely designed page with the following information:
    2020 Hindsight
    Sorry, we couldn’t complete your search because we’re experiencing a high volume of requests right now. Please try again in a minute or add this search to your watchlist to track conversation.
    Ah well. Keep at it. Reminds me of my psych behavior conditioning class: Behavior that is intermittently reinforced (like the Technorati search cosmos results appearing every-once-in-a-while) is the hardest behavior to extinguish. Thus I keep trying. Pathetic, that. The upside for you is that I keep trying. The downside is that my level of dissatisfaction stays elevated.
    But thanks to Scoble, I’m trying out the bloglines citations thing (tho I notice that most recent results are not identical)

  7. Mikhail 07/14/2005, 10:29 pm:

    Dave: That’s just great! However, I do agree with Scoble… Technorati indexing needs to improve quite a bit!

  8. The Basement 07/14/2005, 11:38 pm:

    Size (of the whole Blogosphere) Doesn’t Matter

    Yesterday in Stop
    the Blog Statistic Madness! I touched on something I’ve been
    meaning to blog about for a while now – your base. I’m not
    talking about the base of the classic game Zero Wing, I’m talking
    about the base in the market research sense. …

  9. TechCrunch 07/16/2005, 3:20 am:

    Web 2.0 This Week (July 10 – 16)

    The main Web 2.0 news this week focused on Technorati (is it dyin or is it rockin?) and the release of Atom 1.0. Lots of other random and interesting stuff as well, including important chia-pet news.
    Weekly Summary:
    1. Technorati is Dead! Long Liv…

  10. pointblog.com 07/17/2005, 4:10 am:

    Interrogations sur l’avenir de Technorati

    Technorati occupe une place

  11. padawan 07/17/2005, 4:23 am:

    Scoble’s comparison between Technorati and Bloglines is totally misleading. He confounds sites with links, and quantity with quality. See: http://padawan.info/weblog/weapons_of_mass_delusion.html

  12. lotusmedia 2.0 07/19/2005, 9:40 am:

    TechnoTrouble

    Why doesn’t Technorati ever seem to work quite right (or at least work as they claim it does)? In addition to being slow as molasses, it has recently started to incorrectly index my posts. Maybe I shouldn’t complain since a month ago it …

  13. Ruby 07/19/2005, 1:22 pm:

    Ditto, Omni. It was working for me until some point last week and now my posts are either not showing up on Technorati at all, or they have the wrong URLs and tags with each post. :-(

  14. pointblog.com 07/19/2005, 3:22 pm:

    Interrogations sur l’avenir de Technorati

    Technorati occupe une place

  15. Alex Barnett blog 07/19/2005, 11:48 pm:

    RSS will succeed by disappearing

    Blogspotting quotes Dave Sifry, CEO of Technorati:
    “Sifry: I think RSS needs to be completely transparent….

  16. pointblog.com 07/20/2005, 12:20 am:

    Interrogations sur l’avenir de Technorati

    Technorati occupe une place

  17. Randy Charles Morin 07/26/2005, 10:34 pm:

    Hmmm! Now I’m getting really upset. I don’t have anything to complain about tonight. Technorati seems to be working very well. Congrats to you and the team!

  18. Rexiology... 07/31/2005, 6:18 am:

    [Trackback] Technorati is not linking or tagging my posts…

  19. Rex Tang 07/31/2005, 7:36 pm:

    Hi, David,
    Thanks for the reply at my post http://blog.rex.la/archive/2005/07/31/1821.aspx
    My site at Technorati.com still not update…
    do a site search to technorati will reflect the situation.
    http://www.technorati.com/search/blog.rex.la
    it shows my site is updated 7 days ago, while I just push this post couple hours ago…
    may I ask why is the cause of this problem??
    thanks.
    Rex

  20. Rex Tang 07/31/2005, 9:15 pm:

    Hi, David,
    Thanks for the reply again at my post http://blog.rex.la/archive/2005/07/31/1821.aspx
    and Yes, David,
    It’s working now, thanks a lot!!
    Technorati is really listening, wow!!!
    I am really impressed…
    Now a more big fan of Technorati….
    Rex.

  21. Rexiology... 07/31/2005, 9:25 pm:

    [Trackback] Technorati is not linking or tagging my posts…

Across the Web

  • Facebook
  • Twitter
  • Flickr
  • Youtube

Twitter

Javascript needs to be installed to view the twitterfeed. Get Javacript

Follow Me on Twitter

Blogroll

Personal