Technorati Performance Improvement Update
18I’m incredibly proud of the Technorati Engineering and Ops teams. They have been working day and night to deal with all of the growth that Technorati has been going through. Back in July, we made it our highest priority to improve site performance, and I and our team members kept you updated in some earlier posts.
Here’s some facts:
- Since beginning our infrastructure improvements, Technorati’s uptime has improved significantly – in fact, we had 100% uptime for searches last month!
- According to GrabPerf, even while our overall traffic has increased, our response times have consistently decreased: We are currently averaging subsecond response times for all core products: Search (766 ms), single-word search (651 ms), and Tags (836 ms).
- A quick comparison of search speed of various blog engines is below, and you can see the continuous improvements that our engineers have been making over the past weeks and months. Technorati is the light blue line that is sandwiched right in there with Google Blog Search (731 ms) and Yahoo Blog Search (835 ms).
- Here’s another insight: Open the following three links in separate tabs in your browser, and click back and forth between them to get a quick view of the frequency distribution of search results: Technorati, Google, and Yahoo. The further the results are to the left hand side of the graph (consistently fast results), the better it is.
- Technorati’s index is the most comprehensive, and has the fastest updates. The index is over 3 years old, currently tracks over 21.5 million blogs, over 280 million unique posts and over 1.7 billion links are indexed. We are tracking over 60,000 new blogs each day, and over 700,000 new posts per day. Our median time to index is now under 3 minutes from the moment a blog post is created.
Notes: All engines noted are for the companies’ blog search services. All of those graphs are updated dynamically, so don’t be surprised if the numbers loook a little different when you click on them.
All this, and we rolled out a number of new features and improvements to current services as well. I’m just very, very proud of our team, and I’m really grateful to be able to work with them each day. And we’re constantly striving to do better. Please send us feedback and let us know how we can make things even better for you – what are we missing? How can we be more of service to you?
Update: Many thanks to Randy Charles Morin for pointing out a typo in the stats on number of blogs indexed, I have updated the post to include the correct number of blogs and posts. I also added in the number of new blogs and posts we’re tracking every day as well. Thanks, Randy!
Technorati Tags: blogosphere, blogs, blogsearch, competition, feedback, givingthanks, google, gratitude, scaling, search, search engine, statistics, stats, technorati, wow, yahoo
Related posts:
Technorati posts major server upgrade
Technorati’s founder David Sifry posts an update about his service’s new engineering works. Technorati had 100 percent uptime last month and has reduced its average response times for different forms of search queries to subseconds. This is great news …
Good for you guys! Keep it going!!!
Now, if you can only do something about the splogs in your index.
http://www.technorati.com/search/www.kbcafe.com
That’s 90′s blogspot splog referrers. Maybe remove Blogspot.com blogs?
Dave, thanks to you and your team for making Technorati useful again.
I noticed it subconciously and have a “it is working again” on my to post list – because we should not only give flack when it is not working, but recognize when it does run as we expect it.
Dave:
You and the Technorati team have done some great work over the last few months.
The most impressive gains have been in the Tag search, something you don’t highlight in your post.
http://static.flickr.com/27/65502710_a02e59e113_o.png
smp
Dave, I just realized it says 21.5 million blog posts. Shouldn’t that be total blogs, not blog posts?
I’ve noticed it. I’ve been adding more links from my own personal efforts to Technorati lately because of it and have felt more comfortable mentioning Technorati to others again. Great work!
Keep the good work peeps. Technorati is the search of choice for Illmethinks_Politiq.
http://www.illmethinks.com
peace (literally)
g
Dave,
Now all you have to do is drop about 150 lbs. Hit the treadmill boss!
C
About the histograms for search speed among google, technorati, and yahoo, isn’t the most important thing the tightness of the cluster? While having fast average times is good, I think consistency is really where it’s at…
Technorati Performance Improving: Sifry
Technorati’s David Sifry writes that Technorati’s performance is improving and that they can now claim to be the quickest of the blog search engines. He’s totally right, I’ve recently returned to using Technorati in a big way, and have even relented to…
Geek News Central 2005-11-22 #118
Kept this show to 45 minutes being we are headed into the holiday period I want to encourage all of you to be safe in your travels and enjoy your families during thanksgiving. For listeners outside the United States the…
Technorait Performance Improvement
Technorati has improved its speed tremendously. Scaling a web service is difficult. Scaling one that consumes millions of blogs daily…
Technorati Performance Improvement
Technorati has improved its speed tremendously. Scaling a web service is difficult. Scaling one that consumes millions of blogs daily…
Highly impressed with what you’ve done, especially over the past few months. Technorati has created the ultimate blog community.
-gena
Is there a support forum or email list? I have a few questions and hate emailing the founders all the time
You guys rock I always have considered you guys the Google of blog searches.Keep up the great work!!
Re: Technorati Ping
I have not received response from Technorati to a number of messages (all I got was an acknowledgment). Hope someone here can help.
When Technorati is pinged about a new post and the confirmation appears on screen is there a time lag before the update appears in Technorati ? I have pinged and then gone back to check in search after 5, 10, 15 minutes and received Technorati’s report “last updated 20 hrs ago”–about the previous day’s post. So I repeat the ping and see the confirmation but 10 minutes later still no record
of the current post. It is only after multiple attempts that the post finally appears. Then there are times I see two entries for the same post !
Shall be grateful for comments/advice. I run Win XP and use Firefox or Mozilla browser.
Thank you.