Sifry's Alerts

State of the Blogosphere, October, 2006

Hey, it's that time of the year again! We’re well into the crisp days of autumn and its time for the quarterly State of the Blogosphere report.

The State of the Blogosphere continues to be strong.

The last few months have prompted a great deal of thought amongst the team here about the maturation of the blogosphere since I wrote the first algorithms that led to the creation of Technorati nearly four years ago, and I'll be going into a lot more depth below.

OK, let's start with the overall numbers:

Currently Tracking More than 57 Million Blogs and Counting.

As you can see, growth in the numbers of blogs tracked by Technorati continues to grow briskly. While the doubling of the blogosphere has slowed a bit (every 236 days or so, here's the historical data) , interest in blogging remains considerable. About 55% of all blogs are active, which means that they have been updated at least once in the last 3 months.

To get another view, let's look at the number of new blogs tracked each day:

As of October 2006, about 100,000 new weblogs were created each day, which means that on average, there was a slight decrease quarter-over-quarter in the number of new blogs created each day.

As we’ve said in the past, some of the new blogs in our index are Spam blogs or 'splogs'. The good news is Technorati has gotten much better at preventing these kinds of blogs from getting into our indexes in the first place, which may be a factor in the slight slowing in the average of new blogs created each day.

The spikes in red on the chart above shows the increased activity that occurs when spammers create massive numbers of fake blogs and try to get them into our indexes. As the chart shows, we’ve done a much better job over the last quarter at nearly eliminating those red spikes. While last quarter I reported about 8% of new blogs that get past our filters and make it into the index are splogs, I’m happy to report that that number is now more like 4%. As always, we’ll continue to be hyper-focused on making sure that new attacks are spotted and eliminated as quickly as possible.

My gut feeling is that since we're better at dealing with Spam now, even some of the blue areas in last quarter's graph were probably accountable to spam, which would mean that rather than the bumpy ride shown above, we're actually seeing a steady increased (but slower) growth of the blogosphere. Hopefully we'll be able to have a more detailed analysis of these issues next quarter.

Daily Posting Volume

First off, the total posting volume of the blogosphere has leveled off somewhat, showing about 1.3 million postings per day, which is a little lower than what we were seeing last quarter but still about double the volume of this time last year. This leveling off may be the result of more aggressive and mature spam fighting capabilities as discussed above, but we'll have to see how the next three months progresses to determine if this is the case or if some other trend is at work.

Along with the aggregate posting volume information, we’ve put in some annotations of the events that occurred at the time of the spikes, showing that the blogosphere continues to react strongly to various world events. It is important to note that these spikes are relative to the posting volume at that time. For instance, the big spike in July is related to the Israeli / Hezbollah conflict as well as other escalating tensions in the Middle East. I similarly would expect to see a spike beginning today and throughout this week in response to the upcoming U.S. elections.

Blogs and Mainstream Media

The integration of blogs and traditional media sites on the web continues. We've put together the top 100 sites that make up "The short head" (as opposed to "the long tail") is still predominantly made up of traditional media sites, like The New York Times, Yahoo! News, CNN, and MSNBC.

However, as we move down the curve, blogs become more widespread in the list. There are 12 blogs in the top 100 combined list. 3 are in the top 50, and 9 of the 12 are in the slide below:

By the time you reach the top 5000, blogs have essentially taken over, with very few well-funded mainstream media sites listed. This is partially because of the nature of the medium - that is, the traffic of sites further down the curve make significant staffing and revenue difficult. However, lower cost structures make individual or small group blogs operating at little cost quite efficient at these revenue levels.

The Medium Matures

As I mentioned earlier, we’ve been doing a lot of thinking about the maturation of the blogosphere and the blogging phenomenon in general. We asked ourselves, "What are the common characteristics of top bloggers? Do they behave differently? What can we learn from them?

So, we broke down some basic posting behavior for bloggers that have different Technorati rankings, with the level of influence or authority increasing as you go from left to right in the chart below:

The Low Authority Group (3-9 blogs linking in the last 6 months)

The average blog age (the number of days that the blog has been in existence) is about 228 days, which shows a real commitment to blogging. However, bloggers of this type average only 12 posts per month, meaning that their posting habits are generally dedicated but infrequent.

The Middle Authority Group (10-99 blogs linking in the last 6 months)

This contrasts somewhat with the second group, which enjoys an average age not much older than the first at 260 days and which posts 50% more frequently than the first. There is a clear correlation between posting volume and Technorati authority ranking.

The High Authority Group (100-499 blogs linking in the last 6 months)

The third group represents a decided shift in blog age while not blogging much more frequently than the last. In keeping with the theme of the maturation of the blogosphere, it seems evident that many of these bloggers were previously in category two and have grown in authority organically over time. In other words, sheer dedication pays off over time.

The Very High Authority Group (500 or more blogs linking in the last 6 months)

In the final group we see what might be considered the blogging elite. This group, which represents more than 4,000 blogs, exhibits a radical shift in post frequency as well as blog age. Bloggers of this type have been at it longer – a year and a half on average – and post nearly twice a day, an increase in posting volume of over 100% from the previous group. Many of the blogs in this category, in fact, are about as old as Technorati and we’ve grown up together. Some of these are full-fledge professional enterprises that post many, many times per day and behave increasingly like our friends in the mainstream media. As has been widely reported, the impact of these bloggers on our cultures and democracies is increasingly dramatic.

A note on Ranking

For those of you who are new to Technorati's ranking systems, we establish a blog’s authority (or influence) by tracking the number of distinct blogs that link to it over the past 6 months. In this chart, we’ve looked at folks with at least 3 links or more and grouped them into four separate categories. In total, we’re looking at about 1.5 million blogs of the 57 million total. Even though I labeled the first group as the "Low Authority" group, given that these people are in the top 2% of all of the blogs that exist, the concept of "low" is purely in relation to the other groups above.

Blogging is Global

As we reported last quarter, English and Japanese remain the two most popular languages in the blogosphere. There were, however, some interesting shifts among those languages less well represented in the blogosphere. Holding steady in the number three spot is Chinese, although it has dipped slightly to 10% of the total posting volume. A notable change, however, is that Farsi has pushed its way into the top 10 languages in use in the blogosphere, bumping Dutch, which had held the number 10 spot over the last couple of quarters, into the number 11 spot.

It is important to note that some important caveats apply to this language data.

Posts by language by hour, shown below, looks very similar to last quarter.

We delved a little deeper to see if we could understand any other interesting per-language trends. If you look at the top 4 languages side-by-side and standardize them to their relative posting levels throughout the day, recognizable posting patterns by language begin to emerge. While Japanese and Chinese language posts have a daily pattern that indicate heavily localized posting, both English and Spanish language posts indicate more globalized posting patterns.

Interesting to see how much blogging goes on during work hours!

In Summary:

• Technorati is now tracking more than 57 Million blogs.
• Spam-, splog- and sping-fighting efforts at Technorati are paying dividends in terms of the reduction of garbage in our indexes, even if it does seem to impact overall growth rates.
• Today, the blogosphere is doubling in size approximately every 230 days.
• About 100,000 new weblogs were created each day, again down slightly quarter-over-quarter but probably due in part to spam fighting efforts.
• About 4% of new splogs get past Technorati's filters, even if it is only for a few hours or days.
• There is a strong correlation between the aging and post frequency of blogs and their authority and Technorati ranking.
• The globalization of the blogosphere continues. Our data appears to show both English and Spanish languages are a more universal blog language than the other two most dominant language, Japanese and Chinese, which seem to be more regionally localized.
• Coincident with a rise in blog posts about escalating Middle East tensions throughout the summer and fall, Farsi has moved into the top 10 languages of the blogosphere, indicating that blogging continues to play a critical role in debates about the important issues of our times.

As always, I'm very interested in your comments and feedback.

Technorati Tags: blogging, blogosphere, blogs, blogsearch, charts, language, msm, postingvolume, posts, postvolume, scaling, search, search engine, sotb, sotb2006, spam, spamblog, sping, splog, statistics, stats, study, technorati, top100, update, weblog, weblogs, wow