February 21, 2006

New Technorati Features: Favorites. And Reading Lists/OPML for Blog Finder

The blogosphere doubles every five months. As I'm writing this, Technorati is tracking 28.4 million blogs. That's a whole lot of voices. Here at Technorati, we want to make the world of weblogs accessible, searchable, and fun for everyone to explore. Today we're taking another step in that direction with a brand new feature: Technorati Favorites.

It's really very simple. Just tell us who your favorite bloggers are and you'll get a custom page that lets you monitor, search, and share your Favorites!

Can't wait? Go give it a try! For the curious, read on for more details.

We've made it as easy as possible to add blogs to your Favorites. All over Technorati, just click the star icon () to add that blog to your favorites. Looking for good blogs on a certain topic? Check out Blog Finder and add blogs from there!

There's also a bookmarklet you can take with you, so you can add to your favorites wherever you browse (get it on the Favorites Help page). And if you use another subscription tool or service, you can export your subscriptions and import them using the Favorites Importer!

Once you've got some favorites, you'll see the most recent posts from them, newest to oldest, as they happen. If you prefer RSS, don't worry, it's available.

Got your own blog? Wanna make it easy for Technorati members to mark is as a favorite? Just use one of these handy buttons! You can even display recent posts from your Favorites on your own blog with the Favorites Widget.

But the best part is that now you can search just your favorite blogs. The second tab on every keyword search result is now "Your Favorites" - click it to search just your favorite blogs.

Finally, Technorati Favorites are for sharing! Note that everyone has a public favorites page (here's mine). You can share yours with your friends with the link at the top of your Favorites page.

To help us kick off this new feature, we asked some prominent bloggers what their favorite blogs are, and they obliged! So check out what's new in these people's favorite blogs:

Thanks to these four for helping us kick off Technorati Favorites, and stay tuned - we'll be adding more fun stuff soon!

Reading Lists For Blog Finder Categories

We've also launched a small but really useful feature - OPML export of each Blog Finder category. You can see the OPML button at the bottom of each results page. This means that if you would like to import authoritative blogs in a category like knitting into your RSS Aggregator, you're all set, you can do it in one easy step. Or even better, if you are using an application with Reading List support, you can simply delegate part of your directory to the Blog Finder category for the subject.

I've been very intrigued by the possibilities around Reading Lists, and so we wanted to support them as quickly as possible in the most intuitive place as possible.

As usual, we really want to know what you think of this new feature, so please let us know.

Technorati Tags: , , , , , ,

February 15, 2006

brrreeeport report: Technorati: 422, Google Blog Search: 337

Robert Scoble created a very interesting meme and test: To see how well folks in the blogosphere are indexed by the major search engines. He's on vacation today, so I thought I'd put up a couple of screenshots of the results.

First here's Google Blog Search's results. Note that GBS returns estimates on its first page of results, so you have to click through to find the exact number of results. At this moment, GBS reports on the first page of search results "about 387" results as shown below:

Gbs-Brrreeeport-Page1

However, that number isn't accurate. Here's what you get when you click through the result lists to get to the end of the index (I kept my desktop in there to show the time of the search):

Gbs-Brrreeeport


337 results.

OK, now here's what you get when you go to Technorati, where we always report the up-to-date exact numbers:

Technorati-Brrreeeport

So I just wanted to keep y'all updated. Technorati: 422. Google Blog Search: 337.

By the way, great job, Robert. You've created a whole new meme! And you've got people talking about the accuracy of the numbers that Search Engines report, this is great stuff to have out in the open.

Technorati Tags: , , , , , , , ,

February 13, 2006

State of the Blogosphere, February 2006 Part 2: Beyond Search

In Part 1 of the State of the Blogosphere report, I covered the overall growth of the blogosphere. Today I'm going to cover the growth of the blogosphere as media, and discuss some of the emerging trends that deal with handling information overload. In a world of over 50,000 postings per hour, and over 70,000 new weblogs created each day, keeping on top of and in tune with the most interesting and influential people and topics is the new frontier beyond search. I've also got some surprises for you at the end of this post, two new features that I hope you'll find useful. But first, let's get our hands dirty in the data!

MSM vs. Blogs

To start, let's look at how attention has been shifting in the blogosphere. In the chart below, the top news and media sites are charted according to the number of bloggers linking to them, and clearly, people are still paying a lot of attention to mainstream media stalwarts like The New York Times, CNN, and The Washington Post.

Slide0005-8

For these sites, which sit on what I call The "Big Head" of the curve (as opposed to the now-famous "long tail", four blogs - BoingBoing, Engadget, PostSecret, and Daily Kos show up. This may look a bit smaller than the data of last August, but a quick look a bit further down the tail starts telling a more interesting story (Note that I've flipped the axes so that you can see more data):

Slide0006-5

As you continue down the media attention curve past the "big head", that the number of blogs starts to grow.

The Long Tail

The chart below shows the attention curve once you get past the blogs that look just like mainstream media above. It is important to note how long the long tail really is: this chart at this scale doesn't show it - the long tail of the blogging world goes out to 27.2 million blogs. To give a sense of scale, if this chart was kept to the same scale and I printed out the additional sheets necessary on regular 8.5 x 11 inch sheets of paper in landscape mode to show the entire long tail, the length of the complete graph would be about 120 pages long, making the entire chart about 110 feet long!

Slide0001

Movement along the curve

With so may blogs and bloggers out there, one might think that it is a lost cause for new bloggers to achieve any significant audience, that the power curve means that there's no more room left at the top of the "A-List".

Fortunately, the data shows that this isn't the case.

Thanks to the Wayback machine, here's a look at the Technorati Top 100 as it appeared on November 26, 2002 (bear with me if the wayback machine is slow). Then look at it as it appeared on December 5, 2003. And again on November 30, 2004. And again on April 1, 2005. And now look at it today.

Let's take a few examples. Have a look at PostSecret. It is the #3 site on the Technorati Top 100 today, with over 12,000 sites that have linked to it in the last 180 days. It didn't even exist on the chart in April of 2005. Or look at The Huffington Post. It is #5 on the Top 100. It too, didn't exist on the chart in April of 2005. Or look at the #47 blog in April, 2005 Baghdad Burning. This blog still is regularly posting, but has fallen to #304.

This should not be meant to imply that there are no network effects, or that a power law relationship doesn't exist in the Blogosphere. Of course there are network effects. But I want to go a level or two deeper than just thinking about the blogosphere as an A-List and The Long Tail -- for that's far too simplistic, and leaves out some of the most interesting blogs and bloggers out there.

The Magic Middle

This realm of publishing, which I call "The Magic Middle" of the attention curve, highlights some of the most interesting and influential bloggers and publishers that are often writing about topics that are topical or niche, like Chocolate and Zucchini on food, Wi-fi Net News on Wireless networking, TechCrunch on Internet Companies, Blogging Baby on parenting, Yarn Harlot on knitting, or Stereogum on music - these are blogs that are interesting, topical, and influential, and in some cases are radically changing the economics of trade publishing.

At Technorati, we define this to be the bloggers who have from 20-1000 other people linking to them. As the chart above shows, there are about 155,000 people who fit in this group. And what is so interesting to me is how interesting, exciting, informative, and witty these blogs often are. I've noticed that often these blogs are more topical or focused on a niche area, like gardening, knitting, nanotech, mp3s or journalism and a great way to find them has been through Blog Finder.

Explore: Dealing with Information Overload

Given that there's a lot of interesting topical posts by influential or authoritative bloggers in those topic areas, we formulated an idea: Why not use these authoritative bloggers as a new kind of editorial board? Watch what they do, what they post about, and what they link to as input to a new kind of display - a piece of media that showed you the most interesting posts and conversations that related to a topic area, like food, or technology, or politics, or PR. The idea is to use the bloggers that know the most about an area or topic to help spot the interesting trends that may never hit the "A-list". We call this new section Explore, and we've seeded it with some of the most interesting topics that we could find. But one of the nice things about Explore is that there are no gatekeepers, and that anyone who writes interesting topical blog posts can get included simply by tagging his blog and tagging his posts.

It's still pretty new, and occasionally an irrelevant post or two sneaks into the display. We're working on fixing that, but one of the new features we're launching today is the ability to subscribe to an RSS feed of any explore category, so you can now read the most interesting posts via your favorite newsreader.

These middle tier blogs also define communities of interest in the blogosphere. Its easy to think of the blogosphere as a cacophony of voices spread out over a big long tail distribution. But Blog Finder and Explore help resolve these thousands of blogs into topical, relevant communities of interest that interlink, refer to one another and often wrestle with ideas, discuss them and move them along. People often ask, "what blogs should I read?" And often times a good answer is, "you should read the posts from the leading blogs in topics that of interest you. Blog Finder and Explore make this possible for the first time on a wide variety of topics--- and in so doing we hope will the blgosphere more approachable, useful, and comprehensible to more people than ever before.

Filter By Authority: Giving YOU the power to tune your searches

There's one more big feature that I wanted to write about tonight, our new Filter By Authority feature. You can see this on all keyword search results pages, looking like this:

Explore

Clicking on the green slider allows you to easily refine your search results to show greater or fewer matching blog posts. For some searches, you might want to pick and choose only posts from blogs that have been around a while and are highly influential - so pick "a lot of authority" as shown above. I've found this great for searches on highly trafficked topics, like "George Bush" or Olympics, or on topics that are known to get a lot of spam, like mortgage or refinance. I find that it often helps me to also answer the question, "Who is the most influential blogger talking about XXX this week, and what did she say?"

Clicking lower on the slider gives you the ability to see how different levels of filtering affect your search results. For my ego feeds, I always want to see every single mention, so I turn off filtering for those feeds. I also love looking at the charts on the left-hand side of each search result to see what changes when I change the filter, too.

As we implemented this feature, we spent a lot of time thinking about how to name it. We frequently use the term authority on our site when we talk about inbound links, as in "a link is a vote of authority." So to maintain consistency we called this new feature, " sort by authority." But in no way should this imply a value judgment. More authority doesn't necessarily mean more good or more interesting. In many instances, less authority yields more interesting results: a greater diversity of opinion, less mainstream thinking, more individual voices. The authority filter is a tool to fine tune results, and its a great way to zoom in on the voices that are commanding the most attention, and then zoom back out and listen to the whole diverse medium that is the blogosphere. With so many voices we're happy to add a new tuning control!

This new feature is a beta feature, so we're looking for your feedback! Do you like it? Find it useful? Or is it confusing? What about the name? We tried a number of different names for the feature, but ended up picking "filter by authority" since we speak about a blogger's authority as being based on the number of links he gets from other people, but it isn't a perfect analogy. In the end, we decided that rather than having the perfect name, we'd much rather get the feature out there for all of you to try, and we'd listen intently to your feedback and comments.

In Summary

  • Blogging and Mainstream Media continue to share attention in blogger's and reader's minds, but bloggers are climbing higher on the "big head" of the attention curve, with some bloggers getting more attention than sites including Forbes, PBS, MTV, and the CBC.
  • Continuing down the attention curve, blogs take a more and more significant position as the economics of the mainstream publishing models make it cost prohibitive to build many nice sites and media
  • Bloggers are changing the economics of the trade magazine space, with strong entries covering WiFi, Gadgets, Internet, Photography, Music, and other nice topic areas, making it easier to thrive, even on less aggregate traffic.
  • There is a network effect in the Technorati Top 100 blogs, with a tendency to remain highly linked if the blogger continues to post regularly and with quality content.
  • Looking at the historical data shows that the inertia in the Top 100 is very low - in other words, the number of new blogs jumping to the top of the Top 100 as well as he blogs that have fallen out of the top 100 show that the network effect is relatively weak.
  • The Magic Middle is the 155,000 or so weblogs that have garnered between 20 and 1,000 inbound links. It is a realm of topical authority and significant posting and conversation within the blogosphere.
  • Technorati Explore is a new feature that uses the authoritative topical bloggers as a distributed editorial team, highlighting the most interesting blog posts and links in over 2,500 categories.
  • The new Filter By Authority slider makes it easy to refine a search and look for either a wider array of thoughts and opinions, or to narrow the search to only bloggers that have lots of other people linking to them. This gives you the power to decide how much filtering you want.

Technorati Tags: , , , , , , , , , , , , , , ,

February 6, 2006

State of the Blogosphere, February 2006 Part 1: On Blogosphere Growth

It's been 4 months since last October's State of the Blogosphere report, so it's time to update the numbers! For historical perspective, you can see earlier State of the Blogosphere reports from July 2005, from March 2005, and from October 2004.

The State of the Blogosphere is Strong.

OK, I'm paraphrasing from a more famous speech that happened last week, but the truth is that the blogosphere continues to grow at a quickening pace. Technorati currently tracks 27.2 Million weblogs, and the blogosphere we track continues to double about every 5.5 months, as the chart below shows:

Slide0002-2

The blogosphere is over 60 times bigger than it was only 3 years ago.

New blog creation continues to grow. We currently track over 75,000 new weblogs created every day, which means that on average, a new weblog is created every second of every day - and 13.7 million bloggers are still posting 3 months after their blogs are created. In other words, even though there's a reasonable amount of tire-kicking going on, blogging is growing as a habitual activity. In October of 2005, when Technorati was only tracking 19 million blogs, about 10.4 million bloggers were still posting 3 months after the creation of their blogs.

In addition to that, about 2.7 million bloggers update their blogs at least weekly. Here's a chart of the number of new blogs created each day, from January 2004 to January 2006:

Slide0003-5

Dealing with Spam

There has been an increase in the overall noise level in the blogosphere, most notably in the number of spam and fake pings that are sent - what I call "spings". These spam pings are fake or bogus notifications that a blog has been updated; in some cases, these spings can amount to a denial-of-service attack, and can sometimes account for as much as 60% of the total pings Technorati receives. However, we've built a sophisticated system that mitigates the spings, and helps to keep spam blogs out of our indexes. Beyond that, about 9% of new blogs are spam or machine generated, or are attempts to create link farms or click fraud. Technorati continues to take an ecosystem approach to solving this problem, working closely with other players like Amazon, AOL, Ask Jeeves, Drupal, Google, MSN, Six Apart, Tucows, Wordpress and Yahoo, and there will be another Web 2.0 Spam Squashing Summit this spring, building on the success of the previous two summits.

A News Cycle Measured in Megahertz

Moving beyond spam, the number of people reaching out and reaching each other continues to grow. Daily Posting Volume tracked by Technorati continues to grow, and the blogosphere also reacts to world events. I've pointed out a number of the spikes in posting volume that have accompanied major news events in the chart below of posting volume:

Slide0004-5

We track about 1.2 Million posts each day, which means that there are about 50,000 posts each hour. At that rate, it is literally impossible to read everything that is relevant to an issue or subject, and a new challenge has presented itself - how to make sense out of this monstrous conversation, and how to find the most interesting and authoritative information out there.

The Continued Rise of Tagging

In January 2005, Technorati launched its tagging service, based on the rel=tag microformat, which is a simple way for bloggers to categorize their posts, and to make it easy for people to find interesting posts on a given subject. Today, we have tracked over 81 Million posts with tags or categories - and over 400,000 new tagged posts are created every day. The chart below shows the immense growth of tagging in the past year:

Slide0008-1

Tags for Blogs

There was still a major problem, however - how to easily find the most interesting blogs on the subjects that you cared about. So, in September 2005, Technorati launched Blog Finder, a tags-based way for people to find the most authoritative blogs on a particular subject, allowing bloggers to tag their blogs into the categories that they felt were most relevant for themselves. In 4 months, over 850,000 blogs have been put into Blog Finder, making it the most comprehensive directory of blogs on the web. Over 2,500 categories have already attracted a critical mass of influential bloggers writing about them, from Politics and Technology to Gardening or Erotica. And more are created every day, making it easier for people to find the most interesting blogs in the topics they care about.

In summary:

  • Technorati now tracks over 27.2 Million blogs
  • The blogosphere is doubling in size every 5 and a half months
  • It is now over 60 times bigger than it was 3 years ago
  • On average, a new weblog is created every second of every day
  • 13.7 million bloggers are still posting 3 months after their blogs are created
  • Spings (Spam Pings) can sometimes account for as much as 60% of the total daily pings Technorati receives
  • Sophisticated spam management tools eliminate the spings and find that about 9% of new blogs are spam or machine generated
  • Technorati tracks about 1.2 Million new blog posts each day, about 50,000 per hour
  • Over 81 Million posts with tags since January 2005, increasing by 400,000 per day
  • Blog Finder has over 850,000 blogs, and over 2,500 popular categories have attracted a critical mass of topical bloggers

Tomorrow: Going beyond search and tags, to discovery.

Technorati Tags: , , , , , , , , , , , , , , , , , , , ,

February 5, 2006

Pictures from Rome, Italy

I can't believe that it has been over a month since I last blogged. Well, I've got a bunch of things to blog about, so I'm going to keep it short and sweet, and blog early and blog often.

I had the great privilege of speaking to the 2006 OECD Conference on the Digital Society in Rome last week. It was my first trip to Rome, and I took an extra day to beat the jetlag and to explore Rome a bit. I had a wonderful afternoon wandering around the Vatican, Castel St. Angelo, and The Pantheon with Bradley Horowitz last Wednesday, and then Thursday had a wonderful chance to get a scooter's-eye view of the city with Robin Good. I took lots of pictures, and just put the best ones up on Flickr. You can also see them as a slideshow. Below are some of my personal favorites:

IMG_0040 CRW_2064

CRW_2172 CRW_1967 CRW_2250