The Technorati Generation

Today in Tedium: The thing that we often say about our modern social media ecosystem is that we’re often playing within the walled gardens of others. This is largely by design—Facebook, or Twitter, or Snapchat, or TikTok have more control over the final result if they’re the ones hitting the switches back at HQ. But there was once a time that all that chatter, albeit at a smaller scale, was happening on disparate platforms that nobody owned. There was no walled garden—only a whole lot of weeds, growing freely around the internet. Someone had to make sense of it, and that led to the rise of the blog search engine—a concept popularized by Technorati. (Remember that?) Today’s Tedium considers the forgotten period when blog search engines were a big deal. — Ernie @ Tedium

Thanks to Simon Owens for the idea here. He has a Substack, if you’re curious.

Technorati, as the site appeared in 2007. (buou/Flickr)

The early blogosphere was defined by side projects that became businesses. Technorati was one of those.

When it comes down to it, blogging has always been shorthand for a way to describe conversational writing on the internet, though the specific terminology took a while to find its footing. Some early blogging notables, like late science-fiction author and Byte columnist Jerry Pournelle, were slow to embrace the term.

But it ultimately took off, in no small part thanks to a series of small startups that were generally passionate about the idea. Perhaps the most famous was Blogger, a platform notably created by Evan Williams and Meg Hourihan in 1999 under the Pyra Labs brand. Hourihan notably developed the Kinja blogging platform used by Gawker Media-descendant sites even today; Williams, of course, is a cofounder of Twitter and Medium.

Small startups like Pyra Labs were the lifeblood of early blogging, and played a fundamental role in helping to set the parameters of early internet culture as it evolved from the excesses of Web 1.0 (think Kozmo and Pets.com) into what became known as Web 2.0. South by Southwest’s interactive portion gained much of its modern cultural cachet in large part because Williams and other bloggers decided to embrace it.

It was not a particularly stable time for the web, admittedly. Infamously, Pyra Labs, which did not have a business model for Blogger as the company was intended to develop project management software, saw its staff members leave one-by-one as its funding dried up, leaving Williams as its only employee.

And years later, many of the ideas invented around this time got swallowed up into the walled gardens of the startups and corporate giants that followed these digital pioneers.

One of those digital pioneers was a guy named David Sifry. A blogger himself, he wanted a way to better understand how his own site was doing on the internet. So he built a way to do that. Somewhat similarly to Pyra Labs, the tool that he built, which he called Technorati, was not necessarily intended to be a business. He already had a startup that he was working at, which focused on wireless computing.

“Essentially, it is a site that creates a set of web services that I’ve always wanted for myself—services layered on top of the wealth of current search functionality and tools available for bloggers,” he wrote in a 2002 blog post.

Among those features were a number of basic tools that told users who was writing about or linking to a blog post, how your site rates in relation to specific Google search terms, historical information on how sites are ranking over time, and—most importantly—where everyone ranked.

The ranking element effectively made it a search engine for blogs, a great way to find new voices at a time when the “blogosphere” was a new and vibrant thing. And rather than grabbing this data only occasionally, as many search engines did at the time, it was built around the pulse of the digital ecosystem. Effectively, it filled an immediacy gap for blogs that early search engines were not designed for.

But the data it offered bloggers might have been the most important part of its reason for being.

These days, companies charge hundreds or thousands of dollars a month for the kinds of analytics Technorati generated. Sifry was charging $5 per year for access to historical information on Technorati. This was before we had Google Analytics or anything similar, so this added data was a big deal.

At first, Technorati was a side project for Sifry, just like Blogger was for Pyra Labs. But the cultural tide quickly changed things and made Technorati hugely valuable to large companies. In its first year online, it wasn’t just small-scale bloggers that were interested in the data Sifry had gathered, but major news outlets such as The New York Times and Reuters. As Inc. reported in 2006, Sifry was soon signing a deal to license his technology to AOL—and quitting his job to make Technorati a full-time thing.

Technorati swag, circa 2005. (Nikita Kashner/Flickr)

And, just like blogging itself, Technorati grew fast. To give you an idea, the company scored an impressive partnership with CNN tied to the 2004 Democratic National Convention. Sifry found himself blogging and reporting for CNN, rounding up a variety of voices that Technorati was well-positioned to highlight.

“I hope that this helps people in the living room realize that they can be participants,” Sifry said, according to PR Week.

Not bad for a company that less than two years before was a weekend project for Sifry.

Blogger and Technorati were two of the earliest Web 2.0 tools, predating Friendster, Facebook, MySpace, Digg, Reddit, and numerous other mid-2000s icons of blogging and social media.

At the center of this trend was the concept of the blog, which encouraged people to own their own piece of the internet. Many of the later Web 2.0 tools focused on trying to get people to spend as much time as possible within their URL; Technorati, meanwhile, was really focused on helping this universe of bloggers become better at blogging.

Other services that mined this territory, like Popdex and the Mark Cuban-backed IceRocket, played an important role in the rise of the blog-targeted search engine, whether by ranking the most popular sites or by making it easier to track information in real time. (After all, new posts from amateur pundits were always filtering in.) But none cast a shadow quite as large as that of Technorati, whose ranking system meant something for bloggers. To appear on the list, particularly in a high spot, was a sign of validation. Yes, you were good enough at blogging.

It was only a matter of time before bigger players intruded on Technorati’s territory. And yes, when I say “bigger players,” I mean Google.

Google Blog Search had a lot of Google News energy. (via Alia Al Seeri)

For a brief moment, blog search was a big deal. Until it wasn’t.

In as many ways as one company can, Google has spent much of the past two decades taking control of many of the general elements of running an independent website. For example, Google Analytics, which evolved from the analytics service Urchin, has long been a de facto standard for tracking website data.

And when the RSS feed gained popularity during the first half of the 2000s, Google launched one service, Google Reader, and acquired another, FeedBurner.

But the company is not known for nostalgia, and will remove support when it’s clear a trend has run its course—or Google wants that trend dead. And blogging, while not dead, is a trend the company has clearly moved past. Blogger (acquired in 2003), while still active, languishes, only receiving occasional notice from Google despite being one of the largest sites on the internet. So, too, does FeedBurner. And it famously killed Reader outright.

Such is the story of Google Blog Search, the company’s response to the rise of Technorati. It was there when blogging was a hot trend and gone by the time the trend had started to fade. Launched in 2005, it allowed users to search for individual blog posts, both by relevance (to see the most contextually relevant search results) and by date (to see the latest). It was faster than other options, but with fewer bells and whistles. Early bloggers were skeptical. In a 2005 post on TechCrunch, founder Michael Arrington noted that other tools, particularly Technorati, had more detailed search functionality, and that post relevance wasn’t as good as other options.

“Overall, significant room for improvement exists,” Arrington wrote.

Ultimately, the competition between Technorati and Google was a good thing, no matter who was actually “better” at the job. A 2007 Search Engine Journal blog post noted that the two sites ultimately complemented one another by touching on different technological aspects. While Google was better at a more fine-toothed approach by date, Technorati ultimately brought up better results and had more filtering capabilities.

Other companies, including Microsoft and Yahoo also dived into the space at the time.

For his part, Sifry, who remained CEO of Technorati until 2007, chose to stay the course in the face of the new competition, telling Inc. in 2006, that Google’s competitor was “solid but simple.” Nonetheless, the entry of Google into the space ensured an uphill battle for the little blogging search tool that could.

In the Inc. piece, a number of tech and marketing figures were asked to comment on Technorati’s then-current business model, and the critics seemed less than favorable, suggesting that it wasn’t focused enough, or that the company should focus on an acquisition. One expert quoted, Engadget cofounder Peter Rojas, admitted he was a heavy user of the site, even as he suggested it was not going to win the battle for blog search supremacy.

“Technorati’s strategy is not unique, so its execution has to be perfect,” Rojas stated. “Sooner or later, somebody’s going to get it right. I hate to say it, but it’s probably Google.”

In a way, Google did get it right by not overplaying its hand—creating an effective response to sites like Technorati but not letting them overwhelm their model. Blog Search was just good enough that you didn’t really need to go to another website to find useful results. But it didn’t give the blog outsized influence on its overall results, which was probably the right move, considering the fact that the idea of a blog search engine quickly fell out of date. By the time Twitter gained a footing in the mainstream in 2009—six short years after Ev Williams sold Blogger to Google—searching for things that people said in real-time was no longer a differentiator. It was table stakes.

(Perhaps reflecting this, Google Blog Search was effectively retired in 2011 and exists quietly as a search feature within Google News.)

Technorati tried building an ad business for bloggers starting in 2008. It eventually usurped the rest of the business. (via ProBlogger)

And that was bad news for Technorati, which eventually evolved into more of a services business, with website widgets and advertisements bringing elements of the Technorati experience across the web—even if it dampened the experience of the Technorati website. “Today, only a small percentage of Technorati’s total network traffic of 25 million U.S. unique visitors per month actually visit Technorati.com,” Michael Arrington wrote in TechCrunch in 2009.

As I mentioned earlier, walled gardens won the internet, and Technorati wasn’t a walled garden. It was also too small to win by sheer scale, like Google could.

Sure, its list of top blogs was considered definitive, but was that enough build a company around? In the end, the answer was no.