“I have always thought we needed to index the web every second to allow real time search. At first, my team laughed and did not believe me. With Twitter, now they know they have to do it. Not everybody needs sub-second indexing but people are getting pretty excited about realtime.”
Larry Page, Google co-founder
While watching (with some mixture of amazement and disbelief) the buzz on Twitter during Michael Jackson's Memorial, I became curious about potential real-time monitoring of KFC after Magic Johnson confessed that the greatest moment in his life was eating KFC with the Gloved One. Ahem.
For the record, there was a total of about 800,000 Facebook status updates during the Jackson Memorial.
6,000 updates per minute at its peak.
But there I go again — digressing.
Real-time search is the buzzword of the month. I decided to tackle this topic and began digging through several of the engines (ie: http://www.oneriot.com) until I realized this had been done in a far superior fashion at VentureBeat. Go check out their fine article for some in-depth perspective on the players: Who Rules real-time search? A look at 11 contenders.
So rather than a comprehensive look at the search engine universe, allow me to offer some of my thoughts on this emerging technology as there's a tremendous amount of hype around 'real-time' search. First of all, Google's indexing is fast. Uber fast. You can even search the twitter site (with Google) for near real-time tweets, but it runs 10-15 minutes behind. Secondly, anyone who's done any type of reputation management on the web knows there are a myriad of tools to help keep track of any and all mentions of your brand/vertical in darn near real-time. So there's that.
And tying into my opening quote, Google is all over this.
Google’s Marissa Mayer in a Guardian article about the potential that can be mined from real time data:
“We think the real-time search is incredibly important and the real-time data that’s coming online can be super-useful in terms of us finding out something like, you know, is this conference today any good? Is it warmer in San Francisco than it is in Silicon Valley? You can actually look at tweets and see those sorts of patterns, so there’s a lot of useful information about real time and your actions that we think ultimately will reinvent search.”
That being said, news and/or blog search can't match real-time data. Which is Twitter.
And for that, Twitter search has no peer.
But allow me to segue a moment and mention a few other other twitter tools I've been playing with. There's PeopleBrowsr (in Beta and still stuck in fancypants syndrome) but offers a unique interface to track mentions. Twitterfall is a very cool tool for real-time monitoring of Twitter. I used it during the presidential debates last year and it provided much real-time insight and hilarity. As an aside, there's also PicFog, a real-time Twitter image search tool.
But back to Twitter search and it's value. There's a veritable plethora of advertising options available within this framework. Imagine being able to show Comcast ads to users complaining about a Time-Warner outage in real-time. Tip of the iceberg, my friends.
Microsoft and it's newly refurbished Live search is trying to make a case for what they can do. Am I the only one who thinks it's humorous that Bing (enjoying a small honeymoon spike in usage thanks to a hundred million dollar ad campaign) decided to tout it's real-time search capabilities by incorporating celebrity tweets into its search results?
Nearly two weeks ago, the Bing Community Search Blog announced,
“We're unveiling an initial foray into integrating more real time data into our search results, starting with some of the more prominent and prolific Twitterers from a variety of spheres. When you search for these folks names in association with Twitter, you'll see their latest Tweets come up in real time on Bing's search results.”
Is this what constitutes breakthrough technology over at Bing? Real-time tweets from Shaq and Ashton Kutcher?
Keep up the fine work, Microsoft. You're an inspiration to us all.
So let's get to the heart of it.
Twitter's firehouse feed. Anyone who wants to index in real-time needs access to this — essentially the pipeline of real-time data. Tons and tons of it.
As of last year, the Twitter blog reported that only four companies had access to the data: Twittervision, Zappos, FriendFeed, and Summize. This last partner was purchased by Twitter and now powers their own search.
And you can guess who wants to partner up with Twitter to have access to the firehouse feed? Yup, the Big Three: Google, Bing and Yahoo.
But first they've got to pay up.
And while the search model of Twitter as Google killer is ludicrous, if they do provide the 800 lb gorilla (or the other two) with the firehouse feed, they lose the unique ability to be the only service to search all of their tweets as they appear. I mean, no one has better access to the up-to-the-second universe of tweets than Twitter.
There's some serious value in that.
So one can assume that they are holding out for the big payday in exchange for giving that up.
So for now it's a waiting game. A waiting game with much at stake as marketers (and consumers) become more adept and demanding at using this data.
As things continue to evolve, real-time search will eventually become ubiquitous and we'll wonder what the big deal was. But there are still so many factors to consider — what about spam? How does relevance or trust or PageRank convert into the real-time search world?
I can't wait to see.