One of the magical things about Twitter is that it opens a window to the world in real-time. An event happens, and just seconds later, it’s shared for people across the planet to see.
...when Flight 1549 crashed in the Hudson, @jkrums on twitter shared, "
http://twitpic.com/135xa - There's a plane in the Hudson. I'm on the ferry going to pick up the people. Crazy. 3:36 PM - 15 Jan 2009
When [other] major events happened, such as when Osama bin Laden was killed, and when Mitt Romney mentioned binders during the presidential debates, people instantly came to Twitter – and, in particular, Twitter search – to discover what was happening.
...[there's a] real-time human computation engine we built that allows us to find search queries as soon as they’re trending, ...[these are sent to] real humans to be judged...
...Overview....how the system works.
(1) ...we monitor for which search queries are currently popular.
Behind the scenes: we run a Storm topology that tracks statistics on search queries.
For example: the query “Big Bird” may be averaging zero searches a day, but at 6pm on October 3, we suddenly see a spike in searches from the US.
(2) Next, as soon as we discover a new popular search query, we send it to our human evaluation systems, where judges are asked a variety of questions about the query.
Behind the scenes: when [we detect] that a query has reached sufficient popularity, it ...dispatches the query to Amazon’s Mechanical Turk service, and then polls Mechanical Turk for a response.
For example: as soon as we notice “Big Bird” spiking, we may ask judges on Mechanical Turk to categorize the query, or provide other information (e.g., whether there are likely to be interesting pictures of the query, or whether the query is about a person or an event) that helps us serve relevant tweets and ads.
Photo by Dixon Tam, Flickr CC