Like Bitly, Twitter has a great real-time data set and very smart data scientists and engineers. But instead of relying on a primarily computational solution, Twitter treats real-time search more like a CAPTCHA problem. With this kind of messy data, lots of human brains can find meaning much faster and more accurately than lots of lines of code. So Twitter uses a real-time computation system called Storm to identify search spikes, then Mechanical Turk (Amazon’s crowdsourcing online platform for small jobs) to farm out annotating that data to human beings all over the world. The annotations basically take the spiking search term and tag it for relevance and intent. A human annotator (Twitter calls them “judges”) can tell Twitter’s systems whether searches for “Stanford” refer to a university or to its football team, or that searches for “Big Bird” aren’t primarily referencing a children’s show, but a political debate. This helps Twitter make trending topics smarter and more coherent.

But here’s the dark stroke of genius behind using huge masses of people to help sort out the meaning of Twitter searches: part of the judges’ task is also to match spiking search terms with pictures, events, and other categories that can help Twitter serve up relevant advertising. “For example, suppose our evaluators tell us that [Big Bird] is related to politics; the next time someone performs this search, we know to surface ads by @barackobama or @mittromney, not ads about Dora the Explorer.” The judges are like little focus groups that match intent with revenue.

57 notes

Show

  1. knav3rs3ong reblogged this from darkthoughtsbrightdays
  2. stressfm-feed reblogged this from new-aesthetic
  3. iscarlets reblogged this from new-aesthetic and added:
    Appunto. Come fai? È un casino. Gli algoritmi da soli non ce la fanno, e se sbagliano può darsi che finisci per piazzare...
  4. novemberdepth reblogged this from iamdanw
  5. iamdanw reblogged this from new-aesthetic
  6. tonyhschu reblogged this from new-aesthetic
  7. loganlape reblogged this from new-aesthetic
  8. julianstahnke reblogged this from new-aesthetic
  9. maltstream reblogged this from hibernationstation
  10. mydaddyisarobot reblogged this from new-aesthetic
  11. hibernationstation reblogged this from new-aesthetic
  12. darkthoughtsbrightdays reblogged this from new-aesthetic
  13. dackdel reblogged this from new-aesthetic
  14. christopherpatricktaylor reblogged this from new-aesthetic
  15. artlung reblogged this from new-aesthetic
  16. somethinginparticular reblogged this from new-aesthetic
  17. gadevoi reblogged this from new-aesthetic