Mechanical Turk grades The Economists news article on enterprise prediction markets

No Gravatar

Panos Ipeirotis:

economist-survey

Well, the average was a 5.8/10, meaning that the average detected sentiment was pretty much neutral with some hints on positivity.

I acknowledge this result, brought to us by research scientist and university professor Panos Ipeirotis&#8230- who, 5 minutes ago, was alerting us on the hard fact that Mechanical Turk is not so much of a reliable tool&#8230-

Previously: Enterprise prediction markets… the next big thing —not.

Could a statistical reputation system built on top of Amazons Mechanical Turk be of any help to the prediction market firms?

No Gravatar

Obama-Cinton 1

Obama-Cinton 2

I leave the mic to David Pennock so that he&#8217-ll explain to you what this is all about. (Be sure to check Panos Ipeirotis&#8216- comment on Lukas Biewald&#8216- blog post.) [UPDATE: Panos has just made it into Read &amp- Write Web!!!!!!]

Mechanical Turk

New York Times

I am more interested in what &#8220-Dolores Labs&#8221- does for a living. Here&#8217-s what Lukas Biewald has e-mailed me:

We collect large data sets for fun and profit using Mechanical Turk and a statistical reputation system that we&#8217-ve build on top of it. We&#8217-ve helped companies with tasks ranging from deciding if an analysts opinion about a stock is positive or negative to finding the best price for buying a hot tub online. (You can see other examples at http://doloreslabs.com/examples.html).

Recently we decided to take our small PR budget and instead of buying Google Ads, give our smart friends interesting data sets to play with in exchange for posting on our blog. So far, we&#8217-ve looked at race on Sports Illustrated covers over time (http://blog.doloreslabs.com/?p=10), compared Hillary vs Obama coverage in the media (http://blog.doloreslabs.com/?p=21) and collected subjective data on colors (http://blog.doloreslabs.com/?p=11). The beauty of these experiments is we can do them in just a day or two and they only cost us a few hundred dollars, so we can do lots of them.

We have lots of ideas of more stuff we want to do with our technology &#8212- if you have any thoughts I&#8217-d love to hear them.

Fascinating.

Any applications for the field of prediction markets (InTrade-TradeSports, BetFair-TradeFair, etc.)??

Previous blog posts by Chris F. Masse:

  • IIF’s SIG on Prediction Markets
  • Science
  • Why did prediction markets do well in the pre-polling era, professor Strumpf?
  • Mozilla FireFox users, do you have trouble downloading academic papers (as PDF files) from SSRN?
  • “Impact Matrix. Used to collect and gauge the likelihood and business impact of various events in the very long term.”
  • Ends and Means of Prediction Markets — Tom W. Bell Edition
  • How to run enterprise prediction markets… legally