Panos Ipeirotis depicted my activism to blog about the tone of the news article on enterprise prediction markets published by The Economist —-and its implications. The takeaway that his readers will get is that Chris Masse is on caffeine. 
The important information that our good doctor Panos did not tell is that Chris Masse is the publisher of 2 websites (CFM since 2003 and Midas Oracle since 2006) whose main purpose is to list and/or excerpt the news articles, opinion pieces and research papers that focus on prediction markets. Since 2003, I have seen them all —-in all stripes and colors.
So, it is not like I am a gullible newbie just out of the egg. I have a certain expertise in assessing any media piece on prediction markets. And the same thing can be said about Niall O’-Connor (regarding the betting industry, more generally).
So there we have it – objectivity out of the window, the “wisdom of corwds” as encapuslated in Mechanical Turk in.
This piece of research is the equivalent of a stock market “bear trap.” Anybody that rushes to embrace it to support their pro-prediction market stance, will only serve to make themselves look a bigger fool than they already do.
Let’s have twelve independent and fair minded academics look at the article, and see what they have got to say as to whether its stance is positive or negative vis a vis the prediction market industry.
Niall, I agree. But more than academics, I would like to see what CEOs (the one who would buy EPM services) think about the article from The Economist.
Niall: I find it kind of interesting to mock the “wisdom of the crowds” in a blog about prediction markets…
And the question asked was very very simply: Is the article positive or negative about the concept of prediction markets? Why is it so hard to see that the article is *not* negative? Neutral, yes; indifferent, yes; below expectations; maybe. But no, most of the people that read the article do get the impression that the Economist published an article that is negative and a disaster for the prediction markets.
Panos, you are right, but you forget the [context]. To take off, a brand-new forecasting tool needs to get some “wow” things. The Economist said there is no “Wow”, there is a “all that for that?” ending.
“you forget the concept.”
I meant, you forget the context.
Sorry for the typo.
Chris, the wow moment has happened long ago. The fact that may people were looking at the prediction markets for this years’ election shows the traction.
No technology is accepted in the enterprise through “wow” moments. Things are much much *much* slower. Many technologies have very slow adoption curves. Think about the “the network is the computer” idea from Sun in 1994. About the “thin clients to do computing on the Internet” idea, also in the 1990’s. These ideas are just now becoming mainstream.
Why do you think that prediction markets would take off so much faster as an enterprise tool? We need lots of patience and many small, incremental steps. And, the more people talk about *both* the successes *and* the failures during deployment and adoption, the easier it will be for everyone to fix the issues.
Panos,
Your comment is very well thought, and I acknowledge it.
However, put yourself in the shoes of a CEO reading The Economist on enterprise prediction markets. Why would a CEO pay money for such a tool when The Economist said that nothing great can be expected from it?
@Niall – You seem to miss the point of collective intelligence, wisdom of crowds, and prediction markets. Why would 12 academics be better at evaluating the tone of the article, and moreover, shouldn’t we care more about what a diverse group of people think?
Many persons (tetlock) have repeatedly written about the falibility of the expert. This entire field is to some extent based on this premise. Experts aren’t really experts, and we are better off collecting data from a diverse group of people.
@Chris – What if I’m a CEO reading the economist and I’ve never heard of prediction markets before. Without giving a hard sell on the technology, the article has now informed me about prediction markets, and possibly piqued my interest.
I think Panos comments above are spot on and his use of mechanical turk to evaluate the article sentiment is awesome.
Come off it already, Chris. You’re making a mountain out of a molehill with the Economist thing. And you ARE on caffeine…
=)
“You seem to miss the point of collective intelligence, wisdom of crowds, and prediction markets.”
Daniel, Mechanical Turk is a very basic collective intelligence mechanism as the only judgment aggregation mechanism is the averaging of opinions taken from idle people hired on the cheap (pennies) on the Web at large.
“Why would 12 academics be better at evaluating the tone of the article,”
Daniel, you are not serious. PhD people have better judgment that idiots hired on the cheap on the Web.
“Without giving a hard sell on the technology, the article has now informed me about prediction markets, and possibly piqued my interest.”
See Niall O’Connor’s reading of the article.
–
“You’re making a mountain out of a molehill with the Economist thing.”
Time will tell. We need a prediction market on that.
“Daniel, Mechanical Turk is a very basic collective intelligence mechanism as the only judgment aggregation mechanism is the averaging of opinions taken from idle people hired on the cheap (pennies) on the Web at large.”
Meanwhile, the people that are betting (aka gambling) on presidential election markets are only valedictorians, with a couple of PhDs (one in psychology of the crowds and the other in financial engineering). Maybe the target audience for BetFair, Intrade, and all the other betting exchanges are the well-informed CEO’s, political analysts, and the members of the think tanks.
Come on Chris, Mechanical Turk has the same people that bet online and generate the prices for the prediction markets. Everyday people in one website, everyday people in another. They gamble on the prediction markets, they answer sub-minute questions for a few pennies on Mechanical Turk. Still, the collective accuracy is great.
Are we going to start questioning whether collective intelligence exists, whenever the results are not to our liking?
I posted the task on MTurk without even knowing what to expect. It may have proven Michael wrong. In that was the case, I am sure that Mechanical Turk would be glorified as the tool that shows to the stupid academics how to read stuff like common-sense people.
“Come on Chris, Mechanical Turk has the same people that bet online and generate the prices for the prediction markets.”
Maybe there are more women on Mechanical Turk…
http://www.midasoracle.org/200…..ment-23647
“Daniel, Mechanical Turk is a very basic collective intelligence mechanism as the only judgment aggregation mechanism is the averaging of opinions taken from idle people hired on the cheap (pennies) on the Web at large.”
Not sure how mechanism design or exact incentive is important here. I also believe Mechanical Turk is a promising service/technology that has yet to take off. And, as Panos asks, why this group of people is any different than those using public facing prediction markets.
“Daniel, you are not serious. PhD people have better judgment that idiots hired on the cheap on the Web.”
Actually, you would be selecting a biased group of individuals, so yes I do believe that a random diverse group of people will be better at this task. Again, this is central to the ideas of collective intelligence, wisdom of crowds, and markets as better decision making mechanisms. Of course, there is plenty of research supporting this assertion, maybe you should check your archives.
“Not sure how mechanism design or exact incentive is important here.”
James Surowieki insists on a good mechanism for aggregating judgment from all people.
“Actually, you would be selecting a biased group of individuals, so yes I do believe that a random diverse group of people will be better at this task”
Can we have both? PhD people selected at random?
So there we have it – objectivity out of the window, the “wisdom of corwds” as encapuslated in Mechanical Turk in.
This piece of research is the equivalent of a stock market “bear trap.” Anybody that rushes to embrace it to support their pro-prediction market stance, will only serve to make themselves look a bigger fool than they already do.
Let’s have twelve independent and fair minded academics look at the article, and see what they have got to say as to whether its stance is positive or negative vis a vis the prediction market industry.
Niall, I agree. But more than academics, I would like to see what CEOs (the one who would buy EPM services) think about the article from The Economist.
Niall: I find it kind of interesting to mock the “wisdom of the crowds” in a blog about prediction markets…
And the question asked was very very simply: Is the article positive or negative about the concept of prediction markets? Why is it so hard to see that the article is *not* negative? Neutral, yes; indifferent, yes; below expectations; maybe. But no, most of the people that read the article do get the impression that the Economist published an article that is negative and a disaster for the prediction markets.
Panos, you are right, but you forget the [context]. To take off, a brand-new forecasting tool needs to get some “wow” things. The Economist said there is no “Wow”, there is a “all that for that?” ending.
“you forget the concept.”
I meant, you forget the context.
Sorry for the typo.
Chris, the wow moment has happened long ago. The fact that may people were looking at the prediction markets for this years’ election shows the traction.
No technology is accepted in the enterprise through “wow” moments. Things are much much *much* slower. Many technologies have very slow adoption curves. Think about the “the network is the computer” idea from Sun in 1994. About the “thin clients to do computing on the Internet” idea, also in the 1990’s. These ideas are just now becoming mainstream.
Why do you think that prediction markets would take off so much faster as an enterprise tool? We need lots of patience and many small, incremental steps. And, the more people talk about *both* the successes *and* the failures during deployment and adoption, the easier it will be for everyone to fix the issues.
Panos,
Your comment is very well thought, and I acknowledge it.
However, put yourself in the shoes of a CEO reading The Economist on enterprise prediction markets. Why would a CEO pay money for such a tool when The Economist said that nothing great can be expected from it?
@Niall – You seem to miss the point of collective intelligence, wisdom of crowds, and prediction markets. Why would 12 academics be better at evaluating the tone of the article, and moreover, shouldn’t we care more about what a diverse group of people think?
Many persons (tetlock) have repeatedly written about the falibility of the expert. This entire field is to some extent based on this premise. Experts aren’t really experts, and we are better off collecting data from a diverse group of people.
@Chris – What if I’m a CEO reading the economist and I’ve never heard of prediction markets before. Without giving a hard sell on the technology, the article has now informed me about prediction markets, and possibly piqued my interest.
I think Panos comments above are spot on and his use of mechanical turk to evaluate the article sentiment is awesome.
Come off it already, Chris. You’re making a mountain out of a molehill with the Economist thing. And you ARE on caffeine…
=)
“You seem to miss the point of collective intelligence, wisdom of crowds, and prediction markets.”
Daniel, Mechanical Turk is a very basic collective intelligence mechanism as the only judgment aggregation mechanism is the averaging of opinions taken from idle people hired on the cheap (pennies) on the Web at large.
“Why would 12 academics be better at evaluating the tone of the article,”
Daniel, you are not serious. PhD people have better judgment that idiots hired on the cheap on the Web.
“Without giving a hard sell on the technology, the article has now informed me about prediction markets, and possibly piqued my interest.”
See Niall O’Connor’s reading of the article.
–
“You’re making a mountain out of a molehill with the Economist thing.”
Time will tell. We need a prediction market on that.
“Daniel, Mechanical Turk is a very basic collective intelligence mechanism as the only judgment aggregation mechanism is the averaging of opinions taken from idle people hired on the cheap (pennies) on the Web at large.”
Meanwhile, the people that are betting (aka gambling) on presidential election markets are only valedictorians, with a couple of PhDs (one in psychology of the crowds and the other in financial engineering). Maybe the target audience for BetFair, Intrade, and all the other betting exchanges are the well-informed CEO’s, political analysts, and the members of the think tanks.
Come on Chris, Mechanical Turk has the same people that bet online and generate the prices for the prediction markets. Everyday people in one website, everyday people in another. They gamble on the prediction markets, they answer sub-minute questions for a few pennies on Mechanical Turk. Still, the collective accuracy is great.
Are we going to start questioning whether collective intelligence exists, whenever the results are not to our liking?
I posted the task on MTurk without even knowing what to expect. It may have proven Michael wrong. In that was the case, I am sure that Mechanical Turk would be glorified as the tool that shows to the stupid academics how to read stuff like common-sense people.
“Come on Chris, Mechanical Turk has the same people that bet online and generate the prices for the prediction markets.”
Maybe there are more women on Mechanical Turk…
http://www.midasoracle.org/200…..ment-23647
“Daniel, Mechanical Turk is a very basic collective intelligence mechanism as the only judgment aggregation mechanism is the averaging of opinions taken from idle people hired on the cheap (pennies) on the Web at large.”
Not sure how mechanism design or exact incentive is important here. I also believe Mechanical Turk is a promising service/technology that has yet to take off. And, as Panos asks, why this group of people is any different than those using public facing prediction markets.
“Daniel, you are not serious. PhD people have better judgment that idiots hired on the cheap on the Web.”
Actually, you would be selecting a biased group of individuals, so yes I do believe that a random diverse group of people will be better at this task. Again, this is central to the ideas of collective intelligence, wisdom of crowds, and markets as better decision making mechanisms. Of course, there is plenty of research supporting this assertion, maybe you should check your archives.
“Not sure how mechanism design or exact incentive is important here.”
James Surowieki insists on a good mechanism for aggregating judgment from all people.
“Actually, you would be selecting a biased group of individuals, so yes I do believe that a random diverse group of people will be better at this task”
Can we have both? PhD people selected at random?