Via Yahoo! research scientist David Pennock of Odd Head and YooPick, the dear honorable Duncan Watts:
In part because of disappointing findings such as this, an increasingly popular substitute for expert opinions are so-called “-prediction markets,”- in which individuals buy and sell contracts on various outcomes, such as football game point spreads or presidential elections. The market prices for these contracts then effectively aggregate the knowledge and judgment of the many into a single prediction, which often turns out to be more accurate than all but the best individual guesses.
But even if these markets do perform better than experts, they don’-t necessarily do a good enough job to rely on. Recently, my colleagues have started tracking the performance of one popular prediction market, at forecasting the outcome of weekly NFL games. So far, what they’-re finding is that the market predictions are better than the simple rule of always betting on the home team, but only slightly so —- which, oddly, is very similar to what Tetlock found regarding his experts. Some outcomes, in other words, and possibly the outcomes we care about the most, simply aren’-t “-predictable”- in the way we would like.
–
Prediction markets are not “-a substitute for expert opinions”-. They are a substitute for the averaged probabilistic predictions of a large group of experts polled the traditional way (by phone or by e-mail). In prediction markets, traders (who are not experts, most of the times) collect and aggregate facts and expertise at a lower cost than a poll or survey of experts.
In the research cited by Ducan Watts, the prediction markets are slightly more accurate than the competitive forecasting mechanism. Well, that’-s something we are used to.
What Ducan Watts doesn’-t say is that prediction markets integrate facts and expertise faster than the group of experts polled by his researching colleagues —-for the very crude reason that it takes a certain time to survey a group of experts (be it by e-mail or by phone).
–
If I can count, that’-s 3 reasons why prediction markets can bring in business value:
lower cost-
better accuracy (relatively, and, overall)-
velocity.
That said, it should be repeated that prediction markets feed on facts and expertise —-so the experts remain indispensable in the general forecasting process.
No facts (e.g., political polls) –->- No prediction markets.
No experts (e.g., NFL prognosticators) –->- No prediction markets.
Learning in Investment Decisions: Evidence from Prediction Markets and Polls – (PDF file) – David S. Lee and Enrico Moretti – 2008-12-XX
In this paper, we explore how polls and prediction markets interact in the context of the 2008 U.S. Presidential election. We begin by presenting some evidence on the relative predictive power of polls and prediction markers. If almost all of the information that is relevant for predicting electoral outcomes is not captured in polling, then there is little reason to believe that prediction market prices should co-move with contemporaneous polling. If, at the other extreme, there is no useful information beyond what is already summarized by the current polls, then market prices should react to new polling information in a particular way. Using both a random walk and a simple autoregressive model, we find that the latter view appears more consistent with the data. Rather than anticipating significant changes in voter sentiment, the market price appears to be reacting to the release of the polling information.
We then outline and test a more formal model of investor learning. In the model, investors have a prior on the probability of victory of each candidate, and in each period they update this probability after receiving a noisy signal in the form of a poll. This Bayesian model indicates that the market price should be a function of the prior and each of the available signals, with weights reflecting their relative precision. It also indicates that more precise polls (i.e. polls with larger sample size) and earlier polls should have more effect on market prices, everything else constant. The empirical evidence is generally, although not completely, supportive of the predictions of the Bayesian model.
2 practicing physicians laugh at using collective intelligence for nation-wide flu detection:
[…] Flu Trends tracks almost perfectly with data on influenzalike illnesses that the CDC obtains from doctors’- offices. And as an added bonus, Flu Trends detects outbreaks up to two weeks earlier, when people are still sitting at home sneezing into their keyboards. […]
But if officials monitored only Flu Trends, it would be difficult to sort the signal from the noise —in addition to losing critical details on who is sick. Things besides an actual flu outbreak can cause people to search the Internet for flu information. We would imagine that Flu Trends would spike on the release date for a flu-related movie —maybe Outbreak 2: Electric Booga-Flu. And what happens if a pandemic flu scare hits the nightly news? Flu Trends’- ability to detect when the real pandemic hits will be obliterated when people, including those without symptoms, start to search the Internet. Monitoring drugstore sales has the same issue: A jump in cold-medicine sales may mean a flu outbreak, but it could also mean that CVS is running a sale or that flu fear is causing people to stock their medicine cabinets. […]
–
They end their articles saying that Google can’-t cure the flu, anyway. [???]
–
The response to the objections they jot down in the 2nd paragraph above is easy:
Informed by all other means, the event derivative traders can determine whether the spikes in Google Flu Trends are due to abnormalities (see the 2nd paragraph in the excerpt above) or due to the real spreading of influenza.
Hence, the flu prediction markets have a much higher social utility than Google Flu Trends. Chris Masse said so.
David Pennock, go writing another research paper about that.
History will retain that David Pennock was research scientist under Chris Masse’-s reign in the field of prediction markets.
–
Google Flu Trends
Iowa Health Prediction Market
The “predict flu using search” study you didn’t hear about – by our good Doctor David Pennock
BBC
New York Times
WSJ Health blog
University College Cork (UCC) School of Medicine + Intrade
http://www.cantorexchange.com/ —- A twin site of the Hollywood Stock Exchange.
Here are my early thoughts about the “-Cantor Exchange”-. I regret to say that their “-Cantor Exchange”- website does not seem very usable. (I hope it is not a bad omen.) It is impossible for me to copy their explanations posted on their webpages (other than the 2 press releases) and to republish that material in this post.
The second thought that comes to my mind is that their offerings are not standardized (their event derivatives are described in HSX lingo), and I wonder whether the real-money traders, who are accustomed to dealing with the CME, the NYSE, or InTrade, but who are not familiar with HSX, will make the effort to adapt. We will see. (I don’-t think that the HSX play-money traders will go speculating on this real-money prediction exchange.) The collateral question is, why would the Cantor Exchange (a brand-new exchange with not a single trader, as of today) be better positioned to organize event derivative markets on movie business than, say, HedgeStreet, InTrade, BetFair, or even the CME? Obviously, Cantor Fitzgerald (a bond broker) are thinking that they can leverage their Wall Street clientele and the HSX population to branch out and start up a brand-new, CFTC-regulated, real-money prediction exchange. It’-s quite a big bet. I say “-branch out”- because starting off a real-money prediction exchange (Cantor Exchange) is quite different than running a play-money prediction exchange (HSX). Just look at how difficult it has been for HedgeStreet, which started off in 2004, and escaped bankruptcy in 2007 thanks to their rescue by IG Index. After 4 years, HedgeStreet is still not profitable, and it will probably take more years before it turns the first profit. Let’-s wish a better future for Cantor Exchange.
I will update this present post, later on, linking to the reactions from the media and the blogs. (Cantor will be holding a conf call this Tuesday, so some media coverage will pop up over the next days.)
Here is a Financial Times news article, which won’-t tell you much more than the 2 press releases re-published below.
Here’-s Variety.
–
Press releases:
–
CANTOR ENTERTAINMENT TO ANNOUNCE MOVIE BOX OFFICE CONTRACTS
TO MOTION PICTURE INDUSTRY AND INVESTOR COMMUNITY
Cantor Fitzgerald Files Application for Domestic Box Office Receipt Contracts
Los Angeles, CA- New York, NY – (December 8, 2008) – Cantor Entertainment, which provides various services to the entertainment industry and owns the Hollywood Stock Exchange, is pleased to announce that Domestic Box Office Receipt contracts will soon be available to the motion picture industry and investor community. Cantor Fitzgerald, its parent company, announced earlier today that it has filed an application to launch the Cantor Exchange, whose first listed product will be Domestic Box Office Receipt contracts.
Domestic Box Office Receipt contracts will offer film finance professionals and traders a new opportunity to hedge and speculate on the theatrical performance of wide-release Hollywood movies. Domestic Box Office Receipt contracts will be a next generation film-financing tool that allows market participants to hedge risk and provides them profit opportunities based on the first four weeks of a film’s box office revenues.
“It’s clear from our conversations within the industry and investment community that there is a tremendous opportunity to introduce this exciting new tool to complement existing film financing alternatives. The market for Domestic Box Office Receipt contracts offers the motion picture industry, investment funds, banks and all other prospective investors a federally regulated trading exchange dedicated to the entertainment industry,” said Andrew L. Wing, President and Chief Executive Officer of Cantor Entertainment. “Our involvement in Domestic Box Office Receipt contracts reflects our continuing commitment to expand Cantor Entertainment’s numerous services in the entertainment industry.” The first Domestic Box Office Receipt contract is expected to be listed on the Cantor Exchange in the first quarter of 2009. Cantor Exchange is subject to final approval by the Commodity Futures Trading Commission (“CFTC”).
About Cantor Entertainment Cantor Entertainment, a division of Cantor Fitzgerald, L.P., provides services to the entertainment industry. Cantor Entertainment also owns the Hollywood Stock Exchange (www.HSX.com), the world’s leading virtual entertainment stock market.
About Cantor Fitzgerald, L.P. Cantor Fitzgerald is a leading global financial services firm. The Cantor Fitzgerald franchise includes institutional equity and debt sales and trading, investment banking, private equity, as well as other businesses and ventures. For over 60 years, Cantor Fitzgerald, a proven and resilient leader, has been committed to delivering a unique brand of unparalleled product expertise, innovative technology and customer service to its clients around the world. For more information, please visit www.cantor.com.
–
Cantor Fitzgerald Announces Application for Cantor Exchange
Domestic Box Office Receipt Contracts are Expected to be First Contract Market
A New Tool in Film Finance
NEW YORK–-(BUSINESS WIRE)–- Cantor Fitzgerald, L.P., a leading global financial services firm, announced today that it has filed an application with the Commodity Futures Trading Commission (“CFTC”) to launch the Cantor Exchange. Cantor Exchange intends to list Domestic Box Office Receipt contracts as the exchange’s first traded product.
“The Cantor Exchange and our intention to list Domestic Box Office Receipt contracts reflect our continuing commitment to innovation in the finance and entertainment sectors,” said Howard W. Lutnick, Chairman and Chief Executive Officer of Cantor Fitzgerald.
Subject to final regulatory approval of the Cantor Exchange application, Domestic Box Office Receipt contracts will offer film finance professionals and traders a new opportunity to hedge and speculate on the theatrical performance (ticket sales) of major film titles. Domestic Box Office Receipt contracts will be a next generation financial management tool that allows film professionals to hedge risk and provides speculative opportunities to other market participants based on the first four weeks of a film’s box office performance.
The first Domestic Box Office Receipt contract is expected to be listed on the Cantor Exchange in the first quarter of 2009, subject to final approval of the Cantor Exchange application by the CFTC.
About Cantor Exchange
Cantor Exchange is launching the first trading platform based on movie box office revenue, and expects to begin listing Domestic Box Office Receipt contracts in the first quarter of 2009, subject to final regulatory approval. Cantor Exchange is a division of Cantor Fitzgerald, L.P., one of the world’s leading financial services firms, and is partnered with Cantor Entertainment, another subsidiary of Cantor Fitzgerald, which provides services to the entertainment industry and owns the Hollywood Stock Exchange® (www.HSX.com), the world’s leading virtual entertainment stock market.
About Cantor Fitzgerald, L.P.
Cantor Fitzgerald is a leading global financial services firm. The Cantor Fitzgerald franchise includes institutional equity and debt sales and trading, investment banking, private equity, as well as other businesses and ventures. For over 60 years, Cantor Fitzgerald, a proven and resilient leader, has been committed to delivering a unique brand of unparalleled product expertise, innovative technology and customer service to its clients around the world. For more information, please visit www.cantor.com.
–
UPDATES WILL BE POSTED BELOW, LATER TODAY (AND DURING THE NEXT DAYS)…-
–
[T]he initiative is not for an application for a product extension of HSX. Rather it is an application for the launching of a new futures market, the Cantor Exchange, which will list Domestic Box Office Receipt Contracts. The contracts will also be known as Movie Box Office Contracts.
There are some similarities with HSX in that Movie Box Office Contracts are modeled on the MovieStock methodology of our site. For example, the contracts will be based on four weeks of a film’-s domestic box office revenue. The regulatory approval process is inherently uncertain, so it’-s a bit premature to say we are moving into real-money film trading markets just yet, but that is our intent.
–
That is from Alex Costakis, MD of HSX.
–
Here’-s the Cantor Exchange project leader: Richard Jaycobs.
He seems to be a man open to suggestions.
–
CNBC on Cantor Exchange —- Via Jason Ruspini
Fox Business on Cantor Exchange –-
–
Financial Times:
All eyes on Hollywood futures
–
Previously: Should the Hollywood Stock Exchange become a real-money betting exchange? – 2007-10-04
It’-s “-pretty clear”- that the prediction markets on political elections aggregate information from the polls —-and from the political experts.
–
Previously: #1 – #2 – #3 – #4 – #5 – #6
–
It’-s “-pretty clear”- that:
InTrade has been over-selling the predicting power of its prediction markets.
The prediction markets are information aggregation systems —-not magical tools.
The main benefit of a prediction market is to express an aggregated expected probability. Most of the times, this is of low utility.
In complicated situations, this aggregation will contrast well with a poor reporting. In these instances, the prediction market is a useful source of information.
Management struggles to understand and plan for the future. When forecasts are inaccurate, corporations incur huge costs due to inventory write-offs, stock-outs, misallocated resources or cost of capital. Collective intelligence delivers objective, accurate forecasts in real time, thus saving many millions of dollars for our corporate clients. The solution is not being oversold, to the contrary, the potential vastly exceeds current awareness and adoption.
“-If foresight is not the whole part of management, at least it is an essential part of it”- (Henri Fayol, 1916).
In my 10 years experience as a management accountant and corporate planner, I have witnessed multiple forecasts suffer from inaccuracy due to uncertainty and biases. Whether forecasting a launch date, sales volume or cost of development, it is the systematic biases due to incentive systems, politics and common cognitive errors that contribute more to inaccuracy than the uncertainty. The problem stems from the fact that the owner of a forecast is normally the owner of the business unit / sales team / project, and budgets and bonuses are based on forecasts. This necessitates game playing and politics and makes the development of an objective, accurate forecast near impossible.
Collective intelligence can overcome these problems by incentivising a diverse crowd of knowledgeable employees to share their insight, balancing the resulting estimates, and rewarding accuracy and timeliness.
However, we are at an early stage in the development of this opportunity. There is still work ahead of us to develop the ideal mechanism to combine simplicy of UI [user interface] with richness of information gathering. In addition, we need to further develop the way collective intelligence interfaces with traditional corporate structures, processes and systems. These are Xpree’-s challenges…- stay tuned.
According to Alan Abramowitz, John Tierney has been “-greatly exaggerating the accuracy of the betting markets.”- “-They follow the polls. That’s it.”-
–
–
My comment to Alan Abramowitz and John Tierney:
“-They follow the polls. That’s it.”-
Yes, they follow the polls. No, that’-s not it.
Traders also dig the news of the day and make anticipations about the outcome. For instance, towards the end of the 2008 Democratic primary, the polls and the mass media were still giving Hillary Clinton a very good standing, whereas the prediction markets (informed by a bunch of political experts who did the counting of the delegates and super-delegates) were telling us that she was as toasted as Lehman Brothers in the middle of the credit crunch crisis.
Are prediction markets useful? If John Tierney wants to answer this question, he should pick up a prediction market and put it in the social context of that day. Some prediction markets are more useful than others. In the case of the 2008 Democratic primary (a complicated matter), the prediction markets sided with the best informed political experts against the mass media and the polls. So to speak, they were an umpire. In that case, we see the emergence of a social utility. We now have the case for the media citing more the probabilities of the liquid (play-money and/or real-money) prediction markets.
John Tierney and Jed Christiansen are making the same mistake: they think that people and experts should be impressed by the information aggregation functionality of the prediction markets. They are not —-people still prefer reading Nate Silver and Electoral-Vote.com over InTrade, and the political experts have not added InTrade in their toolbox. (On this last point, do read the very last sentence of that interview.)
You won’-t be impacting if you publish enthusiastically about the features of the prediction markets —-yes, they do incorporate the latest news quickly, they quantify reasonable anticipations, they output probabilities, and they are relatively unbiased. You will be impacting the day you are able to demonstrate the benefits of the prediction markets —-for people, on one hand, and for the experts, on the other hand.
This would require a new focus, and a much bigger effort.
The social utility of most prediction markets is minimal —-busy people (who don’-t have time to read extensively the news) get relatively objective probabilities, real quick. But very few companies are using enterprise prediction markets, as of today. If these new IAM tools were magical (as some sur-excited free-market proponents think they are), all the Fortune-500 companies without any exception of any kind would be using them today.
If you want to discover the true benefits of the prediction markets, you should first be able to rank them by degree of utility. Which ones are more useful than others? Why? To answer this last question, you have to lay out the panorama of all the information sources that people and expert have access to, these days. What were the specific instances where the prediction markets were a tie breaker between the experts and the mass media, or between the decision makers and the experts, or between 2 opposite groups of experts? You should build an airtight, documented case. I haven’-t seen such a case, yet. If some of my readers are interested in such a project, let’-s talk.
InTrade got it [almost] spot on because they were wrong on Missouri (which was predicted to go for Obama but went to McCain) and wrong too on Indiana (which was predicted to go for McCain but went to Obama) —and those 2 opposite mistakes canceled themselves because those 2 states have the exact same number of electoral votes (11). Hence, I disagree with your method.
–
–
APPENDIX:
Here’-s a visual post-mortem of the 2008 US presidential elections.
Pay attention to Missouri and Indiana.
–
A) InTrade, on November 5, 2008 (screen shot taken at 2:00 am):
–
–
Prediction Markets &- State Polls, on November 4, 2008:
–
–
B1) Prediction Markets (on November 4, 2008)
–
–
InTrade (screen shot taken at mid-day ET, November 4, 2008):
–
InTrade (screen shot taken in the morning, November 4, 2008):
–
BetFair (screen shot taken in the morning, November 4, 2008):
–
HubDub (screen shot taken in the morning, November 4, 2008):
–
–
–
–
–
B2) State Polls (on November 4, 2008)
–
–
Karl Rove (on November 4, 2008):
–
CNN (on November 4, 2008):
–
Pollster (on November 4, 2008):
–
Electoral-Vote.com (on November 4, 2008):
–
Nate Silver (on November 4, 2008):
–
–
PREDICTION MARKET PROBABILITIES
–
Explainer On Prediction Markets
–
A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain event (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.
Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism —-with or without an automated market maker.
Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future —-anticipations that will be either confirmed or infirmed.
The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.
–
More Info:
– The Best Resources On Prediction Markets = The Best External Web Links + The Best Midas Oracle Posts
– Prediction Market Science
– The Midas Oracle Explainers On Prediction Markets
– All The Midas Oracle Explainers On Prediction Markets
