Knowledge @ Wharton (on polls):
The Power of Prediction Markets
The rapidly changing landscape of responders and related technology factors are two reasons why Justin Wolfers, Wharton professor of business and public policy, believes in the power of prediction, or betting, markets. Wolfers —- who is associated with several prediction market sites such as InTrade.com or Tradesports.com [*], where participants buy and sell contracts on sports and potential political outcomes —- argues that prediction markets are a more reliable outcome predictor than polls, for three reasons.
“-First, by forcing you to ‘-put your money where your mouth is,’- they yield truthful revelation of beliefs,”- Wolfers notes in a paper on pricing political risks with prediction markets. “-Second, markets provide profit opportunities for those willing to gather new information that helps predict the future. And third, markets aggregate information dispersed across many traders.”-
“-You are not asking who they will vote for, but who they think will win,”- says Wolfers. “-The evidence is overwhelming that prediction markets provide a more accurate prediction than polls. On average, the final forecast from a Gallup poll is within about 2.25 percentage points, and the average for prediction markets is 1.5 percentage points.”-
He points out that “-the idea of betting on presidential elections is not new at all. Betting on elections has been going on for the last 100 years. If you read The New York Times from the turn of the century [**], they will report what is in the prediction markets —- called ‘-betting markets’- back then —- and not polls, which hadn’-t yet been invented. But since 1940, the elections have been dominated by polls.”-
Wolfers predicts that “-within a few years and a couple of election cycles, we will be back to tracking political markets through the lens of prediction markets instead of polls. [***] In fact, in the last few election cycles, we have seen political commentators talking more and more about the race in light of prediction markets.”-
[*] What kind of association is it? I take it that it is an informal association. I have never seen anything written on the InTrade-TradeSports sites.
[**] Thanks to Paul Rhode and Koleman Strumpf. PDF file
[***] I would be more prudent. I’-d say that more and more commentators will follow the prediction markets, but the polls will remain the dominant barometer.
—-
UPDATE: Emile Servan-Schreiber (NewsFutures CEO) comments…-
1) The traders themselves are the first to look at the polls to inform their trades. So the polls are here to stay.
2) Our recent experience in Western Europe seems to indicate that the superior accuracy of markets over polls when predicting elections may be a U.S. artifact that isn’-t so easily reproducible elsewhere. I’-ve discussed this with Forrest Nelson of IEM [Iowa Electronic Markets], and apparently, ever since the Truman-Dewey polling debacle of 1948, U.S. pollsters have adopted a policy of reporting mostly raw numbers rather than projections based on sophisticated secret formulas, so they can’-t be accused of manipulating opinion. However, raw numbers are notoriously unreliable when based on small samples, and Western European pollsters never report them, preferring instead to publish projections based on historically-informed statistical formulas. What we’-ve observed in France and Holland is that it it’-s very hard to beat the accuracy of such projections.
1) The traders themselves are the first to look at the polls to inform their trades. So the polls are here to stay.
2) Our recent experience in Western Europe seems to indicate that the superior accuracy of markets over polls when predicting elections may be a U.S. artifact that isn’t so easily reproducible elsewhere. I’ve discussed this with Forrest Nelson of IEM, and apparently, ever since the Truman-Dewey polling debacle of 1948, U.S. pollsters have adopted a policy of reporting mostly raw numbers rather than projections based on sophisticated secret formulas, so they can’t be accused of manipulating opinion. However, raw numbers are notoriously unreliable when based on small samples, and Western European pollsters never report them, preferring instead to publish projections based on historically-informed statistical formulas. What we’ve observed in France and Holland is that it it’s very hard to beat the accuracy of such projections.
A question for you guys – if we want to compare prediction markets to polls, we have to compare all factors.
From what I’ve see so far in all reported experiments and comparisons, size also plays an important factor.
That is, prediction markets can reach better (or similar) results, but with a much smaller audience (<10%).
Question is – what’s your experience regarding this item?
And another observation –
if we (prediction markets vendors) want to convince the world to use us, we at least have to agree on the “Facts”.
For that matter, the comparison to surveys and polls will always pop up – and we need to have good (and consistent) responses in our arsenal.
So – how about summarizing all the facts so far, and let’s try to agree on this one?
We could always list the top factors/advantages, and then have a poll here in midas oracle to decide…
A bit ironic (using a poll…), but still may be useful.
Noam,
I agree with Naom. It is notable, that prediction market desciples have seemingly turned a blind eye to a paper by Robert S. Erikson and Christopher Wlezien entitled
“Are Political Markets Really Superior to Polls as Election Predictors?”
The authors argue that as prediction market prices reflect forecasts of what will happen on Election Day and trial-heat polls only register preferences on the day of the poll, it is inappropriate to naively compare them on any given day in advance of an election.
Transforming raw poll vote divisions into projections of the Election Day outcome and comparing these projections to vote-share prices they find that daily poll projections are superior to Iowa Electronic Market (IEM) prices. Indeed, in three of the five presidential elections with IEM vote share markets, poll projections were more accurate than market prices. In four of the five elections (with one tie), the week’s average poll projection dominated the daily market price.
The authors assert, on the basis of their study, that it is quite clear that the prices in the IEM vote share market are no better, and in fact they are a bit worse, at predicting the popular vote, than are projections based on the day’s most recent polls; “Where the market holds to a view of the election at odds with the poll projections, it is somewhat more likely to be wrong than right”.
The authors conclude that by their tests the IEM prediction markets are not better than trial-heat polls for predicting elections. In fact, they say, by a reasonable as opposed to naive reading of the polls, the polls dominate the markets as an election forecaster;
“The Iowa election market’s performance has not been so special after all. For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates”.
In “Models, markets, polls and pundits: a case study of informational efficiency” Leighton Vaughan Williams offered a brief perspective on alternative methadologies designed to gauge the state of the 2004 US Presidential election race at a given point in time and to forecast the outcome of the race.
Comparing a number of different betting markets Williams determined the probability of a popular vote victory for each candidate. With regard to George Bush the probabilities of his victory on the individual markets studied were as follows; IEM – 57.9%; Traditional Bookmakers – 62.5%; Betfair – 61.7%; Tradesports – 61%; Play Money Exchanges – 53.5%; Spread Betting Markets – 62%.
Not ones to let the truth get in the way of a good story, advocates of prediction markets may wish to turn a blind eye to these two most revealing papers.
[…] a myriad of others), I break away from Justin Wolfers’ irrational exuberance and I side with Emile Servan-Schreiber of NewsFutures (my preferred play-money prediction exchange). Prediction market reporting will have a function, […]