Jed Christiansen:
[…] When it comes to the first point, forecasting something that the company already forecasts, prediction markets may or may not be an excellent solution. I’ve seen one set of markets that absolutely blew away the accuracy of current forecasts, and I’ve seen other markets that were consistent with current forecasts with little or no accuracy edge. […]
Care to say more about what is the determinant of an EPM success?
I described in a previous post why I delisted his company from my list of prediction market consultants.
I want to share a remark with you, today. Here is a man from Holland who recruited by e-mail some US-based “-advisors”- —-one ocean away. One curious online recruit he made is professor Christopher Wlezien, the co-author of an academic paper…- that claims that prediction markets are *NO* better than damped polls:
For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.
I bet that Florian Riahi didn’-t read that paper, and I bet that professor Christopher Wlezien accepted the advisory slot in order to make the simple point that the “-prediction market advocates”- are just a bunch of baloneys who don’-t read academic papers. 
–
Previously:
– How that prediction market consultant in Holland attracts economic advisers on the cheap
– I bet that those academic scholars…
COALITION FOR INTERNAL MARKETS’- LETTER TO THE CFTC
PDF FILE
Forecasting Principles:
Damping polls
Evidence from the literature shows that polls, in particular early in the campaign, are not reliable in predicting election outcomes but tend to overestimate the extent to which a candidate leads. To deal with these uncertainties, we added a damping factor to the RCP poll average. Damping is used to make forecasts more conservative in situations involving high uncertainty.
Our poll damping is based on research conducted by Campbell (1996) who showed that polls have to be discounted in order to achieve more reliable forecasts. Performing a regression analysis on historical poll data for the elections from 1948 to 2004, he derived a formula for discounting the polls according to their distance from Election Day. Campbell provided Polly the formula, along with a list of damping factors that vary by the number of days left before the election.
Currently, polls are discounted with a damping factor of ${factor}. Applying this factor, we calculate Polly’-s discounted poll based forecast thus:
Polly’-s poll based forecast = ((Latest RCP polling average – 50) * (1 – [damping factor])) + 50 = ((46.1 – 50) * (1 – 0.17)) + 50 = 46.8
Latest RCP polling average 46.1
Damping factor 0.17
Polly’-s poll based forecast 46.8
Thus, our poll damping discounts a candidate’-s lead in the two-party vote, depending on the days left prior to election. The further away the election day, the larger the damping.
Such damped polls have been shown to outperform sophisticated forecasting approaches like prediction markets. Comparing damped polls to forecasts of the Iowa Electronic Markets, Erikson and Wlezien (2008) showed that the damped polls outperformed both the winner-take-all and the vote-share markets.
Thanks to Andreas Graefe for the link.
–
Are Political Markets Really Superior to Polls as Election Predictors? – PDF file
For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.
Previously: The truth about prediction markets
–
Are Political Markets Really Superior to Polls as Election Predictors? – (PDF file) – by Chris Wlezien and Robert Erikson – 2007
Abstract
Election markets have been praised for their ability to forecast election outcomes, and to forecast better than trial-heat polls. This paper challenges that optimistic assessment of election markets, based on an analysis of Iowa Electronic Market (IEM) data from presidential elections between 1988 and 2004. We argue that it is inappropriate to naively compare market forecasts of an election outcome with exact poll results on the day prices are recorded, that is, market prices reflect forecasts of what will happen on Election Day whereas trial-heat polls register preferences on the day of the poll. We then show that when poll leads are properly discounted, poll-based forecasts outperform vote-share market prices. Moreover, we show that win-projections based on the polls dominate prices from winner-take-all markets. Traders in these markets generally see more uncertainty ahead in the campaign than the polling numbers warrant—in effect, they overestimate the role of election campaigns. Reasons for the performance of the IEM election markets are considered in concluding sections.
Conclusion
This paper has tested the claim that the Iowa Electronic Market offers superior predictions of election outcomes than the snapshots from public opinion polls. By our tests, the IEM election markets are not better than trial-heat polls for predicting elections. In fact, by a reasonable as opposed to naive reading of the polls, the polls dominate the markets as an election forecaster. This is true in the sense that a trader in the market can readily profit by “buying” candidates who, according to informed readings of the polls, are undervalued. Moreover, we find that market prices contain little information of value for forecasting beyond the information already available in the polls. Where then do the markets go wrong? To begin with, consider the vote-share market. The histories of market prices show that traders tend to hold persistent beliefs about the vote division that contradict the polls and that these persistent beliefs are often wrong. Incorrect beliefs get corrected only in the last days before the election, when the polls are difficult to ignore. The winner-take-all market tracks the vote-share market but compounds its errors by overvaluing long-shot candidates’ chances of victory, as if the market expects more campaign surprises than occur in reality. The existence of persistent mistakes in the vote-share market compounded by the degree of uncertainty about the vote-share estimates makes the winner-take-all market a particularly poor forecasting tool. Based on the experience of the IEM, if the polls show a candidate to hold a decisive lead but the market is unconvinced, bet on the polls. It should be noted that our daily poll projections are themselves rather crude instruments. Our robotic trading programs are informed by a flat prior, relying solely on the current polls and the days until the election but nothing more. Even when we compare market prices to the weekly average of poll-based forecasts, our instrument is primitive in that the week’s polls are not weighted for relative recency. But further perfection of our forecasting model from the polls would only advance our central argument. If we were to apply more rigorous modeling to obtain a properly weighted average of current polls and earlier polls, the victory of poll forecasts over the market forecast presumably would be more secure. One could argue that the results are drawn from a limited number of election years from a toy market with thin volume and limits on trader spending. With time, the IEM record could improve, and there is some suggestion that it has. Full-blown markets like Tradesports.com [or InTrade.com or BetFair.com] might in the end achieve an efficiency that so far has eluded the Iowa Electronic Market. Additionally, studies like the present one can suggest improved strategies to traders, which in turn improve the efficiency of election markets. Since our results are confined to a few runs of the toy Iowa market, some might claim a “so what” reaction. To such claimants, an important reminder is that the allegedly uncanny performance of the Iowa market has been touted as the primary evidence for the supposed superiority of election markets over the polls as an information source. The Iowa election market’s performance has not been so special after all. For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.
Niall O’-Connor:
A previous Economist article, that I have archived, spoke of how Napster was revolutionising the music industry. Another one, called Betfair a radical upstart. A recent article on Hulu discussed how it was “online videos new model.” By anybody’-s standards, these technologies have unleashed the forces of disintermediation, and affected a paradigmatic shift in the industries in which they operate.
By way of contrast, the Economist article on Prediction Markets states that Koch, one of the biggest users of Prediction markets, asserted that they are a compliment to other forecasting techniques and not a substitute to them. The article aslo raises the issue of cultural barriers that are inhibiting the take up of said Prediction Markets – not least, inertia (etc..).
One can take from the article that Prediction markets are not ground break, not radical, not revolutionising- they are not unleashing the forces of disintermediation. Accordingly, on the evidence presented (”much remains to be done to convince sceptical managers of their value”) the battle is an uphill one. Moreover, one can ask, if the battle was not won during the good times, what is the real chance that it will be won during a recession, when company’s are always more resistent to change.
You guys are all speaking from a position of being laden down with prediction market baggage. Your views are not objective, and one can only hope that you are not collectively suffering from disaster myopia. […]
Niall O’-Connor’-s website
Panos Ipeirotis depicted my activism to blog about the tone of the news article on enterprise prediction markets published by The Economist —-and its implications. The takeaway that his readers will get is that Chris Masse is on caffeine.
The important information that our good doctor Panos did not tell is that Chris Masse is the publisher of 2 websites (CFM since 2003 and Midas Oracle since 2006) whose main purpose is to list and/or excerpt the news articles, opinion pieces and research papers that focus on prediction markets. Since 2003, I have seen them all —-in all stripes and colors.
So, it is not like I am a gullible newbie just out of the egg. I have a certain expertise in assessing any media piece on prediction markets. And the same thing can be said about Niall O’-Connor (regarding the betting industry, more generally).
There is much lying going on in the field of EPM software vendors.
– They lie about the people they hire —-many of the new employees are in fact part-time (at best).
– They lie about their customers —-some of the names you see on their “-clients”- webpages are in fact companies that have abandoned the experiment long ago. By keeping old customers in their listing and adding some brand-new prospects, they create artificially a cumulative effect so as to impress the gullible prospects that they try to hook up at those pitiful $400-a-seat vendor conferences.
– They lie about the benefits of prediction markets. Since (enterprise) prediction markets are just information aggregation mechanisms that can’-t reach omniscience by essence, the only value of (E)PMs comes from the weaknesses of the competitive forecasting tools. Those weaknesses are not that numerous —-hence, the applications of (E)PMs are probably limited.
– They lie about the successes that their customers got. There isn’-t a single detailed business case published about EPMs.
– They lie about the real age of the prediction markets —-they make it like PMs are in childhood, whereas the reality check is that PMs are in adulthood. The first batch of contemporary PMs popped up in 1988 —-that’-s 21 years ago, folks. The starting point of the PM hype was in 2003–-2004 —-that’-s 6 years ago, now. It is not true to say that (E)PMs are a novelty. By now, we should be able to pause, assess their benefits, and tell the world where exactly they can make an impact (if any).
Because the lying is still going on, I have decided to downgrade the prediction market people and the prediction market organizations who are opaque —-and to upgrade the ones who are open. I hope that my tougher stance will incite everyone to be more truthful.
ADDENDUM
An uncertain future – A novel way of generating forecasts has yet to take off. – by The Economist – 2009-02-26
Who are the prediction market consultants who took part of the conversation prompted by the publication of the devastating story by The Economist?
– Adam Siegel of Inkling Markets
– Mat Fogarty of CrowdCast
– George Tziralis of AskMarkets
– Jed Christiansen of Mercury
Notably absent from the conversation:
– David Perry of Consensus Point
– “-Chief Scientist”- Robin Hanson of Consensus Point
– Emile Servan-Schreiber of NewsFutures
– Chris Hibbert of Zocalo
– The HSX people
– The academic canaries who are over-quoted by the New York Times and the Wall Street Journal
Taking all this into account, I am updating my “-Consultants”- listing published at CFM. I am putting the consultants who participate in web conversations ahead of the others, and in bold, so as to signal to my numerous readers who they should contact first —-should they have any inquiry about enterprise prediction markets. And I will consider doing the same for the other Midas Oracle listings.
Starting today, there will be retaliations of measured and graduated amplitude against any prediction market people or prediction market company who snobs the important conversations about prediction markets —-which take place on Midas Oracle or elsewhere.
You can’-t be bragging everywhere that you are a “-prediction market expert”- and be absent from important conversations. If you don’-t converse with us, then you are not such a good expert —-”-you’-re the weakest link, bye bye.”-
NEXT: It is not about Midas Oracle…- It is about taking part of the conversation about (enterprise) prediction markets on the Web.
Scanning the results for the query on “-prediction markets”-, I see that, focusing on the software vendors and prediction market consultants incorporated after the 2003–-2004 starting point (hence, excluding pioneer NewsFutures), Inkling Markets is ranked much higher than Consensus Point.
In the end, whether the judge is Google or Chris Masse, the passing of time is important. It allows us to see thru prediction market people. There are those who matter —-and those who don’-t.
UPDATE:
Google PageRank:
Inkling Markets: 6 / 10
Consensus Point: 5 / 10