Category Archives: All Best Posts Ever

The fact that Inkling needs five bullet points and a graph to explain short selling is a good indication it’s too complicated.

Posted on by
Reply

No Gravatar

That was Jason Trost&#8217-s comment.

But see, first, Chris Hibbert&#8217-s comment:

My main complaint about using the “short-selling” terminology in prediction markets, is that it uses a term from finance that describes a complicated scenario to describe a simple scenario it doesn’t apply to. In financial markets, short selling means that you accrue money in order to take on a conditional obligation. When you bet against a proposition (on InTrade, Foresight Exchange or (I think) Inkling), you spend money and gain a conditional asset. In the prediction market case, you don’t have any further obligation- there’s no possibility of a margin call. The asset has a non-negative value.

I actually think the way NewsFutures describes binary outcomes is the simplest. They never talk about selling unless you already own the asset. If you don’t own any of the asset, you can either buy it, or click a button to see the opposite view, which you can also buy. They don’t have “yes” and “no”, they just have complementary wordings and titles for opposing outcomes.

Go reading all the comments, there.

Some enterprise prediction markets work very well… -some others are just a waste of time.

Posted on by
2

No Gravatar

Jed Christiansen:

[…] When it comes to the first point, forecasting something that the company already forecasts, prediction markets may or may not be an excellent solution. I’ve seen one set of markets that absolutely blew away the accuracy of current forecasts, and I’ve seen other markets that were consistent with current forecasts with little or no accuracy edge. […]

Care to say more about what is the determinant of an EPM success?

Damped polls outperform prediction markets.

Posted on by
4

No Gravatar

Forecasting Principles:

Damping polls

Evidence from the literature shows that polls, in particular early in the campaign, are not reliable in predicting election outcomes but tend to overestimate the extent to which a candidate leads. To deal with these uncertainties, we added a damping factor to the RCP poll average. Damping is used to make forecasts more conservative in situations involving high uncertainty.

Our poll damping is based on research conducted by Campbell (1996) who showed that polls have to be discounted in order to achieve more reliable forecasts. Performing a regression analysis on historical poll data for the elections from 1948 to 2004, he derived a formula for discounting the polls according to their distance from Election Day. Campbell provided Polly the formula, along with a list of damping factors that vary by the number of days left before the election.

Currently, polls are discounted with a damping factor of ${factor}. Applying this factor, we calculate Polly&#8217-s discounted poll based forecast thus:

Polly&#8217-s poll based forecast = ((Latest RCP polling average – 50) * (1 – [damping factor])) + 50 = ((46.1 – 50) * (1 – 0.17)) + 50 = 46.8

Latest RCP polling average 46.1
Damping factor 0.17
Polly&#8217-s poll based forecast 46.8

Thus, our poll damping discounts a candidate&#8217-s lead in the two-party vote, depending on the days left prior to election. The further away the election day, the larger the damping.

Such damped polls have been shown to outperform sophisticated forecasting approaches like prediction markets. Comparing damped polls to forecasts of the Iowa Electronic Markets, Erikson and Wlezien (2008) showed that the damped polls outperformed both the winner-take-all and the vote-share markets.

Thanks to Andreas Graefe for the link.

–

Are Political Markets Really Superior to Polls as Election Predictors? – PDF file

For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.

Previously: The truth about prediction markets

–

Damped polls are superior to prediction markets as election predictors.

Posted on by
Reply

No Gravatar

Are Political Markets Really Superior to Polls as Election Predictors? – (PDF file) – by Chris Wlezien and Robert Erikson – 2007

Abstract

Election markets have been praised for their ability to forecast election outcomes, and to forecast better than trial-heat polls. This paper challenges that optimistic assessment of election markets, based on an analysis of Iowa Electronic Market (IEM) data from presidential elections between 1988 and 2004. We argue that it is inappropriate to naively compare market forecasts of an election outcome with exact poll results on the day prices are recorded, that is, market prices reflect forecasts of what will happen on Election Day whereas trial-heat polls register preferences on the day of the poll. We then show that when poll leads are properly discounted, poll-based forecasts outperform vote-share market prices. Moreover, we show that win-projections based on the polls dominate prices from winner-take-all markets. Traders in these markets generally see more uncertainty ahead in the campaign than the polling numbers warrant—in effect, they overestimate the role of election campaigns. Reasons for the performance of the IEM election markets are considered in concluding sections.

Conclusion

This paper has tested the claim that the Iowa Electronic Market offers superior predictions of election outcomes than the snapshots from public opinion polls. By our tests, the IEM election markets are not better than trial-heat polls for predicting elections. In fact, by a reasonable as opposed to naive reading of the polls, the polls dominate the markets as an election forecaster. This is true in the sense that a trader in the market can readily profit by “buying” candidates who, according to informed readings of the polls, are undervalued. Moreover, we find that market prices contain little information of value for forecasting beyond the information already available in the polls. Where then do the markets go wrong? To begin with, consider the vote-share market. The histories of market prices show that traders tend to hold persistent beliefs about the vote division that contradict the polls and that these persistent beliefs are often wrong. Incorrect beliefs get corrected only in the last days before the election, when the polls are difficult to ignore. The winner-take-all market tracks the vote-share market but compounds its errors by overvaluing long-shot candidates’ chances of victory, as if the market expects more campaign surprises than occur in reality. The existence of persistent mistakes in the vote-share market compounded by the degree of uncertainty about the vote-share estimates makes the winner-take-all market a particularly poor forecasting tool. Based on the experience of the IEM, if the polls show a candidate to hold a decisive lead but the market is unconvinced, bet on the polls. It should be noted that our daily poll projections are themselves rather crude instruments. Our robotic trading programs are informed by a flat prior, relying solely on the current polls and the days until the election but nothing more. Even when we compare market prices to the weekly average of poll-based forecasts, our instrument is primitive in that the week’s polls are not weighted for relative recency. But further perfection of our forecasting model from the polls would only advance our central argument. If we were to apply more rigorous modeling to obtain a properly weighted average of current polls and earlier polls, the victory of poll forecasts over the market forecast presumably would be more secure. One could argue that the results are drawn from a limited number of election years from a toy market with thin volume and limits on trader spending. With time, the IEM record could improve, and there is some suggestion that it has. Full-blown markets like Tradesports.com [or InTrade.com or BetFair.com] might in the end achieve an efficiency that so far has eluded the Iowa Electronic Market. Additionally, studies like the present one can suggest improved strategies to traders, which in turn improve the efficiency of election markets. Since our results are confined to a few runs of the toy Iowa market, some might claim a “so what” reaction. To such claimants, an important reminder is that the allegedly uncanny performance of the Iowa market has been touted as the primary evidence for the supposed superiority of election markets over the polls as an information source. The Iowa election market’s performance has not been so special after all. For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.

Mechanical Turk grades The Economists news article on enterprise prediction markets

Posted on by
7

No Gravatar

Panos Ipeirotis:

economist-survey

Well, the average was a 5.8/10, meaning that the average detected sentiment was pretty much neutral with some hints on positivity.

I acknowledge this result, brought to us by research scientist and university professor Panos Ipeirotis&#8230- who, 5 minutes ago, was alerting us on the hard fact that Mechanical Turk is not so much of a reliable tool&#8230-

Previously: Enterprise prediction markets… the next big thing —not.

Enterprise prediction markets… the next big thing -not.

Posted on by
8

No Gravatar

Niall O&#8217-Connor:

A previous Economist article, that I have archived, spoke of how Napster was revolutionising the music industry. Another one, called Betfair a radical upstart. A recent article on Hulu discussed how it was “online videos new model.” By anybody&#8217-s standards, these technologies have unleashed the forces of disintermediation, and affected a paradigmatic shift in the industries in which they operate.

By way of contrast, the Economist article on Prediction Markets states that Koch, one of the biggest users of Prediction markets, asserted that they are a compliment to other forecasting techniques and not a substitute to them. The article aslo raises the issue of cultural barriers that are inhibiting the take up of said Prediction Markets – not least, inertia (etc..).

One can take from the article that Prediction markets are not ground break, not radical, not revolutionising- they are not unleashing the forces of disintermediation. Accordingly, on the evidence presented (”much remains to be done to convince sceptical managers of their value”) the battle is an uphill one. Moreover, one can ask, if the battle was not won during the good times, what is the real chance that it will be won during a recession, when company’s are always more resistent to change.

You guys are all speaking from a position of being laden down with prediction market baggage. Your views are not objective, and one can only hope that you are not collectively suffering from disaster myopia. […]

Niall O&#8217-Connor&#8217-s website

Opacity versus Openness

Posted on by
1

No Gravatar

There is much lying going on in the field of EPM software vendors.

– They lie about the people they hire &#8212-many of the new employees are in fact part-time (at best).

– They lie about their customers &#8212-some of the names you see on their &#8220-clients&#8221- webpages are in fact companies that have abandoned the experiment long ago. By keeping old customers in their listing and adding some brand-new prospects, they create artificially a cumulative effect so as to impress the gullible prospects that they try to hook up at those pitiful $400-a-seat vendor conferences.

– They lie about the benefits of prediction markets. Since (enterprise) prediction markets are just information aggregation mechanisms that can&#8217-t reach omniscience by essence, the only value of (E)PMs comes from the weaknesses of the competitive forecasting tools. Those weaknesses are not that numerous &#8212-hence, the applications of (E)PMs are probably limited.

– They lie about the successes that their customers got. There isn&#8217-t a single detailed business case published about EPMs.

– They lie about the real age of the prediction markets &#8212-they make it like PMs are in childhood, whereas the reality check is that PMs are in adulthood. The first batch of contemporary PMs popped up in 1988 &#8212-that&#8217-s 21 years ago, folks. The starting point of the PM hype was in 2003&#8211-2004 &#8212-that&#8217-s 6 years ago, now. It is not true to say that (E)PMs are a novelty. By now, we should be able to pause, assess their benefits, and tell the world where exactly they can make an impact (if any).

Because the lying is still going on, I have decided to downgrade the prediction market people and the prediction market organizations who are opaque &#8212-and to upgrade the ones who are open. I hope that my tougher stance will incite everyone to be more truthful.

ADDENDUM

An uncertain future – A novel way of generating forecasts has yet to take off. – by The Economist – 2009-02-26

Enterprise prediction markets: Usability innovation is the answer.

Posted on by
4

No Gravatar

This past week, The Economist wrote on the yet-unfulfilled promise of prediction markets. At CrowdCast (ex-Xpree), we believe prediction markets are not yet mainstream because the current solutions are built on mechanisms designed for the stock market, not for the enterprise.

The stock trading metaphor works for a large, liquid stock market, but is unsuitable for enterprise forecasting. The concept of shorting and covered calls is far from intuitive for your average employee, and the stock mechanism makes it hard to ask the simplest of questions relevant for corporate forecasters. For example, buying or selling a collection of virtual stocks representing probabilities of sales falling in particular ranges is an incredibly obtuse way of asking for a single sales forecast. Finally, the stock mechanism relies on copious liquidity to ensure meaningful metrics, which is often not available with the limited crowds available in the enterprise.

However, innovation moves on and we question the assumption that prediction markets have to rely on the stock market analogy. At CrowdCast, we have been working on a new mechanism, that takes into account participant behavior and aptitude as much as market efficiency. The product we are launching in April will deliver easy, engaging, and expressive information exchanges, without the limitations of traditional notions of stock markets.

When you get the questions, incentives, and mechanism right, a prediction market can be an incredibly powerful management tool. Employees share insights anonymously and are measured and rewarded for their intelligence. Widely deployed, this has the potential to fundamentally change the nature of the organizational contract, moving from information flow based on hierarchy and silos, to enterprise-wide direct communication.

A whole new take on prediction markets- available from CrowdCast in April 2009.

Mat Fogarty

CrowdCast CEO

Cross-posted from the Xpree blog

Previously: Are collective intelligence solutions being oversold?

Inkling Markets CEO Adam Siegel speaks out on the current state of enterprise prediction markets.

Posted on by
1

No Gravatar

Adam Siegel:

Niall,

You are right to question this stuff. There is a lot of bullshit out there and frankly I cringe when I see articles or statements about the “accuracy” of markets because it hurts everyone in the long run. It’s why we’ve written 3 or 4 times on our blog about what “accuracy” in a marketplace really means – that you can’t just look at 5 or 10 markets and say “we nailed it.” Unfortunately it’s something we have to deal with because the application is called a “prediction” market after all so it’s the first question that naturally comes to mind. That said I wish Chris wouldn’t make blanket statements [*] about “the vendors, e.g. Inkling Markets” getting the use cases all wrong in enterprise prediction markets. Because frankly, for quite some time “accuracy” has been a secondary argument to a number of other advantages we discuss about markets (in fact on the page on our website that describes the value proposition to companies, we don’t even list “accuracy” as a key benefit). This is also an expectation we set with our clients right from the beginning.

Anyways, I made the statement about business increasing year over year based on our own experience/numbers and as Jed mentioned, by looking at the activity of some of our competitors who have made hires, added clients, etc. I also base the statement on the types of professional services companies we are working with/hearing from and our discussions about prediction markets and what they are going to try and do in the future. I don’t think so many would be interested in adding markets to their toolset and expending resources putting together offerings if they didn’t see them as a long term, worthwhile business capability. So for those that agree with Chris that 6 years is enough time to evaluate a new business capability, I’d like to politely disagree. I could be wrong but these things haven’t been used beyond the experimental/pilot stage in companies for more than 2 or 3 years. We’re just at the cusp of understanding what benefits they’re going to bring. We’ve seen some promising trends, we’ve also seen people try to use Inkling for something and it failed miserably. This is just standard lifecycle stuff, especially for a capability that is designed, as I said in my blog post, to bring about more transparency and break down organization barriers.

[*] Adam, it was not a &#8220-statement&#8221-, it was a 2-choice hypothesis.

ADDENDUM

An uncertain future – A novel way of generating forecasts has yet to take off. – by The Economist – 2009-02-26