Category Archives: All Best Posts Ever

How vendors are scuttling the field of enterprise prediction markets -and the prediction market industry, as a whole

Posted on by
Reply

No Gravatar

The danger of vendor conferences without any editorial line: It backfires against the whole prediction markets industry &#8212-big time.

sawing

I warned my readers many times against the vendor conferences organized by the San Francisco man. He is so desperate that he invites anybody who will pronounce the word &#8220-prediction&#8221- and &#8220-markets&#8221- in the same paragraph. Many of the invited speakers haven&#8217-t the slightest knowledge of the field of prediction markets. As for the vendors, they are incapable of producing one single case study featuring a success in the use of enterprise prediction markets. Not a single one. (And I won&#8217-t mention the &#8220-flow of information&#8221- &#8212-the worst research ever published on prediction markets.) Their vendor websites publish lists of clients, which, at first glance, look impressive, but many of those so-called customers are in fact ancient clients who have ended pilot programs years ago. To add insult to injury, this fake conference is sold $400 to gullible attendees. It is not even worth 4 cents.

The Economist reporter who attended the San Francisco conference realized what I [*] realized long ago: The field of enterprise prediction markets is all smokes and mirrors. The more the prediction market vendors will participate in such crappy conferences, the more the media will realize that the prediction market vendors are all hat and no cattle, and the more they will publish news stories bursting the prediction market bubble. And in the end of 2009, we will end up with 10 news articles in major media telling the world that prediction markets were a fad. Live by the hype- die by the hype.

The only way to get out of this debacle is to come back to basics: Do the research right, do discover the real value of enterprise prediction markets (velocity), and, then, only when you have something to show for it, go out in postings and conferences.

–

[*] I follow the field of prediction markets since 2003. I saw it in all shapes and stripes. You can fool your mother, but you can&#8217-t fool me.

–

APPENDIX:

An uncertain future – A novel way of generating forecasts has yet to take off. – by The Economist – 2009-02-26

– But although they have spread beyond early-adopting companies in the technology industry, they have still not become mainstream management tools. Even fervent advocates admit much remains to be done to convince sceptical managers of their value.

– Koch says the results so far have been pretty accurate compared to actual outcomes, but stresses that markets are complementary to other forecasting techniques, not a substitute for them.

– A big hurdle facing managers using prediction markets is getting enough people to keep trading after the novelty has worn off.

– Another reason prediction markets flop is that employees cannot see how the results are used, so they lose interest.

– Bosses may also be wary of relying on the judgments of non-experts.

–

Velocity + Inaccuracy

Posted on by
9

No Gravatar

One bit of criticism about my pamphlet (The Truth About Prediction Markets) goes like this: Velocity without accuracy is dumb.

That is not true.

Let&#8217-s imagine, for the sake of the exercise, that Barack Obama does not pick up Kathleen Sebelius to head HHS. The velocity argument remains valid: Fed by the vertical media (in this case, Yahoo News republishing the Associated Press), the prediction markets integrated expectations (informed by facts and expertise) much faster than the mass media did.

Any argument about the velocity of the prediction markets cannot be contradicted. No way.

The HHS-Sebelius prediction market might be (yet) another case-in-point for documenting velocity.

Posted on by
2

No Gravatar

It is only today (February 19) that the New York Times emerges out of hibernation and headlines:

– Kansas Governor Seen as Top Choice in Health Post. &#8212- Gov. Kathleen Sebelius is emerging as President Obama’s top choice for secretary of health and human services.

Now, look at the red line in the HubDub chart below: the prediction markets nailed her since the beginning of February 2009.

Of course, a scientific comparison would have scrutinized more closely than I did all the news articles from the New York Times (and from other mass media). That&#8217-s what we are going to do with the &#8220-Open Institute Of Prediction Markets&#8220-. To this end, I will set up a portable and distributed &#8220-Prediction Markets Consortium&#8221- in the coming days. Then, I will try to anchor it in an institution of higher education, and, after that, I will try to gather support from think tanks and foundations. Not an easy task, but I know now that I can count on many prediction market people and companies. It should be an industry endeavor &#8212-and it should deliver results, in the end (demonstrating the social utility of the prediction markets by documenting velocity, and, from there, following a logical thread which I will talk you about later on).

PS: About velocity&#8230- Remember that we are about the prediction markets versus the mass media (The New York Times, The Times of London, NBC News, BBC News, etc.) &#8212-as opposed to the vertical media (Politico.com, Nate Silver&#8217-s blog, PoliticalBetting.com, etc.). The distinction is very important to keep in mind.

UPDATE: The only stuff I can find about Sebelius for HHS is that February 9 piece from the Associated Press (which didn&#8217-t get a mass audience since it was not-republished in the New York Times or other mass media), saying that she was &#8220-near the top&#8221- for the job. Well, &#8220-near the top&#8221- is not like saying she was &#8220-on top&#8221-.

UPDATE #2: The Sebelius story is picking steam in the mass media. See Nate Silver&#8217-s take.

ADDENDUM: Andrew Gelman tells me that he thinks that &#8220-the Associated Press is a mass medium. It is a cooperative organized by a bunch of newspapers.&#8221- I think that the AP news articles do indeed reach a big audience when they are re-published or cited in the mass media. But in the Sebelius case above, it was not the case.

–

Previously: The truth about prediction markets

–

Who will be the next nominee of the HHS, now that Daschle has withdrawn from consideration?

–

No HHS contract on InTrade, BetFair or NewsFutures. :(

Prediction markets didnt revolutionize decision-making -and will never do. However, they are a nice condiment to the classic forecasting toolkit.

Posted on by
Reply

No Gravatar

I have spent several hours re-reading the 2004 AEI-Brookings book, &#8220-Information Markets&#8221- (by which they mean &#8220-prediction markets&#8221-). It is a collection of un-enlightening research articles &#8212-except for the IEM article, which is outstanding, both on the factual and theoretical sides.

In the conclusion of their introduction, Robert Hahn and Paul Tetlock wrote that they want their readers to contemplate the idea that prediction markets could make a &#8220-big&#8221- difference and &#8220-revolutionize public- and private-sector decision-making&#8221-. Well, 4 years later, it is clear that those big dreams didn&#8217-t pan out. Not a single mass media outlet has praised the public prediction markets for their work on the 2008 US presidential election (I am taking about a post-mortem analysis about Election Day, not the primaries). Not a single one. (Not even Justin Wolfers.) And the number of corporations using enterprise prediction markets is still minute. The thinkers who wrote this book (&#8220-Information Markets&#8221-) all made the mistake to put the emphasis on accuracy instead of efficiency. That was the foundation flaw. We should reset and reboot the field of prediction markets.

Previously: The truth about prediction markets

Velocity is such a potent argument. Why dont we use it more, for Christs sake?

Posted on by
Reply

No Gravatar

I am re-reading a 2007 scientific article from Region Focus’ Vanessa Sumo:

– Ask The Market – Companies are leading the way in the use of prediction markets. The public sector may soon follow. – (PDF)

Here is what I see on the frontpage:

– &#8220-one or two weeks in advance&#8220-

– &#8220-even up to five weeks in advance&#8220-

Marketing-wise, velocity is a much more potent argument than the argument on accuracy. Who cares about an added accuracy of +2.7% (and that&#8217-s debated)? If any, that&#8217-s peanuts.

You cannot make a case against velocity. Impossible.

–

UPDATE: Put the PDF link in the address box of your browser (as opposed to clicking on it, or right-clicking on it).

http://www.richmondfed.org/publications/research/region_focus/2007/spring/pdf/feature1.pdf

The truth about prediction markets

Posted on by
Reply

No Gravatar

Come to the wonderful world of collective intelligence, wisdom of crowds, and prediction markets!&#8230- The sun shines bright, the market-generated predictions are vastly superior to the polls as election predictors, and the track record of the public prediction markets stretches as far as the eye can see. There are opportunities aplenty in the field of prediction markets, and the trading technology is cheap. Every working enterprise can have its own internal prediction exchange, and inside every exchange, a set of enterprise prediction markets that correctly predicts the future of business, which their happy, all-American CEO listens to. Life is good in the magic world of prediction markets&#8230- it&#8217-s paradise on Earth.

Ha! ha! ha! ha!&#8230- That&#8217-s what they tell you, anyway&#8230- &#8212-because they are selling an image (just as Bernie Madoff did). They are selling it thru their vendor websites, vendor conferences, vendor-inspired articles in blogs, newspapers and magazines, and interviews of vendor data-fed professors in the media.

The prediction market technology is not a disruptive technology, and the social utility of the prediction markets is marginal. Number one, the aggregated information has value only for the totally uninformed people (a group that comprises those who overly obsess with prediction markets and have a narrow cultural universe). Number two, the added accuracy (if any) is minute, and, anyway, doesn&#8217-t fill up the gap between expectations and omniscience (which is how people judge forecasters). In our view, the social utility of the prediction markets lays in efficiency, not in accuracy. In complicated situations, the prediction markets integrate expectations (informed by facts and expertise) much faster than the mass media do. Their accuracy/efficiency is their uniqueness. It is their velocity that we should put to work.

–

Here&#8217-s now our definition of prediction markets:

–

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain future outcome (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism &#8212-with or without an automated market maker.

Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future &#8212-anticipations that will be either confirmed or infirmed.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other meta predictive mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other meta predictive mechanisms. A highly accurate set of prediction markets has little value if some other meta predictive mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate predictions on its topic.

–

PS: I am updating a bit the content of this webpage, over time &#8212-so as to finesse the message.

Dealing with public perception and general anti-market sentiment

Posted on by
5

No Gravatar

I posted the following to the Cantor Exchange forum a couple of weeks ago. That same weekend, this piece by Zach Karabell appeared. We make some of the same points that are relevant in a generally hostile environment towards derivatives and markets.

&#8212-

Rich Jaycobs&#8217- expertise and realism on issues such as insider trading and manipulation are invaluable to the Cantor Exchange project, especially given the backdrop of the failure of the financial system. A letter from Max Keiser to the FT and related comments underline the challenge of knee-jerk public reaction to innovative contracts.

This is a typical reaction: &#8220-I can&#8217-t believe this. The financial mess we&#8217-re in right now is, in a very large way, due to this kind of crap &#8230- it&#8217-s simply gambling.&#8221- These sorts of claims need to be dealt with.

First, the contracts that are being proposed are traded on exchanges. As many, including myself and the CFTC have argued, lack of transparency in pricing was one of the main culprits of the financial meltdown. The surest way to deliver a shock, a high standard deviation move, to markets is to just not mark or otherwise mis-mark prices for a while. Without active trading, risk build-ups. Explosion and collapse follows.

Leverage also played a significant role in the crises. After all, without leverage, the bogeyman of derivatives is largely defused. Of course no CFTC-regulated contract, most of which allow for substantial leverage, has yet defaulted.

Nor would the proposed contracts suffer from the specific agency problems that infected credit markets and investment houses, so I&#8217-m not sure what &#8220-kind of crap&#8221- the commenter had in mind precisely. It is meaningless that the box-office contracts happen to be &#8220-derivatives&#8221-.

Max Keiser does propose a specific problem. What if a studio blows-up in the box-office market, forcing it into bankruptcy? This line of thinking quickly becomes absurd. If society were strictly bound to &#8220-do no harm&#8221-, nothing would ever get done. Even doctors do harm in the form of side-effects. They evaluate courses of action in terms of the expected net result and so should we in these cases.

Over time, the net benefit of well-regulated markets will be positive, but realism is needed to stand up to these essentially prudent concerns. It does seem to be the case, for example, that commodity futures exhibit structural influences on prices that are independent of usage-based supply and demand, and that may increase volatility. Whether that is more attributable to the existence of the contract or ultimately fiat money is debatable, but in any case, this should be much less of an issue in markets like the box-office contracts, which are settled objectively in a relatively short period of time. In contrast, the exact &#8220-meaning&#8221- of a perpetuity or commodity future is not clear.

We can imagine self-fulfilling prophesies and other possible side-effects, and of course there are some issues we aren&#8217-t thinking of, but the supporters of innovative contracts have to be on top of the foreseeable pathologies and engage critics in terms of specifics. Generic anti-market, anti-derivative carping is not an argument.

And remember that the eve of the French Revolution, no-one would have predicted Emperor Napoleon.

&#8212-

Yes, Napoleon later &#8220-blew up&#8221-!

Obstacles to Prediction Market Adoption

Posted on by
Reply

No Gravatar

BusinessWeek:

Harrah&#8217-s is setting up a pilot prediction market to forecast customer activity in one of its domestic casino operations. […]

Since the power of prediction markets hinges on effectively tapping into cognitive diversity throughout an organization, Page also argues convincingly that if members of a group do not have enough diversity in their perspectives, prediction markets can actually produce dismal results. […]

Until now, few of the companies sponsoring successful pilots or tests have deployed prediction markets on a broad or sustained basis. Why not? One explanation is that prediction markets are deeply subversive. After all, lots of midlevel executives are consumed with the task of forecasting. If prediction markets do a better job of it, doesn&#8217-t that discredit the efforts (and perhaps even the motives) of these executives? But as prediction markets shift their focus toward new knowledge creation, they may become less threatening within corporations. […]

–

I don&#8217-t buy this explanation &#8212-nor do I buy that other one.

My view is that we haven&#8217-t yet demonstrated clearly when and how prediction markets can be useful.

–

Prediction markets compute facts and expertise quicker that the mass media do.

Posted on by
1

No Gravatar

Political prediction markets react (with a small delay) to political polls &#8212-just like the political experts and the mass media do, too. Hence, in order to discover their true social utility, the prediction markets (which are tools of intelligence) should not be compared to the polls (which are just facts) but to the similar meta intelligence mechanisms (the averaged probabilistic predictions from a large panel of experts, or the averaged probabilistic predictions from the political reporters in the mass media, or else). My bet is that, in complicated situations (such as the 2008 Democratic primary), the prediction markets beat the mass media (in terms of velocity) &#8212-even though the prediction markets are not omniscient and not completely objective (but who is?).

–

You might remember the research article that I have blogged about:

–

Learning in Investment Decisions: Evidence from Prediction Markets and Polls – (PDF file) – David S. Lee and Enrico Moretti – 2008-12-XX

In this paper, we explore how polls and prediction markets interact in the context of the 2008 U.S. Presidential election. We begin by presenting some evidence on the relative predictive power of polls and prediction markers. If almost all of the information that is relevant for predicting electoral outcomes is not captured in polling, then there is little reason to believe that prediction market prices should co-move with contemporaneous polling. If, at the other extreme, there is no useful information beyond what is already summarized by the current polls, then market prices should react to new polling information in a particular way. Using both a random walk and a simple autoregressive model, we find that the latter view appears more consistent with the data. Rather than anticipating significant changes in voter sentiment, the market price appears to be reacting to the release of the polling information.

We then outline and test a more formal model of investor learning. In the model, investors have a prior on the probability of victory of each candidate, and in each period they update this probability after receiving a noisy signal in the form of a poll. This Bayesian model indicates that the market price should be a function of the prior and each of the available signals, with weights reflecting their relative precision. It also indicates that more precise polls (i.e. polls with larger sample size) and earlier polls should have more effect on market prices, everything else constant. The empirical evidence is generally, although not completely, supportive of the predictions of the Bayesian model.

–

polls-prediction-markets

–

You might also have watched Emile Servan-Schreiber&#8217-s videos. Emile is a smart man, and those videos are truly instructive.

  1. In the first part (the lecture), our good doctor Emile Servan-Schreiber sold the usual log lines about the prediction markets &#8212-blah blah blah blah blah.
  2. In the second part, Emile Servan-Schreiber took questions from the audience in the room. &#8220-Aren&#8217-t political prediction markets just following the polls?&#8221-, asked one guy. Emile&#8217-s answer was long and confused. However, in my view, Emile actually did answer that question (before it was ever asked) in his preceding lecture when, at one point, he made the point that the media were slower than the prediction markets to integrate all the facts about the 2008 Democratic primary, around May 2008. That is the right answer to give to a conference attendee who enquires about prediction markets &#8220-following&#8221- the polls. Both the mass media and the prediction markets do follow the polls (since the polls are facts that can&#8217-t be ignored), during political campaigns. Let&#8217-s compare the prediction markets with the mass media, instead, and let&#8217-s see who&#8217-s quicker to deliver the right intelligence..

–

Lance Fortnow gives a good insight about the relationship between polls and prediction markets (see his last paragraph).

Yesterday the Electoral College delegates voted, 365 for Barack Obama and 173 for John McCain. How did the markets do?

To compare, here is my map the night before the election and the final results. The leaning category had Obama at 364. The markets leaned the wrong way for Missouri and Indiana, their 11 electoral votes canceling each other out. The extra vote for Obama came from a quirk in Nebraska that the Intrade markets didn&#8217-t cover: Nebraska splits their votes based on congressional delegations, one of which went to Obama.

Indiana and Missouri were the most likely Republican and Democratic states to switch sides according to the markets, which mean the markets did very well this year again. Had every state leaned the right way (again), one would wonder if the probabilities in each state had any meaning beyond being above or below 50%.

Many argue the markets just followed the predictions based on polls like Nate Silver&#8217-s fivethirtyeight.com. True to a point, Silver did amazingly well and the markets smartly trusted him. But the markets also did very well in 2004 without Silver. [Chris Masse’s remark: In 2004, Electoral-Vote.com (another poll aggregator) was all the rage.] One can aggregate polls and other information using hours upon hours of analysis or one can just trust the markets to get essentially equally good results with little effort.

–

The polls are facts. Prediction markets are meta to facts. Prediction markets are intelligence tools. Let&#8217-s compare them with similar intelligence tools.

–

Lance Fortnow&#8217-s post attracted an interesting comment from one of his readers:

to provide an exciting collection of political and other prediction markets.

These markets are as much a &#8220-prediction&#8221- tool as a wind vane or outdoor thermometer are. They moved up and down according to the daily trends, with very little insight of the longer place phenomena underlying them.

When the weather was hot (Palin&#8217-s nomination announcement) the market swinged widely towards McCain, while ignoring the cold front on the way here (the economic recession + Palin inexperience).

The value of weather forecast is in telling us things we didn&#8217-t know. We don&#8217-t need to trade securities to believe that if McCain is closing on the polls then his chances of wining are higher (duh!), which is what the markets did. We need sophisticated prediction mechanisms to tell us how the worsening economic conditions, the war in Iraq and Palin ineptitude (which in pre-Couric days wasn&#8217-t as well established) will impact this election, today poll&#8217-s be damned.

Looking at the actions by the republican teams, who were trying to read past the daily trend all the way to November 4th, it is clear that they thought all along they were losing by a fair margin. Because of this is they choose moderate, maverick McCain, went for the Palin hail mary fumble^H^H^H^H^H pass and the put-the-campaign-on-hold move.

A full two weeks before the election the McCain team concluded the election was unwinnable, while the electoral college market was still giving 25-35% odds to McCain.

–

As highlighted in bold, the commenter says two things:

  1. The prediction markets are just following the polls.
  2. The prediction markets have a minimal societal value.

–

My replies to his/her points:

  1. That&#8217-s not the whole truth. The polls are just a set of facts, whereas the prediction markets are intelligence tools that aggregate both facts and expertise. The commenter picks up a simple situation (the 2008 US presidential election) where, indeed, anybody reading the latest polls (highly favorable to Barack Obama) could figure out by himself/herself what the outcome would be (provided the polls wouldn&#8217-t screw it).
  2. That&#8217-s true in simple situations, but that&#8217-s wrong in complicated situations (such as the 2008 Democratic primary).

–

The emergence of the social utility of the prediction markets will come more clearly to people once we:

  1. Highlight the complicated situations-
  2. Code the mass media&#8217-s analysis of those complicated situations, and compare that with the prediction markets.

–

–

APPENDIX:

–

–

–

–

No TweetBacks yet. (Be the first to Tweet this post)