Paul Krugman Makes a Boo Boo.

No Gravatar

In Paul Krugman&#8217-s blog entry, Done, at 4:39pm (EDT) on March 21, 2010, he commented: &#8220-OK, nothing is sure in this world. Intrade is still giving Obamacare a 2.2% chance of failing, …&#8221-

He was talking about the InTrade market on Health Care Reform. In theory, the market price in such a derivative market should equal the expectation of the underlying event coming true. However, Paul Krugman (and many others) forgot one of the most basic assumptions of the market model! Transaction costs.

When the market price is over 95, InTrade charges a transaction fee of 3 cents per contract (real money). While market prices are quoted in percentages, the payoff for a winning ticket is $10 (real money). Therefore, the transaction fee is 0.3% of the winning payoff. In addition, InTrade charges 10 cents per contract on expiry (if you &#8220-win&#8221-). That&#8217-s another 1.0%.

So, when the market was quoting 97.8% likelihood of the HCR bill passing before June 2010, this didn&#8217-t really mean that there was a 2.2% chance of the bill not passing. A winning ticket would be subject to 1.3% transaction fees. The real likelihood of failure was 0.9% &#8211- approximating the uncertainty that Obama would be &#8220-hit by a bus&#8221- before signing the bill into law.

No rational investor would wish to purchase a share for more than 98.7, given the transaction costs. In a sense, this is the market&#8217-s &#8220-100%&#8221-. Interestingly, at 1:49pm GMT today (March 23), there are 695 bids at 99.1 and 413 asks at 99.2. Clearly, some traders are not subject to the full transaction fees at InTrade. More about that here.

[Cross-posted from Toronto Prediction Market Blog.]

What has been the best InTrade prediction market ever? Has the ObamaCare prediction market at InTrade been ahead of the commentary?

Jason Ruspini (who feels that the health care reform proposal might well be adopted) wanna feedback from you, folks.

  1. Which InTrade prediction market(s) has/have been ahead of the Press (rather than the other way around)? What is/are the best (most divergent from the commentary, and correct) InTrade prediction market(s) in people&#8217-s memories?
  2. Do you sense that the ObamaCare prediction market at InTrade fits these 2 criteria?

UPDATE: I asked The Brain whether he meant generalist media or political media, and he meant &#8220-generalist&#8221-. That makes all the difference in the world.

Prediction Market Chart

ADDENDUM

More info on health care reform on Memeorandum.

Previously: Insider trading in the InTrade prediction market on health care reform?

Truth in Advertising – Meet Prediction Markets

Most published papers on prediction markets (there aren&#8217-t many) paint a wildly rosy picture of their accuracy. Perhaps it is because many of these papers are written by researchers having affiliations with prediction market vendors.

Robin Hanson is Chief Scientist at Consensus Point. I like his ideas about combinatorial markets and market scoring rules, but I think he over-sells the accuracy and usefulness of prediction markets. His concept of Futarchy is an extreme example of this. Robin loves to cite HP&#8217-s prediction markets in his presentations. Emile Servan-Schreiber (Newsfutures) is mostly level-headed but still a big fan of prediction markets. Crowdcast&#8217-s Chief Scientist is Leslie Fine- their Board of Advisors includes Justin Wolfers and Andrew McAfee. Leslie seems to have a more practical understanding than most, as evidenced by this response to the types of questions that Crowdcast&#8217-s prediction markets can answer well: &#8220-Questions whose outcomes will be knowable in three months to a year and where there is very dispersed knowledge in your organization tend to do well.&#8221- She gets it that prediction markets aren&#8217-t all things to all people.

An Honest Paper

To some extent, all of the researchers over-sell the accuracy and the range of useful questions that may be answered by prediction markets. So, it is refreshing to find an honest article written about the accuracy of prediction markets. Not too long ago, Sharad Goel, Daniel M. Reeves, Duncan J. Watts, David M. Pennock published Prediction Without Markets. They compared prediction markets with alternative forecasting methods for three types of public prediction markets: Football and baseball games and movie box office receipts.

They found that prediction markets were just slightly more accurate than alternative methods of forecasting. As an added bonus, these researchers considered the issue that prediction market accuracy should be judged by its effect on decision-making. So few researchers have done this! A very small improvement in accuracy is not considered material (significant), if it doesn&#8217-t change the decision that is made with the forecast. It&#8217-s a well-established concept in public auditing, when deciding whether an error is significant and requires correction. I have discussed this concept before.

While they acknowledge that prediction markets may have a distinct advantage over other forecasting methods, in that they can be updated much more quickly and at little additional cost, they rightly suggest that most business applications have little need for instantaneously updated forecasts. Overall, they conclude that &#8220-simple methods of aggregating individual forecasts often work reasonably well relative to more complex combinations (of methods).&#8221-

For Extra Credit

When we compare things, it is usually so that we can select the best option. In the case of prediction markets it is not a safe assumption that the choices are mutually exclusive. Especially in enterprise applications, prediction markets are heavily dependent on the alternative information aggregation methods as a primary source of market information. Of course, there are other sources of information and the markets are expected to minimize bias to generate more accurate predictions.

In the infamous HP prediction markets, the forecasts were eerily close to the company&#8217-s internal forecasts. It wasn&#8217-t difficult to see why. The same people were involved with both predictions! The General Mills prediction markets showed similar correlations, even when only some of the participants were common to both methods. The implication of these cases is that you cannot replace the existing forecasting system with a prediction market and expect the results to be as accurate. The two (or more) methods work together.

Not only do most researchers (Pennock et al, excepted) recommend adoption of prediction markets, based on insignificant improvements in accuracy, they fail to consider the effect (or lack thereof) on decision-making in their cost/benefit analysis. Even if some do the cost/benefit math, they don&#8217-t do it right.

Where a prediction market is dependent on other forecasting methods, the marginal cost is the total cost of running the market. There is no credit for eliminating the cost of alternative forecasting methods. The marginal benefit is that expected by choosing a different course of action than the one that would have been taken based on a less accurate prediction. That is, a slight improvement in prediction accuracy that does not change the course of action has no marginal benefit.

Using this approach, a prediction market that is only &#8220-slightly&#8221- more accurate, than those from alternative forecasting approaches, is just not good enough. So far, there is little, if any, evidence that prediction markets are anything more than &#8220-slightly&#8221- better than existing methods. Still, most of our respected researchers continue to tout prediction markets. Even a technology guru like Andrew McAfee doesn&#8217-t get it , in this little PR piece he wrote, shortly after joining Crowdcast&#8217-s Board of Advisors.

Is it a big snow job or just wishful thinking?

[Cross-posted from Toronto Prediction Market Blog]

OSCARS 2010: Did Justin Wolfers brag too much and too loudly? – [RELATIVE ACCURACY DEPARTMENT]

No Gravatar

Jason (a Freakonomics reader):

You are giving yourself WAY too much credit. Siskel and Ebert successfully predict these awards 100% year after year. This isn’t a difficult thing to predict. Predicting something like the NCAA tourney, that would be an accomplishment, but if you look at rankings and your prediction market, you will fail just as much as the average bracket.
— Jason

Nobel Prize for Economics 2009 – Prediction Accuracy

No Gravatar

The Royal Swedish Academy of Sciences has decided to award The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel for 2009 to Elinor Ostrom &#8220-for her analysis of economic governance, especially the commons&#8221- and Oliver E. Williamson &#8220-for his analysis of economic governance, especially the boundaries of the firm&#8221-.

Both the bookmakers and the prection markets are utterly useless in trying to divine who will get the Nobel prize of economics.

Below is the 2009 prediction post-mortem:

1. Bookmakers

Ladbrokes&#8217-s probabilities (odds) for the 2009 Nobel prize in economics:

Eugene Fama 2/1
Paul Romer 4/1
Ernst Fehr 6/1
Kenneth R. French 6/1
William Nordhaus 6/1
Robert Barro 7/1
Matthew J Rabin 8/1
Jean Tirole 9/1
Martin Weitzman 9/1
Chris Pissarides 10/1
Dale T Mortensen 10/1
Xavier Sala-i-Martin 10/1
Avinash Dixit 14/1
Jagdish N. Bhagwati 14/1
Robert Schiller [sic] 14/1
William Baumol 16/1
Martin S. Feldstein 20/1
Christopher Sims 25/1
Lars P. Hansen 25/1
Nancy Stokey 25/1
Peter A Diamond 25/1
Thomas J. Sargent 25/1
Dale Jorgenson 33/1
Paul Milgrom 33/1
Oliver Hart 40/1
Bengt R Holmstrom 50/1
Elhanan Helpman 50/1
Ellinor Ostrom 50/1
Gene M Grossman 50/1
Karl-Goran Maler 50/1
Oliver Williamson 50/1
Robert B Wilson 50/1

2. Betting Pools

Here is the betting in the Nobel pool at Harvard:

Robert Barro -10%
John Taylor &#8211- 8%
Paul Milgrom &#8211- 8%
Jean Tirole &#8211- 6%
Oliver Williamson &#8211- 6%
Martin Weitzman &#8211- 6%
Eugene Fama &#8211- 5%
Richard Thaler &#8211- 5%
Lars Hansen &#8211- 4%
Paul Romer &#8211- 4%

3. Prediction Markets

InTrade:

nobel-econ-2009-intrade

Previously: Nobel Prize for Economics 2009 Predictions

Share This:

The most bombastic prediction markets blogger say that the market has failed if its price close to closing is far away from the final price.

No Gravatar

My dear honorable Eric Crampton,

Here&#8217-s what I published about the &#8220-Olympics in Chicago&#8221- prediction markets:

The market participants did not possess a sufficient level of information completeness to arrive at the correct prediction.

Stay away from these markets where the intention is to divine the decision of a close, opaque group. It is impossible. No good information leaks out.

So, I did not say why you wrote. (I could sue your pants off for defamation. :-D ) I understand that prediction markets have to fail sometimes to be right. But the prediction markets on the Olympics in Chicago are just like playing the lottery. Nobody knows anything. No good information aggregation is possible since no good information is leaking out.

Share This:

If Michael Giberson is wrong, then that means that Chris Masse is right.

Paul Hewitt:

I donta€™ know that you could say Chicago was the a€?weakest linka€?, just because it got dropped first in the voting. The political process caused it to go early. However, Michael Giberson is wrong to imply that the prediction was accurate on the basis that Chicago and Rio were fairly close. Leta€™s keep in mind that the options are about as discrete as they come. Even if Chicago were to have come in a close second, it would have been a complete miss by the market.

If one needed to make a decision that depended on whether Chicago would win the bid, the prior choice would have been completely wrong, once the true outcome was revealed.

I have to agree with Chris. The market participants did not possess a sufficient level of information completeness to arrive at the correct prediction. Furthermore, the discrete nature of the outcomes made it a risky prediction. Finally, Ia€™m guessing that few, if any, of the IOC voting members were involved in the prediction markets, leading one to conclude that all (or almost all) of the market participants were a€?noisea€? traders.

Elsewhere, another commentator claimed that, because the prediction market started to show Chicagoa€™s share falling during the morning of the vote, this was evidence that prediction markets work. Hardly. It does show that prediction markets rarely provide accurate predictions sufficiently in advance of the outcome, in order for useful decisions to be made.

The prediction market industry really needs to investigate the determinants of success and which types of markets (issues) have the potential to provide consistently accurate predictions. Way too much time and effort is being spent arguing about meaningless markets, trivial questions, and false accuracy claims.

Previously: The Chicago candidacy, which was favored by the prediction markets (and gullible bettors like Ben Shannon), is the one that fared the worst.

Previously: Chicago wona€™t have the Olympics in 2016.

ADDENDUM:

IOC

– BetFair&#8217-s event derivative prices:

chicago-olympics-betfair

– InTrade&#8217-s event derivative prices:

chicago-olympics-intrade

– HubDub&#8217-s event derivative prices:

Who will recieve the winning bid to host the 2016 Olympics?