Nobel Prize for Economics 2009 – Prediction Accuracy

No Gravatar

The Royal Swedish Academy of Sciences has decided to award The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel for 2009 to Elinor Ostrom &#8220-for her analysis of economic governance, especially the commons&#8221- and Oliver E. Williamson &#8220-for his analysis of economic governance, especially the boundaries of the firm&#8221-.

Both the bookmakers and the prection markets are utterly useless in trying to divine who will get the Nobel prize of economics.

Below is the 2009 prediction post-mortem:

1. Bookmakers

Ladbrokes&#8217-s probabilities (odds) for the 2009 Nobel prize in economics:

Eugene Fama 2/1
Paul Romer 4/1
Ernst Fehr 6/1
Kenneth R. French 6/1
William Nordhaus 6/1
Robert Barro 7/1
Matthew J Rabin 8/1
Jean Tirole 9/1
Martin Weitzman 9/1
Chris Pissarides 10/1
Dale T Mortensen 10/1
Xavier Sala-i-Martin 10/1
Avinash Dixit 14/1
Jagdish N. Bhagwati 14/1
Robert Schiller [sic] 14/1
William Baumol 16/1
Martin S. Feldstein 20/1
Christopher Sims 25/1
Lars P. Hansen 25/1
Nancy Stokey 25/1
Peter A Diamond 25/1
Thomas J. Sargent 25/1
Dale Jorgenson 33/1
Paul Milgrom 33/1
Oliver Hart 40/1
Bengt R Holmstrom 50/1
Elhanan Helpman 50/1
Ellinor Ostrom 50/1
Gene M Grossman 50/1
Karl-Goran Maler 50/1
Oliver Williamson 50/1
Robert B Wilson 50/1

2. Betting Pools

Here is the betting in the Nobel pool at Harvard:

Robert Barro -10%
John Taylor &#8211- 8%
Paul Milgrom &#8211- 8%
Jean Tirole &#8211- 6%
Oliver Williamson &#8211- 6%
Martin Weitzman &#8211- 6%
Eugene Fama &#8211- 5%
Richard Thaler &#8211- 5%
Lars Hansen &#8211- 4%
Paul Romer &#8211- 4%

3. Prediction Markets

InTrade:

nobel-econ-2009-intrade

Previously: Nobel Prize for Economics 2009 Predictions

Share This:

Assessing the usefulness of enterprise prediction markets

No Gravatar

Do you need to have experience in running an enterprise prediction exchange in order to assess the pertinence of enterprise prediction markets?

Paul Hewitt:

Hi Jed…

As for qualifications, I have been making business decisions for almost 30 years. I am a chartered accountant and a business owner. Starting in university and continuing to this day, I have been researching information needs for corporate decision making. As Chris points out, I’m not a salesperson for any of the software developers. In fact, if I have a bias, it is to be slightly in favour of prediction markets. That said, I still haven’t seen any convincing evidence that they work as promised by ANY of the vendors.

As for whether I have ever run or administered a prediction market, the answer is no. Does that mean I am not qualified to critique the cases that have been published? Hardly. You don’t have to run a PM to know that it is flawed. Those that do, end up trying to justify minuscule “improvements” in the accuracy of predictions. They also fail to consider the consistency of the predictions. Without this, EPMs will never catch on. Sorry, but that is just plain common sense.

The pilot cases that have been reported are pretty poor examples of prediction market successes. In almost every case, the participants were (at least mostly) the same ones that were involved with internal forecasting. The HP markets, yes, the Holy Grail of all prediction markets, merely showed that prediction markets are good at aggregating the information already aggregated by the company forecasters! They showed that PMs are only slightly better than other traditional methods – and mainly because of the bias reduction. Being slightly better is not good enough in the corporate world.

I think I bring a healthy skepticism to the assessment of prediction markets. I truly want to believe, but I need to be convinced. I am no evangelist, and there is no place for that in scientific research. Rather than condemn me for not administering a PM, why not address the real issues that arise from my analyses?

Paul Hewitt&#8217-s blog

Previously: The truth about CrowdClarity’s extraordinary predictive power (which impresses Jed Christiansen so much)

Can we really assess InTrades *very short* prediction market on Van Jones?

No Gravatar

Carlos Graterol has a partial analysis on the Van Jones prediction market at InTrade. Basically, Carlos Graterol (an InTrade fanboy) says that InTrade should be credited for the accurate prediction.

  1. Carlos Graterol should publish what the politicians and editorialists were saying last week (the resignation calls were numerous)-
  2. The InTrade prediction market should have been created much, much earlier &#8212-when Glenn Beck started his &#8216-anti-czars&#8217- guerrilla (at the beginning of August 2009).

vanjonespricehistory

intrade-van-jones-chart

Share This:

Still, as noted, it was a good election for [the] prediction markets and another piece of evidence of their superiority over the pundit[s] (and at least parity with the poll).

No Gravatar

Dixit Nigel Eccles in a comment.

at least parity with the poll

I agree with the above.

their superiority over the pundits

What documented evidence do you have about that, mister the cocky entrepreneurial Scotsman?

John Tierney linked to that Huffington Post that listed the pundits&#8217- predictions about the total number of electoral votes that each presidential candidate would take. But I disagree with that way of predicting the electoral college and assessing these predictions. With this completely flawed method, if you are damn wrong on a state and damn wrong (in the opposite way) about another state that has the exact same number of electoral votes, then you are a bright genius worth the Nobel prize of forecasting. Gimme a break. Enough with that voodoo way of assessing predictions about the electoral college. Do the assessment state by state.

InTrade and HubDub got lucky that their 2 mistakes (so to speak, in a non-probabilistic way) on Missouri and Indiana (both with 11 electoral votes) canceled themselves perfectly. IT WAS PURE LUCK. If their 2 mistakes had been made in the same direction (say, betting on Obama with the outcome going eventually to McCain), and/or their 2 mistakes had been done on 2 very dissimilar states (say, one with 6 electoral votes and the other one with 27 electoral votes), then we would have had reporters and bloggers bashing the prediction markets for the whole month of November.

The Intrade bettors expected Mr. Obama to end up with 364 votes in the Electoral College -one less than he actually got.

No Gravatar

My remark to John Tierney:

InTrade got it [almost] spot on because they were wrong on Missouri (which was predicted to go for Obama but went to McCain) and wrong too on Indiana (which was predicted to go for McCain but went to Obama) —and those 2 opposite mistakes canceled themselves because those 2 states have the exact same number of electoral votes (11). Hence, I disagree with your method.

APPENDIX:

Here&#8217-s a visual post-mortem of the 2008 US presidential elections.

Pay attention to Missouri and Indiana.

A) InTrade, on November 5, 2008 (screen shot taken at 2:00 am):

Prediction Markets &amp- State Polls, on November 4, 2008:

B1) Prediction Markets (on November 4, 2008)

InTrade (screen shot taken at mid-day ET, November 4, 2008):

InTrade (screen shot taken in the morning, November 4, 2008):

BetFair (screen shot taken in the morning, November 4, 2008):

HubDub (screen shot taken in the morning, November 4, 2008):

B2) State Polls (on November 4, 2008)

Karl Rove (on November 4, 2008):

CNN (on November 4, 2008):

Pollster (on November 4, 2008):

Electoral-Vote.com (on November 4, 2008):

Nate Silver (on November 4, 2008):

PREDICTION MARKET PROBABILITIES

Explainer On Prediction Markets

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain event (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism &#8212-with or without an automated market maker.

Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future &#8212-anticipations that will be either confirmed or infirmed.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.

More Info:

– The Best Resources On Prediction Markets = The Best External Web Links + The Best Midas Oracle Posts

– Prediction Market Science

– The Midas Oracle Explainers On Prediction Markets

– All The Midas Oracle Explainers On Prediction Markets

Prediction Market Definition -now updated with the name of Chris Hibbert and Eric Cramptons cult leader built into.

No Gravatar

Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. These event derivative traders feed on the primary indicators &#8212-i.e., the primary sources of information. (Garbage in, garbage out&#8230- Intelligence in, intelligence out&#8230-) Hence, prediction markets are meta forecasting tools.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism.

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative can be interpreted as the objective probability of the future outcome (i.e., its most statistically accurate forecast). A 60% probability means that, in a series of events each with a 60% probability, then 60 times out of 100, the favored outcome will occur- and 40 times out of 100, the unfavored outcome will occur.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.

HUBDUB PUNDIT WATCH: TechCrunch is the bottom of the pool, while VentureBeat and Pat Buchanan are stellar.

No Gravatar

I am not surprised at all by the results.

Maybe a non-profit organization should sponsor PunditWatch.

Robin Hanson (mister &#8220-Track Records&#8221-), take notice.