Tag Archives: Predictions

Robin Hanson: My best idea was prediction markets.

Posted on by
Reply

Robin Hanson‘s auto-biography (i.e., how Our Master Of All Universes views HimSelf):

robin-hanson-drink

–

Robin Hanson:

Do you find it hard to summarize yourself in a few words? Me too.

But I love the above quote. I have a passion, a sacred quest, to understand everything, and to save the world. I am addicted to “viewquakes”, insights which dramatically change my world view. I loved science fiction as a child, and have studied physics, philosophy, artificial intelligence, economics, and political science — all fields full of such insights. Unfortunately, this also tempted me to leave subjects after mastering their major insights.

I also have a rather critical style. I beat hard on new ideas, seek out critics, and then pledge my allegiance only to those still left standing. In conversation, I prefer to identify a claim at issue, and then focus on analyzing it, rather than the usual quick tours past hundreds of issues. I have always asked questions, even when I was very young.

I have little patience with those whose thinking is sloppy, small, or devoid of abstraction. And I’m not a joiner; I rebel against groups with “our beliefs”, especially when members must keep criticisms private, so as not to give ammunition to “them.”  I love to argue one on one, and common beliefs are not important for friendship — instead I value honesty and passion.

In ‘77 I began college (UCI) in engineering, but switched to physics to really understand the equations.  Two years in, when physics repeated the same concepts with more math,  I studied physics on my own, skipping the homework but acing the exams.  To dig deeper, I did philosophy of science grad school (U Chicago), switched back to physics, and was then seduced to Silicon Valley.

By day I did artificial intelligence (Lockheed, NASA), and by night I studied on my own (Stanford) and hung with Xanadu’s libertarian web pioneers and futurists.  I had a hobby of institution design; my best idea was idea futures, now know as prediction markets. Feeling stuck without contacts and credentials, I went for a Ph.D. in social science (Caltech).

The physicist in me respected only econ experiments at first, but I was soon persuaded econ theory was full of insight, and did a theory thesis, and a bit of futurism on the side.  I landed a health policy postdoc, where I was shocked to learn of medicine’s impotency.  I finally landed a tenure-track job (GMU), and also found the wide-ranging intellectual conversations I’d lacked since Xanadu.

My Policy Analysis Market project hit the press shit fan in ‘03, burying me in media attention for a while, and helping to kickstart the prediction market industry, which continues to grow and for which I continue to consult.  The press flap also tipped me over the tenure edge in ‘05; my colleagues liked my being denounced by Senators. :)   Tenure allowed me to maintain my diverse research agenda, and to start blogging at Overcoming Bias in November ‘06, about the same time I became a research associate at Oxford’s Future of Humanity Institute.

My more professional bio is here.

–

Robin Hanson is an associate professor of economics at George Mason University, and a research associate at the Future of Humanity Institute of Oxford University. After receiving his Ph.D. in social science from the California Institute of Technology in 1997, Robin was a Robert Wood Johnson Foundation health policy scholar at the University of California at Berkeley. In 1984, Robin received a masters in physics and a masters in the philosophy of science from the University of Chicago, and afterward spent nine years researching artificial intelligence, Bayesian statistics, and hypertext publishing at Lockheed, NASA, and independently.

Robin has over 70 publications, including articles in Applied Optics, Business Week, CATO Journal, Communications of the ACM, Economics Letters, Econometrica, Economics of Governance, Extropy, Forbes, Foundations of Physics, IEEE Intelligent Systems, Information Systems Frontiers, Innovations, International Joint Conference on Artificial Intelligence, Journal of Economic Behavior and Organization, Journal of Evolution and Technology, Journal of Law Economics and Policy, Journal of Political Philosophy, Journal of Prediction Markets, Journal of Public Economics, Medical Hypotheses, Proceedings of the Royal Society, Public Choice, Social Epistemology, Social Philosophy and Policy, Theory and Decision, and Wired.

Robin has pioneered prediction markets, also known as information markets or idea futures, since 1988. He was the first to write in detail about people creating and subsidizing markets in order to gain better estimates on those topics. Robin was a principal architect of the first internal corporate markets, at Xanadu in 1990, of the first web markets, the Foresight Exchange since 1994, and of DARPA’s Policy Analysis Market, from 2001 to 2003. Robin has developed new technologies for conditional, combinatorial, and intermediated trading, and has studied insider trading, manipulation, and other foul play. Robin has written and spoken widely on the application of idea futures to business and policy, being mentioned in over one hundred press articles on the subject, and advising many ventures, including Consensus Point, GuessNow, Newsfutures, Particle Financial, Prophet Street, Trilogy Advisors, XPree, YooNew, and undisclosable defense research projects.

Robin has diverse research interests, with papers on spatial product competition, health incentive contracts, group insurance, product bans, evolutionary psychology and bioethics of health care, voter information incentives, incentives to fake expertize, Bayesian classification, agreeing to disagree, self-deception in disagreement, probability elicitation, wiretaps, image reconstruction, the history of science prizes, reversible computation, the origin of life, the survival of humanity, very long term economic growth, growth given machine intelligence, and interstellar colonization.

The truth about (enterprise) prediction markets

Posted on by
12

No Gravatar

Paul Hewitt:

[…] In virtually every case, the prediction market forecast is closer to the official HP forecast than it is to the actual outcome. Perhaps these markets are better at forecasting the forecast than they are at forecasting the outcome! Looking further into the results, while most of the predictions have a smaller error than the HP official forecasts, the differences are, in most cases, quite small. For example, in Event 3, the HP forecast error was 59.549% vs. 53.333% for the prediction market. They’re both really poor forecasts. To the decision-maker, the difference between these forecasts is not material.

There were eight markets that had HP official forecasts. In four of these (50%), the forecast error was greater than 25%. Even though, only three of the prediction market forecast errors were greater than 25%, this can hardly be a ringing endorsement for the accuracy of prediction markets (at least in this study). […]

To the despair of the Nashville imbecile, Paul&#8217-s analysis is quite similar to mine (circa February 14, 2009):

The prediction market technology is not a disruptive technology, and the social utility of the prediction markets is marginal. Number one, the aggregated information has value only for the totally uninformed people (a group that comprises those who overly obsess with prediction markets and have a narrow cultural universe). Number two, the added accuracy (if any) is minute, and, anyway, doesn’t fill up the gap between expectations and omniscience (which is how people judge forecasters). In our view, the social utility of the prediction markets lays in efficiency, not in accuracy. In complicated situations, the prediction markets integrate expectations (informed by facts and expertise) much faster than the mass media do. Their accuracy/efficiency is their uniqueness. It is their velocity that we should put to work.

Prediction markets are not a disruptive technology, but merely another means of forecasting.

Go reading Paul&#8217-s analysis in full.

I would like to add 2 things to Paul&#8217-s conclusion:

  1. We have been lied to about the real value of the prediction markets. Part of the &#8220-field of prediction markets&#8221- (which is a terminology that encompasses more people and organizations than just the prediction market industry) is made up of liars who live by the hype and will die by the hype.
  2. Prediction markets have value in specific cases where it could be demonstrated that an information aggregation mechanism is the appropriate method that should be put at work in those cases (and not in others). Neither the Ivory Tower economic canaries nor the self-described prediction market &#8220-practitioners&#8221- have done this job.

Nostradamical.com – 3 months later, they are doing fine.

Posted on by
2

No Gravatar

–

Nostradamical is now 3 months into launch and building a loyal following. Its model is different from our prediction exchanges. It contains more social elements. Nostradamical looks like a multi-user blog with the prediction algorithm(s) as a background feature. Question creators can express themselves on the predictions they are interested in while inviting other users to forecast the predictions &#8212-this is the DIY model pioneered by Inkling Markets in 2006. Users have fun&#8230- Twitter, FaceBook and FriendFeed are integrated, now.

http://www.nostradamical.com/predictions

Damped polls are superior to prediction markets as election predictors.

Posted on by
Reply

No Gravatar

Are Political Markets Really Superior to Polls as Election Predictors? – (PDF file) – by Chris Wlezien and Robert Erikson – 2007

Abstract

Election markets have been praised for their ability to forecast election outcomes, and to forecast better than trial-heat polls. This paper challenges that optimistic assessment of election markets, based on an analysis of Iowa Electronic Market (IEM) data from presidential elections between 1988 and 2004. We argue that it is inappropriate to naively compare market forecasts of an election outcome with exact poll results on the day prices are recorded, that is, market prices reflect forecasts of what will happen on Election Day whereas trial-heat polls register preferences on the day of the poll. We then show that when poll leads are properly discounted, poll-based forecasts outperform vote-share market prices. Moreover, we show that win-projections based on the polls dominate prices from winner-take-all markets. Traders in these markets generally see more uncertainty ahead in the campaign than the polling numbers warrant—in effect, they overestimate the role of election campaigns. Reasons for the performance of the IEM election markets are considered in concluding sections.

Conclusion

This paper has tested the claim that the Iowa Electronic Market offers superior predictions of election outcomes than the snapshots from public opinion polls. By our tests, the IEM election markets are not better than trial-heat polls for predicting elections. In fact, by a reasonable as opposed to naive reading of the polls, the polls dominate the markets as an election forecaster. This is true in the sense that a trader in the market can readily profit by “buying” candidates who, according to informed readings of the polls, are undervalued. Moreover, we find that market prices contain little information of value for forecasting beyond the information already available in the polls. Where then do the markets go wrong? To begin with, consider the vote-share market. The histories of market prices show that traders tend to hold persistent beliefs about the vote division that contradict the polls and that these persistent beliefs are often wrong. Incorrect beliefs get corrected only in the last days before the election, when the polls are difficult to ignore. The winner-take-all market tracks the vote-share market but compounds its errors by overvaluing long-shot candidates’ chances of victory, as if the market expects more campaign surprises than occur in reality. The existence of persistent mistakes in the vote-share market compounded by the degree of uncertainty about the vote-share estimates makes the winner-take-all market a particularly poor forecasting tool. Based on the experience of the IEM, if the polls show a candidate to hold a decisive lead but the market is unconvinced, bet on the polls. It should be noted that our daily poll projections are themselves rather crude instruments. Our robotic trading programs are informed by a flat prior, relying solely on the current polls and the days until the election but nothing more. Even when we compare market prices to the weekly average of poll-based forecasts, our instrument is primitive in that the week’s polls are not weighted for relative recency. But further perfection of our forecasting model from the polls would only advance our central argument. If we were to apply more rigorous modeling to obtain a properly weighted average of current polls and earlier polls, the victory of poll forecasts over the market forecast presumably would be more secure. One could argue that the results are drawn from a limited number of election years from a toy market with thin volume and limits on trader spending. With time, the IEM record could improve, and there is some suggestion that it has. Full-blown markets like Tradesports.com [or InTrade.com or BetFair.com] might in the end achieve an efficiency that so far has eluded the Iowa Electronic Market. Additionally, studies like the present one can suggest improved strategies to traders, which in turn improve the efficiency of election markets. Since our results are confined to a few runs of the toy Iowa market, some might claim a “so what” reaction. To such claimants, an important reminder is that the allegedly uncanny performance of the Iowa market has been touted as the primary evidence for the supposed superiority of election markets over the polls as an information source. The Iowa election market’s performance has not been so special after all. For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.

The truth about prediction markets

Posted on by
Reply

No Gravatar

Come to the wonderful world of collective intelligence, wisdom of crowds, and prediction markets!&#8230- The sun shines bright, the market-generated predictions are vastly superior to the polls as election predictors, and the track record of the public prediction markets stretches as far as the eye can see. There are opportunities aplenty in the field of prediction markets, and the trading technology is cheap. Every working enterprise can have its own internal prediction exchange, and inside every exchange, a set of enterprise prediction markets that correctly predicts the future of business, which their happy, all-American CEO listens to. Life is good in the magic world of prediction markets&#8230- it&#8217-s paradise on Earth.

Ha! ha! ha! ha!&#8230- That&#8217-s what they tell you, anyway&#8230- &#8212-because they are selling an image (just as Bernie Madoff did). They are selling it thru their vendor websites, vendor conferences, vendor-inspired articles in blogs, newspapers and magazines, and interviews of vendor data-fed professors in the media.

The prediction market technology is not a disruptive technology, and the social utility of the prediction markets is marginal. Number one, the aggregated information has value only for the totally uninformed people (a group that comprises those who overly obsess with prediction markets and have a narrow cultural universe). Number two, the added accuracy (if any) is minute, and, anyway, doesn&#8217-t fill up the gap between expectations and omniscience (which is how people judge forecasters). In our view, the social utility of the prediction markets lays in efficiency, not in accuracy. In complicated situations, the prediction markets integrate expectations (informed by facts and expertise) much faster than the mass media do. Their accuracy/efficiency is their uniqueness. It is their velocity that we should put to work.

–

Here&#8217-s now our definition of prediction markets:

–

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain future outcome (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism &#8212-with or without an automated market maker.

Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future &#8212-anticipations that will be either confirmed or infirmed.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other meta predictive mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other meta predictive mechanisms. A highly accurate set of prediction markets has little value if some other meta predictive mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate predictions on its topic.

–

PS: I am updating a bit the content of this webpage, over time &#8212-so as to finesse the message.

The IFTF X2 Project

Posted on by
Reply

No Gravatar

[…] X2 will identify major trends and disruptions in science, technology, and the practice of science over the next twenty years and their impacts on the larger society.

X2 will utilize an open-source web platform that will help individuals and organizations track and analyze global trends in science and technology. The project will employ bottom-up forecasting methods, making use of the collective intelligence of people with different backgrounds, domains of expertise, and geographic locations to synthesize larger patterns and trends. […]

http://www.sciencex2.org/

Prediction Market Definition -now updated with the name of Chris Hibbert and Eric Cramptons cult leader built into.

Posted on by
Reply

No Gravatar

Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. These event derivative traders feed on the primary indicators &#8212-i.e., the primary sources of information. (Garbage in, garbage out&#8230- Intelligence in, intelligence out&#8230-) Hence, prediction markets are meta forecasting tools.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism.

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative can be interpreted as the objective probability of the future outcome (i.e., its most statistically accurate forecast). A 60% probability means that, in a series of events each with a 60% probability, then 60 times out of 100, the favored outcome will occur- and 40 times out of 100, the unfavored outcome will occur.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.

Our play-money prediction exchanges should partner with non-profits.

Posted on by
2

No Gravatar

CORRECTION: It&#8217-s Predictify who comes up with the $100k, not Rock The Vote.

–

Predictify (a prediction platform, not a prediction exchange) is partnering with Rock The Vote.

As I understand it (BEWARE: the Press release is not that clear about where the money comes from), Rock The Vote (a 501c, I suppose) forks over $100.000, which Predictify uses as a prize pool to be handed out between the winners of a 2008 US election forecasting game.

I don&#8217-t get what Rock The Vote gets out of this deal, but that&#8217-s their problem. Rock The Vote gets the free publicity.

I see it as a good idea, and I think that our prediction exchanges should be seeking out deals such as that one with non-profit organizations&#8230- or commercial sponsors. It would attract more traders to our prediction markets.