Tag Archives: accuracy

It wasnt about the predictions.

Posted on by
10

No Gravatar

Let&#8217-s not confuse media visibility with utility. Aside from the depressed Obama-to-win prices on one exchange, prediction market and polling aggregation results for the 2008 election were essentially the same using squared errors. Despite his insane schematics, Emile Servan-Schreiber has a good point about capturing the interest of the public, something that nerdy academic and libertarian-types aren&#8217-t necessarily good at. An Obama-backing baseball statistician out of Daily Kos nailed that part this year, a year where people were especially skeptical of markets, not to mention unregulated &#8220-offshore&#8221- ones. Likewise, if you put down the lens of considering markets as commission generators, you&#8217-ll see the value of contracts tied to social and cultural outcomes. Of one the biggest assets of prediction exchanges is media goodwill, which should be fostered by distilling information on subjects like global development and art prices.

Other things to keep in mind:

  • This year happened to have a lot of favorite-longshot States, which turned-out to be favorable to 538&#8217-s error relative to markets.
  • Prediction markets register information in real time. Since the difference in error is small, this is important.
  • Markets are more flexible, and useful in situations where you don&#8217-t have a rich data set and obvious statistical analyses. Elections are just one type of question. Even if you have data, it might be less expensive to set up a new contract than to undertake the analysis.
  • And of course, prediction markets have functions aside from forecasting, and provide incentives for uncovering new information.

The fact that Emile Servan-Schreiber (usually, a smart man) treats the 2008 US presidential elections, as seen thru the lens of the NewsFutures prediction markets, so lightly, making it a race of spermatozoids swimming their way to the Oval Room, shows you that the prediction market luminaries are i

Posted on by
14

No Gravatar

Download this post to see the NewsFutures widget below.

Political prediction markets should move beyond mere horse-race forecasts to demonstrate larger social value.

Posted on by
12

No Gravatar

I agree with that.

The key, now, is to go beyond the accuracy issue and to move on to the utility issue.

It&#8217-s a much complex problematic, which those who have been over-selling the prediction markets are unwilling to undertake. [*]

Maybe a small bunch of prediction market people, maybe assembled in a new prediction market structure, might go for that lofty goal of fingering the specific instances where prediction markets create real social utility.

–

[*] Yelling across the harbor, like an illuminated Jesus Christ, that prediction markets can help &#8220-avoiding future [financial] crisis&#8221- is a sign that some prediction market practitioners have lost their intellectual compass. To my knowledge, InTrade hadn&#8217-t had any prediction market focused on the &#8220-looming credit crunch crisis&#8221-, last summer. Its CEO should be careful about making any grand statement. As I wrote many times, at best, the prediction markets are the best umpire you can have between either the mass media and the politicians, on one hand, and a group consisting of the best experts, on the other hand. An umpire is only useful during critical times, in a game. But, other than that, most of the times, the umpire is not the determinant of the game &#8212-the players are.

The researchers and practitioners should make a solid case for each of these critical instances where the prediction markets have a real social utility.

Stop the over-selling. Let&#8217-s start the real work.

–

InTrade CEO John Delaney states that prediction markets can prevent the next financial cataclysms. Surely. Prediction markets can also restore womens virginity, and treat mens baldness.

Posted on by
16

No Gravatar

John Delaney states rightfully that the prediction markets are a mechanism that aggregates information dispersed among the population. Then, he goes on at full throttle and states that prediction markets can help &#8220-avoiding future [financial] crisis.&#8221-

Jesus, Mary, Joseph, that&#8217-s quite an extraordinary statement.

John Delaney writes that crucial information is buried deep in the accounting books. That&#8217-s true, but that&#8217-s up to the financial analysts to decipher this problematic &#8212-our event derivative traders can then just pick up on what those experts conclude. The financial experts were unable to prevent the current financial cataclysm. Adding more event derivative traders and more prediction markets won&#8217-t solve any problem.

Prediction markets are only a reflection of the current knowledge of the best experts in town. At best, they are the best umpire you can get between, on one hand, the mass media or the politicians and, on the other hand, the best experts. But when nobody knows anything (or when nobody listens to Nouriel Roubini), the prediction markets are of no help.

What the prediction market industry needs right now is not an ill-informed, bragging rant.

What the prediction market industry needs is a way to discriminate between accuracy and utility.

What we need is more of Robin Hanson.

–

UPDATE:

Prediction Market Definition -now updated with the name of Chris Hibbert and Eric Cramptons cult leader built into.

Posted on by
Reply

No Gravatar

Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. These event derivative traders feed on the primary indicators &#8212-i.e., the primary sources of information. (Garbage in, garbage out&#8230- Intelligence in, intelligence out&#8230-) Hence, prediction markets are meta forecasting tools.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism.

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative can be interpreted as the objective probability of the future outcome (i.e., its most statistically accurate forecast). A 60% probability means that, in a series of events each with a 60% probability, then 60 times out of 100, the favored outcome will occur- and 40 times out of 100, the unfavored outcome will occur.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.

Nobel laureate Gary Becker and judge Richard Posner both wish that, one day, real-money prediction markets will be legal, without restrictions, in the United States of America.

Posted on by
Reply

No Gravatar

Via David Pennock of Odd Head fame

–

Gary Becker:

[…] I believe that online political prediction markets, and other online prediction markets as well, should be legal in the United States and elsewhere, even if the amounts bet were quite large. There is no important substantive difference between such online betting markets and the Chicago Mercantile Exchange and other exchanges that allow individuals and organizations to take positions on movements of stock indexes, housing price indexes, and prices of other derivatives. A distinction is sometimes made between political betting markets and derivative markets since participants in derivative markets may be hedging other risks that they face. Yet this distinction has little substance since if larger bets were allowed in online political markets, groups whose welfare depended greatly on political outcomes would make greater use of these markets. For example, if a Republican presidential win would mean greater spending on military weapons, companies in the arms business might hedge their risks by betting on Barack Obama.

If large bets were allowed, some wealthy groups may bet a lot on their candidates in order to exert bandwagon influences on public opinion through their large bets affecting market odds. If so, these markets likely would become less reliable as predictors of outcomes, and hence would have less influence on opinions. To a large extent, therefore, these markets would be self correcting, although online political markets might place various other restrictions on bets, as is common in derivative and other exchanges.

–

Richard Posner:

[…] There is an interesting question whether prediction markets should be thought of as &#8220-gambling” and perhaps prohibited. As a matter of policy, that would be a mistake, even if one thinks that gambling should be prohibited. The prediction markets are markets for speculation, rather than for game-playing or risk-taking. Slot machines, card-playing, roulette wheels, and other conventional forms of gambling do not generate socially valuable information. Speculation does. Commercial speculation serves to hedge commercial risks and bring prices into closer phase with value. Political, cultural, etc. prediction markets also yield socially valuable information. The outcome of elections is important to companies and even individuals for whom particular public policies are important- they may wish to make adjustments to avert or exploit looming political change. Politicians too need to have as sharp a sense as possible about the effects on the electorate of their and their opponents&#8217- strategies. Apparently they can get more accurate information from the prediction markets than from the public opinion pollsters.

–

Is Intrade out on a limb?

Posted on by
26

No Gravatar

As I write this, Intrade gives the advantage to McCain over Obama and has the Republican party even with the Democratic party to win the election, whereas all the other prediction markets, meaning IEM, Betfair, and the NewsFutures play-money kind still favor a Democrat in the White House. That disconnect prompted Chris to wonder aloud whether Intrade is faster than the other markets to incorporate the latest polls, perhaps because of its &#8220-bigger liquidity&#8221-.

That&#8217-s an interesting reaction on several levels.

First, reactivity and accuracy are not to be confused for one another. Given that market prices are supposed to be more accurate and more stable that fickle U.S. raw polls (Berg et al, 2008), one should not necessarily be impressed by the market that is quickest to mirror the latest polls. I very much doubt that traders in the &#8220-other&#8221- markets have not heard about the latest polls giving McCain an edge. Rightly or wrongly – it is too soon to tell – they just gave those polls less weight that the Intrade traders apparently did.

Second, the argument from &#8220-bigger liquidity&#8221- is not receivable. Recently, Paul Tetlock analyzed Tradesports data in depth and found that more liquidity may in fact make the market dumber. He concludes: &#8220-In both sports and financial prediction markets, the calibration of prices to event probabilities does not improve with increases in liquidity- and the forecasting resolution of market prices actually worsens with increases in liquidity.&#8221-

My personal theory is that Intrade has a hair-trigger Republican bias which is not found in the other markets, because Intrade appeals to, and is marketed to, the more Republican-leaning segments of the U.S. population. In my opinion, the Intrade/Tradesports Republican bias was already evident in the 2004 election, as this analysis shows.

Of course, I may be completely wrong. In any case, I find today&#8217-s dual disconnect between the polls and most of the markets, on the one hand, and between Intrade and the other markets, on the other hand, to be two very interesting data points that should be duly recorded so we can come back to them later, with hindsight.

State Polls versus Electoral College Prediction Markets

Posted on by
7

No Gravatar

Prediction market analyst Lance Fortnow in an e-mail to me:

Right now the electoral college markets are tracking the polls pretty closely. I think we&#8217-ll see some divergence when we get close to the election since the polls can&#8217-t keep up. In past elections the markets were much better than the polls within a few days before the election (though not on election day itself which has too many rumors).

Other thoughts:
– There is a long-shot bias &#8212-states which are above 85% (for one candidate or the other) reflect a probability closer to 100%.
– The state markets are strongly correlated. There is a small but non-trivial chance that many states will be way off this year. And then people will be reluctant to trust the electoral college markets in the future.

So, I have (at least) one answer to my series of provocative questions: Electoral college prediction markets are more useful than the state polls towards the very end of the presidential campaign (but not on Election Day). Interesting. Thanks.

PS: The discussion about this post goes on in the comment area of another post.

Paul Kedrosky … sucks. – Plus, Jason Trost is bashing InTrade and BetFair in order to boost his startup, Smarkets.

Posted on by
Reply

No Gravatar

I will soon publish a full post on the relative accuracy of the VP prediction markets (and whether the prediction markets are useful at all, taking into account the occasional upsets theorized by Koleman Strumpf) &#8212-tackling Paul Kedrosky, Felix Salmon, and Barry Ritholtz.

  • By the way, I was pleased to see (elsewhere) that Emile Servan-Schreiber of NewsFutures has the same (bad) opinion about those VP prediction markets as I do. I will index Emile in this file, next week. Emile is a smart and experienced man, and we have the confirmation of this under our very nose (elsewhere), once again. Emile Servan-Schreiber = one of the great thinkers of the field of prediction markets.
  • I was also pleased to see that our good Wall Street friend Eddy Elfenbein got things right.

–

Now, back to the PM-bashing Paul Kedrosky post:

–

Paul Kedrosky criticizes the &#8220-boosters&#8221- of the prediction markets &#8212-Justin Wolfers, Robin Hanson, John Delaney, Chris Masse, etc.

As for me, I dislike opportunistic bloggers and venture capitalists like Paul Kedrosky who approach the prediction markets without a basic understanding of the forecasting approach.

–

Here is the explainer that I have been publishing on the frontpage of Midas Oracle, for all to see.

–

Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that traders bring when they agree on prices. Prediction markets are meta forecasting tools that feed on the advanced indicators (i.e., the primary sources of information). Garbage in, garbage out&#8230- Intelligence in, intelligence out&#8230-

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative can be interpreted as the objective probability of the future outcome (i.e., its most statistically accurate forecast). A 60% probability means that, in a series of events each with a 60% probability, then 6 times out of 10, the favored outcome will occur- and 4 times out of 10, the unfavored outcome will occur.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism.

–

Finally, for those who missed it, the great interview given by Koleman Strumpf to CNN, in April 2008.

–

CNN:

FOREMAN: I&#8217-ve got something I want you to take a look at. Look at this. It could be the price of a stock or a mutual fund. It isn&#8217-t. It&#8217-s the odds that a particular candidate, the red here is Hillary Clinton, who will become president of the United States. It&#8217-s called the [prediction] market. And while supporters say it&#8217-s no different than any other type of investment, for example, hog bellies or pork futures, or crude oil, it sure looks like gambling.

Economics Professor Koleman Strumpf is on the faculty at the University of Kansas School of Business to talk about it now. And in Las Vegas, another scholar in the art of predicting future events, Johnny Avello, the director of the Sports Book at Wynn, Las Vegas.

Let me start with you, professor. How well does this work? When people start betting online as to who&#8217-s going to win, does it work?

KOLEMAN STRUMPF, UNIVERSITY OF KANSAS: Yes, the markets have a really tremendous track record dating back at least on the online markets to 1988. And actually earlier, there were impromptu markets that existed outside of Wall Street in the early 20th century.

These things have been around for at least 100 years or probably more. 150 years. And with maybe only one or two exceptions that I can think of, they&#8217-ve done a just totally dead on job at forecasting.

[FOREMAN]: Long before polling existed, then you&#8217-re saying we had betting. And there were betting lines in newspapers. And if you want to know who&#8217-s winning the presidential race, that&#8217-s what you looked at?

STRUMPF: That&#8217-s exactly right. So in 1904, &#8220-The New York Times&#8221- reported on the front page what was going on at the Wall Street betting markets since there was no Gallup Poll that existed at that time.

FOREMAN: Johnny, why do you think that this is generally so successful compared to polling?

JOHN AVELLO, DIR. OF RACE &amp- SPORTS, WYNN HOTEL: Well, first of all, you are taking actual bets. And you know, each person that puts their money up is a good indication of, you know, which way they like it.

When you&#8217-re doing polling, you know, that&#8217-s kind of an ambiguous way of finding out who the winner is because you&#8217-re getting a fraction of the people who you&#8217-re actually finding out who they like. So I like to call it money versus unpredictability.

FOREMAN: You&#8217-re saying the difference is that in a poll, somebody may say something that they believe in generally, or they think that the pollster wants to hear. But when they put money down, they&#8217-re going to really bet on what they think is going to happen?

AVELLO: It&#8217-s the real thing.

FOREMAN: It&#8217-s sort of interesting when you look at these different types of polls. There&#8217-s the In Trade system, which is out of Dublin, Ireland that&#8217-s sort of interesting. In Trade allows people to bet actual money and large amounts of it on all sorts of outcomes all around the world.

The Iowa markets is one the people have talked about a lot. Set up by a university there. Basically the Iowa markets allow people to bet in limited amounts of money. About $500. And it&#8217-s used simply to see how well this works for educational purposes mainly.

Professor, when we talk about these, though, why did polling ever become popular if betting works so well in telling how we would win?

STRUMPF: Well, I think a kind of &#8212- much like today, the newspapers, the media was always uncomfortable reporting these markets. There sort of was dubious legal status and maybe some moral issues with it.

Polling for whatever reason seems to have been morally acceptable to the media. And I think as a result when Gallup came around in the late 1930s, the betting markets kind of fell by the wayside. They never of course disappeared, but I think it was sort of a moral issue, the same kind of moral issues that I think arise today in thinking about gambling.

FOREMAN: So let me ask you this, Johnny. Why do you think it&#8217-s been so difficult this year, though? Because as far as we can tell, the betting lines have not done any better than the pollsters this year in predicting this election. It has been all over the map. And the betting has been all over the map.

AVELLO: Well, one thing to remember, Tom, is that when the book maker&#8217-s putt
ing up a line, what they&#8217-re trying to accomplish is divided action. So they&#8217-re not trying to pick the winner because let&#8217-s take for instance if the book maker put up Hillary Clinton at one to two, and she was bet from to three to win the Democratic nomination. You know, and Obama won. People would say, wow, the book maker really got killed on that.

FOREMAN: The book maker&#8217-s not trying to predict it, but obviously, the gamblers are, the people who are betting on these things. And they haven&#8217-t done well this time, not compared to past elections. Why do you think that is? Why is this so hard to sort out?

AVELLO: Because they&#8217-re not always right. No one&#8217-s right at 100 percent of the time. I would say best case. You know, I know that history has shown that the bettor has done well on this. But to be perfectly honest with you, I think if you do 60 percent, you&#8217-ve done a great job of picking the winner.

FOREMAN: And Koleman, do you think there&#8217-s anything unique that&#8217-s making it harder for the betting markets to be as accurate as they had been in the past?

STRUMPF: Well, it&#8217-s obviously a close market and opinions are changing rapidly. I just want to kind of maybe extend a little on what Johnny just said in terms of thinking about what these markets mean.

The markets give us a probability of an event occurring. So even if, for example, Obama is a 80 percent favorite in the upcoming Wisconsin primary, which he is, that still means on the flip side that there&#8217-s a 20 percent chance that he&#8217-s not going to win. So the markets, again as Johnny had said, don&#8217-t – they – by sort of definition can&#8217-t predict an upset. An upset is a surprise which people hadn&#8217-t anticipated. So sometimes there are these quick shifts of opinion, which I &#8212- to the best of my knowledge, there&#8217-s no way to forecast that in advance.

FOREMAN: And this campaign has been just filled with them. Johnny, one last thing here. Any sense of where the smart money is going these days?

AVELLO: Well, let&#8217-s say that there&#8217-s been a shift. I believe the smart money was on Hillary Clinton early, and has shifted to Obama. But surprises do happen. And all you need to do is look at the Superbowl to find that out.

FOREMAN: Johnny, thanks so much. Koleman, as well. We appreciate you being here. And speaking of gambling, Madame Tussaud&#8217-s Wax Museum here in Washington is hedging its bet. And that kicks off our political side show.

–

Back to the PM-bashing Paul Kedrosky post.

Here&#8217-s the sneaky, un-informed and totally biased ( :-D ) comment from our good friend Jason Trost (who should know better), which was attached to the Paul Kedrosky post:

–

UPDATE: Portfolio + Computational Complexity