Emile Servan-Schreiber’-s blog post about the Dutch political elections and the NewsFutures prediction markets:
[…] At first glance, the quality of the market’s vote share predictions were somewhat disappointing compared to the three main polls […] However, the story changes dramatically if, instead of looking at the predictions just before the election, we consider the predictions several weeks earlier. Four to seven weeks before election day, the polls were much less reliable and the market clearly out-predicted them, individually and as a group. […]
This does not fly in my book. To be useful to us, the accuracy of the prediction should increase with time, not decrease.
UPDATE: Mike Giberson…-
[…] Accuracy should improve in most cases as information improves, and generally we expect information about the near future to be better than information about the more distant future.
It is sort of fishy to, after the fact, go sorting back through the data to find “the” accurate prediction. Where is the theory? Better to take the missed predictions as real and try to understand them, rather than engage in after-the-fact data sifting to justify prior beliefs about the accuracy of PMs. […]
The ESS line you highlighted reminds me of something I read recently – maybe the Intel paper? – which showed the same pattern. Increasing accuracy which then tended to go a little haywire just as the predicted quarter itself approached. I think the paper attributed the effect to the rapid increase in information as the event draws near, and suggested that the increased information overwhelms the ability of the mechanism to sort.
I second your reaction on two points:
Accuracy should improve in most cases as information improves, and generally we expect information about the near future to be better than information about the more distant future.
It is sort of fishy to, after the fact, go sorting back through the data to find “the” accurate prediction. Where is the theory? Better to take the missed predictions as real and try to understand them, rather than engage in after-the-fact data sifting to justify prior beliefs about the accuracy of PMs.
(Note that I have not gone back to read the original post, I’m just reacting to what you’ve presented.)