A bunch of Marginal Revolution commenters dared criticizing Keith Jacks Gamble’-s Super Bowl Analysis Highlights (+ addendum). The impudence! The audacity! They called down the thunder- they should get ready for the boom. [*]
Yes, the results of individual plays depend on lots of factors including play selection, and there is no way my analysis can separate all of these factors. After all, football is a team sport, and thus I suspect that cleanly identifying a player’-s sole contribution play-by-play to his team’-s chance of winning may be impossible. Certainly the performance of the offensive line has a ton to do with a team’-s success. But even with standard statistics for measuring a player’-s performance these issues remain. Quarterbacks get credit for completions, passing yards, and throwing touchdowns even though the coach’-s play calling, the line’-s blocking, the receiver’-s route running, ect have everything to do with creating those numbers. Attributing actions to the primary actors on a play is a natural way to compute statistics. At least my net probability points statistic provides some weight to the game situation when measuring a player’-s impact on a given play. Certainly, a 3 yard run on 3rd and 2 at the opponent’-s 4 yard line means more than a 3 yard run on 3rd and 10 at a team’-s own 20. This difference isn’-t captured in rushing yards, but it plays a major role in my numbers.
It’-s not ludicrous to think that the error rate of a betting market is low. In fact, Professor Paul Tetlock’-s research [PDF] shows that it is low. He finds that the implied probabilities (prices) for sports contracts on Tradesports.com are very close to the frequency that these contracts payoff. See Figure 1 on page 41 and notice that for his data the differences between prices and frequencies are actually quite small- in the range of prices from 40 to 80 (where the prices were for most of the Super Bowl) he finds deviations of around 1 (1% in probability). Furthermore, a look at Table 1 on page 34 shows that the standard errors for the estimated deviations in this price range are too large to rule out “-no deviation”- as an unlikely truth.
True, before kickoff the betting market estimated that Florida had about a 30% chance of beating Ohio State, and certainly before kickoff the market would have estimated that Florida had only an very small chance of winning by so many touchdowns. However, I’-m sure that the market’-s estimate of Florida winning by so much was not zero. Thus, this one example doesn’-t say much of anything about the error rate of betting markets. Small probability events do happen. I once saw a lady win $5,000 on a quarter slot machine in Vegas, but that doesn’-t make it ludicrous for me to think that my chance of doing the same was extremely small.
I think that my analysis does account for bruising running, clock control, and ball control. If a Rhodes run wears on the defense, the market sees this fact and will raise the probability of the Colts winning more so than if he had just fallen down at the same spot without knocking into a defender. Also with clock control, if a Colts receiver stays in bounds to keep the clock moving to protect a lead, then the market sees this fact and will raise the probability of the Colts winning more so than if he had run out of bounds at the same spot. Certainly, Addai not fumbling will raise the probability of the Colts winning more so than if he had not controlled the ball. The reason that you don’-t see these factors greatly boosting Rhodes’- and Addai’-s numbers in my analysis is that these things are to be expected of any NFL running back. All running backs pound defenders, stay in bounds when necessary, and hold on to the ball when most important. Thus, market prices only change a little when these actions are done successfully. Doing good things that are to be expected do not count that much towards a player’-s performance in my analysis. However, doing something bad when something good is to be expected, such as fumbling, really hurts a player’-s performance statistic. For example, Addai’-s fumble and Manning’-s interception hurt their overall net performance statistic.
I was surprised by how low Rhodes’- performance measured in my analysis, and I think it’-s in part due to poor play calling that unfortunately gets counted against Rhodes in my analysis. For example, when Rhodes ran for 8 yards on a 3rd and 10 inside the red zone, the market dropped the Colts chance of winning by 4.5% even though I think Rhodes getting 8 yards was a great outcome for a RUN inside the 10. The priced dropped because that play pretty much ended the Colts chance at scoring a touchdown on the drive given that the field goal team was now trotting onto the field. I think this price drop is suggestive that the Colts made a mistake in their play calling. There’-s also a case in which Rhodes ran for 5 yards to midfield on a 2nd and 13, but the probability of the Colts winning dropped by 1%. Even though 5 yards is an above average outcome for a run, it’-s not a good outcome for a play on a 2nd and 13, because the probability of a punt goes up. Although, not related to play calling, all of Rhodes’- yards in the last 5 minutes of the game amounted in just a .5% increase in the Colts probability of winning because at this stage of the game the market was fully convinced that the Bears chance at a comeback was was less than 1%.
It’-s a common misconception that betting lines are set up to get equal money on both sides. Betting lines are set up to maximize the sportsbooks’- profits. See Steven Levitt’-s paper [PDF] or just take a look at Sportsbook.com’-s betting trends. NFL betters overbet on favorites, so I wouldn’-t be surprised to see underdogs beating the spread a little more often than favorites. However, I would be shocked if underdogs can truly be expected to beat the spread more than 52.4% of the time since it would mean that an underdog better with a big pocketbook could expect to make a lot of money from the sportsbooks. Also, my data is taken from Tradesports.com, an online exchange that takes no positions unlike a sportsbook. It’-s a marketplace of betters, and so far, the evidence is that the prices there are pretty good estimates of probabilities. […-]