Systematic Biases in Prediction Market Forecasts

On Super Tuesday, and then again on March 5, there were systematic biases in prediction market forecasts. Specifically, Donald Trump lost four contests that he was predicted to win (Oklahoma, Alaska, Minnesota and Maine) and won no contests that he was predicted to lose (Texas and Kansas).

Another way to express this is as follows: if someone had bet that Trump would lose all eleven states on Super Tuesday at the prevailing prices, they would have secured a substantial positive return, approximately doubling their money, even though he actually won seven states. Each bundle of such bets, one for each state, would have cost about $2 and paid out $4 for a 100% return. On the other hand, if they had bet that Trump would win all eleven states they would have lost money, paying about $9 per bundle to get back $7. And there was enough liquidity available to scale up these bets quite substantially.

The pattern repeated itself on March 5: betting on Trump losses across the board would have made quite a bit of money, betting on him to win everything would have been a money-losing proposition. This was true even though he won two of four states, since betting on him to win was much costlier in the aggregate than betting on him to lose. 

Yet another way to say this is that the markets were not terribly well-calibrated. But this ought to be a temporary phenomenon as wealth is transferred across accounts and traders update their beliefs after each outcome realization.

It's election day again tomorrow, which gives us an opportunity to see if such a correction has in fact occurred. Four states are in play on the Republican side, with Trump heavily favored to win Michigan and Mississippi:

If he fails to win either one of these states it would suggest to me that the biases previously evident have not yet been eliminated. 

As far as the other two contests are concerned, Cruz is favored in Idaho, with Rubio (marginally) favored in Hawaii:

How do these predictions compare with more traditional poll and model based forecasts? On the Republican side, all we have is a Michigan forecast from FiveThirtyEight:

This is in substantial agreement with the prediction markets, so the outcome will not help adjudicate between the two approaches. Similarly, on the Democratic side, there is negligible difference in forecasts: the prediction markets and FiveThirtyEight both agree that Clinton is heavily favored to win Michigan and Mississippi. 

One of the most striking facts that David Rothschild and I uncovered in our analysis of Intrade data from the 2012 election was that the overwhelming majority of traders bet in only one direction. They could be partitioned into Obama enthusiasts and Romney enthusiasts, changing their exposure over time in response to news, but never switching entirely from one side to the other. (These categories are based only on beliefs about the eventual outcome, as represented in bets placed, and need not correspond to political preferences.)

If this pattern holds for the contests currently underway, then poorly calibrated forecasts could be a consequence of over-representation in the market of Trump enthusiasts, or a willingness of such individuals to place larger bets. Even so, the bias should be self-correcting over time, as the wealth of those making consistently losing bets is gradually depleted.

---

Update (March 9). The results are in and it's safe to say that there was no hint of bias in favor of Trump this time: he won Michigan and Mississippi handily and Hawaii became the first state that he won while being predicted to lose. The big surprise, of course, was on the Democratic side where Sanders prevailed over Clinton in Michigan. This outcome was given a likelihood of less than one in ten by FiveThirtyEight and prediction markets, and was even mistakenly called for Clinton at 9pm ET last  night:

We project that Hillary Clinton will defeat Bernie Sanders in Michigan.

— Decision Desk HQ (@DecisionDeskHQ) March 9, 2016

So we still have little to choose from when comparing markets to poll-based predictions, with Kansas being the only state called differently (correctly by markets and incorrectly by FiveThirtyEight). 

The other big news of course was the failure of the Rubio campaign to secure even a single delegate (as of this writing), while Kasich managed to get 17. Even before polls opened on March 5, I wrote:

I suspect that there is a non-negligible probability that Rubio may exit the race before Florida to avoid humiliation there, while Cruz and Kasich survive to the convention.

There's been a lot of chatter about this possibility over the past couple of days, and it seems increasingly likely to me. A Rubio exit before Florida could tip Ohio to Kasich and set up an interesting and unpredictable three-person race going forward. Ohio will also be critical for the continued viability of Sanders. It's a fascinating election cycle.