Saturday, March 05, 2016

Forecasting Elections

This wild and crazy election cycle is generating an enormous amount of data that social scientists will be pondering for years to come. We are learning about the beliefs, preferences, and loyalties of the American electorate, and possibly witnessing a political realignment of historic proportions. Several prominent republicans have vowed not to support their nominee if it it happens to be Trump, while a recent candidate for the Democratic nomination has declared a preference for Trump over his own party's likely nominee. Crossover voting will be rampant come November, but the flows will be in both directions and the outcome remains quite uncertain. 

Among the issues that the emerging data will be called upon to address is the accuracy of prediction markets relative to more conventional poll and model based forecasts. Historically such markets have performed well, but they have also been subject to attempted manipulation, and this particular election cycle hasn't really followed historical norms in any case.

On Super Tuesday, the markets predicted that Trump would prevail in ten of the eleven states in play, with the only exception being a Cruz victory in his home state of Texas. This turned out to be quite poorly calibrated, in the sense that all errors were in a single direction: the misses were Oklahoma and Alaska (which went to Cruz) and Minnesota (where Rubio secured his first victory). But the forecasters at FiveThirtyEight also missed Oklahoma and were silent on the other two so no easy comparison is possible. 

Today we have primaries in a few more states, and another opportunity for a comparison. I'll focus on the Republican side, where voting will occur in Kansas, Kentucky, Louisiana and Maine. Markets are currently predicting a Cruz victory in Kansas (though the odds are not overwhelming):

In contrast, FiveThirtyEight gives the edge to Trump, though again it's a close call:

The only other state for which we have predictions from both sources is Louisiana, but here there is negligible disagreement, with Trump heavily favored to win. Trump is also favored by markets to take Kentucky and Maine, for which we have limited polling data and no predictions from FiveThirtyEight. 

So one thing to keep an eye out for is whether Trump wins fewer than three of the four states. If so, the pattern of inflated odds on Super Tuesday will have repeated itself, and one might be witnessing a systematically biased market that has not yet been corrected by new entrants attracted by the profit opportunity. 

But if the market turns out to be well-calibrated, then it's hard to see how Rubio could possibly secure the nomination. Here's the Florida forecast as of now:

The odds of a Trump victory in Michigan are even higher, while Kasich is slightly favored in Ohio. Plenty of things can change over the next couple of weeks, but based on the current snapshot I suspect that there is a non-negligible probability that Rubio may exit the race before Florida to avoid humiliation there, while Cruz and Kasich survive to the convention. This is obviously not the conventional wisdom in the media, where Rubio continues to be perceived as the establishment favorite. But unless things change in a hurry, I just don't see how this narrative can be sustained.


Update (March 5). The results are in, with Cruz taking Kansas and Maine and Trump holding on to Kentucky and Louisiana. The only missed call by the prediction markets was therefore Maine. Still, the significant margins of victory for Cruz in Kansas and Maine suggest to me that traders in the aggregate continue to have somewhat inflated expectations regarding Trump's prospects. And I'm even more confident than I was early this morning that Rubio faces a humbling and humiliating loss in his home state of Florida, though he may have no option now but to soldier on.


  1. "while a recent candidate for the Democratic nomination has declared a preference for Trump over his own party's likely nominee."

    Jim Webb is really an outlier, and a Republican as recently as 2006.

  2. "Crossover voting will be rampant come November, but the flows will be in both directions"

    I'm not so sure. You do have some democrats switching to Republican or independent to vote in the Republican primary, but that can be for strategic reasons to vote for a candidate they think will be weaker in the general election.

  3. Richard, if Webb campaigns for Trump in VA and elsewhere, or speaks at the convention, I suspect that the impact could be significant. He has a definite constituency and it's up for grabs at the moment.

    I think that strategic voting of the kind you describe is rare. Voting is an expressive act, a private statement of affiliation with a candidate and others who support him or her. Voting for someone you despise, even by accident, is distressing to most people. Think of the butterfly ballot in Florida 2000, and the votes mistakenly cast for Buchanan instead of Gore.

    Trump will attack Clinton from the left on trade, Wall Street ties, and the Iraq war. It will be a brutal and unpredictable campaign.

    1. Trump is scary. Of course, all of the Republicans are horrifying with so much at stake. You will hear my sigh of relief all the way from Tucson if and when Hillary wins.

      But it is fascinating how for so many decades the Republicans were the party of free trade, and the Democrats also, by and large, favored free trade too, but much less so, and with a desire to help those hurt by it. Now, it would be amazing, with Trump, to find the Republicans to be the anti-free-trade party, and the Democrats, correctly, defending free trade (in general)! Who would have ever though that!

      But the Republican party today is really not far from single issue; shovel as much money as possible to the super-rich, especially those in control of the party (Kochs, Adelson, etc.) and their cronies; everything else is pretty much means and political strategy.

  4. I *briefly* looked into the political betting markets last Presidential election. As I recall, the markets were very shallow, and the betting limits were very low, while the fees and hassles were high relative to those betting limits. This would make it so that if the odds were far off of the true odds, highly expert investors would not be able or willing to move those odds close to the true odds. They wouldn’t be able to place nearly a big enough bet. And knowing they could never place very big bets in the first place, they would not spend a lot of time studying these markets, unless it was for another career.

    In addition, with the betting limits so low, top experts just may not find it worth it to make a whole career of betting in these markets, and if you can’t make it a whole career, you can’t diversify away well the idiosyncratic risk.

    Take sports betting, for example. If you think the spread is way off for a single game, you might not bet that much still, because there’s still a lot of idiosyncratic risk with just one game. But if you make a whole career of this, you will be making hundreds of such bets per year, and so you can diversify away the idiosyncratic risk well (hopefully your mistakes are not systematic), and make a lot of money -- if you’re truly, often a lot more expert than the market.

    But you won’t make a career of this as a very smart expert, pushing the odds to be smart, unless you can bet enough per event to make it worth your while, and for enough events in your expertise to diversify away the idiosyncratic risk enough.

    These political betting markets, however, looked like the betting limits were just too low, and the fees and hassles too high, for top experts to make a career in these markets and bet enough to really push the odds. That’s what it looked like to me when I checked around a bit, but I’m sure you have studied the details of these markets a lot more than I have. Do you think this is the case?

    1. IEM has no fees and has predicted well historically. Intrade (now closed) had a small flat fee and no position limits, with very high trading volume and liquidity. PredictIt is new this cycle and has both position limits and fees, but lots of volume and liquidity with narrow spreads.

      The way they are supposed to work is not through expert traders, but through broad participation by a diverse group of amateurs who collectively aggregate scattered information. If the past is any guide, they work well in doing this.

    2. In many markets, including stocks and politics, amateurs are just going to be too lacking in expertise and knowledge to predict very well. The markets pricing well will require experts to jump in and push the prices if they get too far from fundamentals. Ask yourself how well decisions would be made on nuclear power plant design if we just asked a bunch of amateurs to pool their knowledge in a market. Politics and stocks may not be nuclear engineering, but they require far more expertise and knowledge to predict well than the average person has. Without experts, I don’t think these markets would predict nearly as well, so I think depth and betting limits will definitely be a factor in how good these markets are.

      Of course, many amateurs may bet based on what experts say, like in sports, and in that way expert opinion can, to an extent, get into the price, but this can still work far worse than if the experts can enter directly, and big, to place bets. I think, therefore, it’s still an issue the question, how shallow are these markets? How big a bet can be placed?

    3. The point is that the markets predict better than any given pool of experts. Participants are heavily biased one way or another, but the balance of their beliefs as expressed in the price tends to predict very well. There are of course experts also trading - for example James Carville is known to have placed a very early bet on Cruz. But amateurs with good local information have a very important role to play. And there is also selection: better forecasters gain wealth relative to others over time.

    4. "The point is that the markets predict better than any given pool of experts."

      I think this is really going to depend on the market, on what the crowds are giving wisdom on. For some things advanced expertise and information are critical, and aggregating diverse pieces of information (or misinformation, or misunderstanding) that average people bet on matters much less, or can be very statistically biased in a systematic way due to misinformation and misunderstanding.

      What's often best is a deep and low frictions market where amateurs and experts both participate heavily, although there can be exceptions where amateurs are very prone to irrational herd behavior (and experts know they are).

      But where experts cannot prevent well a market from veering far from fundamentals, I'm cautious in interpreting that market, unless perhaps the market for the particular item or event has shown a good and long track record.

      I think the shallowness of these markets is a concern, and the fact that it can be hard to diversify away the idiosyncratic risk like, say, an expert sports or stock bettor can (stock bettors being pooled by people buying their funds or services), where there are just so many similar events that can be pooled, and for big money. An expert in Presidential elections, for example, is probably not very expert in all of the different statewide elections, to diversify his Presidential bets, and there's no network where political expert bettors, expert in different states, etc., can be pooled, like with stock funds.

      I will say, too, as a bachelor's student in economics, an MBA student, and a PhD student in finance, whenever potential market inefficiency would come up, it seems like the defense was almost always that these savvy (expert) marginal investors are what prevents that. They will come in and buy or sell until the market price reflects well the fundamentals (always with pockets deep enough, and no concern about undiversifying their portfolios too much), and they make a living off of the uninformed investors.

      Do you know of a good book or survey article on these market issues and related theory and empirical studies?

    5. Also, if I might ask you an interesting question:

      Suppose there was a market for a coin flip, and the current market probability for heads was 45%. And you know with 100% certainty that the coin is completely fair. You know the market for heads is underpriced for sure by five percentage points. You also know you can't pool well independent bets like this; only one will come about every year. What percentage of your wealth would you place on the heads bet?

      And a further question, suppose you know with 100% certainty that the coin is perfectly weighted so that the true probability of heads is 2%, but the current market probability is 1%. Now how much do you bet, given that the undiversifiable idiosyncratic risk is much worse in this case?

    6. If I had to hold on to my position for a few months until the coin was flipped I would bet nothing. But that's not how prediction markets work. If I could buy a security for 45 that paid 100 with probability 1/2 on election day but could be traded in a liquid market continuously before then, I'd bet a few hundred dollars, maybe more, and sell in a day or two when the price rose up towards 50.

      For the second question I'd bet even more, since the asset is priced at 50% of its intrinsic value, and I'd be reasonably certain of getting a 10 or 20% return within a day or two. Why so certain? Because others would also be buying.

    7. On prediction market accuracy, a good survey is:

      Berg, Joyce, et al. "Results from a dozen years of election futures markets research." Handbook of experimental economic results 1 (2001): 742-751.

      (also linked in the post above)

    8. With regard to the coin flip, yes, I should have made clear that the positive expected return is not over some long period of time. So, suppose the coin will be flipped in just one day, so an extremely high annualized expected return.

      Next, you are assuming that the mispricing will gradually get corrected to the 50% as we approach flip time (maybe it won’t), so you can sell before the flip and make a profit before incurring the big idiosyncratic risk of the flip. It’s a good point to consider, selling before the flip, but suppose you can’t. Suppose you have to risk the flip.

      The analogy is, suppose there is a big idiosyncratic risk over even a short holding period of some new information coming out that is harmful to your position, some new scandal, say, with your political candidate. You know the probability of such a thing is lower than the implied market probability, but still very substantial. What do you bet, given the still high undiversifiable idiosyncratic risk?

      So, my point in these questions is that political market betting may have a high level of undiversifiable idiosyncratic risk, and this will greatly discourage expert investors from pushing the odds to fundamentals even if the expert investor has perfect information that the true odds are very significantly short of the market odds. And your answer of a few hundred dollars shows this. As an economics professor at Columbia for many years, it’s very likely you have access to many orders of magnitude more wealth than a few hundred dollars, yet that’s all you bet, with an annualized expected return that’s absolutely enormous compared to standard investments.

  5. Yes, there's what I like to call fundamental risk - the chance that fundamentals will move against you in the short period before you cash out. That's what limits the investment to a few hundred dollars. But a few hundred traders acting this way gives you a pretty accurate market.

    By the way, if the coin flip were tomorrow I wouldn't bet... In general I never want to have a long or short position when uncertainty is actually realized. Too risky.

  6. "That's what limits the investment to a few hundred dollars. But a few hundred traders acting this way gives you a pretty accurate market."

    I wish I had time to add this earlier, and I'll try to make this point in the future, but a few hundred expert investors in a market can provide great information in some cases, but not if their expert input is massively diluted by investors whose beliefs are far from the fundamentals. Experts controlling only a very tiny percentage of the amount of money invested will not be able to impact the price much, as we see with bubbles, when they're totally outvoted by investors with beliefs far from the fundamentals.

    The experts just don't control enough money to move the price much. It's not the absolute number of experts, but the relative amount of money they put in the market.

  7. Richard, what seems to happen in electoral prediction markets (unlike stock markets) is that there are a lot of people with extreme beliefs but they take opposite sides of the bet so the net effect is not large, and opportunistic traders then carry a lot of weight. My paper with David Rothschild has some evidence on this: