Sunday, March 06, 2011

On the Interpretation of Prediction Market Data

As the election season draws closer, considerable attention will be paid to prices in prediction markets such as Intrade. Contracts for potential presidential nominees are already being scrutinized for early signs of candidate strength. In a recent post on the 2012 Republican field, Nate Silver used prediction market data (among other sources of information) to generate the following very interesting chart:

Source: FiveThirtyEight: Nate Silver's Political Calculus

While Nate's post was concerned primarily with the positioning of candidates along two-dimensions of the political spectrum, he used market prices as a proxy for the probabilities of eventual nomination:
[The] area of each candidate’s circle is proportional to their perceived likelihood of winning the nomination, according to the Intrade betting market. Mitt Romney’s circle is drawn many times the size of the one for the relatively obscure talk-radio host Herman Cain because Intrade rates Mr. Romney many times as likely to be nominated.
This interpretation of prices as probabilities is common and will be repeated frequently over the coming months. But what could the "perceived likelihood according to the market" possibly mean?

Markets don't have perceptions. Traders do, but there is considerable heterogeneity in trader beliefs at any point in time. Prediction market prices contain valuable information about this distribution of beliefs, but there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense. In fact, to make full use of market data to make inferences about the distribution of beliefs, one needs to look beyond the price at last trade and examine the entire order book.

As an example, consider Intrade's market for the presidential election winner by party. This market consists of three contracts comprising a mutually exclusive and exhaustive set of outcomes. One  contract pays out if the winner is a Democrat, a second if the winner is a Republican, and the third if the winner is not the official nominee of either party. The current prices of these contracts are as follows:

Contract Bid AskLast
PRESIDENT.DEM.2012 62.562.9 62.5
PRESIDENT.REP.2012 35.1 35.5 35.0
PRESIDENT.OTHER.2012 2.2 2.3 2.2

These prices are expressed as percentages of contract face value, which in each case is $10. That is, the price at last trade of the DEM contract was $6.25. The buyer risks this amount (per contract purchased) and stands to receive $10 if (and only if) the specified event occurs. The seller risks $3.75 to take the opposite side of the bet.

It's tempting to interpret the price at last trade as a probability because the sum of these prices adds up to approximately 100% of the contract face value. The reason for this is that the sum of the ask prices must be no less than 100, otherwise an arbitrage opportunity would exist: one could buy all contracts and be sure that one will expire at face value, thus generating in a risk-free profit. Similarly, the sum of bid prices must be no greater than 100. If the market is liquid, so that bid-ask spreads are small, then all prices (bid, ask, and last) will sum to approximately 100. This is the basis for the claim that, at current prices, the "market" is predicting that the Democratic nominee will win the White House with probability 62.5%.

But is this interpretation reasonable? All that the price at last trade can tell us about is the beliefs of the two parties to this transaction. If both are risk-averse or risk-neutral, they each must believe that entering their respective positions will yield a positive expected return. Hence the buyer must assign probability at least 62.5% to the event that the Democrat is elected, while the seller assigns a likelihood of at most 62.5% to this event.

This tells us nothing about the beliefs of traders who are not party to this transaction. However, additional information about the distribution of beliefs in the trader population can be obtained by looking at the order book, which at present looks like this:


Note that there are several large orders (in excess of 100 contracts) but these are unevenly distributed on the two sides of the market. Consider, for instance, the bid for 500 contracts at 62. Whenever such an order is placed, Intrade freezes funds in the trader's account equal to the worst case loss, which in this case is $3,100. Upon expiration, these contracts will be worth either $5,000 (if the event occurs) or they will be worthless. Again, assuming risk-aversion or risk-neutrality, one can impute to the potential buyer a belief that the event will occur with probability at least 62%.

But this imputation will be an underestimate for at least two reasons. First, the greater the degree of risk-aversion, the more compensation a trader will demand to enter a risky position. Since these positions are indeed very risky, it is likely that many of those placing large standing bids have significantly positive expected returns, and hence believe that the probability of the event occurring exceeds by some measure the imputed value.

Second, traders placing large bids are aware that conditional on their order being met, it is likely that some news will have emerged that makes the event less likely to occur. That is, they understand that a trade against their order is more likely to occur in the event of bad news (from their perspective) than good news. Taken together, these factors imply that traders placing large bids must be considerably more optimistic about the occurrence of the event than the naive imputation of 62% would suggest.

The same reasoning applies to those taking positions on the sell side: traders placing large limit orders must believe that the event is considerably less likely to occur than a naive reading of their posted price would suggest.

What, then, can one say about the distribution of beliefs in the market? To begin with, there is considerable disagreement about the outcome. Second, this disagreement itself is public information: it persists despite the fact that it is commonly known to exist. That is, traders don't attribute differences in beliefs simply to differences in information applied rationally to a common prior. (This follows from Aumann's famous theorem which states that individuals who have common priors and are commonly known to be rational cannot agree to disagree no matter how different their private information may be.) As a result, the fact of disagreement is not itself considered to be informative, and does not lead to further belief revision. The most likely explanation for this is that traders harbor doubts about the rationality or objectivity of other market participants.

Third, there is a cluster of large buy orders at around 62, and a cluster of large sell orders in the 64-65 range. Hence there are some traders who believe quite confidently that Democrats will hold the White House with probability considerably greater than 62%. And there is another group who believe, also confidently, that the chances of this occurring are quite a bit below 64%. As things stand, the former group appear to be either more numerous or more confident in their judgments.

More generally, it is entirely possible that beliefs are distributed in a manner that is highly skewed around the price at last trade. That is, it could be the case that most traders (or the most confident traders) all fall on one side of the order book. In this case the arrival of seemingly minor pieces of information can cause a large swing in the market price. Of course, such swings may draw into the market other participants whose beliefs are not currently represented in the order book. But the bottom line is this: there is no meaningful sense in which one can interpret the price at last trade as an average or representative belief among the trading population.


  1. Still reading your posts! Always worth reading.

    Plus, people may be betting at (subjectively) unfair odds in order to buy insurance against an unfavourable outcome. We all think there's a 50% risk of a bad harvest, but will pay $60 for a $100 bet on a bad harvest.

  2. Thanks Nick... It's true, I neglected this effect, as well as hedging demand, dealers who trade on both sides of the market, and manipulation.

    Manipulation is particularly interesting in these markets since there is positive feedback between beliefs about viability and success with fundraising. Dealt with this issue in an earlier post.

  3. this is an excellent illustration of the effect of buyers and sellers at the margin, thanks

  4. "This tells us nothing about the beliefs of traders who are not party to this transaction."

    One of the key elements of any prediction market is that it involves people who believe that they are aware of an inaccuracy in the current price AND are willing to put money behind their belief. Eliminating the tepid and unsure from the analysis is one of the purported benefits of prediction markets.

    The big problem in prediction markets isn't that they don't precisely reflect probabilities, because even though there is a disconnect between intuition and the details, that disconnect isn't all that serious (and doesn't differ materially from other gambler set predictions in tradition betting contexts).

    The big problem in political prediction markets is that there is a much strong incentive in prediction markets to act strategically by making bets that the trader believes to be money losing in order to influence the underlying bet. These feedback loops aren't present in traditional gambling contexts (the horses or dogs don't care what the odds are), but are present in real world securities markets (perception of success or impending doom may influence the actually success or doom of a public company). But, in the securities markets it is much more expensive to manipulate the market than it is in a comparatively low economic stakes political market.

  5. Andrew, I agree... feedback effects are interesting and important, though the point of this post was a bit different. Manipulation is most likely to arise with positive feedback effects (as in election markets) rather then negative feedback effects (as in markets for climate change). I have discussed these issues in a couple of earlier posts.