Why Do Groupon Campaigns Damage Yelp Ratings?

One of the many benefits of visiting Microsoft Research this semester is that I get to attend some interesting talks by computer scientists working with social and economic data. One in particular this week turned out to be extremely topical. The paper was on "The Groupon Effect on Yelp Ratings" and it was presented by Giorgios Servas Zervas.

The starting point of the analysis was this: the Yelp ratings of businesses who launch Groupon campaigns suffer a sharp and immediate decline which recovers only gradually over time, with peristent effects lasting for well over a year. The following chart sums it up:

The trend line is a 30 day moving average, but re-initialized on the launch date (so the first few points after this date average just a few observations). There is a second sharp decline after about 180 days, as the coupons are about to expire. The chart also shows the volume of ratings, which surges after the launch date. Part of the surge is driven by raters who explicitly reference Groupon (the darker volume bars). But not all Groupon users identify as such in their reviews, and about half the increase in ratings volume comes from ratings that do not reference Groupon.

As is typical of computer scientists working with social data, the number of total observations is enormous. Almost 17,000 daily deals from over 5,000 businesses in 20 cities over a six month period are included, along with review histories for these businesses both during and prior to the observational window. In addition, the entire review histories of those who rated any of these businesses during the observational window were collected, expanding the set of reviews to over 7 million, and covering almost a million distinct businesses in all.

So what accounts for the damage inflicted on Yelp ratings by Groupon campaigns? The authors explore several hypotheses. Groupon users could be especially harsh reviewers regardless of whether or not they are rating a Groupon business. Businesses may be overwhelmed by the rise in demand, resulting in a decline in quality for all customers. The service provided to Groupon users may be worse than that provided to customers paying full price. Customer preferences may be poorly matched to businesses they frequent using Groupons. Or the ratings prior to the campaign may be artificially inflated by fake positive reviews, which get swamped by more authentic reviews after the campaign. All of these seem plausible and consistent with anecdotal evidence.

One hypothesis that is rejected quite decisively by the data is that Groupon users tend to be harsh reviewers in general. To address this, the authors looked at the review histories of those who identified Groupon use for the businesses in the observational window. Most of these prior reviews do not involve Groupon use, which allows for a direct test of the hypothesis that these raters were harsh in general. It turns out that they were not. Groupon users tend to write detailed and informative reviews that are more likely to be considered valuable, cool and funny by their peers. But they do not rate businesses without Groupon campaigns more harshly than other reviewers.

What about the hypothesis of businesses being overwhelmed by the rise in demand? Since only about half the surge in reviews comes from those who explicitly reference Groupon, the remaining ratings pool together non-Groupon customers with those who don't reveal Groupon use. This makes a decline in ratings by the latter group hard to interpret. John Langford (who was in the audience) noted that if the entire surge in reviews could be attributed to Groupon users, and if undeclared and declared users had the same ratings on average, then one could infer the effect of the campaign on the ratings of regular customers. This seems worth pursuing.

Anecdotal evidence on discriminatory treatment of customers paying discounted prices is plentiful (the authors mention the notorious FTD flowers case for instance). If mistreatment of coupon-carrying customers by a few bad apples were bringing down the ratings average, then a campaign should result in a more negatively skewed distribution of ratings relative to the pre-launch baseline. The authors look for this shift in skewness and find some evidence for it, but the effect is not large enough to account for the entire drop in the average rating.

To test the hypothesis that ratings prior to a campaign are artificially inflated by fake or purchased reviews, the authors look at the rate at which reviews by self-identified Groupon users are filtered, compared with the corresponding rate for reviews that make no mention of Groupon. (Yelp allows filtered reviews to be seen, though they are harder to access and are not used in the computation of ratings). Reviews referencing Groupon are filtered much less often, suggesting that they are more likely to be authentic. If Yelp's filtering algorithm is lenient enough to let a number of fake reviews through, then the post-campaign ratings will be not just more numerous but also more authentic and less glowing.

Finally, consider the possibility of a mismatch between the preferences of Groupon users and the businesses whose offers they accept. To look for evidence of this, the authors consider the extent to which reviews associated with Groupon use reveal experimentation on the part of the consumer. This is done by comparing the business category and location to the categories and locations in the reviewer's history. Experimentation is said to occur when the business category or zipcode differs from any in the reviewer's history. The data provide strong support for the hypothesis that individuals are much more likely to be experimenting in this sense when using a Groupon than when not. And such experimentation could plausibly lead to a greater incidence of disappointment.

This point deserves further elaboration. Even without experimentation on categories or locations, an individual who accepts a daily deal has a lower anticipated valuation for the product or service than someone who pays full price. Even if the expectations of both types of buyers are met, and each feels that they have gotten what they paid for, there will be differences in the ratings they assign. To take an extreme case, if the product were available for free, many buyers would emerge who consider the product essentially worthless, and would rate it accordingly even if their expectations are met.

There may be a lesson here for companies contemplating Groupon campaigns. Perhaps the Yelp rating would suffer less damage if the discount itself were not as steep. At present there is very little variation in discounts, which are mostly clustered around 50%. So there's no way to check whether smaller discounts actually result in better ratings relative to larger discounts. But it certainly seems worth exploring, at least for businesses that depend on strong ratings to thrive.

The Groupon strategy of prioritizing growth above earnings had been criticized on the grounds that there are few barriers to entry in this industry, and no network externalities that can protect an incumbent from competition. But if the link between campaigns and ratings can't be broken, there may be deeper problems with the business model than a change of leadership or strategy can solve.

A Combinatorial Prediction Market

I'm on leave this semester, visiting Microsoft Research's recently launched New York lab. It's a lively and stimulating place and there are a number of interesting projects underway. In this post I'd like to report on one of these, a prediction market developed by a team composed of David Pennock, David Rothschild, Miroslav Dudik, Jennifer Vaughan, and Sébastien Lahaie.

This market is very different from peer-to-peer real money prediction markets such as IEM or Intrade, in that individual participants take positions not against each other but against an algorithmic market maker that adjusts prices in response to orders placed. Furthermore, a broad and complex range of events are priced, orders of arbitrary size can be met, and consistency across prices is maintained by the immediate identification and exploitation of arbitrage opportunities.

The market for Oscar predictions is now live, and it's easy to participate. You can log in with a google account (or facebook or twitter) or create a new PredictWise account. You'll be credited with 1000 points which may be used to buy a range of contracts. These include contracts on simple events, such as "Lincoln to win Best Picture." But they also include events that reference multiple categories: you can bet on the event "Argo to win Best Picture and Daniel Day-Lewis to win Best Actor in a Leading Role," or "Zero Dark Thirty to win between 3 and 5 awards" for example.

All of these contracts are priced but the price is sensitive to order size. For small orders one can buy at the currently posted odds.  For instance, for "Lincoln to win Best Picture," current odds are 10.4%, so an expenditure of 0.104 units will return 1 unit if the event occurs:

But placing a larger order, say for 1.04 units, returns less than 10:

The functional relationship between the price and quantity vectors is deterministic and satisfies three conditions: (i) the purchase of a contract (or portfolio) raises its price smoothly, (ii) this happens in such a manner as to bound the maximum possible loss incurred by the market maker, no matter how large the order size, (iii) contracts that are obvious complements, such as "Lincoln to win Best Picture" and "Lincoln not to win Best Picture" have prices that sum to 1.

What makes this market interesting is that the algorithm ensures consistency in prices across linked contracts, quickly exploiting and eliminating any arbitrage opportunities than might arise. Some of these arbitrage conditions are not immediately transparent, as the following simplified example reveals.

Suppose that there were only two Oscar categories (Best Picture and Best Director) and consider the following seven events:

1. Lincoln to win Best Picture
2. Lincoln not to win Best Picture
3. Lincoln to win Best Director
4. Lincoln not to win Best Director
5. Lincoln to win 0 Oscars
6. Lincoln to win 1 Oscar
7. Lincoln to win 2 Oscars

Let p_i denote the price of contract i, where i = 1,...,7, where each contract pays out one dollar if the event in question occurs. Clearly we must have

p₁ + p₂ =  p₃ + p₄ = 1, 

otherwise there would be an arbitrage opportunity. Similarly, we must have

p₅ + p₆ + p₇ = 1. 

Price adjustments in response to orders are such that these equalities are continuously maintained. Somewhat less obviously, we must also have

p₁ + p₃ =  p₆ + 2p₇.

If this condition were violated, then one could construct a portfolio that guaranteed a positive profit no matter what the eventual outcome may be. To see this, suppose that prices were such that

p₁ + p₃ > p₆ + 2p₇.

In this case, the following portfolio would yield a risk-free profit: buy one unit each of contracts 2, 4, and 6, and two units of contract 7. This would cost

(1 - p₁) + (1 - p₃) +  p₆ + 2p₇  < 2.

The payoff from this portfolio would be exactly 2, no matter how things turn out. If Lincoln wins no Oscars then contracts 2 and 4 each pay out one unit, if it wins one Oscar then contract 6 pays out a unit, in addition to either contract 2 or contract 4, and if it wins two Oscars then each of the two units of contract 7 pays out.

In reality, there are more than two categories for which Lincoln has been nominated, and the arbitrage conditions are accordingly more complex. The point is that whenever trades occur that cause these conditions to be violated, the algorithm itself begins to execute additional trades that exploit the opportunity, shifting prices in such a manner as to restore parity. In peer-to-peer markets this activity is left to the participants themselves; trader developed algorithms have been in widespread use on Intrade for instance.

One problem with peer-to-peer markets is that only a few contracts can have significant liquidity, and complex combinations of events will therefore not be transparently and consistently priced. But the design described here allows for the consistent valuation of any combination of events, at the cost of subjecting the market maker to potential loss. The pricing function is designed to place a bound on this loss, but it cannot be avoided entirely because market participants have access to information that the market maker lacks.

Could this be a template for markets on compound events in the future? It certainly seems possible, if the internally generated arbitrage profits are large enough to compensate for the information disadvantage faced by the market maker. But at the moment this is a research initiative, focused on evaluating the effectiveness of the mechanism for aggregating distributed information. This goal is best served by broad participation and better data, so if you have a few minutes to spare before the Oscar winners are announced on Sunday, why not log in and place a few (hypothetical) bets?

Remembering Albert Hirschman

Albert Hirschman, among the greatest of social scientists, has died. He was truly one of a kind: always trespassing, relentlessly self-subversive, and never constrained by disciplinary boundaries.

Hirschman's life was as extraordinary as his work. Born in Berlin in 1915, he was educated in French and German. He would later gain fluency in Italian, then Spanish and English. He fled Berlin for Paris in 1933, and joined the French resistance in 1939. Fearful of being shot as a traitor by advancing German forces, he took on a new identity as a Frenchman, Albert Hermant. In 1941 he migrated to the United States, met and married Sarah Hirschman, joined the US Army, and soon found himself back in Europe as part of the war effort. After the end of hostilities he was involved in the development of the Marshall Plan, and subsequently spent four years in Bogotá where many of his ideas on economic development took shape. He and Sarah were married for more than seven decades; she died in January of this year.

Not only did Hirschman write several brilliant books in what was his fourth or fifth language, he also entertained himself with the invention of palindromes. Many of these were collected together in a book, Senile Lines by Dr. Awkward, which he presented to his daughter Katya. Forms of expression mattered to him as much as the ideas themselves. In opposition to Mancur Olson, he believed that collective action was an activity that came naturally to us humans, and was thrilled to find that one could invert a phrase in the declaration of independence to express this inclination as "the happiness of pursuit."

Hirschman's intellectual contributions were varied and many but the jewel in the crown is his masterpiece Exit, Voice and Loyalty. In this one slim volume, he managed to overturn conventional wisdom on one issue after another, and chart several new directions for research. The book is concerned with the mechanisms that can arrest and reverse declines in the performance of firms, organizations, and states. It was the interplay of two such mechanisms - desertion and articulation, or exit and voice - which Hirschman considered to be of central importance.

Exit, for instance through the departure of customers or employees or citizens in favor of a rival, can alert an organization to its own decline and set in motion corrective measures. But so can voice, or the articulation of discontent. Too rapid a rate of exit can undermine voice and result in organizational collapse instead of recovery. But a complete inability to exit can make voice futile, and poor performance can continue indefinitely.

Poorly functioning organizations prefer that an exit option be available to their most strident critics, so that they are left with less demanding customers or members or citizens. Hence a moderate amount of exit can result in the worst of all worlds, "an oppression of the weak by the incompetent and an exploitation of the poor by the lazy which is the more durable and stifling as it is both unambitious and escapable." Near-monopolies with exit options for the most severely discontented can therefore function more poorly than complete monopolies. It is not surprising that many dysfunctional states welcome the voluntary exile of their fiercest internal critics.

The propensity to exit is itself determined by the extent of loyalty to a firm or state. Loyalty slows down the rate of exit and can allow an organization time to recover from lapses in performance. But blind loyalty, which stifles voice even as it prevents exit, can allow poor performance to persist. It is in the interest of organizations to promote loyalty and raise the "price of exit", but the short term gains from doing so can lead to eventual collapse as both mechanisms for recuperation are weakened.

Among Hirschman's many targets were the Downsian model of political competition and the Median Voter Theorem. Since he considered collective action to be an expression of voice, readily adopted in response to dissatisfaction, there was no such thing as a "captive voter." Those on the fringes of a political party could not be taken for granted simply because they had no exit option: the inability to exit  just strengthened their inclination to exercise voice. This they would do with relish, driving parties away from the median voter, as political leaders trade-off the fear of exit by moderates against the threat of voice by extremists.

Albert Hirschman lived a long and eventful life and was a joyfully iconoclastic thinker. His books will be read by generations to come. But he will always remain something of an outsider in the profession; his ideas are just too broad and interdisciplinary to find neat expression in models and textbooks. He was an intellectual rebel throughout his life, and it is only fitting that he remain so in perpetuity. 

Risk and Reward in High Frequency Trading

A paper on the profitability of high frequency traders has been attracting a fair amount of media attention lately. Among the authors is Andrei Kirilenko of the CFTC, whose earlier study of the flash crash used similar data and methods to illuminate the ecology of trading strategies in the S&P 500 E-mini futures market. While the earlier work examined transaction level data for four days in May 2010, the present study looks at the entire month of August 2010. Some of the new findings are startling, but need to be interpreted with greater care than is taken in the paper.

High frequency traders are characterized by large volume, short holding periods, and limited overnight and intraday directional exposure:

For each day there are three categories a potential trader must satisfy to be considered a HFT: (1) Trade more than 10,000 contracts; (2) have an end-of-day inventory position of no more than 2% of the total contracts the firm traded that day; (3) have a maximum variation in inventory scaled by total contracts traded of less than 15%. A firm must meet all three criteria on a given day to be considered engaging in HFT for that day. Furthermore, to be labeled an HFT firm for the purposes of this study, a firm must be labeled as engaging in HFT activity in at least 50% of the days it trades and must trade at least 50% of possible trading days. 

Of more than 30,000 accounts in the data, only 31 fit this description. But these firms dominate the market, accounting for 47% of total trading volume and appearing on one or both sides of almost 75% of traded contracts. And they do this with minimal directional exposure: average intraday inventory amounts to just 2% of trading volume, and the overnight inventory of the median HFT firm is precisely zero.

This small set of firms is then further subdivided into categories based on the extent to which they are providers of liquidity. For any given trade, the liquidity taker is the firm that initiates the transaction, by submitting an order that is marketable against one that is resting in the order book. The counterparty to the trade (who previously submitted the resting limit order) is the liquidity provider. Based on this criterion, the authors partition the set of high frequency traders into three subcategories: aggressive, mixed, and passive:

To be considered an Aggressive HFT, a firm must... initiate at least 40% of the trades it enters into, and must do so for at least 50% of the trading days in which it is active. To be considered a Passive HFT a firm must initiate fewer than 20% of the trades it enters into, and must do so for at least 50% of the trading days during which it is active. Those HFTs that meet neither... definition are labeled as Mixed HFTs. There are 10 Aggressive, 11 Mixed, and 10 Passive HFTs.

This heterogeneity among high frequency traders conflicts with the common claim that such firms are generally net providers of liquidity. In fact, the authors find that "some HFTs are almost 100% liquidity takers, and these firms trade the most and are the most profitable."

Given the richness of their data, the authors are able to compute profitability, risk-exposure, and measures of risk-adjusted performance for all firms. Gross profits are significant on average but show considerable variability across firms and over time. The average HFT makes over $46,000 a day; aggressive firms make more than twice this amount. The standard deviation of profits is five times the mean, and the authors find that "there are a number of trader-days in which they lose money... several HFTs even lose over a million dollars in a single day."

Despite the volatility in daily profits, the risk-adjusted performance of high frequency traders is found to be spectacular:

HFTs earn above-average gross rates of return for the amount of risk they take. This is true overall and for each type... Overall, the average annualized Sharpe ratio for an HFT is 9.2. Among the subcategories, Aggressive HFTs (8.46) exhibit the lowest risk-return tradeoff, while Passive HFTs do slightly better (8.56) and Mixed HFTs achieve the best performance (10.46)... The distribution is wide, with an inter-quartile range of 2.23 to 13.89 for all HFTs. Nonetheless, even the low end of HFT risk-adjusted performance is seven times higher than the Sharpe ratio of the S&P 500 (0.31).

These are interesting findings, but there is a serious problem with this interpretation of risk-adjusted performance. The authors are observing only a partial portfolio for each firm, and cannot therefore determine the firm's overall risk exposure. It is extremely likely that these firms are trading simultaneously in many markets, in which case their exposure to risk in one market may be amplified or offset by their exposures elsewhere. The Sharpe ratio is meaningful only when applied to a firm's entire portfolio, not to any of its individual components. For instance, it is possible to construct a low risk portfolio with a high Sharpe ratio that is composed of several high risk components, each of which has a low Sharpe ratio.

To take an extreme example, if aggressive firms are attempting to exploit arbitrage opportunities between the futures price and the spot price of a fund that tracks the index, then the authors would have significantly overestimated the firm's risk exposure by looking only at its position in the futures market. Over short intervals, such a strategy would result in losses in one market, offset and exceeded by gains in another. Within each market the firm would appear to have significant risk exposure, even while its aggregate exposure was minimal. Over longer periods, net gains will be more evenly distributed across markets, so the profitability of the strategy can be revealed by looking at just one market. But doing so would provide a very misleading picture of the firms risk exposure, since day-to-day variations in profitability within a single market can be substantial.

The problem is compounded by the fact that there are likely to by systematic differences across firms in the degree to which they are trading in other markets. I suspect that the most aggressive firms are in fact trading across multiple markets in a manner that lowers rather than amplifies their exposure in the market under study. Under such circumstances, the claim that aggressive firms "exhibit the lowest risk-return tradeoff" is without firm foundation.

Despite these problems of interpretation, the paper is extremely valuable because it provides a framework for thinking about the aggregate costs and benefits of high frequency trading. Since contracts in this market are in zero net supply, any profits accruing to one set of traders must come at the expense of others:

From whom do these profits come? In addition to HFTs, we divide the remaining universe of traders in the E-mini market into four categories of traders: Fundamental traders (likely institutional), Non-HFT Market Makers, Small traders (likely retail), and Opportunistic traders... HFTs earn most of their profits from Opportunistic traders, but also earn profits from Fundamental traders, Small traders, and Non-HFT Market Makers. Small traders in particular suffer the highest loss to HFTs on a per contract basis.

Within the class of high frequency traders is another hierarchy: mixed firms lose to aggressive ones, and passive firms lose to both of the other types.

The operational costs incurred by such firms include payments for data feeds, computer systems, co-located servers, exchange fees, and highly specialized personnel. Most of these costs do not scale up in proportion to trading volume. Since the least active firms must have positive net profitability in order to survive, the net returns of the most aggressive traders must therefore be substantial.

In thinking about the aggregate costs and benefits of all this activity, it's worth bringing to mind Bogle's law:

It is the iron law of the markets, the undefiable rules of arithmetic: Gross return in the market, less the costs of financial intermediation, equals the net return actually delivered to market participants.

The costs to other market participants of high frequency trading correspond roughly to the gross profitability of this small set of firms. What about the benefits? The two most commonly cited are price discovery and liquidity provision. It appears that the net effect on liquidity of the most aggressive traders is negative even under routine market conditions. Furthermore, even normally passive firms can become liquidity takers under stressed conditions when liquidity is most needed but in vanishing supply.

As far as price discovery is concerned, high frequency trading is based on a strategy of information extraction from market data. This can speed up the response to changes in fundamental information, and maintain price consistency across related assets. But the heavy lifting as far as price discovery is concerned is done by those who feed information to the market about the earnings potential of publicly traded companies. This kind of research cannot (yet) be done algorithmically.

A great deal of trading activity in financial markets is privately profitable but wasteful in the aggregate, since it involves a shuffling of net returns with no discernible effect on production or economic growth. Jack Hirschleifer made this point way back in 1971, when the financial sector was a fraction of its current size. James Tobin reiterated these concerns a decade or so later. David Glasner, who was fortunate enough to have studied with Hirshlefier, has recently described our predicament thus:

Our current overblown financial sector is largely built on people hunting, scrounging, doing whatever they possibly can, to obtain any scrap of useful information — useful, that is for anticipating a price movement that can be traded on. But the net value to society from all the resources expended on that feverish, obsessive, compulsive, all-consuming search for information is close to zero (not exactly zero, but close to zero), because the gains from obtaining slightly better information are mainly obtained at some other trader’s expense. There is a net gain to society from faster adjustment of prices to their equilibrium levels, and there is a gain from the increased market liquidity resulting from increased trading generated by the acquisition of new information. But those gains are second-order compared to gains that merely reflect someone else’s losses. That’s why there is clearly overinvestment — perhaps massive overinvestment — in the mad quest for information.

To this I would add the following: too great a proliferation of information extracting strategies is not only wasteful in the aggregate, it can also result in market instability. Any change in incentives that substantially lengthens holding periods and shifts the composition of trading strategies towards those that transmit rather than extract information could therefore be both stabilizing and growth enhancing. 

Death of a Prediction Market

A couple of days ago Intrade announced that it was closing its doors to US residents in response to "legal and regulatory pressures." American traders are required to close out their positions by December 23rd, and withdraw all remaining funds by the 31st. Liquidity has dried up and spreads have widened considerably since the announcement. There have even been sharp price movements in some markets with no significant news, reflecting a skewed geographic distribution of beliefs regarding the likelihood of certain events.

The company will survive, maybe even thrive, as it adds new contracts on sporting events to cater to it's customers in Europe and elsewhere. But the contracts that made it famous - the US election markets - will dwindle and perhaps even disappear. Even a cursory glance at the Intrade forum reveals the importance of its US customers to these markets. Individuals from all corners of the country with views spanning the ideological spectrum, and detailed knowledge of their own political subcultures, will no longer be able to participate. There will be a rebirth at some point, perhaps launched by a new entrant with regulatory approval, but for the moment there is a vacuum in a once vibrant corner of the political landscape.

The closure was precipitated by a CFTC suit alleging that the company "solicited and permitted" US persons to buy and sell commodity options without being a registered exchange, in violation of US law. But it appears that hostility to prediction markets among regulators runs deeper than that, since an attempt by Nadex to register and offer binary options contracts on political events was previously denied on the grounds that "the contracts involve gaming and are contrary to the public interest."

The CFTC did not specify why exactly such markets are contrary to the public interest, and it's worth asking what the basis for such a position might be.

I can think of two reasons, neither of which are particularly compelling in this context. First, all traders have to post margin equal to their worst-case loss, even though in the aggregate the payouts from all bets will net to zero. This means that cash is tied up as collateral to support speculative bets, when it could be put to more productive uses such as the financing of investment. This is a capital diversion effect. Second, even though the exchange claims to keep this margin in segregated accounts, separate from company funds, there is always the possibility that its deposits are not fully insured and could be lost if the Irish banking system were to collapse. These losses would ultimately be incurred by traders, who would then have very limited legal recourse.

These arguments are not without merit. But if one really wanted to restrain the diversion of capital to support speculative positions, Intrade is hardly the place to start. Vastly greater amounts of collateral are tied up in support of speculation using interest rate and currency swaps, credit derivatives, options, and futures contracts. It is true that such contracts can also be used to reduce risk exposures, but so can prediction markets. Furthermore, the volume of derivatives trading has far exceeded levels needed to accommodate hedging demands for at least a decade. Sheila Bair recently described synthetic CDOs and naked CDSs as "a game of fantasy football" with unbounded stakes. In comparison with the scale of betting in licensed exchanges and over-the-counter swaps, Intrade's capital diversion effect is truly negligible.

The second argument, concerning the segregation and safety of funds, is more relevant. Even if the exchange maintains a strict separation of company funds from posted margin despite the absence of regulatory oversight, there's always the possibility that its deposits in the Irish banking system are not fully secure. Sophisticated traders are well aware of this risk, which could be substantially mitigated (though clearly not eliminated entirely) by licensing and regulation.

In judging the wisdom of the CFTC action, it's also worth considering the benefits that prediction markets provide. Attempts at manipulation notwithstanding, it's hard to imagine a major election in the US without the prognostications of pundits and pollsters being measured against the markets. They have become part of the fabric of social interaction and conversation around political events.

But from my perspective, the primary benefit of prediction markets has been pedagogical. I've used them frequently in my financial economics course to illustrate basic concepts such as expected return, risk, skewness, margin, short sales, trading algorithms, and arbitrage. Intrade has been generous with its data, allowing public access to order books, charts and spreadsheets, and this information has found its way over the years into slides, problem sets, and exams. All of this could have been done using other sources and methods, but the canonical prediction market contract - a binary option on a visible and familiar public event - is particularly well suited for these purposes.

The first time I wrote about prediction markets on this blog was back in August 2003. Intrade didn't exist at the time but its precursor, Tradesports, was up and running, and the Iowa Electronic Markets had already been active for over a decade. Over the nine years since that early post, I've used data from prediction markets to discuss arbitrage, overreaction, manipulation, self-fulfilling prophecies, algorithmic trading, and the interpretation of prices and order books. Many of these posts have been about broader issues that also arise in more economically significant markets, but can be seen with great clarity in the Intrade laboratory.

It seems to me that the energies of regulators would be better directed elsewhere, at real and significant threats to financial stability, instead of being targeted at a small scale exchange which has become culturally significant and serves an educational purpose. The CFTC action just reinforces the perception that financial sector enforcement in the United States is a random, arbitrary process and that regulators keep on missing the wood for the trees.

---

Update: NPR's Yuki Noguchi follows up with Justin Wolfers, Thomas Bell, Laurence Lau, and Jason Ruspini here; definitely worth a listen. Brad Plumer's overview of the key issues is also worth a look.

Curtailing Intellectual Monopoly

I never thought I'd see an RSC policy brief referring to mash-ups and mix-tapes, but I was clearly mistaken.

The document deals in an unusually frank manner with the dismal state of US copyright law. Perhaps too frankly: it was quickly disavowed and taken down on the grounds that publication had occurred "without adequate review." Copies continue to circulate, of course (the link above is to one I posted on Scribd). Although lightly peppered with ideological boilerplate, the brief makes a number of timely and sensible points and is worth reading in full.

Aside from extolling the virtues of "a robust culture of DJ’s and remixing" free from the stranglehold of copyright protection, the authors of the report make the following claims. First, the purpose of copyright law, according to the constitution, is to "promote the progress of science and useful arts" and not to "compensate the creator of the content." Copyright law should therefore be evaluated by the degree to which it facilitates innovation and creative expression. Second, unlike conventional tort law, statutory damages for infringement are "vastly disproportionate from the actual damage to the copyright producer." For instance, Limewire was sued for $75 trillion, "more money than the entire music recording industry has made since Edison’s invention of the phonograph in 1877." Third, the duration of coverage has been expanding, seemingly without limit. In 1790 a 14 year term could be renewed once if the the author remained alive; current coverage is for the life of the author plus 70 years. This stifles rather than promotes creative activity.

The economists Michele Boldrin and David Levine have been making these points for years. In their book Against Intellectual Monopoly (reviewed here), they point out that the pace of innovation in industries without patent and copyright protection has historically been extremely rapid. Software could not be patented before 1981, nor financial securities prior to 1998, yet both industries witnessed innovation at a blistering pace. The fashion industry remains largely untouched by intellectual property law, yet new designs keep appearing and enriching their creators. Innovative techniques in professional sports continue to be developed, despite the fact that successful ones are quickly copied and disseminated.

In 19th century publishing, British authors had limited protection in the United States but managed to secure lucrative deals with publishers, allowing the latter to saturate the market at low prices before new entrants could gain a foothold. More recently, commercial publishers have turned a profit selling millions of copies of unprotected government documents. For instance, the 9/11 Commission Report was published by both Norton and Macmillan in 2004, and a third version by Cosimo is now available.

Copyright restrictions for scientific papers are especially illogical, since faculty authors benefit from the widest possible dissemination and citation of their work. Furthermore, in the case of journals owned by commercial publishers, copyright is typically transferred by the author to the publisher. Neither the content creators nor the uncompensated peer-reviewers who evaluate manuscripts for publication benefit from protection in such cases. Fortunately, thanks to the emergence of new high-quality open-source journals sponsored by academic societies, things are starting to change.

It's not clear why the policy brief was taken down, or what motivated it in the first place. Henry Farrell, while agreeing with the positions taken in the report, argues that damage to an industry that has historically supported Democrats may be a factor. In contrast, Jordan Bloom and Alex Tabarrok both believe that pressure on Republicans from the entertainment industry led to the brief being withdrawn. They can't all be right as far as I can see. But less interesting than the motivation for the report is its content, and the long overdue debate on patents and copyrights that could finally be stirred in its wake. 

Prediction Market Manipulation: A Case Study

The experience of watching election returns come in has become vastly more social and interactive than it was just a decade ago. Television broadcasts still provide the core pubic information around which expectations are formed, but blogs and twitter feeds are sources of customized private information that can have significant effects on the evolution of beliefs. And prediction markets aggregate this private information and channel it back into the public sphere.

All of this activity has an impact not only on our beliefs and moods, but also on our behavior. In particular, beliefs that one's candidate of choice has lost can affect turnout. It has been argued, for instance, that early projections of victory for Reagan in 1980 depressed Democratic turnout in California, and that Republican turnout in Florida was similarly affected in 2000 when the state was called for Gore while voting in the panhandle was still underway. For this reason, early exit poll data is kept tightly under wraps these days, and states are called for one candidate or another only after polls have closed.

This effect of beliefs on behavior implies that a candidate facing long odds of victory has an incentive to inflate these odds and project confidence in public statements, lest the demoralizing effects of pessimism cause the likelihood of victory to decline even further. Traditionally this would be done by partisans on television sketching out implausible scenarios and interpretations of the incoming data to boost their supporters. But with the increasing visibility of prediction markets, this strategy is much less effective. If a collapse in the price of a contract on Intrade reveals that a candidate is doing much worse than expected, no amount of cheap talk on television can do much to change the narrative.

Given this, the incentives to interfere with what the markets are saying becomes quite powerful. Even though trading volume has risen dramatically in prediction markets over recent years, the amount of money required to have a sustained price impact for a few hours remains quite small, especially in comparison with the vast sums now spent on advertising.

In general, I believe that observers are too quick to allege manipulation when they see unusual price movements in such markets. As I noted in an earlier post, a spike in the price of the Romney contract a few days ago was probably driven by naive traders over-reacting to rumors of a game-changing announcement by Donald Trump, rather than by any systematic attempt at price manipulation. My reasons for thinking so were based on the fact that frenzied purchases of a single contract (while ignoring complementary contracts) are terribly ineffective if the goal is to have a sustained impact on prices. If one really wants to manipulate a market, it has to be done by placing large orders that serve as price ceilings and floors, and to do this across complementary contracts in a consistent way.

As it happens, this is exactly what someone tried to do yesterday. At around 3:30 pm, I noticed that the order book for both Obama and Romney contracts on Intrade had become unusually asymmetric, with a large block of buy orders for Romney in the 28-30 range, and a corresponding block of sell orders for Obama in the 70-72 range. Here's the Romney order book:

And here's the book for Obama:

Since the exchange requires traders to post 100% margin (to cover their worst case loss and eliminate counterparty risk), the funds required to place these orders was about $240,000 in total. A non-trivial amount, but probably less than the cost of a thirty-second commercial during primetime.

Could this not have been just a big bet, placed by someone optimistic about Romney's chances? I don't think so, for two reasons. First, if one wanted to bet on Romney rather than Obama, much better odds were available elsewhere, for instance on Betfair. More importantly, one would not want to leave such large orders standing at a time when new information was emerging rapidly; the risk of having the orders met by someone with superior information would be too great. Yet these orders stood for hours, and effectively placed a floor on the Romney price and a ceiling on the price for Obama.

Meanwhile odds in other markets were shifting rapidly. Nate Silver noticed the widening disparity and was puzzled by it, arguing that differences across markets should "evaporate on Election Day itself, when the voting is over and there is little seeming benefit from affecting the news media coverage." Much as I admire Nate, I think that he was mistaken here. It is precisely on election day that market manipulation makes most sense, since one only needs to affect media coverage for a few hours until all relevant polls have closed. Voting was still ongoing in Colorado, and keeping Romney viable there was the only hope of stitching together a victory. Florida, Virginia and Ohio were all close at the time and none had been called for Obama. A loss in Colorado would have made these three states irrelevant and a Romney victory virtually impossible.

Given this interpretation, I felt that the floor would collapse once the Colorado polls closed at 9pm Eastern Time, and this is precisely what happened:

Once the floor gave way, the price fell to single digits in a matter of minutes and never recovered.

It turned out, of course, that none of this was to matter: Virginia, Ohio, and (probably) Florida have all fallen to Obama. But all were close, and the possibility of a different outcome could not have been ruled out at the time. The odds were low, and a realistic projection of these odds would have made them even lower. Such is the positive feedback loop between beliefs and outcomes in politics. Under the circumstances, the loss of a few hundred thousand dollars to keep alive the prospect of a Romney victory probably seemed like a good investment to someone.

Should one be concerned about such attempts at manipulation? I don't think so. They muddy the waters a bit but are transparent enough to be spotted quickly and reacted to. My initial post was retweeted within minutes by Justin Wolfers to 24,000 followers, and by Chris Hayes to 160,000 shortly thereafter. Attempts at manipulating beliefs are nothing new in presidential politics, it's just the methods that have changed. And as long as one is aware of the possibility of such manipulation, it is relatively easy to spot and counter. The same social media that transmits misinformation also allows for the broadcast of countervailing narratives. In the end the fog clears and reality asserts itself. Or so one hopes. 

--- 

Update: The following chart shows the Obama price breaking through the ceiling just before the polls closed in Colorado:

It's the extraordinary stability of the price before 8:45pm, which was sustained over several hours, that is suggestive of manipulation.