On Prices, Narratives, and Market Efficiency

The fourth anniversary of the Lehman bankruptcy has been selected as the release date for a collection of essays edited by Diane Coyle with the provocative title: What's the Use of Economics? The timing is impeccable and the question legitimate.

The book collects together some very thoughtful responses by Andrew Haldane, John Kay, Wendy Carlin, Alan Kirman, Andrew Lo, Roger Farmer, and a host of other luminaries (the publishers were kind enough to send me an advance copy). There's enough material there for several posts but I'd like to start with the contribution by John Kay.

This one, as it happens, has been published before; I discussed Mike Woodford's reaction to it in a previous post. But reading it again I realized that it contains a perspective on market efficiency and price discovery that is concise, penetrating and worthy of some elaboration. Kay doesn't just provide a critique of the efficient markets hypothesis; he sketches out an alternative approach based on the idea of prices as the "product of a clash between competing narratives" that can form the basis of an entire research agenda.

He begins with a question famously posed by the Queen of England during a visit to the London School of Economics: Why had economists failed to predict the financial crisis? Robert Lucas pointed out in response that the inability to predict a financial crisis was in fact a prediction of economic theory. This is as pure a distillation of the efficient markets hypothesis is one is likely to find, and Kay uses it to evaluate the hypothesis itself:

Lucas’s assertion that ‘no one could have predicted it’ contains an important, though partial, insight. There can be no objective basis for a prediction of the kind ‘Lehman Bros will go into liquidation on September 15’, because if there were, people would act on that expectation and, most likely, Lehman would go into liquidation straight away. The economic world, far more than the physical world, is influenced by our beliefs about it. 

Such thinking leads, as Lucas explains, directly to the efficient market hypothesis – available knowledge is already incorporated in the price of securities. And there is a substantial amount of truth in this – the growth prospects of Apple and Google, the problems of Greece and the Eurozone, are all reflected in the prices of shares, bonds and currencies. The efficient market hypothesis is an illuminating idea, but it is not “Reality As It Is In Itself”. Information is reflected in prices, but not necessarily accurately, or completely. There are wide differences in understanding and belief, and different perceptions of a future that can be at best dimly perceived. 

In his Economist response, Lucas acknowledges that ‘exceptions and anomalies’ to the efficient market hypothesis have been discovered, ‘but for the purposes of macroeconomic analyses and forecasts they are too small to matter’. But how could anyone know, in advance not just of this crisis but also of any future crisis, that exceptions and anomalies to the efficient market hypothesis are ‘too small to matter’?

The literature on anomalies is not, in fact, concerned with macroeconomic analyses and forecasts. It is rather narrowly focused on predictability in asset prices and the possibility of constructing portfolios that can consistently beat the market on a risk-adjusted basis. And indeed, such anomalies are often found to be quite trivial, especially when one considers the costs of implementing the implied strategies. The inability of actively managed funds to beat the market on average, after accounting for costs and adjusting for risk, is often cited as providing empirical support for market efficiency. But Kay believes that these findings have not been properly interpreted:

What Lucas means when he asserts that deviations are ‘too small to matter’ is that attempts to construct general models of deviations from the efficient market hypothesis – by specifying mechanical trading rules or by writing equations to identify bubbles in asset prices – have not met with much success. But this is to miss the point: the expert billiard player plays a nearly perfect game, but it is the imperfections of play between experts that determine the result. There is a – trivial – sense in which the deviations from efficient markets are too small to matter – and a more important sense in which these deviations are the principal thing that matters. 

The claim that most profit opportunities in business or in securities markets have been taken is justified.  But it is the search for the profit opportunities that have not been taken that drives business forward, the belief that profit opportunities that have not been arbitraged away still exist that explains why there is so much trade in securities. Far from being ‘too small to matter’, these deviations from efficient market assumptions, not necessarily large, are the dynamic of the capitalist economy. 

Such anomalies are idiosyncratic and cannot, by their very nature, be derived as logical deductions from an axiomatic system. The distinguishing characteristic of Henry Ford or Steve Jobs, Warren Buffett or George Soros, is that their behaviour cannot be predicted from any prespecified model. If the behaviour of these individuals could be predicted in this way, they would not have been either innovative or rich. But the consequences are plainly not ‘too small to matter’. 

The preposterous claim that deviations from market efficiency were not only irrelevant to the recent crisis but could never be relevant is the product of an environment in which deduction has driven out induction and ideology has taken over from observation. The belief that models are not just useful tools but also are capable of yielding comprehensive and universal descriptions of the world has blinded its proponents to realities that have been staring them in the face. That blindness was an element in our present crisis, and conditions our still ineffectual responses. 

Fair enough, but how should one proceed? Kay suggests the adoption of more "eclectic analysis... not just deductive logic but also an understanding of processes of belief formation, anthropology, psychology and organisational behaviour, and meticulous observation of what people, businesses, and governments actually do."

I have no quarrel with this prescription, but I'd also like to make a case for more creative and versatile deductive logic. One of the key modeling hypotheses in the economics of information is the so-called Harsanyi doctrine (or common prior assumption), which stipulates that all differences in beliefs ought to be modeled as if they arise from differences in information. This hypothesis implies that individuals can only disagree if such disagreement is not itself common knowledge: they cannot agree to disagree. It is not hard to see that such a hypothesis could not possibly allow for pure speculation on asset price movements, and hence cannot account for the large volume of trade in financial markets. In fact, it implies that order books in many markets would be empty, since a posted price would only be met by someone with superior information.

The point is that over-reliance on deductive logic is not the only problem as far as financial modeling is concerned; the core assumptions to which deductive logic has been applied are themselves too restrictive. To my mind, the most interesting part of Kay's essay suggests how one might improve on this:

You can learn a great deal about deviations from the efficient market hypothesis, and the role they played in the recent financial crisis, from journalistic descriptions by people like Michael Lewis and Greg Zuckerman, who describe the activities of some individuals who did predict it. The large volume of such material that has appeared suggests many avenues of understanding that might be explored. You could develop models in which some trading agents have incentives aligned with those of the investors who finance them and others do not. You might describe how prices are the product of a clash between competing narratives about the world. You might appreciate the natural human reactions that made it difficult to hold short positions when they returned losses quarter after quarter.

There is definitely ongoing work in economics that explores many of these directions, some of which I have surveyed in previous posts. But the idea of prices as the product of a clash between competing narratives about the world reminded me of a paper by Harrison and Kreps, which was one of the earliest models in finance to shed the common prior assumption.

For anyone interested in developing models of heterogeneous beliefs in which trading occurs naturally over time, the Harrison-Kreps paper is the perfect place to start. They illustrate their model with an example that is easy to follow: a single asset provides title to a stream of dividend payments that may be either high or low, and investors disagree about the likelihood of transitions from high to low states and vice versa. This means that investors who value the asset most in one state differ from those who value it most in the other. Trading occurs as the asset is transferred across investors in the two different belief classes each time a transition to a different state occurs. The authors show that the price in both states is higher than it would be if investors were forced to hold the asset forever: there is a speculative premium that arises from the knowledge that someone else will, in due course and mistakenly in your opinion, value the asset more than you do. The contrast with the efficient markets hypothesis is striking and clear:

The basic tenet of fundamentalism, which goes back at least to J. B. Williams (1938), is that a stock has an intrinsic value related to the dividends it will pay, since a stock is a share in some enterprise and dividends represent the income that the enterprise gains for its owners. In one sense, we think that our analysis is consistent with the fundamentalist spirit, tempered by a subjectivist view of probability. Beginning with the view that stock prices are created by investors, and recognizing that investors may form different opinions even when they have the same substantive information, we contend that there can be no objective intrinsic value for the stock. Instead, we propose that the relevant notion of intrinsic value is obtained through market aggregation of diverse investor assessments. There are fundamentalist overtones in this position, since it is the market aggregation of investor attitudes and beliefs about future dividends with which we start. Under our assumptions, however, the aggregation process eventually yields prices with some curious characteristics. In particular, investors attach a higher value to ownership of the stock than they do to ownership of the dividend stream that it generates, which is not an immediately palatable conclusion from a fundamentalist point of view.

The idea that prices are "obtained through market aggregation of diverse investor assessments" is not too far from Kay's more rhetorically powerful claim that they are "the product of a clash between competing narratives".  What Harrison and Kreps do not consider is how diverse investor assessments change over time, since beliefs about transition probabilities are exogenously given in their analysis. But Kay's formulation suggests how progress on this front might be made. Beliefs change as some narratives gather influence relative to others, either though active persuasion (talking one's book for instance) or through differentials in profits accruing to those with different worldviews. While Kay is surely correct that a rich understanding of this process requires more than deductive reasoning, it is also true that deductive reasoning has not yet been pushed to its limits in facilitating our understanding of market dynamics.

Building a Better Dow

The following post, written jointly with Debraj Ray, is based on our recent note proposing a change in the method for computing the Dow.

---

With a market capitalization approaching 600 billion, Apple is currently the largest publicly traded company in the world. The previous title-holder, Exxon Mobil, now stands far behind at 400 billion. But Apple is not a component of the Dow Jones Industrial Average. Nor is Google, with a higher valuation than all but a handful of firms in the index. Meanwhile, firms with less than a tenth of Apple's market capitalization, including Alcoa and Hewlett-Packard, continue to be included.

The exclusion of firms like Apple and Google would appear to undermine the stated purpose of the index, which is "to provide a clear, straightforward view of the stock market and, by extension, the U.S. economy." But there are good reasons for such seemingly arbitrary omissions. The Dow is a price-weighted index, and the average price of its thirty components is currently around $58. Both Apple and Google have share prices in excess of $600, and their inclusion would cause day-to-day changes in the index to be driven largely by the behavior of these two securities. For instance, their combined weight in the Dow would be about 43% if they were to replace Alcoa and Travelers, which are the two current components with the lowest valuations. Furthermore, the index would become considerably more volatile even if the included stocks were individually no more volatile than those they replace. As John Presbo, chairman of the index oversight committee, has observed, such heavy dependence of the index on one or two stocks would "hamper its ability to accurately reflect the broader market."

Indeed, price-weighting is decidedly an odd methodology. IBM has a smaller market capitalization than Microsoft, but a substantially higher share price. Under current conditions, a 1% change in the price of IBM has an effect on the index that is almost seven times as great as a 1% change in the price of Microsoft. In fact, IBM's weight in the index is above 11%, although its valuation is less than 6% of the total among Dow components.

This issue does not arise with value-weighted indexes such as the S&P 500. But as Prestbo and others have pointed out, the Dow provides an uninterrupted picture of stock market movements dating back to 1896. An abrupt switch to value weighting would introduce a methodological discontinuity that would "essentially obliterate this history." Attention has therefore been focused on the desirability of a stock split, which would reduce Apple's share price to a level that could be accommodated by the questionable methodology of the Dow.

But an abrupt switch to a value weighting or the flawed artifice of a stock split are not the only available alternatives. In a recent paper we propose a modification that largely preserves the historical integrity of the Dow time series, while allowing for the inclusion of securities regardless of their market price. Our modified index also leads to a smooth and gradual transition, as incumbent stocks are replaced, to a fully value-weighted index in the long run.

The proposed index is composed of two subindices, one price-weighted to respect the internal structure of the Dow, and the other value-weighted to apply to new entrants. The index has two parameters, both of which are adjusted whenever a substitution is made. One of these maintains continuity in the value of the index, while the other ensures that the two subindices are weighted in proportion to their respective market capitalizations. Stock splits require a change in parameters (as in the case of the current Dow divisor) but only if the split occurs for a firm in the price-weighted subindex.

Once all incumbent firms are replaced, the result will be a fully value-weighted index. In practice this could take several decades, as some incumbent firms are likely to be remain components far into the future. But firms in the price-weighted component of the index that happen to have weights roughly commensurate with their market capitalization can be transferred with no loss of continuity to the value-weighted component. This procedure, which we call bridging, can accelerate the transition to a value-weighted index with minimal short-term disruption. Currently Coca Cola and Disney are prime candidates for bridging.

Under our proposed index, Apple would enter with a weight of less than 13% if it were to replace Alcoa. This is scarcely more than the weight currently associated with IBM, a substantially smaller company. Adding Google (in place of HP or Travelers) would further lower the weight of Apple since the total market capitalization of Dow components would rise. This is a relatively modest change that, we believe, would simultaneously serve the desirable goals of methodological continuity and market representativeness.