Sunday, April 06, 2014

Superfluous Financial Intermediation

I'm only about halfway through Flash Boys but have already come across a couple of striking examples of what might charitably be called superfluous financial intermediation.

This is the practice of inserting oneself between a buyer and a seller of an asset, when both parties have already communicated to the market a willingness to trade at a mutually acceptable price. If the intermediary were simply absent from the marketplace, a trade would occur between the parties virtually instantaneously at a single price that is acceptable to both. Instead, both parties trade against the intermediary, at different prices. The intermediary captures the spread at the expense of the parties who wish to transact, adds nothing to liquidity in the market for the asset, and doubles the notional volume of trade.

The first example may be summarized as follows. A hundred thousand shares in a company have been offered for sale at a specified price across multiple exchanges. A single buyer wishes to purchase the whole lot and is willing to pay the asked price. He places a single buy order to this effect. The order first reaches BATS, where it is partially filled for ten thousand shares; it is then routed to the other exchanges for completion. An intermediary, having seen the original buy order on arrival at BATS, places orders to buy the remaining ninety thousand shares on the other exchanges. This latter order travels faster and trades first, so the original buyer receives only partial fulfillment. The intermediary immediately posts offers to sell ninety thousand shares at a slightly higher price, which the original buyer is likely to accept. All this in a matter of milliseconds.

The intermediary here is serving no useful economic function. Volume is significantly higher than it otherwise would have been, but there has been no increase in market liquidity. Had there been no intermediary present, the buyer and sellers would have transacted without any discernible delay, at a price that would have been better for the buyer and no worse for the sellers. Furthermore, an order is allowed to trade ahead of one that made its first contact with the market at an earlier point in time.

The second example involves interactions between a dark pool and the public markets. Suppose that the highest bid price for a stock in the public exchanges is $100.00, and the lowest ask is $100.10. An individual submits a bid for a thousand shares at $100.05 to a dark pool, where it remains invisible and awaits a matching order. Shortly thereafter, a sell order for a thousand shares at $100.01 is placed at a public exchange. These orders are compatible and should trade against each other at a single price. Instead, both trade against an intermediary, which buys at the lower price, sells at the higher price, and captures the spread.

As in the first example, the intermediary is not providing any benefit to either transacting party, and is not adding liquidity to the market for the asset. Volume is doubled but no economic purpose is served. Transactions that were about to occur anyway are preempted by a fraction of a second, and a net transfer of resources from investors to intermediaries is the only lasting consequence.

Michael Lewis has focused on practices such as these because their social wastefulness and fundamental unfairness is so transparent. But it's important to recognize that most of the strategies implemented by high frequency trading firms may not be quite so easy to classify or condemn. For instance, how is one to evaluate trading based on short term price forecasts based on genuinely public information? I have tried to argue in earlier posts that the proliferation of such information extracting strategies can give rise to greater price volatility. Furthermore, an arms race among intermediaries willing to sink significant resources into securing the slightest of speed advantages must ultimately be paid for by investors. This is an immediate consequence of what I like to call Bogle's Law:
It is the iron law of the markets, the undefiable rules of arithmetic: Gross return in the market, less the costs of financial intermediation, equals the net return actually delivered to market participants.
I hope that the minor factual errors in Flash Boys won't detract from the book's main message, or derail the important and overdue debate that it has predictably stirred. By focusing on the most egregious practices Lewis has already picked the low-hanging fruit. What remains to be figured out is how typical such practices really are. Taking full account of the range of strategies used by high frequency traders, to what extent are our asset markets characterized by superfluous financial intermediation?


Update (4/11). It took me a while to get through it but I’ve now finished the book. It’s well worth reading. Although the public discussion of Flash Boys has been largely focused on high frequency trading, the two most damning claims in the book concern broker-dealers and the SEC.

Lewis provides evidence to suggest that some broker-dealers direct trades to their own dark pools at the expense of their customers. Brokers with less than a ten percent market share in equities trading mysteriously manage to execute more than half of their customers’ orders in their own dark pools rather than in the wider market. This is peculiar because for any given order, the likelihood that the best matching bid or offer is found in a broker’s internal dark pool should roughly match the broker’s market share in equities trading. Instead, a small portion of the order is traded at external venues in a manner that allows the information content of the order to leak out. This results in a price response on other exchanges, allowing the internal dark pool to then provide the best match.

There’s also an account of a meeting between Brad Katsuyama, the book’s main protagonist, and the SEC’s Division of Trading and Markets that is just jaw-dropping. Katsuyama had discovered the reason why his large orders were only partially filled even though there seemed to be enough offers available across all exchanges for complete fulfillment (the first example above). In order to prevent their orders from being front-run after their first contact with the market, Katsuyama and his team developed a simple but ingenious defense. They split each order into components that matched the offers available at the various exchanges, and then submitted the components at carefully calibrated intervals (separated by microseconds) so that they would arrive at their respective exchanges simultaneously. The program written to accomplish this was subsequently called Thor. Katsuyama met with the SEC to explain how Thor worked, and was astonished to find that some of the younger staffers thought that the program, designed to protect fundamental traders from being front-run, was unfair to the high-frequency outfits whose strategies were being rendered ineffective.

This account, if accurate, reveals a truly astonishing failure within the SEC to understand the agency’s primary mandate. If this is the state of our regulatory infrastructure then there really is little hope for reform. 


  1. I see why this is wasteful, but why do you call it unfair? Presumably everyone involved in these markets is attempting to gain at the expense of other participants. Why is one successful trading strategy morally superior to any other?

  2. Well, it's a form of algorithmic front-running, trading based on information contained in orders that were placed (and in fact reached the market) before your own, but have not yet been fully executed. Conceptually not that different from the actions of the CME clerk who was recently convicted of commodities fraud. Most would consider this unfair to the party being front-run.

  3. I guess I agree that it would be a good idea to improve the systme to address the issues you describe in your post, but I have a big problem with the way Michael Lewis's book is being marketed.

    1. The public is being told that the market is "rigged" and they should be afraid to submit an order. They are not being told that this stuff has almost no relevance at all to long term investors or small investors, or index fund investors, and that in fact there has never been a better time to be a small investor in terms of transaction costs. This last point seems beyond debate. How many thousands of people will be scared away from equities because Michael Lewis told them the market is "rigged?"

    2. The public and congress do not understand this issue one bit. They have no idea what liquidity provision even means. Nothing good can come over having a national debate over market micro-structure. This is a case of some Wall Street big boys (the HFT firms) against some other Wall Street big boys (the informed order flow). Let them fight it out with the SEC and fix the kind of problems you mentioned.

    3. You say this problem is "important?" Well, maybe, but I can think of a hundred much more important problems that I'd rather see the public getting outraged about, not this one that they don't even begin to understand. I mean, even if you want to stick to Wall Street, why not take on high fees in people's 401(k)s? This is an order of magnitude bigger problem for people, as Felix Salmon pointed out.

  4. Ed, thanks for your comment. I have some sympathy for your position - terms like "rigged" are designed to inflame rather than inform. But there's a lot of rhetoric from the HFT side that is designed to obfuscate and confuse. You can't just look at spreads and conclude that the costs of financial intermediation have declined on aggregate, or that such firms are net providers of liquidity.

    Retail investors whose entire order is filled at the best bid/ask do benefit from tighter spreads. But the same investors also have retirement savings in mutual funds, and depend on the returns earned by foundations, university endowments, pension plans, municipalities, etc. And through this channel they can be hurt.

    Holding index funds does not solve the problem. At any point in time funds are flowing into and out of such funds and they are forced to buy or sell large blocks of securities. If their order flow can be anticipated their effective spreads will be higher. The spread we see is just the marginal spread, applicable for the first few shares to trade. The average spread depends on order size and the degree to which intermediaries can infer this.

    Are there more important problems? Of course there are. But this is one that needs to be brought to light, since the SEC has done essentially nothing about it. As noted by Lew Burton here, a central limit order book would solve many of these problems.

    In any case, I'm grateful to Lewis for trying to explain some of these issues in plain language, even if the language is occasionally provocative.

  5. Hey Professor Sethi,

    I like the terminology!

    Some corrections though

    Brokers with less than a ten percent market share in equities trading mysteriously manage to execute more than half of their customers’ orders in their own dark pools rather than in the wider market. This is peculiar because for any given order, the likelihood that the best matching bid or offer is found in a broker’s internal dark pool should roughly match the broker’s market share in equities trading.

    This is not a case of having the best price versus not. This is about queue priority. Getting to choose fill your customer internally means you are first in line. This is very valuable to hf trading systems. If you're first in line you will naturally get more fills. It has nothing to do with market share size.

    Further, I doubt the complete accuracy of the account. Programs like Thor have existed for many years. This isn't a shock to anyone.

  6. ZHD, I can't vouch for the accuracy of the accounts, but if you read the chapter you'll see that he's not talking about orders that trade internally right away, but that trade externally in small increments first, until there is a price response. Presumably this is because they don't have the best bid or offer in their pool and have to comply with Reg NMS. According to the book they give up the chance to take offers available at lower price outside the pool, which is clearly counter to client interest.

    Regarding Thor, it's an obvious idea, so I'm sure that it was independently discovered many times. But that doesn't make the SEC reaction any less shocking (at least to me).

    But like I said, I have no way of verifying the accuracy of the accounts. Thanks for your comment.

  7. Usually once someone makes a price improvement, the rest of the market piles onto that price as well. This is known through a market making rule of thumb: be the first in line, but not the only one.

    If they're routing orders away because the main feed shows a price improvement, that often means there will be a large queue joining behind that price. The broker can choose to now fill its customers at that price instead of routing them away. This allows the broker to jump the FIFO.

    In fast moving markets it is obviously more difficult to say where the best price is at any given time. There is always a probability that the price has moved away from the market in the most recent update, regardless of the feed. Orders and cancels are constantly in flight.

    Regarding the "SEC reaction," we don't really know who was there do we? They are a fairly large organization with a lot of people who go to a lot of meetings. If that were the official position it'd be quite different.

  8. All the data in Chapter 7 comes from IEX, itself a dark pool. The broker routes multiple small buy orders to IEX, finds a match at the mid-market price, which is a penny below the best publicly displayed offer. He could trade in volume at this price but chooses not to, filling the bulk of the customer's order internally. The question is, at what price?

    Katsuyama argues that the broker is not acting in the interests of the customer, presumably because the order is filled at a higher average price than was initially available. I'd like to know if he is misinterpreting his data, but it seems like a valid point to me.

    Regarding the SEC, it seems that in the meeting there was a generational divide, with younger staffers arguing that Thor was unfair. It's just an anecdote, and it may not even be accurate, but I just don't see the logic in this position.

  9. That's a fairly outrageous claim based on that data. It's argument from ignorance. A complete logical fallacy.

    IEX is essentially crying about not getting volume.

    Like I said in my previous scenario, if something starts trading on the offer, the rest of the market begins to price improve.

    Scott Locklin covers the general incompetence pretty well:

  10. Interesting summary, Rajiv.

    I also found enlightening Mark Buchanan's take on HFT and Lewis's book. He thinks that the most important danger with HFT is the instability that it creates.

    Also related is Felix Salmon's emphasis on the risk created by complexity:

    "…By far the biggest risk posed by the HFT industry, for instance, is the risk of the kind of event we saw during the flash crash, only much, much worse. The stock market is an insanely complex system, which can fail in unpredictable and catastrophic ways; the HFT industry only serves to make it much more brittle and perilous than it already was. But in Lewis’ book-length treatment of HFT, he barely mentions this risk: I found just one en passant mention of “the instability introduced into the system when its primary goal is no longer stability but speed,” on Page 265, but no elaboration of that idea."

    1. Well, there is also some discussion of Zoran's view that talks about "Normal Accidents" (page 203) which is a classical (and controversial) book by Charles Perrow from 1984, and Richard Cook's paper (page 198). As a person who is quit familiar with the subject of failure of complex systems, I would say that
      A. I agree that this is a very important aspect of HFT that is severely understudied
      B. Lewis should be actually given credit for paying some attention to the subject

  11. Charlie, there's an interesting discussion in Chapter 4 of intraday volatility and the price impact of different order routers. But you're right, there's not much attention to volatility in general. I have argued before (for instance here and here) that order anticipation and momentum based strategies can be stabilizing when rare but destabilizing when sufficiently frequent. HFT is just the latest in a long line of anticipatory strategies, which have been around as long as markets have existed.

  12. Per J. W. Mason question up top: