Friday, June 04, 2010

The New Market Makers

In an earlier post I noted that according to the SEC's preliminary report on the flash crash of May 6, the vast majority of executions against stub quotes of five cents or less were short sales. This, together with the fact that there was also significant "aberrant behavior" on the upside (with Sotheby's trading for almost a hundred thousand dollars a share, for instance) led me to believe that most of this activity was caused by algorithmic trading strategies placing directional bets based on rapid responses to incoming market data.
Two strategies in particular -- momentum ignition and order anticipation -- were explicitly mentioned as potentially destabilizing forces in the SEC's January Concept Release on Equity Market Structure. The SEC invited comments on the release, and dozens of these have been posted to date. There is one in particular, submitted by R.T. Leuchtkafer about three weeks before the crash, that I think is especially informative and analytically compelling (h/t Dr. Duru).
Leuchtkafer traces the history of recent changes in market microstructure and examines the resulting implications for the timing of liquidity demand and supply. The comment is worth reading in full, but here are a few highlights. First, a brief history of the rise of the new market makers:
The last 15 years have seen a radical transformation of the equities markets from highly concentrated, semi-automated and intermediated marketplaces to highly distributed, fully automated and nominally disintermediated marketplaces. Along with or because of these changes, we have seen the rise of new classes of very profitable, aggressive, and technologically savvy participants previously unknown in the U.S. markets. When markets are in equilibrium these new participants increase available liquidity and tighten spreads. When markets face liquidity demands these new participants increase spreads and price volatility and savage investor confidence.
These participants can be more destructive to the interests of long-term investors than most have yet imagined... What is legal in today's market includes an exchange that sells real-time data to high frequency trading ("HFT") firms telling those firms exactly where hidden interest rests and in what direction. What is legal is the replacement of formal and regulated intermediaries with informal and unregulated intermediaries. What is legal is the proliferation of high-speed predatory momentum and order anticipation algorithms unrestrained by the anti-manipulation provisions of the Exchange Act. What is legal is a market structure that dismantled the investing public's order priorities and gave priority to speed and speed alone and then began charging for speed. What is legal is the widespread lack of supervision of the most aggressive and profitable groups of traders in American history. What is legal is exactly what the Release says it is worried about, "a substantial transfer of wealth from the individuals represented by institutional investors to proprietary firms..."
A HFT market making firm does not need to register as a market maker on any exchange. To the regulatory world it can present itself as just another retail customer and make markets with no more oversight than any other retail customer... Some HFT firms do register as market makers. By doing so they get access to more capital through higher leverage, they might get certain trading priority preferences depending on the market center, and they get certain regulatory preferences. They are usually required to post active quotes but quote quality is up to the market center itself to specify and some market centers have de minimis standards... Formal and informal market makers in the equities markets today have few or none of the responsibilities of the old dealers. That was the trade-off as markets transformed themselves during the last decade. In exchange for losing control of the order book and giving up a first look at customer order flow, firms shed responsibility for price continuity, quote size, meaningful quote continuity or quote depth. The result is that firms are free to trade as aggressively or passively as they like or to disappear from the market altogether.
The main problem, as Leuchtkafer sees it, is access to data feeds that make it possible to predict and profit from short term price movements if the information is processed and responded to with sufficient speed:
A classic short-term trading strategy is to sniff out an elephant and trade ahead of it. That is front-running if you are a fiduciary to the elephant but just good trading if you are not, or so we suppose.

Nasdaq sells a proprietary data feed called TotalView-ITCH that specifies exactly where hidden interest lies and whether it is buying or selling interest... Market making, statistical arbitrage, order anticipation, momentum and other kinds of HFT firms are an obvious customer base for this product... The complete details of limit order books can be used to predict short term stock price movements. An order book feed like TotalView-ITCH gives you much more information than just price and size such as you get with the consolidated quote. You get order and trade counts and order arrival rates, individual order volumes, and cancellation and replacement activity. You build models to predict whether individual orders contain hidden size. You reverse engineer the precise behavior and outputs of market center matching engines by submitting your own orders, and you vary order type and pore over the details you get back. If you take in order books from several market centers, you compare activity among them and build models around consolidated order book flows. With all of this raw and computed data and the capital to invest in technology, you can predict short term price movements very well, much better and faster than dealers could 10 years ago. Order book data feeds like TotalView-ITCH are the life's blood of the HFT industry because of it and the information advantages of the old dealer market structure are for sale to anyone.
But this raises a puzzling question: if the information advantages are truly "for sale to anyone" then free entry should drive down profitability until the return on investment is comparable to other uses of capital. In fact, entry has been substantial: "In 2000 as the HFT revolution started, dealer participation rates at the NYSE were approximately 25%. In 2008, the year NYSE specialists phased out, HFT participation rates in the equity markets overall were over 60%." How, then, can one explain the fact that by Leuchtkafer's own estimates, "HFT market making was 10 to 20 times more profitable in 2008 than traditional dealer firms were in 2000, before the HFT revolution?"
One intriguing possibility that Leuchtkafer does not consider is that entry generates increasing tail risk, so while ex ante expected profitability is reduced, this does not show up as declining realized profitability until a major market event (such as the flash crash) materializes. If this interpretation is correct, then some HFT firms must have made significant losses on May 6 that were reversed upon cancellation of trades. This implicit subsidy encourages excessive entry of destabilizing strategies.
The standard argument against increased regulation of the new market makers is that it would interfere with their ability to supply liquidity. Leuchtkafer argues, instead, that the strategies used by these firms cause them to demand liquidity at precisely those moments when liquidity is shortest supply:  
HFT firms claim they add liquidity and they do when it suits them... At any moment when they are in the market with nonmarketable orders by definition they add liquidity. When they spot opportunities or need to rebalance, they remove liquidity by pulling their quotes and fire off marketable orders and become liquidity demanders. With no restraint on their behavior they have a significant effect on prices and volatility. For the vast majority of firms whose models require them to be flat on the day their day-to-day contribution to liquidity is nothing because they buy as much as they sell. They add liquidity from moment to moment but only when they want to, and they cartwheel from being liquidity suppliers to liquidity demanders as their models rebalance. This sometimes rapid rebalancing sent volatility to unprecedented highs during the financial crisis and contributed to the chaos of the last two years. By definition this kind of trading causes volatility when markets are under stress.

Imagine a stock under stress from sellers such was the case in the fall of 2008. There is a sell imbalance unfolding over some period of time. Any HFT market making firm is being hit repeatedly and ends up long the stock and wants to readjust its position. The firm times its entrance into the market as an aggressive seller and then cancels its bid and starts selling its inventory, exacerbating the stock's decline. Unrestrained by affirmative responsibilities, the firm adjusts its risk model to rebalance as often as it wants and can easily dump its inventory into an already declining market. A HFT market making firm can easily demand as much or more liquidity throughout the day than it supplies. Crucially, its liquidity supply is generally spread over time during the trading day but its liquidity demands are highly concentrated to when its risk models tell it to rebalance. Unfortunately regulators do not know what these risk models are. So in exchange for the short-term liquidity HFT firms provide, and provide only when they are in equilibrium (however they define it), the public pays the price of the volatility they create and the illiquidity they cause while they rebalance. For these firms to say they add liquidity and beg to be left alone because of the good they do is chutzpah... 
The HFT firms insist they add liquidity and narrow effective spreads and they do at many instants in time during the day. They also take liquidity and widen realized spreads as they rebalance in narrow time slices and in the aggregate they can easily be as disruptive as supportive.
Paul Kedrosky made the same point immediately following the flash crash, and it is also mentioned in the SEC report. As part of the solution, Leuchtkafer proposes that certain trading strategies be prohibited outright:
The SEC should define both "momentum ignition" and "order anticipation" strategies as manipulation since they are both manipulative under any plain meaning of the Exchange Act. These strategies identify and take advantage of natural interest for a trader's own profit or stimulate artificial professional interest, also for the trader's own profit. They do so by bidding in front of (raising the price) or offering in front of (depressing the price) slower participants they believe are already in the market or that they can induce into the market. They both depend on causing short term price volatility either to prey on lagging natural interest or on induced professional interest. Any reasonable definition of "manipulation" in the equity markets should explicitly ban them by name.
I don't have any quarrel with this analysis and recommendation, but it's also useful to look at the problem from a somewhat broader perspective. Generally speaking, stability in financial markets depends on the extent to which trading is based on fundamental information about the securities that are changing hands. If too great a proportion of total volume is driven by strategies that try to extract information from market data, the data itself becomes less informative over time and severe disruptions can arise. Banning specific classes of algorithms is unlikely to provide a lasting solution to the problem unless the advantage is shifted decisively and persistently in favor of strategies that feed information to the market instead of extracting it from technical data.


Update (6/5). Also worth reading is a more recent comment (dated May 27) by Leuchtkafer on the Concept Release, dealing with some proposed policy responses to the flash crash:
It is strange to hear that "reform" should include a ban on stop loss orders, as if the United States equity markets are at risk because Mrs. Betty Johanssen of Red Lake, Minnesota posted a 300 share stop loss order in 3M, or that our equity markets are at risk because of a 200 share market order in Procter and Gamble. If this is where we end up, we will have failed. It will be an admission that since we won't or can't reform the shadow liquidity system, the only idea we have left is to ban even retail-sized unpriced liquidity demands...

The conventional window now also includes single stock circuit breakers. As I commented earlier... single stock circuit breakers may be useful in the same sense in which air bags are useful to Toyotas, but they don't cure sudden acceleration problems. The car still surges and crashes, but with air bags we can hope the occupants are a little more protected when it does. There is urgency to do something because political and economic exigencies demand action... circuit breakers are something we can do quickly as we continue to try to understand what happened on May 6.
Absolutely correct. Banning market orders or instituting automated single stock circuit breakers deals with symptoms rather than causes; a deeper analysis of structural defects in the current system is essential. 


  1. Why not put a randomized delay into the order processing so that speed is removed as a factor? Depending on the size of the window this would both reduce benefits of getting in front and make detecting trends more difficult.

  2. It's worth considering. David Murphy has a similar suggestion: "If every trade executed in the same, say, five second interval got the same price, instability would be greatly reduced, yet ordinary investors would not notice the effect."

  3. rajiv - i know this wasn't your quote, but the point is definitely not that "equity markets are at risk because Mrs. Betty Johanssen of Red Lake, Minnesota posted a 300 share stop loss order in 3M, or that our equity markets are at risk because of a 200 share market order in Procter and Gamble."

    it's that Mrs. Betty Johanssen HERSELF was at risk when she used a stop loss order (that became a market order) without understanding the consequences. Or that Joe Smith THOUGHT (incorrectly) that his 200 share sell order in PG guaranteed him a fill within a few cents.

    see, our equity markets were never at risk - priced blipped, and quickly mean reverted. it was little more than an opportunity for savvy traders to take advantage of ignorant traders.

    The problem is that individual investors don't understand the implications of the orders they are entering (like market orders, stop loss orders). I don't think anyone sympathizes with algos gone wild creating aberrant stock prices - bad algos should pay for their bad code and bad trading. However, there is general agreement that we want to protect retail investors from plugging themselves like that. I happen to think that the aberrant prints were much more likely the result of retail flow, though (I saw a piece of research recently that cited broker dealers saying that 70% of the canceled trades were retail stop loss orders and market orders).

    If such order types were banned (ie, no more market orders!), there would be no need to cancel trades, as all orders would have a user attached limit on them.

    i wrote more about eliminating market orders here:

  4. KD, thanks for your comment (I've been following your excellent reporting on this). I'm not at all opposed to banning market orders. It wouldn't hurt anyone (since one could specify arbitrarily low or high limit prices in any case). I just think that it would be a cosmetic reform, not dealing with important determinants of volatility. To me the flash crash is just an extreme version of a routine event - departures of prices from fundamentals under pressure of technical trading strategies. It was corrected quickly because the instability was downward. When there is upward instability it can remain not only uncorrected for long periods of time but also undetected.

    Based on the SEC report, I doubt very much that most trades against stub quotes were from retail investors. For example 90% of such trades between 2:50 and 2:55 were short sales. And at the high end most retail investors would not have the funds to cover multimillion dollar orders. I really think that the cancellation was hugely beneficial to some funds using directional algorithms.

    Regardless of whether market orders are permitted, cancellation of trades is a bad idea unless there really is a technical malfunction or human error involved. (I think this is also your position though I'm not sure.) It affects the composition of trading strategies in the market in a manner that is volatility increasing.

    Again - thanks for your comment, I have really enjoyed your posts on this.

  5. "It was corrected quickly because the instability was downward. When there is upward instability it can remain not only uncorrected for long periods of time but also undetected. "

    ahhhh.. interesting... and that's EXACTLY the way THEY want it! isn't it? no one complains on the way up.

    i wasn't aware that the SEC had concluded that most of the trades were short sales - I'd love to hear an explanation from them, then, as to why the trades were canceled. You're right - i absolutely don't think trades should be canceled, but i understand why they are canceled to protect ignorant retail investors. There is no reason to protect professionals who do their job poorly, unless there is a legitimate error (ie, key punch error). as you probably know, one reason many HFT guys said they turned their algos OFF during the crash was that they didn't want to have their trades subsequently canceled!

  6. rajiv check out the first paragraph on page 16 for the data point (anecdotal, though) about stop loss orders

  7. Thanks... but this is data from "retail-oriented brokerages". It's not clear how much of total volume came from these. Looking at all trades, 70% against stub quotes were short sales from 2:45-2:50, and 90% from 2:50-2:55. Details on pages 5 and 35 of the SEC report:

  8. Great post. One of this big problems I see with exchanges is an almost obsessive focus on volume -- from the point of view of the exchange it is about the only relevant statistic when judging the success of a particular product.

    At the CME, the S&P 500 E-Mini futures contract averages about 2 million contracts traded per day. Open interest, however, is typically less than 100,000 sides.

    10-15 years ago, when the S&P futures contract was still largely pit-traded, the typical daily volume of 100,000 contracts supported an open interest of around 500,000 sides. This is not to mention that the S&P pit-traded future is 5 times the notional volume of the E-mini. So, in notional terms, open interest back then was about 20-25 times larger than it is today.

    I thought exchanges are ultimately supposed to serve the interests of the hedgers and speculators who actually take long-term (i.e. longer than 1 day) positions in these markets.

    An exchange that has all volume and no open interest used to be known as a bucket shop. As we comfortable with a product that requires 20 in-and-out trades per day just to support one long-term position?

    To me, its really obvious where the problem lies. The exchanges, almost all of which are now public corporations, rely almost exclusively on share/contract volume for revenue. Their interests are now completely aligned with the firms and trading strategies that generate the maximum amount of volume. At times, this interest might align with the interest of the end users of these products, but, as you correctly point out, in the most critical moments it often does not.

  9. Rajiv - excellent post as always. Your conjecture that HFT firms' profits may only be fair compensation for taking on tail risk is one that I've suspected for a while and the most telling piece of evidence on this is their assertion that they turned their algos off when the market crashed.

    My knowledge of HFT is very thin but from my experience of less than high-frequency algorithmic strategies, such a volatility filter is a tell-tale sign that your underlying strategy is short tail risk/"synthetic short gamma". If your algorithm only works well in relatively "quiet" times then it makes sense to get out when volatility increases - since Mandelbrot, we've known that volatility "clusters". The problem with such an exit strategy, like all other algo strategies, is that its only worth pursuing if you're part of a minority of market participants trying to exit the market in this manner. To put it differently, if everyone's dancing close to the exit door it helps no one's cause - except of course it makes the market susceptible to a collapse as everyone tries to exit at the same time. And of course if you get bailed out in every "collapse", then it makes sense to follow exactly such a strategy that makes pennies with a high degree of certainty and only loses money in a state of the world where a bailout is almost certain.

    I've written a lot on how derivatives have made markets more complete and regulators' jobs harder - the same could be said for algo strategies. Any trader with some rudimentary coding knowledge can code up a momentum strategy with a volatility-based stop-loss mechanism and have it running off a virtual server for a minimal cost. Trying to ban every new innovation that comes into the markets is a game the regulator will never win. Ultimately, some form of market discipline has to be exerted with regulations as a redundant enforcing mechanism.

  10. Andrew, I'm sympathetic to your position on this but it's impractical. As MR points out in his comment it's almost impossible to regulate (or often even to detect) individual strategies. What you might be able to regulate is the flow of information but not what use is made of it.

    ds, I agree completely and would add one thing. In discussions of market microstructure it is usually assumed that more liquidity is always better. And this is true in the following limited sense: people will pay more for any given asset if it is more rather than less liquid, holding constant its price volatility. But if the manner in which liquidity is increased also gives rise to greater volatility, then there's a trade-off and it's no longer true that more liquidity is necessarily better. Liquidity comes at a price that is often neglected.

    MR, thanks, I totally agree. I would really like to see which firms had their accounts credited substantially when trades were canceled, and I suspect that there were a number of HFT firms among them. Those that shut down without being caught long or short got lucky, but it's inconceivable to me that they all managed to exit in time. Someone sold Accenture short for a penny per share, and someone bought Sotheby's for ten million dollars per round lot. And very little of this activity involved retail investors, as I have tried to argue based on the SEC report.

  11. rajiv - your last paragraph in the 8:30 comment (response to MR) is something i'd LOVE to hear about if you follow up. As i wrote previously, I think it was generally agreed that people understood canceling aberrant prints to protect retail investors - but if the SEC's own piece says it wasn't retail (and we can only assume retail wasn't shorting into stub quotes), then they have some 'splainin' to do.

    Maybe part of the problem is that it's not the SEC that cancels trades, right? it's the NASDAQ?

  12. Yes, it was the exchanges that canceled trades, following NASDAQ's lead. But the SEC should obtain and release information about beneficiaries, it would help us understand better what happened. Will follow up.

  13. "These strategies identify and take advantage of natural interest for a trader's own profit or stimulate artificial professional interest, also for the trader's own profit."

    The trader is a small beneficiary of this phenomenon, the major beneficiaries are those investors who trade at better prices due to the dissemination of knowable information by the trader. These strategies only work when the information uncovered is actually valued by the market. When valuable information is not present, the market will prevent anyone from making money consistently via asymmetric market impact of different trades (i.e., you can't buy 10 then sell 10 without ending up where you started, on average, minus costs). If the market allowed non-market-valuable asymmetries, it would be drained of capital very quickly indeed. The total amount of valuable information uncovered by HFT information analysis is quite small relative to the size of the market. Best execution firms have been complaining about HFT recently, because they want the market to stand still while they move large blocks of stock. Good work if you can get it, but very unfair to the other side of the trade, who never seems to get a mention. If you consider all investors, HFT is simply making prices slightly more accurate, given what's knowable. It's a business full of cutthroat competition, and not really subject to much gaming. If you want to create some real gaming of the market, just restrict the free flow of information.

  14. "If you consider all investors, HFT is simply making prices slightly more accurate, given what's knowable."

    This might be true in a world in which the semi-strong form of the efficient markets hypothesis holds, with prices tracking fundamentals quickly and closely. But I do not believe that we live in such a world. I view the flash crash not as a pathological event but as a very extreme version of an event that occurs routinely on a much smaller scale, hard to detect and correct, and causes prices to be much more volatile than fundamentals.

    If you accept this view (and you may not) then the question becomes this: do HFT firms increase price volatility? I believe that all technical trading strategies that make directional bets in response to incoming market data can destabilize markets if they form too great a share of total volume. This is the main point I have been trying to make here and in earlier posts.

  15. Professor Sethi,
    TOTALLY OFF TOPIC--- I have an important question to ask, and as I thought on it, it dawned on me I had never heard anyone else ask this question. I am posting it here in your most recent post in hopes it catches your eye as you are one of the best peoples to answer the question, I think. The question is this: What would happen if a "flash trade" (and a resulting drastic market downturn) happened simultaneously or just before market closing time (4pm eastern time)????

  16. Surely, there needs no ghost come from the grave to tell us this:

    "all technical trading strategies that make directional bets in response to incoming market data can destabilize markets if they form too great a share of total volume."