You probably already know the basics: that high-frequency trading (HFT) is a style of trading that involves placing, transacting and/or rescinding numerous orders over compressed time intervals measured in milliseconds (thousandths of seconds). Positions are rarely held more than a few seconds. The pace is too fast for any human, so trading decisions are made by proprietary computer algorithms called “algos”. To minimize communication time between the computers that host algos and the exchanges or dark pools on which they trade, the computers are typically co-located at those trading venues. During trading, algos can be monitored remotely by high-frequency traders, but trading is too rapid for any sort of real-time intervention. If a trader notices a problem—such as sudden losses or atypical market conditions—the usual response is to close all positions and suspend the algo while the situation is assessed.
High-frequency trading is an evolution of computer-based trading. The origins of computer-based trading can be traced to the 1969 launch of Instinet, a network for off-exchange equity trading. High-frequency trading emerged during the mid-1990s on a nimble competitor of Instinet called Island. Today, high-frequency trading so dominates the US stock market that it more-or-less is the stock market. About 50% of all US equity trading is by high-frequency traders. When investors such as mutual funds, pension plans or individuals trade, the party on the other side is most likely a high-frequency trader.
High-frequency trading is conducted by several large banks, but the real experts are specialized firms. These range from small proprietary firms called “prop shops” to highly-capitalized enterprises with names like Getco, Citadel and Tradebot. Their strategy is not speculation. They take little or no market risk. Those with successful algos tend to earn steady profits. In 2008, Tradebot’s founder, Dave Cummings, said they held positions for an average of 11 seconds and hadn’t had a losing day in four years.
New technology has propelled high-frequency trading, but so has fragmentation in the US equities markets, and flawed SEC regulations. These factors have facilitated a variety of exploitative trading practices.
The NYSE vs. NASDAQ rivalry of the late 1990s has been replaced by a free-for-all between 13 public exchanges and about 45 dark pools. This facilitates various forms of high-speed front-running, although practitioners prefer the name “latency arbitrage”. A typical scenario has an institutional investor breaking up a large stock purchase into small pieces and trying to execute these on different exchanges and dark pools. The orders may be sent simultaneously but, because the venues are geographically separated, they do not arrive simultaneously. On the first venue to receive one of the orders, high-frequency traders detect the order and immediately send orders to other venues to buy the same stock. It all happens in milliseconds but, because the high-frequency traders invest in extremely high speed connections, their orders arrive at the various venues before the institutional investor’s orders. It is front running, plain and simple.
You may be wondering how high-frequency traders would know that one trade hitting one venue signifies a large institutional trade—called a “whale” in the parlance—being split up and sent to multiple venues. The answer is “they don’t”. But this is where sophisticated algos come into play—and high-priced, high-speed data feeds the venues sell for use with those algos. The algos look at a host of clues to assess the source and purpose of trades. If a 1,000 share order for Facebook hits an exchange, the algos can generally tell if it is part of a whale or merely some retail investor’s speculative fling. They don’t have to be right all the time. They just have to be right most of the time—making a small profit on numerous trades, breaking even—called “scratching”—on many others, and maybe taking a small loss on a few. The law of large numbers takes over, and the high-frequency trader earns a predictable revenue stream.
Other high-frequency trading strategies might be called “high-speed arbitrage”. With so many venues for trading US stocks, opportunities for high-speed arbitrage abound. But they shouldn’t. In 2007, the SEC introduced Regulation National Market System (or Reg. NMS) that, among other things, was supposed to prevent arbitrage in the US equity markets. It requires that any order to buy or sell stock be routed to the exchange with the best price. Best-price information is communicated among exchanges and dark pools on a feed called the Securities Information Processor—or “SIP feed” for short. The problem is that the SIP feed is slow. Its prices are routinely stale—not minutes or hours stale, but milliseconds stale. Suppose an investor places a market order to sell a certain stock. The SIP feed indicates a best price of $40, but the price has actually risen to $40.01 on some exchange. A high-frequency trader sticks the investor with the stale SIP price of $40 while selling the same stock for $40.01, making a $.01 profit at the investor’s expense.
High-frequency traders distinguish between “smart money” and “dumb money”—although they avoid the distinction in public. Smart money is the high-frequency traders. Dumb money is everyone else—retail investors, pension funds, mutual funds, lumbering hedge funds, etc. Exchanges consider smart money to be their customers. Sure, dumb money pays the vast majority of rebates, but they generally do so on whatever exchange has the best SIP feed price. It is smart money that decides on which venues to trade, so they drive trading volumes to or away from specific venues. Also, it is smart money that pays for expensive co-location and data feeds. Venues scramble to win high-frequency traders’ business, and they offer a variety of inducements to win their business.
An early inducement was maker-taker payments. These were pioneered by Island but are now universal. The idea is to identify, in each trade, one party as a market maker and the other as a market taker. One is the “liquidity provider” and the other the “liquidity taker”. All this really means is that one party crosses the bid-ask spread to trade whereas the other does not. The exchange extracts a small fee from the market taker and pays a small fee to the market maker. Typical fees today are 30 basis points from the market taker and 20 basis points to the market maker. The exchange keeps the difference. Maker fees—also called rebates—are the primary source of income for most high-frequency traders.
Implementing high-frequency trading strategies, consistently and successfully, requires more than speed. This brings us to one of the most controversial concessions venues have provided HFTs: special order types. Suppose the bid-offer on a stock rises to $60.00 – $60.02. A market order arrives to buy $1,000 shares, and a high-frequency trader would like to place a limit order to sell at $60.02 and capture the rebate. The only problem is that orders are executed according to price-time priority. This means
- best-priced orders are executed first, and
- at any particular price, orders placed first are executed first.
In our example, if there is already a queue of limit orders to sell at $60.02, the high-frequency trader is placed at the back of the queue, missing the trade. There is no market or limit order the high-frequency trader can place to capture the rebate on this order.
Situations like this prompted high-frequency traders to ask venues to implement special order types—in addition to standard market and limit order types. Exchanges and dark pools, eager for high-frequency traders’ business, complied. Special order types have proliferated, with each venue offering its own flavors. They allow high-frequency traders to do things like:
- jump to the head of the queue,
- place orders that are hidden from other market participants until they execute,
- place orders at one price that later “slide” to another price to execute.
The purpose of most special order types is difficult to discern, unless you know the specific trading strategies they are intended to facilitate. Details can be arcane—incomprehensible to anyone not schooled in high-frequency trading.
Special order types give high-frequency traders advantages most people would consider unfair. Venues argue that, because special order types can be used by anyone, no one is favored. But this is only half the story. For a number of years, venues worked closely with high-frequency traders to design and implement special order types to their specifications. All the while, other investors were not informed.
Haim Bodek was a former trader for Hull Trading and Goldman Sachs who set up his own prop shop called Trading Machines. Given his background, Bodek might be considered an insider, but the venues treated him like dumb money. They never told him about special order types. Day after day, Trading Machines was picked off by other high-frequency traders. Bodek worked feverishly to figure out why, tweaking his algos, looking for bugs. After a year of struggles, Bodek was at an industry-related party where he had a conversation with a sales rep for one of the exchanges. Bodek had recently stopped trading on that exchange, and the sales rep wanted him back. He explained special order types to Bodek—how Bodek needed to use them to avoid getting picked off, and how he could use them to pick off others. The information came too late for Trading Machines, which shut down soon after. Bodek went on to become a vocal critic of special order types.
Special order types were a guarded secret until about 2012 when information about them increasingly leaked out. Today, exchanges publicly disclose details about their special order types. To use them effectively, you would need to buy each venue’s data feeds, co-locate at each venue, and develop sophisticated algos to deploy the special order types. In other words, you would have to become a high-frequency trader yourself.
The above discussion only scrapes the surface of controversial practices. Other topics include
- Flash trades are dumb money orders that a venue discloses—or “flashes”—to high-frequency traders milliseconds before they are disclosed publicly. This gives the high-frequency traders time to exploit the orders. The advance notice also gives high-frequency traders an advantage in gauging emerging order imbalances that could move stock prices.
- Dark pools became popular in the early 2000s. They were promoted as safe havens where institutional investors could trade away from high-frequency traders. Many are sponsored by banks or other firms that engage in high-frequency trading firms. Trading is non-transparent. Today, it is widely assumed that sponsors engage in front-running and/or actively promote their dark pools to high-frequency traders with the same sorts of inducements offered by exchanges.
- Some venues have reversed the maker-taker model and actually charge fees of makers and pay fees to takers. One can only assume high-frequency traders find on these venues exploitable opportunities that makes the fees worth paying.
- Many retail orders never make it to an exchange. This is because brokers such as Charles Schwab, E*Trade and TD Ameritrade send their retail order flow directly to high-frequency trading firms called intrnalizers. In exchange, internalizers pay the brokers fees reportedly running into hundreds of millions of dollars.
On May 6, 2010, at 2:42pm, the Dow Jones Industrial Average was down about 300 points for the day when it suddenly plunged an additional 600 in five minutes. High-frequency traders shut down their algos, exiting the market just when it needed liquidity. Stock prices gyrated, with those of prominent companies like Procter & Gamble and Accenture dropping as low as a penny or as high as $100,000. Twenty minutes later, by 3:07pm, it was over. The Dow had recovered most of the 600 point drop, and the markets stabilized.
This was the famous 2010 Flash Crash. The media storm that followed was, for many investors, the first time they heard of high-frequency trading. It put everyone on notice that the US stock market—fragmented, high-speed and driven by competing algos—was unstable in ways never experienced before.
There have been more flash crashes, mostly effecting individual stocks for a few seconds. But in May 2013, a flash crash hit the utilities sector, causing multiple stocks to drop 50% before recovering a few seconds later.
Discussions of high-frequency trading tend to focus on the US stock market, as that is where it originated. But high-frequency trading has spread globally to financial markets in Europe, Asia and the Americas. In Australia, Canada and the United Kingdom, authorities have even allowed their markets to be fragmented like US markets. Ccompeting exchanges linked by high-speed connections provide an infrastructure—with all its exploitable imperfections—for high-frequency trading to thrive. High-frequency trading has also taken over many futures and options markets around the world. Any exchange-traded market is vulnerable.
-  Dealbook: Speedy New Traders Make Waves Far From Wall Street, May 17, 2010.
-  Bowley, Graham, Lone $4.1 billion sale led to ‘flash crash’ in May, New York Times, Oct 1, 2010.
- Flash Boys: A Wall Street Revolt by Michael Lewis
- Dark Pools: The Rise of the Machine Traders and the Rigging of the U.S. Stock Market by Scott Patterson