The future of algo trading: Using deep learning to more accurately predict equity market volumes

Trading algorithms are continuing to gain traction among the buy side, with respondents to a recent report indicating they are using automated tools more than ever. It is further validation of their increasing sophistication and wider acceptance, but the trend is highly concentrated. On average, the largest buy-side firms assign 33% of their order flow to their top provider, according to the report.

But that does not mean they are limiting their choices. Bloomberg Intelligence found these big firms added an average of two additional providers since 2018, bringing the roster of electronic brokers to nearly 12 each. Clearly, getting to the top and staying there is of paramount importance.

That’s creating a new arms race among providers to develop algorithms that use artificial intelligence to understand, or even anticipate, market conditions. Increasingly fragmented and complex market conditions help fuel the constant competition for creating the next innovation that will make electronic trading more agile and effective.

AI can be viewed as more of a marketing buzzword in the electronic trading world, covering a range of different types of algorithms and technologies. However, recent efforts, including those from Mizuho Group’s AI research team, are focused on applying cutting-edge science to the real-world conditions of the financial markets to develop algorithms that are smarter, more adaptable—and most importantly, that add value to trading operations.

Pushing deep learning into volume prediction

Central to achieving these goals is a type of AI called deep learning, used to create algorithms that help execute trades more efficiently and effectively. Deep learning, which is used in Mizuho’s Compass trading algorithm, has proven to be especially effective for financial applications, because it can process huge amounts of data—such as historical market conditions or stock price movements—to identify patterns and determine actions to take based on those conditions.

Of course, price is only one of the variables that traders consider when executing trades. In partnership with AI experts from Peking University, Dr. Ruhan Bao, who leads the Mizuho research team and helped to pioneer the use of deep learning for stock price prediction in Mizuho’s Compass trading algorithm, have been exploring the application of a deep learning technique called “clustering analysis” to handle the kind of real-time judgments that trading desks face every day.

Clustering allows for the examination of characteristics such as volume profile, volatility profile, and liquidity to categorize stocks into like groups. It then analyzes historical data to find the most effective way to trade those stock clusters based on the clients’ benchmark, including adjusting the appropriate amount to trade in the open auction, whether to passively post in the exchange, the exposure given to the dark pool at any given time/price point, and the level of intensity in trading blocks of liquidity that appear on the order book.

To build on the benefits of price prediction and clustering requires the development of a new method for volume prediction to help traders further manage changing market conditions. Think of the rise in passive investing, and what happens to stock volume when indexes are reconstituted. A traditional algorithm might suggest trades based on historic volume profiles, which won’t be ideal on an index event day. But an algorithm with machine learning-enabled volume prediction could study trading volume on all past index event days, detect the patterns, and predict the volume on the next index event day and recommend trades accordingly.

This AI breakthrough caught the attention of the sponsors of the 2021 International Joint Conference on Artificial Intelligence. Dr. Bao and members of the Peking University team were invited to present their paper on volume prediction as part of the conference’s main track in August.

Moving from the research department to the trading desk

While the AI-research community has recognized the importance of new volume prediction capabilities, the most important validation will come from traders who benefit from this new technology. Further incorporation of volume prediction in trading algorithms is expected to help further minimize trading costs. For example, since creating Compass in 2019, we’ve measured a 31.2% improvement in performance versus arrival price, which only inspired us to look for other ways to further improve the algorithm using AI.

This kind of continual research and innovation has helped electronic trading become entrenched in today’s financial markets. And with the kind of breakthroughs that we’ve already seen from AI, we expect that electronic trading will only become a more essential—and powerful—tool for financial institutions in the future.

By Sam Clapp, Mizuho Americas equities, and Don Hundley, Japan head of Mizuho equities electronic trading