Max Bowie: Beating the Storms

The snowstorms in the US and UK that caused Christmas and New Year travel chaos remind us that, just like in the world of market data, a sudden storm can cause havoc for those whose business depends on the smooth and uninterrupted flow of people and information.

New York’s struggle with the snow is a perfect example of how things can go wrong: Without sufficient infrastructure, you can’t deal with extraordinary events. And when those events—such as a blizzard—occur, you don’t have the manpower to plow the streets, and you’re forever playing catch-up while cars and people attempting to go about their normal daily business get stuck in drifts and blocked thoroughfares. Even partly-clear roads hold up the process of getting back to normal. One lane is better than none, but it still takes twice as long for two streams of traffic to navigate.

New York mayor (and Bloomberg LP founder) Michael Bloomberg should understand these principles better than most, since they parallel capacity issues faced by the data industry on a daily basis: a sudden burst of market activity generates unexpected spikes in data volumes, and if vendors and data consumers don’t have sufficient headroom built into their infrastructure, that data finds itself trying to plow through pipes that just aren’t wide enough for multiple lanes of traffic. As a result—since latency is inextricably linked with capacity—everything slows down, and you run the risk of trading on data that is stale by the time it arrives.

Tools of the Trade
Network providers are using all the tools at their disposal to take small but increasingly important steps to mitigate such issues. For example, Spread Networks—which claims the lowest latency between New Jersey and Chicago by digging its own trench between the two cities—enables clients to “light” and run data from different trading desks on separate wavelengths in its dark fiber, to avoid one stream interfering with another.

But it’s not just about choosing the fastest network carrier: It’s also important to have sufficient capacity at both the distribution and consumption ends of the network, and to ensure you are managing your internal flow of data efficiently.

NYSE Euronext, for example, is tackling the issue on multiple fronts. The exchange recently signed up to Correlix’s RaceTeam service to monitor internal latencies of its US markets and make the results available to latency-sensitive clients, while its SIAC division has been able to reduce the latency of the Consolidated Tape Association’s feeds of quote and trade data via a series of initiatives that include capacity upgrades.

And even at the lower end of the scale, OTC Markets (formerly known as the Pink Sheets) has seen such an increase in data traffic as a result of growing order flow and automated quoting engines that—while its volumes are still small when compared to NYSE—it is moving to a new multicast-based datafeed to provide higher capacity and lower latency.

Pay-Off
Vendors are also seeing their efforts pay off. Chicago-based ticker plant vendor SpryWare recently announced that its ticker plant can handle 10 million messages per second (mps) on a single server, as a result of redesigning its software, utilizing the Linux operating system and 10 Gigabit Ethernet (GbE) network adapters from SolarFlare, while the latest generation of fellow low-latency data specialist and SolarFlare 10 GbE switch user Activ Financial’s hardware-accelerated MPU data appliance can handle 5 million mps per FPGA card, with the potential to run multiple cards in the same appliance.

By harnessing new technologies, the industry appears to have overcome the worst of its capacity issues—for now—allowing it to focus its efforts on reducing latency. But with US data rates approaching peaks of 4 million mps and exchange feeds continuing to hit new records, any respite on capacity issues may only be short-lived.