Max Bowie: Do Crisis Closures Make Redundancy Redundant?
Aside from its tragic and destructive impact on the inhabitants of parts of New York and New Jersey, October’s Hurricane Sandy proved two key things for the financial markets: First, the business-continuity plans of datacenters where most marketplaces and trading firms host their trading engines and servers proved effective in keeping everyone’s systems running; and second, in spite of this, the markets still closed for two days because of old-fashioned physical issues that these centers were, in part, designed to eliminate.
The New York Stock Exchange initially planned to open for electronic trading as usual, anticipating that its physical floor would be closed following the storm—as indeed it was, re-opening after two days on generator power, though the floor was never flooded, contrary to some news reports—but changed its mind after discussions with other US exchanges and the Securities and Exchange Commission (SEC). Ultimately, all US equities markets remained closed for a full two days—an occurrence not experienced in decades. And while at time of writing, thousands of New Yorkers remain without power, the US financial markets could have largely continued unhindered.
According to Plan
This is testament to the integrity, security and policies of the datacenter facilities constructed by vendors such as Equinix, Telx and Savvis in recent years—veritable digital fortresses designed to withstand all manner of attacks, whether they be caused by humans or nature. Though new facilities tend to be built to take power from two power grids, most in the New York and New Jersey area experienced outages and had to run on generators during and after the storm. To ensure this could happen smoothly, datacenter operators tested generators beforehand, ensured fuel supplies, arranged hotel rooms for staff near facilities, even bringing cots and ready-to-eat meals into their datacenters, in some cases. This aspect of the markets worked fine, although it does raise the questions of whether so many critical markets should operate from the same datacenters—an outage at Equinix’s NY4 facility, which many markets have found offers the best connectivity to client firms and other venues, for example, could throw multiple markets into disarray—and what impact the changes in latency introduced by switching over to a backup location would have on firms’ trading strategies and routing decisions.
What failed, though, is that in spite of having increasingly electronic, automated and virtual markets, these still rely to a large extent on a legacy physical infrastructure overseeing them—an infrastructure of human administrators and operations staff, whose ability to do their job is limited by, say, flooding at their place of work, subways not running, or trees blocking roads. To a large extent, it seems that the market closures were a result of infrastructure supporting the people who manage the markets not being as resilient as the infrastructures that actually run them. Or perhaps, to look at this from another point of view, people didn’t trust the systems designed to run the markets to do their job without human oversight.
Limits
Some of these limitations cannot be overcome: Storms will always cause havoc. But other factors can be: Administrative and supervisory functions can be automated to the same extent as the markets they oversee, and can be performed as remotely and virtually as the infrastructures of the trading firms they monitor—even possibly by putting the logic to perform these functions on satellites to eliminate the risk of running these from a geographical location subject to disruption.
To a large extent, it seems that the market closures were a result of infrastructure supporting the people who manage the markets not being as resilient as the infrastructures that actually run the markets.
But the more systematic issue at stake has little to do with the systems that support the markets, and more to do with the system that governs them: Under what circumstances should all markets be forced to close, if the systems of some—or all—are able to keep running? After all, aren’t technology and reliability supposed to be differentiators? And what does it say about our confidence in these systems—or the system—if we feel the need to pull the plug when we can’t be in the room?
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: https://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@waterstechnology.com
More on Emerging Technologies
FIS rolls out monthly security patches as Mythos exposes hidden flaws
Anthropic’s Mythos is surfacing vulnerabilities that were once ignored or hidden. FIS is more than doubling its cybersecurity investments and urging clients to upgrade to the latest product versions.
Propellant and Tradeweb expand partnership; NYSE data on Pyth, and more
The Waters Cooler: A recap of the major tech and data news from the past week in the capital markets.
Data vendors are vagueposting about MCP prices
Exchanges and data providers are charging for AI access to their data. Nobody will say how much.
Goldman’s shift on AI highlights the gap between prototype and production
James Harrison expects more institutions will take AI tools away from their traders and bankers—and for good reasons.
Americans voted against datacenters. Datacenters may still win.
The Waters Wrap: The US midterms could spell trouble for datacenters but only if Dems put the screws to Big Tech, too, Reb writes.
AI for data quality control catches on at institutions
Data execs at banks and asset managers say they are working on projects using AI to improve data-quality workflows.
A Q&A with BofA’s Global Markets CIO on AI and problem-solving
Following the bank’s Global Markets Innovation Summit, Duncan McInnes says AI is helping his teams “reimagine” established processes and ask where technology can make them simpler and faster.
Compute is leaving land. Recoverability isn’t following.
The limits of land may push datacenters to the ocean or even outer space, writes Vishal Gupta. But these options introduce new challenges.