Opening Cross: Nightmares and Data Dreamscapes

Market data is scarier than any trick or treat. So either learn to laugh at the frights, or be ready for things to go bump in the night.


Halloween is nearly upon us once again, along with trick-or-treaters, candlelit pumpkins, ghoulish house decorations and garish costumes for children and adults alike. And by the time you’ve gotten over the nightmares from that, the US election will be beating down our door like a young, axe-wielding Jack Nicholson. But even these pale in comparison to the nightmares that wake data managers from their slumbers, leaving them gibbering wrecks in the wee hours.

For example, imagine you’re a company like Estimize, which has just invested a not insignificant amount of money in joining the gameification trend by creating an app that crowd-sources investor opinions on where a stock will rank compared to others, only to find that by charging users to participate and awarding cash prizes to the winners, the contests constitute securities-based swaps, according to US regulator the Securities and Exchange Commission. As a result, despite taking steps to avoid falling under that designation, Estimize was forced to pay a $50,000 fine and remove the payments and incentives from its Forcerank app. Fortunately, officials say, participants continue to rank stocks in the app, and the predictive dataset being created from their recommendations continues to deliver alpha. Certainly, that’s the silver lining in this situation that may yet yield more results over the long term.

And while most of you aren’t Estimize, many readers will know the agony and frustration of sinking investment into a project, only to have it scaled back or even scuppered at the last minute.

In recent years, some of these projects have related to low-latency connectivity, where a firm initially believes it can gain a trading advantage only to realize that the cost of building and maintaining such an infrastructure is greater than the advantage gained, while the firm seeks to balance “good-enough” latency an different itself with smart strategies instead.

In this low-latency world, datacenters became the new central marketplaces, with trading venues and trading firms co-locating side by side. These setups became critical because even if your strategy doesn’t depend on the fastest inbound data, you still need to ensure the ability to execute as fast as possible so the market doesn’t move before you can respond. So whether you consider your business strictly low latency you still need to be in the same place as others who do.

Now consider the nightmare scenario—not necessarily one of those sudden waking episodes, but rather a long, drawn-out cold sweat—of moving your infrastructure between datacenters without impacting your business, such as the London Metal Exchange’s decision to switch its primary datacenter from a CenturyLink facility in Slough, outside London, to Interxion’s datacenter in the heart of the city. Coordinating a move and seamless switchover is no small task, but in this case the burden is offset by latency gains resulting from the move. And while you may question the importance of latency to a commodity futures exchange, consider the stories we’ve written in recent weeks about software vendors releasing high-performance feed handlers for foreign exchange and fixed income markets. Latency is becoming important across all asset classes. And if you thought handling high volumes of low-latency equities and options data was hard, wait until other asset classes get in on the act.

And even buy-side firms seeking to offload most of the burden of handling data can’t escape its clutches, since—according to a panel at last week’s Buy-Side Technology North American Summit—they remain ultimately responsible for data quality and all the business impacts associated with it. 

So the moral of the story is, when dealing with data, be prepared to encounter something terrifying at every turn. Steel yourself against it, if you choose. Or even better, welcome the fright like a horror movie fan, and consider each one a challenge to be faced and dispatched with a silver bullet and a sly wisecrack.  

  • LinkedIn  
  • Save this article
  • Print this page  

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact [email protected] or view our subscription options here:

You are currently unable to copy this content. Please contact [email protected] to find out more.

You need to sign in to use this feature. If you don’t have a WatersTechnology account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here: