Opening Cross: A New Hope for Data Agility

Something similar happened on another Brian de Palma movie, The Untouchables, where Bob Hoskins was supposedly originally cast as gangster Al Capone, only to be replaced by Robert de Niro—though it appears that what actually happened was that Hoskins was enlisted in case de Niro (de Palma’s first choice) dropped out.

Whatever the facts versus Hollywood fable, these tales reflect the importance of agility and being able to shake up the content of your cast list—or in the data world, the content of your content, and the ability to add, cancel and replace data sources to fit specific requirements or based on their price or accuracy.

In the past, changing data sources on the fly has been at best arduous and at worst near-impossible. Not only must data managers contend with end-users demanding data that they’re familiar with; there are also contractual issues that make it harder to switch in a timely manner, as well as technical challenges to integrating new datasets quickly and performing any mapping required to swap them with other datasets, plus legacy architectures that don’t necessarily encourage changing sources of data that’s already tightly embedded in other applications and platforms.

Over the years, the industry recognized that this inability to change data sources—in the same way that one might change any commoditized service, say, your preferred coffee shop, brand of pasta sauce, or even your cable TV or internet supplier—was costing it a lot of money. However, the changes required to enable this level of agility were neither small nor cheap, and with the financial crisis already in full effect when abstraction layer projects like Collaborative Software Initiative emerged, there was little inclination or investment to fund such projects, despite their potential savings.

Then, slowly, things started to change. This was partly a result of the industry adopting new technologies, such as cloud computing, and partly because some vendors realized a pressing need to be able to switch between datasets—even if this was driven not by consumer cost concerns, but rather by resiliency issues.

For example, news and trade indicator provider Benzinga is readying a new cloud-based marketplace of datasets from niche providers that lack the infrastructure and resources to gain the same distribution for their data as large vendors. Though initially aimed at developers more than traders, there’s no reason this couldn’t become the new paradigm of end-user self-sourcing. And Benzinga isn’t alone: Xignite is perhaps the best-known cloud data platform, while newer startups like Tradier are also taking a similar approach, making it easier to subscribe to and start using new datasets.

Then there’s the adapter built by SIX Financial Information and Bloomberg that allows Bloomberg feed clients to use SIX as a backup, and which the vendors are now rolling out in Japan. The Bloomberg Enterprise Adapter for MDFSelect maps SIX’s content to Bloomberg’s symbology so that in the event of an issue with Bloomberg’s B-Pipe feed, clients can switch seamlessly to using SIX data. To take this to its logical conclusion, users could switch between sources based on preference or other factors, rather than having to negotiate separate supplies.

If nothing else, these initiatives will hopefully spur greater openness and integration—and competition—between other vendors, and make them compete on cost and quality, rather than based on incumbency and legacy infrastructure. Now that’s more than just a new hope: That’s truly a force awakening.