Don't Go Small on Big Data

This week the Waters team put together a special report on Big Data. In his opening letter, Waters editor-in-chief Victor Anderson points to the monetary value for those who are on the cutting edge of this large-scale frontier.

I completely agree. Those firms that are at the forefront of mining this data, and those that are innovative in the ways they use the information gained, will have an enormous competitive advantage over the laggards that are in a reactionary position.

Unfortunately, though, the financial services industry has chosen to sit on the sidelines rather than innovate when it has come to things like cloud technology, mobile technology, social media and field-programmable gate arrays (FPGA).

My colleague Tim Murray covered a panel discussion this week that addressed Big Data. Knight Capital's Steven Sadoff told the audience that the market rarely delves into the cutting edge of new technology.

"Advances in technology simply haven't kept up," he said. "Whatever you want to throw at a particular tech problem, we've only seen evolutionary improvements."

There is hope, though, that buy-side firms will look to address the challenge of Big Data internally, rather than outsource it. At that same event, Tudor Investments’ CTO Oleg Olovyannikov said that Big Data storage is something the firm is looking at and that it has tended to build its solutions in-house.

"We have always had a very proprietary mindset—and therefore tend to build everything ourselves," he said. "We manage hundreds of data feeds—real-time as well as research feeds, and track them all in-house. We looked, and simply don't find a lot of savings in outsourcing, when our junior personnel can handle those tasks at lower cost. As a smaller, agile buy-side firm, our challenge is personnel, and keeping the high-value ‘boutique' knowledge in-house."

I would hope that more people in the industry choose to take this approach. For non-proprietary things like market data, the vendors should be leading the charge and rolling out solutions. But when it comes to monetizing Big Data, there's much to be gained by being at the head of the class.