The Next Big Data Debate Emerges
The ongoing discussion about big data, which continued last week in Waters' Big Data Webcast, appears to be turning away from a debate between using cloud computing or the Hadoop standard to a concern with rapidly increasing volume and velocity of data creating a need for greater use of big data systems.
An unspoken context underlying the webcast discussion, which had participants from Credit Suisse, BNY Mellon, Intel, IBM's Platform Computing and Sybase, is that the industry already seems to be leaning or moving away from Hadoop and toward cloud as being more effective for handling big data.
"The cost per gigabyte of storing that transaction over time is pushing us into cheaper, non-SQL, big data-type solutions," said Ed Dabagian-Paul, a vice president at Credit Suisse who works on setting strategy and direction for technology infrastructure at the firm. "The traditional big data solutions haven't mapped to our problems. We can answer most of our existing problems with existing data analytics or very large databases."
Daryan Dehghanpisheh, global director of the financial services segment at Intel, identified "volume, variety, value and velocity" as the four pillars of big data. He had already noted volume, and processing speed and time as key areas for big data when speaking with us in November.
Intel works with partners to produce solutions for operational issues such as big data. According to Dehghanpisheh, the company aims to achieve complex machine learning, statistical modeling and graphing of algorithms within big data, rather than the traditional business intelligence of query reporting and examining historical data trends. Orchestrating use of metadata and setting data usage policies are important parts of administering big data operations, he adds.
An extensible framework is needed to manage the volume and velocity at which big data now pours forth, as Dennis Smith, managing director of the advanced engineering group at BNY Mellon, sees it. "There are tremendous cost benefits to this from a scale standpoint and particularly looking at volume use cases," he said. Cloud computing inherently offers greater scale, of course, and analytics can be layered onto it or attached to it. As Smith also explains, Hadoop-related technologies, or standalone analytics infrastructures and traditional data warehouses as staging areas may all be ways to manage big data in tandem with cloud resources.
The question to ask, or the discussion to have, now, is how to marry big data, sourced from or processed through the cloud, with analytical systems that can derive actionable meaning from the data, for all its increased volume and velocity.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
Deep in the heart of Texas, market data vets sweat the talent gap
The secrets to unlocking all of market data’s questions and the antidote for all its ills were discussed in Texas last week. Unfortunately, we promised not to tell—not all of them, anyway.
Fidelity’s quantum exploration unites theory and proof
The asset manager and Amazon have teamed to put a quantum twist on machine learning.
Interop after acquisitions remains daunting for buyers
Executives from a variety of vendors speaking at WFIC explained why desktop interoperability is important and why progress has been slow.
GenAI could connect ‘constellation’ of vital information technologies
At WFIC, Six Group’s Marion Leslie highlighted the opportunities and risks that the technology holds.
TRG Screen cuts 20 jobs following review and restructure
The review follows an eight-year period of acquisitions and buy-in from a new private equity owner last year.
Why don’t you write a book? Several notable data professionals are doing just that
The IMD Wrap: Max talks to some data architects-turned-authors who are literally sharing volumes of knowledge.
American Bankers Assoc. asks SEC: Do you know what you’re doing?
The industry group disagrees severely with regulators’ interpretation of the Financial Data Transparency Act, hinting at possible legal action in a recently published comment letter.
South Africa’s equity markets court HFTs with tech upgrades
Competition for flow has driven innovation in connectivity, risk, and data provision.