Big Data Vs. Dark Data
Looking at what the buzzword terms really describe for financial data management
In a story last week, I had looked at the use of the term "dark data" and what it means. That is a newer buzzword than one that has been thrown around much more over the past several years, "big data."
Often when "big data" has been used in financial services operations discussions, particularly in data management operations discussions, it really pertains to what is being done with data, such as how firms mine larger amounts of data or how they organize it to get the most insight and value out of it.
So this raises the question of whether "big data" is really even an appropriate term. Increased activity concerning know-your-customer (KYC) data might lead one to think that type of data is rising to the level of "big data," but if compared to something like phone call records, even from just one major cell service provider, KYC data could look small in volume by comparison.
Is there a threshold that KYC data or financial services industry data as a whole needs to cross to truly be "big" and not just "medium," perhaps? Customer details, if cross-referenced with extensive transaction data, could be that "big" in this industry.
Failing that, firms may be better off not getting caught up in "big data" hysteria. It's more effective to work to know exactly what data you do have, how frequently you are getting it and what you are trying to achieve with it—namely, what insights you are trying to generate from it.
In effect, dark data and big data are really about the same thing—data management. Dark data describes knowing where all the meaningful data is and how to aggregate it, while big data infers understanding the complete picture of what data is coming in, especially if the volume of that data is getting larger.
To join a discussion on how "big data" should be defined, visit Inside Reference Data's LinkedIn discussion group.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
Hub to lay off 20% of staff, sources say
Hub’s CEO says this is simply a case of a startup trying to stay nimble and efficient; others say it points to deeper issues.
TS Imagine integrates LTX’s pre-trade analytics tool
Users of the fixed-income EMS will now have access to LTX’s Liquidity Cloud tool, which provides a pre-trade score for the likelihood of trading success.
After contentious Opra upgrades, vendors brace for a faster future
Upgrades to the datafeed widely used to gauge the current market price for options contracts went into effect in February after three separate delays, which market participants say were caused by persistent bandwidth issues at some important recipients.
The IMD Wrap: No more turf wars, or why CDOs should heed the Voice of the CTO
Max reviews how our recent Voice of the CTO series has implications for those beyond a firm’s technology function, and how communication and collaboration between tech, data, and leadership will deliver better results.
Dark horse: Deutsche Börse building dark pool
New functionality allowing exchange members to execute sweep trades comes hot on the heels of European rival Euronext launching its own dark pool.
Man Group’s proprietary data platform is a timesaver for quants
The investment firm’s head of data delves into its alt data strategy and use of AI tools to boost quant efficiency.
Waters Wrap: The tough climb for startups
Anthony speaks with two seasoned technologists to better understand why startups have such a tough time getting banks and asset managers to sign on the dotted line.
As crypto ETFs become reality, benchmark providers take center stage
The SEC’s approval of the first spot bitcoin ETFs will expose a growing number of traditional market participants to the maturing world of crypto data, a moment that some—such as CF Benchmarks, BlackRock’s benchmark provider—have been eagerly awaiting.
Most read
- Women in Technology & Data Awards 2024: All the winners and why they won
- Witad Awards 2024: Above and beyond award (vendor)—Susan Bennett, Tradeweb
- Fighting FAIRR: Inside the bill aiming to keep AI and algos honest