Fool’s Gold: Data Mining Digs Up Explosive Errors
Recent studies reveal the prevalence of poor-quality data, exacerbated by increased use of machine learning that allows users to dredge far bigger datasets and identify spurious correlations.
A review of 100 major psychology studies, for instance, found that only 36 percent had statistical significance. Over half the alien planets identified by Nasa’s Kepler telescope turned out to be stars. And in preclinical cancer research, a mere six out of 53 breakthrough studies were found to be reproducible. Quantitative finance does not fare much better.
“It’s a gigantic problem—spurious results are the norm,” says Zak David, co-founder of analytics firm Mile 59, and former engineer of high
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: https://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
The great migration: Firms work to integrate ISO 20022
Banks and asset managers are working to adopt the new messaging standard, but it’s easier said than done.
PostSig, CJC team up to run data management for London broker
In the first such deployment between the vendors, CJC will act as an outsourced data administrator using PostSig’s technology to manage the firm’s data.
Buy-side firms warn semantic layers not a quick fix for AI integration
Semantic layers promise to teach AI what a firm’s data means. Keeping that meaning current is the part that may get overlooked, tech heads warn.
Mind the stack: the foundation markets run on
Exchange data is an underappreciated component of the AI era, writes Jeff Kimsey.
BlackRock Aladdin prioritizing semantic layer for agentic AI
The asset manager’s tech platform is spending “a lot of time thinking about” semantics before it rolls out AI for a variety of tasks, says Aladdin exec.
The impact of market data and alt data converging
The Waters Wrap: As firms combine traditional data sources with proprietary research and alt-data signals, Max has some questions about where the data market is headed.
GPU compute derivatives are put to the test
Interest rates and onions of the past haunt what could be a booming new index market.
MEMX’s new options venue, ICE launches private credit reference data, and more
The Waters Cooler: A recap of the major tech and data news from the past week in the capital markets.
