Fool’s Gold: Data Mining Digs Up Explosive Errors
Recent studies reveal the prevalence of poor-quality data, exacerbated by increased use of machine learning that allows users to dredge far bigger datasets and identify spurious correlations.
A review of 100 major psychology studies, for instance, found that only 36 percent had statistical significance. Over half the alien planets identified by Nasa’s Kepler telescope turned out to be stars. And in preclinical cancer research, a mere six out of 53 breakthrough studies were found to be reproducible. Quantitative finance does not fare much better.
“It’s a gigantic problem—spurious results are the norm,” says Zak David, co-founder of analytics firm Mile 59, and former engineer of high
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: https://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
After funding boost, Databento preps data expansion
After recently announcing $97 million in funding, the vendor is pressing ahead with global expansion plans.
Can AI beat exceptions out of the back office?
The Waters Wrap: Agentic AI can help operations teams tackle exceptions. But first, they need to get their house in order, writes Wei-Shen.
SIX launches market data platform designed for accessibility
SIX Market Signal aims to bring price transparency, ease of use, and reduced administrative burden to the maddening world of market data.
ISS Stoxx’s new analytics, S&P and Kaiko launch joint indexes, and more
The Waters Cooler: A recap of the major tech and data news from the past week in the capital markets.
Federal court moves to certify class in Cusip antitrust suit
The SDNY judge has handed a procedural win to plaintiffs, allowing swaths of end-user firms and third-party data vendors to join the class action.
The market’s most valuable asset?
DTCC’s Tim Lind writes that using data effectively can provide the intelligence needed to improve investment performance, reduce risk, and inform strategic decision‑making.
Report once: will Esma’s €1bn reforms deliver the full picture?
Critics say plan to merge three reporting regimes will see scant returns, and won’t mesh with single-sided reporting.
Navigating the MarketAxess ICE storm
The Waters Wrap: Most of the discussion surrounding ICE’s MarketAxess buy has focused more on benefits for the front office than the data possibilities.
