Plumbing the Depths of Unstructured Data
Turning to organizing measures like collaboration, ‘microdata’ and application of standards can help manage a deep source of information.

Can collaboration and sharing of data, within the context of adhering to a standard when devising data governance plans, help solve issues with handling unstructured data? This question arises when hearing how firms try to build data foundations to produce higher quality data analytics.
Data governance and analytics can be improved by building mechanisms to share data, says Leif Hanlen, a business development executive at Data61, a data requirements organization that is part of CSIRO, a digital development agency backed by the federal and state governments of Australia.
Yet, however advanced a data sharing system is, the rules that data vendors and exchanges have about their data make it challenging to join such disparate sets together, says John Denheen of Tyler Capital in London. “One of the big issues with getting good reference data is that you need to be able to tack it on to other data sets,” he says.
Before compatible standards can be achieved, firms have to identify the business practices that differ, says Sydney Hassal, a director at Scotiabank. Along with this, standards can vary among jurisdictions, observes Allie Harris of Bank of Montreal.
The US Office of Financial Research (OFR) points to “microdata,” which OFR chief data officer Cornelius Crowley defines as granular, targeted and specific information about entities, instruments, transactions or products. He advises that breaking down standards efforts to individual data elements like those he calls microdata can increase accuracy. In this context, one wonders whether unstructured data can be organized with a “microdata” approach—and what that can mean for following standards.
Harnessing unstructured data could be most useful to support consistent application of data standards.
Analytics Goldmine
Unstructured data is emerging as a “goldmine” for the application of data analytics, CSIRO’s Hanlen says, although it is nothing new—it is akin to “50 million Word documents saved over the last 10 years and put in a box.”
Firms should now understand the two types of unstructured data, says Hanlen. “Unstructured data sitting inside the enterprise—in the customer relationship-management system, in fields called ‘other’—is like a hole in the ground that’s yet to become a goldmine,” he says. External unstructured data from sources such as social media and market surveillance is already being tapped for its golden value, he says.
Analytics can come into play for the former type of unstructured data. “The task for analytics of unstructured data is not to build a brand new goldmine, but to extract elements of information from that unstructured data,” he says.
CSIRO and Data61 have worked on analyzing payments, documents predicting market outcomes and price signals based on the likelihood of future events. The challenge Hanlen finds in working with such unstructured data of a predictive—rather than measuring—nature is placing that data in a credible spectrum, which can be used to anticipate the future or a likely outcome.
Unstructured data may be a goldmine, but it is one that industry experts are just figuring out how to tap, for a variety of purposes. In March, ISITC executive Jeff Zoller pointed to unstructured data about investment behavior and patterns, as well as industry analysts’ commentary, and social and economic behavior, potentially supporting the disruption of data-management technology. In May, MUFG Canada Branch chief information and operations officer Ron Lee, speaking about how his firm is consolidating systems, noted this effort would increase the capability to work with unstructured databases.
Before any of these more aspirational purposes, however, harnessing unstructured data could be most useful to support the consistent application of data standards. The legal entity identifier (LEI) is ripe for this, as Karla McKenna, head of standards at the Global Legal Entity Identifier Foundation, observed in June, saying entity legal form information—a data element in the LEI repository—remains in “unstructured form.”
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
More on Trading Tech
What is Aladdin Trader? BlackRock’s fixed-income tool still a mystery
Sources tell WatersTechnology that Aladdin Trader will expand on the platform’s ‘limited’ execution functionality.
Better tech brings threat of two-speed trading in fixed income
Smaller asset managers may get left behind as automation allows the big players to prosper.
Waters Wrap: Examining ASX’s CHESS do-over
The Australian exchange was the first exchange to be all-in on DLT—and the project failed. Anthony speaks with ASX’s Tim Whiteley to discuss the lessons learned and why he thinks the second attempt will succeed.
Can machine learning help predict recessions? Not really
Artificial intelligence models stumble on noisy data and lack of interpretability
GenAI and HFT: A competitive edge?
Timing is critical in the capital markets. IBM researchers say genAI’s predictive abilities in combination with AI/ML models can translate into profitable HFT opportunities.
Goldman Sachs looks to ‘Amazon’-like ecosystem with MarketView launch
The new research and collaboration tool is delivered through Goldman’s Marquee platform.
Bloomberg deploys new chatbot tool to Terminal
The offering allows users to surface data and notifications from internal systems without ever having to leave Instant Bloomberg chatrooms.
Deutsche Börse to merge SimCorp and Axioma, shuttering Qontigo
The German exchange also plans to have 70% of its ‘IT state’ in the cloud by 2026.