Run the bank, change the bank: CTOs juggle needs and wants

Voice of the CTO: In part two of a five-part series, bank technologists explain where firms go wrong when trying to modernize their tech stacks and manage technical debt.

Voice of the CTO

This article is the second in our new, annual examination of the spending plans of bank technology leaders. Part one of this five-part series can be found here.

The goal is to shine a light on opportunities that some of the world’s biggest banks are trying to grasp—and the obstacles that stand in their way. Some of those hurdles are internal—winning support from the business for important-but-unglamorous refits, for example. Others are external, such as the sheer pace of change. And at the core of everything is data.

Part three, which will be published on Wednesday, February 7, will look at the industry’s increasing adoption of AI throughout organizations. Part four will examine data management and governance issues. And part five will look at interoperability efforts across the front, middle, and back offices.

See the Methodology box below for more details on how the series was created.

A few years ago, a tier-one US bank invested $2 million—allocated over four years—to build an electronic trading platform. It was an efficiency play. Essentially, operations teams were having to look at five different screens to research trades. The goal was to get that number down to one—the much-desired one-stop shop of information. 

As a global head of technology at the bank tells it, a lot of money went into figuring out regulatory control functions before any could be spent on development and testing. The issue was that this new system needed to plug into multiple legacy platforms—hence the five screens—and there were concerns over data licensing and disrupting reporting processes

Once they got to the testing stage, they found databases using old technology became unstable when they added new technology on top. As the bank worked through these issues, “a lot of money” was spent configuring the new platform for individual products (such as non-deliverable forwards) and clients (extending corporate services to smaller clients). From there, the bank had to decommission those older platforms, with each instance creating its own ripple effects. 

But at least at the end of the day they had an all-singing, all-dancing, brand new platform… right? Well, not according to the head of technology. “Very little was delivered,” they tell WatersTechnology.

This is a scenario that every technologist can appreciate—there’s an inexorable push coming from the top to be “data-driven” and “cloud-native”. These are catchy phrases for an earnings call, but they ignore the reality of modernizing and automating legacy systems, and the technical debt they accrue.

Slow train coming

There was a time not so long ago when the largest banks and asset managers preferred to build their own platforms, rather than buy off the shelf—think Goldman Sachs and SecDB or BlackRock and Aladdin. And it was only 15 years ago that financial institutions began tinkering with the cloud in earnest, with most of the work done via a private or hybrid model. “I don’t think an extreme will ever happen. Hybrid is not a path toward full public cloud adoption. There will always be some amount of competitive edge involved with resources you have complete control over,” one technologist told WatersTechnology back in 2010.

While there’s still some truth to that statement, the needle is rapidly moving toward partnerships with the four major cloud providers in the capital markets: Amazon Web Services, Google Cloud, IBM Cloud, and Microsoft Azure.

Cloud has been the great disruptor in the 21st century. It has ushered in the era of Big Data, software-as-a-service (and [fill-in-the-blank]-as-a-service), which gave rise to managed services and wider adoption of APIs. Cloud played a major role in democratizing AI tools and pushed firms to be more open to the ethos of open source. And, while Wall Street was once all about building platforms internally, these shifts have completely reshaped the debate around buy versus build. 

But if banks want to truly embrace these cutting-edge new tools, they first need to deal with their legacy systems. 

“Things like quantum and DLT, we very much keep an eye on. But they’re not really a big deal at all—it’s more about stability,” says the chief information officer at a second global systemically important bank (G-Sib). “How do we get away from our legacy environments and still provide a safe environment? In turn, it might cost more, but we can worry less about infrastructure failure.” 

The evolution of trading technology has been dizzying over the past decade. A little over a year ago, generative AI wasn’t a point of discussion at banks’ board meetings—now, it’s everywhere. But the CIO says the main priority for bank technology teams today is flexibility. 

The financial crisis and the deluge of regulation it brought in its wake taught a lesson—banks were too siloed; data was scattered and pinning down a dataset’s lineage was often impossible. This period also forced banks to look to vendors for help, as their own internal technology teams had been cut to the bone. “So it’s really about cloud, AI, and shifts in development. It all underlines the notion of doing more for less and reducing costs,” says the CIO. 

They add that they are hopeful that as capital markets firms increasingly rely on technology and automation, it will lead to conversations between the business stakeholders and technologists: What should be kept on-prem? What are some of the burdens of client data? How much control do you have over data if it’s off-site versus in-house? And, when it comes to budgeting, how much can be directed toward more forward-looking projects versus modernizing (or decommissioning) legacy systems?

Unfortunately, the ebb and flow of those conversations changes from year to year, they say. “If you’re in cost-reduction mode, you want technologists to be in control, right? Tell us how much to cut and have us figure it out. But when you’re in growth mode, technologists aren’t allowed on the business side; they’re told where to direct their attention. So you go through these natural cycles of cloud, on-prem, distributed, mainframe, and so on. Right now, we’re [being told what to do]—how many years until we’re back in an aligned phase? I don’t know: one year, two years, three years? At least this keeps the consultants happy.”

‘And I’m sayin’, I’m ready to run now’

“Run the organization; change the organization” is a common term used in technology circles—it’s the balancing act that takes place between day-to-day, keep-the-lights-on needs and innovative (potentially alpha-driving) tools. The chief operating officer at a third bank says the budgeting process is split roughly 50/50 between these dichotomies. 

One truth that is often not acknowledged, says the COO, is that systems described as “legacy” today were once considered innovative. Banks have dug themselves a hole in the past by getting, say, a couple decades of use out of a system, but if it was designed for a singular purpose, then the technical debt costs grew each year. As a platform nears the end of its life, technical debt can cost a firm a third of its overall software development budget, if not more, they say.

“You will see some institutions be lopsided toward one or the other [run, or change] depending on where they are on their journey. If you rewind five years, we did an incredible amount of infrastructure modernization. We had end-of-life across our fleet of compute—50,000 to 60,000 servers.” 

Right now, we’re [being told what to do]—how many years until we’re back in an aligned phase? I don’t know: one year, two years, three years? At least this keeps the consultants happy.

This led the bank to build a redundant datacenter to improve its resiliency, because the plan was to invest in more modern tools while decommissioning older ones. For its most critical workloads, it now runs a hot-hot workload configuration, where each environment actively handles requests and transactions at the same time.

“As a result, over the last couple of years the pendulum has swung as we’ve completed a lot of that [infrastructure] work, and we have made sure to swing that capacity toward modernizing our trading and risk systems,” they say. 

The CTO at a fourth tier-one bank notes, though, that getting the business to understand the importance of managing technical debt and spending budget on modernizing aging systems is often difficult. Cloud, machine learning, and cutting-edge analytics systems can improve trading profits, help firms to better manage risk, and reduce staffing costs in the back office—they’re easy sells, even if sometimes overused buzzwords

Technical debt, even though it can eat up millions of dollars of budgetary spend, is a boring topic for the business side of the house, much like data governance and management (more on that next week in part three of this series).

“Why I’m calling out technical debt is because we find a lot of the time for these kinds of more forward initiatives that there’s some degree of retrofitting that has to take place,” says the fourth bank technologist. “So, there are three levels, really: it’s the actual build, the trying to pre-empt those unpleasant surprises—Oh shit, we didn’t know! We didn’t realize we had to do that!’—and managing assumptions and whatnot. That’s your technical debt, and that’s not even including your downstream maintenance.”

Sources say that for any modernization project, technologists need to be able to articulate the drivers, cost, and ROI for these endeavors: Cost control versus business drivers? How will this help the bank to react to market shifts better, faster? How does it help the bank make money? How does it help protect money?

As the first bank technologist learned from the building of a unified trading system that hasn’t yet delivered on its intended ROI, firms should first address downstream legacy systems before creating a one-stop-shop platform; they need to build in regulatory controls so that they do not always have to readjust to new requirements, or risk disrupting current reporting workflows; and they need to have a clear picture as to how the project will create new business and improve the client experience. 

Otherwise, you’re simply building a system that will become another technical debt black hole that will need to be re-platformed at some point in the future. All while competitors are exploring new ways to, say, use AI and other innovations in the investment process.  

Methodology

The “Voice of the CTO” series is based on interviews conducted by WatersTechnology with eight CTOs from a selection of tier-one international banks that took place between October and December of last year. For clarity, the term “CTO” is a catchall that includes chief information officers and various other global heads of capital markets technology—people whose focus is on the corporate and investment bank and who handle a budget. In the story, we will address the individual’s job title, but have granted them anonymity so that they can speak openly about their organization and beliefs.

The series also draws on a limited-circulation survey of handpicked technology end-users, and incorporates new market sizing work by WatersTechnology sibling brand Chartis Research.

We plan to repeat the exercise later this year, and are looking for feedback. If you have any comments or questions, please get in touch: anthony.malakian@infopro-digital.com

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe

You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.

Most read articles loading...

You need to sign in to use this feature. If you don’t have a WatersTechnology account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here