Whatever You Do, Don't Push the Button

Every time I go on vacation, it seems to coincide with another technical meltdown at a major financial services firm. The first time, it was NatWest's mini-meltdown that they still seem to be picking over now. This time, while I was in New York last week, Knight almost imploded with élan, covered in a particularly adroit fashion by my colleague, Jake Thomases, in this column last Friday. Add in the continuing woes at the Tokyo Stock Exchange (TSE), mix in some money laundering accusations and all the rest, and you have to ask; what's going on?

Part of it, I'm sure, is just bad timing. Taken separately, these incidents would be bad enough, but together you can imagine what Antonio felt like when his ship was driven aground by Prospero's tempest. I'm sure I'll start receiving various e-mails as soon as I say this, but glitches do happen. Technology isn't infallible, as much as the vendor community would like you to believe. But continuing issues do start to point to wider, endemic structural issues that could do with some attention.

Plan B
"Another financial sector 'software glitch', this time at the Tokyo Stock Exchange, hits the headlines hot on the heels of Knight Capital, the Spanish bourse and RBS," says David Silverstone, software testing expert at NMQA, which provides services to big names such as Goldman and Barclays. "Firstly, these problems are 100 percent preventable and CIOs should be asking some tough questions of their development and testing teams about the balance between time to market versus quality of delivery. Secondly, if quality isn't aggressively enforced in-house, external regulation will come into play. Senior executives need to understand that IT is the business and if IT fails, the business fails."

IT, I think, is going through something of an identity crisis at the moment. Traditionally it's always been important─a business runs on its technical backbone, right? It's mission-critical in the purest form of the word. But it's never been the sexiest of areas until quite recently, when it's moved from blocky mainframes to sleek blades, and dodgy internet connections to co-location and microwave networks for ultra-low-latency. As a result, some people I speak to are starting to feel that technology is moving out of the hands of IT, and too much into boardrooms.

With that comes a learning curve. While traditionally, senior management in investment banks has come from the trading desks, more influence is starting to come from risk, from compliance, and from the CIO/CTO function, most of whom have a deep understanding of technology, if not always full background in it. These people get it, but the constant pace of change within the industry, the need to be quicker, faster, and better, sooner means that layering occurs and starts to create problems.

You can't just say that, oh, it's because it's too complex. Giving a simple answer to an issue that is supposedly complex, by definition, negates the idea of complexity.

Revamp
Wholesale rip-and-replace is not a practical method of maintaining a conversant IT infrastructure. It's too expensive, and too disruptive, particularly with systems that need to be up and operational 24/7. So you tweak where you can, and adjust where possible, until you can phase in new tech seamlessly.

Some of the technology problems in the industry have come from this sprawling IT estate, and that can't be denied. Some haven't. Knight, for instance, lost an enormous amount of money in a sobering passage of minutes, but that wasn't down to the fact that their execution engine didn't tie up with their reconciliation software, or that their HFT layer was squeezing a legacy backend dry until it finally decided to pull the trigger on itself. It's because an algo went rogue, and there's not a great deal that can be done about it, particularly when the regulators refuse to reverse trades. Blame for that particular can of worms is an entirely separate issue.

The point is that there are clearly problems that need to be worked through, but they can't be lumped into neatly defined brackets (again, I'll expect that sentence to be shot back at me later today). You can't just say that, oh, it's because it's too complex. Giving a simple answer to an issue that is supposedly complex, by definition, negates the idea of complexity.

If you'd like to talk about IT estates, meltdowns, algorithmic glitches, or just how brilliant Errol Flynn was in The Adventures of Robin Hood in Bryant Park last Monday, give me a shout on +44207 316 9811 or james.rundle@incisivemedia.com.