THIS YEAR the UK will waste pounds 10 billion or more on IT projects that go over time, over budget and just plain wrong. Half of the total will be public money. Much of the waste is avoidable, the result not of technical problems but of dismal management and shoddy methods that, like a computer virus, endlessly replicate the mistakes of the past.
This is the grim picture painted in a new report, The Challenge of Complex IT Projects, released last week by the British Computer Society and the Royal Academy of Engineering. Although there are some good examples, Britain's lack of professionalism in software engineering is both dangerous and economically debilitating, says the report.
The background is that as the country struggles to renew its infrastructure and improve delivery of services, it is spending more and more on computerisation. This year total IT spending will hit pounds 23 billion, pounds 12bn of which will come from the public sector, now the largest customer. Yet just 16 per cent of British projects are fully successful, according to one estimate, while others put the success rate even lower. This contrasts poorly with the US - where the success rate has doubled to 34 per cent over the last decade - and with the rest of Europe. The UK seems uniquely uncommitted to finding ways of improving.
Hence the regular diet of computer disaster stories, of which the current cock-ups at the Child Support Agency (a pounds 400 million system that was switched on 18 months late and still leaves staff reliant on pocket calculators) or the magistrates' courts (pounds 318m and still counting - 'one of the worst IT projects I have ever seen,' according the chairman of the Public Accounts Committee last year) are fully representative.
The report lays much of the blame for the failures on management shortcomings, in particular the failure of both customers and suppliers to follow known best practice. Contrary to many assertions, big IT projects have much in common with other large-scale engineering schemes. 'It's project management,' says the Dr Mike Rodd of the British Computer Society (BCS). 'There's not much difference between a big IT project and building a second Severn crossing.'
Professor John McDermid of York University, one of the report's authors, laments that 'in IT best practice is rarely practised. Projects are often poorly defined, codes of practice are frequently ignored and there is a woeful inability to learn from past experience'.
The difference between best and typical, he adds, is enormous. Although the researchers found no hard evidence that the public sector was worse at IT management than the private, the stakes are considerably higher.
For one thing, public sector projects are more visible. For another, they are often larger and subject to both technological and political 'creep'. Moore's Law - which predicts the quadrupling of hardware capability every three years - fuels ambition and complexity, which eat up potential cost gains and far outrun software improvements and the ability to manage them.
The danger is greatly amplified when customers are technologically illiterate, which is generally the case with British managers, but particularly with those in Whitehall and Westminster. The result is often naive over-optimism about the benefits of IT and gross underestimation of the difficulties. 'It's hard to be an "intelligent customer" if you don't know what you're buying,' notes McDermid.
New project-management procedures put in place by the Office of Government Commerce under Sir Peter Gershon will undoubtedly help. Nevertheless alarm bells should be clanging violently around at least three very large and highly ambitious projects now being undertaken in the UK public sector: identity cards, the merger of the Inland Revenue and Customs and Excise, and the NHS computerisation programme.
'I'm not saying they can't be done,' says McDermid. 'But I'd like some reassurance that the analysis has been carried out. Have the risks been analysed? Are the projects implementable? Will they do what Ministers want cost-effectively? Some searching questions need to be asked.'
Part of the questioning, of course, should be about the need for IT in the first place. The fact that the private sector is pulling back from customer relationship management and blanket e-enablement while the public sector is piling in should tell us something.
The e-government initiative has so far absorbed pounds 8bn on the self-evident assumption that it is 'a good thing', with little evidence of benefit or payback. On a smaller scale, central specification of everything from call centres to document-processing technology similarly assumes that IT is the answer - often before the real question has been identified.
What's the remedy for this perennial British weakness? Being clearly linked to wider British management failings, it is not susceptible to easy answers. Better, tougher customers are one part of the equation - McDermid regrets that short awareness courses for senior defence department managers have been discontinued, for example. On the supply side, the report sets store by increasing the professionalism of the software industry. The BCS launched a chartered IT professional qualification last week.
But the key remains breaking cottage-industry mentalities and getting both customers and suppliers to accept that there is a better way. The report mentions the pioneering work of the Software Engineering Institute at Carnegie Mellon University in the US, which has shown that delivering software projects on time and within budget is perfectly possible using well-tried 'lean' principles that build quality in from the start. It argues that Britain needs a similar institute to provide a focus and drive improvements.
In the meantime, we can only share the feelings of the Japanese programmers who have taken to replacing unhelpful and impersonal Microsoft error messages with wistful haikus:
Yesterday it worked.
Today it is not working.
Computers are like that.
Three things are certain:
Death, taxes and lost data.
Guess which has occurred.
The Observer, 2 May 2004