THE SYSTEM for training and employing the UK's junior doctors was always a bit of a black box. Traditionally, the annual announcement of placings was followed by a period of furious informal horse-trading as individuals and institutions swapped places to get as close as possible to their real preferences. They were usually successful. To the government, however, this was a mess that it decided to reform, using an IT system to match applicants precisely with posts.
The result was a disaster. It wasn't that the computer didn't work. It was too precise. While the removal of 'give' in the system made it tidier, it also took away the flexibility in timing and negotiability that made it work. Messy is sometimes more efficient than neat and tidy.
The government's error is a common one. Obsession with order is the malady of modern management. According to the splendid A Perfect Mess: The Hidden Benefits of Disorder , by Eric Abrahamson and David Freedman, 'organising' - decluttering junk-piled desks, closets and spare rooms - is a $100m-a-year industry in the US and growing. Organisations are also compulsive tidiers, addicted to diagrams of boxes and col oured lines, even though these are often more wishful thinking - a sort of naming and taming - than description of reality.
In fact, liberating as this book is, the euphoria at discovering you aren't a freak if your desk is untidy - just human - is rapidly swamped by a wave of despondency as you realise how determined organisations are to stamp out deviance. The point is that because randomness is an essential part of nature (goodbye evolution without it), all organisation comes at a cost - the cost of carrying out the classifying and sorting, and the subsequent, less obvious, cost of complying with it, or preferring one form of organisation over another.
Reasonable organisation owes everything to context - the Japanese '5S' good housekeeping policy ('sort, straighten, sweep, standardise, sustain') makes perfect sense in a busy factory but is anal lunacy when applied to the position of a stapler or paper clips on desks in a revenue office (I'm not kidding).
When costs outweigh benefits, organisation becomes over-organisation - the denial of humanity. It's surprisingly common. Ever get the feeling that these days all change is for the worse? You're right - and it's to do with over-organisation. The parallel growth of regulation and organisational stupidity, for example, is not coincidence: it is driven by obsessive classifying of every form of risk or failure, aided and abetted by the use of computers to analyse the figures.
Or take the ubiquitous automated voice-response systems used by call centres. In effect, these are crude 'categorisation engines' designed to force human variety into arbitrary categories. Unsurprisingly, these categorisations fail to match customers' needs in a huge percentage of cases, breeding contempt and despair - and causing half of them to take their business elsewhere.
Such counterproductive classifying reflects a propensity, common to all institutions, vastly to overestimate the importance of formal organisation. A report by consultants Booz & Company notes that companies have a knee-jerk tendency to ascribe both problems and solutions to organisation - in both cases misguided. While restructuring and organisational change are often the first resort of managers, they don't even figure in the top 10 for effectiveness, according to Booz.
Of course, a degree of creative disorder is not the same as slobbishness or lack of discipline. Newspaper deadlines impose tight timing and resource disciplines no one would want doctors and nurses to ignore infection procedures for the sake of creativity. The difference is well illustrated by Google, as described in this column last week. Managerially, Google can only be described as a mess, with what looks to others like an unfeasibly large amount of play in the system (not least the 20 per cent of time allowed to developers to pursue their own projects). But its aim - to organise the world's information - certainly is not, and its disciplines of fierce product reviews and thrashing out important issues transparently are anything but. The results suggest that Google has the balance much righter than most conventional companies.
So before you guiltily vow to turn out and reorganise your sock drawer, filing cabinet or sales organisation, think again. Of course you don't want to be crushed by an avalanche of books and papers, as happened to one pathological hoarder in the US. But comfort yourself with Abrahamson's finding that work messiness increases with education, salary and experience and remember that a pinch of mess, randomness and redundancy is as essential for innovation and robustness as salt in food.
As Albert Einstein, who apparently maintained his desk in a state of stupendous disarray, sweetly inquired: 'If a cluttered desk is a sign of a cluttered mind, of what, then, is an empty desk?'
The Observer, 3 August 2008