I GROANED last week when, as happens to business journalists several times a year, I was rung up by a polling firm canvassing my views as an 'opinion former' on how a client company ranks across a range of criteria. The interviews are tiresome and (I think) pointless the only reason for doing them is the contribution to charity that the company makes in return.
But that creates problems. The honest answer to most questions ('How do you rate x for innovation?') is, 'I haven't a clue'. But it's embarrassing to admit that. (These are well-known companies, you're an 'opinion former' and they've paid for your views.) So instead you extrapolate from what you do know, usually financial performance. If it's a successful company, it's a fair bet it's OK at innovation, too. The reverse is also plausible.
In other words, you infer a correlation between performance and something else. But in the words of the immortal Bo Diddley, 'You can't judge one by looking at the other' – it's an optical illusion, a 'halo effect', and it renders most of that expensive 'opinion-former' research worthless. The same fault vitiates grander surveys, such as Fortune ' s 'most admired' list, out with the usual fanfare this month. When a company such as Dell or Sony tumbles sharply down the list it's usually because of poor financial results, but the rankings under all the eight headings that make up the index tend to fall in tandem.
Does this matter? More, perhaps, than you think. As Phil Rosenzweig charts in his feisty and entertaining new book The Halo Effect, from which these insights come, problems of research methodology and corrupt data bedevil much management literature, turning it into reassuring parables rather than reliable guidance based on empirical evidence. Such data are particularly dangerous, he charges, when used to support prescriptions about how to succeed - the 'mother of all business questions' - which has generated a lucrative line of management best sellers from Peters and Waterman's In Search of Excellence (1982) on.
Most of them, Rosenzweig shows, are undermined by halo effects, among other manifestations of pseudoscience. Unusually, Rosenzweig, a professor at Swiss management school IMD, has the temerity to name names. For example, In Search of Excellence , the original management blockbuster, is taken to task for the elementary error of looking only at 'excellent' companies, which is like trying to identify what causes high blood pressure without using a control group of non-sufferers. It is also a good example of the halo effect at work. If you start by choosing a group of successful companies and ask managers to account for that success, it would be surprising if they didn't mention listening to customers, good people management, strong values and strategic focus. But these are a rationalisation after the event. Describing success doesn't explain what caused it - strong values could equally well be the result of success as its cause. 'Whether these things drive company performance, or whether they're mainly attributions based on performance, is a different matter. Peters and Waterman went searching for excellence, but they found a hatful of halos.'
Rosenzweig also gives a seeing-to to Jim Collins and Jerry Porras's Built to Last and Collins's Good to Great , perhaps the most influential management volumes of recent years. Both earn ticks for comparing the 'great' exemplars with a control group of less successful companies. But each is riddled with halos and other delusions: delusions of rigorous research (it's quality not quantity that counts), of lasting success (company performance fluctuates, with a tendency to revert to the mean), and of 'organisational physics', the idea that, to quote Built to Last , there exist 'timeless principles of enduring greatness' that apply everywhere.
One reason they don't and can't is the especially treacherous 'delusion of absolute performance': performance is relative, not absolute, which means that the idea of a company achieving success by following a simple formula, irrespective of what competitors do, is just a myth. The margin between success and failure, what's right and wrong, is deceptively narrow, and (despite Bo Diddley) the one can become the other almost overnight.
Rosenzweig attributes the success of the management blockbusters to the authors' ability to spin reassuring, inspiring folk tales which hold out the promise that high performance is within reach of anyone with persistence and focus. In themselves, of course, these things probably aren't wrong, and may even help. But although necessary, they are not sufficient. There are no simple answers to complex questions with many interdependent variables, and managers who believe that there are simply end up taking their own press at face value, a recipe for (relative) failure. As Field Marshall Slim once wisely put it: 'No news is ever as good or bad as it seems at first sight.'
The Observer, 18 March 2007