This is the time when pundits make predictions about what will happen in 2009 – after a year of shocking and unanticipated economic events.  The Queen, visiting the London School of Economics, wondered why the credit crisis and its evolution were not predicted despite the vast resources devoted to financial research.  Here, Ma’am, is the outline of one loyal subject’s answer. You will appreciate, I am sure, that economic prediction is hard.  National economies, financial markets and businesses are complex, dynamic, non-linear systems.  Your economy contains many people and many agents, and there are many interactions between them.  Your Royal Society rightly commemorates the successes of the natural sciences.  

Many of these successes have been achieved because the problems of physics often involve objects large enough to be studied individually – the motion of the earth, for example.  Or components small enough to be subject to statistical regularities – we can never predict the behaviour of an individual molecule or electron, but there are so many molecules and electrons that for most purposes it does not matter.  Many of the phenomena we deal with in economics and business fall in between – the units of analysis are individualistic but also too numerous for their idiosyncrasies to be individually understood.

Economic systems are also dynamic.   Dynamic in the sense that they evolve over time – which makes the mathematics harder, but we are well able to cope with that nowadays.  But also dynamic in the sense that the relevant structural relationships constantly change.  Some economists – perhaps a majority of those in your former American colonies – believe that there is a deep underlying structure from which laws of economic behaviour that are universal in time and space can be deduced.  I think that search is a wild goose chase, and that the best we can do is to identify empirical regularities that apply to particular contexts.  But whoever is right, it is evident that much more work still needs to be done in understanding the relevant relationships.

But the killer is that dynamic complexity interacts with non-linearity.  If that statement sounds like an extract from a monologue by Mr Brown, recall the (false) story that your predecessor, Richard III, lost the crown at Bosworth Field for want of a nail in the shoe of his horse.  The point of the tale is that small differences in initial conditions can have dramatic differences in ultimate outcomes.  The problem is often expressed nowadays through the metaphor of the butterfly which, by flapping its wings on one side of the world, sets in train a chain of consequences that results in a tornado many thousands of miles away.

The nature of such complex, dynamic, non-linear systems is that we may be able to say a lot about their general properties, while being quite unable to make specific predictions.  You will recognise this characteristic in the work of your Meteorological Office, which can tell you fairly reliably when spring will follow summer, or how much cooler or warmer it will be when you visit far-flung outposts, but which can never predict what the weather will be more than a few days ahead, or even with certainty what it will be like tomorrow.

Such limitations on our knowledge lead to a further problem.  Although people endlessly ask for predictions, they rarely really want the answers.  It was only late – too late – in life that I realised that when people said ‘we really want you to challenge our ideas’, they mostly didn’t.  They wanted instead to be congratulated on their wisdom.  Similarly, when they ask ‘what is going to happen?’ they seek reaffirmation and reassurance rather than insight into the future.  

The market for clairvoyants and clairvoyance has existed through history, and is satisfied by messages based on hope and ambiguity. The people who deliver such messages are characterised more by their interpersonal skills than by their prescience.  The market for economic prediction is similar.  Successful proponents are distinguished by their television manner rather than the accuracy of their forecasts.

I learnt during the new economy bubble that people preferred to be told they were right than to be told what would happen. Only when I quit business and academe to write, and did not have to placate clients or potential benefactors, could I openly say that the conventional wisdom was in error.  

Most people in the financial world enjoy no such freedom.  I believed that I would win kudos for my contrarian view when the bubble burst.  But that was a mistake.  People who had not wished to be told they were talking nonsense about the new economy before the bubble burst did not wish to be told that they had been talking nonsense about the new economy after the bubble burst either.  Indeed they generally did not recall that they had been talking nonsense.  Either they had known that it was a bubble all the time, or else they had been victims of events that could not have been predicted.  Frequently the same individual would make both claims.  And the same people would make the same inconsistent, and false, assertions when the credit bubble burst.

Shakespeare, traducing Richard III with the connivance of the first Queen Elizabeth, understood better than anyone that a good story is more compelling than the search for truth.  The American political scientist, Philip Tetlock, has studied the prognostications of pundits over several decades.  He finds that the best indication of the reliability of a prediction is the fame of the person who makes the prediction:  and the correlation is negative.  The better known the forecaster, the less accurate the forecast.  Business people, politicians and journalists value clarity and certainty of view more highly than acknowledgement of the uncertainty of a complex world.  But it is mostly people who appreciate that complexity who have worthwhile things to say about the future.

And that is where you have a role, Ma’am.  The foolish sovereign rewards the courtiers who tell her what she wants to hear:  the wise sovereign bestows her gratitude on those who struggle, however vainly, to reach the truth.  I respectfully hope you will bear that in mind when you graciously bestow your New Year Honours.