John Maynard Keynes, who never tried to conceal that he knew more than most people, also knew the limits to his knowledge. He wrote “about these matters – the prospect of a European war, the price of copper 20 years hence – there is no scientific basis on which to form any calculable probability whatever. We simply do not know.”
And Keynes was right. He published these observations in 1921, and 20 years later Britain was engaged in a desperate, and unpredictable, struggle with Germany.
But lesser men find prognostication easier. I have been looking at some of the models people use, in both the public and private sectors to predict events.
The models share a common approach. They pose the question: “How would we make our decision if we had complete knowledge of the world?” With such information you might make a detailed assessment drawing together many different pieces of relevant information on matters such as costs, benefits, and consequences.
But little of this knowledge exists. So you make the missing data up. You assume the future will be like the past, or you extrapolate a trend. Whatever you do, no cell on the spreadsheet may be left unfilled. If necessary, you put a finger in the air.
This may lead to extravagant flights of fantasy. To use Britain’s Department of Transport scheme for assessing projects, you have to impute values of time in 13 different activities, not just today, but in 2053. Fortunately, you can download answers to these questions from the official website. And answers to many others you probably did not know you wanted to ask. What will be average car occupancy rates, differentiated by time of day, in 2035?
The impression of rationality these procedures convey is spurious. Because so many inputs to the analysis are invented, they can be chosen with a view to the desired result. With monotonous regularity, the Private Finance Initiative route is found to offer slightly better value for money than the public sector comparator. Financial analysts have been told to rework their figures to come up with the right answer sufficiently often that it is now rarely necessary to give them this advice.
The future is assumed to be essentially like the present, with differences mainly derived from mechanical projection of current developments. Uncertainty is ignored, or dealt with in unsatisfactory ways.
Sometimes the analysis will offer probability judgments – typically derived by tweaking some of the arbitrary assumptions and seeing how many of the scenarios generated fall within specified limits. As will the stress tests carried out on Europe’s banks recently. The only information exercises such as these convey is the limits of the imagination of the people who have undertaken them.
Yet the mistaken belief persists that these procedures provide an objective basis for decision making. A consultancy industry has developed that has expertise in particular models. It is almost essential for project sponsors to appoint those companies. To their great profit, the consultancies have developed modelling into a scalable business in which junior analysts can be used to input data.
Models are often useful in illuminating complex problems and quantification is an essential part of decision making. But good models are simplifications, not black boxes whose workings are incomprehensible even to their operators. The relevant model is always specific to the task at hand and there is no objective method of determining the right tool to employ in any particular case. If you do not know the answer to a question, the right response is not to make a number up, but to rethink and frame an alternative question that is capable of being answered.
We do great damage by claiming to know things that are not known, by asserting certainty in the face of uncertainty and ambiguity, and by attaching a veneer of rationality to decisions that have in fact been made on other, rarely articulated, grounds. The paradoxical result is all too obvious. The public sector and large bureaucratic organisations appear as paragons of good decision making process and exemplars of bad decisions.
