The Failures of Economic Forecasting

1453

It is a conventional joke that economic forecasters always disagree, and that there are as many different opinions about the future of the economy as there are economists. The truth is quite the opposite. Economic forecasters do not speak with discordant voices; they all say more or less the same thing at the same time. And what it is that they say is almost always wrong. The differences between forecasts are trivial relative to the differences between all forecasts and what actually happens.

My assessment of the performance of forecasts is based on an analysis by London Economics of the performance of all major forecasting groups since 1987. The forecasts used are the latest made, so that for 1990 I have selected the last forecast made by each group in 1989. Take their forecasts of growth in the economy in 1994, for example. We now know that it was around 4 per cent. The best estimates were made by Patrick Minford of Liverpool University, and a commercial firm, Business Strategies Limited. Both predicted that the economy would grow by 3.3 per cent. But these were not just the best forecasts. They were also the highest. Of the thirty-four forecasts I have obtained, every one was substantially below the outcome. In fact, almost all were between 2 per cent and 3 percent.

Perhaps 1994 was a bad year for forecasters, if not for the economy. It was not an untypical year, however. The same thing happened in 1993. Growth then was 2 per cent. One forecasting group – the National Institute – was spot on. Every other forecaster – every single one – was below the outcome.

Perhaps forecasters are by nature pessimistic. But they were not in 1992. Growth was actually negative that year. That outcome was worse than any one had predicted. And the same was true in 1991. In not one of these four years did the outcome lie within the range of all the forecasts made (Table 1).

Table 1: Growth forecasts, 1991-4 (%)

1991

1992

 

1993

1994

Average forecast

0.8

1.9

1.0
2.6

Outcome

-2.5

-0.6

2.0
3.9

Number of forecasts too high

29

30

0

0

Number of forecasts too low

0

0

29
34

It was dissatisfaction with the outcome of forecasts, and particularly his own, that led the Chancellor to establish his panel of independent economic forecasters – the so-called Wise Men. He need not have bothered. In both the last two years, the average forecast from the Wise Men has been exactly equal to the Treasury’s own forecast, and exactly equal to the average of all forecasters (Table 2). The degree of agreement among the forecasts is astounding. It is just the economy that is different.

Table 2: Growth forecasts, 1993-4(%)

1993

1994

 

Treasury

1.0

2.7

Average of Wise Men

1.0

2.7

Average of all forecasters

1.0

2.6

Outcome

2.0

3.9

Perhaps the forecasters do better with other economic variables. Table 3 shows how they did on the retail prices index. In 1993 and 1994, virtually everyone over-estimated inflation. In 1991, virtually everyone under-estimated it. Only in 1992 – oddly enough, the most dramatic year of the four, the one in which Britain was forced out of the ERM – were the forecasts close to outcome. Even a stopped clock is right some times.

Table 3: Inflation forecasts, 1991-4 (%)

1991

1992

 

1993

1994

Average forecast

5.2

4.0

3.3
3.7

Outcome

5.9

3.7

1.6
2.7

Number of forecasts too high

3

15

26

29

Number of forecasts too low

20

8

0
1

Nor have the last four years been worse than usual for the forecasting profession. They did no better in the 1980s (Table 4). And although 1995 is only two-thirds over, it already looks likely that the consensus will be too high on both inflation and growth.

Table 4: Growth forecasts, 1987-90 (%)

1987

1988

 

1989

1990

Average forecast

2.8

2.5

2.5
1.7

Outcome

4.5

4.2

2.1
1.0

Number of forecasts too high

0

0

15

18

Number of forecasts too low

17

19

3
3

There is a consensus forecast, to such a degree that it is barely worth distinguishing between one forecast and another. Yet the consensus forecast failed to predict any of the most important developments in the economy over the past seven years – the strength and resilience of the 1980’s consumer spending boom, the depth and persistence of the 1990’s recession, or the dramatic and continuing decline in inflation since 1991.

Now there are several reasons for this clustering around a consensus. While some forecasts – such as those of the Treasury, the National Institute and London Business School – are based on elaborate econometric models of the economy, many City and business forecasts are based only on an assessment of the opinions and forecasts of others. So it is not surprising that they are not far apart. And it is always safer to be wrong in a crowd. It is striking that the commercial and City forecasters, whose jobs may be on the line, rarely stray far from the consensus, while the outliers are more often academics. Patrick Minford and Wynne Godley are no more often right than other people, but they are more likely to be different from other people – so Minford’s growth forecast won the golden guru award in 1990 and 1994, but the wooden spoon in 1992.

Even in retrospect, it is important to maintain the consensus. It is an article of faith among banks, for example, that the depth of the recent recession and the magnitude of the property market collapse could not have been predicted, since if it could have been predicted those responsible for the lending excesses of the 1980s would be guilty of gross negligence rather than helpless victims of events. Tim Congdon, whose maverick predictions anticipated the events of the last boom and recession better than anyone, parted company with Shearson Lehman just as events began to prove him right. In large organisations, it is often more important to be wrong for the right reasons than to be correct, and nowhere is this more true than in the Treasury and Bank of England.

But another reason for the near identity of all the major forecasts can be found by looking more carefully at Figures 1 and 2. If you want to add to the forecasts by predicting what the consensus forecast will be, it is not very difficult. The consensus forecast can be derived by taking the average of the present and the past. Today, inflation and interest rates are at historically low levels, and so the consensus is that they will rise. Growth is pretty much in line with its historic average, and that is why most forecasters think that it will stay there. Many so-called forecasters derive their predictions in this way, and if they use the same principle and the same method it is no surprise that they come up with the same answer. What is less obvious is that the Treasury model, and other systems like it, have the same property. In the absence of external shocks, or after them, they revert quickly to the long run trend.

Now if you know very little about what you are forecasting – and when we make macro-economic predictions we do know very little about what we are forecasting – then there are worse rules of thumb than expecting that the future will be like the past, although it is difficult to see why anyone should command the deference accorded to the Chancellor’s Wise Men or the salaries paid to City economists for enunciating this principle. Yet the fundamental weakness of the approach is that it is intrinsically incapable of identifying structural changes in the economy. Changes in asset prices played a role in the boom of the 1980s and recession of the 1990s which had not been seen in previous economic cycles. It was that phenomenon which the consensus forecast almost entirely missed. And the most important current economic policy issue is whether a combination of changes in expectations and micro-economic deregulation has finally brought to an end the 50 year age of inflation. I do not know the answer to that question. But I do know that the consensus forecast, which predicts that inflation will rise because in the past it always has, sheds absolutely no light on the matter.

The boxes explain why reliable macro-economic forecasts for more than a short period ahead are, in principle, impossible. Despite that, they will continue to be made, just as astrologers and quack doctors stay in business. The hope that what they say might be true overrides innate scepticism, and despite the low opinion which both politicians and business people profess for economic forecasts, they continue to listen to them with extraordinary credulousness. Yet once we realise the limitations of our knowledge, there is much that can more sensibly be said. The fall in inflation, and the revival of manufacturing – two current structural changes – each have wide ranging effects on business and finance, and economic analysis can illuminate what they are. But when someone tells you what inflation will be in the third quarter of 1996, or predicts the growth of manufacturing output in 1997, do not listen. He does not know.


How Chaos Theory Undermines Macro-economic Forecasting

Chaos theory is currently one of the most fashionable branches of mathematics – to be found in Stephen Spielberg’s Jurassic Park and Tom Stoppard’s Arcadia as well as in more prosaic textbooks. One of its key insights is that in non-linear systems, which have curves or discontinuities – which certainly include economies and businesses – small differences in initial conditions may lead to large differences in final outcomes.

Consider the effect of dropping a marble into the system below. Whether it ends up at A, B or C is a matter of chance – strictly, it depends on the angle and speed at which you drop the marble. These small differences determine whether what happens is A, B or C. There is no way of predicting where it will end up, but if you are forced to make a prediction, you should guess C, because it will end up there twice as often as at A or B. This is the phenomenon of the consensus forecast.

You can, of course, hire someone who guessed right last time – usually someone who guessed C. You can hire a panel of wise men to pronounce on why they were right, or wrong, last time. Or you can recognise that the marble will end up one quarter of the time at A, one quarter at B, and the rest of the time at C, and that no one can tell you what will be the outcome this time around. Then prepare a strategy which copes with these different outcomes and which reflects the probabilities they will occur. That is what we recommend – both to business people and to Chancellors of the Exchequer.

Britain’s exit from the ERM in September 1992 raises similar issues. That Britain might be forced out at some time was quite likely. The exact timing of the debacle was the product of chance events – a run on the Lire, statements from the Bundesbank, uncertainty about the outcome of the French referendum. Even in summer 1992, no one could have predicted with any confidence that nemesis would come in the autumn, rather than in November or the following spring. In turn, no one could have predicted the course of interest rates, growth or inflation in 1993. In practice, most forecasters simply assume such discontinuities as ERM exit will not occur – which is, in part, why they are so often the same and so often wrong.


Macro-economic Models

A macro-economic model of the economy is a consistent system of relationships, based on past data, which describes the interactions between the main economic variables. For example, investment expenditures by business might depend on the rate of growth of demand for output, the level of interest rates, and on the cash flow of firms; wages might be affected by unemployment and by inflation expectations, which might in turn be determined by past inflation. Each of these elements of the model is validated by examining how well it would have predicted the variable concerned – investment or wages – in previous years.

Structural changes in the economy can raise major problems for this approach. For example, there has been a strong relationship over 20 years between house prices and average incomes. When the affordability ratio has been attractive, house prices have gone up; when the ratio of prices to incomes is too high, they rise less quickly. But this relationship was observed in a period when house prices always went up, so that what you would pay for a house was often limited only by what you could borrow.

So will this historic relationship persist in an era of lower inflation? No one really knows. Ideally, one could construct a mega-model, which delved deeper into underlying behaviour and would describe both inflationary and non-inflationary periods. But most models are very far from achieving this degree of sophistication. Until then, however, all macro-economic models will be biased towards expecting the future to be like the past and will often fail to identify the most important changes in economic behaviour. And even then, we may not be much better off – see box 1.


Print Friendly, PDF & Email