Uncategorized

Teams of foxes make the best forecasts, but expert hedgehogs can help

One of the banes of this economist’s life, and never more so than at the turn of the year, is the belief that economists is clairvoyance. I should be offering prognostications of what GDP growth will be in 2016 and when the central bank will raise interest rates.

Superforecasters, by Philip Tetlock and Dan Gardner, was one of the most interesting business and finance books of 2015.  Tetlock has for two decades collected specific predictions of events from a wide range of forecasters. In his earlier book Expert Political Judgment demonstrated, to few people’s surprise, that forecasters were not very good.  More surprising, however, was his identification of the characteristics of good and bad forecasters.

Tetlock employs the  distinction, due to the Greek poet Archilochus but popularised by Isaiah Berlin, between hedgehogs who know one big thing, and foxes who know many little things.   Hedgehogs tend to have an all encompassing world view, perhaps ideology,  and discover facts which confirm what they already know to be true.  Foxes are eclectic in their sources of information and nuanced in their judgments.  Hedgehogs command more public attention but foxes make better forecasters.

Harry S Truman reportedly (and perhaps apocryphally) sought a one-handed economist, who would not say ‘on the one hand – and on the other’.  But the two handed approach corresponds to the reality of most complex issues.  Yet modern business people, politicians and media seek what Truman wished for – and find it from hedgehogs.  To build attention and a reputation, it is more important to be unequivocal than to be right.

Superforecasters are the winners in an IARPA sponsored competition organised by Tetlock in which individuals and teams were asked to make competitive predictions about specific events.  His superforecasters were indisputably nerds.  They are diligent in pursuing sources of information, and ready to revise their predictions as more data becomes available.  They do not appear on Fox News, advise Bernie Sanders or Jeremy Corbyn; nor earn large speaking fees from business conferences.  But Tetlock not only found that his protégés tended to make good predictions, but that they became better with practice and that teams of superforecasters easily outperformed teams of selected experts.

It would be good to report that the services of these people are being widely sought by governments, businesses and financial institutions.   But they are not.  And perhaps for good(ish) reason.  The questions Tetlock poses to his subjects are – necessarily – highly specific:  otherwise it would not be possible to test the validity of the responses.  But ‘how many Syrian refugees will land in Europe in 2016?’ is at best a proxy for what we really want to know.  The underlying questions are looser and open-ended – how will the political situation in Syria and Iraq evolve?’ and ‘how will Europe respond to the refugee crisis?’.

These bigger questions cannot so readily be approached by the systemization produced by the superforecasters, and may lend themselves more to the narrative approach of the hedgehogs.  What was the right answer on 1st January 1989 to the question ‘will the Berlin Wall be pulled down in 1989?’   I think a shrewd commentator would have said (though few did) something like ‘almost certainly the wall will stand, but you should understand the potentially destructive forces currently undermining the Soviet engine and the East German state’.  That type of response combines probabilistic and narrative thinking.

But people long for certainties even as they know they cannot have them. I have learned that few people really want answers when they ask me to predict growth in GDP or advise whether interest rates will rise in the third quarter.  It is almost always easy to move the subject on to something more interesting than macroeconomic forecasting.