The holiday season requires a close eye on the weather. Modern people do this, not by looking at the sky, but by consulting a smartphone. Yahoo, the most popular weather app, tells you the probability of rain, hour by hour.

But you don’t want to know the probability that it will rain. It either will rain or it won’t and you want to know which of these outcomes will occur. And you can never know whether such a prediction was accurate. Both rain and no rain are consistent with the claim that the probability that it would rain was 10 per cent.

The British meteorological office gives some help in interpretation. It suggests that if the probability of rain was 10 per cent you might be willing to hang your sheets out to dry, but not a shirt needed for an important dinner. While this may be useful advice, however, it is not an explanation of the meaning of the statement that the probability of rain is 10 per cent.

The analysis of probability originates in games of chance, in which the rules are sufficiently simple and well-defined that the game can be repeated in more or less identical form over and over again. If you toss a fair coin repeatedly, it will come up heads about 50 per cent of the time. If you can be bothered, you can verify that fact empirically. Perhaps more strikingly, the theory of probability tells you that if you repeatedly toss that coin 50 times you will get 23 or more heads about 67 per cent of the time, and you can verify that prediction empirically too.

It is a stretch, but perhaps not a very long stretch, to extend this analysis of frequency to single events, and to say that the probability that England will win the toss in the fifth Ashes cricket test against Australia is 50 per cent. And that the probability the home side will win the toss at least 23 times in a decade of five-test match series is 67 per cent. Tosses to start sporting contests are repeated at similar events, and theory and experience validate the probabilistic approach.

Perhaps one could stretch the approach further and apply it to the probability of rain. The Met Office might form a view of tomorrow’s weather. Records might show that it rains on 10 per cent of similar days. The difficulty is defining exactly what is meant by “a similar day”. “A typical April day in England” is a rather loose concept.

However, this is not, in fact, what weather forecasters do. Their analysis is based on elaborate computer models and they tweak the assumptions of these models to generate many different predictions. What they mean when they say the probability of rain is 10 per cent is that rain occurs in 10 per cent of these simulations. The validity of that prediction depends on two rather implausible assumptions; that the model correctly describes the physical world, and that the range of assumptions made by the modellers properly reflects the full range of possible assumptions.

Despite these difficulties – and popular derision comparable to that experienced by economic forecasters – weather forecasters do rather well. Certainly short-term weather forecasts are better than medium-term economic forecasts. These weather forecasting models are constantly calibrated and adjusted in light of the wealth of data that the weather provides hour by hour, and the physical system that governs the weather is relatively constant, even if our understanding of it is limited.

In contrast, severe recessions, property bubbles and bank failures are relatively infrequent, and calibration by economists has come to mean tweaking models to better explain the past rather than revising them to better predict the future – a particularly dangerous methodology when there are many reasons to think that the underlying structure of the economy is in a state of constant flux.

The further one moves from mechanisms that are well understood and events that are frequently repeated, the less appropriate is the use of probabilistic language. What does it mean to say: “I am 90 per cent certain that the extinction of the dinosaurs was caused by an object hitting the earth at Yucatán?” Not, I think, that on 90 per cent of occasions on which the dinosaurs were wiped out, the cause was an asteroid landing in what is now Mexico. There is a difference – often elided – between a probability and a degree of confidence in a forecast. It is one reason why we are better at avoiding drizzle than financial crises.