Why data, soft or hard, cannot replace eyes and ears


In all areas of human endeavour, there are hard data and soft data. The happiness of a society or the progress of a civilisation, are multi-dimensional: components are determined by subjective consensus, not objective measurement.

Last week, the Conference Board announced that European labour productivity growth had overtaken that of the US. The news that growth in Europe in 2006 was 1.5 per cent against 1.4 per cent across the Atlantic was the main headline in this paper.

While the reputation of economists has been in decline, naive acceptance and popular distribution of economic statistics has grown. Figures such as these are taken at face value, as if they were established facts. Markets tremble when an announcement differs from expectations by as little as a single decimal point.

But the margin of error in such data is an order of magnitude larger than these differences. Sampled data may not accurately represent the population. But there is a larger and more subtle problem: the difficulty of translating broad economic concepts, such as national income or productivity, into something that can be measured.

In all areas of human endeavour, there are hard data and soft data. Once you determine a scale for temperature, Fahrenheit or Celsius, every competent observer will arrive at the same answer. But the attractiveness of a face, the happiness of a society, the progress of a civilisation, is multi-dimensional: components of attractiveness, happiness or progress are determined by subjective consensus and are not susceptible to objective measurement.

Scientists and wannabee scientists such as economists resist the use of soft data. Lord Kelvin said that unless you can measure something, your knowledge is of a meagre kind. But Kelvin, although a great physicist, was a silly man and he was as wrong on this as in asserting that manned flight was impossible. A complex world can often only be described through soft data. Every generation has followers of Kelvin, who attempt to measure the unquantifiable and balance the incommensurable. Why we find faces attractive, what makes us happy and how civilisations progress are good questions, but to take measurement too far leads to absurdity. Rankings of national competitiveness, or the assessment of drugs through quality-adjusted life years, are pseudo-science, like listings of the greatest poems.

Some economic data are hard, some soft. The measurement of compensation is rather like the measurement of temperature. You can count the notes in the wages envelope and, while there is some room for argument round the edges, an objective number for payroll can be determined with reasonable accuracy.

But profits and productivity are soft data. Events at Enron and WorldCom may have finally dispelled the fallacy that a true measure of profit can be ascertained by diligent enquiry. Enron, like others, ruthlessly exploited ambiguities in the concept of profit to meet the numbers. The objective recorder is replaced by the accountant or economist who, closing the door, asks the client: “What number would you like?”.

Output seems like a hard number – and would be if it were simply a matter of counting the widgets that leave a production line. But the output of a modern economy is made up of thousands of differentiated products of changing quality and composition. The US productivity miracle was in part created, not by finding new facts about the US economy, but by reclassifying software expenditure as investment and adopting aggressive assumptions about falling computer prices.

The key number used to measure economic performance is gross domestic product. But few politicians or traders could actually define it. GDP is not, exactly, a measure of either business output or consumer welfare, although it is loosely related to both. It is safest to say that GDP is the number you arrive at if you follow an internationally agreed set of statistical conventions.

So long as everyone follows these conventions, movements in GDP tell you something about national prosperity and economic progress, even if it is not entirely clear what. But no economic data, hard or soft, can ever tell the whole story. Prosperity and progress are soft concepts and official statistics are at best a supplement, not a substitute, for evidence of eyes and ears.

Print Friendly, PDF & Email