League tables are in vogue, and despite the difficulties they seem worthwhile. If only it were easy to produce a “value added” table.
It is the league table season. The FT has just produced its annual league of successful companies. The football league is reaching its climax. And last week, another set of educational league tables appeared, this time comparing the performance of primary schools in different local authorities. 85% of children in Rutland passed the official test in basic science, but barely half of those in Sandwell.
Yet, as teachers’ unions were quick to point out, there is a sense in which these educational comparisons are misleading. No-one is likely to be surprised that high scores were obtained in Richmond and in Surrey and low ones in Hackney and Tower Hamlets. Perhaps it is really more significant that Bury did very well and Bromley did not. The government plans value-added tables, to try to meet this criticism. Such a table would compare the actual performance of schools with the performance that could have been expected….
The best example of such a value-added table I know is one that Stefan Szymanski and Tim Kuypers, compiled for English football league clubs. They compared the actual performance of each team with the one which you would have expected from the quality of their players. They estimated the quality of the players by calculating how much each club had spent, in transfer fees and wages, to put them on the field.
So Liverpool and Manchester United had more or less equally good players, but Liverpool had done better. Southampton had not done as well as Manchester United, but – unlike Newcastle – had achieved more than would have been predicted from the club’s players.
The diagram emphasises the difference between the performance of a team – its absolute achievement – and its performance as a team – its achievement relative to its intrinsic quality. Manchester United may score more goals, but Southampton adds more value. In football, we are, to be honest, more concerned with performance than with added value. It is more exciting to watch Manchester United than the school second XI, even if the school second XI has a coach who works wonders with such unpromising material. And in education we are interested in both absolute and relative performance. A good school is one which makes the most of the talents of its students. But when we make choices for our children, it is its overall level of achievement that matters.
Now in business we are very interested in the quality of the coach. It is the capacity to add value that is the measure of the achievement, of an organisation. We admire Marks and Spencer for its ability to obtain consistently extraordinary results from ordinary people. And that is the essential characteristic of the great business. The key distinction is between the firm that truly adds value and the company which simply exploits the favourable environment it finds – the mineral deposit it had the good fortune to win a concession to, the monopoly of local water supply, the ‘licence to print money’ of the early television franchises.
But this emphasises how the football example gains its force from its simplicity. Football teams compete on a standardised basis regulated by the Football Association. There is a clear if subtle distinction between the quality of a team and the quality of the players – a feature which distinguishes football from, say, golf or even cricket, and which emphasises that football really is a team sport.
None of these things are true when we look at education, or at business. In neither case is there any standardisation of the terms of competition. All football teams contain eleven players. But tiny Rutland comes top of the schools league table: could it do as well if it were a much larger authority? Or should we – as we are inclined to do for corporations – simply judge the quality of a school by the number of A-levels its students get, whatever the scale of its entry?
And the sharp distinction between environment and performance blurs on close inspection. When Roy Thomson won ‘his licence to print money’ in Scottish Television, should we attribute the extraordinary profits he obtained to his managerial skills, or the favourable environment within which he operated? Was Bill Gates a far-seeing genius, or simply the man in the right place at the right time? And how should we view people like Rupert Murdoch, who seem to have an extraordinary talent for finding themselves in the right place at the right time?
And that is why, when I wrote at the beginning of this article that we needed a table which would compare the actual performance of schools with the performance that could have been expected, I wanted to complete the sentence – but could not. Could have been expected….. given the quality of their students. But what exactly do we mean by the quality of a student? Could have been expected … given the environment within which the school operated. But are the broken windows of the poor school, or the motivated pupils of the good school, characteristics of the school, or characteristics of its environment?
We need league tables, for football clubs, for schools, and for businesses, and they stimulate better performance in education and commerce just as they do in football. But, outside the football league itself, there is no single measure of performance, and no single league table that tells all, or even much, of what we need to know.