Michael Gove, UK education secretary, described Andreas Schleicher as the most important man in British education. He is the OECD official in charge of the Pisa tests of achievements by pupils in reading, mathematics and scientific understanding.
This month, the OECD produced its latest international comparisons. These results are so widely followed that they now attract equally wide criticism. The BBC devoted an entire programme to an attack on Pisa’s principles and methods.
There are several levels of controversy. Does performance on standardised tests really reflect the objectives of education? Students in some Asian countries are still doing homework at midnight, and attend private crammers to prepare for next week’s lessons. Perhaps children should be allowed to enjoy childhood. And maybe such an intense system stifles rather than fosters the creativity and originality on which economic progress depends. Shakespeare, Churchill and Steve Jobs might not have excelled on standardised tests.
But Pisa does not purport to record the attainment of genius. And Shakespeare would not have been able to produce masterpieces, nor Churchill inspire a nation, had they been unable to read and write. Basic competence in literacy and numeracy is necessary to participate effectively in the social and economic life of a modern society, even if the highest levels of cultural achievement and innovative capacity are not necessarily found in the societies that are most successful in instilling these skills throughout the population.
More technical criticism focuses on whether international comparisons are fair – or even meaningful. Reading Danish is not the same as reading Chinese. But comprehension of what is read has similar significance in Denmark and China.
The more profound difficulty is that comprehension is itself culturally specific. The difficulties Americans and French people have in understanding each other are not just difficulties of language. This issue looms far greater if the comparison is between Denmark and China, and become insuperable if the comparison is made (in Pisa it is not) between Danes and the tribes of the Kalahari. Yet to remove the influence of general knowledge – what we know from our everyday experience – from questions downgrades comprehension to an exercise in formal logic.
The problems accumulate. Even among Danes, students with different backgrounds or abilities will perform relatively better on some questions than others. There can be no objective basis for concluding which ones deserve a higher overall score.
Yet examiners have confronted issues such as these since education began. No doubt many classmates of Plato and Aristotle resented their high marks. To say comparisons of academic performance are always imperfect and open to revision is not to say they cannot be made at all. Any impartial observer should be impressed by the thorough attempt by the OECD experts to deal with problems in a scholarly and objective manner. That team deserves a sympathetic hearing when faced by critics who do not have better ideas of their own but simply think the questions legitimately raised by those concerned about educational standards should not be answered, or do not like the answers that emerge.
So we ought to take the results seriously, accord provisional credence to the findings and consider their policy implications. Recognise the superior performance, especially in science and maths, of many Asian students. Notice the relative decline of the Nordic countries, historically impressive European performers. Observe that devolution of responsibility for education in Britain to England, Scotland and Wales has given us a controlled test of the value of league tables and tests – and it seems that they help. Recognise that wide performance differences between US states seem to have little to do with differences in per pupil spending. And note that, although arguments for freeing schools from political control have gained much attention, the countries that do best in the Pisa tests mostly rely on strong centrally directed public education systems.
Pisa encourages us to assess education policy on the basis of objective research rather than ideological prejudice or our own experience of school. That is a big step forward.
