Skip to main content

Advertisement

Advertisement

They may not find a Jobs, but PISA tests lean in right direction

This month, the Organisation for Economic Co-operation and Development (OECD) produced its latest international comparisons of the Programme for International Student Assessment (PISA) tests of achievements by pupils in reading, mathematics and scientific understanding. These results are so widely followed that they now attract equally wide criticism. The BBC devoted an entire programme to an attack on PISA’s principles and methods.

This month, the Organisation for Economic Co-operation and Development (OECD) produced its latest international comparisons of the Programme for International Student Assessment (PISA) tests of achievements by pupils in reading, mathematics and scientific understanding. These results are so widely followed that they now attract equally wide criticism. The BBC devoted an entire programme to an attack on PISA’s principles and methods.

There are several levels of controversy. Does performance on standardised tests really reflect the objectives of education? Students in some Asian countries are still doing homework at midnight and attend private crammers to prepare for next week’s lessons.

Perhaps children should be allowed to enjoy childhood. And maybe such an intense system stifles rather than fosters the creativity and originality on which economic progress depends. Shakespeare, Churchill and Steve Jobs might not have excelled on standardised tests.

But PISA does not purport to record the attainment of genius. And Shakespeare would not have been able to produce masterpieces, nor Churchill inspire a nation, had they been unable to read and write.

Basic competence in literacy and numeracy is necessary to participate effectively in the social and economic life of a modern society, even if the highest levels of cultural achievement and innovative capacity are not necessarily found in the societies that are most successful in instilling these skills throughout the population.

More technical criticism focuses on whether international comparisons are fair — or even meaningful. Reading Danish is not the same as reading Chinese. But comprehension of what is read has similar significance in Denmark and China.

The more profound difficulty is that comprehension is itself culturally specific. The difficulties Americans and French people have in understanding each other are not only difficulties of language. This issue looms far greater if the comparison is between Denmark and China, and become insuperable if the comparison is made (in PISA it is not) between Danes and the tribes of the Kalahari.

Yet, to remove the influence of general knowledge — what we know from our everyday experience — from questions downgrades comprehension to an exercise in formal logic.

WHY WE SHOULD TAKE THE RESULTS SERIOUSLY

The problems accumulate. Even among Danes, students with different backgrounds or abilities will perform relatively better on some questions than others. There can be no objective basis for concluding which ones deserve a higher overall score.

Yet, examiners have confronted issues such as these since education began. No doubt many classmates of Plato and Aristotle resented their high marks. To say comparisons of academic performance are always imperfect and open to revision is not to say they cannot be made at all.

Any impartial observer should be impressed by the thorough attempt by the OECD experts to deal with problems in a scholarly and objective manner. That team deserves a sympathetic hearing when faced by critics who do not have better ideas of their own, but simply think the questions legitimately raised by those concerned about educational standards should not be answered, or do not like the answers that emerge.

So, we ought to take the results seriously, accord provisional credence to the findings and consider their policy implications. Recognise the superior performance, especially in science and maths, of many Asian students. Notice the relative decline of the Nordic countries, historically impressive European performers.

Recognise that wide performance differences between American states seem to have little to do with differences in per-pupil spending. And note that, although arguments for freeing schools from political control have gained much attention, the countries that do best in the PISA tests mostly rely on strong centrally-directed public education systems.

PISA encourages us to assess education policy on the basis of objective research rather than ideological prejudice or our own experience of school. That is a big step forward.

THE FINANCIAL TIMES LIMITED

ABOUT THE AUTHOR:

John Kay is a visiting professor at the London School of Economics and the author of many books, the latest being Obliquity: Why Our Goals Are Best Achieved Indirectly.

Read more of the latest in

Advertisement

Advertisement

Stay in the know. Anytime. Anywhere.

Subscribe to get daily news updates, insights and must reads delivered straight to your inbox.

By clicking subscribe, I agree for my personal data to be used to send me TODAY newsletters, promotional offers and for research and analysis.