Issues with PISA's Use of its Data in the Context of International Education Policy Convergence

Abstract

In its analysis of data it collected in 2006, the Programme for International Student Assessment (PISA) survey by the Organisation for Economic Co-operation and Development (OECD) used a variety of statistical methods to arrive at the ‘key findings' that school autonomy, school competition and the public posting of student achievement data were associated with higher student performance. The application of one of the methods PISA used — cross-country correlations — to a wider group of school organisation policies found that the consequential accountability measures of using student achievement data to evaluate teachers and allocate resources to schools were associated with worse student performance. PISA applied the cross-country correlation method more consistently to its 2009 data but did not include in its analysis the relationship between using student achievement data to evaluate teachers and student performance. Secondary analysis revealed significant negative relationships between the two. This selective use of data over 2006 to 2009 raises questions about the objectivity of the OECD and PISA and bolsters the views of critics who believe that PISA actively favours convergence around certain schooling policies.