International Large-Scale Assessments (ILSAs) have been used to draw comparisons among countries on a variety of topics in education and, more broadly, for example, in adolescent development. These assessments can inform the public about influential factors on the micro and macro levels, foster interdisciplinary and international collaboration, and provide important data for studying the context and processes of education and development. In short, their potential is great. However, there are also challenges, in particular with the validity of using country rankings to estimate educational achievement of students within countries and the unintended consequences of this use. Promising directions for analysis and reporting are identified in this article – including secondary analysis informed by a conceptual framework that focuses on the context and processes of education. Examples of secondary analysis using data from an ILSA in civic education are given.
References
1.
AmadeoJ.Torney-PurtaJ.LehmannR.HusfeldtV. & NikolovaR. (2002) Civic Knowledge and Engagement: an IEA study of upper secondary students in sixteen countries. Amsterdam: IEA. http://www.terpconnect.umd.edu/∼iea
2.
BerlinerD. (2011) The Context for Interpreting PISA Results in the USA: negativism, chauvinism, misunderstanding and the potential to distort the educational systems of nations, in PereyraM.A.KotthoffH.-G. & CowenR. (Eds) PISA Under Examination: changing knowledge, changing schools, pp. 77–96. Rotterdam: Sense Publishers.
BronfenbrennerU. (1989) The Ecology of Human Development: experiments by nature and design. Cambridge, MA: Harvard University Press.
5.
CarnoyM. & RothsteinR. (2013) What Do International Tests Really Tell Us about US Student Performance?Washington, DC: Economic Policy Institute.
6.
ChatterjiM., (Ed.) (2013) Validity and Test Use: an international dialogue on educational assessment, accountability and equity. Bingley: Emerald.
7.
DeBoerJ. (2010) Why the Fireworks? Theoretical Perspectives on the Explosion in International Assessments, in WisemanA. (Ed.) The Impact of International Assessment Studies on National Education Policy Making, pp. 297–330. Bingley: Emerald Publishing.
8.
DonnellanM.TrzesniewskiK. & LucasR. (2011) Introduction, in TrzewniewskiK.DonnellanM. & LucasR. (Eds) Secondary Data Analysis: an introduction for psychologists, pp. 3–10. Washington, DC: American Psychological Association.
9.
EngelL.WilliamsJ. & FeuerM. (2012) The Global Context of Practice and Preaching: do high-scoring countries practice what US discourse preaches? Working Paper 2–3. Washington, DC: George Washington University.
10.
ErcikanK.RothW.-M. & AsilM. (in press) Cautions about Two Typical Uses of International Assessment Data: the case of PISA 2009, Teachers College Record.
11.
FeuerM. (2006) Moderating the Debate: rationality and the promise of American education. Cambridge, MA: Harvard University Press.
12.
FeuerM. (2013) Validity Issues in International Large-scale Assessment: truth and consequences, in ChatterjiM. (Ed.) Validity and Test Use: an international dialogue on educational assessment, accountability and equity, pp. 197–215. Bingley: Emerald.
13.
FleischmanH.L.HopstockP.J.PelczarM.P. & ShelleyB.E. (2010) Highlights from PISA 2009: performance of US 15-year-old students in reading, mathematics, and science literacy in an international context, NCES 2011–004. US Department of Education, National Center for Education Statistics. Washington, DC: US Government Printing Office.
14.
GonzalesP.WilliamsT.JocelynL.RoeyS.KastbergD. & BrenwaldS. (2008) Highlights from TIMSS 2007: mathematics and science achievement of US fourth- and eighth-grade students in an international context, NCES 2009–001 Revised. National Center for Education Statistics, Institute of Education Sciences, US Department of Education. Washington, DC: US Government Printing Office.
15.
HiebertJ.StiglerJ.W.JacobsJ.K. (2005) Mathematics Teaching in the United States Today (and Tomorrow): results from the TIMSS 1999 Video Study, Educational Evaluation and Policy Analysis, 27(2), 111–132. http://dx.doi.org/10.3102/01623737027002111
16.
HopmannS.T.BrinekG. & RetzlM. (Eds) (2007) PISA According to PISA: does PISA keep what it promises?Vienna: Lit Verlag.
17.
InglehartR. & WelzelC. (2005) Modernization, Cultural Change and Democracy: the human development sequence. Cambridge: Cambridge University Press.
KaneM.T. (2013) Validity and Fairness in the Testing of Individuals, in ChatterjiM. (Ed.) Validity and Test Use: an international dialogue on educational assessment, accountability and equity, pp. 17–59. Bingley: Emerald.
MislevyR.J. (2009) Validity from the Perspective of Model-based Reasoning, in LissitzR.L. (Ed.) The Concept of Validity: revisions, new directions and applications, pp.83–108. Charlotte, NC: Information Age Publishing.
24.
NyroosM. & Wiklund-HörnqvistC. (2012) The Association between Working Memory and Educational Attainment as Measured in Different Mathematical Subtopics in the Swedish National Assessment: primary education, Educational Psychology, 32(2), 239–256. http://dx.doi.org/10.1080/01443410.2011.643578
RutkowskiL.GonzalezE.JoncasM. & von DavierM. (2010) International Large-scale Assessment Data: issues in secondary analysis and reporting, Educational Researcher, 39(2), 142–151. http://dx.doi.org/10.3102/0013189X10363170
27.
SchulzW.AinleyJ.FraillonJ.KerrD. & LositoB. (2011) Civic Knowledge, Attitudes, and Engagement Among Lower-secondary School Students in 38 Countries. Amsterdam: IEA.
28.
Torney-PurtaJ. (1990) International Comparative Research in Education: its role in educational improvement in the US, Educational Researcher, 19, 32–35.
29.
Torney-PurtaJ. & AmadeoJ. (2006) Strengthening Democracy in the Americas through Civic Education. Washington, DC: Organization of American States.
30.
Torney-PurtaJ. & AmadeoJ. (2011) An International Perspective on Developmental Niches for Emergent Citizenship in Early Adolescence, Annals of the American Academy of Political and Science, vol. 663, ‘The Child as Citizen’, 180–200.
31.
Torney-PurtaJ. & AmadeoJ. (2012) The Contribution of International Large-scale Studies in Civic Education and Engagement, in von DavierM.GonzalezE.KirschI. & YamamotoK. (Eds) The Role of International Large Scale Assessments: perspectives from technology, economy, and educational research, pp. 87–114. Dordrecht: Springer.
32.
Torney-PurtaJ.AndolinaM. & AmadeoJ. (2010) A Conceptual Framework and Multi-method Approach for Research on Political Socialization and Civic Engagement, in SherrodL.Torney-PurtaJ. & FlanaganC. (Eds) Handbook of Research on Civic Engagement in Youth, pp. 497–523. Hoboken, NJ: John Wiley.
33.
Torney-PurtaJ.LehmannR.OswaldH. & SchulzW. (2001) Citizenship and Education in Twenty-eight Countries: civic knowledge and engagement at age fourteen. Amsterdam: IEA. http://www.terpconnect.umd.edu/∼iea
34.
Torney-PurtaJ.WilkenfeldB. & BarberC. (2008) How Adolescents in Twenty-seven Countries Understand International Human Rights, Journal of Social Issues, 64(4), 857–880. http://dx.doi.org/10.1111/j.1540-4560.2008.00592.x
35.
Van de VijverF.J.R.ChasiotisA. & BreugelmansS. (2011) Fundamental Questions in Cross-cultural Psychology. Cambridge: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511974090
36.
Van De VliertE. (2011) Using Secondary Data to Advance Cross-cultural Psychology, in TrzewniewskiK.DonnellanM. & LucasR. (Eds) Secondary Data Analysis: an introduction for psychologists, pp. 177–191. Washington, DC: American Psychological Association.
37.
Van HemertD.A. (2011) Frameworks for Explaining Cross-cultural Variance: a meta-analytic examination of their usefulness, in van de VijverF.J.R.ChasiotisA. & BreugelmansS.M. (Eds) Fundamental Questions in Cross-Cultural Psychology, pp. 312–340. Cambridge: Cambridge University Press.
38.
WatermanR. & KliemeE. (2002) Reporting Results of Large-scale Assessment in Psychologically and Educationally Meaningful Terms, European Journal of Psychological Assessment, 18(3), 190–203. http://dx.doi.org/10.1027//1015-5759.18.3.190
39.
WentzelK.R. (2006) Developing and Nurturing Interesting and Researchable Ideas, in ConradC. & SerlinR. (Eds) The Sage Handbook for Research in Education: engaging ideas and enriching inquiry, pp. 315–330. Thousand Oaks, CA: Sage.
40.
WilkenfeldB. & Torney-PurtaJ. (2012) A Cross-context Analysis of Civic Engagement Linking CIVED and US Census Data, Journal of Social Science Education, 11(1), 64–82.
41.
WisemanA.W. (2013) Policy Responses to PISA in Comparative Perspective, in MeyerH.-D. & BenavotA. (Eds) PISA, Power and Policy: the emergence of global educational governance, pp. 303–324. Oxford: Symposium Books.
42.
YoshikawaH.WeisnerT.KililA. & WayN. (2008) Mixing Qualitative and Quantitative Research in Developmental Science, Developmental Psychology, 44(2), 344–354. http://dx.doi.org/10.1037/0012-1649.44.2.344
