This study investigates the effect of mode of administration of the Raven Standard Progressive Matrices test on distribution, accuracy, and meaning of raw scores. A random sample of high school students take counterbalanced paper-and-pencil and computer-based administrations of the test and answer a questionnaire surveying preferences for computer-delivered test administrations. Administration mode effect is studied with repeated measures multivariate analysis of variance, internal consistency reliability estimates, and confirmatory factor analysis approaches. Results show a lack of test mode effect on distribution, accuracy, and meaning of raw scores. Participants indicate their preferences for the computer-delivered administration of the test. The article discusses findings in light of previous studies of the Raven Standard Progressive Matrices test.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. ( 1999). Standards for Educational and Psychological Testing. Washington, DC: Author.
2.
Bandalos, D., & Finney, S. ( 2001). Item parceling issues in structural equating modeling . In G. Marcoulides & R. Schumacker (Eds.), New developments and techniques in structural equation modeling (pp. 269-296). Mahwah, NJ : Lawrence Erlbaum.
3.
Bentler, P., & Bonett, D. ( 1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.
4.
Bollen, K. ( 1989). Structural equations with latent variables. New York: John Wiley.
5.
Boo, J., & Vispoel, W. ( 1998, April). Computer versus paper-pencil assessment of educational development: Score comparability and examinee preference. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
6.
Browne, M., & Cudeck, R. ( 1993). Alternative ways of assessing model fit. In K. Bollen & J. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
7.
Bunderson, V., Inouye, D., & Olsen, J. ( 1989). The four generations of computerized educational measurement . In R. L. Linn (Ed.), Educational measurement (pp. 367-407). Phoenix, AZ: Onyx.
8.
Calvert, E., & Waterfall, R. ( 1982). A comparison of conventional and automated administrations of Raven’s Standard Progressive Matrices. International Journal of Man-Machine Studies, 17, 305-310.
9.
Choi, I., Kim, K., & Boo, J. ( 2003). Comparability of a paper-based language test and a computer-based language test. Language Testing, 20, 295-320.
10.
Chua, S., Chen, D., & Wong, A. ( 1999). Computer anxiety and its correlates: A meta-analysis . Computers in Human Behavior, 15, 609-623.
11.
Dillon, A. ( 1994). Designing usable electronic test. London: Taylor & Francis.
12.
Fan, X., & Thompson, B. ( 2001). Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517-531.
13.
Henson, R.K. ( 2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
14.
International Test Commission. (2005). International guidelines on computer-based and Internet delivered testing . Retrieved February 16, 2006, from http://www.intestcom.org/guidelines
15.
Jensen, A. ( 1998). The g factor. Westport, CT: Praeger.
16.
Kubinger, K.D., Formann, A.K., & Farkas, M.G. ( 1991). Psychometric shortcomings of Raven’s Stan dard Progressive Matrices, in particular for computerized testing. Revue Européenne de Psychologie Appliquée, 41, 295-300.
17.
Lee, J. ( 1986). The effect of past computer experience on computer aptitude test performance. Educational and Psychological Measurement , 46, 727-733.
18.
Leeson, H. ( 2006). The mode effect: A literature review of human and technology issues in computerized testing. International Journal of Testing , 6, 1-24.
19.
Levine, T., & Donitsa-Schmidt, S. (1998). Computer use, confidence, attitude, and knowledge: A causal analysis. Computer in Human Behavior, 14, 125-146.
20.
Luecht, R., Hadadi, A., Swanson, D., & Case, S. ( 1998). Testing the test: A comparative study of a comprehensive basic science test using paper-and-pencil and computerized formats. Academic Medicine, 73, 51-53.
21.
Lynn, R., Backhoff, E., & Contreras, L. ( 2004). Sex differences on the Progressive Matrices among 7 to 10 year olds: Some normative data for Mexico. Personality and Individual Differences, 36, 779-789.
22.
McDonald, A. ( 2002). The impact of individual differences on the equivalence of computer-based and paper-and-pencil educational assessments. Computers and Education, 39, 299-312.
23.
Mead, A., & Drasgow, F. ( 1993). Equivalence of computerized and paper and pencil cognitive ability test: A meta-analysis. Psychological Bulletin, 114, 449-458.
24.
Newman, G., & Baydoun, R. ( 1998). Computerization of paper-and-pencil tests: When are they equivalent? Applied Psychological Measurement, 22, 71-83.
25.
Pommerich, M., & Burden, T. ( 2000, April). From simulation to application: Examinees react to computerized testing. Paper presented at the annual meeting of the National Council on Measurement in Education. New Orleans, LA.
26.
Raven, J. ( 2000). The Raven’s Progressive Matrices: Change and stability over culture and time. Cognitive Psychology, 41, 1-48.
27.
Raven, J., Raven, J.C., & Court, J. ( 1993). Test de matrices progresivas: manual para la aplicaciän .
28.
[Standard progressive matrices test: Direction for administration manual]. Buenos Aires: Paidäs. Rushton, J. (2002). Jensen effects and African/Colored/Indian/White differences on Raven’s Standard Progressive Matrices in South Africa. Personality and Individual Differences, 33, 65-70.
29.
Rushton, J., Skuy, M., & Bons, T. ( 2004). Construct validity of Raven’s Advanced Progressive Matrices for African and Non-African Engineering Students in South Africa . International Journal of Selection and Assessment, 12, 220-229.
30.
Spray, J., Ackerman, T., Reckase, M., & Carlson, J. ( 1989). Effect of the medium of item presentation on examinee performance and item characteristics. Journal of Educational Measurement, 26, 261-271.
31.
Sukigara, M. ( 1996). Equivalence between computer and booklet administrations of the new Japanese version of the Minnesota Multiphasic Personality Inventory . Educational and Psychological Measurement, 56, 570-585.
32.
Taylor, C., Kirsch, I., Eignor, D., & Jamieson, J. ( 1999). Examining the relationship between computer familiarity and performance on computer-based language tasks. Language Learning, 49, 219-274.
33.
Van der Ven, A., & Ellis, L. ( 2000). A Rasch analysis of Raven’s Standard Progressive Matrices. Personality and Individual Differences, 29, 45-64.
34.
Vispoel, W. ( 2000). Computerized versus paper-and-pencil assessment of self-concept: Score comparability and respondent preferences. Measurement and Evaluation in Counseling and Development, 33, 130-143.
35.
Williams, J., & McCord, D. (2006). Equivalence of standard and computerized versions of Raven Progressive Matrices Test. Computers in Human Behavior, 22, 791-800.
