The Score Equivalence of Paper-and-Pencil and Computerized Versions of a Speeded Test of Reading Comprehension

Abstract

This study investigated the equivalence of scores from computerized and paper-and-pencil versions of a reading placement test. Concerns about score equivalence on the computerized versions were warranted because of the speeded nature of the paper-and-pencil version and differences in text delivery and response modes. The results indicated that both forms of the computerized versions produced higher vocabulary scores than the paper-and-pencil format and one form also had higher comprehension and total scores on the computerized version. These difficulty differences, especially for the vocabulary scores, appeared related to the differences in response speed associated with use of a mouse to record responses in contrast to a pencil and answer sheet. Scale scores for the computerized versions had similar predictive power for course placement as paper-and-pencil scores. However, because these results were based on students from only seven institutions, additional studies are needed to investigate the comparability of these versions.

Get full access to this article

View all access options for this article.

References

American Psychological Association . (1986). Guidelines for computer-based tests and interpretations. Washington, DC: Author.

Barnett, V. , & Lewis, T. (1984). Outliers in statistical data. New York: Wiley.

Brown, J. I. , Fishco, V. V. , & Hanna, G. (1993). Nelson-Denny Reading Test: Technical report Forms G&H. Chicago: Riverside.

Clansing, C. , & Schmitt, D. (1990). Paper versus CRT: Are reading rate and comprehension affected? In Proceedings of selected paper presentations at the convention of the Association for Educational Communications and Technology. (ERIC Document Reproduction Service No. ED323924)

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.

Freeberg, N. E. , Rock, D. A. , & Pollack, J. (1989). Analysis of the revised Student Descriptive Questionnaire, Phase II–Predictive validity of academic self-report. (College Board Report No. 89-8 and ETS Research Report 89-49). New York: College Entrance Examination Board.

Glass, G. V. , & Hopkins, K. D. (1996). Statistical methods in education and psychology. Boston: Allyn & Bacon.

Gould, J. D. , Alfaro, L. , Finn, R. , Haupt, B. , & Minuto, A. (1987). Reading from CRT displays can be as fast as reading from paper. Human Factors, 29, 497-517.

Gould, J. D. , & Grischkowsky, N. (1983). Doing the same work with paper and cathode ray tube displays (CRT). Human Factors, 24, 329-338.

10.

Kolen, M. J. , & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer.

11.

Mazzeo, J. , & Harvey, A. (1988). The equivalence of scores from automated and conventional educational and psychological tests. College Board Report 88-8. New York: College Entrance Examination Board.

12.

Mead, A. D. , & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114, 449-458.

13.

Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.

14.

Swineford, F. (1974). The test analysis manual (SR-74-06). Princeton, NJ: Educational Testing Service.

15.

van de Vijver, F.J.R. , & Harsveld, M. (1994). The incomplete equivalence of the paper-and-pencil and computer versions of the General Aptitude Test Battery. Journal of Applied Psychology, 79, 852-859.

16.

Wise, S. L. , & Plake, B. S. (1989). Research on the effects of administering tests via computers. Educational Measurement: Issues and Practices, 8(3), 5-10.