The purposes of this study were to assess the comparability of scores obtained from computer and paper-and-pencil versions of the Iowa Tests of Educational Development and to evaluate examinees' attitudes about multiple aspects of test administration in the two modes. Findings supported the comparability of scores across administration modes with regard to scaling (means and standard deviations), internal consistency, and criterion- and construct-related validity. Overall, examinees preferred taking the computerized tests and valued many operational features of those tests. The least favorable attitudes were reported for the literary skills tests and their scrollable reading passages in particular.
Get full access to this article
View all access options for this article.
References
1.
American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
American Psychological Association. (1986) Guidelines for computer-based tests and interpretation. Washington, DC: American Psychological Association.
3.
BartramD.HambletonR. K. (Eds.) (2006) Computer-based testing and the internet. Hoboken, NJ: Wiley.
4.
BollenK. A.LongJ. S. (Eds.) (1993) Testing structural equation models. Newbury Park, CA: Sage.
5.
Booth-KewleyS.EdwardsJ. E.RosenfeldP. (1992) Impression management, social desirability, and computer administration of attitude questionnaires: does the computer make a difference?Journal of Applied Psychology, 77(4:), 562–566.
6.
BundersonC. V.InouyeD. K.OlsenJ. B. (1989) The four generations of computerized educational measurement. In LinnR. L. (Ed.), Educational measurement. (3rd ed.) Phoenix, AZ: Oryx. Pp. 367–407.
7.
CohenJ. (1988) Statistical power analysis for the behavioral sciences. (2nd ed.) Hillsdale, NJ: Erlbaum.
8.
DeAngelisS. (2000) Equivalency of computer-based and paper-and-pencil testing. Journal of Allied Health, 29(3), 161–164.
9.
DiLilloD.DeGueS.KrasA.Di Loreto-ColganA. R.NashC. (2006) Participant responses to retrospective surveys of child maltreatment: does mode of assessment matter?Violence and Victims, 21(4), 410–424.
10.
DrasgowF.LuechtR. M.BennettR. E. (2006) Technology and testing. In BrennanR. (Ed.), Educational measurement. (4th ed.) Westport, CT: Preager. Pp. 471–516.
DwightS. A.FeigelsonM. E. (2000) A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and Psychological Measurement, 60(3), 340–360.
13.
FanH. M.VispoelW. P. (2008) A meta-analysis of comparability of scores yielded by computerized and paper-and-pencil reading tests. Paper presented at the annual meeting of the National Council on Measurement in Education, New York, April.
14.
FeldtL. S. (1969) A test of the hypothesis that Cronbach's alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 343–373.
15.
FingerM. S.OnesD. S. (1999) Psychometric equivalence of the computer and booklet forms of the MMPI: a meta-analysis. Psychological Assessment, 11(1), 58–66.
16.
ForsythR. A.AnsleyT. N.FeldtL. S.AlnotS. D. (2001) Iowa Tests of Educational Development: complete battery. Itasca, IL: Riverside.
17.
ForsythR. A.AnsleyT. N.FeldtL. S.AlnotS. D. (2003) Iowa Tests of Educational Development: guide to research and development. Itasca, IL: Riverside.
18.
GallagherA.BridgemanB.CahalanC. (2000) The effect of computer-based tests on racial/ethnic, gender, and language groups. (GRE Board Professional Report No. 96-21P) Princeton, NJ: Education Testing Service.
19.
International Test Commission (ITC). (2005) International guidelines on computer-based and internet delivered testing. Retrieved from www.intestcom.org/itc_projects.htm.
20.
LinquistE. F. (1953) Design and analysis of experiments in psychology and education. Boston: Houghton Mifflin.
21.
LumsdenJ. A.SampsonJ. P.Jr.ReardonR. C.LenzJ. G.PetersonG. W. (2004) A comparison study of the paper, personal computer (PC), and Internet versions of Holland's Self-Directed Search. Measurement and Evaluation in Counseling and Career Development, 37(2), 85–94.
22.
MarshH. W.RedmayneR. S. (1994) A multidimensional physical self-concept and its relations to multiple components of physical fitness. Journal of Sport & Exercise Psychology, 16, 43–55.
23.
MazzeoJ.HarveyA. L. (1988) The equivalence of scores from automated and conventional educational and psychological tests: a review of the literature. (Report No. CBR 87-8, ETS RR 88-21) Princeton, NJ: Educational Testing Service.
24.
MeadA. D.DrasgowF. (1993) Equivalence of computerized and paper-and-pencil cognitive ability tests: a meta-analysis. Psychological Bulletin, 114(3), 449–458.
25.
MillsC. N.PotenzaM. T.FremerJ. J.WardW. C. (2002) Computer-based testing: building the foundation for future assessments. Mahwah, NJ: Erlbaum.
26.
NausM. J.PhilippL. M.SamsiM. (2009) From paper to pixels: a comparison of paper and computer formats in psychological assessment. Computers in Human Behavior, 25(1), 1–7.
27.
OlkinI. (1967) Correlations revisited. In StanleyJ. C. (Ed.), Improving experimental design and statistical analysis. Chicago: Rand McNally. Pp. 102–128.
28.
ParshallC. G.SprayJ. A.KalohnJ. C.DaveyT. (2002) Practical considerations in computer-based testing. New York: Springer-Verlag.
29.
PetersonJ. J. (1983) The Iowa Testing Programs. Iowa City, IA: Univer. of Iowa Press.
30.
PowersD. E.O'NeillK. (1993) Inexperienced and anxious computer users: coping with a computer-administered test of academic skills. Educational Assessment, 1(2), 153–173.
31.
RichmanW. L.KieslerS.WeisbandS.DrasgowF. (1999) A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology, 84(5), 754–775.
32.
RosenfeldP.DohertyL. M.VicinoS. M.KantorJ.GreavesJ. (1989) Attitude assessment in organizations: testing three microcomputer-based survey systems. Journal of General Psychology, 116, 145–154.
33.
RyanJ. M.CorryJ. R.AttewellR.SmithsonM. J. (2002) A comparison of an electronic version of the SF-36 General Health Questionnaire to the standard paper version. Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care, and Rehabilitation, 11(1), 19–26.
34.
SalgadoJ. F.MoscosoS. (2003) Internet-based personality testing: equivalence of measures and assessees' perceptions and reactions. International Journal of Selection and Assessment, 11(2-3), 194–205.
35.
SchulenbergS. E.YutrzenkaB. A. (2001) Equivalence of computerized and conventional versions of the Beck Depression Inventory–II (BDI–II). Current Psychology: a Journal for Diverse Perspectives on Diverse Psychological Issues, 20(3), 216–230.
36.
SieppertJ. D.KrysikJ. (1996) Computer-based testing in social work education: a preliminary exploration. Computers in Human Services, 13(1), 43–61.
37.
VispoelW. P.BooJ.BleilerT. (2001) Computerized and paper-and-pencil versions of the Rosenberg Self-Esteem Scale: a comparison of psychometric features and respondent preferences. Educational and Psychological Measurement, 61(3), 461–474.
38.
WangS.JiaoH.YoungM. J.BrooksT.OlsonJ. (2008) Comparability of computer-based and paper-and-pencil testing in K-12 reading assessments: a meta-analysis of testing mode effects. Educational and Psychological Measurement, 68(1), 5–24.
39.
WardT. J.HooperS. R.HannafinK. M. (1989) The effect of computerized tests on the performance and attitudes of college students. Journal of Educational Computing Research, 5(3), 327–333.
40.
WhiteD. M.ClementsC. B.FowlerR. D. (1985) A comparison of computer administration with standard administration of the MMPI. Computers in Human Behavior, 1(2), 153–162.
