This research study was designed to compare student achievement scores from three different testing methods: paper-administered testing, computer-administered testing, and computerized adaptive testing. The three testing formats were developed from the California Assessment Program (CAP) item banks for grades three and six. The paper-administered and the computer-administered tests were identical in item content, format, and sequence. The computerized adaptive test was a tailored or adaptive sequence of the items in the computer-administered test.
Get full access to this article
View all access options for this article.
References
1.
WeissD. J., Computerized Adaptive Ability Measurement, Naval Research Reviews, 28, pp. 1–28, 1975.
2.
LordF. M., Applications of Item Response Theory to Practical Testing Problems, Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1980.
ArmbrusterB. B.StevensR. J.RosenshineB., Analyzing Content Coverage and Emphasis: A Study of Three Curricula and Two Tests, Technical Report No. 26, Center for the Study of Reading, University of Illinois, Urbana—Champaign, 1977.
5.
FlodenR. E.PorterA. C.SchmidtW. H.FreemanD. J., Don't They All Measure the Same Thing? Consequences of Selecting Standardized Tests, Research Series No. 25, Institute of Research on Teaching, Michigan State University, East Lansing, 1977.
6.
FreemanD.KuhsT.KnappenL.PorterA., A Closer Look at Standardized Tests, Research Series No. 53, Institute of Research on Teaching, Michigan State University, East Lansing, 1979.
7.
HouseE. R.GlassG. V.McLeanL.WalkerD. F., No Simple Answer: Critique of the “Follow-Through” Evaluation, Harvard Educational Review, 48, pp. 128–160, 1978.
8.
TylerR. W.WhiteS. H., Testing, Teaching, and Learning, U.S. Department of Health, Education and Welfare and National Institute of Education, Washington, D.C., 1979.
9.
McBrideJ. R., Applications of Latent Trait Theory to Criterion Referenced Testing, WeissD. J. (ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference, Department of Psychology, University of Minnesota, 1977.
10.
McBrideJ. R., Psychometric Considerations for the Computerized Adaptive Testing System, Navy Personnel Research and Development Center, San Diego, California, 1981.
11.
OlsenJ. B.BundersonC. V.GilstrapR. M., Development of a Preliminary Design for a Computerized Adaptive Testing Service for the Department of Defense, Wicat Systems, Orem, Utah, 1982.
12.
OlsenJ. B.BundersonC. V.GilstrapR. M., Prototype Design and Development of a Computerized Adaptive Testing System, Report for the Department of Defense, U.S. Marine Corps Executive Agent, Wicat Systems, Orem, Utah, 1983.
13.
WeissD. J., Computerized Adaptive Achievement Testing, in Procedures for Instructional Systems Development, O'NeilH. F. (ed.), Academic Press, New York, 1979.
14.
WeissD. J., Computerized Adaptive Performance Evaluation, Final Report, Computerized Adaptive Testing Laboratory, University of Minnesota, Minneapolis, 1981.
15.
WeissD. J., New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing, WeissD. J. (ed.), Academic Press, New York, 1983.
16.
McKinleyR. L.ReckaseM. D., Implementing an Adaptive Testing Program in an Instructional Program Environment, paper presented at the annual meeting of the American Educational Research Association, New Orleans, 1984.
17.
MorenoK.WetzelC. D.McBrideJ. R.WeissD. J., Relationships Between Corresponding Armed Service Vocational Aptitude Battery (ASVAB) and Computerized Adaptive Testing (CAT) Subtests, (NPRDC TR 83-27), Navy Personnel Research and Development Center, San Diego, 1983.
18.
ReckaseM. D., The Difficulty of Test Items that Measure More than One Ability, paper presented at the American Educational Research Association Meeting, Chicago, 1985.
19.
HambletonR. K.RovinelliR. J., Assessing the Dimensionality of a Set of Test Items, paper presented at the American Educational Research Association, Montreal, 1983.
20.
SamejimaF., Normal Ogive Model on the Continuous Response Level in the Multi-Dimensional Latent Space, Psychometrika, 39, pp. 111–121, 1974.
21.
WhitelyS. E., Multicomponent Latent Trait Models for Ability Tests, Psychometrika, 45, pp. 479–494, 1980.
22.
BejarI. I.WeissD. J.KingsburyG. G., Calibration of an Item Pool for the Adaptive Measurement of Achievement, Research Report 77-5, Psychometric Methods Program, University of Minnesota, Minneapolis, 1977.
23.
BrownJ. M.WeissD. J., An Adaptive Testing Strategy for Achievement Test Batteries, Research Report 77-6, Psychometric Methods Program, University of Minnesota, Minneapolis, 1977.
24.
HambletonR. K., Applications of Latent Trait Theory to the Development and Use of Criterion Referenced Tests, paper presented at the annual meeting of the American Education Research Association and National Council for Measurement in Education, San Francisco, 1979.
25.
HambletonR. K., Applications of Item Response Models to Criterion-Referenced Assessment, Applied Psychological Measurement, 7:1, pp. 33–44, 1983.
26.
KingsburyG. G.WeissD. J., A Validity Comparison of Adaptive and Conventional Strategies for Mastery Testing, Research Report 81-3, Psychometric Methods Program, Department of Psychology, University of Minnesota, Minneapolis, 1981.
27.
KingsburyG. G.WeissD. J., A Comparison of IRT-Based Adaptive Mastery Testing and a Sequential Mastery Testing Procedure, in New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing, WeissD. J. (ed.), Academic Press, New York, 1983.
28.
SchnitzJ.MaynesD.OlsenJ. B., Evaluation of a Schema-Based Reading Comprehension Curriculum Administered by Computer, Report to the U.S. Department of Education, Wicat Education Institute, Provo, Utah, 1983.
29.
TraubR. E.WolfR. G., Latent Trait Theories and the Assessment of Educational Achievement, Review of Research in Education, 9, pp. 377–433, 1982.
30.
WeissD. J.KingsburyG. G., Application of Computerized Adaptive Testing to Educational Problems, Journal of Educational Measurement, 21:4, pp. 361–375, 1984.
31.
StockingM. L., Two Simulated Feasibility Studies in Computerized Adaptive Testing, paper presented at the American Educational Research Association, New Orleans, 1984.
32.
BockR. D.AitkinM., Marginal Maximum Likelihood Estimation of Item Parameters: An Application of an EM Algorithm, Psychometrika, 46, pp. 443–459, 1981.
33.
BockR. D.MislevyR. J., Adaptive EAP Estimation of Ability in a Microcomputer Environment, Applied Psychological Measurement, 6:4, pp. 431–444, 1982.
34.
StockingM. L.LordF. M., Developing a Common Metric in Item Response Theory, Applied Psychological Measurement, 7, pp. 201–210, 1983.
35.
CalvertE. J.WestfallR. C., A Comparison of Conventional and Automated Administration of Raven's Standard Progressive Matrices, International Journal of Man-Machine Studies, 17, pp. 305–310, 1982.
36.
HardwickeS. B.YoesM. E., Attitudes and Performance on Computerized and Paper and Pencil Tests, paper presented at the second annual Air Force Conference on Technology in Training and Education,
37.
HoffmanK. L.LundbergG. D., A Comparison of Computer-Monitored Group Tests with Paper and Pencil Tests, Educational and Psychological Measurement, 36, pp. 791–809, 1976.
38.
KingsburyG. G.WeissD. J., A Validity Comparison of Adaptive and Conventional Strategies for Mastery Testing, Research Report 81–3, Psychometric Methods Program, University of Minnesota, Minneapolis, 1981.
39.
McBrideJ. R.MartinJ. T., Reliability and Validity of Adaptive Ability Tests in a Military Setting, in New Horizons in Testing, WeissD. J. (ed.), Academic Press, New York, 1983.
40.
SorensonH. B., Cognitive Ability Tests, The Monitor, November/December 1985.
41.
VicinoF. L.HardwickeS. B., An Evaluation of the Utility of Computerized Adaptive Testing, paper presented at the annual meeting of the American Educational Research Association, New Orleans, 1984.
42.
WeissD. J.KingsburyG. G., Application of Computerized Adaptive Testing to Educational Problems, Journal of Educational Measurement, 21, pp. 361–375, 1984.
43.
StockingM. L.LordF. M., Developing a Common Metric in Item Response Theory, Applied Psychological Measurement, 7, pp. 201–210, 1983.
