Sage Journals: Discover world-class research

Abstract

A group of covariance structure models was exam ined to ascertain the similarity between conventionally administered and computerized adaptive (CAT) versions of the complete battery of the Differential Aptitude Tests (DAT). Two factor analysis models developed from classical test theory and three models with a multiplicative structure for these multitrait-multi method data were developed and then fit to sample data in a double cross-validation design. All three di rect-product models performed better than the factor analysis models in both calibration and cross-valida tion subsamples. The cross-validated, disattenuated correlation between the administration methods in the best-performing direct-product model was very high in both groups (.98 and .97), suggesting that the CAT version of the DAT is an adequate representation of the conventional test battery. However, some evidence suggested that there are substantial differences be tween the printed and computerized versions of the one speeded test in the battery. Index terms: adap tive tests, computerized adaptive testing, covariance structure, cross-validation, Differential Aptitude Tests, direct-product models, factor analysis, multitrait-mul timethod matrices.

Get full access to this article

View all access options for this article.

References

Akaike, H. (1987). Factor analysis and AIC. Psychometrika , 52, 317-332.

Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.

Bennett, G.K. , Seashore, H.G. , & Wesman, A.G. (1982). Differential Aptitude Tests Administrator's Handbook . San Antonio TX: The Psychological Corporation.

Bentler, P.M. , & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.

Braun, H.I. , & Holland, P.W. (1982). Observed score test equating: A mathematical analysis of some ETS equating procedures. In P. W. Holland & D. B. Rubin (Eds.), Test equating (pp. 9-49). New York: Academic Press.

Browne, M.W. (1984). The decomposition of multitrait-multimethod matrices by generalized least squares. British Journal of Mathematical and Statistical Psychology, 37, 1-21.

Browne, M.W. , & Cudeck, R.A. (in press). Single-sample cross-validation indices for covariance structures . Multivariate Behavioral Research.

Cudeck, R. (1985). A structural comparison of conventional and adaptive versions of the ASVAB. Multivariate Behavioral Research, 20, 305-322.

Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317-327.

10.

Cudeck, R. , & Browne, M.W. (1983). Cross-validation of covariance structures. Multivariate Behavioral Research, 18, 147-167.

11.

Greaud, V.A. , & Green, B.F. (1986). Equivalence of conventional and computer presentation of speed tests. Applied Psychological Measurement, 10, 23-34.

12.

Gulliksen, H. (1968). Methods for determining equivalence of measures . Psychological Bulletin, 70, 534-544.

13.

James, L.R. , Mulaik, S.A. , & Brett, J.M. (1982). Causal analysis: Assumptions, models and data . Beverly Hills CA: Sage.

14.

Jöreskog, K.G. (1971). Statistical analysis of sets of congeneric tests . Psychometrika, 36, 109-133.

15.

Jöreskog, K.G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443-475.

16.

Lawley, D.N. , & Maxwell, A.E. (1971). Factor analysis as a statistical method (2nd ed.). London: Butterworth.

17.

Lord, F.M. (1974). Individualized testing and item characteristic curve theory. In D. H. Krantz , R. C. Atkinson , R. D. Luce , & P. Suppes (Eds.), Contemporary developments in mathematical psychology (Vol. II). San Francisco: Freeman.

18.

Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.

19.

McBride, J.R. (1986, August). A computerized adaptive edition of the Differential Aptitude Tests. Paper presented at the meeting of the American Psychological Association, Washington DC.

20.

McBride, J.R. , Corpe, V.A. , & Wing, H. (1987, August). Equating the Computerized Adaptive Edition of the Differential Aptitude Tests. Paper presented at the meeting of the American Psychological Association, New York.

21.

Moreno, K.E. , Wetzel, C.D. , McBride, J.R. , & Weiss, D.J. (1984). Relationship between corresponding Armed Services Vocational Aptitude Battery (ASVAB) and computerized adaptive testing (CAT) subtests. Applied Psychological Measurement, 8, 155-163.

22.

Owen, R.A. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351-356.

23.

Rasch, G. (1966). An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 19, 49-57.

24.

Sympson, J.B. , Weiss, D.J. , & Ree, M.J. (1982). Predictive validity of conventional and adaptive tests in an Air Force training environment (Report AFHRL-TR-81-40). Brooks Air Force Base TX: Manpower and Personnel Division .

25.

Tanaka, J.S. (1987). How big is big enough?: Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58, 134-146.

26.

Tanaka, J.S. , & Huba, G.J. (1985). A fit index for covariance structure models under arbitrary GLS estimation. British Journal of Mathematical and Statistical Psychology, 38, 621-635.

27.

U.S. Department of Defense. (1982). Armed Services Vocational Aptitude Battery. North Chicago IL: U.S. Military Entrance Processing Command.

Adaptive and Conventional Versions of the DAT: The First Complete Test Battery Comparison

Abstract

Get full access to this article

References