In this study 590 third-grade students took one of four reading comprehension tests with either multiple- choice items or open-ended items. Each also took 32 tests indicating 16 semantic Structure-of-Intellect (si) abilities. Four conditions or groups were distinguished on the basis of the reading comprehension tests. The four 33 x 33 correlation matrices were analyzed si multaneously with a four-group LISREL model. The 16 intellectual abilities explained approximately 62% of the variance in true reading comprehension scores. None of the SI abilities proved to be differentially re lated to item type. Therefore, it was concluded that item type for reading comprehension is congeneric with respect to the SI abilities measured. Index terms: construct validity, item format, free response, reading comprehension, Structure-of-Intellect model.
Get full access to this article
View all access options for this article.
References
1.
Ackerman, B.P. (1984a). The effects of storage and processing complexity on comprehension repair in children and adults. Journal of Experimental Child Psychology, 37, 303-343.
2.
Ackerman, B.P. (1984b). Storage and processing constraints on integrating storage information in children and adults. Journal of Experimental Child Psychology, 38, 64-92.
3.
Benson, J. (1981). A redefinition of content validity. Educational and Psychological Measurement, 41, 793-802.
4.
Benson, J., & Crocker, L. (1979). The effects of item format and reading ability on objective test performance. Educational and Psychological Measurement, 39, 381-387.
5.
Biemiller, A. (1977). A relationship between oral reading rates for letters, words and simple texts in the development of reading achievement . Reading Research Quarterly, 12, 259-266.
6.
Birenbaum, M., & Tatsuoka, K.K. (1987). Open-ended versus multiple-choice response formats—it does make a difference for diagnostic purposes. Applied Psychological Measurement, 11, 385-395.
7.
Bracht, G.H., & Hopkins, K.D. (1968). Comparative validities of essay and objective tests (Research Paper No. 20). Boulder CO: Laboratory of Educational Research, University of Colorado .
8.
Brown, J. (1976). An analysis of recognition and recall and of problems in their comparison. In J. Brown (Ed.), Recall and recognition. London: Wiley.
9.
Carter, H.D., & Crone, A.P. (1940). The reliability of new-type or objective tests in a normal classroom situation. Journal of Applied Psychology , 24, 353-368.
Cook, D.L. (1955). An investigation of three aspects of free-response and choice-type tests at college level. Dissertation Abstracts , 1351.
12.
Coombs, C.H., Milholland, J.E., & Womer, F.B. (1956). The assessment of partial knowledge. Educational and Psychological Measurement, 16, 13-37.
13.
Croft, A.C. (1982). Do spelling tests measure the ability to spell?Educational and Psychological Measurement, 42, 715-723.
14.
Cronbach, L.J. (1966). New light in test strategy from decision strategy . In A. Anastasi (Ed.), Testing problems in perspective. Washington DC: American Council on Education.
15.
Culpepper, M.M., & Ramsdale, R. (1983). A comparison of multiple-choice and essay tests of writing skill. Research in the Teaching of English, 41, 295-298.
16.
Davis, F.B. (1968). Research in comprehension in reading. Reading Research Quarterly, 3, 499-535.
Fienberg, S.E. (1980). The analysis of cross-classified categorical data . Cambridge: MIT Press.
19.
Frary, R.B. (1985). Multiple-choice versus free-response: A simulation study. Journal of Educational Measurement, 22, 21-31.
20.
Gillund, G., & Shiffrin, R.M. (1984). A retrieval model for both recognition and recall. Psychological Review, 9, 2-67.
21.
Gipps, C., Steadman, S., Blackstone, T., & Stiener, B. (1983). Testing children: Standardized testing in local educational authorities and schools. London: Heinemann.
22.
Goslin, D.A. (1967). Teachers and testing. New York : Russell Sage Foundation.
23.
Guilford, J.P. (1971). The nature of human intelligence. London: McGraw-Hill.
24.
Guilford, J.P. (1979). Cognitive psychology with a frame of reference . San Diego: EDITS.
25.
Guilford, J.P., & Hoepfner, R. (1971). The analysis of intelligence. New York: McGraw-Hill.
26.
Gulliksen, M. (1950). Theory ofmental tests. New York: Wiley.
27.
Guttman, L. (1965). A faceted definition of intelligence. In K. Eiferman (Ed.), Studies in psychology, Scripta Hierosolymitana 14 (pp. 166-181). Jerusalem: Hebrew University.
28.
Heim, A.W., & Watts, R.B. (1967). An experiment on multiple-choice versus open-ended answering in a vocabulary test. British Journal of Educational Psychology, 37, 339-346.
29.
Hoeks, J. (1985). Vaardigheden in begrijpend lezen [Abilities in reading comprehension]. Unpublished doctoral dissertation, University of Amsterdam.
30.
Hogan, T.P. (1981). Relationships between free-response and choice type tests of achievement: A review of literature. Washington DC: National Institute of Education.
31.
Hopkins, K.D., George, C.A., & Williams, D.D. (1985). The concurrent validity of standardized achievement tests by content area using teachers' ratings as criteria. Journal of Educational Measurement, 22, 177-182.
32.
Hopkins, K.D., & Stanley, J.C. (1981). Educational and psychological measurement and evaluation (6th ed.). Englewood Cliffs NJ: Prentice-Hall.
33.
Hurd, A.W. (1930). Comparison of short-answers and multiple-choice type tests covering identical subject content. Journal of Educational Research, 26, 28-30.
34.
Jöreskog, K.G. (1971). Statistical analysis of sets of congeneric tests . Psychometrika, 36, 109-132.
35.
Jöreskog, K.G., & Sörbom, D. (1986). USREL: Analysis of linear structural relationships by maximum likelihood, instrumental variables and least squares methods. Uppsala, Sweden: University of Uppsala.
36.
Just, M.A., & Carpenter, P.A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329-354.
37.
Kinney, C.B., & Eurich, A.C. (1938). A summary of investigations comparing different types of tests. School and Society, 36, 540-544.
38.
Kintsch, W., & van Dijk, T.A. (1978). Toward a model of discourse comprehension of simple prose. Psychological Review, 85, 363-394.
39.
Klein-Braley, C. (1983). A cloze is a cloze is a question. In J. W. Oller (Ed.), Issues in language testing research. Rowley MA: Newbury House.
40.
Lorch, R.F., Lorch, E.P., & Matthews, P.D. (1985). On-line processing of the topic structure of a text. Journal of Memory and Language , 24, 350-362.
41.
Lorch, R.F., Lorch, E.P., & Mogan, A. (1987). Task effects and individual differences in on-line processing of the topic structure of a text. Discourse Processes , 10, 63-80.
42.
Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
43.
Mellenbergh, G.J. (1971). Studies in studietoetsen [Studies in educational testing]. Unpublished doctoral dissertation, University of Amsterdam.
44.
Meuffels, B. (1982). Studies over taalvaarrdigheid [Studies in language abilities]. Unpublished doctoral dissertation, University of Amsterdam.
45.
Mezynsky, K. (1983). Issues concerning the acquisition of knowledge: Effects of vocabulary training on reading comprehension. Review of Educational Research, 53, 253-279.
46.
Murphy, R.J. (1982). Sex differences in objective test performance . British Journal of Educational Psychology, 52, 213-219.
47.
Neville, M.H., & Pugh, A.K. (1977). Context in reading and listening: Variations in approach to cloze tasks. Reading Research Quarterly, 11, 12-31.
48.
Paterson, D.G. (1926). Do old and new type examinations measure different mental functions?School and Society, 24, 246-248.
49.
Samson, D.M.M. (1983). Rasch and reading. In J. van Weeren (Ed.), Practice and problems in language testing 5. Arnhem, The Netherlands: CITO .
50.
Shohamy, E. (1984). Does the testing method make a difference? The case of reading comprehension. Language Testing, 1, 147-169.
Spearman, C. (1927). The abilities of man. London : McMillan.
53.
Stemberg, R.J. (1980). A proposed resolution of curious conflicts in the literature of linear syllogisms. In R. S. Nickerson (Ed.), Attention and performance (vol. VII). Hillsdale NJ: Erlbaum.
Traub, R.E., & Fisher, C.W. (1977). On the equivalence of constructed-response and multiple-choice tests. Applied Psychological Measurement, 1, 355-369.
57.
van denBergh, H. (1987). Open en meerkeuzevragen tekstbegrip: Een onderzoek naar de relatie tussen prestaties op en meerkeuzevragen [Open-ended and multiple-choice items: A study of the relationships between achievements on open-ended and multiple-choice items for reading comprehension]. Tijdschrift voor Onderwijsonderzoek, 12, 304-312.
58.
van den Bergh, H. (1989). De correlationele structuur van enkele schriftelijke taalvaardigheden [The correlational structure of several language abilities] . Tijdschrift voor Taalbeheersing, 11, 38-59.
59.
van denBergh, H., Eiting, M., & Otter, M. (1988). Differentiële effecten van vraagvorm bij aardrijkskunde en natuurkunde examens [Differential effects of item format in geography and science exams]. Tijdschrift voor Onderwijsonderzoek, 13, 270-284.
60.
Vernon, P.E. (1962). The determinants of reading comprehension. Educational and Psychological Measurement, 22, 269-286.
61.
Ward, W.C. (1982). A comparison of free-response and multiple-choice forms of verbal aptitude tests. Applied Psychological Measurement , 6, 1-11.
62.
Ward, W.C., Frederiksen, N., & Carlson, S.B. (1980). Construct validity of free-response and machine-scorable forms of a test. Journal of Educational Measurement, 17, 11-29.
63.
Weiss, D., & Jackson, R. (1983). The validity of descriptive tests of language skills: Relationships to direct measures of writing skill ability and grades in introductory college English courses (Report LB-83-4 ; ETS-RR-83-27). New York: College Entrance Examination Board.
