An Item Discriminability Study of Textually Explicit,Textually Implicit,and Scriptally Implicit Questions

Abstract

This paper reports the effect that background knowledge has on the item discriminability of reading comprehension items; item discriminability is an index of how well a test item discriminates bet ween weak and strong examinees in the ability being tested.

The questions from three reading comprehension tests were placed into one of the following categories: (1) textually explicit; (2) textually implicit; and (3) scriptally implicit. An item discriminability index was computed for each item. The results indicated that there were significant differences in the three categories of itèms with respect to item discriminability and that reading comprehension items which are heavily dependent on background knowledge do not ex hibit good item discriminability. The findings indicate that researchers must control for background knowledge in a reading test when they use test products as the basis for making inferences about the pro cesses underlying reading comprehension.

Get full access to this article

View all access options for this article.

References

Campbell, D.T. , and Stanley, J.C. (1963). Experimental and quasi-experimental designs for reseacrh. Chicago: Rand McNally College Publishing Company.

Coleman, E.B. (1971). Developing a technology of written instruction: Some determinants of the complexity of prose. In E. Rothkopf and P. Johnson (Eds.), Verbal learning research and the technology of written instruction. New York: Teachers College Press.

Farr, R. , and Carey, R.F. (1986). Reading: What can be measured?, second edition Newark, DE: International Reading Association.

Farr, R. , Carey, R.F. , and Tone, B. (1986). Recent theory and research into the reading process: Implications for reading assessemnt. In J. Orasanu (Ed.), Reading comprehension: From research to practice . Hillsdale, NJ: Lawrence Erlbaum Associates.

Henning, G. (1987). A guide to language testing. Cambridge, MA: Newbury House Publishers.

Johnston, P. (1981). Prior knowledge and reading comprehension test bias. Unpublished doctoral dissertation, Univeristy of Illinois.

Johnston, P. (1983). Reading comprehension assessment: A cognitive basis. Newark, DE: International Reading Association.

Johnston, P. (1984). Assessment in reading, In P.D. Pearson (Ed.), Handbook of reading research. New York and London: Longman, Inc.

Johnston, P. , and Pearson, P.D. (1982). Prior knowledge, connectivity, and the assessment of reading comprehension. Technical report no. 245. Champaign, IL: Center for the Study of Reading, University of Illinois.

10.

Kintsch, W. (1974). The representation of meaning in memory. Hillsdale, NJ: Lawrence Erlbaum Associates.

11.

McLaughlin, G. (1969). SMOG grading — a new readability formula . Journal of Reading. 12:639-646.

12.

Pearson, P.D. , and Johnson D.D. (1978). Teaching reading comprehension. New York: Holt, Rinehart and Winston.

13.

Royer, J.M. , and Cunningham, D.J. (1978). On the theory and measurement of reading comprehension. Technical report no. 91. Champaign, IL: Center for the Study of Reading, University of Illionis.

14.

Searle, J. (1975). A taxonomy of illocutionary acts. In K. Gunderson (Ed.), The philosophy of science: VII. I Language, mind and society. Minneapolis: University of Minnesota Press.

15.

Schank, R.C. (1975). The structure of episodes in memory. In D.G. Bob-row and A. Collins (Eds.), Representation and understanding: Studies in cognitive science. New York: Academic Press .

16.

Schlesinger, I.M. , and Weiser, Z. (1970). A facet design for tests of reading comprehension . Reading Research Quarterly, 5:566-580.

17.

Tuinman, J.J. (1979). Reading is recognition — when reading is not reasoning. In J. Harste and R.F. Carey (Eds.), New perspectives on comprehension . Bloomington, IN: Indiana University Press.