A priori estimates of item characteristics are necessary for the effi cient development of sound tests. Judges were asked to rate the For mat, Relevancy, Difficulty, Discrimination, and Overall Quality of multiple-choice items for an examination covering health science in formation. Ratings of Relevancy were least reliable. The combined estimates of Difficulty did not correspond with empirical values of item difficulty; however, the combined ratings for Discrimination did correlate (point-biserial) significantly (p < .05) with item-total test scores. Additionally, combined ratings of Overall Quality were correlated significantly (p < .05) with item-total correlations.
Get full access to this article
View all access options for this article.
References
1.
Anastasi, Anne.Psychological testing. New York: The Macmillan Company, 1976.
2.
Campbell, A.C.Some determinants of the difficulty of nonverbal classification items , EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1961, 21, 899-913.
3.
Dimond, E.G.Academic plan of the School of Medicine. Kansas City: University of Missouri-Kansas City, 1976.
4.
Guilford, J.P.Psychometric methods. New York: McGraw-Hill Book Company, 1954.
5.
Ebel, R.L.Estimation of the reliability of ratings. Psychometrika , 1951, 16, 407-424.
6.
Lippey, G.Computer-assisted test construction. Englewood Cliffs, New Jersey: Educational Technology Publications , 1974.
7.
Lorge, I. and Diamond, L.The value of information to good and poor judges of item difficulty . EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1954, 14, 29-33.
8.
Lorge, I. and Kruglov, L.The improvement of estimates of test difficulty. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1953, 13, 34-36.
9.
Peterson, C.C. and Peterson, J.L.Linguistic determinants of the difficulty of true-false items. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENTS, 1976, 36, 161-164.
10.
Quereshi, M.Y.Mental test performance as a function of payoff conditions, item difficulty, and degree of speeding. Journal of Applied Psychology, 1960, 44, 65-77.
11.
Quereshi, M.Y. and Fisher, T.L.Logical versus empirical estimates of item difficulty. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1977, 37, 81-100.
12.
Smull, N.W., Willoughby, T.L., Kerber, J.A. and Tyson, J.G.A profile examination system for physician extenders and a method for curricular prescription (N01-AH-44108. Washington, D.C.: Department of Health, Education and Welfare , 1977.
13.
Willoughby, T.L., Dimond, E.G., and Smull, N.W.Correlation of Quarterly Profile Examination and National Board of Medical Examiner Scores, EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT , 1977, 37, 445-449.
