This article addresses standard-setting
for written and interactive (oral) examinations in the health professions. The
currently used methods are explained
and classified, and their strengths and
weaknesses are discussed. We argue
that standard-setting must be understood as an interactive system involving
decision maker(s), subject area, and
methods. Thus, standard-setting is a
methods. Thus, standard-setting is
a psychological/social psychological
process as well as a psychometric one.
It rests upon a foundation ofjudgment.
For written examinations, normative
and content-referenced (absolute)
methods are discussed. In interactive
examinations, judges' standards are
inherently absolute; design considerations are presented to systematize the
context for these judgments.
Get full access to this article
View all access options for this article.
References
1.
Abrahamson, S. (1975) "The oral examination: the case for the case against." Presented at American Board of Specialties Conference on the Oral Examination, Chicago, March.
2.
American Board of Internal Medicine (1979) Clinical Competence in Internal Medicine. Philadelphia : American Board of Internal Medicine, 1979.
3.
American Board of Medical Specialties ( 1975 ) "Conference on the oral examination." Chicago, March.
4.
Andrew, B. J. AND J. T. Hecht (1976) "A preliminary investigation of two procedures for setting examination standards." Educ. and Psych. Measurement36: 45-50.
5.
Asch, S. E. (1948) "The doctrine of suggestion prestige, and imitation in social psychology." Psych. Rev.55: 250-276.
6.
Block, J. H. (Ed). Mastery Learning: Theory and Practice. New York: Holt, Rinehart, and Winston, 1971.
7.
Bloom, B. J., J. T. Hastings , and G. F. Madaus ( 1971 ) Handbook on Formative and Summartive Evaluation of Student Learning. New York: McGraw-Hill .
8.
Cahn, S. M. (1974) "Philosophical reflections on evaluation." American J. of Medicine57: 152-156.
9.
Cahn, S. M. (1973) " Philosophical reflections on evaluation." Presented at the annual conference of the American Board of Internal Medicine, August
10.
Ebel, R. L. (1972) Essentials of Educational Measurement. Englewood Cliffs, NJ: Prentice-Hall.
11.
Elstein, A. S., L. S. Shulman , and S. A. Sprafka ( 1978) Medical Problem Solving: An Analysis of Clinical Reasoning . Cambridge, MA: Harvard Univ. Press.
12.
Evans, L. R., R. W. Ingersoll , and E. J. Smith ( 1966 ) "The reliability, validity, and taxonomic structure of the oral examination. J. of Medical Education41: 651-657.
13.
Festinger, L. (1954) "A theory of social comparison processes." Human Relations7: 117-140.
14.
Foster, J. T., S. Abrahamson , S. Lass, R. Girard, and R. Garris ( 1969) "Analysis of an oral examination used in specialty board certification." J. of Medical Education44: 951-954.
15.
Glass, G. V. (1978) "Standards and criteria." J. of Educ. Measurement15: 237-261.
16.
Gulliksen, H. (1950) Theory of Mental Tests. New York: John Wiley .
17.
Hambleton, R. K., H. Swaminathan , L. L. Cook, D. R. Eignor, and J. A. Gifford (1978) "Developments in latent trait theory: models, technical issues, and applications."Rev. of Educ. Research48: 467-510.
18.
Hubbard, J. P. (1978) Measuring Medical Education: The Tests and the Experience of the National Board of Medical Examiners. Philadelphia: Lea & Febiger.
19.
Klatzky, R. L. (1975) Human Memory: Structures and Processes. San Francisco: Freeman.
20.
LaDuca, A. (1979a) "The structure of competence in health professions." Presented at annual meeting of American Educational Research Association, San Francisco, April.
21.
LaDuca, A. (1979b) " Solving the medical problem solving problem." Presented at the Association of American Medical Colleges Conference on Research in Medical Education, November.
22.
Levine, H. G., and P. J. Forman ( 1973) "A study of retention of knowledge of neurosciences information." J. Medical Education48: 867-869.
23.
Lord, F. M. and M. R. Novick (1968) Statistical theories of mental test scores." Reading MA: Addison-Wesley .
24.
Lorge, I. (1936) "Prestige, suggestion and attitudes." J. of Social Psychology7: 386-402.
25.
Marshall, V. R. and J. Ludbrook (1972) "The relative importance of patient and examiner variability in a test of clinical skills." British J. of Medical Education6: 212-217.
26.
McGuire, C. (1975) "Experiences with orthopedic surgery." Presented at the American Board of Medical Specialties Conference on the Oral Examination, Chicago, March.
27.
Meskauskas, J. A. (1976) "Evaluation models for criterion-referenced testing: views regarding mastery and standard-setting ." Rev. of Educ. Research46: 133-158.
28.
Meskauskas, J. A. (1975) Cardiology and the American Board of Internal Medicine. Presented at the American Board of Medical Specialties Conference on the Oral Examination, Chicago, March.
29.
Meskauskas, J. A. and J. J. Norcini (1979) "Validity PMP/ CASE redundancy, and some comments about the structure of relationships among measures. Presented at the Association of American Medical Colleges Conference on Research in Medical Education, Washington, D.C., November.
30.
Meskauskas, J. A., and G. W. Webster (1975) "The American Board of Internal Medicine recertification examination process and results." Annals of Internal Medicine, 82: 577-581.
31.
Miller, G. A. (1956) "The magical number seven, plus or minus two: some limits on our capacity for processing information." Psych. Rev.63: 81-97.
32.
Millman, J. (1973) "Passing scores and test lengths for domain-referenced measures." Rev. of Educ. Research, 43: 205-216.
33.
Nedelsky, L. (1954) "Absolute grading standards for objective tests." Educ. and Psych. Measurement14: 3-19.
34.
Raju, N. S. (1977) "A generalization of coefficient alpha." Psychometrika42: 549-565.
35.
Rasch, G. (1966) "An item analysis which takes individual differences into account." British J. of Mathematical and Statistical Psychology19: 49-57.
36.
Rosenthal, R. (1966) Experimenter Effects in Behavioral Research. Englewood Cliffs, NJ: Prentice-Hall.
37.
Rosenthal, R.(1964) " The effect of the experimenter on the results of psychological research," pp. 79-114 in B. A. Maher (ed.) Progress in experimental personality research, Volume I. New York: Academic.
38.
Rosenthal, R. and L. Jacobson (1968) Pygmalion in the Classroom: Teacher Expectation and Pupils' Intellectual Development. New York: Holt, Rinehart, and Winston.
39.
SAS Institute, Inc. (1979) SAS User's Guide. Raleigh, NC: Author .
40.
Schumacher, C. F. et al. (1979) " Applying the Rasch model to equate examinations in the field of medicine ." Symposium presented at the annual meeting of the American Educational Research Association, San Francisco, April.
41.
Stanley, J. C. (1961) "Analysis of unreplicated 3-way classifications with applications to rater bias and trait independence." Psychometrika26: 205-219.
42.
Taylor, D. D., J. C. Reid , D. A. Senhauser, and J. A. Shively (1971) "Use of minimum pass levels on pathology examinations." J. of Medical Education46: 876-881.
43.
Van Wart, A. D. (1974) "A problem-solving oral examination for family medicine." J. Medical Education49: 673-680.
44.
Ware, J. E. and R. G. Williams (1975) "The doctor fox effect: a study oflecturer effectiveness and ratings of instruction. J. of Medical Education50: 149-156.
45.
Winer, B. J. (1971) Statistical Principles in Experimental Design. New York: McGraw-Hill.
46.
Wright, B. D. (1977) "Solving measurement problems with the Rasch model." J. of Educ. Measurement , 14: 97-116.
47.
Wright, B. D. ( 1968) " Sample-free test calibration and person measurement," in Proceedings of the 1967 Invitational Conference on Testing Problems . Princeton, NJ: Educational Testing Service .
48.
Wright, B. D. and G. A. Douglas (1977) "Best procedures for sample-free item analysis." Applied Psychological Measurement1: 281-295.
49.
Wright, B. D. and N. Panachapakesan (1969) "A procedure for sample-free item analysis ." Educ. and Psych. Measurement29: 23-48.
50.
Wright, B. D. and M. H. Stone (1979) Best Test Design: Rasch Measurement . Chicago: MESA Press.
51.
Wrightsman, L. S. (1972) Social Psychology in the Seventies. Monterey, CA: Brooks/Cole.
