The oral examination used in licensing clinical social workers in Virginia was revised by the
Board of Social Work in 1991. This article describes the process used to evaluate the oral
examination and reviews research on testing and clinical judgment as background to the
discussion of the validity and reliability of the revised oral examination. The article reports
statistical analyses of interrater reliability for two administrations of the exam. Using kappa (a
stringent test of interrater reliability), moderate interjudge ratings were obtained. The oral exam
was eliminated as a method of testing in 1993. Recommendations for evaluating licensure tests in general are discussed
Get full access to this article
View all access options for this article.
References
1.
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1974). Standards for educational and psychological tests . Washington, DC: Author.
2.
American Psychological Association. (1985). Standards for educational and psychological testing. Washington, DC: Author.
3.
Anastasi, A. (1968). Psychological testing (3rd ed.). New York: Macmillan.
4.
Arnhoff, R.N. (1954). Some factors influencing the unreliability of clinical judgments. Journal of Clinical Psychology, 10, 272-275.
5.
Bieri, J., Atkins, A.L., Briar, S., Leaman, R.L., Miller, H., & Tripodi, T. (1966). Clinical and social judgment: The discrimination of behavioral information. New York: Wiley.
6.
Biggerstaff, M.A. (1976). Social work practitioners' conceptions of sex and social roles. Unpublished doctoral dissertation, University of Southern California.
7.
Biggerstaff, M.A. (1992). Evaluating the oral examination in Virginia's licensing of clinical social workers. Research on Social Work Practice, 2, 184-197.
8.
Biggerstaff, M.A. & Eisenberg, R. (1989). Summary report: Job analysis verification study . Culpeper, VA: American Association of State Social Work Boards.
9.
Briar, S. (1961). Use of theory in studying effects of client social class on students' judgments. Social Work, 6(3), 91-97.
10.
Carmines, E.G., & Zeller, R.A. (1979). Reliability and validity assessment. Beverly Hills, CA: Sage.
11.
Cocozzelli, C.L. (1986). Social workers'theoretical orientations. New York: University Press of America.
12.
Cohen, J. (1960). A coefficient of agreement for nominal scales . Educational and Psychological Measurement, 20, 37-46.
13.
Cowen, E.L. (1961). The experimental analogue: An approach to research in psychotherapy. Psychological Reports, 8, 9-11.
14.
Dobrin, A. (1989). Ethical judgments of male and female social workers . Social Work, 34, 451-455.
15.
Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice. (1978 ). Adoption by four agencies of Uniform Guidelines on employee selection procedures. Federal Register, 43, 38290-38315.
16.
Fleiss, J.L. (1970). Estimating the reliability of interview data. Psychometrika, 35, 143-162.
17.
Fleiss, J.L. (1973). Statistical methods for rates and proportions . New York: Wiley.
18.
Fleiss, J.L., Spitzer, R.L., & Burdock, E.I. (1965). Estimating accuracy of judgment using recorded interviews. Archives of General Psychiatry, 12, 562-567.
19.
Fleiss, J.L., Spitzer, R.L., Endicott, J., & Cohen, J. (1972). Quantification of agreement in multiple psychiatric diagnosis. Archives of General Psychiatry, 26, 168-171.
20.
Franklin, D.L. (1985). Differential clinical assessments: The influence of class and race. Social Service Review, 59, 44-61.
21.
Garb, H.N. (1989). Clinical judgment, clinical training, and professional experience. Psychological Bulletin, 105, 387-396.
22.
Green, B.F. (1981). A primer of testing. American Psychologist, 36, 1001-1011.
23.
Hill, D.S., Edwards, H.P., Crawford, W., & Warner, J. (1986). Oral examinations for licensing or certification: A manual for psychology boards. Birmingham, AL: American Association of State Psychology Boards.
24.
Hollon, S.D., & Kriss, M.R. (1984). Cognitive factors in clinical research and practice . Clinical Psychology Review, 4, 35-76.
25.
Kane, M.T. (1982). The validity of licensure examinations. American Psychologist, 37, 911-918.
26.
Krathwohl, D.R., Bloom, B.S., & Masia, B.B. (1964). Taxonomy of educational objectives: The classification of educational goals. Handbook II: Affective domain. New York: David McKay.
27.
Kutchins, H., & Kirk, S.A. (1988). The business of diagnosis. Social Work, 33, 215-220.
28.
Lodge, M. (1981). Magnitude scaling: Quantitative measurement of opinions. Beverly Hills, CA: Sage .
29.
McIver, J.P., & Carmines, E.G. (1981). Unidimensional scaling . Beverly Hills, CA: Sage.
30.
National Association of Social Workers. (1959 ). Use of judgments as data in social work research. Proceedings of a conference held by the Research Section of the National Association of Social Workers, Michigan State University, June 1958. New York: Author .
31.
Nunnally, J.C. (1964). Psychometric theory. New York : McGraw-Hill.
32.
Nurius, P.S., & Gibson, J.W. (1990). Clinical observation, inference, reasoning, and judgment in social work: An update. Social Work Research & Abstracts, 26, 18-25.
33.
Orcutt, B.A. (1964). A study of anchoring effects in clinical judgment . Social Service Review, 38, 408-417.
34.
Rock, D.L., Bransford, J.D., Maisto, S.A., & Morey, L. (1987). The study of clinical judgment: An ecological approach. Clinical Psychology Review, 7, 645-661.
35.
Sattler, J.M. (1988). Assessment of children (3rd ed.). San Diego: Jerome M. Sattler .
36.
Selltiz, C., Wrightsman, L.S., & Cook, S.W. (1976). Research methods in social relations (3rd ed.). New York: Holt, Rinehart & Winston.
37.
Shimberg, B. (1981). Testing for licensure and certification. American Psychologist, 36, 1138-1146.
38.
Spitzer, R.L., Cohen, J., Fleiss, J.L., & Endicott, J. (1967). Quantification of agreement in psychiatric diagnosis . Archives of General Psychiatry, 17, 83-87.
39.
Thomas, E.J. (1962). Experimental analogs of the casework interview . Social Work, 7, 24-30.
40.
Thorndike, R.L., & Hagen, E.P. (1977). Measurement and evaluation in psychology and education (4th ed.). New York : Wiley.
41.
U.S. Department of Health, Education, and Welfare, Public Health Services. (1977). Credentialing health manpower (DHEW Publication No. [05] 77-50057. Washington, DC: Author.
