Given the paucity of previous research, we examined the occurrence of scoring error on widely used objective personality tests and examined its possible relation to two factors: scoring procedure complexity (SPC) and commitment to accuracy (CTA). We doublechecked the scoring of three tests (MMPI, Beck Depression Inventory, Spielberger State/Trait Anxiety Inventory) across three settings. Each of the tests were misscored at a surprisingly high rate in at least one setting, and some such errors altered major interpretive implications. Tests of higher SPC showed greater error rates, but high CTA greatly reduced the occurrence of error across levels of SPC. Unexpected sources of error were also uncovered, such as commercial computer scoring errors and disagreement in scoring standards among test publishers. Practical suggestions for improving scoring accuracy are offered.
Allard, G. A.
, Butler, J., Shea, M. T., & Faust, D. (1995). Errors in hand scoring objective personality tests: The case of the Personality Diagnostic Questionnaire-Revised (PDQ-R). Professional Psychology: Research and Practice, 26, 304-308.
2.
Beasley, M. G.
, Lobasher, M., Henley, S., & Smith, I. (1988). Errors in computation of WISC and WISC-R intelligence quotients from raw scores. Journal of Child Psychology and Psychiatry, 29, 101-104.
3.
Beck, A. T.
, & Steer, R. A. (1987). Beck Depression Inventory manual. San Antonio, TX: The Psychological Corporation.
4.
Blakey, W. A.
, Fantuzzo, J. W., Gorsuch, R. L., & Moon, G. W. (1987). A peer-mediated, competency-based training package for administering and scoring the WAIS-R. Professional Psychology: Research and Practice, 18, 17-20.
5.
Boehm, A. E.
, Duker, J., Haesloop, M. D., & White, M. A. (1974). Behavioral objectives in training for competence in the administration of individual intelligence tests. Journal of School Psychology, 12, 150-157.
6.
Butcher, J. N.
, Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). Minnesota Multiphasic Personality Inventory-2: Manual for administration and scoring. Minneapolis: University of Minnesota Press.
7.
Connor, R.
, & Woodall, F. E. (1983). The effects of experience and structured feedback on WISC-R error rates made by student-examiners. Psychology in the Schools, 20, 376-379.
8.
Dahlstrom, W. G.
, Welsh, G. S., & Dahlstrom, L. E. (1972). An MMPI handbook: Volume 1: Clinical interpretation. Minneapolis: University of Minnesota Press.
9.
Faust, D.
(1984). The limits of scientific reasoning. Minneapolis: University of Minnesota Press.
10.
Fowler, R. D.
, & Coyle, F. A. (1968). Scoring error on the MMPL. Journal of Clinical Psychology, 25, 62-63.
11.
Greene, R. L.
(1991). The MMPI-2/MMPI: An interpretive handbook. Boston: Allyn and Bacon.
12.
Johnson, D. L.
, & Candler, A. C. (1985). Using a small computer for test score conversion. College Student Journal, 19, 102-106.
13.
Levenson, R. L., Jr.
, Golden-Scaduto, C. J., Aiosa-Karpas, C. J., & Ward, A. W. (1988). Effects of examiners' education and sex on presence and type of clerical errors made on WISC-R protocols. Psychological Reports, 62, 659-664.
14.
Miller, C. K.
, & Chansky, N. M. (1972). Psychologists' scoring of WISC protocols. Psychology in the Schools, 9, 144-152.
15.
Miller, C. K.
, Chansky, N. M., & Gredler, G. R. (1970). Rater agreement on WISC protocols. Psychology in the Schools, 7, 190-193.
16.
Moreland, K. L.
, Eyde, L. D., Robertson, G. J., Primoff, E. S., & Most, R. B. (1995). Assessment of test user qualifications: A research-based measurement procedure. American Psychologist, 50, 14-23.
17.
Oakland, T.
, Lee, S. W., & Axelrod, K. M. (1975). Examiner differences on actual WISC protocols.Journal of School Psychology, 13, 227-233.
18.
Piotrowski, C.
, & Keller, J. W. (1989). Psychological testing in outpatient mental health facilities: A national study. Professional Psychology: Research and Practice, 20, 423-425.
19.
Piotrowski, C.
, & Keller, J. W. (1992). Psychological testing in applied settings: A literature review from 1982-1992. The Journal of Training & Practice in Professional Psychology, 6, 74-82.
20.
Piotrowski, C.
, & Lubin, B. (1990). Assessment practices of health psychologists: Survey of APA Division 38 clinicians. Professional Psychology: Research and Practice, 21, 99-106.
21.
Ryan, J. J.
, Prifitera, A., & Powers, L. (1983). Scoring reliability on the WAIS-R. Journal of Consulting and Clinical Psychology, 51, 149-150.
Sherretts, S.
, Gard, G., & Langner, H. (1979). Frequency of clerical errors on WISC protocols. Psychology in the Schools, 16, 495-496.
24.
Slate, J. R.
, & Chick, D. (1989). WISC-R examiner errors: Cause for concern. Psychology in the Schools, 26, 78-84.
25.
Slate, J. R.
, & Hunnicutt, L. C., Jr., (1988). Examiner errors on the Wechsler scales. Journal of Psychoeducational Assessment, 6, 280-288.
26.
Slate, J. R.
, & Jones, C. H. (1990). Identifying students' errors in administering the WAIS-R. Psychology in the Schools, 27, 83-87.
27.
Slate, J. R.
, Jones, C. H., & Murray, R. A. (1991). Teaching administration and scoring of the Wechsler Adult Intelligence Scale-Revised: An empirical evaluation of practice administrations. Professional Psychology: Research and Practice, 22, 375-379.
28.
Slate, J. R.
, Jones, C. H., Murray, R. A., & Coulter, C. (1993). Evidence that practitioners err in administering and scoring the WAIS-R. Measurement and Evaluation in Counseling and Development, 25, 156-161.
29.
Spielberger, C. D.
(1983). State-Trait Anxiety Inventory for adults: Sampler set, manual, test, scoring key. Palo Alto, CA: Consulting Psychologists Press.
30.
University of Minnesota
. (1983). Minnesota Multiphasic Personality Inventory manual for administration and scoring. Minneapolis: University of Minnesota Press.
31.
Wade, T. C.
, & Baker, T. B. (1977). Opinions and use of psychological tests: A survey of clinical psychologists. American Psychologist, 32, 874-882.
