Sage Journals: Discover world-class research

Abstract

This study investigated examiner accuracy in the completion of protocols during actual test sessions. The performances of 22 language handicapped preschoolers, each tested by a familiar and strange examiner, were videotaped. Eleven examiners were inexperienced; another 11 examiners were professionally experienced. Protocols generated by the actual examiners and by independent raters, who viewed the tapes, were scored. Analysis of variance revealed that examiners' protocols yielded significantly higher scores than did raters' protocols; children scored significantly higher when tested by familiar, rather than unfamiliar examiners; and children's scores were similar whether examiners were experienced or inexperienced. Additionally, a significant three-way interaction among protocol completer (rater vs. examiner), familiarity, and examiner experience indicated that examiner in- accuracy was greater for inexperienced testers during familiar testing sessions, but greater for experienced testers during unfamiliar sessions. Implications for psychoeducational assessment are discussed.

Get full access to this article

View all access options for this article.

References

Anastasi, A. (1982). Psychological testing (5th ed.). New York, MacMillan.

Einhorn, H. J. , & Hogarth, R. M. (1978). Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 85 (5), 395-416.

Fiscus, E. G. (1975). The effects of pre-test information on school psychologists' scoring of the Wechsler Intelligence Scale for Children. Unpublished doctoral dissertation. Ohio State University.

Fuchs, D. , Featherstone, N. , Garwick, D. R. , & Fuchs, L. S. (1984). Effects of examiner familiarity and task characteristics on speech- and language-impaired children's test performance. Measurement and Evaluation in Guidance 16 (4), 198-204.

Fuchs, L. S. , & Fuchs, D. (1984). An analysis of examiner error patterns. Unpublished manuscript.

Fuchs, D. , Fuchs, L. S. , Dailey, A. M. , & Power, M. H. (1983). Effects of pretest contact with experienced and inexperienced examiners on handicapped children's performance (Reseach Report No. 110). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.

Fuchs, D. , Fuchs, L. S. , Garwick, D. R. , & Featherstone, N. (1983). Test performance of language handicapped children with familiar and unfamiliar examiners. Journal of Psychology, 114, 37-46.

Fuchs, D. , Fuchs, L. S. , Power, M. H. , & Dailey, A. M. (1983). Systematic bias in the assessment of handicapped children (Research Report No. 134). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.

Fuchs, L. S. , & Fuchs, D. (in press). Criterion-referenced assessment without measurement: How accurate for special education? Remedial and Special Education.

10.

Guilford, J. P. (1936). Psychometric methods. New York: McGraw Hill.

11.

Kaspar, J. C. , Throne, F. M. , & Schulman, J. L. (1968). A study of the interjudge reliability in scoring the responses of a group of mentally retarded boys to three WISC subscales. Educational and Psychological Measurement, 28, 469-477.

12.

Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABC's of reliability. Journal of Applied Behavior Analysis, 10, 141-150.

13.

Mahan, T. W. (1963). Diagnostic consistency and prediction: A note on graduate student skills. Personnel and Guidance Journal, 42, 364-367.

14.

McDermott, P. A. (1980). Congruence and typology of diagnosis in school psychology: An empirical study. Psychology in the Schools, 17, 12-24.

15.

Miller, C. K. , & Chansky, N. M. (1972). Psychologists' scoring of WISC protocols. Psychology in the Schools, 9, 144-152.

16.

Miller, C. K. , Chanksy, N. M. , & Gredler, G. R. (1970). Rater agreement on WISC protocols. Psychology in the Schools, 7. 190-193.

17.

Plumb, G. R. , & Charles, D. C. (1955). Scoring difficulty of Wechsler Comprehension responses. Journal of Educational Psychology, 46, 179-183.

18.

Rosenthal, R. (1980). Examiner effects in behavioral research (Enlarged paperback edition). New York: Irvington.

19.

Thompson, R. H. , White, K. R.. & Morgan, D. P. (1982). Teacher-student interaction patterns in classrooms with main-streamed mildly handicapped students. American Educational Research Journal, 19, 220-236.

20.

Warren, S. A. , & Brown, W. G. (1973). Examiner scoring errors on individual intelligence tests. Psychology in the Schools, 10, 118-122.

21.

Watson, C. G. (1967). Interjudge agreement of draw-a-person diagnostic impressions. Journal of Projective Techniques and Personality Assessment, 31, 42-45.

22.

Zimmerman, I. L. , Steiner, V. G. , & Pond, R. E. (1979). Preschool language scale. Columbus, Ohio: Charles E. Merrill.

Examiner Accuracy during Protocol Completion

Abstract

Get full access to this article

References