Sage Journals: Discover world-class research

Abstract

Studies that show marginal generality of test-session behaviors to non-test situations are criticized for addressing the question of validity without first establishing evidence of observational reliability. Internal consistency indices of reliability for observational instruments, while necessary, are insufficient without evidence of interobserver and intraobserver agreement. Data are presented that show that interobserver and intraobserver agreement is inconsistent both in direction and level of ratings when 42 test-session behaviors are rated during each of 21 test sessions in which the WPPSI-R was used as the standardized assessment instrument. A systematic research program on the utility of this clinical practice is needed, with a primary focus on improving observational reliability. Until data that substantiate this clinical practice are accumulated, clinicians are urged to exercise caution when they are drawing inferences from test-session behaviors.

Get full access to this article

View all access options for this article.

References

Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.

Fassnacht, G. (1982). Theory and practice of observing behavior. New York: Academic Press.

Glutting, J. J. , & McDermott, P. A. (1988). Generality of test-session observations to kindergarteners' classroom behavior. Journal of Abnormal Child Psychology, 16, 527-537.

Glutting, J. J. , & Oakland, T. (1993). Guide to the assessment of test session behavior. San Antonio, TX: Psychological Corporation.

Glutting, J. J. , Oakland, T. , & McDermott, P. A. (1989). Observing child behavior during testing: Constructs, validity, and situational generality. Journal of School Psychology, 27, 155-164.

Gordon, M. , DiNiro, D. , Mettelman, B. B. , & Tallmadge, J. (1989). Observations of test behavior, quantitative scores, and teacher ratings. Journal of Psychoeducational Assessment, 7, 141-147.

Kaufman, A. S. , & Reynolds, C. R. (1984). Intellectual and academic achievement tests. In T. H. Ollendick & M. Hersen (Eds.), Child behavioral assessment: Principle and procedures (pp. 185-200). New York: Pergamon Press.

McDermott, P. A. (1988). Agreement among diagnosticians or observers: Its importance and determination. Professional School Psychology, 3, 225-240.

Oakland, T. , & Glutting, J. J. (1990). Examiner observations of children's WISC-R test-related behaviors: Possible socioeconomic status, race, and gender effects. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 86-90.

10.

Sattler, J. M. (1990). Assessment of children. San Diego, CA: Author.

Reliability and Validity of Test-Session Behavior Observations: Putting the Horse Before the Cart

Abstract

Get full access to this article

References