Measurement Issues in EBD Research: What we know and how we Know It

Abstract

Since the passage of the No Child Left Behind Act in 2002, educators have been advised that it is no longer acceptable to allow ideology, fads, or untested procedures to define teaching. Therefore, knowing what counts as evidence and why it counts is the primary issue of the early 21st century in the field of Emotional and Behavioral Disorders (EBD). Over the last several years, we have been engaged in an effort to define our field as a set of standardized, evidence-based, and sustainable practices. Issues related to the validity of research are at the center of that discussion. This article responds to the work of other authors in this special issue through a discussion of the role of science in EBD, various aspects of validity related to current interventions, and threats to the systematic determination of a valid research base in the field.

Get full access to this article

View all access options for this article.

References

Baer

D. M.

, Wolf

M. M.

, & Risley

T. R.

(1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97.

Bergin

A. E.

(1966). Some implications of psychotherapy research for therapeutic practice. Journal of Abnormal Psychology, 71, 235–246.

Brigham

F. J.

, Gustashaw

W. E.

, Wiley

, & St. Peter Brigham

(2004). Research in the wake of the No Child Left Behind Act: Why the controversies will continue and some suggestions for controversial research. Behavioral Disorders, 29, 300–310.

Campbell

D. T.

, & Stanley

J. C.

(1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.

Carr

E. G.

, Dunlap

, Horner

R. H.

, Koegel

R. L.

, Turnbull

A. P.

, Sailor

, Anderson

J. L.

, Albin

R. W.

, Koegel

L. K.

, & Fox

(2002). Positive behavior support: Evolution of an applied science. Journal of Positive Behavior Interventions, 4, 4–16.

Carr

J. E.

, Austin

J. L.

, Britton

L. M.

, Kellum

K. K.

, & Bailey

J. S.

(1999). An assessment of social validity trends in applied behavior analysis. Behavioral Interventions, 14, 223–231.

Carr

J. E.

, & Sidener

T. M.

(2002). On the relation between applied behavior analysis and positive behavioral support. The Behavior Analyst, 25, 245–253.

Cook

B. G.

, Tankersley

, Cook

, & Landrum

(2000). Teachers’ attitudes toward their included students with disabilities. Exceptional Children, 67(1), 115–135.

Cook

T. D.

, & Campbell

D. T.

(1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.

10.

Crockett

J. B.

, & Kauffman

J. M.

(1999). The least restrictive environment: Its origins and interpretation in special education. Mahwah, NJ: Lawrence Erlbaum Associates.

11.

Cronbach

L. J.

, & Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

12.

Eysenck

H. J.

(1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16, 319–324.

13.

Feynman

R. E.

(1985). Surely you're joking, Mr. Feynman: Adventures of a curious character. New York: Bantam Books.

14.

Forness

S. R.

(in press). Strengthening the research base in special education: The pursuit of evidence-based practice. In Landrum

, Crockell

, & Gerber

(Eds.), Strategies for achieving reform in special education. Mahwah, NJ: Lawrence Erlbaum Associates.

15.

Fuchs

, & Fuchs

L. S.

(1994). Inclusive schools movement and the radicalization of special education reform. Exceptional Children, 60, 294–309.

16.

Gallagher

D. J.

, Heshusius

, Iano

R.P.

, & Skrtic

T. M.

(2004). Challenging orthodoxy in special education. Denver: Love Publishing.

17.

Gellner

E. S.

(1991). Where's the validity in social validity? Journal of Applied Behavior Analysis, 24, 179–187.

18.

Hawkins

R. P.

(1991). Is social validity what we are interested in? Argument for a functional approach. Journal of Applied Behavioral Analysis, 24, 205–213.

19.

Herschbach

(1996). Imaginary gardens with real toads. In Gross

, Levitt

, & Lewis

M. W.

(Eds.), The flight from science and reason. Baltimore: Johns Hopkins University Press.

20.

Heward

W. L.

(2003). Ten faulty notions about teaching and learning that hinder the effectiveness of special education. The Journal of Special Education, 36, 186–205.

21.

Horner

R. H.

, Carr

E. G.

, Halle

, McGee

, Odom

, & Wolery

(in press). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children.

22.

Horner

R. H.

, Todd

A. W.

, Lewis-Palmer

, Irvin

L. K.

, Sugai

, & Boland

J. B.

(2004). The school-wide evaluation tool (SET). Journal of Positive Behavior Interventions, 6, 3–12.

23.

Hunter

(1999). Systems change and the transition to inclusive systems. In Coutinho

M. J.

, & Repp

A. C.

(Eds.), Inclusion: The integration of students with disabilities (pp. 135–151). New York: Wadsworth.

24.

Jensen

P. S.

(2003). What is the evidence for evidence-based treatments? Emotional and Behavioral Disorders in Youth, 3, 37–41, 48.

25.

Kanfer

F. H.

, & Phillips

J. S.

(1970). Learning foundations of behavior therapy. New York: Wiley.

26.

Kauffman

(2001). Characteristics of emotional and behavioral disorders of children and youth. Upper Saddle River, NJ: Merrill/Prentice Hall.

27.

Kauffman

J. M.

(2002). Education deform: Bright people sometimes say stupid things about education. London: Scarecrow Press.

28.

Kazdin

A. E.

(1977). Assessing the clinical or applied importance of behavior change through social validation. Behavioral Modification, 1, 427–452.

29.

Kazdin

A. E.

(1978). History of behavior modification: Experimental foundations of contemporary research. Baltimore: University Park Press.

30.

Kazdin

A. E.

(1992). Research design in clinical psychology (2nd ed.). Boston: Allyn & Bacon.

31.

Krasner

(1971). Behavior therapy. In Mussen

P.H.

(Ed.), Annual Review of Psychology, 22, (pp. 114–138) Palo Alto, CA: Annual Reviews.

32.

Lewis

T. J.

, Hudson

, Richter

, & Johnson

(2004). Scientifically supported practices in emotional and behavioral disorders: A proposed approach and brief review of current practices. Behavioral Disorders, 29, 247–259.

33.

Lieberman

L. M.

(1992). Preserving special education … For those who need it. In Stainback

, & Stainback

(Eds.), Controversial issues confronting special education (pp. 13–25). Boston: Allyn & Bacon.

34.

McClellan

J. M.

, & Werry

J. S.

(2003). Evidence-based treatments in child and adolescent psychiatry: An inventory. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 1388–1400.

35.

McLeskey

, & Waldron

(2002). Professional development and inclusive schools: Reflections on effective practice. The Teacher Educator, 37(3), 159–172.

36.

Moats

L. C.

(2000, October). Whole language lives on: The illusion of “balanced” reading instruction. Learning Disabilities OnLine. Retrieved September 3, 2004, from http://www.ldonline.org/ld_indepth/reading/whole_language_lives_on.html

37.

Mostert

M. P.

, Kauffman

J. M.

, & Kavale

K. A.

(2003). Truth and consequences. Behavioral Disorders, 28, 333–347.

38.

Nickerson

R. S.

(1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2, 175–220.

39.

Odom

S. L.

, Brantlinger

, Gersten

, Horner

, Thompson

, & Harris

(in press). Research in special education: Scientific methods and evidence-based practices. Exceptional Children.

40.

Peck

, Sasso

G. M.

, & Stambaugh

(1999). Structural analysis of social behavior: Reliability and validity issues. Preventing School Failure, 43(1), 14–18.

41.

Sailor

, & Paul

(2004). Framing positive behavior support in the ongoing discourse concerning the politics of knowledge. Journal of Positive Behavior Interventions, 6, 37–49.

42.

Sailor

, & Skrtic

T. M.

(1996). School-linked services integration: Crisis and opportunity in the transition to a postmodern society. Remedial and Special Education, 17, 171–183.

43.

Sasso

G. M.

(2001). The retreat from inquiry and knowledge in special education. The Journal of Special Education, 34, 178–193.

44.

Sasso

G. M.

(2003). An examined life: A response to James Kauffman's reflections on the field. Behavioral Disorders, 28, 209–211.

45.

Sasso

G. M.

(2004, January). Evidence-based practice. Keynote presentation to the MSLBD (Midwest Symposium for Leadership in Behavioral Disorders) Think Tank, Kansas City, MO.

46.

Schwartz

I. S.

, & Baer

D. M.

(1991). Social validity assessments: Is current practice state of the art? Journal of Applied Behavior Analysis, 24, 189–204.

47.

Scott

T. M.

, Bucalos

, Liaupsin

, Nelson

C. M.

, Jolivette

, & DeShea

(2004). Using functional behavior assessment in general education settings: Making a case for effectiveness and efficiency. Behavioral Disorders, 29, 189–201.

48.

Scruggs

, & Mastropieri

(1996). Teacher perceptions of mainstreaming/inclusion, 1958–1995: A research synthesis. Exceptional Children, 63(1), 59–74.

49.

Shermer

(2002). Why people believe weird things: Pseudoscience, superstition and other confusions of our time. New York: Holt, Rinehart and Winston.

50.

Sidman

(1960). Tactics of scientific research: Evaluating experimental data in psychology. New York: Basic Books.

51.

Simpson

, & Sasso

(1992). Full inclusion of students with autism in general education settings: Values vs. science. Focus on Autistic Behavior, 7(3), 1–13.

52.

Sommers

C. H.

(2000). The war against boys. New York: Simon & Schuster.

53.

Stainback

, & Stainback

(1991). A rationale for integration and restructuring: A synopsis. In Lloyd

J. W.

, Singh

N. N.

, & Repp

A. C.

(Eds.), The regular education initiative: Alternative perspectives on concepts, issues, and model (pp. 225–239). Sycamore, IL: Sycamore.

54.

Stanovich

K. E.

(2004). How to think straight about psychology. New York: Pearson.

55.

Sugai

, Lewis-Palmer

, Todd

, & Horner

R. H.

(2001). School-wide evaluation tool. Eugene, OR: University of Oregon.

56.

Turnbull

H.R.

III , Wilcox

B. L.

, Stowe

, & Turnbull

A. P.

(2001). IDEA requirements for use of PBS: Guidelines for responsible agencies. Journal of Positive Behavior Interventions, 3, 11–18.

57.

U.S. Department of Education. (2002). No child left behind: A desktop reference. Washington, DC: Author.

58.

Vitiello

(2002). Current research on mental health treatments for children and adolescents. Journal of Emotional and Behavioral Disorders in Youth, 3, 87–88, 99–100.

59.

Wacker

D. P.

, & Berg

W. K.

(2002). PBS as a service delivery system. Journal of Positive Behavioral Interventions, 4, 25–28.

60.

Wolf

M. M.

(1978). Social validity: the case for subjective measurement, or how behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.

61.

Young

(1999). Ceasefire: Why women and men must join forces to achieve true equality. New York: The Free Press.