Problems in Logic and Interpretation with Quantitative Syntheses of Single-Case Research: Mathur and Colleagues (1998) as a Case in Point

Abstract

This article focuses attention on some of the logic and interpretation problems inherent in doing quantitative syntheses of single-case research. Particular attention is given to the meta-analytic method of using percentage of nonoverlapping data, highlighting specific concerns related to withdrawal designs, sensitivity to variability in behavior, social significance of behavior change, and understanding the complex multiple variables that influence intervention effectiveness. As a case in point, we critique the use of PND in an article by Mathur, Kavale, Quinn, Forness, and Rutherford (1998) recently published in Behavioral Disorders.

Get full access to this article

View all access options for this article.

References

Allport

G. W.

(1962). The general and the unique in psychological science. Journal of Personality 30, 405–422.

Baer

D. M.

(1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167–172.

Baer

D. M.

, Salzberg

C. L.

, & Strain

P. S.

(1987). The nature of review: A polite response. Remedial and Special Education, 8, 204.

Baer

D. M.

, & Wolf

M. M.

(1970). The entry into natural communities of reinforcement. In Ulrich

, Stachnik

, & Mabry

(Eds.), Control of human behavior (Vol. 2, pp. 319–324). Glenview, IL: Scott, Foresman.

Baer

D. M.

, Wolf

M. M.

, & Risley

T. R.

(1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97.

Bandura

(1969). Principles of behavior modification. New York: Holt, Rinehart, & Winston.

Barlow

D. H.

, & Hersen

(1984). Single case experimental designs (2nd ed.). Boston: Allyn and Bacon.

Bergrin

A. E.

, & Strupp

H. H.

(1972). Changing frontiers in the science of psychotherapy. New York: Aldine-Atherton.

Blair

, & Ramey

C. T.

(1997). Early intervention for low-birth-weight infants and the path to second-generation research. In Guralnick

M. J.

(Ed.), The effectiveness of early intervention (pp. 93–201). Baltimore: Brookes.

10.

Blimling

G. S.

(1988). Meta-analysis: A statistical method for integrating the results of empirical studies. Journal of College Student Development, 20, 543–549.

11.

Busk

P. L.

, & Serlin

R. C.

(1992). Meta-analysis for single-case research. In Kratochwill

T. R.

, & Levin

J. R.

(Eds.), Single-case research design and analysis (pp. 187–211). Hillsdale, NJ: Erlbaum.

12.

Center

B. A.

, Skiba

R. J.

, & Casey

(1985–1986). A methodology for the quantitative synthesis of intra-subject design research. Journal of Special Education, 19, 387–400.

13.

Coie

J. D.

(1990). Adapting intervention to the problems of aggressive and disruptive rejected children. In Asher

S. R.

, & Coie

J. D.

(Eds.), Peer rejection in childhood (pp. 93–109). Cambridge, England: Cambridge University Press.

14.

Cook

T. D.

, & Leviton

L. C.

(1980). Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality, 48, 449–472.

15.

Dishion

J. J.

, & Patterson

G. R.

(1992). Age effects in parent training outcomes. Behavior Therapy, 23, 719–729.

16.

Glass

G. V.

(1976). Primary, secondary, and meta-analysis of research. Educational Research, 5, 3–8.

17.

Gresham

F. M.

(1997). Social competence and students with behavioral disorders: Where we've been, where we are, and where we should go. Education and Treatment of Children, 20, 233–249.

18.

Haring

(1992). The context of social competence: Relations, relationships, and generalization. In Odom

, McConnell

, & McEvoy

(Eds.), Social competence of young children with disabilities: Issues and strategies for intervention (pp. 307–320). Baltimore: Brookes.

19.

Haring

T. G.

, & Breen

C. G.

(1992). A peer-mediated social network intervention to enhance the social integration of persons with moderate and severe disabilities. Journal of Applied Behavior Analysis, 25, 319–333.

20.

Hawkins

(1991). Is social validity what we are interested in? Journal of Applied Behavior Analysis, 24, 205–213.

21.

Hersen

, & Barlow

D. H.

(1976). Single case experimental designs. New York: Pergamon.

22.

Jackson

G. B.

(1980). Methods of integrative research. Review of Educational Research, 50, 438–460.

23.

Kamps

D. M.

, Leonard

B. R.

, Vernon

, Dugan

E. P.

, & Delquadri

J. C.

(1992). Teaching social skills to students with autism to increase peer interactions in an integrated first-grade classroom. Journal of Applied Behavior Analysis, 25, 281–288.

24.

Kavale

(1984). Potential advantages of the meta-analysis technique for research in special education. The Journal of Special Education, 18, 61–72.

25.

Kavale

(1995). Meta-analysis at 20. Evaluation and the Health Professions, 18, 349–369.

26.

Kerlinger

F. N.

(1964). Foundations of behavioral research. New York: Holt, Rinehart & Winston.

27.

Kohler

F. W.

, Strain

P. S.

, Maretsky

, & DeCesare

(1990). Promoting positive and supportive interactions between preschoolers: An analysis of group-oriented contingencies. Journal of Early Intervention, 14, 327–341.

28.

Kratochwill

T. R.

, & Levin

J. R.

(1992). Single-case research design and analysis. Hillsdale, NJ: Erlbaum.

29.

Loeber

(1991). Antisocial behavior: More enduring than changeable? Journal of American Academy of Child and Adolescent Psychiatry, 30, 393–397.

30.

Mathur

S. R.

, Kavale

K. A.

, Quinn

M. M.

, Forness

S. R.

, & Rutherford

R. B.

(1998). Social skills intervention with students with emotional and behavioral problems: A quantitative synthesis of single-subject research. Behavioral Disorders, 23, 193–201.

31.

McEvoy

M. A.

, Nordquist

V. M.

, Twardosz

, Heckman

K. A.

, Wehby

J. H.

, & Denny

R. K.

(1988). Promoting autistic children's peer interaction in an integrated early childhood setting using affection activities. Journal of Applied Behavior Analysis, 21, 193–200.

32.

McFall

(1982). A review and reformulation of the concept of social skills. Behavioral Assessment, 4, 1–33.

33.

McReynolds

L. V.

, & Kearns

K. P.

(1983). Single-subject experimental designs in communicative disorders. Baltimore: University Park Press.

34.

Paine

S. C.

, Hops

, Walker

H. M.

, Greenwood

C. R.

, Fleischman

D. H.

, & Guild

J. J.

(1982). Repeated treatment effects. Behavior Modification, 6, 171–199.

35.

Parsonson

B. S.

, & Baer

D. M.

(1992). The visual analysis of data, and current research into the stimuli controlling it. In Kratochwill

T. R.

, & Levin

J. R.

(Eds.), Single-case research design and analysis (pp. 20–40). Hillsdale, N): Erlbaum.

36.

Patterson

G. R.

, Reid

, & Dishion

T. J.

(1992). Antisocial boys. Eugene, OR: Castalia.

37.

Scotti

J. R.

, Evans

I. M.

, Meyer

L. H.

, & Walker

(1991). A meta-analysis of intervention research with problem behavior: Treatment validity and standards of practice. American Journal on Mental Retardation, 96, 233–256.

38.

Scruggs

T. E.

, Mastropieri

M. A.

, & Castro

(1987). The quantitative synthesis of single subject research: Methodology and validation. Remedial and Special Education, 8, 24–33.

39.

Sechrest

, McKnight

, & McKnight

(1996). Calibration of measures for psychotherapy outcome studies. American Psycholgist, 51, 1065–1071.

40.

Sidman

(1960). Tactics of scientific research. New York: Research Press.

41.

Slavin

R. E.

(1984). Meta-analysis in education: How has it been used? Educational Researcher, 15, 5–11.

42.

Strain

P. S.

(1987). Comprehensive evaluation of young autistic children. Topics in Early Childhood Special Education, 7, 97–110.

43.

Strain

P. S.

, & Kohler

F. W.

(1995). Analyzing predictors of daily social skill performance. Behavioral Disorders, 21, 78–87.

44.

Strain

P. S.

, Kohler

F. W.

, Storey

, & Danko

(1994). Teaching preschoolers with autism to self-monitor their social interactions: An analysis of results in home and school settings. Journal of Emotional and Behavioral Disorders, 2, 78–88.

45.

Strain

P. S.

, Shores

R. E.

, & Timm

M. A.

(1977). Effects of peer initiations on the social behavior of withdrawn preschoolers. Journal of Applied Behavior Analysis, 10, 289–298.

46.

Strain

P. S.

, Steele

, Ellis

, & Timm

M. A.

(1982). Long-term effects of oppositional child treatment with mothers as therapists and therapist trainers. Journal of Applied Behavior Analysis, 15, 163–169.

47.

Tawney

J. W.

, & Gast

D. L.

(1984). Single subject research in special education. Columbus, OH: Merrill.

48.

Walker

H. M.

, Greenwood

C. R.

, Hops

, & Todd

N. M.

(1979). Differential effects of reinforcing topographic components of social interaction. Behavior Modification, 3, 291–321.

49.

White

D. M.

, Rusch

F. R.

, Kazdin

A. E.

, & Hartmann

D. P.

(1989). Applications of meta-analysis in individual-subject research. Behavioral Assessment, 11, 281–296.

50.

White

J. L.

, Moffitt

T. E.

, Earls

, Robins

, & Silva

P. A.

(1990). How early can we tell? Predictors of early childhood conduct disorder and adolescent delinquency. Criminology, 28, 507–533.