Percentage of All Non-Overlapping Data (PAND)

Abstract

Although single-case researchers are not accustomed to analyzing data statistically, standards for research and accountability from government and other funding agents are creating pressure for more objective, reliable data. In addition, “evidence-based interventions” movements in special education, clinical psychology, and school psychology imply reliable data summaries. Within special education, two heavily debated single-case research (SCR) statistical indices are “percentage of non-overlapping data” (PND) and the regression effect size, R² . This article proposes a new index—PAND, the “percentage of all non-overlapping data”—to remedy deficiencies of both PND and R² . PAND is closely related to the established effect size, Pearson's Phi , the “fourfold point correlation coefficient.” The PAND/ Phi procedure is demonstrated and applied to 75 published multiple baseline designs to answer questions about typical effect sizes, relationships with PND and R² , statistical power, and time efficiency. Confidence intervals and p values for Phi also are demonstrated. The findings are that PAND/ Phi and PND correlate equally well to R² . However, only PAND/Phi could show adequate power for most of the multiple baseline designs sampled. The findings suggest that PAND/Phi may meet the requirement for a useful effect size for multiple baseline and other longer designs in SCR.

Get full access to this article

View all access options for this article.

References

Abelson, R.P. (1985). A variance explanation paradox: When a little is a lot. Psychological Bulletin, 97, 129—133.

Allison, D.B. , & Gorman, B.S. (1993). Calculating effect sizes for meta-analysis: The case of the single case. Behaviour Research and Therapy, 31(6), 621—631.

Allison, D.B. , & Gorman, B.S. (1994). Make things as simple as possible, but no simpler: A rejoinder to Scruggs and Mastropieri. Behaviour Research and Therapy, 32(8), 885—890.

Bloom, M. , Fischer, J. , & Orme, J.G. (2003). Evaluating practice: Guidelines for accountable professionals (4th ed.). Needham Heights, MA: Allyn & Bacon.

Breckler, S.J. (1990). Application of covariance structure modeling in psychology: Cause for concern? Psychological Bulletin, 107, 260—273.

Brossart, D.F. , Parker, R.I. , Olson, E.A. , & Lakshmi, M. (2005). The relationship between visual analysis and five statistical analyses in a simple AB single-case research design. Behavior Modification.

Busk, P.L. , & Marascuilo, L.A. (1992). Statistical analysis in single-case research: Issues, procedures, and recommendations, with applications to multiple behaviors. In T. R. Kratochwill & J. Levin , R. (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 159—185). Hillsdale, NJ: Erlbaum.

Busk, P.L. , & Serlin, R.C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 187— 212). Hillsdale, NJ: Erlbaum.

Chambless, D.L. , & Ollendick, T.H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685—716.

10.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

11.

Davison, A.C. , & Hinkley, D.V. (1997). Bootstrap methods and their applications. Cambridge, UK: Cambridge University Press.

12.

DeProspero, A. , & Cohen, S. (1979). Inconsistent visual analyses of intra-subject data. Journal of Applied Behavior Analysis, 12, 573—579.

13.

Education Sciences Reform Act of 2002 (P.L. 107—279). Retrieved September 3, 2003, from http://www.ed.gov/news/pressreleases/2002/11/11062002a.html

14.

Fleiss, J.L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.

15.

Forness, S.R. (2001). Special education and related services: What have we learned from meta-analysis? Exceptionality, 9(4), 185—197.

16.

Good, P.I. (2001). Resampling methods: A practical guide to data analysis. Boston: Birkhauser.

17.

Harbst, K.B. , Ottenbacher, K.J. , & Harris, S.R. (1991). Interrater reliability of therapists' judgments of graphed data. Physical Therapy, 71, 107— 115.

18.

Hays, W.L. (1988). Statistics (4th ed.). Philadelphia: Holt, Rinehart & Winston.

19.

Hintze, J. (2004). NCSS and PASS: Number cruncher statistical systems [Computer software]. Kaysville, UT.

20.

Horner, R.H. , Carr, E.G. , Halle, J. , McGee, G. , Odom, S. , & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165—179.

21.

Kavale, K.A. , Mathur, S.R. , Forness, S.R. , Quinn, M.M. , & Rutherford, R.B. , Jr. (2000). Right reason in the integration of group and single-subject research in behavioral disorders. Behavioral Disorders, 25, 142— 157.

22.

Kazdin, A.E. (1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford University Press.

23.

Keselman, H.J. , Huberty, C. , Lix, L.M. , Olejnik, S. , Cribbie, R.A. , Donahue, B. , et al. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68, 350—386.

24.

Kirk, R.E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746—759.

25.

Kratochwill, T.R. , & Brody, G.H. (1978). Single subject designs: A perspective on the controversy over employing statistical inference and implications for research and training in behavior modification. Behavior Modification, 2, 291—307.

26.

Kratochwill, T.R. , & Stoiber, K.C. (2000). Empirically supported interventions and school psychology: Conceptual and practical issues: Part II. School Psychology Quarterly, 15, 233—253.

27.

Kratochwill, T.R. , & Stoiber, K.C. (2002). Evidence-based interventions in school psychology: Conceptual foundations of the Procedural and Coding Manual of Division 16 and the Society for the Study of School Psychology Task Force. School Psychology Quarterly, 17, 341—389.

28.

Linden Software Ltd, (1998). i-extractor [Graph digitizing software]. United Kingdom: Author.

29.

Mostert, M.P. (2001). Characteristics of meta-analyses reported in mental retardation, learning disabilities, and emotional and behavioral disorders . Exceptionality, 9(4), 199—225.

30.

31.

Odom, S.L. , Brantlinger, E. , Gersten, R. , Horner, R.H. , Thompson, B. , & Harris, K.R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71(2), 137—148.

32.

Ottenbacher, K.J. (1990). Visual inspection of single-subject data: An empirical analysis. Mental Retardation, 28, 283—290.

33.

Park, H. , Marascuilo, L. , & Gaylord-Ross, R. (1990). Visual inspection and statistical analysis of single-case designs. Journal of Experimental Education, 58, 311—320.

34.

Parker, R.I. , Brossart, D.F. , Callicott, K.J. , Long, J.R. , Garcia de Alba, R. , Baugh, F.G. , et al. (2005). Effect sizes in single case research: How large is large? School Psychology Review, 34(1), 116—132.

35.

Parker, R.I. , Cryer, J. , & Byrns, G. (in press). Controlling trend in single case research. School Psychology Quarterly.

36.

Parker, R.I. , Cryer, J. & Byrns, G. (2006). Controlling trend in single case research. School Psychology Quarterly. 21(3) 418-440.

37.

Parsonson, B.S. , & Baer, D.M. (1992). The visual analysis of data, and current research into the stimuli controlling it. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 15—40). Hillsdale, NJ: Erlbaum.

38.

Rosenthal, R. (1991). Meta-analysis procedures for social science research . Beverly Hills, CA: Sage.

39.

Rosenthal, R. , Rosnow, R.L. , & Rubin, D.B. (2000). Contrasts and e fect sizes in behavioral research: A correlational approach. Cambridge University Press: Cambridge, UK.

40.

Scruggs, T.E. , & Mastropieri, M.A. (1994). The utility of the PND statistic: A reply to Allison and Gorman. Behaviour Research and Therapy, 32, 879—883.

41.

Scruggs, T.E. , & Mastropieri, M.A. (1998). Summarizing single-subject research: Issues and applications. Behavior Modification, 22, 221—242.

42.

Scruggs, T.E. , & Mastropieri, M.A. (2001). How to summarize single-participant research: Ideas and applications. Exceptionality, 9, 227—244.

43.

Scruggs, T.E. , Mastropieri, M.A. , & Casto, G. (1987). The quantitative synthesis of single subject research: Methodology and validation. Remedial and Special Education , 8(2), 24—33.

44.

Shavelson, R. J. , Towne, L. , (Eds.) & the Committee on Scientific Principles for Education Research. (2002). Scientific research in education. Washington, DC: National Academy Press.

45.

Stats Direct Ltd (2005). StatsDirect [Computer software]. Cheshire, UK: Author.

46.

Steiger, J.H. (2005) NDC (Non-Central Distribution Calculator). Down-loadable from http://www.statpower.net/page5.html

47.

Steiger, J.H. , & Fouladi, R.T. (1992). R2: A computer program for interval estimation, power calculation, and hypothesis testing for the squared multiple correlation. Behavior Research Methods, Instruments, & Computers, 4, 581—582.

48.

Tawney, J.W. , & Gast, D.L. (1984). Single subject research in special education. Columbus, OH: Merrill.

49.

Thompson, B. , Diamond, K.E. , McWilliam, R. , Snyder, P. , & Snyder, S.W. (2005). Evaluating the quality of evidence from correlational research for evidence-based practice. Exceptional Children , 71, 181—194.

50.

White, O.R. (1987). Some comments concerning: “The quantitative synthesis of single-subject research.” Remedial and Special Education, 8(2), 34—39.

51.

Whitehurst, G. (2004, April). Wisdom of the head, not the heart. Distinguished Public Policy Lecture, Institute for Policy Research, North-western University, Evanston, IL.