Investigation of Two Preliminary Analysis-Altering Elements: Ordinate Scaling and DPPXYR

Abstract

The purpose of this pre-registered study (Peltier & McKenna) was to conceptually replicate if the truncation of the ordinate and DPPXYR increased analysts’ estimation of a functional relation and magnitude of treatment effect. Visual analysts (n = 27) evaluated eight data sets reporting null (n = 2), small (n = 2), moderate (n = 2), and large (n = 2) effects. Each data set was graphed six times with manipulations of the ordinate and DPPXYR, resulting in 48 ABAB graphs. We estimated two separate three-level mixed effect models with variations nested in datasets and nested in participants to evaluate the impact of graph characteristics for (1) confidence in determining a functional relation and (2) the estimated magnitude of the treatment effect. We included ordinate scaling and DPPXYR at level 1 and graph effect size at level 2, including all interactions. Overall, graph manipulation consistently did not impact confidence in a functional relation. Results suggest mixed findings for graph manipulation on the estimated magnitude of the treatment effect. Findings will be couched in current literature and recommendations for graph construction and future research will be discussed.

Keywords

single-case research designs visual analysis graph manipulation ordinate scaling DPPXYR

Get full access to this article

View all access options for this article.

References

Arizona Software. (2010). GraphClick. http://www.arizona-software.ch/graphclick/

Bailey

D. B.

(1984). Effects of lines of progress and semilogarithmic charts on ratings of charted data. Journal of Applied Behavior Analysis, 17(3), 359–365. https://doi.org/10.1901/jaba.1984.17-359

Bates

Maechler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.

*Briere

D. E.

III Simonsen

(2011). Self-monitoring interventions for at-risk middle school students: The importance of considering function. Behavioral Disorders, 36(2), 129–140. https://doi.org/10.1177%2F019874291103600204

*Bruhn

Watt

(2012). Improving behavior by using multicomponent self-monitoring within a targeted reading intervention. Behavioral Disorders, 38(1), 3–17. https://doi.org/10.1177%2F019874291203800102

*Coogan

B. A.

Kehle

T. J.

Bray

M. A.

Chafouleas

S. M.

(2007). Group contingencies, randomization of reinforcers, and criteria for reinforcement, self-monitoring, and peer feedback on reducing inappropriate classroom behavior. School Psychology Quarterly, 22(4), 540–556. https://psycnet.apa.org/doi/10.1037/1045-3830.22.4.540

Dart

E. H.

Radley

K. C.

(2017). The impact of vertical axis scaling on the visual analysis of single-case data. Journal of School Psychology, 63, 105–118. https://doi.org/10.1016/j.jsp.2017.03.008

Dart

E. H.

Radley

K. C.

(2018). Toward a standard assembly of linear graphs. School Psychology Quarterly, 33(3), 350–355. https://doi.org/10.1037/spq0000269

Dowdy

Jessel

Saini

Peltier

(2022). Structured visual analysis of single-case experimental design data: Developments and technological advancements. Journal of Applied Behavior Analysis, 55(2), 451–462. https://doi.org/10.1002/jaba.899

10.

Hancock

G. R.

(2003). Fortune cookies, measurement error, and experimental design. Journal of Modern Applied Statistical Methods, 2(2), 293–305. https://doi.org/10.22237/jmasm/1067644980

11.

Hartmann

D. P.

Barrios

B. A.

Wood

D. D.

(2004). Principles of behavioral observation. In Haynes

S. N.

Hieby

E. M.

(Eds.), Comprehensive handbook of psychological assessment, behavioral assessment (Vol. 3, pp. 108–127). John Wiley.

12.

*Hodgetts

Magill-Evans

Misiaszek

J. E.

(2011). Weighted vests, stereotyped behaviors and arousal in children with autism. Journal of Autism and Developmental Disorders, 41(6), 805–814. https://doi.org/10.1007/s10803-010-1104-x

13.

Horner

R. H.

Carr

E. G.

Halle

Mcgee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177/001440290507100203

14.

Huang

F. L.

(2020). MANOVA: A procedure whose time has passed? Gifted Child Quarterly, 64(1), 56–60. https://doi.org/10.1177/0016986219887200

15.

Koller

(2016). Robustlmm: An R package for robust estimation of linear mixed-effects models. Journal of Statistical Software, 75(6), 1–24. https://doi.org/10.18637/jss.v075.i06

16.

Kratochwill

T. R.

Levin

J. R.

(2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15(2), 124–144. https://doi.org/10.1037/a0017736

17.

Kubina

R. M.

Kostewicz

D. E.

Brennan

K. M.

King

S. A.

(2017). A critical review of line graphs in behavior analytic journals. Educational Psychology Review, 29, 583–598. https://doi.org/10.1007/s10648-015-9339-x

18.

Lane

J. D.

Shepley

Spriggs

A. D.

(2019). Issues and improvements in the visual analysis of AB single-case graphs by pre-service professionals. Remedial and Special Education, 42(4), 235–247. https://doi.org/10.1177/0741932519873120

19.

Ledford

J. R.

Barton

E. E.

Severini

K. E.

Zimmerman

K. N.

Pokorski

E. A.

(2019). Visual display of graphic data in single case design studies: Systematic review and expert preference analysis. Education and Training in Autism and Developmental Disabilities, 54(4), 315–327.

20.

Ledford

J. R.

Chazin

K. T.

Lane

J. D.

Zimmerman

K. N.

Ayres

K. A.

(2020, September). Single case analysis and review framework (SCARF). http://ebip.vkcsites.org/scarfv2

21.

Ledford

J. R.

Lane

J. D.

Severini

K. E.

(2018). Systematic use of visual analysis for assessing outcomes in single case design studies. Brain Impairment, 19(1), 4–17. https://doi.org/10.1017/BrImp.2017.16

22.

Manolov

Moeyaert

Fingerhut

J. E.

(2022). A priori justification for effect measures in single-case experimental designs. Perspectives on Behavior Science, 45(1), 153–186. https://doi.org/10.1007/s40614-021-00282-2

23.

*Mitchell

R. R.

Tingstrom

D. H.

Dufrene

B. A.

Ford

W. B.

Sterling

H. E.

(2015). The effects of the good behavior game with general-education high school students. School Psychology Review, 44(2), 191–207. https://doi.org/10.17105/spr-14-0063.1

24.

Ninci

Vannest

K. J.

Willson

Zhang

(2015). Interrater agreement between visual analysts of single-case data: A meta-analysis. Behavior Modification, 39(4), 510–541. https://doi.org/10.1177/0145445515581327

25.

Parsonson

Baer

(1978). The analysis and presentation of graphic data. In Kratochwill

(Ed.), Single subject research (pp. 101–166). Academic.

26.

Peltier

McKenna

J. W.

Sinclair

T. E.

Garwood

Vannest

K. J.

(2022). Brief report: Ordinate scaling and axis proportions of single-case graphs in two prominent EBD journals from 2010 to 2019. Behavioral Disorders, 47(2), 134–148. https://doi.org/10.1177%2F0198742920982587

27.

Peltier

McKenna

(2020). Investigation of two preliminary analysis-altering elements: Ordinate scaling and DPPXYR. https://doi.org/10.17605/OSF.IO/BD8ZQ

28.

Peltier

Morano

Shin

Stevenson

McKenna

J. W.

(2021). A decade review of single-case graph construction in the field of learning disabilities. Learning Disabilities Research & Practice, 36(2), 121–135. https://doi.org/10.1111/ldrp.12245

29.

Peltier

Muharib

Haas

Dowdy

(2022). A decade review of two potential analysis altering variables in graph construction. Journal of Autism and Developmental Disorders, 52, 714–724. https://doi.org/10.1007/s10803-021-04959-0

30.

Peltier

McKenna

van Dijk

(2020). Investigation of two preliminary analysis-altering elements: Ordinate scaling and DPPXYR. https://doi.org/10.17605/OSF.IO/R5U2T

31.

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

32.

Radley

K. C.

Dart

E. H.

Wright

S. J.

(2018). The effect of data points per x-to y-axis ratio on visual analysts evaluation of single-case graphs. School Psychology Quarterly, 33(2), 314–322. https://doi.org/10.1037/spq0000243

33.

*Ramsey

M. L.

Jolivette

Patterson

D. P.

Kennedy

(2010). Using choice to increase time on-task, task-completion, and accuracy for students with emotional/behavior disorders in a residential facility. Education and Treatment of Children, 33(1), 1–21. https://doi.org/10.1353/etc.0.0085

34.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage.

35.

Reichow

Barton

E. E.

Maggin

D. M.

(2018). Development and applications of the single-case design risk of bias tool for evaluating single-case design research study reports. Research in Developmental Disabilities, 79, 53–64. https://doi.org/10.1016/j.ridd.2018.05.008

36.

Roane

H. S.

Fisher

W. W.

Kelley

M. E.

Mevers

J. L.

Bouxsein

K. J.

(2013). Using modified visual-inspection criteria to interpret functional analysis outcomes. Journal of Applied Behavior Analysis, 46(1), 130–146. https://doi.org/10.1002/jaba.13

37.

Saini

Fisher

W. W.

Retzlaff

B. J.

(2018). Predictive validity and efficiency of ongoing visual-inspection criteria for interpreting functional analyses. Journal of Applied Behavior Analysis, 51(2), 303–320. https://doi.org/10.1002/jaba.450

38.

Snijders

R. J.

Bosker

T. A. B.

(2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Sage.

39.

*Stichter

J. P.

Sasso

G. M.

Jolivette

Carr

E. G.

(2004). Structural analysis and intervention in a school setting: Effects on problem behavior for a student with an emotional/behavioral disorder. Journal of Positive Behavior Interventions, 6(3), 166–177. https://doi.org/10.1177%2F10983007040060030401

40.

Swan

D. M.

Pustejovsky

J. E.

Beretvas

S. N.

(2020). The impact of response-guided designs on count outcomes in single-case experimental design baselines. Evidence-Based Communication Assessment and Intervention, 14(1–2), 82–107. https://doi.org/10.1080/17489539.2020.1739048

41.

Tanious

Onghena

(2021). A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis. Behavior Research Methods, 53(4), 1371–1384. https://doi.org/10.3758/s13428-020-01502-4

42.

Tate

R. L.

Perdices

Rosenkoetter

Shadish

Vohra

Barlow

D. H.

Horner

Kazdin

Kratochwill

McDonald

Sampson

Shamseer

Togher

Albin

Backman

Douglas

Evans

J. J.

Gast

Manolov

. . . Wilson

(2016). The single-case reporting guidelines in behavioral interventions (SCRIBE) 2016 statement. Remedial and Special Education, 37(6), 370–380. https://doi.org/10.1177%2F0741932516652893

43.

Taylor

Lanovaz

M. J.

(2022). Agreement between visual inspection and objective analysis methods: A replication and extension. Journal of Applied Behavior Analysis, 55(3), 986–996. https://doi.org/10.1002/jaba.921

44.

Travers

J. C.

Cook

B. G.

Therrien

W. J.

Coyne

M. D.

(2016). Replication research and special education. Remedial and Special Education, 37(4), 195–204. https://doi.org/10.1177%2F0741932516648462

45.

Wolfe

Seaman

M. A.

Drasgow

Sherlock

(2018). An evaluation of the agreement between the conservative dual-criterion method and expert visual analysis. Journal of Applied Behavior Analysis, 51(2), 345–351. https://doi.org/10.1002/jaba.453

46.

Zimmerman

K. N.

Pustejovsky

J. E.

Ledford

J. R.

Barton

E. E.

Severini

K. E.

Lloyd

B. P.

(2018). Single-case synthesis tools II: Comparing quantitative outcome measures. Research in Developmental Disabilities, 79, 65–76. https://doi.org/10.1016/j.ridd.2018.02.001

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.02 MB

1.60 MB

0.01 MB