Direct behavior ratings (DBR) refer to a family of assessments that educators use to measure behavioral change. To facilitate interpretations of changes in student behavior, it is beneficial to measure rates of change over pre-determined spans of time. This pilot study estimated the score change variability and the change sensitivity of the DBR-Single Item Scales (DBR-SIS) for academic engagement and disruptive behavior in typical U.S. educational settings. A series of multivariate multilevel models (MMLMs) were fitted to an extant dataset of progress-monitoring observations for students identified with academic behavior and social behavior risk in grades 1 through 5. Rates of change in both academic engagement and disruptive behavior exhibited considerable between-student variability after 3–4 weeks of progress monitoring. Academic engagement was slightly more sensitive to capturing between-student differences in rates of change. Considerations for selecting certain single-item scales are discussed in addition to the utility of interpreting score change for decision-making purposes.
ChafouleasS. M.Riley-TillmanT. C.ChristT. J. (2009). Direct behavior rating (DBR): An emerging method for assessing social behavior within a tiered intervention system. Assessment for Effective Intervention, 34(4), 195–200. https://doi.org/10.1177/1534508409340391
4.
ChristT. J.Riley-TillmanT. C.ChafouleasS. M. (2009). Foundation for the development and use of direct behavior rating (DBR) to assess and evaluate student behavior. Assessment for Effective Intervention, 34(4), 201–213. https://doi.org/10.1177/1534508409340390
5.
DemarayM. K.ElliottS. N. (2001). Perceived social support by children with characteristics of attention-deficit/hyperactivity disorder. School Psychology Quarterly, 16(1), 68–90. https://doi.org/10.1521/scpq.16.1.68.19156
6.
DiPernaJ. C.ElliottS. N. (2002). Promoting academic enablers to improve student achievement: An introduction to the mini-series. School Psychology Review, 31(3), 293–297. https://doi.org/10.1080/02796015.2002.12086156
7.
FoxR. A.LeifE. S.MooreD. W.FurlongerB.AndersonA.SharmaU. (2021). A systematic review of the facilitators and barriers to the sustained implementation of school-wide positive behavioral interventions and supports. Education & Treatment of Children, 45(1), 105–126. https://doi.org/10.1007/s43494-021-00056-0
8.
FuchsD.FuchsL. S. (2006). Introduction to response to intervention: What, why, and how valid is it?Reading Research Quarterly, 41(1), 93–99. https://doi.org/10.1598/RRQ.41.1.4
9.
GastD. L. (2014). General factors in measurement and evaluation. In GastD. L.LedfordJ. R. (Eds.), Single case research methodology: Applications in special education and behavioral science (pp. 85–104). Routledge.
JohnsonA. H.MillerF. G.ChafouleasS. M.WelshM. E.Riley-TillmanT. C.FabianoG. A. (2016). Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening: A multisite investigation. Journal of School Psychology, 54, 39–57. https://doi.org/10.1016/j.jsp.2015.10.001
15.
JonesV.JonesL. (2007). Comprehensive classroom management. Creating communities of support and solving problems (8th ed.). Pearson.
16.
KazdinA. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). Oxford University Press.
17.
KilgusS. P.ChafouleasS. M.Riley-TillmanT. C. (2013). Development and initial validation of the social and academic behavior risk screener for elementary Grades. School Psychology Quarterly, 28(3), 210–226. https://doi.org/10.1037/spq0000024
18.
KilgusS. P.TaylorC. N.von der EmbseN. P. (2018). Screening for behavioral risk: Identification of high risk cut scores within the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS). School Psychology Quarterly, 33(1), 155–159. https://doi.org/10.1037/spq0000230
19.
KilgusS. P.von der EmbseN. P. (2015). Social, academic, and emotional behavior risk screener (SAEBRS). Theodore J. Christ & Colleagues.
20.
LaneK. L.SmitherR. L.HusemanR.GuffeyJ.FoxJ. G. (2007). A function-based intervention to decrease disruptive behavior and increase academic engagement. The Journal of Early and Intensive Behavioral Intervention, 4(1), 348–364. https://doi.org/10.1037/h0100348
21.
LeidigT.CasaleG.WilbertJ.HennemannT.VolpeR. J.BrieschA. M.GroscheM. (2022). Individual, generalized, and moderated effects of the good behavior game on at-risk primary school students: A multilevel multiple baseline study using behavioral progress monitoring. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.917138
22.
LongJ. D. (2011). Longitudinal data analysis for the behavioral sciences using R. Sage.
23.
MacCallumR. C.KimC.MalarkeyW. B.Kiecolt-GlaserJ. K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32, 215–253. https://doi.org/10.1207/s15327906mbr3203_1
24.
MattaM.VolpeR. J.BrieschA. M.OwensJ. S. (2020). Five direct behavior rating multi-item scales: Sensitivity to the effects of classroom interventions. Journal of School Psychology, 81, 28–46. https://doi.org/10.1016/j.jsp.2020.05.002
25.
MillerF. G.CrovelloN. J.ChafouleasS. M. (2017). Progress monitoring the effects of daily report cards across elementary and secondary settings using direct behavior rating—single item scales. Assessment for Effective Intervention, 43(1), 34–47. https://doi.org/10.1177/1534508417691019
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage.
29.
R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
30.
ReisenerC. D.DufreneB. A.ClarkC. R.OlmiD. J.TingstromD. H. (2016). Selecting effective interventions to increase math computation fluency via brief experimental analyses. Psychology in the Schools, 53(1), 39–57. https://doi.org/10.1002/pits.21887
31.
Riley-TillmanT. C.ChafouleasS. M.SassuK. A.ChaneseJ. A. M.GlazerA. D. (2008). Examining the agreement of direct behavior ratings and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions, 10(2), 136–143. https://doi.org/10.1177/1098300707312542
32.
ThornbladS. C.ChristT. J. (2014). Curriculum-based measurement of reading: Is 6 weeks of daily progress monitoring enough?School Psychology Review, 43(1), 19–29. https://doi.org/10.1080/02796015.2014.12087451
33.
ThumY. M. (1997). Hierarchical linear models for multivariate outcomes. Journal of Educational and Behavioral Statistics, 22, 77–108. https://doi.org/10.3102/10769986022001077
University of Oregon, Center on Teaching and Learning. (2018). Understanding the research behind DIBELS® 8th Edition (Technical Report 1801).
36.
Van NormanE. R. (2021). The effect of nonlinear growth on the accuracy of curriculum-based measurement of reading decision rules. School Psychology, 36(1), 60–68. https://doi.org/10.1037/spq0000416
37.
Van NormanE. R.NelsonP. M.ParkerD. C. (2018). A comparison of nonsense-word fluency and curriculum-based measurement of reading to measure response to phonics instruction. School Psychology Quarterly, 33(4), 573–581. https://doi.org/10.1037/spq0000237
38.
VolpeR. J.BrieschA. M. (2012). Generalizability and dependability of single-item and multiple-item direct behavior rating scales for engagement and disruptive behavior. School Psychology Review, 41(3), 246–261. https://doi.org/10.1080/02796015.2012.12087506
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
