To make valid decisions about intervention effectiveness in a tiered intervention system, it is essential to formatively assess treatment integrity along with student outcomes. Despite significant advances in technologies for ongoing assessment of student outcomes, research regarding treatment integrity assessment has not shared the same progress in that most available methods lack adequate psychometric evidence and require significant resources. Direct behavior rating has been discussed and evaluated as a tool for assessing student behavior, yet the technology could be extended to result in an efficient treatment integrity assessment method with utility in a multi-tiered model of school-based intervention delivery. This article reviews options in treatment integrity assessment, with emphasis on how direct behavior rating technology might be incorporated within a multi-tiered model of intervention delivery. Implications for both practice and research are addressed.
Burton, S., & Blair, E. ( 1991). Task conditions, response formulation processes, and response accuracy for behavioral frequency questions in surveys. Public Opinion Quarterly, 33, 50-79.
2.
Chafouleas, S.M., Briesch, A.M., Riley-Tillman, T.C., Christ, T.J., & Kilgus, S.P. ( 2008). An investigation of the generalizability and dependability of Direct Behavior Ratings (DBRs) to measure engagement and disruptive behavior of middle school students. Unpublished manuscript.
3.
Chafouleas, S.M., Riley-Tillman, T.C., & Christ, T.J. ( 2009). Direct behavior rating (DBR): An emerging method for assessing social behavior within a tiered intervention system. Assessment for Effective Intervention, 34, 195-200.
4.
Chafouleas, S.M., Riley-Tillman, T.C., & Christ, T.J. ( 2008). Direct behavior ratings (DBRS): A manual for the effective use. Unpublished manual.
5.
Chafouleas, S.M., Christ, T.J., Riley-Tillman, T.C., Briesch, A.M., & Chanese, J.M. ( 2007). Generalizability and dependability of daily behavior report cards to measure social behavior of preschoolers. School Psychology Review, 36, 63-79.
6.
Chafouleas, S.M., Riley-Tillman, T.C., & Sugai, G. ( 2007). School-based behavioral assessment: Informing instruction and intervention. New York: Guilford .
7.
Christ, T.J., Chafouleas, S.M., & Riley-Tillman, T.C. (2008). Direct behavior rating (DBR): Alternate coding and interpretation of teacher ratings on a single item graphic rating scale. Unpublished manuscript.
8.
Christ, T.J., Riley-Tillman, T.C., & Chafouleas, S.M. (2009). Foundation for the development and use of direct behavior rating (DBR) to assess and evaluate child behavior. Assessment for Effective Intervention, 34, 201-213.
Dane, A.V., & Schneider, B.H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control ? Clinical Psychology Review, 18, 23-45.
11.
DiMatteo, M.R. ( 2004). Variations in patients’ adherence to medical recommendations: A quantitative review of 50 years of research. Medical Care , 42, 200-209.
12.
Gresham, F.M. ( 1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 37-50.
13.
Hagermoser Sanetti, L.M., & Kratochwill, T.R. (in press). Treatment integrity assessment in the schools: An evaluation of the treatment integrity planning protocol (TIPP) . School Psychology Quarterly.
14.
Hirschstein, M.K., Edstrom, L.V., Frey, K.S., Snell, J.L., & MacKenzie, E.P. (2007). Walking the talk in bully prevention: Teacher implementation variable related to initial impact of the Steps to Respect program. School Psychology Review, 36, 3-21.
15.
Jimerson, S.R., Burns, M.K., & VanDerHeyden, A.M. (2007). Handbook of response to intervention: The science and practice of assessment and intervention. New York: Springer.
16.
Jones, H.A., Clarke, A.T., & Power, T.J. ( 2008). Expanding the concept of intervention integrity: A multidimensional model of participant engagement. In Balance, 23, 4-5.
17.
Lane, K.L., Bocian, K.M., MacMillan, D.L., & Gresham, F.M. ( 2004). Treatment integrity: An essential-but often forgotten-component of school-based interventions. Preventing School Failure, 48, 36-43.
National Association of State Directors of Special Education. (2008). Response to intervention: Blueprints for implementation. Alexandra, VA: Author.
20.
Noell, G.H. ( 2007). Research examining the relationships among consultation process, treatment integrity, and outcomes. In W. P. Erchul & S. M. Sheridan (Eds.), Handbook of research in school consultation: Empirical foundations for the field (pp. 315-334). Mahwah, NJ: Lawrence Erlbaum.
21.
Noell, G.H., & Gansle, K.A. ( 2006). Assuring the form has substance: Treatment plan implementation as the foundation of assessing response to intervention. Assessment for Effective Intervention, 32, 32-39.
22.
Noell, G.H., Gresham, F.M., & Gansle, K.A. ( 2002). Does treatment integrity matter? A preliminary investigation of instructional implementation and mathematics performance. Journal of Behavioral Education, 11, 51-67.
23.
Noell, G.H., Witt, J.C., Slider, N.J., Connell, J.E., Gatti, S.L., Williams, K.L., et al. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34, 87-106.
24.
Perepletchikova, F., & Kazdin, A.E. ( 2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365-383.
25.
Peterson, L., Homer, A., & Wonderlich, S. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477-492.
26.
Power, T.J., Blom-Hoffman, J., Clarke, A.T., Riley-Tillman, T.C., Kellerher, C., & Manz, P. ( 2005). Reconceptualizing intervention integrity: A partnership-based framework for linking research with practice. Psychology in the Schools, 42, 495-507.
27.
Quittner, A.L., Espelage, D.L., Ievers-Landis, C., & Drotar, D. ( 2000). Measuring adherence to medical treatments in childhood chronic illness: Considering multiple methods and sources of information. Journal of Clinical Psychology in Medical Settings, 7, 41-54.
28.
Rand, C.S., & Wise, R.A. ( 1994). Measuring adherence to asthma medication regimens. American Journal of Respiratory and Critical Care Medicine, 149, S69-S76.
29.
Riekert, K.A. ( 2006). Integrating regimen adherence assessment into clinical practice. In W. T. O’Donohue & E. R. Levensky (Eds.), Promoting treatment adherence: A practical handbook for health care providers (pp.17-34). Thousand Oaks, CA: Sage.
30.
Riley-Tillman, T.C., Chafouleas, S.M., Christ, T.J., Briesch, A.M., & LeBel, T.J. (in press). The impact of wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs). School Psychology Quarterly.
31.
Roach, A.T., & Elliott, S.N. ( 2008). Best practices in facilitating and evaluating intervention integrity. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (Vol. 2., 5th ed., pp. 195-208). Bethesda, MD: National Association of School Psychologists.
32.
Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
33.
Steele, D.J., Jackson, T.C., & Gutmann, M.C. ( 1990). Have you been taking your pills? The adherence-monitoring sequence in the medical interview. Journal of Family Practice , 30, 294-299.
34.
Stone, A.A., Turkkan, J.S., Bachrach, C.A., Jobe, J.B., Kurtzman, H.S., & Cain, V.S. ( 2000). The science of self-report: Implications for research and practice. Mahwah, NJ: Lawrence Erlbaum.
35.
Vitolins, M.Z., Rand, C.S., Rapp, S.R., Ribisl, P.M., & Sevick, M.A. ( 2000). Measuring adherence to behavioral and medical interventions . Controlled Clinical Trials, 21, 188S-194S.
36.
Waltz, J., Addis, M.E., Koerner, K., & Jacobson, N.S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620-630.
37.
Wickstrom, K.F., Jones, K.M., LaFleur, L.H., & Witt, J.C. ( 1998). An analysis of treatment integrity in school-based behavioral consultation. School Psychology Quarterly, 13, 141-154.
38.
Yeaton, W.H., & Sechrest, L. ( 1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156-167.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
