Measuring Implementation of Multi-Tiered Systems of Support (MTSS): Development and Validation of the Integrated MTSS Fidelity Rubric (IMFR)

Abstract

The Integrated MTSS Fidelity Rubric (IMFR) is a 14-item measure of elementary school implementation of integrated multi-tiered systems of support (I-MTSS). With I-MTSS, schools strategically use assessment to guide intervention that addresses the integrated nature of students’ academic, social, emotional, and behavioral needs. The goal for this study was to develop and validate the IMFR as a measurement tool that is useful and relevant for school practitioners and researchers who deliver and study intervention for students in elementary schools. The IMFR was validated through 3 years of iterative administration, psychometric testing, and refinement, using a nationwide U.S. sample that ranged from 65 to 87 elementary schools across 13–20 districts in a given administration year. Analyses examined content validity, substantive validity, structural validity, and generalizability. In addition, the study examined usability through focus groups with participating school teams. Overall, we conclude that the IMFR is a reliable and valid measure of I-MTSS implementation and is useful to school and district practitioners.

Keywords

interventions progress monitoring screening/benchmarking

Get full access to this article

View all access options for this article.

References

AIR. (2015). DBI implementation rubric and interview. https://intensiveintervention.org/resource/dbi-implementation-rubric-and-interview

AIR. (2025). Multi-Tiered System of Supports (MTSS) fidelity of implementation rubric (version 3). https://mtss4success.org/sites/default/files/2025-01/mtss-fidelity-rubric-2025.pdf

Algozzine

Barrett

Eber

George

Horner

Lewis

Putnam

Swain-Bradway

McIntosh

Sugai

(2019). School-wide PBIS Tiered Fidelity Inventory. OSEP Center on Positive Behavioral Interventions and Supports.

Algozzine

Wang

White

Cooke

Marr

M. B.

Algozzine

Helf

S. S.

Duran

G. Z.

(2012). Effects of multi-tier academic and behavior instruction on difficult-to-teach students. Exceptional Children, 79(1), 45–64. https://doi.org/10.1177/001440291207900103

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.

Anderson

Childs

Kincaid

Horner

R. H.

George

Todd Spaulding

(2009). Benchmarks for advanced tiers (BAT). Educational and Community Supports, University of Oregon & University of South Florida.

Anderson

C. M.

Lewis-Palmer

Todd

A. W.

Horner

R. H.

Sugai

Sampson

N. K.

(2012). Individual Student Systems Evaluation Tool (ISSET). Educational and Community Supports, University of Oregon.

Arden

S. V.

Gandhi

A. G.

Zumeta Edmonds

Danielson

(2017). Toward more effective tiered systems: Lessons from national implementation efforts. Exceptional Children, 83(3), 269–280. https://doi.org/10.1177/0014402917693565

Baker

S. K.

Fien

Baker

D. L.

(2010). Robust reading instruction in the early grades: Conceptual and practical issues in the integration and evaluation of Tier 1 and Tier 2 instructional supports. Focus on Exceptional Children, 42(9), 1. https://doi.org/10.17161/foec.v42i9.6693

10.

Baumgartner

Hombur

(1996). Applications of structural equation modeling in marketing and consumer research: A review. International Journal of Research in Marketing, 13, 139–161. https://doi.org/10.1016/0167-8116(95)00038-0

11.

Berkeley

Scanlon

Bailey

T. R.

Sutton

J. C.

Sacco

D. M.

(2020). A snapshot of RTI implementation a decade later: New picture, same story. Journal of Learning Disabilities, 53(5), 332–342. https://doi.org/10.1177/0022219420915867

12.

Burns

M. K.

Jimerson

S. R.

VanDerHeyden

A. M.

Deno

S. L.

(2015). Toward a unified response-to-intervention model: Multi-tiered systems of support. In Jimerson

S. R.

Burns

M. K.

VanDerHeyden

A. M.

(Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (pp. 719–732). Springer.

13.

Castillo

J. M.

Batsche

G. M.

Curtis

M. J.

Stockslager

March

Minch

Hines

(2012). Problem solving/response to intervention evaluation tool technical assistance manual revised. Educational and Psychological Studies Faculty Publications. https://digitalcommons.usf.edu/esf_facpub/39

14.

Center on Multi-Tiered Systems of Support. (2025). MTSS Fidelity of Implementation Rubric, updated. American Institutes for Research.

15.

Center on PBIS. (2025, February). Tiered Fidelity Inventory (TFI) manual (Version 30). University of Oregon.

16.

Chafouleas

S. M.

Miller

F. G.

Briesch

A. M.

Neugebauer

S. R.

Riley-Tillman

T. C.

(2012). Usage rating profile: Assessment. University of Connecticut.

17.

Childs

K. E.

Kincaid

George

H. P.

(2011). The revised school-wide PBS Benchmarks of Quality (BoQ). OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports.

18.

Cohen

(1968). Weighted Kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220. https://doi.org/10.1037/h0026256

19.

Condliffe

Zhu

Doolittle

van Dok

Power

Denison

Kurki

(2022). Study of training in multi-tiered systems of support for behavior: Impacts on elementary school students’ outcomes (Appendix NCEE 2022-008a). National Center for Education Evaluation and Regional Assistance.

20.

Fixsen

D. L.

Naoom

S. F.

Blase

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature (FMHI Publication #231). University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network. https://nirn.fpg.unc.edu/resource/implementation-research-a-synthesis-of-the-literature/

21.

Fuchs

L. S.

(2006). Introduction to response to intervention: What, why, and how valid is it? Reading Research Quarterly, 41(1), 93–99. https://doi.org/10.1598/RRQ.41.1.4

22.

Gandhi

Lembke

Riley-Tillman

T. C.

Pierce

Smith

Casasanto-Ferro

Majeika

C. E.

(2024). Integrated MTSS fidelity rubric (IMFR): Materials packet. American Institutes for Research.

23.

Horner

R. H.

Sampson

N. K.

Anderson

C. M.

Todd

A. W.

Eliason

B. M.

(2012). Monitoring advanced tiers tool (MATT). Educational and Community Supports, University of Oregon.

24.

Horner

R. H.

Todd

A. W.

Lewis-Palmer

Irvin

L. K.

Sugai

Boland

J. B.

(2004). The school-wide evaluation tool (SET): A research instrument for assessing schoolwide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12. https://doi.org/10.1207/S15327035EX0803_6

25.

L.-t.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

26.

Lane

K. L.

Oakes

W. P.

Buckman

M. M.

Menzies

H. M.

(2022). Comprehensive, integrated, three-tiered (Ci3T) models of prevention: Prioritizing integrated systems. In Lemons

C. J.

Powell

S. R.

Lane

K. L.

Aceves

T. C.

(Eds.), Handbook of special education research (Vol. 2, pp. 4-18). Routledge.

27.

Linacre

J. M.

(1989). Many facet Rasch measurement. MESA Press.

28.

McIntosh

Goodman

(2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. Guilford Press.

29.

Messick

(1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741

30.

National Center on Intensive Intervention. (2015). Data-based individualization: A rubric for evaluating DBI implementation. U.S. Office of Special Education Programs.

31.

O’Donnell

C. L.

(2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78(1), 33–84. https://doi.org/10.3102/0034654307313793

32.

Pierce

Arnold

Majeika

C. E.

Gandhi

Lembke

(2025). An evolution of multi-tiered systems of support (MTSS): Integrated MTSS (in review).

33.

Pierce

Thomas

Majeika

Casasanto-Ferro

Gandhi

Lembke

(2025). Developing and testing the integrated multi-tiered systems of support fidelity rubric. Global Implementation Research and Applications, 5, 556–567. https://doi.org/10.1007/s43477-025-00191-5

34.

Powell

B. J.

Stanick

C. F.

Halko

H. M.

Dorsey

C. N.

Weiner

B. J.

Barwick

M. A.

Damschroder

L. J.

Wensing

Wolfenden

Lewis

C. C.

(2017). Toward criteria for pragmatic measurement in implementation research and practice: A stakeholder-driven approach using concept mapping. Implementation Science, 12(1), 1–7. https://doi.org/10.1186/s13012-017-0649-x

35.

Simonsen

Coyne

Goodman

(2024). Implementing practices within an integrated multi-tiered system of support: Introduction to the special series. Intervention in School and Clinic, 60(1), 3–5. https://doi.org/10.1177/10534512241257435

36.

St Martin

Harms

Walsh

Nantais

(2023). Reading Tiered Fidelity Inventory elementary-level edition (Version 2.2). Michigan Department of Education, Michigan’s Multi-Tiered System of Supports Technical Assistance Center.

37.

Stockslager

Castillo

Brundage

Childs

Romer

(2016). Self-assessment of MTSS (SAM) technical assistance manual. Florida’s Problem Solving/Response to Intervention Project and Florida’s Positive Behavior Intervention and Support Project, University of South Florida.

38.

Sugai

Lewis-Palmer

Todd

Horner

R.H

. (2001). School-wide evaluation tool. University of Oregon.

39.

Sugai

Horner

R. H.

(2002). Introduction to the special series on positive behavior support in schools. Journal of Emotional and Behavioral Disorders, 10(3), 130–135. https://doi.org/10.1177/10634266020100030101

40.

Sugai

Horner

R. H.

(2009). Responsiveness-to-intervention and school-wide positive behavior supports: Integration of multi-tiered system approaches. Exceptionality, 17(4), 223–237. https://doi.org/10.1080/09362830903235375

41.

Thompson

(2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association. https://doi.org/10.1037/10694-000.

42.

Wolfe

Smith

Jr. (2007). Understanding Rasch measurement: Instrument development tools and activities for measure validation using Rasch models: Part II—Validation activities. Journal of Applied Measurement, 8(2), 204–234.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB

0.75 MB