Development of a Brief Measure for Multi-Tiered Systems of Support Sustainability

Abstract

This study focused on the development of the Multi-Tiered Systems of Support–Sustainability Scale (MTSS-SS). Review of the literature identified factors associated with sustainability for multi-tiered systems of support (MTSS), and it indicated few sustainability measures currently exist for practitioners and researchers to incorporate into MTSS program evaluation. This study endeavored to create a brief measure of MTSS sustainability that could be used for evaluation in research and practice by school interventionists. Study participants included a national sample of 598 school counselors and school psychologists who worked as interventionists in schools using MTSS. This group completed the 10-item MTSS-SS. Outcomes from content validity ratings, principal axis factoring, internal consistency reliability analyses, and construct validity with known groups indicated the MTSS-SS possessed initial evidence of psychometric adequacy when used by interventionists from the disciplines of school counseling or school psychology. Discussion focuses on the use of the MTSS-SS, need for further development, and study limitations.

Keywords

multi-tiered systems of support sustainability interventionists

Get full access to this article

View all access options for this article.

References

American Educational Research Association. (2014). Standards for educational and psychological tests. AERA.

American Institutes for Research. (2021). Center on multi-tiered system of support. https://mtss4success.org/

Arrimada

Torrance

Fidalgo

(2021). Response to intervention in first-grade writing instruction: A large-scale feasibility study. Reading and Writing: An Interdisciplinary Journal. https://doi.org/10.1007/s11145-021-10211-z

Bastable

Massar

M. M.

McIntosh

(2019). A survey of team members’ perceptions of coaching activities related to tier 1 SWPBIS implementation. Journal of Positive Behavior Interventions, 22, 51–61. https://doi.org/10.1177%2F1098300719861566

Burns

M. K.

Jimerson

S. R.

VanDerHeyden

A. M.

Deno

S. L.

(2016). Toward a unified response-to-intervention model: Multi-tiered systems of support. In Jimerson

S. R.

Burns

M. K.

VanDerHeyden

A. M.

(Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (2nd ed., pp. 719–732). Springer. https://doi.org/10.1007/978-1-4899-7568-3_41

Castillo

J. M.

Dedrick

R. F.

Stockslager

K. M.

March

A. L.

Hines

C. V.

Tan

S. Y.

(2015). Development and initial validation of a scale measuring the beliefs of educators regarding response to intervention. Journal of Applied School Psychology, 31, 1–30. https://doi.org/10.1080/15377903.2014.938282

Castillo

J. M.

March

A. L.

Stockslager

K. M.

Hines

C. V.

(2016). Measuring educators’ perceptions of their skills relative to response to intervention: A psychometric study of a survey tool. Assessment for Effective Intervention, 4, 94–108. https://doi.org/10.1177%2F1534508415616583

Chaparro

E. A.

Smolkowski

Jackson

K. R.

(2020). Scaling up and integrating Effective Behavioral and Instructional Support Systems (EBISS): A study of one state’s professional development efforts. Learning Disability Quarterly, 43, 4–17. https://doi.org/10.1177/0731948719851752

Charlton

C. T.

Sabey

C. V.

Dawson

M. R.

Pyle

Lund

E. M.

Ross

S. W.

(2018). Critical incidents in the scale-up of state multitiered systems of supports. Journal of Positive Behavior Interventions, 20, 191–202. https://doi.org/10.1177/1098300718770804

10.

Clemens

N. H.

Keller-Margulis

M. A.

Scholten

Yoon

(2016). Screening assessment within a multi-tiered system of support: Current practices, advances, and next steps. In Jimerson

S. R.

Burns

M. K.

VanDerHeyden

A. M.

(Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (2nd ed., pp. 187–213). Springer. https://doi.org/10.1007/978-1-4899-7568-3_12

11.

Cohen

R. J.

Schneider

W. J.

Tobin

(2022). Psychological testing and assessment (10th ed.). McGraw Hill.

12.

Colorado Department of Education. (2021). Multi-tiered systems of support: Guidance article. https://www.cde.state.co.us/mtss/whatismtssarticle

13.

Cureton

E. E.

D’Agostino

R. B.

(1993). Factor analysis: An applied approach. Psychology Press. https://doi.org/10.4324/9781315799476

14.

Dallas

W. P.

(2017). Systemic sustainability in RtI using intervention-based scheduling methodologies. Learning Disability Quarterly, 40, 105–113. https://doi.org/10.1177/0731948717690141

15.

Dart

E. H.

Arora

P. G.

Collins

T. A.

Doll

(2019). Progress monitoring measures for internalizing symptoms: A systematic review of the peer-reviewed literature. School Mental Health: A Multidisciplinary Research and Practice Journal, 11, 265–275. https://doi.org/10.1007/s12310-018-9299-7

16.

Ennis

R. P.

Flemming

S. C.

Michael

Lee

E. O.

(2020). Using a tiered approach to support early childhood educators’ use of behavioral strategies. Education & Treatment of Children, 43, 265–277. https://doi.org/10.1007/s43494-020-00027-x

17.

Every Student Succeeds Act, 20 U.S.C. § 6301. (2015). https://www.congress.gov/bill/114th-congress/senate-bill/1177

18.

Fallon

L. M.

Mueller

M. R.

Kurtz

K. D.

Cathcart

S. C.

(2019). Tier 1 basics. In Radley

K. C.

Dart

E. H.

(Eds.), Handbook of behavioral interventions in schools: Multi-tiered systems of support (pp. 180–120). Oxford University Press. https://doi.org/10.1093/med-psych/9780190843229.003.0010

19.

Fixsen

D. L.

Naoom

S. F.

Blasé

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature National Implementation Research Network. https://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf

20.

Fletcher

J. M.

Vaughn

(2009). Response to intervention: Preventing and remediating academic difficulties. Child Development Perspectives, 3, 30–37. https://doi.org/10.1111/j.1750-8606.2008.00072.x

21.

Gibbons

Coulter

W. A.

(2016). Making response to intervention stick: Sustaining implementation past your retirement. In Jimerson

S. R.

Burns

M. K.

VanDerHeyden

A. M.

(Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (2nd ed., pp. 641–660). Springer. https://doi.org/10.1007/978-1-4899-7568-3_36

22.

Gonzales

Tejero Hughes

(2021). Leveraging a Spanish literacy intervention to support outcomes of English learners. Reading Psychology, 42, 411–434. https://doi.org/10.1080/02702711.2021.1888352

23.

Hagermoser Sanetti

L. M.

Luh

H. J

. (2019). Fidelity of implementation in the field of learning disabilities. Learning Disability Quarterly, 42, 204–216. https://doi.org/10.1177/0731948719851514

24.

Han

S. S.

Weiss

(2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665–679. https://link.springer.com/article/10.1007/s10802-005-7646-2

25.

Hudson

T. M.

McKenzie

R. G.

(2016). The impact of RTI on timely identification of students with specific learning disabilities. Learning Disabilities: A Multidisciplinary Journal, 21, 46–58. https://doi.org/10.18666/LDMJ-2016-V21-I2-7722

26.

Hughes

Dexter

D. D.

(2021). Field studies of RTI programs, revised. RTI Action Network, National Center for Learning Disabilities. http://www.rtinetwork.org/learn/research/field-studies-rti-programs

27.

King

K. R.

Lembke

E. S.

Reinke

W. M.

(2016). Using latent class analysis to identify academic and behavioral risk status in elementary students. School Psychology Quarterly, 31, 43–57. https://doi.org/10.1037/spq0000111

28.

Lawshe

C. H.

(1975). A quantitative approach to content validity. Personnel Psychology, 28, 563–575.

29.

Mason

E. N.

Benz

Lembke

E. S.

Burns

M. K.

Powell

S. R.

(2019). From professional development to implementation: A district's experience implementing mathematics tiered systems of support. Learning Disabilities Research and Practice, 34, 207–214. https://doi.org/10.1111/ldrp.12206

30.

Mathews

McIntosh

Frank

J. L.

May

S. L

. (2014). Critical features predicting sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 16, 168–178. https://doi.org/10.1177%2F1098300713484065

31.

McIntosh

MacKay

L. D.

Hume1

A. E.

Doolittle

Vincent

C. G.

Horner

R. H.

Ervin

R. A.

(2011). Development and initial validation of a measure to assess factors related to sustainability of school-wide positive behavior support. Journal of Positive Behavior Interventions, 13, 208–218. https://doi.org/10.1177%2F1098300710385348

32.

McIntosh

Mercer

S. H.

Nese

R. N. T.

Strickland-Cohen

M. K.

Hoselton

(2016). Predictors of sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 18, 209–218. https://doi.org/10.1177/1098300715599737

33.

McIntosh

Predy

L. K.

Upreti

Hume

A. E.

Turri

M. G.

Mathews

(2014). Perceptions of contextual features related to implementation and sustainability of school-wide positive behavior support. Journal of Positive Behavior Interventions, 16, 31–43. https://doi.org/10.1177/1098300712470723

34.

Mercer

S. H.

McIntosh

Strickland-Cohen

M. K.

Horner

R. H.

(2014). Measurement invariance of an instrument assessing sustainability of school-based universal behavior practices. School Psychology Quarterly, 29, 125–137. https://doi.org/10.1037/spq0000054

35.

Miller

Freeman

(2016). Challenges faced by new implementation sites: The role of culture in the change process. In Jimerson

S. R.

Burns

M. K.

VanDerHeyden

A. M.

(Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (2nd ed., pp. 627–640). Springer. https://doi.org/10.1007/978-1-4899-7568-3_35

36.

Morrison

J. Q.

Newman

D. S.

Erickson

A. G.

(2020). Process evaluation of literacy practices within a multi-tiered system of supports framework. Journal of Applied School Psychology, 37, 140–164. https://doi.org/10.1080/15377903.2020.1804030

37.

Neitzel

A. J.

Lake

Pellegrini

Slavin

R. E.

(2021). A synthesis of quantitative research on programs for struggling readers in elementary schools. Reading Research Quarterly. https://doi.org/10.1002/rrq.379

38.

Rinaldi

Averill

O. H.

Stuart

(2010–2011). Educators’ perceptions of a three-year RTI collaborative reform effort in an urban elementary school. Journal of Education, 191, 43–53. http://www.bu.edu/journalofeducation/

39.

Rodgers

(2016). Scaling and sustaining an intervention: The case of reading recovery. Journal of Education for Students Placed at Risk, 21, 10–28. https://doi.org/10.1080/10824669.2015.1110026

40.

Sailor

(2015). Advances in schoolwide inclusive school reform. Remedial and Special Education, 36, 94–99. https://doi.org/10.1177%2F0741932514555021

41.

Sailor

Skrtic

T. M.

Cohn

Olmstead

(2021). Preparing teacher educators for statewide scale-up of multi-tiered system of support. Teacher Education and Special Education, 44, 24–41. https://doi.org/10.1177/0888406420938035

42.

Thomas

E. R.

Conoyer

S. J.

Lembke

E. S.

(2020). Districtwide evaluation of RTI implementation: Success, challenges, and self-efficacy. Learning Disabilities Research and Practice, 35, 118–125. https://doi.org/10.1111/ldrp.12226

43.

Turri

M. G.

Mercer

S. H.

McIntosh

Nese

R. N. T.

Strickland-Cohen

M. K.

Hoselton

(2016). Examining barriers to sustained implementation of school-wide prevention practices. Assessment for Effective Intervention, 42, 6–17. https://doi.org/10.1177%2F1534508416634624

44.

Van Norman

E. R.

Nelson

P. M.

Klingbeil

D. A

. (2020). Profiles of reading performance after exiting tier 2 intervention. Psychology in the Schools, 57, 757–767. https://doi.org/10.1002/pits.22354

45.

Weisenburgh-Snyder

A. B.

Malmquist

S. K.

Robbins

J. K.

Lipshin

A. M.

(2015). A model of MTSS: Integrating precision teaching of mathematics and a multi-level assessment system in a generative classroom. Learning Disabilities: A Contemporary Journal, 13, 21–41. https://eric.ed.gov/?id=EJ1080598