Development and Initial Validation of Scores on the School Counselor/MTSS Alignment Scale

Abstract

Multi-tiered Systems of Supports (MTSS) is a framework for organizing and providing school-based support in various areas such as academics, behavior, social and emotional development, and mental health. A foundational aspect of MTSS is evaluation, including the fidelity of alignment. For over a decade, scholars have proposed frameworks for school counselor (SC)/MTSS alignment. However, the extant literature is missing an assessment with valid scores for measuring SC/MTSS alignment. Accordingly, the aim of the present study was to develop and validate scores on the School Counselor/MTSS Alignment Scale (SC/MTSS-AS), a newly developed assessment for measuring SC/MTSS alignment among three independent U.S. national samples of SCs. The three following aims of this study included: (a) testing the dimensionality of scores on the SC/MTSS-AS, (b) confirming the emergent factor structure with a second independent sample of SCs, and (c) testing the convergent validity of scores on the SC/MTSS-AS. Results of internal structure validity testing revealed and confirmed a five-dimensional MTSS alignment model, including a higher-order latent factor of total MTSS alignment. Convergent validity testing also supported the psychometric properties of the SC/MTSS-AS.

Keywords

School Counselor/MTSS Alignment Scale internal structure validity convergent validity

Get full access to this article

View all access options for this article.

References

Alvarez

J. M.

Saunders

Neubauer

Brown

C. H.

(2022). School counselors implementing a trauma-informed approach through evidence-based practices. Professional School Counseling, 26(1a), 1–8. https://doi.org/10.1177/2156759X221086742

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/uploads/7/6/6/4/76643089/standards_2014edition.pdf

American School Counselor Association. (2019). ASCA national model: A framework for school counseling programs (4th ed.).

American School Counselor Association. (2021a). ASCA research report: State of the profession 2020. https://www.schoolcounselor.org/getmedia/bb23299b-678d-4bce-8863-cfcb55f7df87/2020-State-of-the-Profession.pdf

American School Counselor Association. (2021b). The school counselor and multitiered system of support. https://www.schoolcounselor.org/Standards-Positions/Position-Statements/ASCA-Position-Statements/The-School-Counselor-and-Multitiered-System-of-Sup

Betters-Bubon

Donohue

(2016). Professional capacity building for school counselors through School-Wide Positive Behavior Interventions and Supports Implementation. Journal of School Counseling, 14(3), 1–35.

Clemens

E. V.

Carey

J. C.

Harrington

K. M.

(2010). The school counseling program implementation survey: Initial instrument development and exploratory factor analysis. Professional School Counseling, 14(2), 125–134. https://doi.org/10.1177/2156759X1001400201

Clemons

K. L.

(2024). It takes a village: A conceptual model for Black American community partnerships. Professional School Counseling, 28(1), 1–11. https://doi.org/10.1177/2156759X231225222

Cook

R. M.

(2021). Addressing missing data in quantitative counseling research. Counseling Outcome Research and Evaluation, 12(1), 43–53. https://doi.org/10.1080/21501378.2019.1711037

10.

Credé

Harms

(2015). 25 years of higher-order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations. Journal of Organizational Behavior, 36(6), 845–872. https://doi.org/10.1002/job.2008

11.

Dimitrov

(2012). Statistical methods for validation of assessment scale data in counseling and related fields. American Counseling Association.

12.

Edirmanasinghe

Attia

Brant-Rajahn

Staton

A. R.

(2022). Working with immigrant children in schools: Applying a multi-tiered approach. Journal of Child and Adolescent Counseling, 8(2), 59–71. https://doi.org/10.1080/23727810.2022.2074771

13.

Edwin

Bahr

M. W.

(2021). Development and exploratory factor analysis of the Interventionist Multitiered Systems of Support Multicultural Competence Scale. Professional School Counseling, 25(1), 1–13. https://doi.org/10.1177/2156759X211050409

14.

Edwin

Bahr

M. W.

(2023). Multi-tiered System of Supports-Interventionist Beliefs Scale: Development and initial validation with school counselors. Journal of Psychoeducational Assessment, 41(7), 748–763. https://doi.org/10.1177/07342829231184293

15.

Goodman-Scott

(2014). Maximizing school counselors’ efforts by implementing School-wide Positive Behavioral Interventions and Supports: A case study from the field. Professional School Counseling, 17(1), 111–119. https://doi.org/10.5330/prsc.17.1.518021r2x6821660

16.

Goodman-Scott

Betters-Bubon

Donohue

(2015). Aligning Comprehensive School Counseling Programs and Positive Behavioral Interventions and Supports to maximize school counselors’ efforts. Professional School Counseling, 19(1), 57–67. https://doi.org/10.5330/1096-2409-19.1.57

17.

Goodman-Scott

Grothaus

(2017a). RAMP and PBIS: “They definitely support one another.” A phenomenological study. Professional School Counseling, 21(1), 119–129. https://doi.org/330/1096-2409-21.1.119

18.

Goodman-Scott

Grothaus

(2017b). School counselors’ roles in RAMP and PBIS: A phenomenological investigation. Professional School Counseling, 21(1), 130–141. https://doi.org/10.5330/1096-2409-21.1.130

19.

Goodman-Scott

Betters-Bubon

Donohue

(2019). A School Counselor’s Guide to Multi-tiered Systems of Support(1st ed.). Routledge.

20.

Goodman-Scott

Edirmanasinghe

Moe

Boulden

(2022b). Assessing the influence of Multitiered Systems of Support training on school counselors’ perceptions of school counseling activities: Results of a national study. Professional School Counseling, 26(1), 1–10. https://doi.org/10.1177/2156759X221138232

21.

Goodman-Scott

Kalkbrenner

M. T.

Olsen

Betters-Bubon

J. J.

(2022). The School Counselor/MTSS Alignment Scale (SC/MTSS-AS) [Screening Tool].

22.

Goodman-Scott

Ziomek-Daigle

(2022a). School counselors’ leadership experiences in Multi-tiered Systems of Support: Prioritizing relationships and shaping school climate. Journal of Counseling and Development, 100, 266–277. https://doi.org/10.1002/jcad.12426

23.

Hines

E. M.

Mayes

R. D.

Harris

P. C.

Vega

(2022). Using a culturally responsive MTSS approach to prepare black males for postsecondary opportunities. School Psychology Review, 52(3), 357–371. https://doi.org/10.1080/2372966X.2021.2018917

24.

Kalkbrenner

M.T.

(2021). A practical guide to instrument development and score validation in the social sciences: The MEASURE Approach. Practical Assessment, Research & Evaluation, 26(1), Article 1. https://doi.org/10.7275/svg4-e671

25.

Lenz

A. S.

Ault

Balkin

R. S.

Barrio Minton

Erford

B. T.

Hays

D. G.

Kim

B. S. K.

(2022). Responsibilities of users of standardized tests (RUST-4E): Prepared for the Association for Assessment and Research in Counseling. Measurement and Evaluation in Counseling and Development, 55(4), 227–235. https://doi.org/10.1080/07481756.2022.2052321

26.

Mayes

R. D.

Byrd

J. A.

(2022). An antiracist framework for evidence-informed school counseling practice. Professional School Counseling, 26(1a), 1–10. https://doi.org/10.1177/2156759X221086740

27.

McIntosh

Goodman

(2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. Guilford Press.

28.

Mvududu

N. H.

Sink

C. A.

(2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4(2), 75–98. https://doi.org/10.1177/2150137813494766

29.

National Association of School Psychologists. (2021). ESSA and multitiered systems of support for school psychologists. https://www.nasponline.org/research-and-policy/policy-priorities/relevant-law/the-every-student-succeeds-act/essa-implementation-resources/essa-and-mtss-for-school-psychologists

30.

Ockerman

M. S.

Mason

E. C. M.

Hollenbeck

A. F.

(2012). Integrating RTI with school counseling programs: Being a proactive professional school counselor. Journal of School Counseling, 10(15), 1–37. https://jsc.montana.edu/articles/v10n15.pdf

31.

Olsen

Parikh Foxx

Flowers

(2020). A confirmatory factor analysis of the school counselor knowledge and skills survey for multi-tiered systems of support. The Professional Counselor, 10(3), 376–392. https://doi.org/10.15241/jo

32.

Olsen

Parikh Foxx

Flowers

(2021). A confirmatory factor analysis of the school counseling program implementation survey. Journal of School Counseling, 19(44), 1–38. http://www.jsc.montana.edu/articles/v19n44.pdf

33.

Olsen

Betters-Bubon

Edirmanasinghe

(2024). Is it students or the system? Infusing a culturally sustaining approach to Tier 2 groups within multi-tiered systems of support. Professional School Counseling, 28(1a), 2156759X241234926.

34.

Patrikakou

Ockerman

M. S.

Hollenbeck

A. F.

(2020). Perceptions of leadership and clarity of roles within multi-tiered systems: A structural equation model. The Journal of Counselor Preparation and Supervision, 13(4), 7. https://digitalcommons.sacredheart.edu/jcps/vol13/iss4/7

35.

Santiago

R. T.

Hall

G. J.

Garbacz

S. A.

Gulbrandson

Albers

C. A.

(2025). Examining an integrated factor structure of schoolwide MTSS implementation measures. Journal of Positive Behavior Interventions, 27(1), 39–49. https://doi.org/10.1177/10983007241249524

36.

School Social Work Association of America. (2021). Who are school social workers. https://www.sswaa.org/_files/ugd/426a18_98102fe3074e4529a17602419df0be59.pdf

37.

Schreiber

J. B.

Nora

Stage

F. K.

Barlow

E. A.

King

(2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Educational Research, 99(6), 323–337. https://doi.org/10.3200/JOER.99.6.323-338

38.

Smith

L. C.

Garnett

B. R.

Herbert

Grudev

Vogel

Keefner

Barnett

Baker

(2017). The hand of professional school counseling meets the glove of restorative practices: A call to the profession. Professional School Counseling, 21(1), 1–10. https://doi.org/10.1177/2156759X18761899

39.

Steen

Ieva

Beasley

DeLucia

(2024). Visualizing systemic change through group counseling: Established and emerging approaches. Professional School Counseling, 28(1a), 1–4. https://doi.org/10.1177/2156759X241235118

40.

Swank

J. M.

(2022). Multitiered system of support with attention problems: The role of the school counselor. Journal of Professional Counseling: Practice, Theory & Research, 49(2), 76–90. https://doi.org/10.1080/15566382.2022.2141486

41.

Sugai

La Salle

T. P.

Everett

Feinberg

A. B.

(2019). Multi-tiered systems of support. In Goodman-Scott

Betters-Bubon

Donohue

(Eds.), The school counselor’s guide to multi-tiered systems of support (1st ed.; pp. 1–28). Routledge. https://doi.org/10.4324/9781315144467

42.

Taylor

J. V.

Burgess

(2023). Using data in MTSS to improve systems and student outcomes. In Goodman-Scott

Betters-Bubon

Donohue

Olsen

(Eds.), The school counselor’s guide to multi-tiered systems of support (2nd ed.; pp. 297–316). Routledge. https://doi.org/10.4324/9781003306412

43.

Weinberg

J. R.

Cooper

J. M.

(2023). Examining the mental health needs of sexual and gender minority youth to articulate a multitiered system of supports for schools. Psychology in the Schools, 60(9), 3612–3632. https://doi.org/10.1002/pits.22933

44.

Vagias

W. M.

(2006). Likert-type scale response anchors. Clemson International Institute for Tourism & Research Development, Department of Parks, Recreation and Tourism Management. Clemson University. http://media.clemson.edu/cbshs/prtm/research/resources-for-research-page-2/Vagias-Likert-Type-Scale-Response-Anchors.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.02 MB

0.00 MB