We report results from this psychometric study examining convergent validity between internalizing subscale (SRSS-I4) scores from the revised version of the teacher-completed Student Risk Screening Scale for Internalizing and Externalizing behavior (SRSS-IE 9) and the internalizing subscale from the Teacher Report Form (TRF). Using the sample of nine middle and high schools across two U.S. Midwestern states with 227 sixth- through twelfth-grade students, we replicated the data analytic plan used to determine initial cutting scores for use in secondary schools. Results from logistic regression and receiver operator characteristic curves suggested the following preliminary SRSS-I4 risk categories for secondary students: 0 to 2 low, 3 moderate, and 4 to 12 high risk for internalizing behavior patterns relative to the TRF internalizing subscale score. We intended to conduct subsequent analysis to address the nested nature of the data. However, 43.70% of teachers rated only one student on the TRF, resulting in there not being enough observations per classroom estimate a meaningful and accurate random slope in many cases. We discuss limitations and directions for future research, encouraging replication before shifting screening practices in middle and high schools.
AchenbachT. M. (1991). Manual for the child behavior checklist/4-18 and 1991 profile. Department of Psychiatry, University of Vermont.
2.
AchenbachT. M.RescorlaL. A. (2001). Manual for the ASEBA school-age forms & profiles. Research Center for Children, Youth, & Families, University of Vermont.
3.
AllenG. E.CommonE. A.GermerK. A.LaneK. L.BuckmanM. M.OakesW. P.MenziesH. M. (2020). A systematic review of the evidence base for active supervision in PK-12 settings. Behavioral Disorders, 45(3), 167–182. https://doi.org/10.1177/0198742919837646
4.
AlrubayieS. A.AlkahtaniK. D. F. (2024). Evidence for reliability and validity of the Arabic version of the Student Risk Screening Scale for Internalizing and Externalizing behaviors (SRSS-IE). Journal of Behavioral and Brain Science, 14(1), 32–45. https://doi.org/10.4236/jbbs.2024.141004
5.
BrieschA. M.LaneK. L.CommonE. A.OakesW. P.BuckmanM. M.ChafouleasS. M.SherodR. L.AbdulkerimN.RoyerD. J. (2022). Exploring views and professional learning needs of Comprehensive, Integrated, Three-Tiered (Ci3T) leadership teams related to universal behavior screening implementation. Education and Treatment of Children, 45, 245–262. https://doi.org/10.1007/s43494-022-00080-8
6.
BrieschA. M.ChafouleasS. M.DineenJ. N.McCoachD. B.DonaldsonA. (2022). School building administrator reports of screening practices across academic, behavioral, and health domains. Journal of Positive Behavior Interventions, 24(4), 266–277. https://doi.org/10.1177/10983007211003335
7.
BuckmanM. M.LaneK. L.CommonE. A.RoyerD. J.OakesW. P.AllenG. E.LaneK. S.BrunstingN. (2021). Treatment integrity of primary (tier 1) prevention efforts in tiered systems: Mapping the literature. Education and Treatment of Children, 44, 145–168. https://doi.org/10.1007/s43494-021-00044-4
8.
BuckmanM. M.LaneK. L.OakesW. P. (2022). Treatment integrity and social validity in tiered systems: Using data to inform implementation efforts. In LemonsC. J.PowellS. R.LaneK. L.AcevesT. C. (Eds.), Handbook of special education research: Research based practices and intervention innovations (pp. 19–30). Routledge.
9.
BuneviciusA.PeceliunieneJ.MickuvieneN.ValiusL.BuneviciusR. (2007). Screening for depression and anxiety disorders in primary care patients. Depression and Anxiety, 24, 455–460. https://doi.org/10.1186/1471-2296-10-58
10.
Ci3T Project ENHANCE Research Team. (2024, June). Selecting and Installing Behavior Screeners: Student Risk Screening Scale—internalizing and Externalizing (SRSS-IE). https://doi.org/10.17161/ci3t.98025
11.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. https://doi.org/10.1007/BF02310555
12.
DengJ.ZhouF.HouW.HeybatiK.LohitS.AbbasU.SilverZ.WongC. Y.ChangO.HuangE.ZouQ. K.MoskalykM.RamarajuH. B.HeybatiS. (2023). Prevalence of mental health symptoms in children and adolescents during the COVID-19 pandemic: A meta-analysis. Annals of the New York Academy of Sciences, 1520, 53–73. https://doi.org/10.1111/nyas.14947
13.
DrummondT. (1994). The Student Risk Screening Scale (SRSS). Josephine County Mental Health Program.
14.
EnnisR. P.RoyerD. J.LaneK. L.GriffithC. E. (2017). A systematic review of precorrection in PK-12 settings. Education and Treatment of Children, 40, 465–496. https://dx.doi.org/10.1353/etc.2017.0021
GandhiA. G.ClemensN.CoyneM.GoodmanS.LaneK. L.LembkeE.SimonsenB. (2023). Integrated multi-tiered systems of support (I-MTSS): New directions for supporting students with or at risk for learning disabilities. In Handbook of learning disabilities (3rd ed.). Guilford Press.
Individuals with Disabilities Education Improvement Act of 2004, Pub. L. No. 20 U.S.C. § 1400et seq. (2004).
19.
JonesC.GraybillE.BargerB.RoachA. T. (2020). Examining the predictive validity of behavior screeners across measures and respondents. Psychology in the Schools, 57(6), 923–936. https://doi.org/10.1002/pits.22371
20.
KamphausR. W.ReynoldsC. R. (2007). BASC–2 Behavioral and Emotional Screening System (BASC-2 BESS). Pearson Assessments.
21.
KazdinA. E. (1982). Symptom substitution, generalization, and response covariation: Implications for psychotherapy outcome. Psychological Bulletin, 91(2), 349–365. https://doi.org/10.1037/0033-2909.91.2.349
22.
KazdinA. E. (1987). Treatment of antisocial behavior in children: Current status and future directions. Psychological Bulletin, 102(2), 187–203. https://doi.org/10.1037/0033-2909.102.2.187
23.
KraemerH. C. (1992). Evaluating medical tests: Objective and quantitative guidelines. Sage Publications.
24.
LaneK. L.KalbergJ. R.ParksR. J.CarterE. W. (2008). Student Risks Screening Scale: Initial evidence for score reliability and validity at the high school level. Journal of Emotional and Behavioral Disorders, 16, 178–190. https://doi.org/10.1177/1063426608314218
25.
LaneK. L.LittleM. A.CaseyA. M.LambertW.WehbyJ. H.WeisenbachJ. L.PhillipsA. (2009). A comparison of systematic screening tools for emotional and behavioral disorders: How do they compare?Journal of Emotional and Behavioral Disorders, 17, 93–105. https://doi.org/10.1177/1063426608326203
26.
LaneK. L.MenziesH. M. (2009). Student Risk Screening Scale for Internalizing and Externalizing behavior (SRSS-IE). http://www.ci3t.org/screening
27.
LaneK. L.OakesW. P.BuckmanM. M.LaneN. A.LaneK. S.FlemingK.Swinburne RomineR.SherodR.ChangC.JonesJ.CantwellE.CrittendenM. (2023). Examination of the factor structure and measurement invariance of the SRSS-IE. Remedial and Special Education, 45(3), 152–172. https://doi.org/10.1177/07419325231193147
28.
LaneK. L.OakesW. P.BuckmanM. M.LaneN. A.LaneK. S.FlemingK.Swinburne RomineR.SherodR. L.CantwellE. D.ChangC. (2024). New evidence of predictive validity of SRSS-IE scores with middle and high school students. Frontiers in Education, 8, Article 1251063. https://doi.org/10.3389/feduc.2023.1251063
29.
LaneK. L.OakesW. P.BuckmanM. M.MenziesH. M. (2022). Comprehensive Integrated, Three-Tiered (Ci3T) models of prevention: Prioritizing integrated systems. In LemonsC. J.PowellS. R.LaneK. L.AcevesT. C. (Eds.), Handbook of special education research: Research based practices and intervention innovations (pp. 4–18). Routledge. https://doi.org/10.4324/9781003156888-2
30.
LaneK. L.OakesW. P.CantwellE. D.CommonE. A.RoyerD. J.LekoM.SchatschneiderC.MenziesH. M.BuckmanM. M.AllenG. E. (2019). Predictive validity of Student Risk Screening Scale for Internalizing and Externalizing (SRSS-IE) scores in elementary schools. Journal of Emotional and Behavioral Disorders, 27(4), 221–234. https://doi.org/10.1177/1063426618795443
31.
LaneK. L.OakesW. P.CantwellE. D.SchatschneiderC.MenziesH.CrittendenM.MessengerM. (2016). Student Risk Screening Scale for Internalizing and Externalizing behaviors: Preliminary cut scores to support data-informed decision making in middle and high schools. Behavioral Disorders, 42, 271–284. https://doi.org/10.17988/BD-16-115.1
32.
LaneK. L.OakesW. P.CarterE. W.LambertW.JenkinsA. (2013). Initial evidence for the reliability and validity of the Student Risk Screening Scale for internalizing and externalizing behaviors at the middle school level. Assessment for Effective Intervention, 39, 24–38. https://doi.org/10.1177/1534508413489336
33.
LaneK. L.OakesW. P.CommonE. A.BrunstingN.ZorigianK.HicksT.LaneN. A. (2019). A comparison between SRSS-IE and BASC-2 BESS scores at the middle school level. Behavioral Disorders, 44(3), 162–174. https://doi.org/10.1177/0198742918794843
34.
LaneK. L.OakesW. P.LuskM. E.CantwellE. D.SchatschneiderC. (2016). Screening for intensive intervention needs at the secondary level: Directions for the future. Journal of Emotional and Behavioral Disorders, 24, 159–172. https://doi.org/10.1177/1063426615618624
35.
LaneK. L.OakesW. P.MenziesH. M. (2021). Considerations for systematic screening PK-12: Universal screening for internalizing and externalizing behaviors in the COVID-19 era. Preventing School Failure: Alternative Education for Children and Youth, 65(3), 275–281. https://doi.org/10.1080/1045988X.2021.1908216
36.
LaneK. L.OakesW. P.MenziesH. M. (2023). Using low-intensity strategies to support engagement: Practical applications in remote learning environments for teachers and families. Preventing School Failure: Alternative Education for Children and Youth, 67, 79–82. https://doi.org/10.1080/1045988x.2023.2181300
37.
LaneK. L.OakesW. P.MenziesH. M.OyerJ.JenkinsA. (2013). Working within the context of three-tiered models of prevention: Using school wide data to identify high school students for targeted supports. Journal of Applied School Psychology, 29, 203–229. https://doi.org/10.1080/15377903.2013.778773
38.
LaneK. L.ParksR. J.KalbergJ. R.CarterE. W. (2007). Systematic screening at the middle school level: Score reliability and validity of the Student Risk Screening Scale. Journal of Emotional and Behavioral Disorders, 15, 209–222. https://psycnet.apa.org/doi/10.1177/10634266070150040301
MajeikaC. E.WalderJ. P.HubbardJ. P.SteebK. M.FerrisG. J.OakesW. P.LaneK. L. (2011). Improving on-task behavior using a functional assessment-based intervention in an inclusive high school setting. Beyond Behavior, 20, 55–66.
41.
McDonaldR. (1999). Test theory: A unified treatment. Lawrence Erlbaum.
42.
MenziesH. M.OakesW. P.LaneK. L.RoyerD. J.BuckmanM. M. (2023). Understanding teachers’ adoption of a comprehensive reform program. Leadership and Policy in Schools, 22, 1066–1081. https://doi.org/10.1080/15700763.2022.2081213
43.
OakesW. P.LaneK. L.CantwellE. D.RoyerD. J. (2017). Systematic screening for behavior in K-12 settings as regular school practice: Practical considerations and recommendations. Journal of Applied School Psychology, 33, 369–393. https://doi.org/10.1080/15377903.2017.1345813
44.
PeltonK. S. L.LaneK. L.OakesW. P.BuckmanM. M.RoyerD. J.SherodR. L. (2024). Mapping the research base for universal behavior screeners. Review of Educational Research, 00346543251315168.
45.
PetrasH.ChilcoatH.LeafP. J.IalongoN. S.KellamS. G. (2004). Utility of TOCA-R scores during the elementary school years in identifying later violence among adolescent males. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 88–96. https://doi.org/10.1097/00004583-200401000-00018
46.
RacineN.McArthurB. A.CookeJ. E.EirichR.ZhuJ.MadiganS. (2021). Global prevalence of depressive and anxiety symptoms in children and adolescents during COVID-19: A meta-analysis. Journal of the American Medical Association Pediatrics, 175(11), 1142–1150. https://doi.org/10.1001/jamapediatrics.2021.2482
47.
R Core Team. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.R-project.org/
48.
RoyerD. J.LaneK. L.CantwellE. D.MessengerM. (2017). A systematic review of the evidence base for instructional choice in k-12 settings. Behavioral Disorders, 42, 89–107. https://doi.org/10.1177/0198742916688655
49.
SchatschneiderC.LaneK. L.OakesW.KalbergJ. (2014). The Student Risk Screening Scale: Exploring dimensionality and differential item functioning. Educational Assessment Journal, 19(3), 185–203. https://doi.org/10.1080/10627197.2014.934608
50.
UmbreitJ.FerroJ.LaneK. L.LiaupsinC. (2024). Functional assessment-based intervention: A practical, effective, and integrated approach. Guilford.
51.
WalkerH. M.FornessS. R.LaneK. L. (2014). Design and management of scientific research in applied school settings. In CookB.TankersleyM.LandrumT. (Eds.), Advances in learning and behavioral disabilities (Vol. 27, pp. 141–169). Emerald.
52.
WalkerH. M.SeversonH. H.FeilE. G. (2014). Systematic Screening for Behavior Disorders (SSBD) (2nd ed.). Ancora Publishing.
