This article documents the results of a meta-analysis of available correlational validity evidence for the Social Skills Improvement System Performance Screening Guide (SSIS-PSG), which is a brief teacher-completed rating scale designed to be used as part of universal screening procedures. Article inclusion criteria included (a) published in English in a peer-reviewed journal, (b) administration of the PSG, and (c) provided validity evidence representative of the relationship between PSG scores and scores on related variables. Ten studies yielding 147 correlation coefficients met criteria for inclusion. Data were extracted following established procedures in validity generalization and meta-analytic research. Extracted coefficients were of the expected direction and magnitude with theoretically aligned constructs, thereby providing evidence of convergent validity (e.g., PSG Math and Reading items were most strongly correlated with academic performance and academic behavior variables, with effect sizes ranging from .708 to .740; PSG Prosocial Behavior and Motivation to Learn items were most strongly correlated with broadband externalizing/internalizing problems, with effect sizes ranging from −.706 to −.717), although Prosocial Behavior and Motivation to Learn were not as effective at discriminating among divergent constructs. These results generally support the utility of the PSG in correlating with academic and social/behavioral outcomes in the schools.
AllenA. N.KilgusS. P.BurnsM. W.HodgsonC. (2019). Surveillance of internalizing behaviors: A reliability and validity generalization study of universal screening evidence. School Mental Health, 11, 194–209. https://doi.org/10.1007/s12310-018-9290-3
2.
AssinkM.WibbelinkC. J. M. (2016). Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology, 12, 154–174. https://doi.org/10.20982/tqmp.12.3.p154
3.
BembenuttyH. (2016). Motivation and self-regulated learning among preservice and in-service teachers enrolled in educational psychology courses. Scholarship of Teaching and Learning in Psychology, 2, 231–244. http://dx.doi.org/10.1037/stl0000068
4.
BensonN. F.FloydR. G.KranzlerJ. H.EckertT. L.FeferS. A.MorganG. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 National Survey. Journal of School Psychology, 72, 29–48. https://doi.org/10.1016/j.jsp.2018.12.004
5.
BeretvasS. N.PastorD. A. (2003). Using mixed-effects models in reliability generalization studies. Educational & Psychological Measurement, 63, 75–95.
CarsonK. P.SchriesheimC. A.KinickiA. J. (1990). The usefulness of the “fail-safe” statistic in meta-analysis. Educational and Psychological Measurement, 50, 233–243. doi:10.1177/0013164490502001
8.
CartaJ. J. (2019). Introduction to multi-tiered systems of support in early education. In CartaJ. J.YoungR. M. (Eds.), Multi-tiered systems of support for young children: Driving change in early education (pp. 1–14). Brookes.
9.
ChafouleasS. M.KilgusS. P.JafferyR.Riley-TillmanT. C.WelshM.ChristT. J. (2013). Direct behavior rating as a school-based behavior screener for elementary and middle grades. Journal of School Psychology, 51, 367–385. https://doi.org/10.1016/j.jsp.2013.04.002
10.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Routledge.
11.
DiazA.EisenbergN.ValienteC.VanSchyndelS.SpinardT. L.BergerR.. . . SouthworthJ. (2017). Relations of positive and negative expressivity and effortful control to kindergarteners’ student-teacher relationship, academic engagement, and externalizing problems at school. Journal of Research in Personality, 67, 3–14. https://doi.org/10.1016/j.jrp.2015.11.002
12.
DiPernaJ. C.VolpeR. J.ElliottS. N. (2002). A model of academic enablers and elementary reading/language arts achievement. School Psychology Review, 31, 298–312.
13.
DiPernaJ. C.VolpeR. J.ElliottS. N. (2005). A model of academic enablers and mathematics achievement in the elementary grades. Journal of School Psychology, 43, 379–392. doi:101016/jjsp200509002.
14.
DowdyE.KamphausR. W.AbdouA. S.TwyfordJ. M. (2012). Detection of symptoms of prevalent mental health disorders of childhood with the parent form of the Behavioral and Emotional Screening System. Assessment for Effective Intervention, 38, 192–198. https://doi.org/10.1177/1534508412447009
15.
*ElliottS. N.DaviesM. D.FreyJ. R.GreshamF.CooperG. (2018). Development and initial validation of a social emotional learning assessment for universal screening. Journal of Applied Developmental Psychology, 55, 39–51. https://doi.org/10.1016/j.appdev.2017.06.002
16.
ElliottS. N.GreshamF. K. (2008). SSIS performance screening guide. Pearson Assessments.
17.
GloverT. A.AlbersC. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117–135. doi:101016/jjsp200605005.
18.
GreshamF. M.ElliottS. N. (2008). Social skills improvement system: Rating scales manual. NCS Pearson.
19.
GreshamF. M.ElliottS. N. (2017). Social skills improvement system social-emotional learning edition. NCS Pearson.
20.
*HartmanK.GreshamF. M.ByrdS. (2017). Student internalizing and externalizing behavior screeners: Evidence for reliability, validity, and usability in elementary schools. Behavioral Disorders, 42, 108–118. https://doi.org/10.1177/0198742916688656
21.
JenkinsL. N.DemarayM. K.WrenN. S.SecordS. M.LyellK. M.MagersA. M.. . . TennantJ. (2014). A critical review of five commonly used social-emotional and behavioral screeners for elementary or secondary school. Contemporary School Psychology, 18, 241–254. https://doi.org/10.1007/s40688-014-0026-6
22.
KamphausR. W.ReynoldsC. R. (2015). Behavior Assessment System for Children—Third Edition (BASC-3): Behavioral and Emotional Screening System (BESS). Pearson.
23.
*KettlerR. J.ElliottS. N.DaviesM.GriffinP. (2012). Testing a multi-stage screening system: Predicting performance on Australia’s national achievement test using teachers’ ratings of academic and social behaviors. School Psychology International, 33, 93–111. doi:10.1177/0143034311403036
24.
*KilgusS. P.ChafouleasS. M.Riley-TillmanT. C.WelshM. E. (2012). Direct behavior rating scales as screeners: A preliminary investigation of diagnostic accuracy in elementary school. School Psychology Quarterly, 27, 41–50. https://doi.org/10.1037/a0027150
25.
KilgusS. P.EklundK.MagginD. M.TaylorC. N.AllenA. N. (2017). The Student Risk Screening Scale: A reliability and validity generalization meta-analysis. Journal of Emotional and Behavioral Disorders, 26, 143–155. https://doi.org/10.1177/1063426617710207
26.
KilgusS. P.EklundK.von der EmbseN. P.TaylorC. N.SimsW. A. (2016). Psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) Teacher Rating Scale and multiple gating procedure within elementary and middle school samples. Journal of School Psychology, 58, 21–39. https://doi.org/10.06/j/jsp.2016.07.001
27.
KilgusS. P.ReinkeW. M.JimersonS. R. (2015). Understanding mental health intervention and assessment within a multi-tiered framework: Contemporary science, practice, and policy. School Psychology Quarterly, 30, 159–165. https://doi.org/10.1037/spq0000118
28.
*KrachS. K.McCreeryM. P.WangY.MohammadiaminH.CirksC. K. (2017). Diagnostic utility of the social skills improvement system performance screening guide. Journal of Psychoeducational Assessment, 35, 391–409. https://doi.org/10.1177/0734282916636500
29.
*LaneK. L.OakesW. P.CommonE. A.ZorigianK.BrunstingN. C.SchatschneiderC. (2015). A comparison between SRSS-IE and SSIS-PSG scores: Examining convergent validity. Assessment for Effective Intervention, 40, 114–126. https://doi.org/10.1177/1534508414560346
30.
*LaneK. L.OakesW. P.EnnisR. P.RoyerD. J. (2015). Additional evidence of convergent validity between SRSS-IS and SSiS-PSG scores. Behavioral Disorders, 40, 213–229.
31.
*LaneK. L.Richards-TutorC.OakesW. P.ConnorK. (2014). Initial evidence for the reliability and validity of the student risk screening scale with elementary age English learners. Assessment for Effective Intervention, 39, 219–232. https://doi.org/10.1177/1534508413496836
32.
MegaC.RonconiL.De BeniR. (2014). What makes a good student? How emotions, self-regulated learning, and motivation contribute to academic achievement. Journal of Educational Psychology, 106, 121–131. https://doi.org/10.1037/a0033546
33.
*MillerF. G.CohenD.ChafouleasS. M.Riley-TillmanT. C.WelshM. E.FabianoG. A. (2015). A comparison of measures to screen for social, emotional, and behavioral risk. School Psychology Quarterly, 30, 184–196. https://doi.org/10.1037/spq0000085
34.
MooreS. A.Widales-BenitezO.CarnazzoK. W.KimE. K.MoffaK.DowdyE. (2015). Conducting universal complete mental health screening via student self-report. Contemporary School Psychology, 19, 253–267. https://doi.org/101007/s406880150062x
35.
*OakesW. P.LaneK. L.EnnisR. P. (2016). Systematic screening at the elementary level: Considerations for exploring and installing universal behavior screening. Journal of Applied School Psychology, 32, 214–233. https://doi.org/10.1080/15377903.2016.1165325
36.
OrwinR. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159. https://doi.org/10.2307/1164923
37.
QuintanaD. S. (2015). From pre-registration to publication: A non-technical primer for conducting a meta-analysis to synthesize correlational data. Frontiers in Psychology, 6, 1–9. https://doi.org/10.3389/fpsyg.2015.01549
38.
R Core Team. (2015). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.Rproject.org/
39.
ReinkeW. M.HermanK. C. (2016). Using brief assessments of important indicators to inform school-based interventions and practice. Assessment for Effective Intervention, 42, 3–5.
40.
ShultzK. S.WhitneyD. J. (2005). Measurement theory in action: Case studies and exercises. Sage.
41.
StormontM. A.HermanK. C.ReinkeW. M.KingK. R.OwensS. (2015). The Kindergarten academic and behavior readiness screener: The utility of single-item teacher ratings of kindergarten readiness. School Psychology Quarterly, 30, 212–228.
42.
StormontM. A.ThompsonA. M.HermanK. C.ReinkeW. M. (2017). The social and emotional dimensions of a single item overall school readiness screener and its relation to academic outcomes. Assessment for Effective Intervention, 42, 67–76.
43.
SullivanJ. R.ConoleyJ. C. (2004). Academic and instructional interventions with aggressive students. In ConoleyJ. C.GoldsteinA. P. (Eds.), School violence intervention: A practical handbook (2nd ed., pp. 235–255). Guilford Press.
44.
ThorneS.KampsD. (2008). The effects of group contingency intervention on academic engagement and problem behavior of at-risk students. Behavior Analysis in Practice, 1, 12–18.
VillarrealV.SullivanJ.LeethC. (2019). BASC-2 Behavioral and Emotional Screening System: A validity generalization meta-analysis. Manuscript submitted for publication.
47.
WangM.EcclesJ. S. (2013). School context, achievement motivation, and academic engagement: A longitudinal study of school engagement using a multidimensional perspective. Learning and Instruction, 23, 12–23. https://doi.org/10.1016/j.learninstruc.2013.04.002
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
