The purpose of this study was to evaluate the psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS): a quick and easy universal screener for behavioral and emotional risk. Elementary school teachers completed the SAEBRS with 346 students in Grades 3 to 5. Teachers also completed two criterion measures, including the Student Risk Screening Scale (SRSS) and the Student Internalizing Behavior Screener (SIBS). Additional extant behavioral and academic data sources were collected including office discipline referrals, suspensions, curriculum-based measurement scores, and statewide achievement test scores. Reliability analyses were indicative of the internal consistency of all four SAEBRS scales, whereas correlational analyses and Mann–Whitney–Wilcoxon tests supported the criterion-related and construct validity. Receiver operating characteristic curve analyses suggested each SAEBRS scale was associated with acceptable or optimal diagnostic accuracy. However, cut scores selected as most appropriate within each SAEBRS scale were found to differ from those identified in previous studies, potentially suggesting the influence of criterion outcome under consideration on SAEBRS diagnostic accuracy. Limitations and future directions for research are discussed, with emphasis on the need for continued examination of the extent of variability in SAEBRS cut score performance.
BearG. G.YangC.PellM.GaskinsC. (2014). Validation of a brief measure of teachers’ perceptions of school climate: Relations to student achievement and suspensions. Learning Environments Research, 17, 339–354.
2.
CampbellD. T.FiskeD. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
3.
CarranD. T.ScottK. G. (1992). Risk assessment in preschool children: Research implications for the early detection of educational handicaps. Topics in Early Childhood Special Education, 12, 196–211.
4.
ChafouleasS. M.KilgusS. P.WallachN. (2010). Ethical dilemmas in school-based behavioral screening. Assessment for Effective Intervention, 35, 245–252.
5.
CohenJ. (1988). Statistical power analysis for the behavior sciences (2nd ed.). New York, NY: Academic Press.
6.
CookC. R.RasetshwaneK. B.TruelsonE.GrantS.DartE. H.CollinsT. A.SpragueJ. (2011). Development and validation of the Student Internalizing Behavior Screener: Examination of reliability, validity, and classification accuracy. Assessment for Effective Intervention, 36, 71–79.
7.
CortinaJ. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal or Applied Psychology, 78, 98–104.
8.
CostenbaderV.MarksonS. (1998). School suspension: A study with secondary school students. Journal of School Psychology, 36, 59–82.
CurranP. J.WestS. G.FinchJ. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29.
11.
DarneyD.ReinkeW. M.HermanK. C.StormontM.IalongoN. S. (2012). Children with co-occurring academic and behavior problems in first grade: Distal outcomes in twelfth grade. Journal of School Psychology, 51, 117–128.
12.
DenoS. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37, 184–192.
13.
DiPernaJ. C. (2006). Academic enablers and student achievement: Implications for assessment and intervention services in the schools. Psychology in the Schools, 43, 7–17.
14.
DowdyE.DoaneK.EklundK.DeverB. V. (2011). A comparison of teacher nomination and screening to identify behavioral and emotional risk within a sample of underrepresented students. Journal of Emotional and Behavioral Disorders, 21, 127–137.
15.
DrummondT. (1994). The Student Risk Screening Scale (SRSS). Grants Pass, OR: Josephine County Mental Health Program.
16.
EndersC. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
17.
GloverT. A.AlbersC. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117–135.
18.
GreshamF. M.ElliottS. N. (2008). Social skills improvement system: Rating scales. Bloomington, MN: Pearson.
19.
GrissomR. J.KimJ. J. (2012). Effect sizes for research: Univariate and multivariate applications (2nd ed.). New York, NY: Taylor & Francis.
20.
HintzeJ. M.SilberglittB. (2005). A longitudinal examination of the diagnostic accuracy and predictive validity of R-CBM and high-stakes testing. School Psychology Review, 34, 372–386.
21.
IrvinL. K.TobinT. J.SpragueJ. R.SugaiG.VincentC. G. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions, 6, 131–147.
22.
JenkinsJ. R.HudsonR. F.JohnsonE. S. (2007). Screening for at-risk readers in a response to intervention framework. School Psychology Review, 36, 582–600.
23.
JenkinsL. N.DemarayM. K.WrenN. S.SecordS. M.LyellK. M.MagersA. M.. . . TennantJ. (2014). A critical review of five commonly used social-emotional and behavioral screeners for elementary or secondary schools. Contemporary School Psychology, 18, 241–254.
24.
KamphausR. W.ReynoldsC. R. (2007). BASC-2 Behavioral and Emotional Screening System. Minneapolis, MN: Pearson.
25.
KaneM. T. (2013). Validating the interpretations and uses of tests scores. Journal of Educational Measurement, 50, 1–73.
26.
KilgusS. P.ChafouleasS. M.Riley-TillmanT. C. (2013). Development and initial validation of the Social and Academic Behavior Risk Screener for elementary grades. School Psychology Quarterly, 28, 210–226.
27.
KilgusS. P.ChafouleasS. M.Riley-TillmanT. C.von der EmbseN. P. (2014). Social, Academic, and Emotional Behavior Risk Screener (SAEBRS). Minneapolis, MN: Theodore J. Christ & Colleagues.
28.
KilgusS. P.EklundK. R.von der EmbseN. P.TaylorC.SimsW. (2016). An evaluation of the concurrent validity and diagnostic accuracy of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) within elementary and middle school samples. Manuscript in preparation.
29.
KilgusS. P.MetheS. A.MagginD. M.TomasulaJ. L. (2014). Curriculum-based measurement oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52, 377–405.
30.
KilgusS. P.Riley-TillmanT. C.ChafouleasS. M.ChristT. J.WelshM. E. (2014). Direct behavior rating as a school-based behavior universal screener: Replication across sites. Journal of School Psychology, 52, 63–82.
31.
KilgusS. P.SimsW. A.von der EmbseN. P.Riley-TillmanT. C. (2015). Confirmation of models for interpretation and use of the Social and Academic Behavior Risk Screener (SABRS). School Psychology Quarterly, 30, 335–352.
32.
KingK.ReschlyA. L.AppletonJ. J. (2012). An examination of the validity of the Behavioral and Emotional Screening System in a rural elementary school. Journal of Psychoeducational Assessment, 30, 527–538.
33.
KlineR. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.
34.
KwonK.KimE.SheridanS. (2012). Behavioral competence and academic functioning among early elementary children with externalizing problems. School Psychology Review, 41, 123–140.
35.
LaneK. L.BruhnA. L.EisnerS. L.KalbergJ. R. (2010). Score reliability and validity of the Student Risk Screening Scale: A psychometrically sound, feasible tool for use in urban middle schools. Journal of Emotional and Behavioral Disorders, 18, 211–224.
36.
LaneK. L.KalbergJ. R.LambertE. W.CrnoboriM.BruhnA. L. (2010). A comparison of systematic screening tools for emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 18, 100–112.
37.
LaneK. L.MenziesH. M.OakesW. P.KalbergJ. R. (2012). Systematic screenings of behavior to support instruction: From preschool to high school. New York, NY: Guilford Press.
38.
McIntoshK.CampbellA. L.CarterD. R.ZumboB. D. (2009). Concurrent validity of office discipline referrals and cut points used in schoolwide positive behavior support. Behavioral Disorders, 34, 100–113.
39.
MoilanenK. L.ShawD. S.MaxwellK. L. (2010). Developmental cascades: Externalizing, internalizing, and academic competence from middle childhood to early adolescence. Development and Psychopathology, 22, 635–653.
40.
ParkerR. I.VannestK. (2009). An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy, 40, 357–367.
41.
ReschlyA. L.BuschT. W.BettsJ.DenoS. L.LongJ. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47, 427–469.
42.
ReynoldsC. R.KamphausR. W. (2004). Behavior Assessment System for Children–Second Edition (BASC-2). Circle Pines, MN: AGS.
43.
StreinerD. L.CairneyJ. (2007). What’s under the ROC? An introduction to receiver operating characteristic curves. The Canadian Journal of Psychiatry, 52, 121–128.
44.
SugaiG.SpragueJ. R.HornerR. H.WalkerH. M. (2000). Preventing school violence: The use of office discipline referrals to assess and monitor school-wide discipline interventions. Journal of Emotional and Behavioral Disorders, 8, 94–101.
45.
von der EmbseN. P.PendergastL.KilgusS. P.EklundK. R. (in press). Evaluating the applied use of a mental health screener: Structural validity of the Social, Academic, and Emotional Behavior Risk Screener. Manuscript submitted for publication.
46.
WalkerH. M.IrvinL. K.NoellJ.SingerG. H. S. (1992). A construct score approach to the assessment of social competence: Rationale, technological considerations, and anticipated outcomes. Behavior Modification, 16, 448–474.
47.
WalkerH. M.SeversonH. (1992). Systematic screening of behavior disorders (SSBD): A multiple gating procedure. Longmont, CO: Sopris West.
