An identified barrier to the widespread adoption of universal socioemotional and behavioral screening in schools may be that existing instruments may be too burdensome or costly to administer. As a result, the County Schools Mental Health Coalition came together to develop a common assessment system, the Early Identification System (EIS). The purpose of this study was to evaluate the psychometric properties of the EIS-student report, using a sample of third- to eighth-grade students. A series of exploratory and confirmatory factor analytic models were used with one exploratory (n1 = 450) and two holdout samples (n2 = 450, n3 = 940). A six correlated factor model was supported showing adequate model fit, reliability (range = .73-.90), and convergent validity. Results suggest that the EIS is a promising, cost-free, and quick-to-administer universal screening tool that may assist educators and school psychologists identify students at risk of future socioemotional and behavioral difficulties.
AndersonJ. C.GerbingD. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411-423.
2.
BrowneM. W.CudeckR. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230-258.
3.
BurnsJ. R.RapeeR. M. (2016). Screening for mental health risk in high schools: The development of the Youth RADAR. Psychological Assessment, 28, 1220-1231.
4.
Centers for Disease Control and Prevention. (2013). Mental health surveillance among children—United States, 2005-2011. Morbidity and Mortality Weekly Report, 62, 1-35.
5.
CicchettiD.TuckerD. (1994). Development and self-regulatory structures of the mind. Development and Psychopathology, 6, 533-549.
6.
CicchettiD. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290.
7.
ConnollyS. D.BernsteinG. A, &Work Group on Quality Issues (2007). Practice parameter for the assessment and treatment of children and adolescents with anxiety disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 267-283.
8.
CookC. R.VolpeR. J.LivanisA. (2010). Constructing a roadmap for future universal screening research beyond academics. Assessment for Effective Intervention, 35, 197-205.
9.
CornellD.KleinJ.KonoldT.HuangF. (2012). Effects of validity screening items on adolescent survey data. Psychological Assessment, 24, 21-35.
10.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
11.
DowdyE.RitcheyK.KamphausR. W. (2010). School-based screening: A population-based approach to inform and monitor children’s mental health needs. School Mental Health, 2, 166-176.
12.
DunnT. J.BaguleyT.BrunsdenV. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399-412.
13.
EssauC. A. (2005). Frequency and patterns of mental health services utilization among adolescents with anxiety and depressive disorders. Depression and Anxiety, 22, 130-137.
14.
FabrigarL. R.WegenerD. T. (2012). Exploratory factor analysis. New York, NY: Oxford University Press.
15.
FinneyS.DiStefanoC. (2006). Non-normal and categorical data in structural equation modeling. In HancockG.MuellerR. (Eds.), Structural equation modeling: A second course (pp. 439-492). Greenwich, CT: Information Age Publishing.
16.
GadermannA.GuhnM.ZumboB. (2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation, 17. Retrieved from http://www.pareonline.net/getvn.asp?v=17&n=3
17.
GloverT. A.AlbersC. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117-135.
18.
Hall-LandeJ. A.EisenbergM. E.ChristensonS. L.Neumark-SztainerD. (2007). Social isolation, psychological health, and protective factors in adolescence. Adolescence, 42, 265-286.
19.
HermanK. C.LambertS. F.IalongoN. S.OstranderR. (2007). Academic pathways between attention problems and depressive symptoms among urban African American children. Journal of Abnormal Child Psychology, 35, 265-274.
20.
HoganT. P.BenjaminA.BrezinskiK. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523-531.
21.
HornJ. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185.
22.
HornerR. H.SugaiG.SmolkowskiK.EberL.NakasatoJ.ToddA. W.EsperanzaJ. (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-144.
23.
HuL. T.BentlerP. M. (1995). Evaluating model fit. In HoyleR. H. (Ed.) Structural equation modeling: Concepts, issues, and applications (pp. 76-99). Thousand Oaks, CA: SAGE.
24.
HuangF. (2014). Analyzing group level effects with clustered data using Taylor series linearization. Practical Assessment, Research, and Evaluation, 19, 1-9.
25.
HuangF. (2016). Alternatives to multilevel modeling for the analysis of clustered data. Journal of Experimental Education, 84, 175-196.
26.
KataokaS. H.ZhangL.WellsK. B. (2002). Unmet need for mental health care among US children: Variation by ethnicity and insurance status. American Journal of Psychiatry, 159, 1548-1555.
27.
KelleyK.PornprasertmanitS. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21, 69-92.
28.
KlineR. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.
29.
KuhnC.AebiM.JakobsenH.BanaschewskiT.PoustkaL.GrimmerY.. . . SteinhausenH. C. (2017). Effective mental health screening in adolescents: Should we collect data from youth, parents or both?Child Psychiatry and Human Development, 48, 385-392.
30.
LaneK. L.CarterE. W.JenkinsA.DwigginsL.GermerK. (2015). Supporting comprehensive, integrated, three-tiered models of prevention in schools: Administrators’ perspectives. Journal of Positive Behavior Interventions, 17, 209-222.
31.
LevittJ. M.SakaN.RomanelliL. H.HoagwoodK. (2007). Early identification of mental health problems in schools: The status of instrumentation. Journal of School Psychology, 45, 163-191.
32.
LewinsohnP. M.RohdeP.SeeleyJ. R. (1998). Major depressive disorder in older adolescents: Prevalence, risk factors, and clinical implications. Clinical Psychology Review, 18, 765-794.
33.
McNeishD. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23, 412-433.
NelsonJ. R.StageS.Duppong-HurleyK.SynhorstL.EpsteinM. H. (2007). Risk factors predictive of the problem behavior of children at risk for emotional and behavioral disorders. Exceptional Children, 73, 367-379.
36.
OdgersC. L.CaspiA.NaginD. S.PiqueroA. R.SlutskeW. S.MilneB. J.. . . MoffittT. E. (2008). Is it important to prevent early exposure to drugs and alcohol among adolescents?Psychological Science, 19, 1037-1044.
37.
PattersonG. R.ReidJ. B.DishionT. J. (1992). Antisocial boys (Vol. 4). Eugene, OR: Castalia.
38.
ReidJ. B.PattersonG. R.SnyderJ. E. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, DC: American Psychological Association.
39.
ReidR.GonzalezJ. E.NordnessP. D.TroutA.EpsteinM. H. (2004). A meta-analysis of the academic status of students with emotional/behavioral disturbance. The Journal of Special Education, 38, 130-143.
40.
RevelleW. (2015). psych: Procedures for psychological, psychometric, and personality research. Evanston, IL: Northwestern University.
41.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36.
42.
ScottM. A.WilcoxH. C.SchonfeldI. S.DaviesM.HicksR. C.TurnerJ. B.ShafferD. (2009). School-based screening to identify at-risk students not already known to school professionals: The Columbia suicide screen. American Journal of Public Health, 99, 334-339.
43.
SeversonH. H.WalkerH. M. (2002). Proactive approaches for identifying children at risk for sociobehavioral problems. In LaneK.GreshamF.O’ShaughnessyT. (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 33-53). Boston, MA: Allyn & Baco.
44.
SeversonH. H.WalkerH. M.Hope-DoolittleJ.KratochwillT. R.GreshamF. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193-223.
TaylorL. D.Davis-KeanP.MalanchukO. (2007). Self-esteem, academic self-concept, and aggression at school. Aggressive Behavior, 33, 130-136.
47.
TemcheffC. E.SerbinL. A.Martin-StoreyA.StackD. M.HodginsS.LedinghamJ.SchwartzmanA. E. (2008). Continuity and pathways from aggression in childhood to family violence in adulthood: A 30-year longitudinal study. Journal of Family Violence, 23, 231-242.
48.
ThompsonA.ReinkeW.HermanK. (2014). In MassatC.KellyM.ConstableR. (Eds.), School social work: Practice, policy, and research (8th ed.). New York, NY: Oxford.
49.
ThompsonB.DanielL. G. (1996). Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56, 197-208.
50.
TremblayR. E.NaginD. S.SeguinJ. R.ZoccolilloM.ZelazoP. D.BoivinM.. . . JapelC. (2004). Physical aggression during early childhood: Trajectories and predictors. Pediatrics, 14, 3-9.
51.
VolpeR. J.BrieschA. M.ChafouleasS. M. (2010). Linking screening for emotional and behavioral problems to problem-solving efforts: An adaptive model of behavioral assessment. Assessment for Effective Intervention, 35, 240-244.
52.
WestenD.RosenthalR. (2003). Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology, 84, 608-618.
53.
WillnerC. J.Gatzke-KoppL. M.BrayB. C. (2016). The dynamics of internalizing and externalizing comorbidity across the early school years. Development and Psychopathology, 28, 1033-1052.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
