This study examined the item and category response option functioning of items on the Student Risk Screening Scale for Internalizing and Externalizing Behaviors (SRSS-IE), using item response theory (IRT) methods with a sample of 2,122 middle school students. The SRSS-IE is a screening instrument used to identify students who are potentially at risk for emotional and behavioral disorders (EBD). Utilizing the nominal response and generalized partial credit models of IRT, items from the SRSS-IE were evaluated in terms of the degree to which the response options for each item functioned as intended and how well those response options discriminated among students who exhibited varying levels of EBD risk. Results indicate that the four-response option configuration used in this study may not adequately discriminate regarding the frequency of externalizing and internalizing behaviors demonstrated by middle school students. Recommendations for revising the response options and scoring procedures are discussed.
AchenbachT. M.RescorlaL. A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington: Research Center for Children, Youth, & Families, University of Vermont.
2.
AkaikeH. (1974). A new look at the statistical identification model. IEEE Transaction Automatic Control, 19, 716–723.
3.
BockR. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
4.
CaiL. (2013). flexMIRT® version 2: Flexible multilevel multidimensional item analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group.
5.
CookB. G. (2014). A call for examining replication and bias in special education research. Remedial and Special Education, 35, 233–246.
6.
CunninghamJ. M.SuldoS. M. (2014). Accuracy of teachers in identifying elementary school students who report at-risk levels of anxiety and depression. School Mental Health, 6, 237–250.
7.
DrummondT. (1994). The Student Risk Screening Scale (SRSS). Grants Pass, OR: Josephine County Mental Health Program.
8.
FarmerT. W.IrwinM. J.MotocaL. M.LeungM. C.HutchinsB. C.BrooksD. S.HallC. M. (2015). Externalizing and internalizing behavior problems, peer affiliations, and bullying involvement across the transition to middle school. Journal of Emotional and Behavioral Disorders, 23, 3–16.
9.
FornessS. R.FreemanS. F.PaparellaT.KauffmanJ. M.WalkerH. M. (2012). Special education implications of point and cumulative prevalence for children with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 20, 4–18.
10.
GloverT. A.AlbersC. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117–135.
11.
GreenJ. G.KeenanJ. K.GuzmánJ.VinnesS.HoltM.ComerJ. S. (2017). Teacher perspectives on indicators of adolescent social and emotional problems. Evidence-Based Practice in Child and Adolescent Mental Health, 2, 96–110.
12.
HardyC. L.BukowskiW. M.SippolaL. K. (2002). Stability and change in peer relationships during the transition to middle-level school. The Journal of Early Adolescence, 22, 117–142.
13.
HaydenE. P.MashE. J. (2014). Child psychopathology: A developmental-systems perspective. In MashE. J.BarkleyR. A. (Eds.), Child psychopathology (3rd ed., pp. 3–72). New York, NY: Guilford Press.
14.
KruegerR. F.MarkonK. E. (2006). Reinterpreting comorbidity: A model-based approach to understanding and classifying psychopathology. Annual Review of Clinical Psychology, 2, 111–133.
15.
LaneK. L.MenziesH. M. (2009). Student Risk Screening Scale for Early Internalizing and Externalizing Behavior (SRSS-IE). Retrieved from http://www.ci3t.org/screening#srssie
16.
LaneK. L.OakesW. P.CantwellE. D.MenziesH. M.SchatschneiderC.LambertW.CommonE. A. (2017). Psychometric evidence of the SRSS-IE scores in middle and high schools. Journal of Emotional and Behavioral Disorders, 25, 233–245. doi:10.1177/1063426616670862
17.
LaneK. L.OakesW. P.CantwellE. D.SchatschneiderC.MenziesH.CrittendenM.MessengerM. (2016). Student Risk Screening Scale for Internalizing and Externalizing Behaviors: Preliminary cut scores to support data-informed decision making in middle and high schools. Behavioral Disorders, 42, 271–284.
18.
LaneK. L.OakesW. P.CarterE. W.LambertW. E.JenkinsA. B. (2013). Initial evidence for the reliability and validity of the Student Risk Screening Scale for Internalizing and Externalizing Behaviors at the middle school level. Assessment for Effective Intervention, 39, 24–38.
19.
LaneK. L.OakesW. P.EnnisR. P.CoxM. L.SchatschneiderC.LambertW. (2011). Additional evidence for the reliability and validity of the Student Risk Screening Scale at the high school level: A replication and extension. Journal of Emotional and Behavioral Disorders, 21, 97–115.
20.
LaneK. L.OakesW. P.HarrisP. J.MenziesH. M.CoxM.LambertW. (2012). Initial evidence for the reliability and validity of the Student Risk Screening Scale for Internalizing and Externalizing Behaviors at the elementary level. Behavioral Disorders, 37, 99–122.
21.
LaneK. L.OakesW. P.LuskM. E.CantwellE. D.SchatschneiderC. (2016). Screening for intensive intervention needs in secondary schools: Directions for the future. Journal of Emotional and Behavioral Disorders, 24, 159–172.
22.
LaneK. L.OakesW. P.SwoggerE. D.SchatschneiderC.MenziesH. M.SanchezJ. (2015). Student Risk Screening Scale for Internalizing and Externalizing Behaviors: Preliminary cut scores to support data-informed decision making. Behavioral Disorders, 40, 159–170.
23.
LittleR. J. A.RubinD. B. (2002). Statistical analysis with missing data (2nd ed.). Hoboken, NJ: John Wiley.
24.
McDermottP. A.WeissR. V. (1995). A normative typology of healthy, subclinical, and clinical behavior styles among American children and adolescents. Psychological Assessment, 5, 162–170.
25.
MorrisR. J.ShahK.MorrisY. P. (2002). Internalizing behavior disorders. In LaneK. L.GreshamF. M.O’ShaughnessyT. E. (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 223–241). Boston, MA: Allyn & Bacon.
MurakiE. (1997). A generalized partial credit model. In van der LindenW. J.HambletonR. K. (Eds.), Handbook of modern item response theory (pp. 153–164). New York, NY: Springer.
28.
MurrayA. L.BoothT.MolenaarD. (2016). When middle really means “top” or “bottom”: An analysis of the 16PF5 using Bock’s nominal response model. Journal of Personality Assessment, 98, 319–331.
29.
PrestonK. S. J. (2014, April). Advanced topics in IRT: Evaluating the effectiveness of each response option with the nominal response model. PowerPoint presentation at the 94th Annual Convention of the Western Psychological Association, Portland, OR.
30.
PrestonK. S. J.ReiseS. P. (2014). Estimating the nominal response model under nonnormal conditions. Educational and Psychological Measurement, 74, 377–399.
31.
PrestonK. S. J.ReiseS. P. (2015). Detecting faulty within-item category functioning with the nominal response model. In ReiseS. P.RevickiD. (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 386–405). New York, NY: Routledge.
32.
PrestonK. S. J.ReiseS. P.CaiL.HaysR. D. (2011). Using the nominal response model to evaluate response category discrimination in the PROMIS emotional distress item pools. Educational and Psychological Measurement, 71, 523–550.
33.
ReddyL. A.NewmanE.De ThomasC. A.ChunV. (2009). Effectiveness of school-based prevention and intervention programs for children and adolescents with emotional disturbance: A meta-analysis. Journal of School Psychology, 47, 77–99.
34.
RenshawT. L.LongA. C. J.CookC. R. (2014). Assessing adolescents’ positive psychological functioning at school: Development and validation of the Student Subjective Wellbeing Questionnaire. School Psychology Quarterly, 30, 534–552.
35.
RStudio Team. (2015). RStudio: Integrated development for R. Boston, MA: RStudio, Inc. Available from http://www.rstudio.com/
36.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34(4, Pt. 2), 1–100.
37.
SamejimaF. (1996). Evaluation of mathematical models for ordered polychotomous responses. Psychometrika, 23, 17–35.
38.
SchatschneiderC.LaneK. L.OakesW.KalbergJ. (2014). The Student Risk Screening Scale: Exploring dimensionality and differential item functioning. Educational Assessment Journal, 19, 185–203.
39.
SchwarzG. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
40.
SeversonH. H.WalkerH. M.Hope-DoolittleJ.KratochwillT. R.GreshamF. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193–223.
41.
SuldoS. M.ShafferE. J. (2008). Looking beyond psychopathology: The dual-factor model of mental health in youth. School Psychology Review, 37, 52–68.
42.
ThissenD.SteinbergL. (1986). A taxonomy of item response models. Psychometrika, 49, 501–519.
43.
ThissenD.SteinbergL.FitzpatrickA. R. (1989). Multiple-choice models: The distractors are also part of the item. Journal of Educational Measurement, 26, 161–176.
44.
WagnerM.KutashK.DuchnowskiA. J.EpsteinM. H.SumiW. C. (2005). The children and youth we serve: A national picture of the characteristics of students with emotional disturbances receiving special education. Journal of Emotional and Behavioral Disorders, 13, 79–96.
45.
WaldA. (1945). Sequential tests of statistical hypotheses. The Annuals of Mathematical Statistics, 16, 117–186. doi:10.1214/aoms/1177731118
46.
YoungE. L.CaldarellaP.RichardsonM. J.YoungK. R. (2012). Positive behavior support in secondary schools: A practical guide. New York, NY: Guilford Press.
