The role of measuring functional impairment holds an important place in research, clinical practice, and service provision for children and adolescents. Responding to the growing need to measure serious emotional disturbances at the local, state, and national level, the Columbia Impairment Scale (CIS) was developed in the early 1990s and has remained one of the several popular scales for assessing functional impairment. However, despite the growing popularity of the instrument in research and practice, only a few studies to date have specifically examined the psychometric properties of the CIS. In this article, we describe the results of the first item response theory analysis of the CIS utilizing nationally representative data from the Medical Expenditure Panel Survey (N = 69,966). The results of our analysis lend support to the essential unidimensionality of the CIS and demonstrate that the scale is most reliable for those who exhibit high levels of functional impairment. Given the psychometric properties of the scale identified by our analysis, we contend that the CIS is a viable measure in the ongoing efforts to establish a national epidemiologic surveillance system to track the prevalence and impact of serious emotional disturbances in children and adolescents.
Agency for Healthcare Research and Quality. (2017). Medical Expenditure Panel Survey. Retrieved fromhttps://meps.ahrq.gov/mepsweb/
2.
BarchD. M.GotlibI. H.BilderR. M.PineD. S.SmollerJ. W.BrownC. H.…FarberG. K. (2016). Common measures for national institute of mental health funded research. Biological Psychiatry, 79, e91–e96.
3.
BirdH. R.AndrewsH.Schwab-StoneM.GoodmanS.DulcanM.RichtersJ.…GouldM. S. (1996). Global measures of impairment for epidemiologic and clinical use with children and adolescents. International Journal of Methods in Psychiatric Research, 6, 295–307.
4.
BirdH. R.CaninoG. J.DaviesM.ZhangH.RamirezR.LaheyB. B. (2001). Prevalence and correlates of antisocial behaviors among three ethnic groups. Journal of Abnormal Child Psychology, 29, 465–478.
5.
BirdH. R.GouldM. S. (1995). The use of diagnostic instruments and global measures of functioning in child psychiatry epidemiological studies. In VerhulstF. C.KootH. M. (Eds.), The epidemiology of child and adolescent psychopathology (pp. 86–103). New York, NY: Oxford University Press.
6.
BirdH. R.ShafferD.FisherP.GouldM. S. (1993). The Columbia Impairment Scale (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research, 3, 167–176.
7.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.
8.
BrannanA. M.BrashearsF.GyamfiP.ManteuffelB. (2012). Implementation and development of federally-funded systems of care over time. American Journal of Community Psychology, 49, 467–482.
9.
BrennanE. M.NygrenP.StephensR. L.CroskeyA. (2016). Predicting positive education outcomes for emerging adults in mental health systems of care. The Journal of Behavioral Health Services & Research, 43, 564–581.
10.
BurnsB. J.CostelloE. J.AngoldA.TweedD.StanglD.FarmerE. M.ErkanliA. (1995). Children’s mental health service use across service sectors. Health Affairs, 14, 147–159.
CaninoG.CostelloE. J.AngoldA. (1999). Assessing functional impairment and social adaptation for child mental health services research: A review of measures. Mental Health Services Research, 1, 93–108.
13.
CaninoG. J.FisherP. W.AlegriaM.BirdH. R. (2013). Assessing child impairment in functioning in different contexts: Implications for use of services and the classification of psychiatric disorders. Open Journal of Medical Psychology, 2, 29–34.
14.
CattellR. B. (1966). The meaning and strategic use of factor analysis. In CattellR. B. (Ed.), Handbook of multivariate experimental psychology (pp. 174–243). Chicago, IL: Rand McNally.
15.
ChalmersR. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29.
16.
ChildD. (2006). The essentials of factor analysis (3rd ed.). New York, NY: Continuum International.
17.
EdelenM. O.ReeveB. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16, 5–18.
18.
EmbretsonS. E.ReiseS. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
19.
EzpeletaL.GraneroR.OsaN. D. L.DoménechJ. M.BonilloA. (2006). Assessment of functional impairment in Spanish children. Applied Psychology, 55, 130–143.
20.
GardnerW.LucasA.KolkoD. J.CampoJ. V. (2007). Comparison of the PSC-17 and alternative mental health screens in an at-risk primary care sample. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 611–618.
21.
GarlandA. F.LauA. S.YehM.McCabeK. M.HoughR. L.LandsverkJ. A. (2005). Racial and ethnic differences in utilization of mental health services among high-risk youths. American Journal of Psychiatry, 162, 1336–1343.
22.
GoldL. H. (2014). DSM-5 and the assessment of functioning: The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0). Journal of the American Academy of Psychiatry and the Law Online, 42, 173–181.
23.
GordonM.AntshelK.FaraoneS.BarkleyR.LewandowskiL.HudziakJ. J.…CunninghamC. (2006). Symptoms versus impairment: The case for respecting DSM-IV’s Criterion D. Journal of Attention Disorders, 9, 465–475.
24.
GoudieA.HavercampS.JamiesonB.SahrT. (2013). Assessing functional impairment in siblings living with children with disability. Pediatrics, 132, e476–e483.
25.
GouldM. S.MunfakhJ. L. H.LubellK.KleinmanM.ParkerS. (2002). Seeking help from the internet during adolescence. Journal of the American Academy of Child & Adolescent Psychiatry, 41, 1182–1189.
26.
HammondS. (2006). Using psychometric tests. In BreakwellG. M.HammondS.Fife-SchawC.SmithJ. A. (Eds.), Research methods in psychology (3rd ed., pp. 182–209). Thousand Oaks, CA: Sage.
27.
HarrisE. S.CanningR. D.KelleherK. J. (1996). A comparison of measures of adjustment, symptoms, and impairment among children with chronic medical conditions. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1025–1032.
28.
HaysR. D.MoralesL. S.ReiseS. P. (2000). Item response theory and health outcomes measurement in the 21st century. Medical Care, 38, II28–II42.
29.
HoffmanL. (2015). Longitudinal analysis: Modeling within-person fluctuation and change. New York, NY: Routledge; Taylor & Francis.
30.
HoldenE. W.SantiagoR. L.ManteuffelB. A.StephensR. L.BrannanA. M.SolerR.…ZaroS. (2003). Systems of care demonstration projects: Innovation, evaluation, and sustainability. In PumariegaA. J.WintersN. C. (Eds.), The handbook of child and adolescent systems of care: The new community psychiatry (pp. 432–458). San Francisco: Jossey-Bass.
31.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.
32.
KaiserH. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.
33.
KleinD. N.DoughertyL. R.OlinoT. M. (2005). Toward guidelines for evidence-based assessment of depression in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 412–432.
34.
KlineR. B. (2015). Principles and practice of structural equation modeling (4th ed.). New York, NY: Guilford.
35.
KlomekA. B.KleinmanM.AltschulerE.MarroccoF.AmakawaL.GouldM. S. (2011). High school bullying as a risk for later depression and suicidality. Suicide and Life-Threatening Behavior, 41, 501–516.
36.
KolenM. J.BrennanR. L. (2014). Test equating, scaling, and linking: Methods and practices. Berlin, Germany: Springer Science & Business Media.
37.
KooT. K.LiM. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163.
38.
KramerT. L.PhillipsS. D.HargisM. B.MillerT. L.BurnsB. J.RobbinsJ. M. (2004). Disagreement between parent and adolescent reports of functional impairment. Journal of Child Psychology and Psychiatry, 45, 248–259.
39.
LambertM. C.JanuaryS. A. A.EpsteinM. H.SpoonerM.GebreselassieT.StephensR. L. (2015). Convergent validity of the behavioral and emotional rating scale for youth in community mental health settings. Journal of Child and Family Studies, 24, 3827–3832.
40.
ManteuffelB.StephensR. L.SantiagoR. (2002). Overview of the national evaluation of the comprehensive community mental health services for children and their families program and summary of current findings. Children’s Services: Social Policy, Research, and Practice, 5, 3–20.
41.
ManteuffelB.StephensR. L.SondheimerD. L.FisherS. K. (2008). Characteristics, service experiences, and outcomes of transition-aged youth in systems of care: Programmatic and policy implications. The Journal of Behavioral Health Services & Research, 35, 469–487.
42.
McMahonR. J.FrickP. J. (2005). Evidence-based assessment of conduct problems in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 477–505.
43.
MerikangasK. R.BrometE. J.DrussB. G. (2017). Future surveillance of mental disorders in the United States: Count people, not disorders. JAMA Psychiatry, 74, 431–432.
44.
MillerM.HinshawS. P. (2010). Does childhood executive function predict adolescent functional outcomes in girls with ADHD?Journal of Abnormal Child Psychology, 38, 315–326.
45.
National Academies of Sciences, Engineering, and Medicine. (2016). Measuring serious emotional disturbance in children: Workshop Summary. Washington, DC: National Academies Press.
46.
OlfsonM.DrussB. G.MarcusS. C. (2015). Trends in mental health care among children and adolescents. New England Journal of Medicine, 372, 2029–2038.
47.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.
48.
OrlandoM.ThissenD. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289–298.
49.
NguyenT. H.HanH. R.KimM. T.ChanK. S. (2014). An introduction to item response theory for patient-reported outcome measurement. The Patient: Patient-Centered Outcomes Research, 7, 23–35.
50.
PainterK. (2012). Outcomes for youth with severe emotional disturbance: A repeated measures longitudinal study of a wraparound approach of service delivery in systems of care. Child & Youth Care Forum, 41, 407–425.
51.
R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved fromhttp://R-project.org/
RingeisenH.StambaughL.BoseJ.CasanuevaC.HeddenS.AvenevoliS.…CopelandW. E. (2017). Measurement of childhood serious emotional disturbance: State of the science and issues for consideration. Journal of Emotional and Behavioral Disorders, 25, 195–210.
54.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded responses. Psychometrika, Monograph Supplement, 34, 100.
55.
SingerJ. B.EackS. M.GreenoC. M. (2011). The Columbia Impairment Scale: Factor analysis using a community mental health sample. Research on Social Work Practice, 21, 458–468.
56.
Slocum-GoriS. L.ZumboB. D.MichalosA. C.DienerE. (2009). A note on the dimensionality of quality of life scales: An illustration with the Satisfaction with Life Scale (SWLS). Social Indicators Research, 92, 489–496.
57.
SnyderF. J.RobertsY. H.CrustoC. A.ConnellC. M.GriffinA.FinleyM. K.…KaufmanJ. S. (2012). Exposure to traumatic events and the behavioral health of children enrolled in an early childhood system of care. Journal of Traumatic Stress, 25, 700–704.
58.
StarinA. C.AtkinsM. S.WehrmannK. C.MehtaT.Hesson-McInnisM. S.Marinez-LoraA.MehlingerR. (2014). Moving science into state child and adolescent mental health systems: Illinois’ evidence-informed practice initiative. Journal of Clinical Child & Adolescent Psychology, 43, 169–178.
59.
SteinhausenH. C.Winkler MetzkeC. (2001). Global measures of impairment in children and adolescents: Results from a Swiss community survey. Australian & New Zealand Journal of Psychiatry, 35, 282–286.
60.
StreinerD. L.NormanG. R.CairneyJ. (2015). Health measurement scales: A practical guide to their development and use. Oxford, England: Oxford University Press, USA.
61.
TolandM. D. (2014). Practical guide to conducting an item response theory analysis. The Journal of Early Adolescence, 34, 120–151.
62.
VishnevskyT.StrompolisM.ReeveC. L.KilmerR. P.CookJ. R. (2012). Using latent growth curve modeling to examine changes in mental health outcomes for children enrolled in a system of care. American Journal of Orthopsychiatry, 82, 121.
63.
WalrathC.MiechR.HoldenE. W.ManteuffelB.SantiagoR.LeafP. (2003). Child functioning in rural and nonrural areas: How does it compare when using the service program site as the level of analysis?. The Journal of Behavioral Health Services & Research, 30, 452–461.
64.
WellsC. S.SubkoviakM. J.SerlinR. C. (2002). The effect of item parameter drift on examinee ability estimates. Applied Psychological Measurement, 26, 77–87.
65.
WintersN. C.CollettB. R.MyersK. M. (2005). Ten-year review of rating scales, VII: Scales assessing functional impairment. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 309–338.
66.
WuP.HovenC. W.BirdH. R.MooreR. E.CohenP.AlegriaM.…NarrowW. E. (1999). Depressive and disruptive disorders and mental health service utilization in children and adolescents. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 1081–1090.
67.
YehM.HoughR. L.McCabeK.LauA.GarlandA. (2004). Parental beliefs about the causes of child problems: Exploring racial/ethnic patterns. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 605–612.
68.
YehM.McCabeK.HoughR. L.DupuisD.HazenA. (2003). Racial/ethnic differences in parental endorsement of barriers to mental health services for youth. Mental Health Services Research, 5, 65–77.
69.
ZanonA.TomassoniR.GarganoM. L.GranaiM. G. (2016). Reliability and validity of the Columbia Impairment Scale (CIS) for adolescents: Survey among an Italian sample in Lazio Region. Epidemiology, Biostatistics and Public Health, 13, e116501–e11650-8.
70.
ZielinskiK.WoodJ. J.RennoP.WhithamS.SterlingL. (2014). Examining the validity of the Columbia impairment scale for assessing level of functioning in youth with autism spectrum disorder. Journal of Child and Adolescent Psychopharmacology, 24, 509–512.
