Psychometric Evaluation of the Affective Reactivity Index Among Children and Adolescents in China: A Multi-Method Assessment Approach

Abstract

The Affective Reactivity Index (ARI) is one of the most studied scales for assessing youth irritability, but little is known about its measurement performance in community populations. This study applied item response theory (IRT), network analysis, and classical test theory (CTT) to examine the psychometric properties of the ARI in a sample of n = 395 community-based children (M_age = 13.44, SD = 2.51) and n = 403 parents. In this sample, the ARI demonstrated good reliability, as well as convergent and concurrent validity. The one-factor structure was supported by both confirmatory factor analysis (CFA) and network analysis. IRT analysis revealed that the ARI effectively distinguished between various levels of irritability within the community population. Network analysis identified “Loses temper easily,”“Gets angry frequently,” and “Often loses temper” are central aspects of irritability. The findings support the ARI as a brief, reliable, and valid instrument to assess irritability in community children and adolescents.

Keywords

irritability Affective Reactivity Index psychometric properties item response theory network analysis

Get full access to this article

View all access options for this article.

References

Achenbach

Rescorla

. (2001). Manual for the ASEBA school-age forms and profiles: An integrated system of multi-informant assessment. Research Center for Children, Youth and Families, University of Vermont.

Aebi

Barra

Bessler

Steinhausen

H. C.

Walitza

Plattner

. (2016). Oppositional defiant disorder dimensions and subtypes among detained male adolescent offenders. Journal of Child Psychology and Psychiatry, 57(6), 729–736. https://doi.org/10.1111/jcpp.12473

Aebi

Plattner

Metzke

C. W.

Bessler

Steinhausen

H. C

. (2013). Parent- and self-reported dimensions of oppositionality in youth: Construct validity, concurrent validity, and the prediction of criminal outcomes in adulthood. Journal of Child Psychology and Psychiatry, 54(9), 941–949. https://doi.org/10.1111/jcpp.12039

Althoff

R. R.

Kuny-Slock

A. V.

Verhulst

F. C.

Hudziak

J. J.

van der Ende

. (2014). Classes of oppositional-defiant behavior: Concurrent and predictive validity. Journal of Child Psychology and Psychiatry, 55(10), 1162–1171. https://doi.org/10.1111/jcpp.12233

Ambrosini

P. J.

Bennett

D. S.

Elia

. (2013). Attention deficit hyperactivity disorder characteristics: II. Clinical correlates of irritable mood. Journal of Affective Disorders, 145, 70–76. https://doi.org/10.1016/j.jad.2012.07.014

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Baker

F. B

. (2001). The basics of item response theory (2nd ed.). ERIC Clearinghouse on Assessment and Evaluation. https://files.eric.ed.gov/fulltext/ED458219.pdf

Beauducel

Herzberg

P. Y

. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13(2), 186–203. https://doi.org/10.1207/s15328007sem1302_2

Bock

R. D.

Aitkin

. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443–459. https://doi.org/10.1007/bf02293801

10.

Borsboom

Cramer

A. O. J

. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9, 91–121. https://doi.org/10.1146/annurev-clinpsy-050212-185608

11.

Bringmann

L. F.

Elmer

Epskamp

Krause

R. W.

Schoch

Wichers

Wigman

J. T. W.

Snippe

. (2019). What do centrality measures measure in psychological networks? Journal of Abnormal Psychology, 128(8), 892–903. https://doi.org/10.1037/abn0000446

12.

Brotman

M. A.

Kircanski

Leibenluft

. (2017). Irritability in children and adolescents. Annual Review of Clinical Psychology, 13(1), 317–341. https://doi.org/10.1146/annurev-clinpsy-032816-044941

13.

Brotman

M. A.

Schmajuk

Rich

B. A.

Dickstein

D. P.

Guyer

A. E.

Costello

E. J.

Egger

H. L.

Angold

Pine

D. S.

Leibenluft

. (2006). Prevalence, clinical correlates, and longitudinal course of severe mood dysregulation in children. Biological Psychiatry, 60(9), 991–997. https://doi.org/10.1016/j.biopsych.2006.08.042

14.

Burger

Isvoranu

A.-M.

Lunansky

Haslbeck

J. M. B.

Epskamp

Hoekstra

R. H. A.

Fried

E. I.

Borsboom

Blanken

T. F

. (2023). Reporting standards for psychological network analyses in cross-sectional data. Psychological Methods, 28, 806–824. https://doi.org/10.1037/met0000471

15.

Burke

J. D

. (2012). An affective dimension within oppositional defiant disorder symptoms among boys: Personality and psychopathology outcomes into early adulthood. Journal of Child Psychology and Psychiatry, 53(11), 1176–1183. https://doi.org/10.1111/j.1469-7610.2012.02598.x

16.

Burke

J. D.

Boylan

Rowe

Duku

Stepp

S. D.

Hipwell

A. E.

Waldman

I. D

. (2014). Identifying the irritability dimension of ODD: Application of a modified bifactor model across five large community samples of children. Journal of Abnormal Psychology, 123(4), 841–851. https://doi.org/10.1037/a0037898

17.

Burke

J. D.

Hipwell

A. E.

Loeber

. (2010). Dimensions of oppositional defiant disorder as predictors of depression and conduct disorder in preadolescent girls. Journal of the American Academy of Child and Adolescent Psychiatry, 49(5), 484–492. https://doi.org/10.1016/j.jaac.2010.01.016

18.

Burke

J. D.

Johnston

O. G.

Butler

E. J

. (2021). The irritable and oppositional dimensions of oppositional defiant disorder integral factors in the explanation of affective and behavioral psychopathology. Child and Adolescent Psychiatric Clinics of North America, 30(3), 637–647. https://doi.org/10.1016/j.chc.2021.04.012

19.

Caprara

G. V.

Paciello

Gerbino

Cugini

. (2007). Individual differences conducive to aggression and violence: Trajectories and correlates of irritability and hostile rumination through adolescence. Aggressive Behavior, 33(4), 359–374. https://doi.org/10.1002/ab.20192

20.

Caprara

G. V.

Renzi

Daugello

Dimperio

Rielli

Travaglia

. (1986). Interpolating physical exercise between instigation to aggress and aggression: The role of irritability and emotional susceptibility. Aggressive Behavior, 12(2), 83–91. https://doi.org/10.1002/1098-2337(1986)12:2%3C83::AID-AB2480120202%3E3.0.CO;2-S

21.

Cheung

G. W.

Rensvold

R. B

. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/s15328007sem0902_5

22.

Cohen

. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155

23.

Copeland

W. E.

Shanahan

Egger

Angold

Costello

E. J

. (2014). Adult diagnostic and functional outcomes of DSM-5 disruptive mood dysregulation disorder. American Journal of Psychiatry, 171(6), 668–674. https://doi.org/10.1176/appi.ajp.2014.13091213

24.

DeSousa

D. A.

Stringaris

Leibenluft

Koller

S. H.

Manfro

G. G.

Salum

G. A

. (2013). Cross-cultural adaptation and preliminary psychometric properties of the Affective Reactivity Index in Brazilian youth: Implications for DSM-5 measured irritability. Trends in Psychiatry and Psychotherapy, 35(3), 171–180. https://doi.org/10.1590/S2237-60892013000300004

25.

Dougherty

L. R.

Galano

M. M.

Chad-Friedman

Olino

T. M.

Bufferd

S. J.

Klein

D. N

. (2021). Using item response theory to compare irritability measures in early adolescent and childhood samples. Assessment, 28(3), 918–927. https://doi.org/10.1177/1073191120936363

26.

Dougherty

L. R.

Smith

V. C.

Bufferd

S. J.

Stringaris

Leibenluft

Carlson

G. A.

Klein

D. N

. (2013). Preschool irritability: Longitudinal associations with psychiatric disorders at age 6 and parental psychopathology. Journal of the American Academy of Child and Adolescent Psychiatry, 52(12), 1304–1313. https://doi.org/10.1016/j.jaac.2013.09.007

27.

Epskamp

Borsboom

Fried

E. I

. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212. https://doi.org/10.3758/s13428-017-0862-1

28.

Epskamp

Cramer

A. O. J.

Waldorp

L. J.

Schmittmann

V. D.

Borsboom

. (2012). Qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software, 48(4), 1–18. https://doi.org/10.18637/jss.v048.i04

29.

Epskamp

Maris

Waldorp

L. J.

Borsboom

. (2018). Network psychometrics. In Irwing

Hughes

Booth

(Eds.), The Wiley handbook of psychometric testing. 2 volume set: A multidisciplinary reference on survey, scale and test development (pp. 953–986). Wiley. https://doi.org/10.1002/9781118489772.ch30

30.

Epskamp

Rhemtulla

Borsboom

. (2017). Generalized network psychometrics: Combining network and latent variable models. Psychometrika, 82(4), 904–927. https://doi.org/10.1007/s11336-017-9557-x

31.

Evans

I. M.

Heriot

S. A.

Friedman

A. G

. (2002). A behavioural pattern of irritability, hostility and inhibited empathy in children. Clinical Child Psychology and Psychiatry, 7, 211–224. https://doi.org/10.1177/1359104502007002008

32.

Evans

S. C.

Abel

M. R.

Doyle

R. L.

Skov

Harmon

S. L

. (2021). Measurement and correlates of irritability in clinically referred youth: Further examination of the Affective Reactivity Index. Journal of Affective Disorders, 283, 420–429. https://doi.org/10.1016/j.jad.2020.11.002

33.

Evans

S. C.

Burke

J. D.

Roberts

M. C.

Fite

P. J.

Lochman

J. E.

de la Pena

F. R.

Reed

G. M

. (2017). Irritability in child and adolescent psychopathology: An integrative review for ICD-11. Clinical Psychology Review, 53, 29–45. https://doi.org/10.1016/j.cpr.2017.01.004

34.

Ezpeleta

Granero

de la Osa

Penelo

Domenech

J. M

. (2012). Dimensions of oppositional defiant disorder in 3-year-old preschoolers. Journal of Child Psychology and Psychiatry, 53(11), 1128–1138. https://doi.org/10.1111/j.1469-7610.2012.02545.x

35.

Ezpeleta

Granero

de la Osa

Trepat

Domenech

J. M

. (2016). Trajectories of oppositional defiant disorder irritability symptoms in preschool children. Journal of Abnormal Child Psychology, 44(1), 115–128. https://doi.org/10.1007/s10802-015-9972-3

36.

Ezpeleta

Penelo

de la Osa

Navarro

J. B.

Trepat

. (2020). How the Affective Reactivity Index (ARI) works for teachers as informants. Journal of Affective Disorders, 261, 40–48. https://doi.org/10.1016/j.jad.2019.09.080

37.

Fortunato

. (2010). Community detection in graphs. Physics Reports, 486, 75–174. https://doi.org/10.1016/j.physrep.2009.11.002

38.

Frazier

E. A.

Liu

R. T.

Massing-Schaffer

Hunt

Wolff

Spirito

. (2016). Adolescent but not parent report of irritability is related to suicidal ideation in psychiatrically hospitalized adolescents. Archives of Suicide Research, 20(2), 280–289. https://doi.org/10.1080/13811118.2015.1004497

39.

Golino

H. F.

Demetriou

. (2017). Estimating the dimensionality of intelligence like data using exploratory graph analysis. Intelligence, 62, 54–70. https://doi.org/10.1016/j.intell.2017.02.007

40.

Golino

H. F.

Epskamp

. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PLOS ONE, 12(6), Article e0174035. https://doi.org/10.1371/journal.pone.0174035

41.

Golino

H. F.

Shi

D. J.

Christensen

A. P.

Garrido

L. E.

Nieto

M. D.

Sadana

Thiyagarajan

J. A.

Martinez-Molina

. (2020). Investigating the performance of Exploratory Graph Analysis and traditional techniques to identify the number of latent factors: A simulation and tutorial. Psychological Methods, 25(3), 292–320. https://doi.org/10.1037/met0000255

42.

Gomez

Stavropoulos

Gomez

Brown

Watson

. (2022). Network analyses of Oppositional Defiant Disorder (ODD) symptoms in children. BMC Psychiatry, 22(1), Article 263. https://doi.org/10.1186/s12888-022-03892-5

43.

Goodman

. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry, 40(11), 1337–1345. https://doi.org/10.1097/00004583-200111000-00015

44.

L. t.

Bentler

P. M

. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

45.

Hukkelberg

S. S.

Ogden

. (2018). Dimensionality of oppositional defiant disorder. Child and Adolescent Mental Health, 23(2), 121–129. https://doi.org/10.1111/camh.12248

46.

Kolko

D. J.

Pardini

D. A

. (2010). ODD dimensions, ADHD, and callous-unemotional traits as predictors of treatment response in children with disruptive behavior disorders. Journal of Abnormal Psychology, 119(4), 713–725. https://doi.org/10.1037/a0020910

47.

Leadbeater

B. J.

Homel

. (2015). Irritable and defiant sub-dimensions of ODD: Their stability and prediction of internalizing symptoms and conduct problems from adolescence to young adulthood. Journal of Abnormal Child Psychology, 43(3), 407–421. https://doi.org/10.1007/s10802-014-9908-3

48.

Leibenluft

Kircanski

. (2021). Chronic irritability in youth: A reprise on challenges and opportunities toward meeting unmet clinical needs. Child and Adolescent Psychiatric Clinics of North America, 30(3), 667–683. https://doi.org/10.1016/j.chc.2021.04.014

49.

Leibenluft

Stoddard

. (2013). The developmental psychopathology of irritability. Development and Psychopathology, 25(4), 1473–1487. https://doi.org/10.1017/s0954579413000722

50.

Mallidi

Meza-Cervera

Kircanski

Stringaris

Brotman

M. A.

Pine

D. S.

Leibenluft

Linke

J. O

. (2023). Robust caregiver-youth discrepancies in irritability ratings on the affective reactivity index: An investigation of its origins. Journal of Affective Disorders, 332, 185–193. https://doi.org/10.1016/j.jad.2023.03.091

51.

McDonald

R. P

. (1999). Test theory: A unified treatment. Psychology Press. https://doi.org/10.4324/9781410601087

52.

Meade

A. W.

Johnson

E. C.

Braddy

P. W

. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. https://doi.org/10.1037/0021-9010.93.3.568

53.

Meredith

. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. https://doi.org/10.1007/bf02294825

54.

Mikita

Stringaris

. (2013). Mood dysregulation. European Child & Adolescent Psychiatry, 22, S11–S16. https://doi.org/10.1007/s00787-012-0355-9

55.

Mulraney

M. A.

Melvin

G. A.

Tonge

B. J

. (2014). Psychometric properties of the affective reactivity index in Australian adults and adolescents. Psychological Assessment, 26(1), 148–155. https://doi.org/10.1037/a0034891

56.

Orri

Galera

Turecki

Boivin

Tremblay

R. E.

Geoffroy

M. C.

Cote

S. M

. (2019). Pathways of association between childhood irritability and adolescent suicidality. Journal of the American Academy of Child and Adolescent Psychiatry, 58(1), 99–107. https://doi.org/10.1016/j.jaac.2018.06.034

57.

Pan

P.-Y.

Yeh

C.-B

. (2018). Irritability and maladaptation among children: The utility of Chinese versions of the Affective Reactivity Index and Aberrant Behavior Checklist-Irritability Subscale. Journal of Child and Adolescent Psychopharmacology, 29(3), 213–219. https://doi.org/10.1089/cap.2018.0070

58.

Pickles

Aglan

Collishaw

Messer

Rutter

Maughan

. (2010). Predictors of suicidality across the life span: The Isle of Wight study. Psychological Medicine, 40(9), 1453–1466. https://doi.org/10.1017/s0033291709991905

59.

Pituch

K. A.

Stevens

J. P

. (2016). Applied multivariate statistics for the social sciences: Analyses with SAS and IBM’s SPSS (6th ed.). Routledge. https://doi.org/10.4324/9781315814919

60.

Reise

S. P.

Ainsworth

A. T.

Haviland

M. G

. (2005). Item response theory: Fundamentals, applications, and promise in psychological research. Current Directions in Psychological Science, 14(2), 95–101. https://doi.org/10.1111/j.0963-7214.2005.00342.x

61.

Reise

S. P.

Rodriguez

. (2016). Item response theory and the measurement of psychiatric constructs: Some empirical and conceptual issues and challenges. Psychological Medicine, 46(10), 2025–2039. https://doi.org/10.1017/s0033291716000520

62.

Reise

S. P.

Waller

N. G

. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 27–48. https://doi.org/10.1146/annurev.clinpsy.032408.153553

63.

Riglin

Eyre

Thapar

A. K.

Stringaris

Leibenluft

Pine

D. S.

Tilling

Smith

G. D.

O’Donovan

M. C.

Thapar

. (2019). Identifying novel types of irritability using a developmental genetic approach. American Journal of Psychiatry, 176(8), 635–642. https://doi.org/10.1176/appi.ajp.2019.18101134

64.

Rowe

Costello

E. J.

Angold

Copeland

W. E.

Maughan

. (2010). Developmental pathways in oppositional defiant disorder and conduct disorder. Journal of Abnormal Psychology, 119(4), 726–738. https://doi.org/10.1037/a0020798

65.

Roy

A. K.

Comer

J. S

. (2020). Advances in the conceptualization, assessment, and treatment of pediatric irritability. Behavior Therapy, 51(2), 207–210. https://doi.org/10.1016/j.beth.2019.10.010

66.

Samejima

. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17, Part 2). Psychometric Society.

67.

Smith

T. E.

Lee

C. A.

Martel

M. M.

Axelrad

M. E

. (2017). ODD symptom network during preschool. Journal of Abnormal Child Psychology, 45(4), 743–748. https://doi.org/10.1007/s10802-016-0196-y

68.

Stoddard

Stringaris

Brotman

M. A.

Montville

Pine

D. S.

Leibenluft

. (2014). Irritability in child and adolescent anxiety disorders. Depression and Anxiety, 31(7), 566–573. https://doi.org/10.1002/da.22151

69.

Stringaris

. (2011). Irritability in children and adolescents: A challenge for DSM-5. European Child & Adolescent Psychiatry, 20(2), 61–66. https://doi.org/10.1007/s00787-010-0150-4

70.

Stringaris

Cohen

Pine

D. S.

Leibenluft

. (2009). Adult outcomes of youth irritability: A 20-year prospective community-based study. American Journal of Psychiatry, 166(9), 1048–1054. https://doi.org/10.1176/appi.ajp.2009.08121849

71.

Stringaris

Goodman

. (2009a). Longitudinal outcome of youth oppositionality: Irritable, headstrong, and hurtful behaviors have distinctive predictions. Journal of the American Academy of Child and Adolescent Psychiatry, 48(4), 404–412. https://doi.org/10.1097/CHI.0b013e3181984f30

72.

Stringaris

Goodman

. (2009b). Three dimensions of oppositionality in youth. Journal of Child Psychology and Psychiatry, 50(3), 216–223. https://doi.org/10.1111/j.1469-7610.2008.01989.x

73.

Stringaris

Goodman

Ferdinando

Razdan

Muhrer

Leibenluft

Brotman

M. A

. (2012). The Affective Reactivity Index: A concise irritability scale for clinical and research settings. Journal of Child Psychology and Psychiatry, 53(11), 1109–1117. https://doi.org/10.1111/j.1469-7610.2012.02561.x

74.

Stringaris

Vidal-Ribas

. (2019). Probing the irritability-suicidality nexus. Journal of the American Academy of Child and Adolescent Psychiatry, 58(1), 18–19. https://doi.org/10.1016/j.jaac.2018.08.014

75.

Sugaya

L. S.

Kircanski

Stringaris

Polanczyk

G. V.

Leibenluft

. (2022). Validation of an irritability measure in preschoolers in school-based and clinical Brazilian samples. European Child & Adolescent Psychiatry, 31(4), 577–587. https://doi.org/10.1007/s00787-020-01701-6

76.

Thomas

M. L

. (2011). The value of item response theory in clinical assessment: A review. Assessment, 18(3), 291–307. https://doi.org/10.1177/1073191110374797

77.

Toland

M. D

. (2015). Practical guide to conducting an item response theory analysis. Journal of Early Adolescence, 35(3), 120–151. https://doi.org/10.1177/0272431615572521

78.

Toohey

M. J.

DiGiuseppe

. (2017). Defining and measuring irritability: Construct clarification and differentiation. Clinical Psychology Review, 53, 93–108. https://doi.org/10.1016/j.cpr.2017.01.009

79.

Tseng

W. L.

Moroney

Machlin

Roberson-Nay

Hettema

J. M.

Carney

Stoddard

Towbin

K. A.

Pine

D. S.

Leibenluft

Brotman

M. A

. (2017). Test-retest reliability and validity of a frustration paradigm and irritability measures. Journal of Affective Disorders, 212, 38–45. https://doi.org/10.1016/j.jad.2017.01.024

80.

van Borkulo

C. D.

van Bork

Boschloo

Kossakowski

J. J.

Tio

Schoevers

R. A.

Borsboom

Waldorp

L. J

. (2022). Comparing network structures on three aspects: A permutation test. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000476

81.

van Dijk

Claassen

Suwartono

van der Veld

W. M.

van der Heijden

P. T.

Hendriks

M. P. H

. (2017). Evaluating WAIS-IV structure through a different psychometric lens: Structural causal model discovery as an alternative to confirmatory factor analysis. Clinical Neuropsychologist, 31, 1141–1154. https://doi.org/10.1080/13854046.2017.1352029

82.

Vidal-Ribas

Brotman

M. A.

Valdivieso

Leibenluft

Stringaris

. (2016). The status of irritability in psychiatry: A conceptual and quantitative review. Journal of the American Academy of Child and Adolescent Psychiatry, 55(7), 556–570. https://doi.org/10.1016/j.jaac.2016.04.014

83.

Whelan

Y. M.

Stringaris

Maughan

Barker

E. D

. (2013). Developmental continuity of oppositional defiant disorder subdimensions at ages 8, 10, and 13 years and their distinct psychiatric outcomes at age 16 years. Journal of the American Academy of Child and Adolescent Psychiatry, 52(9), 961–969. https://doi.org/10.1016/j.jaac.2013.06.013

84.

Wiggins

J. L.

Briggs-Gowan

M. J.

Estabrook

Brotman

M. A.

Pine

D. S.

Leibenluft

Wakschlag

L. S

. (2018). Identifying clinically significant irritability in early childhood. Journal of the American Academy of Child and Adolescent Psychiatry, 57(3), 191–199. https://doi.org/10.1016/j.jaac.2017.12.008

85.

Wilson

M. K.

Cornacchio

Brotman

M. A.

Comer

J. S

. (2022). Measuring irritability in early childhood: A psychometric evaluation of the Affective Reactivity Index in a clinical sample of 3- to 8-year-old children. Assessment, 29(7), 1473–1481. https://doi.org/10.1177/10731911211020078

86.

C. Y

. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes [Doctoral dissertation, University of California, Los Angeles].