Restricted accessResearch articleFirst published online 2024-7
Measurement Invariance of the Neurobehavioral Symptom Inventory in Male and Female Million Veteran Program Enrollees Completing the Comprehensive Traumatic Brain Injury Evaluation
This study evaluated measurement invariance across males and females on the Neurobehavioral Symptom Inventory (NSI) in U.S. military veterans enrolled in the VA Million Veteran Program. Participants (N = 17,059; males: n = 15,450; females: n = 1,609) included Veterans who took part in the VA Traumatic Brain Injury (TBI) Screening and Evaluation Program and completed the NSI. Multiple-group confirmatory factor analyses investigated measurement invariance of the NSI 4-factor model. The configural (comparative fit index [CFI] = 0.948, root mean square error of approximation [RMSEA] = 0.060) and metric (CFI = 0.948, RMSEA = 0.058) invariance models showed acceptable fit. There was a minor violation of scalar invariance (Δχ2 = 232.50, p < .001); however, the degree of noninvariance was mild (ΔCFI = −0.002, ). Our results demonstrate measurement invariance across sex, suggesting that the NSI 4-factor model can be used to accurately assess symptoms in males and females following TBI. Findings highlight the importance of considering validity of measurement across study groups to increase confidence that a measure is interpreted similarly by respondents from different subgroups.
BelangerH. G.Powell-CopeG.SpeharA. M.McCranieM.KlancharS. A.Yoash-GantzR.KosasihJ. B.ScholtenJ. (2016). The Veterans Health Administration’s traumatic brain injury clinical reminder screen and evaluation: Practice patterns. Journal of Rehabilitation Research and Development, 53(6), 767–780. https://doi.org/10.1682/JRRD.2015.09.0187
3.
BelangerH. G.UomotoJ. M.VanderploegR. D. (2009). The Veterans Health Administration system of care for mild traumatic brain injury: Costs, benefits, and controversies. The Journal of Head Trauma Rehabilitation, 24(1), 4–13.
4.
BengeJ. F.PastorekN. J.ThorntonG. M. (2009). Postconcussive symptoms in OEF-OIF veterans: Factor structure and impact of posttraumatic stress. Rehabilitation Psychology, 54(3), 270–278. https://doi.org/10.1037/a0016736
5.
BontempoD. E.GrouzetF. M. E.HoferS. M. (2012). Measurement issues in the analysis of within-person change. In NewsomJ.JonesR. N.HoferS. M. (Eds.), Longitudinal data analysis: A practical guide for researchers in aging, health, and social sciences (pp. 97–142). Routledge/Taylor & Francis Group.
6.
BrickellT. A.LippaS. M.FrenchL. M.KennedyJ. E.BailieJ. M.LangeR. T. (2017). Female service members and symptom reporting after combat and non-combat-related mild traumatic brain injury. Journal of Neurotrauma, 34(2), 300–312. https://doi.org/10.1089/neu.2016.4403
7.
CaplanL.IvinsB.PooleJ.VanderploegR.JaffeeM.SchwabK. (2010). The structure of postconcussive symptoms in 3 US military samples. The Journal of Head Trauma Rehabilitation, 25, 447–458. https://doi.org/10.1097/HTR.0b013e3181d5bdbd
8.
ChenF.CurranP. J.BollenK. A.KirbyJ.PaxtonP. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research, 36(4), 462–494.
9.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.
10.
CiceroneK.KalmarK. (1995). Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury. The Journal of Head Trauma Rehabilitation, 10, 1–17. https://doi.org/10.1097/00001199-199506000-00002
11.
Department of Veterans Affairs (2007). Screening and evaluation of possible traumatic brain injury in Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) veterans: VHA Directive 2007–013. Veterans Health Administration.
12.
Department of Veterans Affairs (2010). Screening and evaluation of possible traumatic brain injury in Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) veterans: VHA Directive 2010–012. Veterans Health Administration.
13.
EapenB. C.BowlesA. O.SallJ.LangA. E.HoppesC. W.StoutK. C.KretzmerT.CifuD. X. (2022). The management and rehabilitation of post-acute mild traumatic brain injury. Brain Injury, 36(5), 693–702.
14.
GallagherV.KramerN.AbbottK.AlexanderJ.BreiterH.HerroldA.LindleyT.MjaanesJ.ReillyJ. (2018). The effects of sex differences and hormonal contraception on outcomes after collegiate sports-related concussion. Journal of Neurotrauma, 35(11), 1242–1247.
15.
GrayM.AdamsonM. M.ThompsonR. C.KapphahnK. I.HanS.ChungJ. S.HarrisO. A. (2020). Sex differences in symptom presentation and functional outcomes: A pilot study in a matched sample of veterans with mild TBI. Brain Injury, 34(4), 535–547. https://doi.org/10.1080/02699052.2020.1725979
16.
GazianoJ. M.ConcatoJ.BrophyM.FioreL.PyarajanS.BreelingJ.WhitbourneS.DeenJ.ShannonC.HumphriesD. (2016). Million Veteran Program: A mega-biobank to study genetic influences on health and disease. Journal of Clinical Epidemiology, 70, 214–223.
17.
HaagsmaJ. A.ScholtenA. C.AndriessenT. M.VosP. E.Van BeeckE. F.PolinderS. (2015). Impact of depression and post-traumatic stress disorder on functional outcome and health-related quality of life of patients with mild traumatic brain injury. Journal of Neurotrauma, 32(11), 853–862. https://doi.org/10.1089/neu.2013.3283
18.
HanK.ColarelliS. M.WeedN. C. (2019). Methodological and statistical advances in the consideration of cultural diversity in assessment: A critical review of group classification and measurement invariance testing. Psychological Assessment, 31(12), 1481–1496. https://doi.org/10.1037/pas0000731
19.
HaynesN.GoodwinT. (2023). Literature review of sex differences in mTBI. Military Medicine, 188(5–6), e978–e984.
20.
IversonK. M.HendricksA. M.KimerlingR.KrengelM.MeterkoM.StolzmannK. L.BakerE.PogodaT. K.VasterlingJ. J.LewH. L. (2011). Psychiatric diagnoses and neurobehavioral symptom severity among OEF/OIF VA patients with deployment-related traumatic brain injury: A gender comparison. Womens Health Issues, 21(Suppl. 4), S210–S217. https://doi.org/10.1016/j.whi.2011.04.019
21.
KingP. R.DonnellyK. T.DonnellyJ. P.DunnamM.WarnerG.KittlesonC. J.BradshawC. B.AltM.MeierS. T. (2012). Psychometric study of the Neurobehavioral Symptom Inventory. Journal of Rehabilitation Research and Development, 49(6), 879–888. https://doi.org/10.1682/jrrd.2011.03.0051
22.
LaiM. H. C.LiuY.TseW.-Y. (2022). Adjusting for partial invariance in latent parameter estimation: Comparing forward specification search and approximate invariance methods. Behavior Research Methods, 54(1), 414–434.
23.
LaiM. H. C.RichardsonG. B.MakH. W. (2019). Quantifying the impact of partial measurement invariance in diagnostic research: An application to addiction research. Addictive Behaviors, 94, 50–56. https://doi.org/10.1016/j.addbeh.2018.11.029
24.
LaiM. H. C.ZhangY. (2022). Classification accuracy of multidimensional tests: Quantifying the impact of noninvariance. Structural Equation Modeling: A Multidisciplinary Journal, 29(4), 620–629. https://doi.org/10.1080/10705511.2021.1977936
25.
LangeR. T.FrenchL. M.LippaS. M.BailieJ. M.BrickellT. A. (2020). Posttraumatic stress disorder is a stronger predictor of long-term neurobehavioral outcomes than traumatic brain injury severity. Journal of Traumatic Stress, 33(3), 318–329. https://doi.org/10.1002/jts.22480
26.
LippaS. M.BrickellT. A.BailieJ. M.FrenchL. M.KennedyJ. E.LangeR. T. (2018). Postconcussion symptom reporting after mild traumatic brain injury in female service members: Impact of gender, posttraumatic stress disorder, severity of injury, and associated bodily injuries. Journal of Head Trauma Rehabilitation, 33(2), 101–112. https://doi.org/10.1097/htr.0000000000000353
27.
The Management of Concussion/mTBI Working Group. (2021). VA/DoD clinical practice guidelines for the management of concussion/mild traumatic brain injury (mTBI).
28.
MenattiA. R. R.MelinderM. R. D.WarrenS. L. (2020). Limited prediction of performance validity using embedded validity scales of the Neurobehavioral Symptom Inventory in an mTBI veteran sample. Journal of Head Trauma Rehabilitation, 35(1), E36–E42. https://doi.org/10.1097/htr.0000000000000467
29.
MerrittV. C.JurickS. M.SakamotoM. S.CrockerL. D.SullanM. J.HoffmanS. N.DaveyD. K.JakA. J. (2020). Post-concussive symptom endorsement and symptom attribution following remote mild traumatic brain injury in combat-exposed Veterans: An exploratory study. Journal of Psychiatric Research, 130, 224–230.
30.
MerrittV. C.PadgettC. R.JakA. J. (2019). A systematic review of sex differences in concussion outcome: What do we know?The Clinical Neuropsychologist, 33(6), 1016–1043. https://doi.org/10.1080/13854046.2018.1508616
31.
MeterkoM.BakerE.StolzmannK. L.HendricksA. M.CiceroneK. D.LewH. L. (2012). Psychometric assessment of the Neurobehavioral Symptom Inventory-22: The structure of persistent postconcussive symptoms following deployment-related mild traumatic brain injury among veterans. The Journal of Head Trauma Rehabilitation, 27(1), 55–62. https://doi.org/10.1097/HTR.0b013e318230fb17
32.
MillsapR. E.KwokO. M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9(1), 93–115. https://doi.org/10.1037/1082-989x.9.1.93
NyeC. D.DrasgowF. (2011). Effect size indices for analyses of measurement equivalence: Understanding the practical importance of differences between groups. Journal of Applied Psychology, 96(5), 966–980.
35.
OzturkE. D.Chanfreau-CoffinierC.SakamotoM. S.Delano-WoodL.MerrittV. C. (2022). Characterization of Million Veteran Program (MVP) enrollees with Comprehensive Traumatic Brain Injury Evaluation (CTBIE) data: An analysis of neurobehavioral symptoms. Journal of Psychiatric Research, 145, 230–242.
RaykovT.MarcoulidesG. A. (2018). A course in item response theory and modeling with Stata. Stata Press.
38.
ScholtenJ. D.SayerN. A.VanderploegR. D.BidelspachD. E.CifuD. X. (2012). Analysis of US Veterans Health Administration comprehensive evaluations for traumatic brain injury in Operation Enduring Freedom and Operation Iraqi Freedom veterans. Brain Injury, 26(10), 1177–1184.
39.
SchwabK.TerrioH. P.BrennerL. A.PazdanR. M.McMillanH. P.MacDonaldM.HindsS. R.2ndScherA. I. (2017). Epidemiology and prognosis of mild traumatic brain injury in returning soldiers: A cohort study. Neurology, 88(16), 1571–1579. https://doi.org/10.1212/wnl.0000000000003839
40.
SilvaM.BarwickF.KretzmerT.VanderploegR.BelangerH. (2013). Reliable change indices for the Neurobehavioral Symptom Inventory and Brief Symptom Inventory-18 in a mild traumatic brain injury sample. The Clinical Neuropsychologist, 27, 601.
41.
SobleJ. R.SilvaM. A.VanderploegR. D.CurtissG.BelangerH. G.DonnellA. J.ScottS. G. (2014). Normative data for the Neurobehavioral Symptom Inventory (NSI) and post-concussion symptom profiles among TBI, PTSD, and nonclinical samples. The Clinical Neuropsychologist, 28(4), 614–632. https://doi.org/10.1080/13854046.2014.894576
42.
SolomitoM. J.ReumanH.WangD. H. (2019). Sex differences in concussion: A review of brain anatomy, function, and biomechanical response to impact. Brain Injury, 33(2), 105–110.
43.
SteinM. B.UrsanoR. J.Campbell-SillsL.ColpeL. J.FullertonC. S.HeeringaS. G.NockM. K.SampsonN. A.SchoenbaumM.SunX.JainS.KesslerR. C. (2016). Prognostic indicators of persistent post-concussive symptoms after deployment-related mild traumatic brain injury: A prospective longitudinal study in U.S. army soldiers. Journal of Neurotrauma, 33(23), 2125–2132. https://doi.org/10.1089/neu.2015.4320
44.
VanderploegR. D.SilvaM. A.SobleJ. R.CurtissG.BelangerH. G.DonnellA. J.ScottS. G. (2015). The structure of postconcussion symptoms on the Neurobehavioral Symptom Inventory: A comparison of alternative models. Journal of Head Trauma Rehabilitation, 30(1), 1–11. https://doi.org/10.1097/HTR.0000000000000009
45.
VosL.WhiteneckG. G.NganE.Leon NoveloL.HarikL. M.ShererM. (2019). Comparison of the Neurobehavioral Symptom Inventory and the Rivermead Postconcussion Symptoms Questionnaire. Brain Injury, 33(9), 1165–1172. https://doi.org/10.1080/02699052.2019.1637024
46.
WidamanK. F.ReiseS. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In BryantK. J.WindleM.WestS. G. (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). American Psychological Association. https://doi.org/10.1037/10222-009
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
