Restricted accessResearch articleFirst published online 2020-4
Comparison of structural equation modelling,item response theory and Rasch measurement theory-based methods for response shift detection at item level: A simulation study
When assessing change in patient-reported outcomes, the meaning in patients’ self-evaluations of the target construct is likely to change over time. Therefore, methods evaluating longitudinal measurement non-invariance or response shift at item-level were proposed, based on structural equation modelling or on item response theory. Methods coming from Rasch measurement theory could also be valuable. The lack of evaluation of these approaches prevents determining the best strategy to adopt. A simulation study was performed to compare and evaluate the performance of structural equation modelling, item response theory and Rasch measurement theory approaches for item-level response shift detection. Performances of these three methods in different situations were evaluated with the rate of false detection of response shift (when response shift was not simulated) and the rate of correct response shift detection (when response shift was simulated). The Rasch measurement theory-based method performs better than the structural equation modelling and item response theory-based methods when recalibration was simulated. Consequently, the Rasch measurement theory-based approach should be preferred for studies investigating only recalibration response shift at item-level. For structural equation modelling and item response theory, the low rates of reprioritization detection raise issues on the potential different meaning and interpretation of reprioritization at item-level.
NippRDTemelJS.Harnessing the power of patient-reported outcomes in oncology.Clin Cancer Res Off J Am Assoc Cancer Res2018;
24: 1777–1779.
2.
KluetzPGO’ConnorDJSoltysK.Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada.Lancet Oncol2018;
19: e267–e274.
3.
WidamanKFFerrerECongerRD.Factorial invariance within longitudinal structural equation models: measuring the same construct across time.Child Dev Perspect2010;
4: 10–18.
4.
VerhagenJFoxJ-P.Longitudinal measurement in health-related surveys. A Bayesian joint growth model for multivariate ordinal responses.Stat Med2013;
32: 2988–3005.
5.
PastorDABeretvasSN.Longitudinal rasch modeling in the context of psychotherapy outcomes assessment.Appl Psychol Meas2006;
30: 100–120.
6.
SprangersMASchwartzCE.Integrating response shift into health-related quality of life research: a theoretical model.Soc Sci Med1982;
48: 1507–1515.
7.
McClimansLBickenbachJWestermanM, et al.
Philosophical perspectives on response shift.Qual Life Res 22: 1871–1878.
8.
PostulartDAdangEM.Response shift and adaptation in chronically ill patients. Med Decis Mak IntJ Soc Med Decis Mak2000;
20: 186–193.
9.
OortFJ.Using structural equation modeling to detect response shifts and true change.Qual Life Res2005;
14: 587–598.
10.
SajobiTTBrahmbattRLixLM, et al.
Scoping review of response shift methods: current reporting practices and recommendations.Qual Life Res2018;
27: 1133–1146.
11.
SchwartzCE.Introduction to special section on response shift at the item level.Qual Life Res2016;
25: 1323–1325.
12.
VanierASébilleVBlanchinM, et al.
Overall performance of Oort’s procedure for response shift detection at item level: a pilot simulation study.Qual Life Res2015;
24: 1799–1807.
13.
NolteSMierkeAFischerHF, et al.
On the validity of measuring change over time in routine clinical assessment: a close examination of item-level response shifts in psychosomatic inpatients.Qual Life Res2016;
25: 1339–1347.
14.
GandhiPKSchwartzCEReeveBB, et al.
An item-level response shift study on the change of health state with the rating of asthma-specific quality of life: a report from the PROMIS® Pediatric Asthma Study.Qual Life Res2016;
25: 1349–1359.
15.
VerdamMGEOortFJSprangersMAG.Using structural equation modeling to detect response shifts and true change in discrete variables: an application to the items of the SF-36.Qual Life Res2016;
25: 1361–1383.
16.
AhmedSSawatzkyRLevesqueJ-F, et al.
Minimal evidence of response shift in the absence of a catalyst.Qual Life Res2014;
23: 2421–2430.
17.
GuilleuxABlanchinMVanierA, et al.
RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies.Qual Life Res2015;
24: 553–564.
18.
AnotaABascoul-MolleviCConroyT, et al.
Item response theory and factor analysis as a mean to characterize occurrence of response shift in a longitudinal quality of life study in breast cancer patients.Health Qual Life Outcomes2014;
12: 32.
19.
BlanchinMSébilleVGuilleuxA, et al.
The Guttman errors as a tool for response shift detection at subgroup and item levels.Qual Life Res Int J Qual Life Asp Treat Care Rehabil2016;
25: 1385–1393.
20.
FischerGHMolenaarIW.Rasch models: foundations, recent developments, and applications.
New York, NY:
Springer, 1995, 470 p.
de BockÉHardouinJ-BBlanchinMLe NeelT, et al.
Rasch-family models are more valuable than score-based approaches for analysing longitudinal patient-reported outcomes with missing data.Stat Meth Med Res2016;
25: 2067–2087.
23.
HamelJ-FHardouinJ-BLe NeelT, et al.
Biases and power for groups comparison on subjective health measurements.PLoS ONE2012;
7: e44695.
24.
de BockEHardouinJ-BBlanchinM, et al.
Assessment of score- and Rasch-based methods for group comparison of longitudinal patient-reported outcomes with intermittent missing data (informative and non-informative).Qual Life Res2015;
24: 19–29.
25.
NolteSElsworthGRSinclairAJ, et al.
Tests of measurement invariance failed to support the application of the “then-test.”J Clin Epidemiol2009;
62: 1173–1180.
26.
MillerRGJ.Simultaneous statistical inference. 2nd ed.
New York, NY:
Springer-Verlag, 1981.
27.
MurakiE.A generalized partial credit model: application of an EM algorithm.Appl Psychol Meas1992;
16: 159–176.
28.
FischerGPonocnyI.An extension of the partial credit model with an application to the measurement of change.Psychometrika1994;
59: 177–192.
29.
MastersGN.A rasch model for partial credit scoring.Psychometrika1982;
47: 149–174.
30.
WareJESherbourneCD.The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection.Med Care1992;
30: 473–483.
31.
AaronsonNKAhmedzaiSBergmanB, et al.
The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology.J Natl Cancer Inst1993;
85: 365–376.
32.
ZigmondASSnaithRP.The hospital anxiety and depression scale.Acta Psychiatr Scand1983;
67: 361–370.
33.
King-KallimanisBLOortFJNolteS, et al.
Using structural equation modeling to detect response shift in performance and health-related quality of life scores of multiple sclerosis patients.Qual Life Res2011;
20: 1527–1540.
34.
King-KallimanisBLOortFJLynnN, et al.
Testing the assumption of measurement invariance in the SAMHSA mental health and alcohol abuse stigma assessment in older adults.Ageing Int2012;
37: 441–458.
35.
DolanCV.Factor analysis of variables with 2, 3, 5 and 7 response categories: a comparison of categorical variable estimators using simulated data.Br J Math Stat Psychol1994;
47: 309–326.
36.
MuthenBKaplanD.A comparison of some methodologies for the factor analysis of non-normal Likert variables: a note on the size of the model.Br J Math Stat Psychol1992;
45: 19–30.
37.
GadermannAMSawatzkyRPalepuA, et al.
Minimal impact of response shift for SF-12 mental and physical health status in homeless and vulnerably housed individuals: an item-level multi-group analysis.Qual Life Res Int J Qual Life Asp Treat Care Rehabil2017;
26: 1463–1472.
38.
Barclay-GoddardREpsteinJDMayoNE.Response shift: a brief overview and proposed research priorities.Qual Life Res2009;
18: 335–346.
