Background In order to make meaningful cross-cultural or cross-linguistic comparisons of health-related quality of life (HRQL) or to pool international research data, it is essential to create unbiased measures that can detect clinically important differences. When HRQL scores differ between cultural/linguistic groups, it is important to determine whether this reflects real group differences, or is the result of systematic measurement variability.
Purpose To investigate the linguistic measurement equivalence of a cancer-specific HRQL questionnaire, and to conduct a sensitivity analysis of treatment differences in HRQL in a clinical trial.
Methods Patients with newly diagnosed chronic myelogenous leukemia (n = 1049) completed serial HRQL assessments in an international Phase III trial. Two types of differential item functioning (uniform and non-uniform) were evaluated using item response theory and classical test theory approaches. A sensitivity analysis was conducted to compare HRQL between treatment arms using items without evidence of differential functioning.
Results Among 27 items, nine (33%) did not exhibit any evidence of differential functioning in both linguistic comparisons (English versus French, English versus German). Although 18 items functioned differently, there was no evidence of systematic bias. In a sensitivity analysis, adjustment for differential functioning affected the magnitude, but not the direction or interpretation of clinical trial treatment arm differences.
Limitations Sufficient sample sizes were available for only three of the eight language groups. Identification of differential functioning in two-thirds of the items suggests that current psychometric methods may be too sensitive.
Conclusions Enhanced methodologies are needed to differentiate trivial from substantive differential item functioning. Systematic variability in HRQL across different groups can be evaluated for its effect upon clinical trial results; a practice recommended when data are pooled across cultural or linguistic groups to make conclusions about treatment effects.
Get full access to this article
View all access options for this article.
References
1.
1. U.S. Department of Health and Human Services (FDA). Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims (draft guidance), 2006.
2.
2. Angel R, Thoits P.The impact of culture on the cognitive structure of illness. Culture, Medicine and Psychiatry1987; 11: 23–52.
3.
3. Bullinger M, Anderson R, Cella D, Aaronson N.Developing and evaluating cross-cultural instruments from minimum requirements to optimal models. Qual Life Res1993; 2: 451–459.
4.
4. Johnson TP, O'Rourke DP, Chavez N, Sudman S, Warnecke RB, Lacey L.Cultural variations in the interpretation of health questions. Health survey research methods: conference proceedings1996: 57-57, 62-62.
5.
5. Hayes RP, Baker DW.Methodological problems in comparing English-speaking and Spanish-speaking patients' satisfaction with interpersonal aspects of care. Med Care1998; 36: 230–236.
7. Drasgow F, Kanfer R.Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology1985; 70: 662–680.
8.
8. Hui CH, Triandis HC.Measurement in cross-cultural psychology. Journal of Cross-Cultural Psychology1985; 16: 131–152.
9.
9. Hahn EA, Cella D.Health outcomes assessment in vulnerable populations: Measurement challenges and recommendations. Arch Phys Med Rehabil2003; 84: S35–S42.
10.
10. Flaherty JA, Gaviria FM, Pathak D et al. Developing instruments for cross-cultural psychiatric research. The Journal of Nervous and Mental Disease1988; 176: 257–263.
11.
11. Herdman M, Fox-Rushby J, Badia X.‘Equivalence’ and the translation and adaptation of health-related quality of life questionnaires. Qual Life Res1997; 6: 237–247.
12.
12. Stewart AL, Napoles-Springer AM.Advancing health disparities research: Can we afford to ignore measurement issues?Med Care2003; 41: 1207–1220.
13.
13. Holland PW, Wainer H.Differential item functioning. Lawrence Erlbaum Associates, 1993.
14.
14. Zumbo BD.A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and likert-type (ordinal) item scores. Directorate of Human Resources Research and Evaluation, Department of National Defense, 1999.
15.
15. Embretson SE, Reise SP.Item response theory for psychologists. Lawrence Erlbaum Associates, 2000.
16.
16. Millsap RE, Everson HT.Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement1993; 17: 297–334.
17.
17. Camilli G, Shepard LA.Methods for identifying biased test items. Sage Publications, 1994.
18.
18. Bjorner JB, Kreiner S, Ware JE, Damsgaard MT, Bech P.Differential item functioning in the Danish translation of the SF-36. J Clin Epidemiol1998; 51: 1189–1202.
19.
19. Petersen MA, Groenvold M, Bjorner JB et al. Use of differential item functioning analysis to assess the equivalence of translations of a questionnaire. Qual Life Res2003; 12: 373–385.
20.
20. Fumimoto H, Kobayashi K, Chang CH et al. Cross-cultural validation of an international questionnaire, the General Measure of the Functional Assessment of Cancer Therapy scale (FACT-G), for Japanese. Qual Life Res2001; 10: 701–709.
22. van der Linden WJ, Hambleton RK.Handbook of modern item response theory. Springer-Verlag, 1997.
23.
23. Hays RD, Morales LS, Reise SP.Item Response Theory and health outcomes measurement in the 21st century. Med Care2000; 38: II-28–II-42.
24.
24. Cella D, Chang CH.A discussion of Item Response Theory (IRT) and its applications in health status assessment. Med Care2000; 38: II66–II72.
25.
25. Andrich D.Scaling attitude items constructed and scored in the Likert tradition. Educational and Psychological Measurement1978; 38: 665–680.
26.
26. Raczek AE, Ware JE, Bjorner JB et al. Comparison of Rasch and summated rating scales constructed from SF- 36 physical functioning items in seven countries: Results from the IQOLA Project. International Quality of Life Assessment. Journal of Clinical Epidemiology1998; 51: 1203–1214.
27.
27. Orlando M, Marshall GN.Differential item functioning in a Spanish Translation of the PTSD checklist: Detection and evaluation of impact. Psychological Assessment2002; 14: 50–59.
28.
28. Hahn EA, Holzner B, Kemmler G, Sperner-Unterweger B, Hudgens SA, Cella D.Cross-cultural evaluation of health status using item response theory: FACT-B comparisons between Austrian and U.S. patients with breast cancer. Evaluation and the Health Professions2005; 28: 233–259.
29.
29. Bacik J, Mazumdar M, Murphy BA et al. The functional assessment of cancer therapy-BRM (FACTBRM): A new tool for the assessment of quality of life in patients treated with biologic response modifiers. Qual Life Res2004; 13: 137–154.
30.
30. Bonomi AE, Cella DF, Hahn EA et al. Multilingual translation of the Functional Assessment of Cancer Therapy (FACT) quality of life measurement system. Qual Life Res1996; 5: 309–320.
31.
31. Cella D, Hernandez L, Bonomi AE et al. Spanish language translation and initial validation of the functional assessment of cancer therapy quality-of-life instrument. Med Care1998; 36: 1407–1418.
32.
32. Lent L, Hahn E, Eremenco S, Webster K, Cella D.Using cross-cultural input to adapt the Functional Assessment of Chronic Illness Therapy (FACIT) scales. Acta Oncol1999; 38: 695–702.
33.
33. Cella DF, Tulsky DS, Gray G et al. The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. J Clin Oncol1993; 11: 570–579.
34.
34. O'Brien SG, Guilhot F, Larson RA et al. Imatinib compared with interferon and low-dose cytarabine for newly diagnosed chronic-phase chronic myeloid leukemia. The New England Journal of Medicine2003; 348: 994–1004.
35.
35. Hahn EA, Glendenning GA, Sorensen MV et al.Quality of life in patients with newly diagnosed chronic phase chronic myeloid leukemia on imatinib versus interferon alfa plus low-dose cytarabine: Results from the IRIS Study. J Clin Oncol2003; 21: 2138–2146.
37. Bentler PM, Bonnett DG.Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin1980; 88: 588–606.
38.
38. Steiger JH, Lind JC.Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA, 1980.
39.
39. Bentler PM.Comparative fit indexes in structural models. Psychological Bulletin1990; 107: 238–246.
40.
40. Hu L, Bentler PM.Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equations Modeling1999; 6: 1–55.
41.
41. Marsh HW, Hau KT, Wen Z.In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural Equation Modeling2004; 11: 320–341.
42.
42. Maruyama GM.Basics of structural equation modeling.Sage Publications, Inc., 1998.
43.
43. Wright BD, Linacre JM.Reasonable mean square fit values. Rasch Measurement Transactions1994; 8: 370-370.
44.
44. Smith RM.A comparison of the Rasch separate calibration and between fit methods of detecting item bias. Educational and Psychological Measurement1996; 56: 403–418.
48. Fairclough DL.Design and analysis of quality of life studies in clinical trials. Chapman & Hall/CRC, 2002.
49.
49. Little RA.Modeling the drop-out mechanism in repeated-measures studies. Journal of the American Statistical Association1995; 90: 1112–1121.
50.
50. Hedeker D, Gibbons RD.Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods1997; 2: 64–78.
51.
51. Yost KJ, Sorensen MV, Hahn EA, Glendenning GA, Gnanasakthy A, Cella D.Using multiple achor-and distribution-based estimates to evaluate clinically meaningful change on the Functional Assessment of Cancer Therapy-Biologic Response Modifiers (FACTBRM) Instrument. Value Health2005; 8: 117–127.
52.
52. Teresi JA, Kleinman M, Ocepek-Welikson K.Modern psychometric methods for detection of differential item functioning: Application to cognitive assessment measures. Stat Med2000; 19: 1561–1683.
