Objective: To provide a practical guide on how to avoid the pitfalls of correlated correlation coefficients when comparing multiple instruments in rehabilitation research.
Design: An observational study comparing a number of instruments measuring quality of life (QoL) compared with an external criterion.
Subjects: Sixty-eight patients admitted to a rheumatology ward for intensive treatment of rheumatoid arthritis.
Methods: Patients completed three new (QoL) instruments and an established instrument before and after intensive treatment for rheumatoid arthritis.
Main outcome measures: Correlation coefficients together with their confidence intervals and a test for the difference between a set of correlated correlation coefficients for the change in the EuroQoL Quality of Life scale (EuroQoL), the World Health Organization Quality Of Life-Abbreviated version (WHOQoL-BREF) and the Quality of Life Profile (QLP) against the Stanford Health Assessment Questionnaire (HAQ).
Results: Although the range of correlation between the new instruments and the external criterion was between -0.37 and -0.59 and suggested that one new instrument was far more responsive than the others,; an omnibus test for an overall difference could find no difference in responsiveness.
Conclusions: It is conceptually simple to use correlation coefficients to assess the properties of multiple instruments measured on the same subjects to find a ‘best’ instrument. However, proper interpretation of results when correlated correlation coefficients are calculated is complex. We recommend analysis includes: (a) that simple plots of the pairs of analysed variables are shown, (b) that simple linear model-fitting statistics, e.g., the R-squared statistic, accompany the plots, (c) that confidence intervals are presented for correlation coefficients, (d) that an omnibus statistical test for the difference between correlated correlation coefficients is presented, and (e) that normal model assumptions are tested.
Get full access to this article
View all access options for this article.
References
1.
Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: A critical review. J Clin Epidemiol2000; 53: 459–468.
2.
Fitzpatrick R, Fletcher A, Gore S, Jones D, Spiegelhalter D, Cox D. Quality of life measures in health care I: applications and issues in assessment. BMJ1992; 305: 1074-1074.
3.
Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. Chichester: J Wiley and Sons, 2000.
4.
Hobart JC, Lamping DL, Thompson AJ. Evaluating neurological outcome measures: the bare essentials. J Neurol Neurosurg Psychiatry1996; 60: 127–130.
5.
Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures: Statistics and strategies for evaluation. Controlled Clin Trials1991; 12: 142S–158S.
6.
Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostic test performance. J Chronic Disabil1986; 39: 897–906.
7.
Fitzpatrick R, Ziebland S, Jenkinson C, Mowat A, Mowat A. A comparison of sensitivity to change of several health status instruments in rheumatoid arthritis. J Rheumatol1993; 20: 429–436.
8.
Wright JG, Young NL. A comparison of different indices of responsiveness. J Clin Epidemiol1997; 50: 239–246.
9.
Meenan RF, Anderson JJ, Kazis Le et al. Outcome assessment in clinical trials: evidence for the sensitivity of a health status measure. Arthritis Rheum1984; 27: 1344–1352.
10.
Stucki G, Liang MH, Fossel AH, Katz JN. Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol1995; 48: 1369–1378.
11.
Bain LJ, Engelhardt M. Introduction to probability and mathematical statistics, second edition. Boston: PWS-Kent, 1991.
12.
May K, Hittner JB. A note on statistics for comparing dependent correlations. Psychol Rep1997; 80: 475–480.
Taylor WJ, McPherson K. Measuring health-related quality of life. NZ Med J1999; 112: 239–240.
15.
EuroQol group. EuroQol: a new facility for the measurement of health related quality of life. Health Policy1990; 16: 199–208.
16.
Harper A, Power M. Development of the World Health Organisation WHOQOL-BREF quality of life assessment. Psychol Med1998; 28: 551–558.
17.
Raphael D, Brown I, Renwick R, Cava M, Weir N, Heathcote K. The quality of life of seniors living in the community: a conceptualization with implications for public health practice. Can J Public Health1995; 86: 228–233.
18.
Raphael D, Rukholm E, Brown I, Hill-Bailey P, Donato E. The quality of life profile —Adolescent version: background, description, and initial validation. J Adolescent Health1996; 19: 366–375.
19.
Fries JF, Spitz P, Kraines G, Holman HR. Measurement of patient outcome in arthritis. Arthritis Rheum1980; 23: 137–145.
20.
Pincus T, Summey JA, Soraci SA, Wallston KA, Hummon NP. Assessment of patient satisfaction in activities of daily living using a modified Stanford health assessment questionnaire. Arthritis Rheum1983; 26: 1346–1353.
21.
Bombardier C, Raboud J. A comparison of health related quality-of-life measures for rheumatoid arthritis research. Controlled Clin Trials1991; 12:243S-256S.
22.
Wells GA, Tugwell P, Kraag GR, Baker PRA, Groh J, Reidlmeier DA. Minimum important difference between patients with rheumatoid arthritis: The patient's perspective. J Rheumatol1993; 20: 557–560.
23.
Buchbinder R, Bombardier C, Yeung M, Tugwell P. Which outcome measurements should be used in rheumatoid arthritis trials. Arthritis Rheum1995; 38: 1568–1580.
24.
Kosinski M. Zhao SZ, Dedhiya S, Osterhaus JT, Ware JE. Determining the minimally important changes in generic and disease-specific health-related quality of life questionnaires in clinical trials of rheumatoid arthritis. Arthritis Rheum2000; 43: 1478–1487.
25.
Tugwell P, Wells G, Strand V et al. Clinical improvement as reflected in measures of function and health-related quality of life following treatment with leflunomide compared with methotrexate in patients with rheumatoid arthritis: sensitivity and relative efficiency to detect a treatment effect in a twelve-month, placebo-controlled trial. Leflunomide Rheumatoid Arthritis Investigators Group. Arthritis Rheum2000; 43: 506–514.
26.
Tugwell P, Boers M. Developing consensus on preliminary core efficacy endpoints for rheumatoid arthritis clinical trials. OMERACT Committee. J Rheumatol1993; 20: 555–556.
27.
Vliet Vlieland TPM, Zwinderman AH, Vanendbroucke JP, Breedveld FC, Hazes JMW. A randomized clinical trial of inpatient
