Regression to the Mean: Statistical Bias Can Mislead Interpretation in Cartilage and Osteoarthritis Clinics and Research

Introduction

Assessment of the effect of treatment is one of the key fundamental challenges in both clinical research and practice. It is well known that misinterpretation of apparent improvement is lurking due to placebo- or contextual effects. However, the recent publication of Englund and Turkiewicz in Lancet Rheumatology ¹ shows that there is a phenomenon that may explain a significant portion of the effect sizes of orthopedic clinical trials: regression to the mean. They found that in knee osteoarthritis (OA) trials, this purely statistical effect can be as much as a full point on a 10-point Numeric Rating Scale (NRS) for pain.

Variance in Measurement of Pain Sensation in Osteoarthritis

OA is a chronic whole-joint disease, with severe impact on quality of life due to immobilizing joint pain. ² Due to its chronic nature, the average reported discomfort or pain is not likely to decrease in the long term. ³ However, day-to-day variations are significant, with as much as 7 points reported on a 10-point scale difference from 1 day to the next for the pain on that day.^3,4 This may lead to significant recall bias with retrospective pain assessment, as patients are more likely to remember a recent outlier in pain than the average over time. ⁴ The natural history of pain and discomfort differs with each patient, with some patients reporting constant pain levels over time, while others suffer more from intermittent pain episodes. ⁵ Besides the natural history, each measurement tool has inherent measurement error. That means that the reflection of the measurement of the “real” state of the patient is subjected by variance unrelated to the disease.

Explanation of Regression to the Mean

Regression to the mean was first described in a paper by Francis Galton in 1886, ⁶ wherein he observed that parents who are relatively tall tend to have children who are shorter than them, while parents who are relatively short tend to have children who are taller than them.

Regression to the mean refers to the tendency for extreme measurements or observations to move closer to the average when measured again because these extreme measurements are more likely to be affected by variance. This variance can be caused by either the natural fluctuations of the disease or by measurement error. That means that if you measure any variable twice, the highest scores in the first measurement will tend to be on average lower (closer to the mean) in the second measurement and vice versa. Regression to the mean affects a clinical trial if you select your subjects based on your outcome variable (e.g., high pain scores as inclusion criterion, Fig. 1 ). However, in orthopedic trials, this is unavoidable.

Figure 1.

An example of the natural course of disease for a hypothetical osteoarthritis patient. The patient is only eligible for inclusion during high NRS readings (by course of disease or measurement error, yellow), which means that the patient will report a relatively high NRS at baseline. On follow-up, there is no restriction, and the NRS will tend to regress to the long-term mean value, independent of treatment.

Estimation of the Effect

A frequently encountered example of regression to the mean in clinical practice is the incidental encounters of either a high or low blood pressure during routine checkup. Since there is a known high variation affecting blood pressure measurements, single measurements of abnormal blood pressure are usually repeated, with regression to the mean resulting in less extreme values. ⁷

Englund and Turkiewicz ¹ ran a hypothetical trial on the data of an existing cohort of knee OA patients, ⁸ whose NRS scores were collected over a period of 8 years, in which they received no treatment. In the hypothetical trial, the patients were included each year that they met the inclusion criteria of the FORWARD trial ⁹ (NRS>4.0 and use of pain medication). The NRS scores at the moment of inclusion were compared to the NRS scores of the same patients the years before or after hypothetical inclusion. Due to exclusion of the patient years of mild or no reported pain, the average NRS at the moment of inclusion was higher than average for the cohort (6.2 vs. 4.8). If they looked at the same patients 1 year before or after the hypothetical inclusion, the average NRS was an astonishing 0.9 points lower, which even further reduced when looking longer back or ahead in time. We know, however, that in this cohort, there was no treatment and that the mean NRS of this group was in reality constant over the 8 years. This proves that the observed decrease in the average NRS at the time of inclusion can be attributed to the exclusion of patients with low NRS scores resulting from measurement error (see Fig. 1 for an explanation).

In orthopedic trials, regression to the mean is almost inevitable: even without explicit inclusion criteria, patients are eligible for and seek treatment at a moment in time when they score high on the typical outcome measures (e.g., pain, loss of function). As there usually is a high natural variation in these variables, especially in chronic conditions such as OA, ⁴ the trials will include an overrepresentation of patients who report a higher pain score compared to their average. Any retest will therefore regress toward the mean, independent of treatment effects. Regression to the mean can explain in large why placebo-controlled knee OA trials show a consistent improvement in the placebo arm. ¹⁰

Implications for Clinics and Research

In our RCT on the effect of platelet-rich plasma (PRP) on ankle OA,^11,12 we found a 10-point improvement on the American Orthopaedic Foot and Ankle Society (AOFAS) score in both the saline and PRP arm of the trial. Previous non-controlled studies concluded that this was a significant improvement, despite that similar effect sizes were found. ¹³ Considering the effect size of regression to the mean, as found by Englund and Turkiewicz, ¹ most or all found improvement in the ankle PRP studies may be due to regression to the mean.

First and foremost, awareness of regression toward the mean is crucial when evaluating the efficacy of treatments or interventions in clinical practice. Clinicians should be aware that in chronic diseases, the patients are more likely to seek medical assistance when in a period of higher pain, and improvement of the patient at follow-up can therefore not always be attributed to the treatment alone. By considering regression to the mean, clinicians can better discern whether the observed improvements are genuinely attributable to the intervention or simply the natural tendency of extreme values to regress toward the mean over time. Multiple baseline measurements over time, if the trial or clinical practice allows it, can be a way to reduce the effect of variance. Furthermore, researchers in the field of cartilage research could account for regression toward the mean during study design and statistical analysis. In design, a comparative trial with proper randomization will equally distribute the effects of regression to the mean over the groups. If this is not feasible, multiple baseline measurements, preferably over time, will reduce the measurement error and variance. ¹⁴ Also, decoupling of the inclusion and evaluation criteria (e.g., inclusion based on symptoms, evaluation based on image parameters) may reduce the regression to the mean, although that is not always feasible. In the analysis, the effect of regression to the mean can be estimated by inspection if the effect of treatment is larger in patients with extreme baseline values. If so, the effect size can be adjusted using the methods described elsewhere.^14,15 Without such correction, it is important to realize that significant improvement from baseline in a cohort study is not a proof of effect of treatment, and control groups are a requisite. Likewise, the minimal clinically important difference (MCID), that is regularly used to power and evaluate the introduction of new treatments, should be interpreted with caution in cohort studies where no control group is present. ¹⁵ Likewise, the estimation of the MCID can be biased by effects of regression to the mean, which was found to be a significant factor in the determination of the MCID of the International Knee Documentation Committee (IKDC) questionnaire. ¹⁵

Conclusion

Regression to the mean is a major contributor to apparent treatment effect in orthopedic research and practice and should be taken into account when evaluating clinical results.