Grab Control! Choosing the Right Comparison Group in an Observational Study

Randomized controlled trials (RCTs) are not always practical to conduct among patients receiving surgical intervention for spine conditions. They may, for example, be unfeasible as in the case of identifying a rare complication. Such a study would require thousands of patients to determine the potential harm of an intervention. Time and resources make an RCT undoable in this situation. In some circumstances, it may be unethical to randomize patients in order to determine if a surgical procedure is effective, as when there is a lack of clinical equipoise. In either case, observational studies can provide evidence to help answer specific clinical questions. Observational studies can introduce bias at several stages of an investigation. Today we will focus on one of those stages, choosing a control or comparison group against which to compare an intervention.

Observational Study Designs

Selecting an appropriate comparison group depends in part on the study design. Let’s consider 2 common observational study designs, the cohort study and the case-control study.

Cohort Study Design

In a cohort study, 2 or more groups are formed based on exposure (eg, surgical procedure or risk factor). The outcome is then determined and compared between groups. As a result, a cohort study can measure several outcomes in the same study. This can be done either prospectively or retrospectively, depending on when the study begins. Studies that are initiated prior to the occurrence of outcomes are prospective, while those starting after the outcomes have been collected are retrospective. Figure 1 illustrates this design.

Figure 1.

Cohort study design.

Case-Control Study Design

In a case-control study, groups are formed based on outcome (eg, nonunion or deep infection). The exposure is then determined and compared between groups. Case-control studies are suitable for study of rare outcomes and can measure multiple exposures (risk factors) in the same study. A case-control study is retrospective and often misunderstood and confused with a retrospective cohort study. In fact, one recent report assessing studies labeled as case-control in the Journal of Neurosurgery Publishing Group (Journal of Neurosurgery, Journal of Neurosurgery: Pediatrics, Journal of Neurosurgery: Spine, and Neurosurgical Focus) or Neurosurgery found only 48% of the studies were in fact case-control studies. ¹ Many of the rest were retrospective cohort studies. Remember, a retrospective cohort study identifies groups based on their treatment while a case-control study identifies groups based on their outcome. Figure 2 illustrates this design.

Figure 2.

Case-control study design.

Control or Comparison Groups

For the sake of this article, the terms “control group” and “comparison group” will be used interchangeably. The goal in selecting patients for a control group is to have a group similar to the intervention group in terms of the presence of prognostic factors. Any unbalance in prognostic factors between groups can lead to confounding, either an underestimation or overestimation of the effect of the surgical procedure or risk factor. Let’s look at some potential control groups for the 2 study designs we identified above.

Control Groups for Cohort Studies

Concurrent Controls

Concurrent controls are those that receive a treatment during the same time period as those that receive the intervention under study. Since the goal is to have a control group as similar to the intervention group as possible, comparing 2 surgical procedures from the same institution would be preferable to comparing the same 2 procedures from different institutions. This would help reduce potential bias that may arise from different hospital-based patient selection criteria, different case complexity, discordant surgical volume, and dissimilar levels of perioperative and postoperative hospital-based care. ² Better still is to compare 2 procedures at the hands of the same surgeon at the same institution given that surgeon experience and technique has been shown to have prognostic value. ^{3 -5}

Historical Controls

Distinct from concurrent controls are historical controls. Historical controls include a set of patients outside the study population that were treated in the past. At first blush, a historical control group may seem to be a natural choice when there has been a change from one surgical treatment to another. However, historical controls have all the potential biases as concurrent controls with the added uncertainty that comes with changes that occur over time. These changes over time often include improvement in technology and perioperative care, an evolving quality assurance program, and a more experienced staff. ⁶ Furthermore, changes in the severity or operational definition of the spinal condition as well as changes in outcome risks or outcome definitions that commonly occur over time can add to potential study bias. ⁷ Therefore, historical controls should be considered only when enrollment of concurrent controls is not possible.

Control Groups for Case-Control Studies

Remember that case-control studies have groups formed based on the outcome. Those with the outcome of interest are referred to as “cases,” while those without the outcome of interest are the “controls”. Selection of controls is often the most difficult aspect of conducting a case-control study.

To reduce selection bias, controls should be representative of the same source population that produced the cases. This can be a bit tricky and is illustrated by a study attempting to identify risk factors associated with ischemic optic neuropathy after spinal fusion. ⁸ Since ischemic optic neuropathy is a rare complication following spine surgery, a case-control study design is appropriate. The investigators identified cases from a volunteer registry with anonymous submission. The type and location of hospitals from where the cases came are not known. The controls, on the other hand, were selected from among academic medical centers that perform a large volume of spine fusions. It is possible that spine fusion patients referred to an academic medical center are different with respect to demographic and severity characteristics than spine fusion patients treated at nonacademic centers. Furthermore, some of the effect of risk factors may be the effect of the facility (eg, facility and surgeon volume), and this information is not available. This study had limited choices in choosing their control group. Nonetheless, their selection increased the risk of bias. A better option if it were possible would include some nonacademic institutions.

Summary

Selecting an appropriate control group in an observational study depends in part on the study design, whether the design is a cohort or case-control study.

The goal in selecting patients for a control group is to have a group similar to the surgical intervention group in terms of the presence of prognostic factors.

For cohort studies, concurrent controls are better than historical controls. Historical controls have all the potential biases as concurrent controls with the added uncertainty that comes with changes that occur over time.

Selecting controls in case-control studies requires they be representative of the same source population that produced the cases.