Misclassification of Case–Control Studies in the Orthopedic Literature is Common: A Bibliometric Analysis

Abstract

Background

Observational studies constitute the majority of the orthopedic literature, each type distinct in terms of what it can and cannot measure. Case–control studies select participants based on outcome status, not exposure status, and therefore differ from other observational studies in their aims, limitations, and conclusions. Misclassification of a different kind of study as a case–control study can lead to misinterpretation of the data and misreporting of its level of evidence (LOE), either “overselling” or “underselling” its importance.

Questions/Purposes

We sought to answer three questions: (1) How frequently do studies reported to be orthopedic case–control studies actually reflect other study designs? (2) What factors might be associated with misclassification? (3) How does study design misclassification affect LOE reporting?

Methods

A bibliometric analysis was performed to identify all studies published in 75 orthopedic journals over a one-year period (January 2017 through December 2017) that included the term “case–control” in the title, abstract, or main text. We identified the proportion of studies that were misclassified as “case–control” in design and recorded the associated changes in reported LOE. We also examined associations between study misclassification (and by extension LOE misclassification) and the study specialty, journal specialty, and journal impact factor.

Results

Of 339 studies that reported a case–control design, 227 (67%) were misclassified and reflected other study designs. The study designs most often misclassified as case–control designs were retrospective cohort studies (n = 97; 43%) and cross-sectional studies (n = 88; 39%). The frequency of misclassification was associated with the subspecialty of the journal and the impact factor but not the study subspecialty. After correction of the LOE in the misclassified studies that reported an LOE (n = 193), it was found that 28 (15%) had underreported their LOE, and eight (4%) had overreported their LOE.

Conclusions

Studies reported in the orthopedic literature to have a case–control design frequently have another study design, and this pattern is consistent across subspecialties. Enhanced rigor in accurately defining study designs in orthopedics could be achieved through training and stricter review processes.