A regulator’s view of comparative effectiveness research

Abstract

Background ‘Comparative effectiveness’ is the current enthusiasm, and for good reason. After knowing a treatment works, the most critical question is how it compares with alternatives. Comparative studies are not commonly conducted by drug companies and they represent a significant methodological challenge. Comparative data could include evidence of overall superiority to an alternative or advantages in identifiable subsets, for example, people who do not respond to or tolerate alternatives, or members of a genetic subset and could also include convincing evidence that there is little difference between two treatments.

Purpose To describe regulations, guidance, and Food and Drug Administration experience related to studies of comparative effectiveness, including approaches to showing superiority and problems encountered in showing similarity.

Methods Review of Food and Drug Administration regulations and guidance and experience with showing superiority and similarity, particularly related to randomized trials and epidemiologic studies.

Results Methods exist, and they have been successful for showing overall superiority of one drug over another, advantages in specific population subsets. Efforts to show true equivalence face problems of definition and very large sample sizes needed to rule out small differences.

Limitations There is need for further discussion of what is meant by similarity or equivalence of two treatments.

Conclusion Comparative studies are challenging because differences between effective therapies are likely to be small and can be detected reliably only in randomized trials, often large ones. Despite the difficulties, comparative trials have been successful and we clearly would like to see more of them.