Prediction of therapeutic value of new drugs approved by health Canada from 2011−2020: A cross-sectional study

Abstract

Objectives

To examine whether a combination of three characteristics of new drugs – review type, outcome of premarket trials (surrogate or clinical) and first-in-class is associated with significant therapeutic value.

Design

Cross-sectional analysis of new drugs approved by Health Canada from January 1, 2011 to December 31, 2020.

Setting

Canada.

Participants

New drugs approved by Health Canada for which therapeutic evaluations, trial outcomes and first-in-class status was available.

Main outcome measures

Distribution of therapeutic value (major, moderate, little to no) depending on how many of the three characteristics were present for each drug.

Results

Health Canada approved 340 drugs of which 243 had data available for analysis. If all three characteristics were present 10 out of the 20 drugs had a major therapeutic rating. Conversely if none were present only 2 drugs out of 37 had a major therapeutic rating.

Conclusion

This study introduces a new evaluation method for determining whether new drugs will have major therapeutic value that appears to be more successful than relying only on the type of review that drugs receive.

Introduction

Previous research from Canada, ¹ France ² and the United States ³ has found that only 10%−31% of new drugs offer significant therapeutic value compared to drugs already on the market. However, predicting which new products will be therapeutically innovative has proven difficult. Single factors such as first-in-class status ⁴ or expedited review status^3,5 have not proven to be good predictors of therapeutic value as rated by independent organizations.

The Canadian Food and Drug Regulations require manufacturers to present evidence that products are efficacious, safe and manufactured under quality conditions in order to be approved. However the Regulations do not require any evidence about how a new drug compares with existing products in terms of efficacy or safety in order to be approved. ⁶ Similarly, the United States (US) Food and Drug Administration (FDA) ⁷ and the European Medicines Agency ⁸ do not compare the therapeutic value of new drugs with ones already available. Therefore, when new drugs appear on the Canadian market there is no information available about how they compare therapeutically to existing products. Being able to predict the therapeutic value of new drugs relatively early in their lifecycle would be useful for clinical practice guideline developers, public and private drug insurers deciding whether to cover the drug and individual clinicians trying to decide whether to prescribe new drugs.

This study examines whether a combination of multiple characteristics – review type, drug characteristics and outcomes used in premarket clinical studies – is associated with major therapeutic value, as rated by independent agencies, for new drugs approved by Health Canada. Secondarily, each of these factors was investigated individually to determine if there was any change from the results of previous studies that showed a poor association with therapeutic value.

Methods

Data sources

A list of all drugs approved by Health Canada from January 1, 2011 to December 31, 2020 was compiled from annual reports from Health Canada (available by contacting [e-mail]) and the generic and brand names along with type of review – standard or expedited was entered into an Excel spreadsheet. Expedited status was either a priority review (180 days as opposed to a standard review time of 300 days) ⁹ or a review under Health Canada's Notice of Compliance with conditions (NOC/c) policy that allows drugs to be marketed on the basis of limited evidence of efficacy. ¹⁰

Health Canada does not identify first-in-class status for the drugs that it approves but there is significant overlap between drugs available in Canada and the US. ¹¹ Therefore, the identification of first-in-class status used American data – a combination of papers by Lanthier and colleagues and Brown and Wobst that rated first-in-class status for drugs approved by the US FDA from 1987–2011 ¹² and 2010−2019, ¹³ respectively. First-in-class status for 2020 used an FDA publication on new drug approvals for that year. ¹⁴ Outcomes (surrogate or clinical) used in premarket trials were determined by consulting the Summary Basis of Decision documents on Health Canada's website that explain the rationale for approving new drugs and describe the outcomes used in the pivotal trials. ¹⁵ Only outcomes from pivotal trials were recorded.

All data was retrieved by a single individual between January 4 and February 5, 2023.

Therapeutic evaluations

Therapeutic value was graded as major, moderate or little to no according to ratings by the Canadian Patented Medicine Prices Review Board (PMPRB), the independent French drug bulletin Prescrire International and the independent, nongovernmental German health technology assessment agency IQWiG. These agencies evaluate therapeutic value of new drugs typically early in their life-cycle, once they have received regulatory approval. In deciding on the level of therapeutic innovation the PMPRB considers two primary factors: increased efficacy and reduction in incidence or grade of important adverse reactions and nine secondary factors: route of administration, patient convenience, compliance improvements leading to improved therapeutic efficacy, caregiver convenience, time required to achieve the optimal therapeutic effect, duration of usual treatment course, success rate, percentage of affected population treated effectively and disability avoidance/savings. The primary factors are given the greatest weight, followed by an assessment of any additional improvement as a result of the secondary factors. ¹⁶ If the rating was not available on either the PMPRB website or in its annual report, the organization was contacted directly. Prescrire assesses the therapeutic value of medicines through a multistep process. First, it examines the proposed indication for the drug, the natural course of the disease, the harms and benefits of existing treatments and the most relevant outcome measures. Then, it conducts a systematic search for clinical data on the efficacy and adverse effects of new drugs and an assessment of the level of evidence and uses this information to grade the level of therapeutic improvement. ¹⁷ IQWiG uses the evidence in dossiers supplied by pharmaceutical companies to determine the extent of added benefit of a new drug. In doing so, it examines the outcomes of treatment in the following ranked order: effects on all-cause mortality, serious (or severe) symptoms (or late complications); serious (or severe) adverse events, health related quality of life; and nonserious (or non-severe) symptoms (or late complications), nonserious (or non-severe) adverse events. Health-related quality of life is regarded to be of equal importance as serious (or severe symptoms), late complications, and adverse events. ¹⁸

Table 1 shows how the ratings from the three agencies were equated and translated into the major, moderate and little to no categories used in this study. These three organizations were chosen because they use mutually exclusive ordinal ratings for therapeutic value, report in English and their ratings have been accepted in other research.^3,19 If all three organizations rated a drug then the highest rating was used.

Table 1.

Evaluation of therapeutic value.

Organization	New therapeutic value			Website for definition of therapeutic value
	Major	Moderate	Little to none
Human Drug Advisory Panel (Patented Medicine Prices Review Board)	Breakthrough, substantial improvement	Moderate improvement – primary and moderate improvement – secondary	Slight or no improvement	http://www.pmprb-cepmb.gc.ca/view.asp?ccid = 492#1635
IQWiG	Major added benefit	Considerable added benefit	Minor (more than marginal) added benefit, no added benefit, less benefit (than the appropriate comparator therapy)	https://www.iqwig.de/methoden/general-methods_version-6-0.pdf
Prescrire International	Bravo, a real advance	Offers an advantage	Possibly helpful, nothing new, not acceptable	http://english.prescrire.org/en/81/168/46800/0/NewsDetails.aspx

Data analysis

The distribution of therapeutic value (major, moderate, little to no) was determined for drugs with all three characteristics (expedited review, first-in-class status, clinical outcome in premarket trials), two of the three, one and no characteristics. A two-tailed Chi-square analysis was used to determine if the distribution differed depending on the number of characteristics present for each drug.

Review status was used as one characteristic as Health Canada states that priority reviews and approvals using the NOC/c pathway are both used for drugs that offer therapeutic options for serious, life-threating or severely debilitation disease for which current treatments are limited.^9,10 Clinical trial outcome was included as it is generally considered to offer more definitive evidence about the efficacy of drugs compared to a surrogate outcome. ²⁰ Finally, first-in-class was included as those drugs are ones with a new and unique mechanism of action for treating a medical condition. These products are often referred to as innovative and cited as offering new treatment options for patients.^21,22

A single linear regression analysis tested whether any of the three individual characteristics – expedited review, clinical outcome and first-in-class status – were individually statistically significantly associated with therapeutic value. For the equations, therapeutic value was rated as 1 = major, 0.5 = moderate, 0 = little to no. The linear regression analysis was repeated separately for drugs with a priority review and those approved through the NOC/c policy.

All analyses were done with Prism 9.5.0 (GraphPad Software, http://www.graphpad.com) and p < 0.05 was considered statistically significant.

Ethics and public/patient involvement

All data were publicly available and ethics approval was not required. The public and patients were not involved.

Results

Over the 10-year period, Health Canada approved 340 new drugs; 55 had no therapeutic evaluations, 4 had no information about trial outcomes and the first-in-class status could not be determined for 38, leaving 243 (71.5%) drugs for analysis. Thirty-six (14.8%) had major therapeutic value as rated by the PMPRB, IQWiG and/or Prescrire, 61 (25.1%) were of moderate therapeutic value and the remaining 146 (60.1%) were little or no therapeutic value over existing drugs. The Supplementary File provides all of the data gathered for this study.

Table 2 shows the distribution of therapeutic value depending on the number of characteristics (expedited review, clinical outcome, first-in-class) present. Twenty (8.2%) of the 243 drugs had all three characteristics and 10 (50%) of those 20 were major therapeutic advances. On-the-other hand, when all three characteristics were absent only two (5.4%) of 37 drugs had a major therapeutic value. The distribution of therapeutic value was statistically significantly different depending on the number of characteristics present (Chi-square, p < 0.0001), indicating that all three characteristics were more accurate in predicting major therapeutic value. Overall, the 20 drugs with all three characteristics accounted for just over 8% of the total number of drugs, but 27.8% of those with major therapeutic value.

Table 2.

Therapeutic value as a function of the number of drug characteristics* for all drugs.

Number of characteristics	Number of drugs	Number of drugs with therapeutic value†
		Major	Moderate	Little to no
Three	20	10	4	6
Two	77	14	25	38
One	109	10	25	74
None	37	2	7	28

*Expedited review, clinical trial outcome, first-in-class.

^†Distribution of therapeutic value statistically significantly different, Chi-square, p < 0.0001.

Individually, expedited review and first-in-class status were statistically significantly associated with therapeutic value in single linear regression analyses (p < 0.0001), although the associations were weak, with R² = 0.08441 (p < 0.0001) and 0.02057 (p = 0.0254), respectively. Breaking down expedited review into priority and NOC/c reviews, only priority review was statistically significant, p < 0.0001 but the association was weak, R² = 0.1283.

Interpretation

Previous research looking at the association with therapeutic value focused on single characteristics such as review type or first-in-class status. This study took a different approach combining decisions by regulatory agencies about review status (standard versus expedited), with chemical features of drugs (first-in-class status) and the methodology used in premarket clinical trials (surrogate versus clinical outcome). The presence of all three characteristics indicates a 50% chance that a drug will have a major therapeutic value whereas the absence of all three is associated with a minimal chance of major therapeutic value. Only a small minority (8.2%) of the 243 drugs had all three characteristics.

Similar to other research, this study also found that review status was associated with therapeutic value, but the association was weak. Both FDA and EMA expedited programs were not strong predictors of high therapeutic value. ³ Canadian research looking at priority review status found that the positive predictive value of Health Canada's decision about review status was 36.3%, that is, it gave a priority review to 91 drugs, but only 33 were evaluated as major therapeutic advances. ⁵

Two arguments have been advanced in favour of approving and marketing drugs with lower therapeutic value. First, that they offer choice to patients who are unable to tolerate the standard treatment and second, that the presence of additional drugs fosters price competition. Although more drugs in a class do potentially offer choice, the value of that choice is unknown when these drugs appear because companies do not test their new drugs on patients who cannot tolerate or do not improve on standard treatment. In the Canadian context, new drugs in the same class do not promote price competition. The mean introductory price of new medications was 96% of the price of existing brand-name products in the same therapeutic class and 92% of the price of the most expensive brand-name product in the class. ²³

Limitations

The lack of therapeutic evaluations and first-in-class data eliminated 28.5% of the drugs approved by Health Canada from analysis. All of the data was gathered by a single investigator and this could have introduced transcription errors. There are discrepancies in the ratings from the PMPRB, IQWiG and Prescrire. In one study, measuring the agreement between PMPRB and Prescrire, the Kappa value, a measure of the inter-rater reliability, was 0.2624 indicating only fair agreement. ¹ These differences may be due to the variability in the factors that the three organizations consider and also in how the data are interpreted. Therapeutic value is also not necessarily fixed and can change over time but this study used a static evaluation of value.

Conclusion

This study introduces a novel evaluation method for determining whether new drugs will have major therapeutic value that appears to be more accurate than relying only on the type of review that drugs receive or other single measures. The absence of one or more characteristics lowers the likelihood that a drug will have major therapeutic value. However, this method still has limitations in its predictive value. Information about therapeutic value and first-in-class status for more of the drugs would allow more precision in determining the strength of this method. The model could also be prospectively evaluated by selecting a group of newly approved drugs, determining which of the three characteristics are present for each drug and then looking at future therapeutic evaluations of the drugs to see how accurately the model predicted the evaluations.

Supplemental Material

sj-pdf-1-shr-10.1177_20542704231166620 - Supplemental material for Prediction of therapeutic value of new drugs approved by health Canada from 2011−2020: A cross-sectional study

Supplemental material, sj-pdf-1-shr-10.1177_20542704231166620 for Prediction of therapeutic value of new drugs approved by health Canada from 2011−2020: A cross-sectional study by Joel Lexchin in JRSM Open