Abstract
Introduction
As the pace of health-technology innovations increases, greater pressure is placed on health-technology assessment (HTA) agencies to review and make recommendations promptly. One response to this pressure is the use of rapid reviews (RRs). Since 2009 all new drugs in Ireland are first subject to a RR. The RR process refers costly drugs and those with uncertainty surrounding their cost effectiveness for a full HTA. The objective of this study is to explore differences between drugs that were subject to RR only and a full HTA, compare the number of HTAs in Ireland to those in other jurisdictions and examine what factors determine the outcome of an RR, i.e. if a full HTA is recommended or not.
Methods
Data on drug evaluations from 2009 to July 2015 were extracted from the NCPE, SMC and NICE websites to support the univariate and comparative analysis. A logit regression was employed to determine the factors influencing the outcome of the RR, i.e. the recommendation for a full HTA. Also, propensity score matching was employed to determine if there was an expected difference in achieving a positive reimbursement between drugs recommended and not recommended for full HTAs.
Results
Between 2009 and July 2015 a total of 199 drugs were evaluated in Ireland: 53% of them were recommended for a full HTA (n = 105) returning a 76% reimbursement rate. This compares to 120 and 355 HTAs and reimbursement rates of 68% and 80% in Scotland and England and Wales, respectively. Results of the logit reveal that first in class drugs and those indicated for cancer and musculoskeletal disorders were more likely to require a full HTA in Ireland. The matching technique revealed there was an expected difference in achieving a positive reimbursement between drugs recommended and not recommended for full HTAs.
Conclusions
RRs support decision making in Ireland by reserving full scientific rigor (without comprising outcomes) for technologies that put most pressure on health budgets.
Introduction
Health-technology assessments (HTAs) have become a common feature of health systems globally and play an important role in informing reimbursement decisions. While there appears to be consensus surrounding the rationale and importance of their use, there is wide variation in what is included in them and how they are employed globally. There have been many studies describing and examining the role, function and workings of HTAs for health systems on a standalone and comparative basis highlighting similarities and differences. HTA methods however are not static and best practice evolves over time. Furthermore, the number and pace of health-technology innovations has been increasing, placing increased pressure on already scarce resources and on decision makers to make decisions in short timeframes (shorter than time required to produce a full HTA (1)). In response to these advances and demands a novel response is warranted. One advancement is the use of rapid reviews (RRs) (2).
While definitions and descriptions of what RRs entail vary, they aim in theory to synthesize evidence to support informed decisions in a timely manner, without sacrificing scientific rigor. RRs are emerging as an approach to support HTAs and decision making in an environment with increasingly competing demands (3) from patients who want access to health technologies, manufacturers who want their products reimbursed and health payers who want to curtail spending. Lack of standardized definitions and methods for conducting RRs makes comparative analyses difficult, so attention is focused on single-system analysis. Khangura et al (3) describes the methods and experiences of using RRs by the Canadian Agency for Drugs and Technologies in Health and this paper considers their use in Ireland.
There have been several studies published describing the role of HTAs/pharmacoeconomic evaluation process in the Irish healthcare system (4–6) as well as an examination of the criteria influencing reimbursement decisions in Ireland using a retrospective analysis of completed HTAs (7). This paper complements these analyses and considers the role of the RR process in Ireland. Specifically, the objective of this study is to explore differences in the RR only and HTA pathways in the Irish reimbursement system and to show what factors influence whether a full HTA is required or not. The Irish reimbursement system is also compared to that in Scotland and England and Wales to consider the efficiency of one containing a RR.
HTAs and rapid reviews in Ireland
In Ireland, the use of HTAs is governed by agreements between the Irish Pharmaceutical Health Association (IPHA) and the Health Service Executive (HSE). The agreement outlines the supply and pricing conditions of medicines supplied to the health services in Ireland. It covers all medicines reimbursable in the primary care reimbursement schemes (PCRS) and to hospitals (HSE/state funded) and state agencies whose functions involve the provision of medicines (8). There are three broad reimbursement schemes under which medicines can be reimbursed. These are as follows:
General Medicines Scheme (GMS): covering drugs that are prescribed and dispensed in community pharmacies. It is means tested and those eligible receive free medicines subject to a €2.50 prescription charge (up to a maximum €25 a month);
High Technology Scheme (HTS): covering mainly oral high cost drugs that are prescribed in hospitals but dispensed in the community.
Hospital scheme: covering IV drugs prescribed and dispensed in hospitals.
While there have been a series of agreements since the 1990s, HTA officially came into effect in Ireland with the 2006 IPHA agreement. Under the agreement, the National Centre for Pharmacoeconomics (NCPE) evaluates HTAs on behalf of the HSE and after review recommends if the technology should be reimbursed. The primary outcome of an HTA is cost effectiveness as measured by the incremental cost effectiveness ratio (ICER).
Since 2009, RRs are required for all new drugs and are a preliminary examination of the clinical evidence and the budgetary impact of the new drug to assess whether a formal HTA is required. RRs are also reviewed by the NCPE and, according to them: “
Following a review of the RR, which takes 2-4 weeks, the NCPE either recommends a full HTA or not. According to Schmitz et al (2016), ‘
For HTAs, a cost-effectiveness threshold acts as the decision rule supporting a reimbursement recommendation. Specifically, the NCPE were operating under an explicit cost-effectiveness threshold of €45,000 per quality adjusted life year (QALY) from 2006 to 2010; it was reduced to €20,000/QALY between 2010 and 2012 and, under the 2012 agreement it was €45,000/QALY. Furthermore, the agreement indicates that products with ICERs above the €45,000/QALY threshold that are considered exceptional may be considered in discussions with HSE, relevant stakeholders and marketing authorization holders (MAH) (7) and proceed to be reimbursed following negotiations. Also, an ICER below the threshold does not result in automatic endorsement. For example, if there are outstanding validity concerns surrounding the cost-effectiveness analysis, a negative recommendation may result (7). Thus, according to Schmitz et al (7), the threshold is flexible in practice and there is no explicit list of relevant criteria or their influence on the decision available limiting transparency. There are no such decision rules supporting the requirement for a full HTA or not.
Methods
Data
All evaluations of drugs from 2009 up to July 2015 were extracted from the NCPE website (vaccines and devices were excluded). These dates were chosen because RRs came into effect in 2009 and July 2015 was the cut-off used in Schmitz et al (7) and thus provided a source of validation and a point of comparison. The NCPE website provided all the data for the products evaluated through the HTA pathway; this includes therapy area, reimbursement scheme, year, reimbursement recommendation. This information was not available for the drugs evaluated through the RR pathway because, in contrast to HTAs, a summary report is not published for these drugs. As such, various sources were used for the data supporting this pathway. Specifically, the NCPE website was the source of the HTA recommendation and year of recommendation (9). In the absence, of any other information on this site regarding the eventual reimbursement of drugs subject to RR only, we assumed that those that were not recommended for further assessment were reimbursed. This assumption was based on the description of the process in Schmitz at al (7) and the lack of information on the NCPE website to suggest that RR only drugs were not automatically reimbursed.
Furthermore, the PCRS website was the source of the reimbursement scheme, while the World Health Organization ICD-10 classification was used to categorize the drugs into therapeutic areas (10). Finally, the US Food and Drug Administration (FDA) annual report on novel drugs was used to classify drugs as first in class. This indicates that they are medicines that offer a new and unique mechanism of action for treatment (11).
For the comparison with Scotland and England and Wales, the number of single technology assessments (STAs) from January 2009 up to July 2015 and corresponding reimbursement recommendations were abstracted from the Scottish Medicine Consortium and NICE websites, respectively.
Methods
This analysis seeks to examine if there are differences between the medicines requiring RR only and those requiring a full HTA and if so what are the influencing factors. The methods required for this were three-fold. Firstly, univariate analyses are used to examine differences in the two pathways in Ireland and how they compare with Scotland, England and Wales. Second, to determine the factors influencing the outcome of a RR, i.e. if a full HTA is recommended, a logit regression was estimated (using Stata 14) with binary dependent variable (1 = RR only, 0 = HTA). Independent variables included were year of assessment, if the product was first in class (1, 0), if the product was seeking reimbursement under the GMS scheme (1, 0), if the product was seeking reimbursement under the HT scheme (the reference category was the hospital scheme); and therapeutic areas. For the latter, five dummy variables representing circulatory, respiratory, neoplasms, endocrine and musculoskeletal (1, 0) (the reference category was other) were included.
Third, propensity score matching is employed to determine that there is no expected difference in outcome (i.e. in achieving positive reimbursement recommendation) between those drugs recommended for a full HTA compared with those not recommended for full HTA (i.e. RR only). Matching estimators are based on the idea of comparing the outcomes of observations that are as similar as possible with the sole exception of their “treatment status”. This research compares drugs in two pathways that have similar characteristics and establish if any differences exist with respect to their reimbursement decision. Several control variables are used (year, first in class, reimbursement scheme [GMS and HTDS] and therapeutic area), which may affect both the probability of getting a positive reimbursement decision.
Where: Y is the outcome variable (1 if a positive reimbursement, 0 if not a positive reimbursement) of individual
The coefficient β
Given Equation (1) above:
The average treatment effect (ATE) is given by:
The average treatment effect of the treated is given by:
This gives the average over the subpopulation of treated drugs of the treatment effect.
The nearest-neighbor matching (NNM) estimator was employed for the ATE and the average treatment effect on the treated (ATET) using Stata 14 and generated t scores.
Results
Analysis of rapid review outcomes
In total, 199 evaluations of drugs were conducted between 2009 and up to July 2015 (Fig. 1). All were subject to an RR and in 47% of cases an HTA was not recommended (n = 94). This is the RR only pathway, illustrated on the right-hand side of Figure 1. For the other 53%, an HTA was recommended and in 71% of these cases an HTA was completed. This is the HTA pathway, illustrated on the left-hand side of Figure 1. In total, there were 151 positive reimbursement decisions (including reimbursement following price negotiations), suggesting a 76% positive reimbursement rate overall (HTA and RR) for the period (this is assuming a 100% reimbursement rate following an RR only (7)). When the drugs for which an HTA was requested but not submitted (n = 30) are included, the reimbursement rate decreases to 54%.
Submissions to NCPE 2009-2015. Data from (9).
Most drugs (n = 199) evaluated were for consideration in the GMS scheme (40%), followed by hospital (32%) and HTS (28%). With regard to therapeutic area, 20% were neoplasms, 14% circulatory, 10% endocrine, nutritional and metabolic, 9% respiratory and 9% musculoskeletal (the remaining 38% were for all other areas). 31% of all drugs were first in class. Table I provides descriptive statistics on the reviewed drugs.
The main differences between drugs that went down the RR only and HTA pathways were in relation to reimbursement scheme, whether the drug was first in class and therapeutic class. Specifically, amongst the drugs that went down the RR only pathway (n = 94), 50% and 20% were recommended for reimbursement under the GMS and HT schemes, respectively. The equivalent figures for the HTA pathway were 35% and 35%, respectively. With respect to therapeutic area, 4% of RR only drugs were neoplasms and 5% were in the musculoskeletal class. The corresponding figures for the HTA pathway were 33% and 13%. Finally, 20% of the drugs that went down the RR only pathway, were categorized as being first in class compared to 40% for the HTA pathway.
An analysis of the drugs for which an HTA was recommended but not completed revealed that they were more likely to be hospital drugs, in the cancer area and less likely to be first in class compared to those for which an HTA was completed.
Comparison with Scotland, England and Wales
The number of HTAs recommended (n = 105) and completed (n = 75) is less in Ireland compared to Scotland (n = 355) (12) and England and Wales (n = 120) (13). The comparative analysis also shows that when all evaluations are accounted for, the overall positive reimbursement rate (with and without restrictions) are similar in Ireland (76%) to that of England and Wales (80%) and greater than in Scotland (68%). Figure 2 illustrates these findings.
Factors influencing rapid review outcome
The results of the logit regression (Tab. II) reveal that if the drug is classified as being first in class it is statistically significant and decreases the log-odds of requiring an RR only. Similarly, if the drug is indicated for the musculoskeletal system (or connective tissue) therapeutic area, it is statistically significant and decreases the log-odds of requiring an RR only. Likewise, if the drug is a neoplasm it is statistically significant and decreases the log-odds of requiring an RR only. The year in which the assessment was conducted and the scheme under which reimbursement is sought are not statistically significant. The results of the logistic regression suggest that drugs that are first in class and indicated for the musculoskeletal system and neoplasm are more likely to require a full HTA.
Logit regression results
RR = rapid reviews; SE = standard error; GMS = General Medicines Scheme; HTS = High Technology Scheme; Coeff. = coefficient.
Statistically significant at 5% level.
Matching results
Propensity score matching, using the NNM technique, was employed to determine if there was a difference between the drugs considered for RR only and full HTA. The analysis revealed (t = 6.143) that there is a difference in outcomes (i.e., achieving a positive reimbursement recommendation) between drugs for which an RR only was completed compared to where a full HTA was completed. Thus, it appears that the RR stage is an efficient filter, had those RR only drugs been recommended for HTA the outcome would have remained a positive reimbursement.
Conclusions
There is a derived demand for HTAs from manufacturers, physicians, patients and decision makers who want timely access to safe, effective and cost-effective medicines. As healthcare budgets come under increasing strain and the pace of technology grows, pressure to deliver and review HTAs in an efficient time frame increases. For example, from HTAs formally coming into effect in Ireland in 2006 and 2008, the number of HTAs rapidly increased. As a means of ensuring timely decisions, RRs were introduced in 2009. The RR is a preliminary look at the budget impact and likely value for money of a drug to determine whether an HTA is required. If the drug is not likely to significantly add to the drugs budget and likely to be value for money, an HTA is not required and is automatically reimbursed.
This study examined the factors influencing the outcome of an RR, i.e. if a full HTA is recommended or not in Ireland and in combination with Schmitz et al (7) gives a complete picture of the reimbursement process in Ireland. Specifically, in the absence of guidelines or thresholds for decision making surrounding the need for a full HTA, this analysis revealed that the therapeutic area (specifically neoplasms and musculoskeletal system) and being first in class, increase the odds of a full HTA being requested. While Schmitz et al (7) show that cost effectiveness and quality of evidence influence a positive reimbursement recommendation following an HTA. Combined, these studies show that the likelihood of a drug being recommended for reimbursement is greater if they are not in the cancer and musculoskeletal areas, not first in class, have good quality evidence and demonstrated to be cost effective. These results are expected as medicines that are first in class and in the oncology and musculoskeletal areas tend to be high cost and have a big budget impact.
The effect of the RR process is that scientific rigor is reserved for the high-cost medicines that are driving growth in the medicines budget (14). At the same time, access to medicines for patients is not compromised, as demonstrated by the broadly similar positive reimbursement recommendation rates between Ireland and Scotland and England and Wales.
Furthermore, although all new medicines undergo an RR in Ireland, only half of them require further assessment via an HTA. Thus, the RR has the pragmatic effect of reducing the number of HTAs required and optimizing resources within the NCPE as shown by the fewer HTAs conducted here compared to Scotland and England and Wales. This suggests that a system which includes an RR can be considered efficient and potentially cost effective, avoiding duplication of HTAs in drug classes where an HTA for the first in class medicine has already been considered.
The benefits of this extend beyond the NCPE. Shortening times between submission and reimbursement can get products to market faster benefiting manufacturers and patients. While medicines reimbursed without an HTA are less likely to be first in class, they may be more affordable because manufacturers are incentivized to decrease the price at the RR stage in order to avoid the time-consuming HTA process. Indeed, McCullagh and Barry (5) have recently shown that in 25% of cases, a lower price was negotiated with the manufacturer following an RR to avoid a full HTA. Furthermore, the latest Framework Agreement on the supply and pricing of medicines, between IPHA and the HSE, which was published in July 2016 (15) now has an explicit option for price negotiation prior to a full HTA, officially endorsing the advantages of early price negotiations.
It is also reassuring to see from the Framework Agreement (15) that the RR is now embedded in the reimbursement process, further strengthening the benefits of RR. However, there is a need to improve transparency on the criteria that influence the decision on whether an HTA is required or not, given that it is an integral part of the reimbursement process. Although, McCullagh and Barry (5) refer to three criteria (robust clinical data, small eligible population, and low-budget impact) that are used to determine whether a full HTA is required or not, this is not explicit and not laid out in the Framework Agreement (15) or in the reimbursement process (8). The new Framework Agreement (15) now contains a multilayered reimbursement decision process following an HTA, taking budget impact and cost-effectiveness thresholds into account to determine the level at which the reimbursement decision is made in the HSE. A similar process may be useful at the RR stage.
One of the limitations of this study is that the analysis is based on publicly available data, which may not reflect reality due to withholding of information for confidentiality reasons. Specifically, some negative reimbursement recommendations following an HTA may have been subsequently overturned following negotiations with the manufacturer on price and reimbursement conditions. We have controlled for this as far as possible given publicly available data. However, this may explain why the overall positive reimbursement rate in McCullagh and Barry (5) of 79% is slightly higher than the 76% found in this study. In addition, McCullagh and Barry (5) showed that in 8% of cases reimbursement was not recommended following an RR only, whereas we have assumed that all drugs subject to an RR only are subsequently reimbursed in the absence of other information on the NCPE website. Another limitation, is that we have not included all of the factors that influence whether an HTA is required or not in the regression equations, such as budget impact and high cost, because this information is not publicly available. However, the ‘first in class’ variable and therapeutic area, which were statistically significant, were used as proxy indicators for high cost and budget impact, respectively to overcome this limitation.
Despite their advantages, it is evident that RRs need to be used carefully and deliberately. Particularly for drugs that are not first in class, we need to ensure that not doing an HTA does not erode any potential cost savings which could have been captured had cost effectiveness been considered. Furthermore, former HTAs for the drug that was first in that class may have been informed by data which had substantial uncertainties, subject to a learning curve and is now outdated etc., which may affect the potential cost effectiveness of new drugs.
This case study of Ireland demonstrates that RRs are a useful way to approach the increased burden on health system decision makers and when well executed can be efficient. Nevertheless, there are improvements that could be made in the Irish case, for example more transparency around the decision rules and outcomes, inclusion of some cost-effectiveness analyses (even secondary through systematic literature review).
Footnotes
Financial support: No grants or funding have been received related to this study.
Conflict of interest: Sandra Redmond is an employee of GlaxoSmithKline.