Abstract
Background:
Various health technology assessment (HTA) agencies review new medical devices worldwide, and their recommendations can be useful in guiding clinical decision making. However, different agencies use different processes and methodologies, resulting in variation in recommendations.
Objectives:
The objectives were to review full HTAs for a new technology for diabetes management, flash glucose monitoring (FGM), with the aim of summarizing similarities/differences in processes, methodologies, and recommendations from the perspective of everyday clinical practice.
Methods:
A literature review was conducted using online HTA resources.
Results:
Four full HTAs were identified (Canary Islands, France, Catalunya, and Norway); one issued a conditional recommendation for patients with type 1 diabetes mellitus (T1DM) with controlled glycated hemoglobin (HbA1c) (Spain; Canary Islands), one issued a broader recommendation for patients with T1DM and T2DM (France), and two reported that there was insufficient evidence to support a recommendation (Spain [Catalunya] and Norway). The most comprehensive and stringent of the available HTAs were those in the Canary Islands and Norway, which included systematic literature reviews (SLRs), consultation with patient groups and clinicians, GRADE evidence quality assessments, and full economic models. Comprehensive HTAs either did not recommend FGM (Norway) or restricted the recommendation to a small subpopulation of the overall diabetes population (Canary Islands).
Conclusion:
HTAs represent a valuable additional resource for clinicians to consider alongside clinical evidence, guidelines, and consensus papers; however, interpreting recommendations requires an understanding of the processes behind these recommendations. In this review, comprehensive HTAs either recommended for a selected subpopulation based on RCT evidence or found insufficient evidence for a recommendation.
Introduction
New medical devices are constantly being developed, increasing the available options for patients. However, innovation alone is not sufficient to warrant the use of a new device in clinical practice, and any new technology must deliver additional clinical value for patients and health care systems (at an acceptable cost) to gain clinical acceptance.
Medical devices are regulated by the Food and Drug Administration (FDA) in the United States 1 and via the CE mark in the European Union. 2 The processes for each agency primarily focus on the safety of a device, and initial clinical trials to gain regulatory approval may therefore have a limited focus. 3 As design and quality of post launch trials can vary widely, interpreting the data from medical device trials, and the relevance of their data to daily clinical practice, can therefore be challenging.
Clinicians routinely review and critically appraise the evidence available for new technologies, using resources such as clinical trial publications, clinical guidelines, and consensus statements. Additional resources that can help inform clinical decision making are the assessment documents and guidance produced by health technology assessment (HTA) agencies, which can provide a broader perspective than published clinical evidence alone.
Essentially, HTA is often used to answer a fundamental question: should health care systems fund a new technology? More technically, HTA refers to the systematic evaluation of the properties, effects, and/or impacts of a specific health technology, and is a multidisciplinary process to evaluate the social, economic, organizational, and ethical issues of a health intervention or health technology. 4 HTAs are conducted at a national (eg, UK, France) or regional/decentralized level (eg, regions in Italy, Spain), or both, depending on the processes in that country. HTAs are conducted by specialized, dedicated agencies such as the Haute Autorité de Santé in France or the National Institute of Health and Care Excellence in the UK. These agencies may conduct full assessments, which are a comprehensive review of all available evidence in line with the standards published by each agency, or limited assessments which are less comprehensive but are quicker to complete. Typically, the manufacturer of the new technology prepares a submission detailing the clinical and economic evidence supporting their product, which is then reviewed in detail by the HTA agency in collaboration with relevant stakeholders such as clinicians and patient organizations. The processes and criteria used to evaluate this submission vary between countries (and regions), 5 with some countries focusing primarily on clinical evidence, 6 while others place greater importance on economic impact. 7 In Europe, the European Network of HTA Agencies (EUnetHTA) aims to harmonize these processes and criteria, however, differences still exist.6-8 An additional factor which can influence assessments is the societal context of each country/region. In theory, HTAs are an unbiased assessment of the evidence and agencies that follow similar processes and methodologies should reach similar conclusions; however, this is not always the case and unreported human and social factors may influence their outcomes.
Following the review, the HTA agency will provide one of three recommendations: a recommendation for the full treatable population, a restricted recommendation which limits use to well-defined subpopulations, or no recommendation. Where a new technology can be used in a broad and/or large population (eg, patients with diabetes) HTA agencies will often seek to define the subpopulation in which it is of most benefit; full recommendations are therefore less common than other outcomes. The recommendation or rejection typically relates to funding for the new technology. A recommended technology will therefore (in theory) be ‘easy’ to use and fund in clinical practice. In the case of a rejection or restriction, use of the new technology may still be possible; however, it is likely that funding will have to be sought on a case-by-case basis and justification provided as to why it is necessary. For example, in Germany self-monitoring of blood glucose (SMBG) is recommended for patients receiving insulin but not for patients receiving oral antidiabetic drugs; 9 however, SMBG can still be justified from a clinical point of view for some patients in the latter segment, and will be funded if deemed appropriate. In the case of a rejection or restriction to a subpopulation, published HTA recommendations may later be revised if further evidence can be provided. Published positive recommendations are occasionally reversed depending on the reimbursement process in that country/region, although this is generally infrequent.
The most thorough HTAs therefore represent a comprehensive and in-depth evaluation of the evidence for new technologies, viewed through the prism of the factors deemed most important to each country; this can provide clinicians with an additional perspective on a new technology that goes beyond, but includes, the clinical evidence routinely considered in everyday practice. However, understanding the methodologies and processes involved in HTA recommendations and country-/region-specific preferences is highly relevant when critically evaluating their findings.
HTA is used in diabetes in both the therapeutic (eg, drugs and insulin) and devices/diagnostic (eg, continuous glucose monitoring) spaces; while HTAs of therapies are common and required for countries with HTA processes, device/diagnostic HTAs are less common and evolving processes for these mean that there is often no clear HTA pathway that new devices must follow. Two examples of recently assessed therapies (2017 to June 2018) are insulin aspart (recommended in France, Scotland, and Sweden)10-12 for adults with type 2 diabetes mellitus (T2DM), which was one of seven injectable treatments assessed in HTAs, and dapagliflozin plus saxagliptin (no recommendation in Australia, 13 restricted recommendations in Scotland and Sweden),14,15 which was one of five oral treatments assessed in HTAs. 16 HTA is consequently an important factor in determining access to new treatments or technologies. The purpose of this study was therefore to consider similarities/differences in the processes, methodology, and recommendations between diabetes HTAs from the perspective of everyday clinical practice, using a new technology in diabetes care (flash glucose monitoring [FGM], FreeStyle Libre™; Abbott Diabetes Care Inc, CA, USA) as a case study. FGM monitors glucose via a factory-calibrated, on-body sensor that is worn on the back of the arm for up to 14 days and continuously monitors interstitial glucose levels. This sensor can be scanned by a reader to provide both a real-time level and a graphical trace of glucose values for the past 8 hours.17,18
Methods
A literature review was conducted using the Centre for Reviews and Dissemination: Database of Abstracts of Reviews of Effects, Health Technology Assessment database, websites of HTA agencies (eg, NICE), and reference lists from identified HTAs available in February 2018. Further hand-searching of gray literature was also conducted. National and regional HTAs were considered. The following criteria were used in evaluating each of the identified HTAs: clinical evidence evaluated, process for identification of clinical evidence, quality assessment of clinical evidence, inclusion or exclusion of economic/budget impact models, and consultation with clinicians/patient organizations (Table 1).
Criteria Used to Assess HTAs and Their Relevance.
Results
Assessments Identified
In total, eight reviews were identified (Table 2). Of these, three were limited assessments that did not follow the standard process for full HTAs in that country (Canada, the Netherlands, and the UK)17,19,20 and one was a priority assessment representing the first of three steps in a full HTA (Italy). 21 Two of the limited appraisals provided no official recommendation on the use of FGM (Canada and the UK),17,19 while one rejected FGM for insurance coverage (Netherlands). 20 The Italian assessment was the first stage of a three-step process and issued a temporary, patient group specific recommendation for FGM (Lombardia; Italy). 21 These will not be discussed further in this article. Four other assessments could be considered full HTAs (Canary Islands, France, Catalunya, and Norway)18,22-24 applying defined methodologies and criteria and are discussed in detail. A summary of the HTAs is provided in Table 2; the clinical evidence supporting these assessments is provided in the following section, along with a discussion of each HTA.
Summary of Assessments Examining FGM.
Compared with HTA processes within each country. Limited assessments are not discussed in detail in this article.
FGM Clinical Evidence
Based on the most recent HTA (Norwegian submission), the majority of clinical evidence for FGM is based on single-arm trials (10 of 15 trials) of <2 weeks in duration which did not consider outcomes related to glucose control.25-34 These trials are consequently of limited interest when considering the clinical benefits of FGM. Three single-arm trials examined outcomes related to glucose control over 10-16 weeks;35-37 however, single-arm studies are considered a lower grade of evidence than randomized controlled trials (RCTs). 38 Two open-label controlled trials were conducted over 6 months using SMBG as the comparator;39,40 both formed the basis of the clinical evidence for all full HTAs.
The RCTs were published by Bolinder et al 39 and Haak et al. 40 Both trials were open-label studies in adults lasting 6 months and used SMBG as the comparator, with both studies using one specific Abbott blood glucose meter for SMBG. Bolinder et al (published in the Lancet) conducted a prospective, open-label, RCT across 23 European centers in adult patients with well-controlled type 1 diabetes mellitus (T1DM) (HbA1c ⩽7.5%), comparing FGM (n = 119) with SMBG (n = 120). 39 The primary outcome was the difference in change in time in hypoglycemia (<70 mg/dL) between baseline and 6 months, for which there was a significant 38% reduction for FGM versus SMBG (between-group difference −1.24 [standard error 0.239], P < .0001). Haak et al (published in Diabetes Therapy) conducted an open-label RCT in adults with T2DM with HbA1c 7.5-12.0% on intensive insulin therapy across 26 European centers, comparing FGM (n = 149) with SMBG (n = 75). 40 The primary outcome (change from baseline in HbA1c at 6 months) was not met in this study, with no significant difference observed between FGM and SMBG for the overall population (change from baseline in HbA1c at 6 months: −0.29 ± 0.07% vs −0.31 ± 0.09%, respectively; P = .8222). A summary of the key clinical endpoints assessed in these trials is provided in Table 3, of which the primary outcomes are of most relevance.
Summary of Key Efficacy Outcomes in RCTs of FGM.
indicates primary outcomes; all other outcomes were secondary.
Full Assessments: Regional Decentralized HTAs
Spain: Canary Islands
The assessment in the Canary Islands considered both RCTs along with an economic model provided by the manufacturer based on the IMS CORE Diabetes Model. Based on the GRADE system, 41 the assessment group reported that the clinical evidence for FGM compared with SMBG was of moderate quality for events of and time in hypoglycemia without worsening in HbA1c, poor quality for rate of severe hypoglycemia in T1DM, and low quality for safety and treatment satisfaction. The committee conditionally recommended FGM for well controlled adult patients with T1DM with HbA1c <7.5% and good prior adherence to SMBG based on the population in the Bolinder et al study. 39 The parameters used in the economic model submitted by the manufacturer were considered uncertain by the committee, and it was noted that more clinical and economic evidence is required before any further conditional or strong recommendations could be made.
Spain: Catalunya
The assessment for Catalunya considered both RCTs. 22 The committee stated that there were limitations to the technology and the evidence supporting its use. It was noted that FGM did not eliminate the need for SMBG, although it reduced the frequency, and that FGM alone may not stimulate the behavioral changes required for more effective diabetes management. FGM was considered to have the potential to improve treatment satisfaction and adherence; however, the high cost of the device and discrepancies in the comparative efficacy data were stated as barriers to access. In addition, a budget impact analysis was provided only for patients with T1DM testing >6 and >9 times daily. The committee therefore concluded that insufficient evidence had been presented to support the use of FGM and that no recommendation could be made.
Full Assessments: National HTAs
France
The French HTA considered both RCTs for FGM based on the manufacturer’s submission as these were not published at the time of assessment, along with the Bailey et al 27 and BEAGLE 29 single-arm studies. 23 It was stated that FGM improved individual comfort and quality of life (QoL) for patients (although no significant differences were observed for QoL in the RCTs) and the committee provided a relatively broad recommendation for patients with T1DM and T2DM receiving intensive insulin therapy and performing SMBG >3 times daily. This recommendation goes beyond the RCT trial populations (well-controlled T1DM and poorly controlled T2DM), including a recommendation for a population in which the only RCT failed to meet its primary endpoint (T2DM, Haak et al). 40 Given that no assessment of the quality of the clinical evidence nor economic evidence was considered, it is therefore possible that factors beyond effectiveness in clinical trials were considered by the committee but are not reported in the documents.
Norway
The Norwegian HTA considered both RCTs and an economic model based on the IMS CORE Diabetes Model. 18 The GRADE system 41 was used to evaluate the quality of this clinical evidence; a rating of low or very low was assigned for all outcomes for HbA1c, hypo/hyperglycemia, QoL, and treatment satisfaction. The submitted economic model was not considered sufficiently transparent or flexible, and it was concluded that FGM does not appear to provide higher efficacy, fewer adverse events, or improved QoL compared with SMBG. The manufacturer’s conclusion of lower costs being associated with FGM versus SMBG was therefore considered difficult to support. The committee consequently made no recommendation for FGM.
Summary of Full HTA Processes and Recommendations
Two open-label RCTs comparing FGM with SMBG (based on a device from the same manufacturer) exist, one in well-controlled T1DM and one in poorly controlled T2DM, with all other studies single-arm investigations; both RCTs therefore represent the best available evidence for FGM. For FGM versus SMBG, these studies reported significant improvements for some outcomes related to hypoglycemia (Table 3) and treatment satisfaction, and no difference in HbA1c and QoL.39,40 The primary outcome was not met in one RCT (HbA1c) 39 and was met in one RCT (change in time in hypoglycemia). 40
Four full HTAs have been conducted using these RCTs as the evidence basis. One issued a conditional recommendation for patients with T1DM with controlled HbA1c based on the population of the IMPACT RCT (Spain; Canary Islands), 24 one issued a broader recommendation for patients with T1DM and T2DM undergoing intensive insulin therapy and performing SMBG >3 times daily following a 1 month trial period and certain other preconditions (France), 23 and two reported that there was insufficient evidence to support a recommendation (Spain [Catalunya] and Norway)18,22 (Table 4). Only one of the four full HTAs issued a recommendation beyond the population included in the assessed clinical trials (France), which may suggest that external factors beyond the clinical evidence, and not reported in the available documents, played a part in the recommendation in this country. Overall, it therefore appears that HTA agencies (with the exception of France) do not believe that the clinical benefits shown in the RCTs (Table 3) are sufficient to provide full funding for FGM.
Summary of Full HTAs Examining FGM.
All full HTAs considered both RCTs, with only one also considering non-RCTs (France). The quality of these RCTs was not evaluated in two HTAs (Catalunya and France) and was evaluated in two HTAs (Canary Islands and Norway), with both concluding that RCT evidence quality was generally low.18,24 A summary of the quality assessments from these HTAs is shown in Table 5.
Summary of Quality Assessments of Trials for FGM as Evaluated in Norway and Canary Islands HTAs.
GRADE ratings: low = confidence in the effect estimate is limited, the true effect may be substantially different from the estimate of the effect; very low = very little confidence in the effect estimate, the true effect is likely to be substantially different from the estimate of effect.
Full economic evaluations were included in two of the four full HTAs (Spain [Canary Islands] and Norway), both of which were based on the IMS CORE Diabetes Model.18,24 Both HTAs reported that the parameters used in the economic model were uncertain.
The recommendations for FGM in full HTAs were produced in April 2016 (Canary Islands) and July 2016 (France), with subsequent full HTAs finding insufficient evidence to support a recommendation in October 2016 (Catalunya) and August 2017 (Norway). This suggests that the HTA-relevant evidence base for FGM has not evolved over time.
It should be noted that the Canary Islands and Norway HTAs followed very similar processes, with both including systematic literature reviews (SLRs), consultation with patient groups, GRADE evidence quality assessments, and full economic models, and may therefore be considered to be the most comprehensive and stringent of the available HTAs.
Discussion
All identified HTAs were based on two RCTs that showed no difference in HbA1c or QoL between FGM and SMBG,39,40 but demonstrated improvements in some hypoglycemia outcomes (Table 3) and treatment satisfaction. A variety of processes were used across the identified HTAs; of the eight identified,17-24 only two could be considered comprehensive.18,24 These comprehensive HTAs either did not recommend FGM (Norway) 18 or restricted the recommendation to a small subpopulation of the overall diabetes population (patients with T1DM with controlled HbA1c, Canary Islands). 24 Differences in recommendations between other HTAs may be due to differences in how the RCT evidence was interpreted between HTAs; it is possible that some countries considered changes in hypoglycemia to be of sufficient clinical benefit to warrant a recommendation, while other countries did not consider this sufficient and placed more weight on HbA1c outcomes, which showed no difference. In addition, the relevance of the hypoglycemia outcomes may have been interpreted differently in different countries. In both trials outcomes related to time in hypoglycemia were assessed as secondary endpoints (other than time <70 mg/dL over 24 hours in Bolinder et al), with FGM significantly favored over SMBG for different hypoglycemia endpoints;39,40 however, the authors note that these outcomes are highly interrelated and should not be considered in isolation. No outcomes assessed the impact of time in hypoglycemia on related events, such as hospitalizations. In Bolinder et al these results should also be considered in the context of the relatively high time in hypoglycemia at baseline, which was 3.38-3.44 hours for time with glucose <3.9 mmol/L (70 mg/dL) within 24 hours. 39 This highlights the need for additional studies on the new technology, such as the real-world study by Dunn et al, 42 who evaluated 50 831 readers worldwide. This study was published after the most recent HTA and was therefore not considered; however, it is unlikely that this study would have altered any of the HTA recommendations if included as it was a retrospective study with no transparency around data selection and no comparator. While an association was shown between scan frequency and HbA1c/time in hypoglycemia, data were not presented for the overall population (eg, vs baseline). Despite this, the relevance of real-world evidence is growing and future HTAs may consider such studies alongside RCTs. This raises important questions around the weight given to large, uncontrolled studies relative to RCTs.
To the best of our knowledge, a review of HTA recommendations for FGM across different countries has not previously been published. While this publication represents a relatively brief overview of a complex topic, it can therefore serve as a starting point for future research and discussions.
As with any study there are limitations associated with our research. To simplify the search strategy, only reviews/HTAs published online were considered, and regional HTAs were only considered for Italy and Spain; it is therefore possible that HTAs not available online or from regional bodies outside of Italy and Spain have not been identified. Another limitation of this review is that the health care landscape surrounding the HTA recommendation has not been considered. While HTAs should be an impartial and unbiased assessment of the evidence for a new technology, the preexisting perceptions that governments, clinicians, patient organizations, and the general public have of specific technologies and the disease in which they are to be used will inevitably have some degree of influence on the recommendation.
Future research could apply the methods described here to other diabetes devices, such as real-time continuous glucose monitoring, and compare the range of recommendations with this study. If such research were conducted, it may also be interesting to apply the framework of this article to compare HTAs for different products in the therapeutic diabetes field, such as those mentioned previously in the introduction,10-12,14,15 to consider variability between device and therapeutic HTA outcomes.
Conclusion
The most comprehensive HTAs represent a solid and objective overview of the evidence for a new technology that, along with individual assessment of clinical data, can aid clinicians in their decision making. For the technology considered in this study (FGM), two of the four full HTAs could be considered a comprehensive review of the evidence; based on RCT data, one provided a conditional recommendation for the subpopulation of patients with T1DM with controlled HbA1c (Canary Islands, Spain), while the other concluded that there was insufficient evidence to support a recommendation (Norway). HTAs represent an additional resource for clinicians to consider alongside clinical evidence, guidelines, and consensus papers; however, interpreting the recommendations from HTA agencies requires an understanding of the processes behind these recommendations.
Footnotes
Acknowledgements
The authors would like to thank Ewan Bennett of DRG Abacus for his support in preparing the manuscript.
Abbreviations
CGM, continuous glucose monitoring; EUnetHTA, European Network of HTA Agencies; FDA, Food and Drug Administration; FGM, flash glucose monitoring; GRADE, Grading of Recommendations Assessment, Development and Evaluation; HbA1c, glycated hemoglobin; HTA, health technology assessment; NA, not applicable; QoL, quality of life; RCT, randomized controlled trial; SLR, systematic literature review; SMBG, self-monitoring of blood glucose; T1DM, type 1 diabetes mellitus; T2DM, type 2 diabetes mellitus.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: MS is a full-time employee of Johnson & Johnson Medical GmbH. OS has received honorarium from Cilag GmbH Int.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by Cilag GmbH Int (LifeScan division).
