Abstract
The National Institute for Health and Clinical Excellence (NICE) underwent a Health Select Committee (HSC) Inquiry in April 2007, the second since its inception in 1999. The inquiry was set up to examine the reasons why NICE's decisions have witnessed increasing challenges and whether public confidence in NICE has waned. Consequently the focus was on the technology appraisal programme and its mandatory recommendations, although its clinical guidelines programme came under scrutiny too. We appraise the challenges now facing NICE in light of the HSC's conclusions, 1 the subsequent government response to the HSC's recommendations 2 published in March 2008 and the recent Darzi report. 3
The HSC report is exhaustive, running to over 100 pages, and draws upon submissions from over 116 different groups with witness testimony from over 30 representatives including patients, clinical and non clinical health professionals, senior NICE representatives, Department of Health representatives and a Health Minister. The task of the HSC was formidable – to investigate the function of what is essentially a regulatory agency by evaluating feedback from those whom it regulates. The challenges facing such an inquiry are obvious – the submissions are from self-selected groups, many of whom are complainants, and the inquiry aspired to report objective findings to improve NICE processes.
Despite these challenges, the inquiry has largely succeeded in identifying and differentiating the ‘special pleading’ of vested interests from genuine issues. We review the findings of the inquiry in relation to (1) the current role of NICE, (2) the role of health care professionals and their representative organizations, (3) the role of patient interest groups, (4) the role of the pharmaceutical and devices industry, (5) NICE, the media and public perception and (6) guidance development and implementation.
The current role of NICE
Resource allocation is inherently controversial as all needs cannot be met within a financially constrained healthcare system and there is public expectation that healthcare should be universally and comprehensively available to all – a still prevalent view amongst some professional groups. The HSC has acknowledged the allocative decision maker's role and recognized that the UK population is ageing and healthcare costs are rising. In particular drug expenditure has been identified as one of the largest contributors to increased costs. 4
The underlying remit of NICE, to address challenging issues and justify often difficult and emotionally fuelled decisions, represents a justifiable appropriation of public funds. Recommendations are inevitably not always beneficial to all parties and those recommendations that do not ‘bite’ might reflect impotence on behalf of the regulator. A recognition that NICE's processes and decisions will inevitably be subjected to criticism underpins the entire HSC report and sets a high evidence threshold for the inquiry that separates the ‘signal’ from the ‘noise’ within the many self-selecting submissions received.
The HSC commented that ‘we became increasingly aware that the job of NICE is much more difficult than was first imagined’. NICE may increasingly finding itself in the invidious position of regulating the availability of healthcare technologies while the relative availability of resources decreases due to the widely expected slowdown in NHS spending from 8% of GDP towards the long term average of 3%, 5 coupled with demographic change and an epidemiological transition, a issue recognized by the HSC. Consequently, the long-term trend for allocative decision-making could be perceived as increasingly restrictive due to the need to adopt lower cost-effectiveness thresholds arising from increased demands on the healthcare budget.
Healthcare professionals and clinical practice
NICE relies on multidisciplinary and multispecialty panels that include patients, generalists and specialists, supported by a broad consultation process to ensure a balanced view. Based on the English legal system, the process utilizes non- specialist panels that hear evidence from specialist groups and other stakeholders and assesses evidence provided by manufacturers of technologies, usually with independently commissioned evidence assessment. Though the work of its committees is inquisitorial, it is inevitable that some stakeholders may adopt an adversarial approach. The lack of inclusion of experts was a frequent criticism from many respondents during the inquiry; however, to understand how healthcare professional groups may perceive NICE, one must first understand the sociology of medical specialties.
Clinical care has traditionally been guided by professional groups, which apart from the Royal Colleges have often been poorly integrated into the NHS and take little account of the consequences of their recommendations beyond their single specialty focus. Specialty guidelines are derived differently from NICE guidance with no mandatory assurance from conflicts of interest, multidisciplinary compilation or systematic appraisal of an entire evidence base. 6, 7, 8, 9, 10, 11 Single specialty panels are also more likely to demonstrate a pro-intervention (“optimism”) bias when asked to appraise the same evidence as a multidisciplinary panel. 12, 13 ‘Expert’ opinion thus does deserve a place within an evidence pyramid – as part of the evidence rather than ‘the’ evidence.
The HSC heard and rejected a number of suggestions for alternatives to the decision making process used in NICE's technology appraisals. These included recruiting experts on the basis of their publication record – thereby restricting input to academia as opposed to those whose expertise resides in providing direct clinical care. Another was to allow participation by experts whose pharmaceutical commissions are below some arbitrary percentage of their gross income – a proposal certain to lead to accusations of bias. Finally, populating NICE's appraisal committees only with experts was also suggested.
The HSC has taken a less iatro-centric view and concluded that NICE should appoint non-voting clinical advisors to support its technology appraisal committees. It is noteworthy that the HSC linked this recommendation with the potential to improving uptake and this duality of purpose, rather than a need for improved evaluation per se, supports their conclusion. In their response the government has identified stakeholder engagement as being key to involving expert opinion and regard the issue of whether or not use specialist advisors as being an operational matter for NICE. Maintaining the integrity of the NICE process would require that advisors are free from potential financial and academic conflicts of interest. Although their input may provide some fine discrimination in complex evaluations or in situations where there is a small degree of pivotal ‘uncertainty’ in a health economic evaluation, major changes in recommendations made would be unlikely.
A previous inquiry that examined the influence of the pharmaceutical industry recommended that a register of interests be instituted but this has never been implemented. 14 Transparency around potential competing interests, both fiscal and academic, is essential to achieving a level playing field and it is disappointing that the HSC has not pursued the lack of implementation of its own previous recommendation – a missed opportunity.
The HSC also asked NICE to examine disinvestment opportunities in more detail and cites data to suggest considerable potential here. However, close review of this data reveals that only 4% of commonly used treatments are thought ineffective or harmful, 6% unlikely to be beneficial and 46% are of unknown effectiveness. 15 This suggests a conflation of absence of evidence and evidence of absence and we believe that adopting this as a priority may also be wasteful given the modest evidence for disinvestment opportunities. The HSC's observation that clinical guidelines provide an opportunity to consider disinvestment opportunities across an entire pathway of care may be limited by the potential optimism bias of specialty focused guideline groups. The government has indicated that the efforts made by NICE to tackle disinvestment must be considered in the context of its other business priorities.
The role of patient interest groups
NICE's processes are designed to have the characteristics of equity of voice and a high standard of transparency yet also operate an appeals system that may be freely used by third parties with the potential to further self-interest over the public good. The first two of these observations have been complemented by an objective review from the World Health Organization that evaluated methods and processes within the NICE technology appraisal programme and declared them to be sound 16 while a recent independent report from the Office of Fair Trading (OFT) similarly reported positively and proceeded to further recommend an extension of its powers to undertake drug pricing. 17
Despite considerable criticism from many patient groups and an ongoing judicial review of NICE's recommendation on Alzheimer's drugs during the inquiry, there was considerable praise for NICE and recognition of its role in ensuring the rational allocation of healthcare resources. In our experience as members of a patient advocacy group, NICE has allowed a level playing field to address south Asian health issues, in contrast to the situation a decade ago, ‘when access was more or less zero.’ Resource allocation is no longer a case of ‘he who shouts loudest’. 18 It is unsurprising that the HSC has not made any direct recommendations that NICE modify its approach to stakeholder engagement as there was considerable evidence of engagement, irrespective of agreement.
The key lesson here is that voluntary sector agencies should dutifully champion a particular cause but also accept that their interests should not disadvantage other groups or be perceived as detrimental to general population welfare. However, engagement is a two way process and difficulties arise when stakeholders do not demonstrate their patient advocacy role in an even handed manner as is expected of any group genuinely committed to seeking benefits for a particular patient group. Criticism of NICE from patient groups is rarely, if ever, accompanied by simultaneous lobbying of industry (frequently the sponsors or ‘friends’ of charities) that could result in access to medicines through instantaneous price re-adjustments that represent fair value to the NHS. This anomalous behaviour is a matter of concern.
The role of the pharmaceutical industry
The pharmaceutical industry has to ensure its products receive recommendation for NHS use that allows as widespread dissemination as possible. With drug development costs of approximately $800 million 19 and the cost of challenging NICE's decision within an appeal being relatively small, a reasonable and appropriate strategy is for industry to challenge unfavourable decisions from NICE as frequently as necessary to seek competitive advantage. This is perceptibly an Achilles heel of NICE and the potential for ‘gaming’ of the appeals system was accepted as a genuine dynamic by the HSC. The commercial factors which motivate promotion of some drugs beyond their optimum use have also been described in the recent Office of Fair Trading Review of the PPRS Scheme. 17
The HSC also heard several concerns from industry and patient groups around the use of health economics, quality adjusted life years (‘QALYs’) and the specific cost-effectiveness range employed by NICE to determine value for money. Described as a ‘dark art’, a similar analogy could be drawn between health economics and clinical trials research as the latter is rooted in the theory of logical induction, based on statistical inference, essentially probabilistic in nature and ultimately refutationist. Ultimately, there is no practical alternative and at the present time, ‘the perfect is the enemy of the good’.
The need to validate NICE's cost-effectiveness threshold was identified as being of serious concern. Arguments were put that a more permissive threshold be employed (from manufacturers) and also in support of a more conservative threshold that reflects the resources available within the NHS to fund new technologies (from health economists and testimony from NHS groups). Health economic modelling was submitted that suggested that that in 2004/06, for a PCT with average needs and expenditure, the marginal cost of a life saved was £13,100 in cancer and £8000 for circulatory disease. Adjustment for QALYs suggested a cost/QALY of £19,100 for cancer and £12,000 for circulatory disease. A tentative conclusion was drawn by the authors that ‘the cost of a QALY secured in these programmes compare favourably with the threshold of £30,000 employed by NICE’. 20
Whilst seductively simple, these analyses can overlook important observations such as the contribution of secular lifestyle trends to reductions in coronary heart disease mortality which account for a significant part of the decline seen since 1992 and are difficult to attribute to PCT programme budgeting. Adjustment for these effects might diminish the impressive cost-effectiveness of these programmes. Estimating a cost-effectiveness threshold for priority setting based on historical levels of PCT spending that reflect poor financial management 21 and delayed uptake of recommended technologies 22 also risks institutionalizing local inefficiency within national priority setting – the antithesis of quality improvement.
Acknowledging that NICE is commissioning empirical research, the HSC has recommended that an independent body be constituted to set the cost threshold. An independent process could ensure that the integrity with which NICE undertakes its central role is separated from more peripheral arguments around affordability per se. The HSC has accepted calls for wider societal costs such as those relating to disability or employment to be more fully incorporated, though the modelling of such costs becomes increasingly challenging as the remit of NICE widens and of course the allocative ‘cake’ is still the same size. The government sagely note that the implications of such an approach would be the privileged economic value attributed to improving the health of working age people over those who are young, old or not in employment. This might be seen as entrenching the pursuit of healthcare provided for its own sake, however this is a normative societal view and in a free society these issues should be resolved by social consensus.
In their response, the government have endorsed NICE's efforts to set the QALY threshold on a more scientific footing through commissioning further research. This research will look at the monetary value that society attaches to particular health gains and issues such as whether QALY's should receive a greater value for children as opposed to adults. There does not appear to be any enthusiasm for establishing an independent body to determine a cost-effectiveness threshold, with the government citing the interplay between the threshold and NICE's assessment methodologies and the cost implications of establishing another body that would ultimately only have similar evidence available to it. Regardless of who is ultimately responsible for determining the threshold, it should be based upon empirical research, however arbitrary that may be.
Implementing the OFT Report recommendations 17 to enable NICE to define the maximum price the NHS should pay for novel technologies might circumvent the evident diffidence of some stakeholders to place pressure on their sponsors by emphasizing the need for drugs to be priced affordably in the first instance. The HSC inquiry recommends that NICE undertake evaluations before drug launch using a lower threshold to increase the speed at which guidance is produced. Issuing guidance at the point when data is most sparse would be challenging but the experience and proficiency gained may provide a valuable platform for value-based drug pricing in the future. The government response to this suggestion has been muted, not least because of their intention to review the PPRS scheme. However, the difficulty in undertaking complex evaluations and the use of a two-tier system of cost-effectiveness thresholds and possible legal challenges are pragmatic concerns voiced by the government that cast considerable doubt on the feasibility of this specific approach.
NICE, the media and public perception
Medical trade publications often report divisive headlines with respect to NICE. However there is evident support for NICE's role expressed by prominent UK medical journals such as the BMJ and the Lancet which have both endorsed the rigour of NICE's methodology and called for the government to lead a public debate around rationing. 23, 24
Public confidence – that is the view of the integrity with which NICE makes decisions (regardless of whether a particular party finds them agreeable or not) – is a distinct issue from public perception. The HSC has focused considerable attention on this, the central issue in its enquiry, and has differentiated between the appropriateness of NICE's decisions in relation to its remit (value for money) as opposed to a populist perception of NICE based on the number of positive recommendations it makes. Nevertheless, opinion polls do suggest that public confidence in NICE is satisfactorily high. 25
It is satisfying that the HSC is explicitly clear that more collaborative efforts should be undertaken across the health service to support NICE's role. The perception of political pressure preceding NICE's appraisal of herceptin had a negative impact and the commitment given by the HSC, Health Minister and government to maintaining a commitment to the independence of NICE is invaluable.
Guidance development and implementation
A detailed WHO review in 2006 identified several important opportunities to develop NICE's guideline methodology. Developmental areas include improving the systematic approach to gathering, using and interpreting evidence. 26 Tools for reducing variability should not be sources of variability in themselves as this reduces their utility for quality improvement purposes. This is particularly important given the proposals within the Darzi report for the greater use of NICE guidance or guidelines as a foundation for clinical quality improvement. 3
The speed of guidance production is often frustrating for service users. 27 This statement is tempered by the realization that there was little robust guidance at all before NICE was created and that previous HSC recommendations that NICE receive additional funding have yet to be implemented.
Comparison between NICE and the work of the Scottish Medicines Consortium (SMC) are inevitable. The SMC issues guidance at the time of launch and its decisions are produced within approximately six months. Although costs are not known they are likely to be less than the £250,000 cost of a NICE appraisal. There are important differences too. The SMC does not operate the same standards of ‘accountability for reasonableness’ in terms of publicity (there is no scoping or public consultation process), reduced opportunity for revision (arising from a lack of public consultation) and it does not operate a formal appeals process.
These aspects of NICE's process establish the legitimacy of NICE in the social context. As one of the major themes raised during the HSC was the issue of transparency it seems implausible that reducing NICE's accountability for reasonableness in order to expedite decisions for the considerably larger pharmaceutical market in England and Wales could be successful. Such an approach would, we believe, be seen as overly technocratic and fuel the increasing perception of a democratic deficit in public administration. 28, 29 This issue also highlights the importance of not only scientific evidence in considering policy issues but also important epistemological differences between research evidence, organizational evidence (‘what seems to work’) and political evidence (‘what is acceptable in the wider social context’).
A study in 2004 suggested that implementation of NICE guidance was variable and more likely to be adopted when there is strong professional support; a stable and convincing evidence base, and no increased or unfunded costs. 30 Further research from 2005 covering 28 appraisals suggests the situation is improving: 12 appraisals were implemented fully, 12 were incompletely implemented, and four over-implemented. 21 An enquiry by the Audit Commission identifies the need for improvements at local health economy level – ascribing lack of implementation to poor financial management in local NHS trusts. 21
The notion that quality improvement is an implementation issue is based on a top down perspective to improve system performance, and disregards the responsibility of the ‘meso’ organizational level of healthcare and the micro level where direct care is delivered. 31 Each level of healthcare is subject to an ambiguity of goals, conflicting interests among decision makers and inadequate use of the available evidence leading to suboptimal performance.
This ambiguity is most obvious in the confinement of cost-effectiveness to NICE's work programme while fiscal balance drives cost control within primary care trusts, 32, 33, 34 where neither evidence or cost-effectiveness based policy are embedded. Fiscal balance is influenced by many non-pharmaceutical cost drivers such as manpower costs and waste. Until efficiency and cost-effectiveness are applied consistently throughout the health economy, technologies towards the upper part of NICE's cost-effectiveness range cannot be argued as the sole reason why others are displaced. 30, 35
The HSC has recognized that NICE is only partly responsible for uptake of guidance and has identified resource, organizational and professional barriers to implementation. It has recommended more involvement from primary care trusts and experts; more help for PCTs to implement guidance, better assessment of guideline uptake (after finding low verification of self assessment by PCTs) and that funding for technology appraisals no longer be ring fenced outside the payment by results tariff. Significantly, it has also recommended that elements of guidelines now also become mandatory and following the adoption of the WHO recommendations highlighted earlier this might become feasible.
The Association of British Pharmaceutical Industries has also called for guidance to be made mandatory. 36 The HSC has partially responded by recommending that technology appraisal guidance be renamed as ‘directives’ to differentiate them from guidelines which act as developmental standards. Though this may reduce ambiguity there remain key practical implementation issues at organizational level that need to be addressed and autocratic terminology such as ‘directives’ is unlikely to foster better engagement. The government has noted that such terminology might also imply that the status of technology appraisals might be seen to incorrectly over-ride clinical judgement in the treatment of individual patients.
The post-Darzi future
The HSC recommendation that NICE move towards conducting early stage appraisals carried the widest process implications of the 32 recommendations in the report. 37 This has been partly adopted, albeit following release, for the most ‘important drugs’, as an important priority in the recent Darzi report. 3 The particular level of theNICE cost-effectiveness threshold was a key issue identified within the HSC report and this awaits further research, to be published later this year. On the issue of more public support for NICE the government acknowledges that ‘in some cases’ more can be done to improve the transparency and clarity of its recommendations and indicates that NICE will plan to hold its Appraisal Committee meetings in public. Whether this alone will be sufficient to address the fundamental tensions in managing resources in a tax funded healthcare system seems improbable and may in fact have the opposite effect unless there is unambiguous public support for NICE’s principles as set by the government in its establishment directives for the institution.
Technical considerations aside, it is inevitable the focus should eventually move from structure and process to outcome, with clearer linkage between clinical policy and its practice, as the greatest gains lie not in the methodological refinements of guidance production, but in their translation into improvements in patient's health status. The HSC report highlights the findings of a verification audit undertaken by the Audit Commission that found that the rate of adoption of technology appraisals by PCTs was 25%, in contrast to figures of 84.6% submitted by PCTs themselves. 1 This supports Darzi’s proposal 3 for the establishment of an NHS constitution guaranteeing patient access to clinically indicated and approved drugs, albeit through the safety net of consumer demand should organizational delivery fail.
The Darzi report’s 3 proposals for the increased use of quality metrics and standards based on NICE guidance should ensure that the scope of unmeasured healthcare quality is reduced: self-reported data should be avoided and a wider array of measures that recognize the need for organizational support may need to be considered. Valuable, too, is the opportunity to exploit the synergies between the initial steps for identifying valid process measures and the process for evidence synthesis that underpins guideline development.
The NICE fellowship programme also recommended within the Darzi report is a welcome step towards local implementation and is most likely to be effective if it is integrated within a policy framework that reflects the evidence base for guideline implementation or other quality improvement initiatives (e.g. disease management) that have shown modest effect sizes. 3, 38 This is also an opportunity to promote clinical engagement within the framework of quality improvement.
Meanwhile, the government should reflect on its own opportunities to promote evidence based policy and consider the concerns of Wanless et al who noted that ‘the failure by government to evaluate policies that absorb billions of pounds contrasts strongly with the efforts made by NICE to evaluate new healthcare technologies’. 39 The barriers to undertaking evaluation are only partly due to the epistemological immaturity of evidence-based medicine; 40 after all, measurement is as fundamental to laying the foundations for improving quality as it is to conducting evaluations. Pilot programmes undertaken by NICE, such as the Patient Safety Solutions Project, have shown that well-defined and discrete technical interventions can be addressed within current methodology, though admittedly more work needs to be done in quantifying the value of improved access and patient satisfaction measures as noted by Wanless. 39
Conclusion
NICE has emerged from its review with its status and fundamental processes entrenched – the locus of contention has now irreversibly moved from the ‘why’ to the ‘how’.
The Darzi report effectively underwrites the government’s commitment to NICE, and although NICE has still to face the ‘acid test’ of a change in government, cross-party support 41 suggests a reasonably secure future. The welcome progression from process to outcome poses a challenge for NICE in maintaining its exceptionally process-orientated and unitary organizational culture while simultaneously extending its activity through a pluralistic healthcare system. That NICE has effectively been chosen to lead the drive for clinical quality comes as no surprise, despite earlier indications that a de novo ‘Health Innovation Council’ might be established.
The first decade of NICE’s existence can be summed up by meeting the challenge of ‘survival’ and its future work programme is likely to focus exclusively on its impact. Recent reforms promise to address, at least in part, the key three organizational barriers of conflicting interests among decision makers, ambiguous goals and insufficient use of the available evidence. 31
The challenges faced by NICE should not be forgotton or under-estimated. It was established on a national fault line where societal, economic, professional and commercial tectonic plates abut. It has undergone two health select committee inquiries, one high court judicial review and been subject to ritual opprobrium by the media. Following these experiences, one might reasonably conclude that its greatest achievement is that it is still there.
Footnotes
DECLARATIONS
Footnotes
Acknowledgements
None