Abstract
Introduction
Chidgey, Leng and Lacey 1 review the successes and some outstanding issues facing the UK National Institute for Health and Clinical Excellence (NICE) in improving the routine delivery of care in the National Health Service and similar organizations. The focus of their essay is the NICE programme of guideline development, ‘arguably the largest in the world’, whose objective is to carry out rigorous reviews of the evidence for alternative treatments, develop clinical guidance that ensures clinical decisions are based on the best evidence, target use of resources optimally and maintain ongoing reviews of new evidence. Chidgey et al. point to convincing examples of successes in changing practice, while accepting that overall the record of translating guidance into successful implementation is ‘mixed’.
Editorials in the same issue of JRSM discuss this assessment. Iain Chalmers asks ‘How are we to know whether all this work [on guideline development] has led to better patient care in the NHS?’ Gupta and Warner observe that ‘although NICE guidelines can be very helpful in guiding clinicians and patients, they focus on the clinical problem and cannot take into account various other factors (e.g. physical and psychological co-morbidities, social and cultural issues) that make each patient unique’. Finally the editor of the JRSM, Kamran Abbasi, comments that ‘Guidelines have a miserable record in changing clinical practice’ and ‘The central problem for NICE is that too many clinicians view its rationing role with displeasure and each decision and guidance brings its own enemies. … Clinicians feel increasingly isolated from the decision-making process and increasingly resentful of the inflexibility of high level commandments.’
Chidgey et al. suggest a number of ways to improve guideline implementation, ranging from educational and outreach mechanisms to the use of clinical audit and reminders and ‘computer-based decision support’. In this paper we wish to pick up the last remark because it was only mentioned in passing and we believe that Clinical Decision Support (CDS) technology actually offers a major new implementation strategy. Specifically, we wish to draw attention to ways in which CDS can make a significant contribution to the effective dissemination of evidence-based practice. Furthermore we will argue that decision support technology can address a number of issues raised in the editorials: providing clinical guidance in a form that is specific to individual patients; permitting and indeed supporting the exercise of professional clinical judgement if this conflicts with general guidelines, and involving working clinicians in the translation of research into practice. This is not a commentary on NICE, or the UK, but about a strategy for improving the quality and safety of patient care in modern medical services.
We will explain these claims in the context of a long-term research programme supported
by Cancer Research UK (CRUK) which, while focused on cancer, has resulted in techniques
for supporting clinical decision-making which are believed to be applicable across
clinical specialties, sectors and countries. CDS technology is a rapidly developing
field.
2
We have not attempted to write a detailed review of different approaches to CDSs
for this paper because several already exist.
3,
4,
5
Short overviews of many systems can be found at
In the first section of the paper we briefly overview international efforts to develop and disseminate evidence-based clinical practice guidelines (CPGs), their perceived impact on clinical practice and the issues that have emerged. 2 Next we describe advances in the field of decision support, drawing particularly though not exclusively on our own experience. The last part of the paper focuses on how CDS technology can address challenges in implementing clinical guidelines and help clinicians and others to comply with evidence-based recommendations. A novel capability that CDS services can offer is to capture clinicians' experience in using a CPG, thereby informing the treatment policies of the healthcare organization, and feeding back information about CPG use and impact to the authors and reviewers of guidance and clinical researchers.
Practice guidelines and decision support
Cancer Research UK has supported work on theory and applications of CDS technology for more than 20 years. The clinical focus has been primarily in oncology but the techniques have been widely used by individuals and organizations in other clinical specialties. 6 The motivation for research in this area arose from CRUK's early recognition of variability in quality of cancer care in the UK, together with wide acceptance of the need to establish treatment protocols and guidelines which would facilitate more consistent practice based on secure scientific foundations.
Over this period issues of medical error and patient safety have also come to the fore. The US Institute of Medicine's influential report To Err Is Human created awareness internationally that medical error is a major cause of avoidable mortality, morbidity and inappropriate use of resources. Studies by Vincent in the acute sector of the UK National Health Service 7 and McGlynn et al. 8 in the US, among others, have clearly demonstrated major shortcomings in service delivery and the incidence of avoidable adverse events across medical specialties. With the increasing recognition of shortcomings of healthcare systems, practice guidelines came to be widely advocated as a means of summarizing and encouraging compliance with evidence-based practice, and the last decade has seen the growth of the ‘guidelines movement’ throughout medicine.
A typical CPG is a text document (paper or electronic) which contains a summary of recommended clinical practice for a specific condition together with the rationale and supporting evidence. CPGs may include ‘clinical algorithms’ in the form of flowcharts to be followed in appropriate situations, though despite the computer-inspired notation these are usually intended for humans to follow. Major repositories of practice guidelines are now available in the USA (e.g. National Guideline Clearing House 9 ), the UK (e.g. NICE 10 ) and internationally (e.g. the Guideline International Network, G-I-N 11 ). These indicate a significant cultural and professional shift with great potential for improving quality and safety of clinical practice.
While these efforts are producing an impressive and important body of material there are major questions to be addressed about whether they lead directly to improvements in clinical practice. Although there is evidence that guidelines can improve clinical outcomes 12, 13, 14 there are grounds for concern that the potential created by the enormous effort that goes into creating them may not be matched by the level of adherence to them in practice. 15, 16, 17, 18 Chidgey et al. acknowledge this in the context of the NICE programme and Chalmers observes that ‘the available evidence suggests that there is plenty of room for improvement in the difficult area of changing clinical practice’. We now review a number of obstacles to changing clinicians' behavior by means of CPGs, before turning to a discussion of how decision support can help to mitigate these difficulties.
Pros and cons of practice guidelines
CPGs have their roots in issues that face most healthcare systems: variability in provision of clinical services; rising costs fuelled by new and more expensive technologies; an ageing population and increased demands for care, together with the intrinsic desire of healthcare professionals to offer, and of patients to receive, the best possible care. In reality we still see over-use or under-use of services and wide variations in service quality among providers, hospitals and geographic regions, and we know that at least some of thisvariation stems from inappropriate care. 19
Despite the seemingly uncontroversial objectives of CPGs, they have frequently attracted fierce criticism and resistance from clinicians. Various studies have identified objections that are commonly raised regarding the value and usability of conventional practice guidelines.
Inaccessibility of guidance at the point of care: Intended users are often unaware of the availability of CPGs 20, 21 and even when they are aware, the guideline recommendations are embedded in lengthy documents that are not easily accessible or practical to use in the clinical setting. 22
Absence of patient-specific guidance: CPGs cannot deal with the
specifics of each case because their authors cannot foresee the details of every
possible clinical situation in which they may be applied.
23
The conventional text CPG suffers from:
Over-simplification: Most CPGs address single diseases while
in reality, especially with an ageing population, patients have multiple
co-morbidities making the application of single disease CPG recommendations
more difficult;
24
Ambiguity: Many guidelines are written in an imprecise way and
may lack clarity about their recommendations;
25
Weak evidence base: Sometimes there is no consensus among
specialists about the appropriate approach due to lack of clear evidence.
26
Clinicians are more likely to follow guideline recommendations based on
higher grades of evidence but this is often lacking;
a
Patient values and goals: Guideline recommendations are based
on a ‘prototype’ patient and do not provide explicit support for incorporating
patient values or preferences into decision-making.
Inapplicability in local settings: Lack of resources such as personnel, skills or equipment, and organizational constraints or cost may mean that a CPG developed by a national or international organization is not helpful in a local clinical setting. 27
Long lifecycle of guideline development: New medical knowledge is continuously generated and, in certain areas like cancer, the pace of change is fast. However, it takes time and resources to update conventional guidelines and unless they are promptly revised to reflect new research can hinder, not foster, improvements in quality of care. 28
Lack of active user involvement: As Iain Chalmers points out (op cit) ‘… doctors have a professional responsibility to help address uncertainties about the effects of treatments’ and the explicit call in the NHS Plan (2000) for hospitals and trusts to ‘identify “procedures that should be modified or abandoned and new practices that will lead to improved patient care”’. Yet conventional CPGs provide little opportunity for involvement by clinicians, so much so that doctors frequently perceive them to be a threat to their autonomy. 29
Absence of tools to assess the impact of CPG: Chalmers also notes the General Medical Council's expectation that ‘… doctors have a professional responsibility to help address uncertainties about the effects of treatments’ yet the current guideline development lifecycle does not provide appropriate tools to assess their impact on clinical practice. Audit processes that rely on written medical records may under- or over-estimate what really happened to a patient especially if documentation is poor. 30 Conventional paper-based audit and feedback methods are only modestly effective 31 and are difficult to maintain on a continuous basis.
Chidgey et al. describe efforts by NICE to promote implementation, and the many tools it makes available to support implementers. These include practical advice on implementation barriers and ways of overcoming them, audit criteria and costing templates, education tools and commissioning guides. Valuable as these methods probably are we believe CDS services represent an important new option for delivering guidance at the point of care. They offer patient-specific advice while allowing the clinician to exercise professional judgement where guidance appears inappropriate, reporting the reasons for deviating from them and contributing information about alternative options into the review process.
From clinical guidelines to decision support services
The recognition that healthcare services all over the world are falling significantly short of the highest quality standards is now complemented by a body of evidence that decision support services can make a contribution. Among many services they can offer are: summarizing patient data; providing alerts and reminders; retrieving and filtering information which is relevant to a specific decision; and weighing up the pros and cons of clinical options in a patient-specific way. Recent systematic reviews of trials of CDS applications are encouraging.
Garg et al. 32 reviewed 100 published trials of simple CDSs such as alerts and reminders. The review showed that 64% of applications produced significant improvements in decision-making. Kawamoto et al. 33 carried out a review of 70 systems with similar results (68% were successful). In addition Kawamoto analysed the primary success factors and found that when four particular design features were all present 94% of the applications produced significant improvements in decision-making (e.g. the guideline must be on computer not paper, and it should be automatically available and not wait for the clinician to request it).
More complex CDS techniques also appear to have considerable promise. For example an
assessment of the pros and cons of alternative treatments can be given together with
various styles of evidence-based justification. Our own trials have consistently shown
the potential of such services, in primary, secondary and tertiary settings.
34
Applications which have been trialed to date include family history and genetic
risk assessment tools;
35,
36
interpretation of medical images;
37
prescribing, in general practice
38
and in specialist treatment;
39
test and treatment selection;
40
and personalized treatment planning.
41
All indicated substantial improvements using various measures of decision
quality. While much of this work has been in oncology CRUK has developed a generic
approach which has been used by ourselves and others to develop applications for general
practitioners (e.g.
The evidence seems clear that computerized decision support systems can add value to traditional documentary guidelines by actively offering evidence-justified, patient-specific advice at the point of decision-making.
Closing the loop: from decision support to clinical guidelines
‘Uncertainties about treatment effects can almost never be eliminated; but they can often be reduced by further research to an extent that facilitates decision-making’ (Chalmers op cit). Chalmers' point is that even when guidelines give clear advice the production and aftercare processes should be part of a closed loop from clinical research into clinical practice and back again, so that the experience of working clinicians can contribute directly to guideline maintenance, and even into the clinical research process itself. He is speaking about NICE guidelines here but his point is directed at the guidelines movement generally. This section discusses how CDS may offer a further significant opportunity for improving clinical practice by feeding back experience using CPGs in the clinic into continuing development of the guideline and clinical research.
The conventional guideline development process, or lifecycle, is ‘top down’ with the guideline authority typically having responsibility only for studying available evidence on a clinical topic and developing and publishing the guideline document with regular (if infrequent) reviews. On this conventional publishing model there is little or no opportunity to monitor experience in using the CPG in the clinic or feed this back into ongoing reviews and guideline improvements. A practical mechanism for achieving these goals would allow clinicians and healthcare service organizations to become active participants in the guideline development lifecycle.
In order to illustrate such a mechanism we will assume the form of decision support
described earlier. Here the CDS determines a set of clinical options
for the clinical decision being taken. For each option it then compiles all the
arguments for and against the option which are relevant and valid
for the individual patient and makes an assessment of the options in terms of whether
they are acceptable or unacceptable, identifying the most preferred options ( (a) a CDS application in breast cancer. A screen showing decision options for
the imaging for one case, to be taken after medical history and examination.
The system recommends an ultrasound scan but recommends against mammography and
against doing nothing. For the decision option ‘do an ultrasound of an affected
area’, a supporting argument has been expanded to show the justifying evidence
(this is an option available to the clinician for all decisions, options and
arguments). Links are provided to the relevant supporting literature, which can
be accessed by the user if required (e.g. from PubMed); (b) The system allows
the user to reject all the arguments of the same grade or lower and at the same
time requests reasons for rejections. Here all arguments deriving from
recommendations of grade D or lower are rejected and the system will update the
suggestions accordingly. The information is captured and will be available for
further processing by the system
If a clinician wishes to exercise personal judgement during the decision process it is possible to interact with the displayed information in various ways, the most important of which are below.
If the clinical user considers that a CDS recommendation is inappropriate in some
respect, s/he may respond by: rejecting the recommended option or its supporting arguments. In this case the
user may be prompted for the reason for the rejection (e.g. quality, relevance
or type of evidence) which is recorded in the patient's record and/or a
clinical audit database. The CDS may then offer a revised recommendation based
only on the acceptable options and arguments; selecting an option other than the one recommended by the CDS. Again the user
would be prompted to provide a rationale to be included in the patient record
and recorded for future review; introducing a new option not included in the set of options considered by the
CDS (and presumably not considered in or even excluded from the original
guideline) together with a rationale for introducing this option (e.g. absence
of local resources to implement other options).
Otherwise the clinician accepts the suggestion offered by the CDS, though not uncritically since the arguments and evidence for every option are always available and can be investigated in depth should there be any uncertainty about appropriateness or relevance.
Whatever the outcome, the decision can be monitored and captured in a database or (electronic) patient record. The decision record can also be accompanied by the rationale as approved or amended by the user. In principle a clinician can act in similarly critical ways when using a paper CPG, but the key difference is that the decision support service is interactive and many aspects of the decision process can be automatically monitored and recorded. The decision may result in an appointment or the placing of a clinical order or some other automated service but in addition the records can be used for clinical audit and research, providing data for the healthcare organization to assess its practices and the guideline development organization to reflect clinical experience in its guideline refinement and revision processes (Figure 2). b
Overcoming the shortcomings of conventional CPGs
Three interaction loops for clinicians using CPGs within CDS services. Loop 1:
Guideline publishing and feedback – represents the flow of guidance from the
guideline authority to the point of care and back. The top-to-bottom part is
similar the traditional development and maintenance of paper guidelines; the
bottom-up part represents feedback to the authority about use of the guideline
in practice; Loop 2: Organizational adaptation and local feedback – represents
organization-specific guidance and feedback. A CDS can be customized to meet
organizational requirements and constraints, thereby requiring a distinct
feedback and monitoring process; Loop 3: Clinician assessment and record –
deviations from guidance, if any, are recorded. Clinician modifications to the
argumentation (e.g. rebuttals or additions) are taken into account in the CDS's
subsequent advice to the clinician and recorded for later review, for
self-audit and/or research
Once the knowledge that a guideline contains has been formalized it can be used to interpret patient data in the clinical setting, and to address a number of the barriers to acceptance summarized above.
Inaccessibility of guidance at point of care: Decision support can be conveniently delivered in many ways and styles in the clinic. The service may be accessed using a desktop computer or, increasingly, a hand-held PDA which is wirelessly linked to a hospital server or a wider network providing interactive access to a national or other decision support service. The clinic computer can send encrypted or anonymous patient data to the service which will respond with a summary of options in a succinct and structured form (Figure 1). An Internet connection will permit the clinician to access the guideline developer's web site in order to access the original CPG, the systematic review or the research studies on which the CPG is based, or carry out wider searches if required.
Absence of patient-specific guidance: CPGs are designed to summarize general principles of good, evidence-based practice for defined groups of patients. In order to use CPGs as they are intended, a clinician must not just commit the principles to memory, but also judge when they are relevant (and when not) and adopt, adapt or ignore the guideline as appropriate. As Gupta and Warner comment 46 a recommendation in a general guideline may be inappropriate due to ‘physical and psychological co-morbidities, social and cultural issues’ but also for reasons of patient preferences (e.g. the cancer patient who values increased survival over poor quality of life due to an aggressive treatment, and the patient with the opposite valuation). A suitably designed CDS can take such factors into account when formulating its options and provide a patient-centred rationale in the form of personal pros and cons. Another approach is the ALCHEMIST 47 decision support system which is based on statistical decision analysis and allows users to incorporate personal utilities.
Inapplicability in local settings: If a practice guideline is inappropriate in some respect due to local circumstances (such as institutional policy, lack of specialist equipment or budgetary constraints) it can be adapted and refined to include additional arguments for and against the various options. Alternatively a clinician can simply override a recommendation, as described above, but in this case a brief record of the rationale for this decision should be made for subsequent policy review, ethicolegal and research purposes.
Long lifecycle of guideline development: Current guideline reviews and publishing processes are time-consuming and, due to weight of work, infrequently revised. Since new research can become available at any time a CPG may easily get into disrepute, even if it is relatively recent, or preferred practice has only changed in a small way. Again, the interactive style of CDS may be useful here in that clinical users can be given the option of annotating an argument to indicate that it is outdated or otherwise questionable. There are several possible ways of exploiting this, from simply indicating the presence of the annotation for consideration by other users on subsequent occasions, to automatically generating an alert to the guideline developers that an issue has been flagged. Where the annotation is accepted and a modification is required changes to the medical logic of the CDS may be quite localized and approved without waiting for the scheduled CPG review date.
Lack of active user involvement: Some kinds of decision support services, which provide alerts for potential drug interactions and service reminders for example, are relatively easy to implement and known to be effective. 32, 33 However, as with conventional guidelines such mechanisms do not address situations where the clinician is doubtful about the appropriateness of advice or wishes to refine the clinical question or restrict the options to be considered. The evidence-based approach outlined above mitigates this key problem because it empowers clinical users to exclude decision options, arguments based on poor grades of evidence, and even the criteria which are deemed relevant in a particular clinical decision. The user can reject recommendations or require the CDS to refine its suggestions and supporting justifications in response to such exclusions.
Absence of tools to assess the impact of CPGs: A computerized decision support system may not only give advice but for every case in which it is used it can also record the advice offered and the rationale for it, the user's clinical decision with any amendments to the rationale and, with patient tracking software, c the clinical outcome when it becomes known. It can furthermore capture many different kinds of data that can be used with statistical software and other ‘data mining’ technologies to analyse the impact of a clinical practice guideline: what recommendations were made? When were they accepted and when rejected, or modified? What was the rationale given when recommendations were rejected? Are there significant patterns in the adoption of the guidance? And so forth. The clinicians then begin to have a role in contributing to the organizational memory and policy development since these data can be fed back to guideline developers and publishing organizations.
Discussion
In his JRSM editorial, Kamran Abbasi remarks that ‘the problem for NICE, as with many national organizations, is that clinicians feel increasingly isolated from the decision-making process and increasingly resentful of the inflexibility of high-level commandments’. We have argued that CDS technology not only provides a new and flexible way of delivering patient-specific clinical guidance, it also opens new communication channels for feeding back information about the use of the guidance, into CPG maintenance and clinical research which builds on rather than ignores clinician experience.
An example of an authoring, editorial, publishing and maintenance process for
active decision support and guideline services
Figure 3 shows the publishing lifecycle that we
have developed for the creation, dissemination, open review and revision of active
decision support services in the OpenClinical project (
The objective is that a suitably qualified clinician who uses a clinical guideline in practice should be able to benefit from the enhanced services made possible by CDS technology and participate in the clinical assessment of the recommendations formulated in the source guideline. Following Chalmers we believe this will not only support greater compliance with an evidence-based policy, it will also permit monitoring of the use of the CPG in practice. In due course this will lead to reduced uncertainty about the effects of the clinical interventions that the guideline is concerned with, more refined guidance and improved quality and safety of care. The feasibility of closed loop feedback has been technically demonstrated (Figure 1) though requires evaluation in a large-scale trial.
Closing remarks
About the time that NICE was set up, one of us contributed an essay on ‘Computers, decision making and clinical effectiveness’ to a UK Institute of Public Policy report Rethinking IT and Health. 48 The thrust of the essay was that many decision-making services can be provided by computers and that these could and should be evidence-based so this might be an area that NICE could have responsibility for in due course. At the time there were significant technical obstacles to the deployment of CDS technology in the clinic but in the intervening years many of those obstacles have been taken down.
Clinical decision support is now a hot topic 49 with healthcare organizations in the UK and other countries identifying it and associated technologies as critical for future medical services. 50 The American Medical Informatics Association has published a ‘road map’ for their successful development and deployment 51 and major texts covering the subject are in print (notably the volume Clinical Decision Support: the Road Ahead, edited by Robert A Greenes. 52 Connecting for Health, the UK body overseeing the development of the IT infrastructure for the NHS, requires its lead suppliers to support the delivery of decision support products and services in due course and major medical publishers see decision support as a new and important market for reusing much of their archive of conventional published material. So all is apparently set for a major take-off of decision support, with the potential – unique in medical research we believe – that a major technical breakthrough is in progress which could improve consistency, quality and safety of patient care throughout medicine, and support healthcare professionals in their wishes to disseminate good practices and participate in the development of policy.
Footnotes
DECLARATIONS
Footnotes
Acknowledgements
The authors would like to thank Sir Iain Chalmers (Oxford University) andMor Peleg (Stanford University) for many helpful comments on an earlier version of this paper
a
As Gupta and Warner note in the case of the NICE guideline for management of depression: 'Only four of the 32 recommendations in the full guidelines were based on grade “A” evidence (i.e. supported by at least one adequate quality randomized controlled trial), and the majority of recommendations were supported by grade “C” recommendation (i.e. based on expert committee report or respected opinion).
b
An earlier approach was demonstrated in the NewGuide system
" href="#bibr-b44" id="a-69">
44
which allows clinician users to deviate from a CDS' recommendation and choose
alternative actions that are related to the system's recommendation (selected from a
hierarchy of SNOMED-CT terms) and the clinician is required to write comments
explaining the deviation. In the approach described here we demonstrate how the
clinician can interact with the medical logic underlying the decision, resulting in a
different recommendation.
c
A Google™ search for ‘patient tracking software’ identifies various products of this type.
d
According to Wikipedia a passive decision support system is a system that aids the process of decision-making, but cannot bring out explicit decision suggestions or solutions in contrast to active DSS which can. The term was applied by Michael Stein to the Map of Medicine as it is essentially a software tool for navigating between CPGs and other documentary sources of clinical guidance on the web.