Abstract
INTRODUCTION
The modern hospital with its growing IT infrastructure provides fertile ground for the development and exploitation of clinical decision support systems (l). The combination of the increasing power and sophistication of information systems and the rapid development of networks spanning the clinical domain presents the laboratory with unprecedented opportunities to expand its role into the clinical arena. In particular the increasing interdependence of clinical and diagnostic services makes this area a focus for the early implementation of such systems. To date the main emphasis of information systems in the laboratory environment has been on control of analytical systems and administrative support of operational processes (2,3,4). Support for the use of analytical results in clinical decision-making, apart from a few isolated examples such as in thyroid test interpretation, has generally been a manual process exploiting the skills of senior medical and scientific staff.
This short paper will explore some of the issues involved in the provision of automated clinical information support drawing on examples of existing systems. It should help readers to fully understand the requirements for future systems and be in a better position to exploit such systems to enhance the performance of their laboratories.
BACKGROUND
The practice of medicine is an inherently decision-based process. Though there are undoubtedly areas where manual skills are important, as in surgery and areas of interventive radiology, for the most part medicine is a cerebral activity. Data gained from clinical examination and investigative procedures is combined by the physician and used to inform decisions about diagnosis or future management (5). At a higher level, aggregation of data on individual patients is used epidemiologically to inform policies. This happens at both the level of an individual clinician's practice and at the institutional level. The synthesis of such data together with that gained from clinical trials serves to inform strategic decision-making on a regional, national or global basis. Thus, at all levels we can see the primacy of decision making in clinical activity a fact now well represented in the development and expansion of the creed of ‘Evidence-based Practice’ (5, 6).
THE DIAGNOSTIC PROCESS
The diagnostic process can be thought of as an iterative cycle (Figure 1) in which clinicians form hypotheses based on clinical examination of the patient and order investigations to rule in or rule out diagnoses. The cycle is iterative in that the information gained by each testing cycle feeds forward into the next cycle informing the selection of subsequent tests (Figure 2). In acute disease states the cycle may turn only once or twice and for sake of speed many tests may be ordered in parallel. In chronic disease states the cycle may continue indefinitely, though under those circumstances ongoing monitoring and management of disease are the goals rather than the generation of diagnostic evidence.
If viewed in this way one can identify many decision steps which would be open to the potential of decision support systems and indeed within the automated laboratory many of the internal systems are in place. (Table 1) As yet the use of systems outside the laboratory sphere, i.e. in steps 1, 2 and 6, is not well established in the majority of hospitals.
DEFINITIONS
In considering clinical decision support it is important to be clear about the definition and purpose of a decision support system (2,3,4,7). To many people such systems are synonymous with so-called expert systems. However, this is too restrictive and narrow a definition as decision support tools need not be computer-based. However for the most part they are computer orientated and most exploit the computer in one way or another. The following definition from the Harvard Business School Review provides for me the best generic description of the qualities of a decision support system. It rightly emphasises the powerful synergy between the power of the computer to handle data and the human to evaluate it.
“Decision support systems couple the intellectual resources on individuals with the capabilities of the computer to improve the quality of decisions. It is a computer-based support system for decision makers who deal with semi-structured problems.”
Keen & Scott-Morton Harvard Business Review 1984.
In contrast the following widely used definition of an expert system concentrates on the computational aspects alone and does not consider the entire context of use.
“An expert system embodies in a computer the knowledge based component of an expert's skill in such a form that the system can offer intelligent advice and on demand justify its own line of reasoning.”
Donald Mitchie, 1982
Of course expert systems are widely used for decision support but many of the best computer-based decision systems do not use this technology at all (7).
As an alternative to the computational definitions of decision support systems I prefer to consider them from a functional standpoint, in particular drawing a distinction between those which offer passive as opposed to active support (1,8). I define Passive systems as ones that manipulate data or information in ways which make the appropriateness of decisions more apparent to human users but that in themselves do not dictate the action to be taken. In these terms a computer generated, paper-based graphical presentation of analytical results can be viewed as a decision-support aid. In contrast active support systems are those which dictate the outcome and leave no role for the human decision-maker, other than in the definition of the rule set and the range of possible outcomes. Table 2 illustrates some examples of systems broken down in this way. Those in the passive category can be considered ‘open-loop systems’ whereas those in the active category generally are ‘closed-loop systems’. It is important to recognise this as on the whole the closed loop systems are required to perform to more exacting standards since the element of human evaluation is removed from the decision cycle. However, in general, the increase in performance achieved by passing the responsibility for decision making to the system justifies the effort involved in engineering systems to this level. It should be noted however that this does not necessarily imply that passive systems can be built to less exacting standards. Firstly, performance gains are just as important and hence optimisation is still a key factor in the development process. But perhaps more importantly, though designed for passive use such systems can be used in a close-loop manner if the human user never questions the outcome. If the user, often a relatively unskilled user, is unaware of potential deficiencies such usage can become extremely dangerous. Passive systems when introduced into the clinical arena need therefore to be robustly developed and their use rigorously controlled.
Examples of different types of Decision Support Systems in current clinical use.
SPECIFIC SYSTEMS
There are hundred of examples in the literature of decision support systems, which have been developed in the clinical domain. What is striking however is how few are used in day to day clinical practice. Some have been extremely successful. Almost all electrocardiograms taken in the developed world are run on instruments which provide automated, expert-system reporting and by the mid-1990s more than 100 million ECGs per day were being routinely reported through decision support aids (9). There are many reasons why decision support systems have been less successful elsewhere probably the most major being the difficulty of transferring the technology between users due to portability barriers of conventional IT platforms. However with the growing openness of systems and the establishment of standards of data access using modern languages, SQL databases and the growth of network protocols. This is likely to change.
There are many good listings of exiting decision support systems (see for example http://www.acb.org.uk/Technical%20Affairs/ITG/decision_support/decision.htm) and here I will draw attention to a small selection merely to emphasise particular issues.
TELEGASTRO
The Telegastro system can be regarded as one of the milestone systems in medical decision support (10). Though it is based on a relatively simple Bayesian algorithm its importance lies in the fact that it was the first clinical decision support system to be subjected to a full randomised controlled clinical trial. Indeed it is possible that it remains in a unique position in this regard. The system is designed to aid in the diagnosis of acute abdominal pain and using very simple laboratory and clinical data is able to easily perform as well as the expert surgical specialist. In a trial of 16,000 patient episodes it produced an improvement in initial diagnostic accuracy from 45% to 65%, reduced unnecessary operations and perforations by 15% and saved 8000 bed-days of hospitalisation (11). Importantly its use clearly reduced the number of avoidable deaths from this condition. In respect of the fact that it provides gold standard evidence of effectiveness it should be seen as one of the seminal developments in the field alongside better known systems such as MYCIN (12) and INTERNIST (13).
LUMPS (Liver Unit Medical Protocol System)
LUMPS is a system designed to provide clinical protocol-driven control of test ordering. It works by creating a diary of tests for a patient based on their clinical diagnosis, which can be modified interactively on the basis of results or changes in clinical status (14). The system has been thoroughly tested in the field of the care of patients under-going liver transplantation and benefits were achieved in terms of the quality and cost of care. Though the numbers of tests were not reduced through the use of the system the appropriateness of their use increased and the need for stat testing fell by 40%. The major benefit was in the saving of medical staff time amounting to a 1 hour per day average reduction in time spent on laboratory tasks by junior physicians. This raises the interesting issue of activities sustained by the laboratory which bring benefit to users which may not be rewarded by increases in laboratory revenues and therefore the need for laboratories to define their role not in terms of test generation but in terms of broader-based clinical service provision.
PEIRS
PEIRS (Pathology Expert Interpretative Reporting System) appends interpretative comments to pathology reports (15, 16). The knowledge acquisition strategy is the ‘ripple down rules’ method, which has allowed a pathologist to build over 2300 rules without knowledge engineering or programming support. New rules are added in minutes, and maintenance tasks are a trivial extension to the pathologist's routine duties. PEIRS can generate interpretative comments for a wide range of reports including thyroid function tests, arterial blood gases, glucose tolerance tests, hCG, catecholamines and a range of other hormones. Its strength is in the fact that it captures the pathologist's own behaviour in a self correcting and adaptive way and can become an extremely personalised tool. The downside is that the rule set can never be formally proven which makes it difficult to see how a rule set could be transferred easily from site to site.
VALAB
Valab was developed as a rule based system for clinically validating sets of analytical data (17). As well as checking datasets against conventional reference ranges and action limits it includes sophisticated delta checking and auto-correlation techniques to identify result sets which may be aberrant and warrant further inspection by pathologists prior to reporting. It is available as a standalone system, which can be used as an add-on to a LIMS. It has found limited success for widespread use as the rule set is not transparent and the benefits of the system over the validation performance of many conventional LIMS have been difficult to prove.
PROMD
The Pro.M.D. is a development system for laboratory medical experts to formulate their expert knowledge, experience and problem solving strategies (18, 19). The result of this process is a knowledge base of rules defined in a structured (formal) medical language that can be used to create knowledge driven programmes for laboratory medicine test result interpretation. Several such subsystems have been developed in the areas of thyroid interpretation, liver function testing and Epstein Barr virus test evaluation, and several are in routine use as test report generators. The system was initially developed in Prolog but has been recoded such that it can be implemented on general computing platforms and can be used to add functionality to conventional laboratory systems.
HELP
The HELP system (Health Evaluation through Logical Processing) has been developed in Salt Lake City and comprises a protocol driven system that complements the conventional hospital information system (20). Its role is to monitor the conventional process of test ordering and reporting providing alerts to warn of inappropriate measures and rules to constrain inappropriate testing and suggestions to guide clinicians towards more appropriate actions. One example of its use is that when a drug that can potentially impair renal function is ordered the LIMS system is interrogated to establish that patient's renal status and the pharmacist is alerted if another drug may be preferred. The laboratory benefits through access to the patients' drug regimens particularly if drugs are known to interfere with certain laboratory tests. The system has been made more generally available bur the investment in rule generation at the scale of an entire hospital means that the effort required for full implementation is large and it is doubtful the benefits of the system can ever be fully realised. However, when considering the scale of spending on drugs targeted implementations in the control of therapeutic interventions can provide for good cost benefit outcomes.
BARRIERS TO USE
Given the success of these systems, what steps might lead to greater uptake of systems on a widespread basis? It is important to try to identify the barriers which have to date prevented their widespread implementation since by identifying such barriers some possible directions of future development might be more apparent (4, 8).
IT PROBLEMS
Embedding decision support systems in conventional laboratory information systems has undoubtedly produced many good algorithms and evidence of utility. However this monolithic approach has not led to propagation of even the most successful systems. One possible direction of change might be to adopt a modular approach to the provision of decision support thereby separating the decision support element from the central LIMS functionality. In this way it might be possible to create a plug and play scenario when individual best of breed DDSs could be attached to a LIMS providing for highly focussed support for particular problems. My own group has adopted such an approach for Downs Syndrome Screening Calculation and we have developed a calculation engine which can be called by a client database to estimate risks on the basis of clinical and biochemical data (21). The communication is managed through a database interface reminiscent of an analyser interface. Taking this approach one could view the DDS as downstream laboratory analysers where the input is a dataset as opposed to a biological specimen and the output is diagnostic information rather than analytical results (Figure 3). One obvious challenge is to create an interface standard along the lines of the HL7 interface standard which would truly allow plug and play approach to the selection of the most appropriate decision support system for a particular clinical function.
PROFESSIONAL ISSUES
There are also many professional barriers to change. In the late 1980s professionals viewed these systems with great suspicion partly because of the threat to professional status. If computers could be more accurate than doctors could then how come doctors could still command respect and high salaries! With time this view has changed. We do not expect planes to be flown without computer support and there is no reason why we should expect doctors to perform unsupported. The information explosion in medicine is now recognised and with the increasing demands on the time of medical professionals any help is now generally received in a positive light. However there are still many problems to be solved as regard the validation of systems and of professional understanding of what constitutes acceptable performance of these systems. Computers are generally expected to be infallible but decision support systems are based at some level on statistical procedures. Hence they will never be 100% accurate. They are often then unfairly compared to human decision-makers since humans are known to be fallible and can be excused mistakes where as a single computer error is viewed as failure. A better comparison is to look at the marginal levels of improvement that can be achieved with computers. We need to recognise that experts do not take most decisions in medicine and that even experts in some areas perform sub-optimally. The de Dombahl abdominal pain diagnosis system was not perfect but it instantly brought performance standards of all novice users up to that of experts. The marginal improvements in performance were highly significant.
Two other areas that I will highlight are the need to develop methods to make comparative evaluations of decision support systems and the need to provide methods of professional accreditation, QA and licensing. If in the future a modular approach is taken to decision support application then laboratories should be provided with a choice of which system to adopt, entirely analogous to the freedom to choose the best instrument or method for a particular analysis. Just as there are scores of published examples of systems for thyroid function test interpretation algorithm one can envisage a situation where for any decision process several alternative decision support aids will become available. Of these some will clearly be better engineered and will perform better and we will need robust methods of comparison, akin to those for analytical methods, in order to select those best suited to a particular purpose. Similarly if we are to place reliance on such systems one can see a coming requirement for them to be accredited through a body such as the FDA and for there to be ongoing QA of their performance, especially true of systems where local customization is included. The French Agency of Medicine has recently embarked on a National programme of Downs' Syndrome Screening and has far-sightedly undertaken a programme of accreditation of the calculators used in the risk assessment process. This could be an important indication of what the future might hold for these systems. At present the field is so new such agencies lack the expertise to undertake this function on a widespread basis and the tools for evaluation are crude. This presents an interesting challenge to professionals in the field.
CONCLUSION — POTENTIALS FOR DEVELOPMENT
There is no doubt that decision-support systems will play an increasingly important role in the future of medicine. Enough evidence is available to show that they can significantly enhance the diagnostic and therapeutic performance of doctors and nurses without threatening their professional status. Furthermore they can do this in ways which can provide huge savings of scarce resources, both of time and material and can thus contribute to alleviating the pressures on health resources worldwide. However there are many challenges in coupling this technology into the current systems of health provision since their use requires changes in many of the business processes currently in place. Laboratories are already in the vanguard of use of such systems and being amongst the most informatics literate of the health-care organisations currently have great opportunities to be involved in the extension and implementation of decision support systems into the clinical world.