Abstract
Clinical indicators are tools used to monitor important processes in the clinical care of patients. They have been used extensively in other areas of medicine [1]. They aim to improve quality [2] and promote uniformity in patient care [3]. The Australian Council on Health Care Standards incorporates clinical indicators into its accreditation programme through a ‘continuum of care’ model [4].
A clinical indicator is used within a cycle of quality improvement. The cycle contains a number of steps: (i) the clinical indicator is defined, and a threshold marking adequate performance is established; (ii) the indicator is measured over a period of time; (iii) the indicator is reviewed in reference to the threshold; (iv) if there is consistent failure to reach the threshold, an action is proposed to improve this component of service delivery; and (v) the effects of the intervention are monitored to assess their efficacy. Therefore, clinical indicators can be useful in demonstrating the maintenance of a good standard of service, and in detecting and stimulating solutions to problems in service delivery.
Clinical indicators and database at the Royal Melbourne Hospital
Consultation–liaison (C–L) psychiatry databases have been used in a range of Australian [5–7] and international settings [8]. They are increasingly recognized as a key component of a modern C–L service [9]. Previous experience with databases has highlighted problems of administrative burden, inflexibility and problems with feedback of results.
Clinical indicators and a C–L database were developed and implemented in the consultation–liaison psychiatry service at the Royal Melbourne Hospital (RMH) in 1998 [10]. The indicators and database were applied within a system of weekly clinical review and monthly administration meetings. Their development had been stimulated by a range of factors, including the increasing use and relevance of clinical indicators in other medical and psychiatric settings; a recognition of the importance of developing objective markers of service performance; the importance of defining the nature of C–L work; and the need to develop systems that are able to detect changes in service needs.
Paramount in the development of the RMH C–L database was that it be efficient, useful, flexible and user friendly. It was foreseen that information from the database would be summarized on an annual basis. The annual data would provide a context for the interpretation of clinical indicators and a baseline for monitoring change in service function over subsequent years.
Three clinical indicators are associated with the time taken to see a referral in different parts of the service. The first indicator measures the time to see a referral in the emergency department. The threshold was met if 90% of cases were seen within 4 h. For general C–L, the threshold was met if 90% of patients were seen within 24 h, and for specialist C–L, the threshold was met if 70% of patients were seen within 48 h. The different thresholds reflect the variations in urgency and resources in different areas of the service.
Three indicators monitor the process of communication. Communication was said to have occurred if there was oral exchange of information at referral and after the C–L assessment, or if there was oral or written communication with the service providing psychiatric follow up. The initial thresholds were set at communication occurring in at least 90% of cases.
Method
Data collection
Data was collected for each patient using a data collection sheet [10]. This data was reviewed at weekly clinical meetings and thereafter entered into a computerized database.
Review of clinical indicators and database
The progress of the system was monitored through weekly clinical review meetings, monthly administration meetings and through regular contact with the C–L secretary. The clinical indicators and data were presented at the monthly administration meetings. Comments and recommendations were minuted. After the system had been in place 12 months, the data collected and the way it had been used was reviewed. The results of this review are presented here.
Results
Data collection
The data collection sheets were found to be useful in providing a format for weekly clinical reviews and for keeping track of patient progress. The trainees described an increased burden of record keeping in filling out the sheets, each taking about 1 min to complete.
Computerized database
The entry of the data into the computerized database occurred without major difficulties. The secretary needed an orientation to the database and required a number of weeks to reach a stable level of efficiency. The entry of information relating to each patient took about two minutes. The operating platform was found to be solid, but minor flaws in the data entry screen and production of reports required attention.
Generation of reports
The monthly reports required reformatting before being tabled at the administration meeting. The database produced a monthly summary of results on a number of sheets of paper. These figures were then entered into a preformatted computer spreadsheet to produce the clinical indicator charts. This process took half an hour each month and required some knowledge of the system.
Clinical indicators
Clinical indicator 6 (Fig. 1) is presented in the same format used for the review at the monthly administration meeting. When presented at the monthly meeting, responses to individual indicators varied depending on the relationship to the threshold. If the indicator was above the threshold, discussion was often cursory. In contrast, attention often focused on those below the threshold.
Royal Melbourne Hospital consultation–liaison clinical indicator 6. ▪, threshold; ♦, clinical indicators for each month.
During the first months, a number of staff expressed uncertainty as to the definition and significance of the indicators. The definitions of each indicator were recirculated and training undertaken using case examples. As the year progressed, staff developed familiarly with the indicators and an increased understanding of the process being monitored.
Clinical indicators 1 and 3 (timeliness of assessment in the emergency department and for specialist C–L) were consistently above the threshold. These indicated consistent ‘good enough’ service delivery. Clinical indicator 2 (timeliness of assessment in general C–L) remained around the threshold, except for during the month of August, which corresponded with a change of psychiatry trainees. The trainees confirmed that they experienced difficulties orienting themselves to a new hospital. In response, the orientation process and documentation was reviewed and improved.
Clinical indicator 4 is a measure of oral communication with the referrer at the time of referral. In the early months of 1999, oral communication with the referrer occurred in less than 90% of cases. In over 10% of cases only the name of the patient, without the reason for referral, was being given. In response to this, the service instigated a policy of speaking to the referrer prior to seeing all patients, with a subsequent improvement in the measure.
Clinical indicator 5 is a measure of the feedback of the results of the C–L assessment to the referrer. This measure remained above the threshold of 90%, except during times of increased referral rate or decreased staffing. This finding suggests that increased workloads leads to a reduction in communication and therefore a decreased quality of service.
Clinical indicator 6 is a measure of communication with the service providing psychiatric follow-up, and was only applied to patients receiving psychiatric follow up. Figure 1 demonstrates the variability in performance of this measure and the difficulty in reaching the threshold. A significant number of patients were discharged without there being a handover of psychiatric issues to the follow-up agency. This was an unexpected finding. Once brought to light, communication improved over the subsequent six months, although it did remain below threshold.
Discussion
This article reviews 12 months of experience using clinical indicators at the RMH C–L service. The development and use of clinical indicators in a C–L service have not previously been reported. Overall, the experience has been positive. The use of the indicators was found to be practical and useful.
The difficulties identified with the use of the clinical indicators relate to the burden of maintaining the system. The data sheets need to be fully completed and reviewed, the data needs to be entered into the database, the results need to be formatted and presented, and actions have to be proposed and followed through. This requires the application of resources from all parts of the team, which amount to some hours per month. It may be argued that, at the RMH, the commitment to clinical indicators was derived, at least in part, from the interest and enthusiasm of building a new system. This commitment may wane with time.
The difficulties in keeping the process timely were significant, and it might be expected that similar difficulties would occur in other settings. The choice of monthly feedback was based on the standards for review of clinical indicators already in place in the hospital. It may be argued that the overly frequent review of clinical indicators, although leading to a sense of security about the functioning of a system, can outweigh their utility. In practice, two- or three-monthly measures may provide similar information, with less administrative burden, although this might increase the time lag for dealing with specific problems.
The thresholds related to each indicator are arbitrary. They were selected to reflect our expectations of good clinical practice. In a review of the area, Fauman was unable to identify any published guidelines as to where to set thresholds [3]. A particular threshold may be appropriate for one hospital, but not for another. The thresholds chosen for the RMH C–L service took into account factors of clinical need, resources and a commitment to high standards. To apply these standards more generally, a consensus between different hospitals reflecting a range of services and patient populations would be needed. In practice, a combination of generic and specific indicators would be most useful.
Clinical indicators can improve the clinical process. This improvement occurs through a number of mechanisms, namely (i) by bringing to light a problem that had previously been hidden (clinical indicator 6), which occurs at the systemic level and in individual cases as data is recorded; (ii) through specific interventions; (iii) through monitoring the effect of an intervention (clinical indicator 4). These mechanisms are practical examples of the cycle of quality improvement.
The use of guidelines for clinical practice is increasing becoming a part of contemporary medicine [11], and this is also the case for C–L psychiatry [12]. The publishing of guidelines alone does not necessarily lead to a change in practice [13]. The evidence from our use of clinical indicators is that they can change clinical practice, in the absence of guidelines. It is likely that guidelines in combination with clinical indicators is a more effective combination than guidelines alone.
Monitoring efficiency and effectiveness are essential in the context of a limited health dollar. Programs not only need to be efficient and effective, but also must be seen to be so. This is especially true for specialist services such as C–L psychiatry [8]. The use of clinical indicators is one means of demonstrating efficiency and effectiveness. The C–L clinical indicators used at RMH demonstrated both the maintenance of high standards in some areas and improved standards in others. Further measures of effectiveness and efficiency may be evident in service-use data, such as that produced through the C–L database and the application of specific outcome measures.
A perennial argument used against processes that attend to the broader system is that resources would be better spent in direct clinical care. The doctor caring for the patient has always been the cornerstone of medical practice. The application of a systems perspective to medicine has been slower in finding voice, although less so in psychiatry than in other areas of medicine. Using Engel's paradigm [14], the use of clinical indicators in C–L is an intervention at the group level of the system. Our experience has been that clinical indicators can be an important intervention in the overall functioning of a consultation–liaison service.