Abstract
A range of pressures (consumer advocacy, growth in health services spending, expectations of improvements in quality of health services) has led to an emphasis on financial and clinical accountability. Introduction of active processes for improved accountability require a systematic approach. One relevant methodology is ‘total quality management’ that emphasizes that the workplace unit or team, rather than the individual, should introduce improvements in clinical practice [1]. Consequently, an emphasis is placed on improving work structures and interaction between parts of a clinical team or system of care. This is a move away from traditional methods that have focused on individual practitioner performance and productivity.
In Australia, the first stage of the National Mental Health Strategy (1993–1998) [2] focused on improved quality of specialist mental health services and achieving transition to greater community-based care. The impact of these policy directives on the management of depression in older persons is of special interest. Depression in older patients is often complicated by medical and neurological factors [3] and the need for longer term medical and psychiatric care. Designated psychogeriatric services are one way of providing services which meet the specific needs of older patients. Research suggests, however, that the same type of integrated assessment is required for people with depression who present for treatment after the age of 50 years [3].
An earlier study of older persons with depression (aged over 50 years) treated in our services [4] demonstrated that relevant medical and psychiatric assessment practices were implemented least often by the generic adult community-based health services (as compared with specialized or inpatient services). In each of the services, approximately one-third of patients were being treated for their first ever episode of depression. Of note, key factors such as medical, neurocognitive and risk assessment were frequently omitted. Relevant laboratory investigations were often not performed and communication with general practitioners was poor. Further, although patients attending the generic adult services were the youngest (mean age = 57.3 years), they had the worst outcomes at 2.5 years follow up (presumably, reflecting the chronic and/or relapsing nature of their early-onset disorders). The research highlighted the need to develop: (i) clear, evidence-based assessment tools for patients with depression aged over 50 years presenting for psychiatric care; (ii) educational systems for the staff of mental health services and associated general practitioners; and (iii) depression management programmes to monitor treatment adherence and long-term patient outcomes. That is, there was a clear need for a combined education and service development approach [5].
As the first step in the evaluation of quality improvement initiatives, it is often most relevant to look at changes in key process variables before attempting to assess relevant individual patient outcomes. While one cannot simply assume that improvement in such processes will necessarily result in improved patient outcomes, a failure to detect such changes would indicate that the specific initiative required immediate modification. This paper reports the implementation and evaluation of our specific education and management initiative for older patients with depression within a district mental health service. As such it makes a contribution to the emerging debate about implementation of evidencebased mental health practice in Australia.
Method
Subjects
Patients were included in this phase of the project if they: (i) presented to district mental health services over an 8-month period following implementation of the specific education and service development programme; (ii) had a primary diagnosis of major depression; (iii) were aged over 50 years; and (iv) were able to read English. As the educational and self-evaluation materials provided required reasonable English literacy and/or good eyesight, we did not attempt to recruit those persons who would be unable to complete the basic assessments. Of 81 persons identified as potential subjects, 23 failed to meet the inclusion criteria. Three patients had depression which was clearly secondary to a medical illness, eight had a primary diagnosis of dementia (DSM-IV, made by an experienced clinician), four were alcohol dependent and eight were from non-English-speaking backgrounds. Due to the process of stepwise introduction of the programme and some staff reluctance to utilize formal programmes, 14 patients who were otherwise eligible did not receive the assessment/educational programme during the 8-month evaluation phase. Systematic evaluation of the intervention was therefore possible in 76% (44/58) of subjects who were identified and met the inclusion criteria.
Implementation of the improved service delivery system
The strategy comprised: (i) clinician education about evidencebased treatment of depression; (ii) provision of standardized assessment and treatment tools for mental health clinicians and general practitioners; (iii) changes in policy and procedures to require clinical teams to complete the assessment forms and to introduce patients to the education and self-evaluation formats; and (iv) employment of a project officer to support clinicians in their skill development and changed clinical practice. The project officer was a research psychologist who did not have prior clinical training or experience. The project officer was employed over a 2-year period from Commonwealth grants ($50 000 per year) to conduct the initial evaluation [4], participate in the implementation of the programme and then evaluate the implementation. Clinician education was provided in the form of seminars for groups, supplemented by individual discussions between the project officer and each mental health clinician. Staff were provided with a detailed description of the project including the initial evaluation of care systems. Their questions regarding the rationale of the project were addressed. Every new staff member was provided with this education on entry to the service. Journal articles relating to the assessment and management of depression were made available to all clinicians.
In addition to the one-to-one discussions, education booklets regarding the pharmacological and psychological treatment of depression in older persons were provided to clinicians for use with their individual patients [6]. The booklets were designed for use by mental health clinicians to assist them to educate patients during therapy sessions and for patients to keep for the purpose of self-education.
To achieve greater standardization of assessment, clinicians were provided with protocols which contained checklists for making formal diagnoses of depression (DSM-IV) [7], as well as information on patient medical and psychiatric history, depression severity, suicide risk assessment, neuroimaging, blood tests (electrolytes, full blood screen, thyroid and liver function tests), medical examination and the Mini-Mental State Examination (MMSE) [8] to screen for cognitive impairment. Patients with MMSE scores less than 27 were reviewed by senior clinicians for possible diagnoses of dementia. A format for designing an evidence-based treatment plan was supplied.
At the same time separate booklets of patient treatment aids were also provided. These included: (i) forms to monitor symptoms of depression [9] (completed at baseline assessment, each week for the first 4 weeks of treatment, and again at 6, 12, 26 and 52 weeks); (ii) self-rated disability measures (baseline assessment usingthe 12-item Short-Form Health Survey [10], and the Brief Disability Questionnaire [11] and again at 6, 12, 26 and 52 weeks); (iii) patient activity schedules to use in conjunction with behavioural treatment; and (iv) a self-report monitor of side-effects and benefits of antidepressant medication. Each of these were used at the discretion of the treating clinician or team.
The project officer provided practical support for health professionals in using the education assessment and treatment aids, facilitated the recruitment of appropriate patients, compiled and distributed materials, issued reminders to engage general practitioners in each patient's care, and organized the educational components. For the duration of the implementation the project officer was located at an adult community mental health centre with the clinicians.
A specific strategy to ensure engagement of general practitioners was devised. With patient consent, a copy of the diagnostic and assessment module for each patient was sent to their general practitioner. General practitioners were also sent a pamphlet which described the aims and process of the project and a newsletter which discussed identification and management of patients with depression in general practice. General practitioners were invited to attend a seminar on assessment and management of older patients with depression which was conducted by a staff specialist in the psychiatry of old age. Prior to discharge from mental health services, patients participating in the project were reminded to visit their general practitioner for monitoring of symptoms at specified time-points (every week for the first 4 weeks and 6, 12, 26 and 52 weeks following initial presentation).
A formal evaluation of clinical practice over an 8-month period was conducted. As in the initial phase of the study [4], this comprised a collation of diagnostic and assessment modules and a manual audit of clinical files from the three service components. Patients who met inclusion criteria for the project but who had not been entered in the project were also identified and probable reasons for exclusion of patients were noted. In the earlier study, approximately one-third of the patients in each service were presenting for treatment of their first ever depressive episode. Entry to each service type is determined by age and location of care [4]. The specialist psychogeriatric team provides hospital and community services to patients over 65 years of age at presentation to the service. The population with depression that they service is older (mean age = 75.8 years) with more concurrent medical problems (54%), fewer persons living independently (57%) and at more risk of developing significant cognitive impairment (32%). By contrast, the community-based adult teams provide services to persons who have the onset of their disorders before age 65 years. Consequently, they treat a younger group of patients (mean = 57.3 years) who have fewer concurrent medical problems (26%), are more likely to be living independently (79%) and who are at less risk of developing cognitive impairment (8%). This service then continues to see these patients as they grow older.
Results
A comparison of the baseline (1995) and postimplementation (1999) data is presented. Demographic and clinical characteristics of the two patient samples are presented in Table 1. Of the patients who were involved in the assertive implementation phase of the study, 39% (n = 17) were managed by the specialized psychogeriatric service, 23% (n = 10) by the general adult community mental health service, and 39% (n = 17) required initial admission to the inpatient unit. Patients seen by the general adult community team were particularly likely to have an age of onset of illness before age 50 years [50% vs 33% (psychogeriatrics) and 24% (inpatient services)]. Patients involved in the implementation phase of the study were more likely to be female, have a history of anxiety and to be treated by inpatient services. The increase in diagnoses of anxiety appears to have been a consequence of an emphasis on recognition of these comorbid diagnoses across all services. A comparison between patients included (n = 44) and not included (n = 14) in the assertive implementation phase was performed. Females were preferentially recruited to the project (91% of included patients vs 67% of patients who were not included).
Demographic characteristics, psychiatric status, medical illness history and psychiatric services attended by patients
The improvements in assessment and investigation practices between baseline (12 months of 1995) and following the assertive improvement strategy (8 months of 1999) are outlined in Table 2. It can be noted that in all three service centres there was an increase in the use of standardized medical and psychiatric assessments and investigations. All clinicians with patients participating in the project used the diagnostic and assessment modules, treatment packages and the education books.
Specific clinical assessments or investigations provided by psychiatric services before and after implementation strategy
Discussion
As it was too early after the implementation of key system changes to evaluate long-term patient outcomes, we have confined ourselves here to the evaluation of changes in those processes which are thought to underpin improved outcomes. Within health services research this is a common first step, though in many clinical services it is the only evaluation that is ever able to be achieved. For the key clinical, cognitive and biomedical assessments, and communications with other practitioners, improvements were achieved across all service sectors. While the two assessment periods were some time apart (1995and 1999) there were no other major changes in service structure during the period that could easily account for the improvement in the specific process variables. As those assessment and communication parameters that were measured had also been targeted specifically by the intervention, it is reasonable to assume that the improvements occurred largely as a direct consequence of the quality improvement project rather than other non-specific service enhancement factors.
Mental health services traditionally are provided by clinicians from a large variety of professional backgrounds. In our service, clinicians act as part of a multidisciplinary team where the responsibility to provide comprehensive assessment and management rests with the team rather than any specific individual. It is only when we examine closely the needs of a particular illness group, such as older patients with depression, that we are confronted by discontinuities in services within these multidisciplinary systems. Typically, when mental health services have been confronted with such discontinuities, and the need to implement evidence-based practice, they have established special interest groups. This is exemplified currently by the growth of first-onset psychosis programmes in most mental health services [12].
While the advantages of training a specific group of enthusiastic clinicians in evidence-based practice may be apparent, it can be argued that this is not a practical solution within comprehensive psychiatric services. A proliferation of specialist teams runs the risk that individual patients, and particularly those with comorbid and multiple illnesses, will receive even more erratic care [12]. Thus an alternative solution is to develop clinical management modules (or streams) with well-defined assessment, investigation, intervention and evaluation protocols.
Regardless of model of service delivery, such management decisions must be accompanied by a commitment to implementing effective strategies for evidence-based care. This study describes the use of an overarching theoretical framework of ‘total quality management’. This approach emphasizes the need for collaboration between management directives (top-down) and active clinician involvement (bottom-up) [13]. Within this framework, the strategies employed included an intensive education programme which contained traditional teaching approaches (seminars) a variant of academic detailing (oneto- one discussion between project officer and clinician) and provision of significant clinical support materials.
This broad ranging education strategy was devised by an academic department but the commitment of staff time to participate, required active management support. Other aspects of management commitment were demonstrated by the changes in policy and procedures which were required to facilitate staff moving from previous clinical practice to more structured approaches. This partnership between academic and management psychiatry has been discussed elsewhere [13] and was further demonstrated by the systematic evaluation of the project.
While evidence-based practice and total quality improvement approaches have broad clinical and research support, the implications for management have not been well elucidated. This project demonstrated that a considerable resource commitment was required. It included: (i) support for the production of multiple copies of the packages; (ii) investment of staff time in new learning; and (iii) management time to implement and monitor the new service delivery model (with its emphasis on improved assessment, investigation and systematized intervention). The additional costs of greater use of laboratory and radiology tests also require adjustments in funding.
An apparent bias against the recruitment of male patients was an unexpected finding. While we have no clear explanation, this may reflect the fact that the project was most enthusiastically received within the specialized psychogeriatric service which sees more older females in non-institutional settings [4] and/or that male patients, particularly those seen in institutional settings, were perceived by staff to be less suited to the initiative. Some initial staff reluctance to take up the project was most evident in the community-based general adult teams and the inpatient service (both of whom service a higher proportion of males) [4]. In the former, this reflected staff reluctance to participate in ‘research’ (reflecting a lack of familiarity with quality improvement processes) and a concern that the model presented was too ‘biomedical’. For the inpatient staff, the issues reflected the difficulty they perceived in implementing the educational and selfevaluation components with more severely ill patients. The way in which staff perceptions impact on the pattern of uptake of service enhancement projects requires further direct evaluation.
In each of the service sectors, although different groups of patients were receiving different styles of treatment, the main process variables assessed were clinically relevant. The specialist psychogeriatric services focus on treating older patients with depression in non-institutional settings and provide treatment in close collaboration with general practitioners. Even within such targeted services, however, improvements in the key assessment parameters were achieved. For the general adult community services, very significant gains were achieved largely as a consequence of recognition that these services do provide mental health services to a group of patients with depression (largely aged 50–65 years) with very specific but unmet biomedical and clinical needs. This service had not previously developed a specific process for clinical evaluation of these patients. While patients attending this service are younger than those seen by the psychogeriatric service [4], they are in a high-risk age group for development of comorbid medical problems. Additionally, as one-third of cases are having their first episode of depression after 50 years of age [4] they are also at high risk of underlying cerebral disease [3]. The inpatient services demonstrated the least change in assessment practices as they already had in place protocols to achieve some of the key clinical and biomedical assessments and transfer of information to the general practitioner.
While the total cost of this new style of mental health practice was not estimated in this project, if protocoldriven clinical practice can be sustained it will be possible to do realistic cost assessments. The patient benefits will need to be evaluated not only in terms of clinical outcomes but also from resource efficiencies (through shorter lengths of time in mental health services, less use of inpatient admission and few complications of badly treated depression). At this point, the important factors being demonstrated were that quality improvement techniques and combined management and academic commitment were successful in bringing about evidence-based care in a mental health system.
The role of the project officer as a supporter and champion of the project cannot be overemphasized. She engaged individual clinicians in discussions relevant to their perspective, thus reducing the previously described staff antipathy towards ‘evaluation’ and/or ‘research’ [13]. The practical support required to ensure packages and protocols were readily available, and her constant reminders to clinicians to recruit patients, have been considered indispensable to the level of clinician uptake of the evidence-based approach. She attended clinical team meetings (the community equivalent of hospital ward rounds) serving as a continual reminder to clinicians of the priority status of the project within the service.
Involving general practitioners in partnership care with specialist public sector mental health services is a priority at national and State levels. Many initiatives at a local level have promoted this view [5, 14, 15]. The key issue for mental health services is the generation of sustainable shared care models. Within such models it should be standard practice for shared education, patient information and care protocols to occur. Whether this project has achieved a culture change in mental health clinicians' attitudes towards the roles of the general practitioner in the ongoing management of this common mental illness can only be evaluated in the longer term.
In conclusion, the implementation of evidence-based practice in mental health services is a key national priority. We initially demonstrated that for older patients with depression it was not occurring and that health outcomes for patients receiving routine clinical care were poor [4]. We have described a successful implementation strategy to engage clinicians from multiple professional backgrounds in delivering evidence-based practice. However, the initiative was costly ($100 000 in direct funding), time consuming and required considerable academic leadership and management commitment over a relatively long period of time. The pertinent questions which must be asked include: Will the changes be sustained after the life of the project?; Are they generalizable to other mental health services?; Can they be replicated at a lower cost now that the lessons have been described?; and Can we demonstrate a significant cost benefit for this change in service delivery model?