Abstract
Summary
Objective
To examine the performance assessments and cognitive function of practitioners referred to the National Clinical Assessment Service (NCAS).
Design
Retrospective observational study.
Setting
Practitioners referred to NCAS for performance assessment due to suspected performance problems.
Participants
One hundred and nine practitioners over the age of 45 years referred to NCAS between 1 September 2008 and 30 June 2012.
Main outcome measures
Reasons for referral of practitioners and their characteristics; details of their assessments including screening for cognition using Addenbrooke's Cognitive Examination Revised (ACE-R); outcome of the process.
Results
Reasons for referral included ‘clinical difficulties’ and ‘governance or safety issues’. Eighty-seven practitioners scored above 88 on ACE-R. Twenty-two were found to have an ACE-R score of ≤88. On further assessment, 14 of these 22 practitioners were found to have cognitive impairment. The majority of all practitioners were found to be performing below the expected level of practice for someone at their grade and specialty. Of those scoring ≤88 on the screening, only seven continued in clinical practice.
Conclusions
A high proportion of practitioners scoring poorly on ACE-R were found to have cognitive impairment following detailed neuropsychological testing, the youngest aged 46 years. Many were working in isolation. Nearly all practitioners scoring poorly on ACE-R were international medical graduates; reasons for this are unclear. Performance assessment results showed persisting failings in the practitioners' record keeping and in their assessment of patients. Our findings highlight the need for increased vigilance and training of responsible officers to recognise performance problems and emphasise the importance of comprehensive assessment.
Keywords
Introduction
The UK's National Clinical Assessment Service (NCAS) was set up in 2001 and helps improve patient safety by working with the National Health Service (NHS) and other healthcare organisations to resolve concerns about the professional practice of doctors, dentists and pharmacists. NCAS provides a range of services – from telephone advice, through to more detailed and ongoing support, to a full assessment of the practitioner's performance. This consists of a workplace-based assessment, which includes a health assessment by a specialist in occupational medicine, a behavioural assessment by an occupational psychologist and an assessment of clinical practice. The components of the clinical assessment include a record review, observation of clinical practice, case-based assessment, site visit, and peer and patient feedback. It is conducted by a team of trained peer and lay assessors whose performance as assessors is quality assured.
Following assessment, a detailed report is produced which covers the assessment findings mapped to the domains of Good Medical Practice described by the General Medical Council (GMC). Within each domain, a practitioner is either found to be ‘poor’, ‘inconsistent’ (assessment displays a combination of poor and satisfactory examples) or ‘satisfactory’. A conclusion is drawn stating that the practitioner's performance overall is at the level or below that expected for the grade and specialty and recommendations made to address the deficiencies identified.
The health assessment takes place first and is conducted by an occupational health (OH) physician who determines whether a practitioner has any health concerns, which may be contributing to poor performance. The assessment includes physical problems and screening for anxiety, depression, drugs or alcohol misuse. The OH physician will explore whether the assessment process itself could exacerbate any health problems, and whether the remediation programme would need to take account for them.
One component of the OH assessment is a screening for cognitive function. NCAS searched for an instrument that would identify those who may have cognitive impairment accounting for their performance issues. The decision was informed by a review of 39 tests 1 which takes less than 20 min which suggested that while there may be no single fail-safe instrument the Addenbrooke's Cognitive Examination Revised (ACE-R) had certain advantages including its sensitivity and specificity for the diagnosis of dementia at cut off scores of ≤88 although it does not specifically test reasoning and judgement. 2 –4 The ACE-R has now been superseded by Addenbrooke's Cognitive Examination – Third edition (ACE-III). See appendix 1 for further information regarding tests of cognitive function.
An important finding since instituting this process has been that a number of practitioners have been found to have previously unrecognised cognitive deficit. This paper describes the characteristics of practitioners undergoing an NCAS assessment who scored poorly on ACE-R, the results of the assessment, the findings of the specialist assessment and the outcome of the process in terms of the practitioner's continuing practice. It compares the results of the NCAS performance assessments of those practitioners who scored above the 88 cut off point in the ACE-R screening test with practitioners who scored below. There are no studies exploring cognitive impairment in physicians with performance concerns in the UK and the effect this may have on the ability to practise safely.
Methods
Since September 2008, NCAS has performed neuropsychological screening on all practitioners referred for performance assessment using the ACE-R. The test takes between 12 and 20 min to perform and assesses five cognitive domains: attention/orientation (18 points), memory (26 points), verbal fluency (14 points), language (26 points) and visuospatial abilities (16 points), giving a total score of 100 points. 4 NCAS uses the results of this test and information from the OH assessment to identify those practitioners requiring detailed neuropsychological assessment. Practitioners who score ≤88 on ACE-R are referred for specialist neurological assessment. The referring body is advised that those practitioners scoring 82 or below should refrain from practice until further investigations have been performed.
All practitioners referred to NCAS for a performance assessment from 1 September 2008 until 1 June 2012 were identified and information collected for those scoring 88 and above or below on ACE-R. Details including demographics, specialty, concerns leading to referral, results from the assessment of clinical performance and the occupational assessment were collected and analysed. Because of the relatively small number of practitioners assessed, formal testing for the statistical significance of differences between subgroups was not carried out.
Results
There are approximately 1000 new referrals per year made to NCAS. From 1 September 2008 until 30 June 2012, 135 healthcare practitioners, all of whom were working in the NHS, who were referred to NCAS required assessment due to suspected performance problems. Of those, 22 practitioners were found to have an ACE-R score of ≤88, the youngest being 46 years old. The data on all practitioners referred over the age of 45 years were analysed (109 in total). Eighty-seven practitioners scored ≥88 on ACE-R. At the point of referral there were no suspicions of impaired cognition.
Demographic characteristics of practitioners referred for assessment to NCAS.
Twenty-one out of 22 (95.5%) practitioners scoring 88 and below on ACE-R obtained their qualification outside the European Economic Area (EEA), compared with 39/88 (44.3%) in the group scoring >88 (risk ratio 2.15, 95% confidence interval 1.68–2.77).
Concerns leading to referral
Reasons for referral.
Performance assessment
Overview of performance assessment.
Nature of impairment
Results following detailed neuropsychological testing.
No new physical health problems were identified. Existing conditions included ischaemic heart disease, hypertension, diabetes mellitus, aortic valve disease, hypothyroidism, hyperthyroidism, glaucoma, arthritis, hypercholesterolaemia and hepatitis. One practitioner was known to have dyslexia. Of the 14 practitioners found to have cognitive impairment, eight were found not to be performing at the expected level of practice. Four practitioners did not proceed to a full assessment, two practitioners were found to be performing at the expected level of practice.
Outcome
Details of outcome.
Discussion
Twenty-two practitioners (doctors and dentists) referred to NCAS for performance problems had an ACE-R score of ≤88, indicating a need for further assessment. All but one were international medical graduates. Fourteen of these practitioners were diagnosed with cognitive impairment. The youngest found to have cognitive impairment was aged 46, with two in the 45–49 age group. The oldest was aged over 70. At the point of referral, there were no suspicions of cognitive impairment in any of the cases.
In the group of practitioners scoring ≤88, 15 had a work place performance assessment; 10 were found to be performing below expected level of practice. The assessments were further explored; the subdomains ‘Record keeping’ and ‘Assessment of Patient's Condition’ were found to be poor in the majority. However, ‘relationships with patients’ and ‘maintaining good medical practice’ were mainly judged to be ‘inconsistent’ with examples of satisfactory practice along with examples of poor practice. Practitioners appear to be maintaining their verbal skills; which could explain why cognitive difficulties were unrecognised until formally assessed.
There was an over representation of men in both groups, reasons for this are unclear. There is also an over representation of GPs in the groups with an ACE-R score ≤88, but a more equal representation of GPs and hospital specialties in the 88 and over group. Single-handed GPs were over represented (eight in those scoring ≤88). An explanation for this could be that symptoms and signs of cognitive impairment in those working in isolation are less likely to be identified. If the referring body had suspicions of practitioners suffering with cognitive deficit, they would not have been referred to NCAS.
On observing the outcome, a greater proportion of practitioners in the group scoring ≥88 on ACE-R were able to continue working, either as normal or within a remediation programme, despite a similar proportion in both groups found to be performing below expected level of practice. This could be attributed to the finding of cognitive impairment. Two practitioners diagnosed with cognitive impairment were found to be performing at expected level of practice, showing that it is possible to accommodate cognitive decline in clinical practice. 5 Assessment tools are unable to predict how cognitive impairment will impact on professional practice. 6 There are no agreed guidelines to help decide when a practitioner should cease medical practice. 5,7
Strengths and weaknesses
To our knowledge, this is the only study in the UK looking at the finding of cognitive impairment in healthcare practitioners with performance concerns. Although this has been demonstrated internationally, this study is unique in that it also looks at reasons for referral, and looks at practitioners' performance in a clinical setting and highlighting concerns.
A key limitation of this study was that there was no control group. Further work could include testing a group of practitioners not known to have performance concerns. It is also difficult to estimate the number of practitioners with performance problems secondary to cognitive deficit, as this study only includes those where it was not suspected. An explanation for this could be that practitioners were referred for performance difficulties rather than behavioural changes. It could be that neuropsychological testing is being carried out at a presymptomatic stage before most people are suspected to have cognitive impairment and subsequently referred for testing. Another theory to explain why cognitive impairment is not suspected is that physicians are able to mask the symptoms of cognitive impairment due to their cognitive reserve, which is likely related to their pre-morbid IQ. 6
Given that the majority of those scoring ≤88 did not possess English as their primary language, this raises questions about how it effects the validity and reliability of ACE-R, which was designed with native English speakers in mind, or at least those who speak English fluently. 2 In order to practice effectively in the healthcare professions in the UK, practitioners need to be proficient in English. However, if communication was an issue at presentation, NCAS would have organised an expert communication competency assessment as part of the NCAS assessment. Although communication proficiency is not formally screened for, the health and behavioural assessments assist in identifying if it is a cause for concern.
For those who did not undergo an NCAS assessment, the full cognitive evaluation would have highlighted linguistic issues if it were the primary concern. It therefore remains unclear why those scoring ≤88 on ACE-R were all international medical graduates.
Another issue to consider is whether the cut off scores for this group of professionals are correct. The ACE-R normative values were based on volunteers from the Medical Research Council and spouses of patients attending clinics. 2 Although education was taken into account, there is no evidence on whether the cut off score of 88 is still as sensitive and specific for those with a higher IQ than the general population. Reports state that according to the overall data for MicroCog, physicians' scores are better than non-physicians. 8,9 A study exploring the detection of cognitive impairment in highly intelligent individuals was compared using IQ-adjusted norms to predict cognitive decline with age and education-adjusted norms. 10 It found that IQ-adjusted norms provided a better predictor of cognitive decline. 10 As IQ-adjusted impairments were measured from pre-morbid intelligence instead of a normative average, IQ-adjusted cut off scores could detect early changes in cognition that would have been normalised by standardised test norms. 10
Comparison to previous work
Findings of this study mirror what has been found elsewhere; several prior studies have explored cognitive difficulty in physicians who have performance concerns, and have been referred for competency assessment. In Australia, there has been an increase in the number of notifications for suspected cognition problems. 5 A recent study found that cognitive impairment in physicians is responsible for 57% of adverse medical events, most of which were determined to be preventable. 11,12
A study by Turnbull et al. in 2000 based in Ontario performed cognitive screening tests on 27 physicians with performance concerns who were in the Physician Review Program (PREP). Seven were found to have moderate or severe cognitive impairment. The age range of those with neuropsychological difficulties was 43 years to 76 years. 13 Forty-five physicians were assessed in the follow on study in 2006. Thirty-one physicians performed poorly on competency assessment, 12 were found to have moderate or severe cognitive impairment, which was likely to have led to their poor performance. 14
Studies by Korinek et al., 15 Peisah and Wilhelm 16 also report similar findings. Korinek et al. completed a study with 335 physicians in Colorado. Two hundred sixty-seven were referred for competency evaluation due to performance concerns, and 68 participants were in the control group. Physicians in the competency group scored lower than the control in processing speed, processing accuracy and cognitive proficiency. Twenty-four per cent of physicians in the competency group had scores suggesting cognitive impairment, compared with none in the control group. The referred physicians showed slower processing, less accuracy and less proficiency. 15
A descriptive study by Peisah and Wilhelm looked at 41 case records of notifications to the Impaired Registrants Program of the New South Wales Medical Board, Australia, of those aged over 60 years. Cognitive impairment or dementia was found in 22 cases. 16 Another study performed by the California Medical Board (CMB) looked at 148 physicians referred for competency assessment secondary to performance concerns. 17 A number of tests of neuropsychological functioning were used; they found the physicians to be performing lower than expected on tests of intellectual and neuropsychological functioning. Studies by Korinek et al., Turnbull et al., and Perry and Crean all used screening tests to identify neuropsychological difficulties but comprehensive neuropsychological assessments were not performed.
We found the majority of those scoring poorly on ACE-R in our study were aged between 55 and 59 years. In the group scoring >88, the majority were aged between 50 and 54 years. This supports previous work, which has found that there is a negative correlation between performance on cognitive screening and age. 13,14,18 –20 Turnbull et al. 13,14 found that the correlation increased when the sample was altered by excluding physicians with neuropsychological disorders.
There was an over representation of men in both groups.
As mentioned previously, 21 practitioners who scored poorly on ACE-R were international medical graduates. The study by Turnbull et al. 14 found that English as a first language was found to be a positive predictor of performance on PREP. In the studies by Korinek et al. 15 and Perry and Crean 17 those who trained outside the United States or who did not have English as their first language were excluded.
Peisah and Wilhelm 16 found that older doctors suffer from ‘the four D's – drugs, drink, depression, dementia’ and reported that the presence of substance misuse in 29% of their sample. Substance misuse issues were not found in our group of practitioners, if this were a presenting concern, they would likely be referred locally or to the GMC.
Implications for policy and practice
This study shows an association between performance problems and cognitive impairment. It suggests cognitive screening for health practitioners aged 45 years and above with professional performance problems may be necessary. Selecting a validated screening tool needs further exploration, as this study showed an association between practitioners who trained overseas and scoring poorly on ACE-R for reasons that are not understood. ACE-III has now superseded ACE-R and is very similar in comparison.
Detection of cognitive impairment is essential to determine whether patient safety is at risk and safeguard quality patient care as well as to take it into account when creating and implementing an effective remediation programme. Tailoring to individuals is crucial as well as considering whether the same level of cognition is required for different specialities such as surgery or academia. 6,7 Frequent assessment would also be required to ascertain whether remediation to safe practice is proving effective. Further work is required to explore effective methods of remediation, and also to identify those in which it would be unlikely to be successful. A wider study would be required to assess whether regular medical and physical screening for healthcare practitioners is necessary.
Our study does however highlight the need for increased vigilance and training of responsible officers to recognise performance problems and emphasise the importance of a comprehensive assessment, which takes into account cognitive function.
Conclusion
A high proportion of practitioners in those referred to NCAS for performance assessment score ≤88 on ACE-R, a validated screening tool for detection of cognitive impairment. The majority of those scoring below the predetermined cut off were found to have cognitive impairment following detailed neuropsychological testing. Persisting failings were detected in the practitioners' record keeping and patient assessment. Although the reasons for this are not clear, a consistent finding was that all practitioners with an ACE-R score of ≤88 were international medical graduates. We recommend ensuring that appraisers and responsible officers are trained to detect performance problems and should arrange for cognitive screening such as that provided by NCAS.