Abstract
Summary
Objective
To track the success rates in obtaining a specialist training (ST) post of a cohort of medical training application system (MTAS) applicants, as well as canvassing their views of the process and measuring symptoms of psychological distress during application and after job allocation.
Design
A follow-up cross-sectional survey.
Setting
Two large south London psychiatry training schemes.
Participants
101 MTAS applicants.
Main outcome measures
Success in obtaining an ST post, respondents' views about MTAS and General Health Questionnaire-12 (GHQ-12) scores.
Results
48 of the original sample responded to the follow-up survey. Data were available about post-MTAS job status for a further 41 trainees. 64% of candidates obtained an ST post. Of those, however, 12% were only offered a fixed-term post. Most unsuccessful candidates were doing locum (34%) or non-training grade (21%) jobs. UK/European Economic Area (EEA) doctors were more successful than non-EEA applicants in obtaining an ST post (odds ratio 5.5, 95% confidence interval 2.1–14.3), as were women candidates compared to men (3.7, 0.5–9.3). The respondents' views about MTAS were globally negative (even if they were successful in obtaining an ST post). The median GHQ-12 score for respondents was 7 out of a potential total score of 12. 79% of the sample scored above the threshold (GHQ-12 ≥4) for psychological distress and 21% experienced significant distress (GHQ-12 ≥8).
Conclusions
MTAS was a flawed system. A considerable proportion (36%) of candidates who had been on a Senior House Officer training scheme were left without a training post after MTAS. The system seems to favour UK/EEA applicants. Applicants suffered significant psychological distress during and after the MTAS process.
Introduction
In the UK, Medical Training Application Service (MTAS) was a single electronic portal for doctors to apply for the specialist ‘run through’ training scheme given the umbrella title of Modernising Medical Careers (MMC). Such single portals have been successfully used in other developed countries for many years because they obviate the need for multiple applications and can support a consistent selection process. The implementation of MTAS in the UK was rushed, undermined by technical failings, and has not been supported by the profession as an effective selection process. 1 The perceived failings of MTAS, particularly with the huge disparity between available training places and the number of applicants, has led to the abandonment of MTAS for the 2008 application round, and to a wholesale review of MMC.
We previously reported the findings of a survey of the views of MTAS applicants from two London psychiatry training schemes, and a request by the Royal College of Psychiatrists for emails from trainers and trainees documenting problems with the system. 2 The survey was conducted during the application process and before the MMC Review Group decided that the short-listing process was seriously flawed and it was decided to give all applicants (who were in the UK and eligible to work in the UK) an interview with a deanery of their choice (i.e. between MTAS round 1a and 1b).
The MTAS system has met with widespread disapproval from the medical community, 3 , 4 and the interim report from the MMC inquiry has been released, recommending yet further changes to postgraduate medical education in the UK. 1 Many junior doctors have been affected both practically – a number who had been on the old senior house officer (SHO) rotations have been left without training jobs – and emotionally or in their sense of well-being. A contemporaneous survey had reported that three-quarters of trainees suffered symptoms of psychological distress, and the majority (96%) attributed these to MTAS and/or MMC. 5
In light of these issues, we decided to re-survey the 101 respondents to the original training scheme survey following the allocation of jobs through round 1 of MTAS.
Aims
To track how the cohort of trainees who responded to the original survey had fared following the allocation of jobs.
To re-canvass their views on the process.
To measure symptoms of stress experienced during MTAS application/after job allocation.
Method
The 101 trainees who responded to the original survey were contacted, where possible, using the email addresses from the original survey in July and August 2007, following the announcement of jobs after the first round of MTAS (i.e. rounds 1a and 1b) but before round 2 had started. They were asked if they were successful in obtaining a run-through post and, if so, to provide the details of their appointment. If unsuccessful, they were asked if they had arranged an alternative job and what their plans were for the future. Successful candidates were asked if they obtained a run-through post or a fixed-term specialist training appointment (FTSTA), if their job was in the deanery of their first choice, and whether they had to move to work in another geographical area as a result of the process. Respondents were asked to give their views on the MTAS process as a whole in a free text section (including indicating potential areas of improvement for future selection processes). We wanted to canvass their views after jobs had been allocated because the original survey had been conducted during round 1 of MTAS.
As a result of changed employment a number of email addresses were not available. We used personal contacts to communicate with those trainees without working email addresses, and asked them to complete a paper version of the survey. If it was not possible to make contact with the trainee, we sought information about how they fared in terms of job allocation from the clinical tutors and human resources departments of the Trusts they had worked in.
In addition to the survey's questions, respondents were asked to complete the General Health Questionnaire-12 (GHQ-12) covering the previous four weeks during application and following the announcement of jobs though MTAS. As some of the trainees knew the authors personally, the GHQ-12 was an optional part of the survey. For the same reason, respondents were given the option of completing the GHQ-12 anonymously, and the score was later linked by number (rather than name) to the other survey results.
Results
Forty-eight trainees responded to the survey. We were able to obtain details on how
another 41 candidates fared through the MTAS process from the clinical tutors and human
resources departments. Therefore, post-MTAS job status was available for 89 trainees
(88.1% of the original sample), as were the views of 48 trainees (47.5%). We were unable
to trace 12 (11.9%) of the respondents to the original survey. MTAS timeline and study flow diagram
How the applicants fared
Data analyses were conducted using Numbers for Apple Macintosh. Figure 1 is a flow diagram reporting how the cohort of trainees fared in terms of success in obtaining a training post through MTAS, as well as a timeline of the process. Of the 89 candidates for whom we had data regarding their post-MTAS job status, 57 (64.1%) obtained an ST post. Of those, 46 (80.1%) obtained a full run-through and seven (12.3%) FTSTA jobs in psychiatry. One other successful respondent secured an academic psychiatry run-through post, while three changed speciality (two to neurology and one to general practice – all run-through). Forty-three (75.4%) successful candidates got a training post in the deanery of their choice. The 14 (24.6%) who didn't get their first choice had to move considerable distance to work in their non-preferred deanery.
Thirty-two (36%) former trainees were unsuccessful in obtaining an ST post. Of those, 11 (34.4%) had arranged or had started locum jobs and seven (21.9%) had taken staff grade posts. Another three unsuccessful MTAS candidates got jobs abroad (one each for Ireland, Greece and Australia). One respondent had decided to take a career break.
Candidates' MTAS success rates
EEA, European Economic Area; MTAS, Medical Training Application System; ST, specialist training
When the results of UK and EEA candidates were pooled into an overall EEA-combined (EEA-c) category, 81.2% of applicants from this group were successful in obtaining an ST post compared to 43.9% of non-EEA candidates. Thus the odds ratio of an EEA-c doctor obtaining a training job compared to a non-EEA applicant was 5.5 (95% CI 2.1–14.3). Interestingly, 80% of women candidates were successful compared to only 50% of men. Thus the odds ratio of a woman obtaining a training post compared to a man was 3.7 (95% CI 1.5–9.3).
Respondents' views about MTAS
Problem areas identified by respondents (n=48)
FTSTA, fixed-term specialist training appointment; MMC, modernising medical careers; MTAS, medical training application system
The selection interviews were described as ‘rushed’ and ‘mechanical’. Rather worryingly, two trainees responded that the interview questions had been leaked before round 1b. The themes that emerged from the original survey were repeated in the trainees' responses to this survey. Themes that emerged in this survey that didn't occur in the original survey related to problems with job allocations and concerns about the futures of trainees who obtained FTSTA posts or no job at all. There was greater focus on the broader implications of MMC, the mismatch between the numbers of trainees and consultant posts, and lack of proper consultation with junior doctors about MTAS/MMC. In the original survey many of the candidates, whose views of the MTAS process were predominantly poor, recognized that there were some positives, mainly relating to a need to change the old system. However, in the follow-up survey the mood had changed: the respondents' comments about MTAS were globally negative, even for those who had been successful in obtaining a training job.
Respondents' recommendations for future application processes and training systems (n=48)
CV, curriculum vitae; SHO, senior house officer; SpR, specialist registrar
GHQ-12 scores
Forty-two of the 48 respondents to the survey completed the GHQ-12. Data were
analysed using Statistical Package for Social Sciences version 14. The responses were
scored using the standard 0-0-1-1 method and a relatively high threshold of 3/4 was
used to indicate psychological distress.
6
Candidates' General Health Questionnaire-12 scores
(n=42)
Discussion
MTAS was the biggest overhaul in modern times of the way in which junior doctors in the UK applied for training posts. It was introduced in a rushed fashion, without adequate piloting and with insufficient testing of the computer system required to handle tens of thousands of applications, and despite the concerns expressed by many in the profession, including the British Medical Association. 7 The people most affected (i.e. junior doctors) were not consulted sufficiently during the development of the system. Selection methodology in MTAS was designed for ST1, not for thousands of doctors already working in the specialities. 8 This is particularly relevant given that the interview component of the selection process at ST1 level has shown a high degree of validity and reliability but may not be transferable to other levels of training. 9
The respondents to our survey were experiencing considerable levels of psychological distress during and shortly after the MTAS application and job allocation process. Jones et al. found that only 20% of soldiers scored above the same threshold on the GHQ-12 in a large survey of psychological symptoms experienced by the Armed Forces in the UK. 10 The soldiers' median score was one, compared to seven for the doctors in our survey. Avery et al. looked at the mental health of miners following the 1992 national pit closure programme in the UK, finding that the percentage of responders with GHQ-12 scores above the same cut-off point was 46% for those still employed in the mining industry, 52% for unemployed former miners and 22% for working non-miners, 11 thus indicating that unemployment contributes to stress.
The proportion of trainees scoring above the threshold was much higher than in a previous study by Clarke et al. looking at stress levels in hospital doctors in New Zealand. 12 They found that psychological distress was associated with life events thereby confirming the role of stressors. The high levels of psychological distress in our sample are in keeping with the recent online survey by Lydall et al., which found that nearly three-quarters of trainees were feeling low energy levels and half were feeling hopeless about their future. 5 In addition, one third were drinking more and nearly one third said that they were making more mistakes at work.
The sample we report here were doctors who had already commenced specialist training through the previous SHO training scheme system prior to MTAS. Their sense of dissatisfaction with and unfairness of MTAS (as reported in the original survey) was well placed: approximately one-third were left without training posts as a result of the process. Of those who obtained training posts, 12% were left facing the uncertainty of what will happen when their FTSTA contracts expire after a year. Although the two were linked, MTAS was a separate entity to the broader issue of an overhaul of the UK medical education and training system (i.e. MMC). The ongoing discussions and debate about further changes only add to the continuing uncertainty, and are likely to produce further stress: employers and trainers must be aware of these.
Limitations
The numbers involved in the survey are small and represent only two psychiatry training schemes. However, the views expressed are in keeping with those expressed by junior doctors at the MMC inquiry road shows, in the media and through Remedy UK. 13
The GHQ-12 data need to be interpreted with caution. No baseline scores were available to act as a comparator. It is possible that doctors, 5 and psychiatry trainees in particular, have high GHQ-12 scores normally given the stressful nature of their work and possible vulnerability factors. Furthermore, GHQ scores for the general population have been reported to be high. 14 , 15 As only half of the candidates from the original survey responded, a possible respondent bias occurred (i.e. that the candidates who experienced less psychological distress didn't respond to the follow-up survey). We were unable to trace details for 12 (11.9%) respondents to the original survey. Most of those doctors (nine) were not from the EEA. Thus we may have under-reported the plight of non-EEA doctors.
Summary
Despite these limitations, it is evident from this survey that a number of former SHO rotational trainees were disadvantaged by MTAS/MMC, that their views of the process are negative, and that they experienced high levels of psychological distress during and shortly after the application process. However, it is reassuring that a number of the problem areas highlighted by this study were also identified by the MMC inquiry's interim report, and that Professor Tooke's recommendations are similar to those of doctors who responded to the survey.
Footnotes
DECLARATIONS
Footnotes
Acknowledgements
None