Abstract
Membership examinations of the Australian and New Zealand College of Psychiatrists (ANZCP) were instituted in 1970. Despite five major revisions since (1976, 1987, 1992, 1996 and 2004), with more pending [1], concerns about the examination process and results have persisted. These have focused on the perceived low pass rates [2,3] (e.g. approximately 65% for the ‘Day 1 clinicals’ from 1980–1988 [2] and 1995–1999 [4]). The reliability and validity of the oral (viva) examination has also caused concern [5,6].
The reliability and validity of the examination have been improved considerably in recent years, as commended in a recent Australian Medical Council accreditation report [7]. This has been achieved by examiner training before clinical exams; processes for setting a final consensus mark; criterion-referenced standard setting; and continuous quality improvement calibration after each examination [8]. Notwithstanding, the pass rate for the clinical examination has remained largely unchanged (53–69% for the equivalent clinical examination in 2007–2009 [9–13]).
As noted by Adler, ‘Such pass rates would be a cause for critical review in any other higher educational setting with questions asked about the quality of teaching and the expectations of the examination’ [3]. To date the quality of the examinations has been the focus of attention [14]. Less scrutiny has been directed at ‘the quality of teaching’ and ‘expectations of the examination’. However, ‘assessment is not merely a measurement problem’ but also ‘an instructional design problem’ [15]. If the purpose of the examination process is ‘to assess what you have learned in your training and attest that you have reached a certain level of competence’ [8], we should scrutinize what is being learnt in training, as well as refining the examination, in our efforts to improve pass rates.
Supervision is one form of teaching during psychiatric training. While the latest by-laws [16] mandate training and ongoing education of supervisors, what occurs during supervision has not been studied empirically, although recent examiners’ comments suggest deficiencies in candidates’ learning. While ‘the candidates appeared well prepared and managed their time well’, they had difficulties with the mental state examination, formulation and cognitive testing, all ‘fundamental skills’ of psychiatrists [9–13].
During research on RANZCP registrar supervision it became evident that a focus on the examination became a key issue in supervision later in training, as examinations loomed [17]. This material was therefore reviewed to assess the role of supervision in ensuring that registrars are trained to meet the standards of the RANZCP examinations and to consider the implications for examination pass rates.
Objective
The research questions that this study sought to answer on supervision in the RANZCP were:
What happens during the mandated individual supervision hour?
How does that supervision contribute to the production of psychiatrists?
Why is it configured as it is?
The objectives of this paper are:
to review the results from the study of supervision that are relevant to the examination process;
to consider their implications for improving examination pass rates.
Methods
This was a qualitative study of supervision approved by the Central Regional Ethics Committee.
Data collection
In order to ‘illuminate the different facets of the reality being investigated’ [18] three approaches to data collection were used:
Individual interviews were conducted with eight trainees, and repeated a year later. The purpose of these was to explore their experiences of supervision.
Four focus groups, two with supervisors and two with trainees, were run by J.M. The aim of these was to allow for ‘the discussion of taboo topics’, to ‘provide mutual support’ and ‘highlight (sub)cultural values or group norms’ [19].
Fifty sessions of supervision were audio-recorded in vivo.
Data collection took place from 2000 to 2003.
As there was little mention of the examination process in the focus groups or individual interviews, no extracts from those data sources are included in the following material. The material from those sources informed other aspects of the analysis, which will be the focus of forthcoming publications.
The supervisors recorded their supervision sessions in their offices using a supplied audiotape recorder. In the middle of each six-month attachment, J.M. posted them four coded audiotapes to record four sessions and return the set of tapes in the accompanying courier bags. It was intended to tape four sessions consecutively during the middle part of each attachment for three attachments.
A professional transcriber transcribed the material.
Recruitment and sampling
Participants were identified by a combination of ‘mixed purposeful’ and ‘opportunistic’ sampling [20, pp.169–181]. J.M. approached the Directors of Training from Auckland, Christchurch and Dunedin, requesting the names of trainees and supervisors who were not in their first year of supervising or being supervised, and including a mix of genders, different ages and ethnicity, and particularly including any Maori supervisors and trainees. As the Director of Training for Wellington at the time, J.M. did not recruit locally. Hamilton did not have a sufficiently large programme at that time to ensure anonymity. The participants were all the supervisors and their trainees who were willing to participate.
Analysis
Analysis was interpretive, informed by Foucaultian and postmodern concepts. It was based on Tesch's detailed description of the ‘mechanics of interpretational qualitative analysis’ [21], and influenced by Dey's descriptions of ‘reading and annotating’; ‘creating’ and ‘assigning categories’; and ‘splitting and splicing’ [22].
J.M. listened to all the tapes as part of correcting the transcriptions, which provided a thorough grasp of their content as well as the nuances conveyed by tone of voice and humour. After a careful and detailed reading of the transcripts as a whole, J.M. repeated this, looking for, and marking by hand, initial key themes, contradictions, paradoxes, confusions, omissions and missing information. Some transcripts were jointly analysed with one of her thesis supervisors (K.D.), as a means of reflecting upon and examining her assumptions. The transcripts were then coded using HyperRESEARCH software [23], which was used to generate ‘reports’. These are documents that group all of the extracts that have a particular code. This de-contextualizing renders the themes more apparent (although the individual elements can still be viewed in context as needed) thus allowing examination of commonalities and discrepancies. It also allows grouping of codes capturing similar underlying themes.
Four key themes emerged from this iterative process of reading and re-reading the grouped codes. Discussions with P.E. and other psychiatric and non-psychiatric colleagues and trainees supported these themes. One of these four key themes was ‘the examination’. The remaining three themes (exercising power, being a psychiatrist, and support) will be the focus of further publications.
An analysis with respect to ethnicity was not undertaken because of the probability of identifying the participants.
The quotations below from the audiotaped material were chosen to illustrate key points, mindful of journal space limitations. Such excerpts are, of necessity, decontextualized, and represent words, without intonation. Accordingly, while the reader may be able to suggest other possible interpretations of the material (in keeping with a postmodern epistemology), the authors suggest that the interpretations offered are congruent with their thesis, which is based on a wealth of data that they are unable to present in this context.
The data
All names are pseudonyms. To preserve anonymity, I have represented the three regions as given as X, Y, and Z and genders de-identified by using s/he or his/her throughout.
Supervision sessions are labelled with the region, then the supervisor's number, and the number of the recorded session. For example, X1-1 indicates a quote from supervisor number one, from region X, from the first of their four tapes.
Speakers are represented by
The transcriptions
To facilitate reading, the transcriptions are given in grammatically correct language, as far as possible without altering the meaning. Variations in prosody are shown as follows:
? indicates a rising inflexion.
indicates a pause.
[?] indicates inaudible material, the number of question marks indicating the number of possible words.
… indicates that material that preceded or followed this section of transcript has been omitted.
Where two speakers overlapped, this is shown by a comma at the end of the first speaker's speech, followed by the interruption and then continuation of the first speaker.
Results
The supervisors were four women and two men; the trainees were six women and seven men, in their third to final years of training, of various ages but none older than their supervisor. The quotations below are representative of the group, although not every supervisor is quoted. Trainees gave informed consent to participate after an initial approach by their supervisor.
It proved impossible to obtain recordings from four consecutive sessions and thus to explore the continuity of issues from one session to another. Not all supervisors completed 12 recordings for various reasons (e.g. registrar refusal, machine malfunction, not being assigned a registrar).
During this study, the examination was in its previous format of five written case histories; a written examination; and the clinical component, comprising an examination (‘viva’) based on an unobserved interview; an examination in general medicine; and a ‘consultancy viva’.
Supervisory discussions focused on three elements of the examination process: the case histories, written examination, and the clinical examination. Four subthemes emerged from the data, three of which suggested that supervisors used the examination as a justification for requiring good standards of practice. These were categorized as: the examination as a reason for studying and learning good psychiatric practice; what ‘the examiners’ want; and how to pass or fail. A fourth subtheme, being ‘a good practising psychiatrist’, presented the trainees with a paradox. Despite the coaching apparently required to meet the standards of the examinations, supervisors also told trainees that all that was required to pass the examination was to be ‘a good practising psychiatrist.’ While perhaps intended as supportive, supervisors seemed unaware of the contradiction inherent in this statement.
The examination as a justification for good practice
The examination as a reason for studying and learning good psychiatric practice
Until an examination loomed, the supervisors rarely expected evidence of theoretical study. In the face of the written examination, discussion focused on the need to study, with the implication that study was required purely because of the examination, rather than as an integral part of professional practice.
S (supervisor): Especially as you get towards the writtens and that, obviously you do, you have to bone up everything so that you're as crisp as you can be, you know in terms of all the basic stuff, but you do actually need to make sure that you've got yourself up to speed with the bits that you haven't had so much clinical experience with.
T (trainee): Mm.
S: You do actually need to do that reading about the bits that you haven't had so much of a chance to see. And to some extent although we see it the whole time, that includes alcohol stuff as well,
T: Yes.
S: because we often - you haven't done that run, the alcohol and drug one?
T: No.
S: And you won't, won't have done, you won't have done the kind of academic reading about that, about what works, what doesn't, what the syndromes are you know, um, yeah. (X1-9)
Although the supervisor acknowledged that alcohol and drug problems were seen ‘the whole time’, s/he had not suggested that the trainee study this area until it was necessary for the examination.
Trainees recognized this lack of discipline at times.
T: I'm sort of emerging now where I need to be really sort of focused on the writtens but I think that I should use supervision to actually present people in a reasonably standard type of way, which I haven't been doing, and I think I should be doing that as a bare minimum. (X2-2)
Supervisors also justified asking questions that probed a trainee's theoretical knowledge by referring to them as practice for the examination.
S Just for practice's sake, tell me what's an overvalued idea as opposed to a delusion or how do we define it, sort of thing?
T: I probably can't actually give you a textbook definition of an overvalued idea, but it's an idea, um… (X1-6)
This was a basic question asked of a senior trainee. It raises issues of why the supervisor did not have a clearer idea of the trainee's knowledge base, or deficits; why the trainee had reached this stage of training with such a deficit. Of particular note was the way in which the supervisor justified asking a theoretical question by referring to it as ‘for practice's sake’, which, in context, referred to examination practice.
There was a differentiation between exam-style, and ‘non-exam type’, management illustrated by a comment by a supervisor in the context of giving end-of-attachment feedback.
S: Management plans, again in terms of quickly making a management plan, and then presenting it verbally that, that's what you need to work on – in terms of your overall sort of non-exam type management planning I think, that's fine but in terms of an exam skill you need to just keep working, being able to do that in a relatively short space of time, pull it all together and present it in a clear manner (X1-3)
What the examiners want
The Committee for Examinations consists of a heterogeneous group of psychiatrists from varying subspecialisms, only some from academic backgrounds, both genders; and a wide range of ages. However, both supervisors and trainees referred to ‘the examiners’ as a homogeneous group who, at times, assumed almost mythical status in discussions of the examination process. This was often conveyed by discussions of ‘what the examiners want’. The basis for these statements was unclear. Although some supervisors had previously been examiners themselves, they still portrayed the examiners as demanding, rather hostile people with high, often unrealistic, expectations, particularly regarding the clinical examinations.
In the following example, the trainee had presented a case to practise ‘in exam style’. ‘They’ referred to the examiners.
S: I mean in the exam it's absolutely without doubt that you've got to, um, you know you're supposed to have a comprehensive management plan, all sorted out,
T: Yes.
S: you present that to me and,
T: Yes.
S: um, you need to sort of have your own head of steam up and to be sort of clear about it. If you get, if you sort of pause, and are vague about it and that, they won't be happy about that (X1-2)
The next extract refers to the importance of interesting the examiners.
S: Yeah, and with a really complicated history, where somebody's been able to give you lots of detailed history you can't actually do a ten minute presentation without formulating, say, the past history or the personal history in some way. Sometimes you have to kind of present a mini-formulation inside your presentation to actually fit the detail in. And that will work for the Examiners. If you can kind of tell them a story that makes sense even though you're not dotting every ‘i’ and crossing every ‘t’, you know if the examiners are fascinated by some tiny detail in the history that you've summarized basically, or formulated in that way, they'll bring you back to it and they'll say, ‘Just tell us a bit more about such and such.’ (X1-12)
In the following, the supervisor was discussing the management plan that the trainee had presented.
S: So if you told me it was appropriate for her to be here because of suicidal ideation would be a good way to start, um - usually you need more information. I know the examiners get really sick of this one. All the registrars say, ‘Well I need more information,’ and they get really bored with it but you might want to talk about interviewing the family or sister or whoever. (Z2-5)
The focus here was how to present the intention to gather information without ‘boring’ the examiners. The examiners featured less frequently and in less daunting terms when discussing the case histories, perhaps reflecting the higher pass rates for these [3].
How to pass (or fail)
The issues raised in this subtheme were similar to the issues raised above, but focused on the behaviours or ways of presenting material required to pass the examination, rather than focusing on the behaviours necessary to please the examiners. The key areas were prioritization; timing; being concise, thoughtful, and sensible; and, for the case histories, the choice of a case to present.
Prioritization
Supervisors emphasized the issue of prioritization in the clinical examination:
S: The point is not that you just sort of somehow squish everything into ten minutes, like, for example, by talking really fast. The point is the selection process you use. Which bits you leave out, which bits you put in, because in effect you're kind of in a way formulating even when you're doing the presentation in that you're,
T: These are the important things.
S: Yes, selecting the important bits yeah.
T: These are the salient points. (X1-12)
The requirement to prioritize was justified, in part, by the examination requirement to present the clinical findings, formulation and differential diagnosis in ten minutes.
Timing
Timing was therefore a focus in most practice presentations for the clinical examinations:
S: OK, two minutes over, about two and a bit minutes over.
T: Right.
S: So if you'd had more history you'd have had to summarize much more,
T: Yes.
S: carefully. Um, though, OK with the, um, I think you'd probably, if you'd had a person where you'd been able to get a bit more history and so you'd have had to reorganize it differently. I think you probably would have had to summarize down a bit on the presenting complaint perhaps. (X1-10)
Concerns about timing led to many detailed discussions.
T: And the time, should it be like three minutes on this and five minutes on that?
S: Oh, I don't know, now. I mean it really depends on the patient you've got. Um, I think you kind of get a feel for it as time goes on. I mean I wouldn't plot it but you just need to slightly watch your watch so that if it's sort of seven minutes and you haven't done, you're miles away from mental state, you might want to abbreviate your personal and developmental history and get on to the mental state. If you're going too long they'll tell you but it's better if you mark it for yourself. (Z1-5)
One of the skills necessary in order to ‘fit’ the presentation into 10 minutes was identifying key ‘relevant’ material, and knowing what to omit.
S: You've actually got quite a lot of detail, and it would have been good
T: Yep.
S: to practice, you know, summarizing it down, picking out the meat of it, making sure it fitted inside ten minutes. (X1-12)
Being concise, thoughtful and sensible
Other skills necessary to meet presentation time limits and case history word limits were the ability to be concise, thoughtful, and sensible. The first example was in response to the trainee having presented a case history as examination practice.
S: You were much more succinct than you sometimes have been. You've just told me what I needed to know without elaborating it. (Z1-5)
This seemed to be a form of praise or approval.
In the next example, during a discussion of why the trainee may have recently failed a clinical examination, the need to show balance was emphasized.
S: Still sometimes you get focused on one small part of the comprehensive management rather than getting the big over-picture, and just mentioning the individual components, short-term and medium-term. But getting that sort of balanced clinical judgement and conveying it is really important. (Y2-1)
Later in the same discussion the focus moved to demonstrating the capacity to be thoughtful, and taking time to think.
S: But I do, particularly when I've been asked a real curly question, take time to think ‘Now how am I going to answer this?’ And sometimes recently you've still been launching into your answer immediately, and you know you and I are working on that. Slow down, have a think, list in your mind the number of points you need to cover. (Y2-1)
Trainees also had to demonstrate being ‘sensible’. Similar to relevance, this involved being comprehensive without being over-inclusive.
S: It's always hard with a complicated person, to get the balance right between sort of mentioning the main things and yet going on and on with a way-too-big list which sort of leaves the examiner thinking, ‘Well, he's just gone down all the chapter headings in DSM and,
T: Yeah, right.
S: he's not very good at filtering and sorting.’
T: Being sensible.
S: Yeah. ‘He's not quite sensible enough’, sort of thing. (X1-6)
In all of these discussions, the performance aspect of the examination was emphasized. The discussions used words like, ‘show’, ‘demonstrate’, ‘look’ and ‘appear’, indicating the importance of how the candidates were seen by the examiners. This is demonstrated below, in which the trainee had just presented a patient management plan ‘in exam-style’.
S: If you're doing something like adding an antipsychotic for someone who's not psychotic, you need to say why. I mean is it perfectly reasonable, rational why you did that?
T: Right. It would actually pay to put it in there?
S: They may not ask you but it just looks better. (Z1-5)
Good ordinary practice
Despite the examination-focused suggestions above, there were, paradoxically, several statements suggesting that the examination merely required ‘good ordinary practice’. If so, there should have been little need for the degree of practice and preparation evident as the trainees neared their examinations! This discrepancy between the exhortations of behaviours necessary ‘for the exam’, and comments suggesting that the examination was a reflection of ‘ordinary practice’, is evident below.
S: So that the exams really ought not to come out of the blue as being completely unreasonable. (X2-2)
Similar advice was offered by another supervisor:
S: And secondly, particularly with the clinicals,
T: Yes,
S: I mean, what you need is to be in good ordinary practice in terms of being a good sensible clinician. (X1-3)
This apparent paradox was further complicated by supervisors referring to the ‘good, ordinary practice’ of a consultant psychiatrist, not a trainee.
S: The challenge for the clinical exam is that you have to demonstrate that both you are a proficient registrar, a very proficient registrar, that you've really got all of the skills required to be a very good registrar, and at the same time that you understand what the role of being a junior consultant is. Because you're not being examined to be a registrar. (X2-2)
One means that supervisors used to encourage trainees to make this transition from trainee to psychiatrist was to repeatedly emphasize the need to think ‘like a psychiatrist’.
S: But before you consider prescribing psychotherapy, you've got another role as her psychiatrist, which is to discuss lifestyle issues with her on an ongoing and reviewable basis. (Y2-3)
The current training process, in which trainees complete two years of advanced training after passing their exam, makes this even more challenging.
Discussion
This is the only published study that has examined psychiatric supervision in vivo to illuminate what takes place in supervision [17]. Many of the findings are congruent with other studies of supervision and teaching in undergraduate and other branches of postgraduate medicine. It also raises a number of issues about the practice of supervision in psychiatry that, if addressed, might significantly improve examination pass rates.
Until an examination was imminent, supervisors did not expect evidence of theoretical knowledge or formal case presentations. This may explain why trainees did not refer to examination preparation as a key component of supervision during their individual interviews and focus groups. That examinations drive learning, termed ‘consequential validity’ [24], is well recognized in the educational literature [25,26]. However, few authors examine ‘the educational impact of assessment on learning’ [27]. This impact is evident in the present material; the examinations are frequently ‘the reason’ for trainees being expected to study, learn, and demonstrate a knowledge base to their supervisors. At first glance, this does not seem problematic. It ensures that at some point trainees are forced to acquire a body of theoretical knowledge and the skills to formulate patients’ problems succinctly and produce coherent management plans, which are clearly desirable skills. The requirement to fine-tune these skills at some stage of training seems necessary. However, there is a danger that candidates attain only ‘mastery of the examination’, rather than true clinical expertise [5]. It also seems that, until the examination looms, supervisors have rarely scrutinized trainees’ knowledge and skills (as is mandatory) despite some 3–6 years of training. As shown above, some relatively advanced trainees were unable to describe basic elements of the mental state examination. The authors suggest that, rather than being acquired for the examination, the discipline of understanding the theoretical underpinnings of psychiatry, and being able to prioritize, be succinct and precise, are basic to good psychiatric practice and as such, should be fostered from the outset of training.
While supervisors described the examination as requiring demonstration of ‘good ordinary practice’, they also portrayed it as different from everyday work and the reason why trainees should, after some years in training, begin to study and learn to present cases formally. The distinction of ‘exam’ from ‘non-exam’ styles of presentation supports the contention that something different from ‘everyday practice’ is required to pass the examination. This is consistent with findings in other medical specialisms. Almost 30 years ago, Norman wrote that ‘it is increasingly evident that physicians frequently do not practice medicine in the manner they have been taught in medical school’[28, p. 25]. Similarly, more recently, ‘expert presentations are known for their brevity, sometimes to the point of taking a form of cryptic shorthand recognizable only to specialist colleagues’ [29] Atkinson asserted that ‘the extent to which the ritual formats are followed reflects the degree of formality of the occasion on which the presentation is made’ [30, p. 93]. Thus, trainees rarely see their supervisors present cases using this ritual format in the course of their daily work, although this may occur at formal presentations. One of the reasons for the low pass rate may, therefore, be that the everyday practice of psychiatrists, modelled to trainees, is different from the requirements of the examination.
This brevity in the presentations of supervisors may represent the practice of ‘illness script formation’, which has been proposed as characteristic of experts [31]. Thus examination practice (which coaches trainees to prioritize and present information succinctly) could be seen, using this framework, as encouraging progression from a novice to intermediate level of practice (ibid.). This is in keeping with the ability to prioritize developing as the trainee gains the experience to judge what is important. The process by which senior registrars/junior consultants later learn to abbreviate presentations and utilize ‘illness scripts’ as experts, is not apparent.
The ‘performance’ element of the examination, reflected in words like ‘show’, ‘demonstrate’ and ‘appear’, is also described in other branches of medicine, in both undergraduate and postgraduate settings. Atkinson referred to this as the ‘dramatical and liturgical’ aspects of case presentations [30] and the ‘element of performance that runs through so much medical culture’ (ibid, p. 149).
Supervisors emphasized the skills of prioritization, relevance, being sensible, concise and thoughtful as necessary for passing the RANZCP examinations. These same attributes are considered necessary for undergraduate and other postgraduate medical students [30,32–35, p.483], and noted in examination ‘hints’ for the RANZCP examinations [36]. Prioritization is an ‘essential strategy’ for medical students learning to present case histories [29]. Haber and Lingard found medical students were repeatedly advised to practise ‘relevance’, although this was neither defined, nor able to be defined, by their teachers [34]. Similarly the Royal College of General Practitioners examination emphasizes ‘rationality’, ‘coherence’, and ‘consistency’ [35, p.483]. It seems plausible that this emphasis on brevity, conciseness and prioritization may serve a similar function here as elsewhere: i.e. the ‘rhetorical function’ of proving the trainees’ competence and suitability to become a member of the discipline [9]. Conversely, if these are essential skills of a psychiatrist, it is unclear why these are not a focus of supervision until the trainee is approaching an examination.
The portrayal of the examiners as hostile, demanding adversaries to be ‘pleased’ resonates with examiners’ experience of how they are viewed, although it contrasts with their avowed aim to help candidates do their best. Perhaps this allows supervisors to abrogate to the examiners the expectation of high standards and allows them to occupy a supportive position, avoiding the evaluative and managerial roles of a supervisor [37,38]. This desire to be seen as supportive may also explain the apparent contradiction of the supervisors asserting that the exam merely requires ‘good ordinary practice’ (a supportive and reassuring statement), while requiring the demonstration of knowledge and skills ‘for the exam’ (which relieves them of the responsibility for being demanding). The issue of the roles, and particularly the supportive role, of supervision for trainees and supervisors is the subject of a forthcoming article.
The major limitations of this study are its timing and its focus on the individual supervision hour. Since this data was collected, the examination format and supervision requirements have changed. However, the examinations pass rates remain similar. Whether the increased emphasis on training supervisors has altered the nature of supervision is unclear. The focus on the individual hour of supervision mandated by the College by-laws was because individual supervision is seen as a central element of how trainees are supported in developing their clinical skills. However, it is acknowledged that this is not the only venue in which supervision occurs. As a qualitative study, this was not aiming for generalizability, but typicality. It could be argued that these supervisors were not ‘typical’ in that they were prepared to expose themselves by participating in such a study, suggesting an interest in supervision. However, the effect of this on the findings is unknown.
Conclusion
The RANZCP has devoted considerable time and effort in recent years to improving the metrics of its examinations, particularly their reliability. Despite these efforts, the pass rates have remained largely unchanged, particularly in the clinical examination. This report suggests a possible explanation, in that supervisors do not see the examinations as a reflection of the daily practice of psychiatrists or trainees, but as a performance, requiring specific preparation. This is in keeping with the concern that ‘assessment methods that are cost- and reliability-driven threaten to overlook vital needs that the associated learning is relevant to practice and reinforces lifelong learning.’ [27]. While this mismatch between ‘exam style’ and routine presentations continues, we suggest that pass rates will remain low. For this to change, the daily work of trainees needs to be aligned with the requirements of the examination, or vice versa. The authors suggest that, if the requirements of the examination do represent the standards expected of psychiatrists, then it is the daily practice and supervision of trainees that needs to change, rather than the examination. Van der Vleuten has recently made a ‘plea for a shift of focus regarding assessment’, so that assessment ‘changes from a psychometric problem…to an educational design problem that encompasses the entire curriculum’ [15]. However, such a realignment, while in keeping with the curricular changes proposed by the Board of Education [1], would require supervisors to change from their current predominantly supportive role to also actively evaluating trainee performance from the start of training, and using this evaluation to foster development of clinical attitudes, knowledge and skills. Although the proposals for change recognize the need for supervisor training, to be successful they must also inculcate a major change in supervisor roles and attitudes.
Footnotes
Acknowledgements
I wish to thank Kevin Dew of Victoria University, a supervisor of my PhD, for his invaluable support and intellectual contribution to the project; and the supervisors and trainees who participated in the study.