A comparative discourse study of simulated clinical roleplays in two assessment contexts: Validating a specific-purpose language test

Abstract

The aim of this paper is to investigate from a discourse analytic perspective task authenticity in the speaking component of the Occupational English Test (OET), an English language screening test for clinicians designed to reflect the language demands of health professional–patient communication. The study compares the OET speaking sub-test roleplay performances of 12 doctors who were successful OET candidates with practice Objective Structured Clinical Examination (OSCE) roleplay performances of 12 international medical graduates (IMGs) preparing for the Australian Medical Council clinical examination. The premise for the comparison is that the OSCE roleplays can represent communication practices that are valued within the medical profession; therefore a finding of similarity in the discourse structure across the OET and the OSCE roleplays could be taken as supporting the validity of the OET as a tool for eliciting relevant communication skills in the medical profession.

The study draws on genre theory as developed in Systemic Functional Linguistics (SFL) in order to compare the roleplay discourse structure and the linguistic realizations of the two tasks. In particular, it examines the role relationships of the participants (i.e. the tenor of the discourse), and the ways in which content is represented (i.e. the field of the discourse) by roleplay participants. The findings reveal some key similarities but also important differences. Although both tests inevitably fall short in terms of authentic representation of real world interactions, the findings suggest that the OET task, for a range of reasons including time allowances, training of test interlocutors, and the limits of contextual information provided to candidates, constrains candidate topic exploration and treatment negotiation, compared to the OSCE format. The paper concludes with proposals for mitigating these limitations in the interests of enhancing the OET’s capacity to elicit more professionally relevant language and communication skills.

Keywords

Assessing language for specific purposes discourse analysis medical discourse Occupational English Test Objective Structure Clinical Examination (OSCE)roleplay interaction systemic functional linguistics

Get full access to this article

View all access options for this article.

References

Basturkmen

Elder

(2004). The practice of LSP. In Davies

Elder

(Eds.), Handbook of applied linguistics. Malden, MA: Blackwell.

Chapelle

(2012). Conceptions of validity. In Fulcher

Davidson

(Eds.), The Routledge handbook of language testing (pp. 21–33). Abingdon: Routledge.

De Haes

Bensing

(2009). Endpoints in medical communication research, proposing a framework of functions and outcomes. Patient Education Counselling, 74, 287–94.

Dorgan

K. A.

Lang

Floyd

Kemp.

(2009). International medical graduate-patient communication: a qualitative analysis of perceived barriers. Academic Medicine, 84, 1567–1575.

Douglas

(2000). Assessing languages for specific purposes. Cambridge: Cambridge University Press.

Douglas

(2001). Three problems in testing language for specific purposes: Authenticity, specificity and inseparability. In Elder

Brown

Grove

Hill

Iwashita

Lumley

McNamara

O’Loughlin

(Eds.), Experimenting with uncertainty: Essays in honour of Alan Davies. Cambridge: Cambridge University Press.

Eggins

Slade

(1997). Analysing casual conversation. London: Cassell.

Elder

Pill

Woodward-Kron

McNamara

Manias

McColl

Webb

(2012). Health professionals’ views of communication: Implications for assessing performance on a health-specific English language test. TESOL Quarterly, 46, 409–419.

Elder

McNamara

Woodward-Kron

Manias

McColl

Webb

Pill

O’Hagan

(2013). Developing and validating language proficiency standards for non-native English speaking health professionals. Papers in Language Testing and Assessment: An International Journal of the Association for Language Testing and Assessment of Australia and New Zealand, 2(1), 66–70.

10.

Fulcher

(2003). Testing second language speaking. Harlow: Pearson Longman.

11.

Halliday

M. A. K.

(1994). An introduction to functional grammar. London: Arnold.

12.

Harden

R. M.

Stevenson

Wilson Downie

Wilson

G. M.

(1975). Assessment of clinical competence using objective structured examination. British Medical Journal, 1(5955), 447–451.

13.

Johnson

(2001). The art of non-conversation: A reexamination of the validity of the oral proficiency interview. New Haven, CT: Yale University Press.

14.

Lumley

Brown

(1996). Specific-purpose language performance tests: Tasks and interaction. Australian Review of Applied Linguistics, Series S, 13, 105–136.

15.

Martin

J. R.

(1992). English text: System and structure. Philadelphia, PA: John Benjamins.

16.

Martin

J. R.

(1997). Analysing genre: functional parameters. In Christie

Martin

J. R.

(Eds.), Genre and institutions: Social processes in the workplace and school (pp. 3–39). London: Continuum.

17.

Martin

J. R.

(2001). Language, register and genre. In Burns

Coffin

(Eds.), Analysing English in a global context. London: Routledge.

18.

McNamara

(1996). Measuring second language performance. London: Longman.

19.

McNamara

T. F.

(1997). ‘Interaction’ in second language performance assessment: Whose performance? Applied Linguistics, 18, 446–466.

20.

McNamara

Hill

May

(2002). Discourse and assessment. Annual Review of Applied Linguistics, 22, 221–242.

21.

Pill

(2013). What doctors value in consultations and the implications for specific-purpose language testing. PhD thesis, University of Melbourne.

22.

Pill

Woodward-Kron

(2012). How professionally relevant can language tests be? A response to Wette. Language Assessment Quarterly, 9,105–108.

23.

Pryor

Woodward-Kron

(2014). International medical graduate doctor to doctor telephone communication: A genre perspective. English for Specific Purposes, 35, 41–53.

24.

Rothman

A. I.

Cusimano

(2000). A comparison of physician examiners’, standardized patients’, and communication experts’ ratings of international medical graduates’ English proficiency. Academic Medicine, 75, 1206–1211.

25.

Rothman

A. I.

Cusimano

(2001). Assessment of English proficiency in international medical graduates by physician examiners and standardized patients. Medical Education, 35(8), 762–766.

26.

Spence-Brown

(2001). The eye of the beholder: Authenticity in an embedded assessment task. Language Testing, 18, 463–481.

27.

Van Zanten

Boulet

J. R.

McKinley

Whelan

G. P.

(2003). Evaluating the spoken English proficiency of international medical graduates: Detecting threats to the validity of standardised patient ratings. Medical Education, 37, 69–76.

28.

Van Zanten

Boulet

J. R.

Norcini

J. J.

McKinley

(2005). Using a standardised patient assessment to measure professional attributes. Medical Education, 39(1), 20–29.

29.

Woodward-Kron

Fraser

Pill

Flynn

(2014). How we developed Doctors Speak Up: an evidence-based language and communication skills open access resource for International Medical Graduates. Medical Teacher, 37(1), 31–33.