Development of a Mental Health Objective Structured Clinical Examination (OSCE): A Pilot Study

Abstract

Background.An Objective Structured Clinical Examination (OSCE) is an effective means for clinical evaluation related to clinical competence. Historically, clinical assessment is based on direct observation which can be rooted in subjectivity. Evidence is minimal regarding psychometric properties of an OSCE in undergraduate mental health nursing education, although established as a valid measure of clinical competence.

Method.A pilot study was conducted with a convenience sample of 13 sixth semester nursing students to evaluate the validity and reliability of an OSCE framework designed to assess undergraduate mental health nursing students’ clinical competence. The research questions asked: What are the psychometric properties of a mental health OSCE developed for undergraduate nursing education? How do students perceive the OSCE?

Results.Preliminary validity and reliability measures were established, including content and criterion validities and inter-rater reliability. Students perceived the OSCE as both a beneficial and challenging learning experience.

Conclusion.The OSCE was initially established as a valid, reliable tool for the assessment of clinical competence in undergraduate mental health nursing education. This evaluation method may be used to objectively assess mental health nursing knowledge, skills, and attitudes in conjunction with existing clinical assessment methods to provide a valid, reliable measure of clinical competence. More testing is needed.

Keywords

clinical competence nursing education OSCE reliability validity

Get full access to this article

View all access options for this article.

References

Alinier

(2003). Nursing students’ and lectures’ perspectives of Objective Structured Clinical Examination incorporating simulation. Nurse Education Today, 23(6), 419-426. doi:10.1016/S0260-6917(03)00044-3

American Association of Colleges of Nursing (AACN). (2008). The essentials of baccalaureate education for professional nursing practice. Retrieved from http://www.aacn.nche.edu/education-resources/baccessentials08.pdf

American Nursing Association. (2015). Nursing scope and standards of practice (3rd ed.). Silver Springs, MD: Author.

Aronowitz

Mardin-Small

Kim

(2017). Using Objective Structured Clinical Examination (OSCE) as education in advanced practice registered nursing education. Journal of Professional Nursing, 33(2), 119-125. doi:10.1016/j.profnurs.2016.06.003

Bartfay

W. J.

Rombough

Howse

Leblanc

(2004). The OSCE approach in nursing education. Canadian Nurse, 100(3), 18-23. Available from https://www.canadian-nurse.com

Cant

R. P.

Cooper

S. J.

(2010). Simulation-based learning in nurse education: Systematic review. Journal of Advanced Nursing, 66(1), 3-15. doi:10.1111/j.1365-2648.2009.05240.x

Cant

McKenna

Cooper

(2013). Assessing preregistration nursing students’ clinical competence: A systematic review of objective measures. International Journal of Nursing Practice, 19(2), 163-176. doi:10.1111/ijn.12053

Dreifuerst

K. T.

(2015). Getting started with Debriefing for Meaningful Learning. Clinical Simulation in Nursing, 11(5), 268-275. doi:10.1016/j.ecns.2015.01.005

Goh

H. S.

Tang

M. L.

Devi

M. K.

K. C. E.

Lim

L. M.

(2016). Testing the psychometric properties of Objective Structured Clinical Examination (OSCE) in nursing education in Singapore. Singapore Nursing Journal, 43(1), 12-18. Retrieved from http://www.sna.org.sg/publications/singapore-nursing-journal/

10.

Hall

(2017). Simulation-based learning in Australian undergraduate mental health nursing curricula: A literature review. Clinical Simulation in Nursing, 13(8), 380-389. doi:10.1016/j.ecns.2017.04.002

11.

Harden

R. M.

Lilley

Patricio

(2016). The definitive guide to the OSCE: The Objective Structured Clinical Examination as a performance assessment. New York, NY: Elsevier.

12.

Harden

R. M.

Stevenson

Downie

W. W.

Wilson

G. M.

(1975). Assessment of clinical competence using objective structured examination. The British Medical Teacher Journal, 1(5955), 447-451. doi:10.1136/bmj.1.5955.447

13.

Hodges

Hanson

McNaughton

Regehr

(2002). Creating, monitoring, and improving a psychiatry OSCE: A guide for faculty. Academic Psychiatry, 26(3), 134-161. doi:10.1176/appi.ap.26.3.134

14.

IBM Corp. (2016). IBM SPSS statistics for windows, version 24.0. Armonk, NY: Author.

15.

Jones

Pegram

Fordham-Clarke

(2010). Developing and examining an Objective Structured Clinical Examination. Nurse Education Today, 30(2), 137-141. doi:10.1016/j.nedt.2009.06.014

16.

Kardong-Edgren

Adamson

K. A.

Fitzgerald

(2010). A review of currently published evaluation instruments for human patient simulation. Clinical Simulation in Nursing, 6(1), e25-e35. doi:10.1016/j.ecns.2009.08.004

17.

Kardong-Edgren

Mulcock

P. M.

(2016). Angoff method of setting cut scores for high-stakes testing: Foley catheter checkoff as an exemplar. Nurse Educator, 41(2), 80-82. doi:10.1097/NNE.0000000000000218

18.

Kaufman

D. M.

Mann

K. V.

Miujtjens

A. M. M.

Van der Vleuten

C. P. M.

(2000). A comparison of standard setting procedures for an OSCE in undergraduate medical education. Academic Medicine, 75(3), 267-271.

19.

Krippendorff

(2012). Content analysis: An introduction to its methodology (3rd ed.). Thousand Oaks, CA: SAGE.

20.

Lewis

K. L.

Bohnert

C. A.

Gammon

W. L.

Hölzer

Lyman

Smith

. . . Gliva-McConvey

(2017). The Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP). Advances in Simulation, 2(10), 1-8. doi:10.1186/s41077-017-0043-4

21.

Lioce

Meakim

C. H.

Fey

M. K.

Chmil

J. V.

Mariani

Alinier

(2015). Standards of best practice: Simulation standard IX: Simulation design. Clinical Simulation in Nursing, 11(6), 309-315. doi:10.1016/j.ecns.2015.03.005

22.

Lopreiato

J. O.

(Ed.), Downing

Gammon

Lioce

Sittner

Slot

Spain

A. E.

(Assoc. Eds.), and the Terminology & Concepts Working Group. (2016). Healthcare Simulation Dictionary. Rockville, MD: Agency for Healthcare Research and Quality; October 2016. AHRQ Publication No. 16(17)-0043.

23.

Lynn

M. R.

(1986). Determination and quantiﬁcation of content validity. Nursing Research, 35(6), 382-385. Retrieved from https://journals.lww.com/nursingresearchonline/pages/default.aspx

24.

McNaughton

Anderson

(2017). Standardized patients: It’s all in the words (Editorial). Clinical Simulation in Nursing, 13(7), 293-294. doi:10.1016/j.ecns.2017.05.014

25.

McWilliam

Botwinski

(2008). Developing a successful nursing Objective Structured Clinical Examination. Journal of Nursing Education, 49(1), 36-41. doi:10.3928/01484834-20090915-01

26.

Miller

G. E.

(1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9, Suppl.), S63-S67. Retrieved from https://journals.lww.com/academicmedicine/pages/default.aspx

27.

Mitchell

M. L.

Henderson

Groves

Dalton

Nulty

(2009). The Objective Structured Clinical Examination (OSCE): Optimising its value in the undergraduate nursing curriculum. Nurse Education Today, 29(4), 398-404. doi:10.1016/j.nedt.2008.10.007

28.

Newble

(2004). Techniques for measuring clinical competence: Objective Structured Clinical Examinations. Medical Education, 38(2), 199-203. Retrieved from https://onlinelibrary.wiley.com/journal/13652923

29.

Norman

I. J.

Watson

Murrells

Calman

Redfern

(2002). The validity and reliability of methods to assess the competence to practise of pre-registration nursing and midwifery students. International Journal of Nursing Studies, 39(2), 133-145. doi:10.1016/S0020-7489(01)00028-1

30.

Oermann

M. H.

Kardong-Edgren

Rizzolo

M. A.

(2016). Summative simulated-based assessment in nursing programs. Journal of Nursing Education, 55(6), 323-328. doi:10.3928/01484834-20160516-04

31.

Pierre

R. B.

Wierenga

Barton

Branday

J. M.

Christie

C. D. C.

(2004). Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BMC Medical Education, 4, Article 22. doi:10.1186/1472-6920-4-22

32.

Polit

D. F.

Beck

C. T.

(2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489-497. Retrieved from https://onlinelibrary.wiley.com/journal/1098240x

33.

Polit

D. F.

Beck

C. T.

(2017). Nursing research: Generating and assessing evidence for nursing practice (10th ed.). Philadelphia, PA: Wolters Kluwer.

34.

Quality and Safety Education for Nurses Institute (QSEN). (2011). Competencies. Retrieved from http://qsen.org/competencies

35.

Rushforth

H. E.

(2007). Objective Structured Clinical Examination (OSCE): Review of literature and implications for nursing education. Nurse Education Today, 27(5), 481-490. doi:10.1016/j.nedt.2006.08.009

36.

Selim

A. A.

Ramadan

F. H.

El-Gueneidy

M. M.

Gaafer

M. M.

(2012). Using Objective Structured Clinical Examination (OSCE) in undergraduate psychiatric nursing education: Is it reliable and valid? Nurse Education Today, 32(3), 283-288. doi:10.1016/j.nedt.2011.04.006

37.

Smith

Muldoon

Biesty

(2012). The Objective Structured Clinical Examination (OSCE) as a strategy for assessing clinical competence in midwifery education in Ireland: A critical review. Nurse Education in Practice, 12(5), 242-247. doi:10.1016/j.nepr.2012.04.012

38.

Vijayalakshmi

Revathi

(2017). Performance of nursing students in psychiatric nursing using Objective Structured Clinical Examination (OSCE) versus traditional practical examination (TPE)- a Comparative approach. Asian Journal of Nursing Education and Research, 7(4), 561-568. doi:10.5958/2349-2996.2017.00109.4

39.

Wallace

Rao

Haslam

(2002). Simulated patients and Objective Structured Clinical Examinations: Review of their use in medical education. Advances in Psychiatric Treatment, 8(5), 342-350. doi:10.1192/apt.8.5.342

40.

Walsh

Bailey

P. H.

Koren

(2009). Objective structured clinical evaluation of clinical competence: An integrative review. Journal of Advanced Nursing, 65(8), 1584-1595. doi:10.1111/j.1365-2648.2009.05054.x

41.

Watson

Stimpson

Topping

Porock

(2002). Clinical competence assessment in nursing: A systematic review of the literature. Journal of Advanced Nursing, 39(5), 421-431. Retrieved from https://onlinelibrary.wiley.com/journal/13652648