Supporting Canadian Nursing Students to Write the NCLEX-RN Exam: A Three-Phased Mixed Methods Descriptive Design

Abstract

Background

In 2015, the College of Nurses of Ontario, replaced the Canadian Registered Nurse Examination with the NCLEX-RN exam as entry-to-practice. Faculty in a college-university partnership searched for products to provide nursing students with focused practice in writing exams modelled on the Canadian NCLEX-RN test plan.

Purpose

The aim of this three-phased evaluation study was to test and validate NCLEX-RN exam preparation materials newly developed for the Canadian context.

Methods

A mixed methods descriptive design was used to capture subjective perspectives and objective measures. After ethical approval was obtained, 13 students assessed the e-learning platform's usability. Eight faculty/clinical experts assessed the content validity of materials using a content validity index (CVI) at both item (I-CVI), and scale (S-CVI) levels. Lastly, 72 completed tests served as the basis for assessing psychometric properties of selected test items.

Results

Materials were assessed as useful and easy to use and navigate. I-CVIs ranged between 0.5 to 1.0 with none falling below 0.5 while S-CVIs were above the standard for acceptability of greater than 0.8 with none falling below 0.9. Overall test reliability measured by the Kuder-Richardson formula was 0.73. Many items assessed for difficulty (64%) showed a proportion of correct responses within desired ranges, and most point-biserial indices ranged from fair to very good.

Conclusion

Strong evidence supported the usability and content validity of the materials assessed. Item difficulty and discrimination analyses were within acceptable ranges. Suggestions for improvements were offered. Predictive analysis should form the basis of future research in this area.

Keywords

NCLEX-RN exam preparation NCLEX-RN exam resources entry-to-practice usability content validity item difficulty

Get full access to this article

View all access options for this article.

References

Braun

Clarke

. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun

Clarke

. (2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238

Canadian Council of Registered Nurse Regulators. (2021). NCLEX-RN 2020: Canadian and international results. http://www.ccrnr.ca/assets/ccrnr-nclex-rn-2020---canadian-and-international-results---en.pdf.pdf

Canadian Council of Registered Nurse Regulators. (2017, May 11). NCLEX-RN 2016: Canadian results. http://www.ccrnr.ca/assets/nclex-rn-2016-canadian-results.pdf

Canadian Nurses Association. (2016, March 7). 2015 NCLEX-RN results signal several major issues with nurse licensing exam [Press Release]. https://www.cnaaiic.ca/en/news-room/news-releases/2016/2015-nclex-rn-results-signal-several-major-issues-with-nurse-licensing-exam

Clarke

S. P

. (2017). What you need to know about the NCLEX-RN®. Nursing Management, 48(10), 21–23. https://doi.org/10.1097/01.NUMA.0000524821.72029.0a

Cox-Davenport

R. A.

Phelan

J. C

. (2015). Laying the groundwork for NCLEX success. CIN: Computers, Informatics, Nursing, 33(5), 208–215. https://doi.org/10.1097/CIN.0000000000000140

Creswell

Plano Clark

. (2011). Designing and conducting mixed methods research (2nd ed.). Sage.

Czekanski

Mingo

Piper

. (2018). Coaching to NCLEX-RN success: A postgraduation intervention to improve first-time pass rates. Journal of Nursing Education, 57(9), 561–565. https://doi.org/10.3928/01484834-20180815-10

10.

Davids

R. M.

Chikte

U. M. E.

Halperin

M. L

. (2014). Effect of improving the usability of an e-learning resource: A randomized trial. Advances in Physiology Education, 38(2), 155–160. https://doi.org/10.1152/advan.00119.2013

11.

Davis

F. D

. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008

12.

Davis

J. H.

Marrow

M. R

. (2021). Professional preparation: faculty practices for NCLEX-RN success. Nursing Science Quarterly, 34(4), 360–365. https://doi.org/10.1177/08943184211031581

13.

Davis

L. L

. (1992). Instrument review: getting the most from your panel of reviewers. Applied Nursing Research, 5(4), 194–197. https://doi.org/10.1016/S0897-1897(05)80008-4

14.

De Lima

London

Manieri

. (2011). Looking at the past to change the future: A retrospective study of associate degree in nursing graduates’ national council licensure examination scores. Teaching & Learning in Nursing, 6(3), 119–123. https://doi.org/10.1016/j.teln.2011.01.001

15.

De Von

H. A.

Block

M. E.

Moyle-Wright

Ernst

D. M.

Hayden

S. J.

Lazzara

D. J.

Savoy

S. M.

Kostas-Polston

. (2007). A psychometric toolbox for testing validity and reliability. Journal of Nursing Scholarship, 39(2), 155–164. https://10.1111/j.1547-5069.2007.00161.x https://doi.org/10.1111/j.1547-5069.2007.00161.x

16.

Farley Serembus

. (2016). Improving NCLEX first-time pass rates: A comprehensive program approach. Journal of Nursing Regulation, 6(4), 38–44. https://doi.org/10.1016/S2155-8256(16)31002-X

17.

Fielding

N. G

. (2012). Triangulation and mixed methods designs: data integration with new research technologies. Journal of Mixed Methods Research, 6(2), 124–136. https://doi.org/10.1177/1558689812437101

18.

Freire

L. L.

Azeres

P. M.

Campos

J. C

. (2012). A literature review about usability evaluation methods for e-learning platforms. Work, 41(Suppl. 1), 1038–1044. https://doi.org/10.3233/WOR-2012-0281-1038

19.

Glasgow

M. E. S.

Dreher

H. M.

Schreiber

. (2019). Standardized testing in nursing education: Preparing students for NCLEX-RN and practice. Journal of Professional Nursing, 35(6), 440–446. https://doi.org/10.1016/j.profnurs.2019.04.012

20.

Havrilla

Zbegner

Victor

. (2018). Exploring predictors of NCLEX-RN success: One school’s search for excellence. Journal of Nursing Education, 57(9), 554–556. https://doi.org/10.3928/01484834-20180815-08

21.

Holloway

Galvin

(2017). Qualitative research in nursing and healthcare (4th ed). Wiley-Blackwell.

22.

Holmes

K. M

. (2017). Research at colleges in ontario: Learning from the past and looking towards the future. College Quarterly, 20(3), 1–16. http://collegequarterly.ca/2017-vol20-num03-summer/research-at-colleges-in-ontario.html

23.

Horton

Polek

Hardie

T. L

. (2012). The relationship between enhanced remediation and NCLEX success. Teaching & Learning in Nursing, 7(4), 146–151. https://doi.org/10.1016/j.teln.2012.06.002

24.

House

S. K.

Sweet

. (2021). Adapting NCLEX-RN remediation during the COVID-19 pandemic. Journal of Teaching and Learning in Nursing, 17(2), 237–239. https://doi.org/10.1016/j.teln.2021.11.005

25.

Joseph

. (2018, March 15). A multiple case study of associate degree nursing student experiences on NCLEX-RN preparation. Paper presented at the Nursing Education Research Conference 2018: Generating and Translating Evidence for Teaching Practice. Washington, DC, from https://sigma.nursingrepository.org/handle/10755/623850

26.

Kingsbury

G. G.

Hauser

. (2004, April). Computer adaptive testing and the No child left behind act. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Retrieved from http://www.psych.umn.edu/psylabs/catcentral/pdf%20files/ki04-01.pdf

27.

Koch

. (2014). The nursing educator's role in e-learning: A literature review. Nurse Education Today, 34(11), 1382–1387. https://doi.org/10.1016/j.nedt.2014.04.002.

28.

Laitusis

C. C.

Cook

L. L.

Buzick

H. M.

Stone

(2011). Adaptive testing options for accountability assessments. In Russell

(Ed.), Assessing students in the margins: challenges, strategies, and techniques. Information Age Publishing.

29.

Lawshe

C. H

. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

30.

Lobiondo-Wood

Haber

Cameron

Singh

M. D.

(2018). Nursing research in Canada: methods and critical appraisal and utilization (4th Canadian Ed.). Elsevier Canada.

31.

Lynn

. (1986). Determination and quantification of content validity index. Nursing Research, 35(6), 382–386. https://doi.org/10.1097/00006199-198611000-00017

32.

McCloskey

Stewart

Burke

L. K

. (2019). Predictors of success in the NCLEX-RN for Canadian graduates. Nursing Leadership, 32(4), 30–45. https://doi.org/10.12927/cjnl.2020.26103

33.

McDonald

M. E.

(2008). Developing trustworthy classroom tests. In Penn

B. K.

(Ed.), Mastering the teaching role: A guide for nurse educators (pp. 275−286). FA Davis.

34.

McGahee

T. W.

Ball

. (2009). How to read and really use an item analysis. Nurse Educator, 34(4), 166–171. https://doi.org/10.1097/NNE.0b013e3181aaba94

35.

McGillis Hall

Lalonde

Kashin

. (2016, November). People are failing! something needs to be done: Canadian students’ experience with the NCLEX-RN. Nurse Education Today, 46, 43–49. https://doi.org/10.1016/j.nedt.2016.08.022.

36.

Meehan

C. D.

Barker

. (2021). Remediation for NCLEX-RN success in high-risk nursing students. Journal of Teaching and Learning in Nursing, 16(3), 254–257. https://doi.org/10.1016/j.teln.2021.02.003

37.

Menon

Korner-Bitensky

Chignell

Status

. (2012). Usability testing of two e-learning resources: methods to maximize potential for clinician use. Journal of Rehabilitation Medicine, 44(4), 338–345. https://doi.org/10.2340/16501977-0952

38.

Monroe

H. E.

Dunemn

. (2020). Determining postgraduation readiness to take the NCLEX-RN. Journal of Nursing Education, 59(2), 101−106. https://doi.org/10.3928/01484834-20200122-09

39.

National Council of State Boards of Nursing. (2022). NCLEX And other exams. Computerized Adaptive Testing (CAT). https://www.ncsbn.org/1216.htm

40.

National Council of State Boards of Nursing (2014, Spring). Pencils down, booklets closed. The evolution of the NCLEX®: 20 years as a computer adaptive exam. In Focus. ncsbn.org/InFocus_Spring2014.pdf

41.

Opsahl

A. G.

Auberry

Sharer

Shaver

. (2018). A comprehensive educational approach to improving NCLEX-RN pass rates. Nursing Forum, 53(4), 549–554. https://doi.org/10.1111/nuf.12285

42.

Owusu

Sweetman

. (2015). Regulated health professions: outcomes by place of birth and training. Canadian Public Policy 41(S1), 98–115. https://doi.org/10.3138/cpp.2015-008

43.

Pence

Wood

. (2018). Using computer-adaptive quizzing as a tool for national council licensure examination success. Nursing Education Perspectives, 39(3), 164–166. https://doi.org/10.1097/01.NEP.0000000000000289

44.

Petrovic

Doyle

Lane

Corcoran

. (2019). The work of preparing Canadian nurses for a licensure exam originating from the USA: A nurse educator’s journey into the institutional organization of the NCLEX-RN. International Journal of Nursing Education Scholarship, 16(1), 1–8. https://doi.org/10.1515/ijnes-2018-0052

45.

Polit

D. F.

Beck

C. T

. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing and Health, 29(5), 489–497. https://doi.org/10.1002/nur.20147

46.

Polit

D. F.

Tatano Beck

C. T.

Owen

S. V

. (2007). Focus on research methods. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing and Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199

47.

Quinn

B. L.

Smolinski

Peters

. (2018). Strategies to improve NCLEX-RN success: A review. Journal of Teaching and Learning in Nursing, 13(1), 18–26. https://doi.org/10.1016/j.teln.2017.09.002

48.

Romeo

E. M

. (2013). The predictive ability of critical thinking, nursing GPA, and SAT scores on first-time NCLEX-RN performance. Nursing Education Perspectives, 34(4), 248–253. https://doi.org/10.5480/1536-5026-34.4.248

49.

Salfi

Carbol

. (2017). The applicability of the NCLEX-RN to the Canadian testing population: A review of the regulatory body evidence. International Journal of Nursing Education Scholarship, 14(1), 1–10. https://doi.org/10.1515/ijnes-2016-0078

50.

Samuels

. (2015). Advice on reliability analysis with small samples [Technical Report]. https://doi.org/10.13140/RG.2.1.1495.5364

51.

Sandars

. (2010). The importance of usability testing to allow e-learning to reach its potential for medical education. Education for Primary Care, 21(1), 6–8. https://doi.org/10.1080/14739879.2010.11493869

52.

Sandars

Lafferty

. (2010). Twelve tips on usability testing to develop effective e-learning in medical education. Medical Teacher, 32(12), 956–960. https://doi.org/10.3109/0142159X.2010.507709

53.

Shatto

Shagavah

Krieger

Lutz

Duncan

C. E.

Wagner

E. K

. (2019). Active learning outcomes on NCLEX-RN or standardized predictor examinations: an integrative review. Journal of Nursing Education, 58(1), 42–46. https://doi.org/10.3928/01484834-20190103-07

54.

Shoemaker

J. R.

Chavez

R. A.

Keane

Butz

Yowler

S. K

. (2017). Effective utilization of computerized curricular assistive tools in improving NCLEX-RN pass rates for a baccalaureate nursing program. CIN: Computers, Informatics, Nursing, 35(4), 194–200. https://doi.org/10.1097/CIN.0000000000000311.

55.

Six

J. M.

,, & Macefield

(2016). How to determine the right number of participants for usability studies. https://www.uxmatters.com/mt/archives/2016/01/how-to-determine-the-right-number-of-participants-for-usability-studies.php

56.

Trofino

R. M

. (2013). Relationship of associate degree nursing program criteria with NCLEX-RN success: what are the best predictors in a nursing program of passing the NCLEX-RN the first time? Teaching & Learning in Nursing, 8(1), 4–12. https://doi/10.1016/j.teln.2012.08.001

57.

Varma

. (2006). Preliminary item statistics using point-biserial correlation and p-values. Educational data Systems, Inc [PDF]. https://eddata.com/wpcontent/uploads/2015/11/EDS_Point_Biserial.pdf

58.

Yaghmaie

. (2003). Content validity and its estimation. Journal of Medical Education, 3(1), 25–27. https://doi.org/10.22037/jme.v3i1.870.