This study was a formative evaluation of a retention program for nursing students from disadvantaged backgrounds underrepresented in the nursing workforce. The program was challenging, with multilevel interventions, some implemented sequential and some concurrently. The program team was interested to identify problematic components and address them quickly, as part of implementing a rapid cycle improvement process to ensure program success. Evaluators proposed a novel three-wave formative evaluation to meet this need. Results indicated high satisfaction with the program as well as several unintended consequences related to clinical rotations and interaction with faculty and nonparticipant students. The contribution of this study is demonstrating a practical application of an innovative evaluation approach and tool, to assist program planning and to inform remedial actions in a timely manner. Lessons learned from this study contribute to explaining the dynamic nature of unintended effects of program components in the real world and their assessment.
American Association of Colleges of Nursing. (2016). 2015–2016 Salaries of instructional and administrative nursing faculty. Washington, DC: AACN.
2.
BambergerM.TarsillaM.Hesse-BiberS. (2016). Why so many “rigorous” evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute. Evaluation and Program Planning, 55, 155–162. doi:10.1016/j.evalprogplan.2016.01.001
3.
BravemanP.EgerterS.WilliamsD. R. (2011). Social determinants of health: Coming of age. Annual Reviews of Public Health, 32, 381–398. doi:10.1146/annurev-publhealth-031210-101218
4.
ChenH. (2015). Practical program evaluation: Theory-driven evaluation and the integrated evaluation perspective (2nd ed.). Los Angeles, CA: Sage.
5.
CondonV. M.MorganC. J.MillerE. W.MamierI.ZimmermanG. J.MazharW. (2013). A program to enhance recruitment and retention of disadvantaged and ethnically diverse baccalaureate nursing students. Journal of Transcultural Nursing, 24, 397–407. doi:10.1177/1043659613493437
6.
CreswellJ. W. (2012). Qualitative inquiry and research design: Choosing among five approaches. Thousand Oaks, CA: Sage.
7.
Health Workforce Technical Assistance Center. (2016). Rapid cycle quality improvement resource guide: A resource guide to accelerate improvement efforts for Health Resources and Services Administration (HRSA) Grantees (p. 20). Albany: School of Public Health University at Albany, State University of New York.
8.
JabeenS. (2016). Do we really care about unintended outcomes? An analysis of evaluation theory and practice. Evaluation and Program Planning, 55, 144–154. doi:10.1016/j.evalprogplan.2015.12.010
9.
JeffreysM. (2012). Nursing student retention: Understanding the process and making a difference (2nd ed.). New York, NY: Springer.
10.
KrippendorffK. (1980). Content analysis: An introduction to its methodology. London, England: Sage.
MilesM. B.HubermanA. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.
13.
MorellJ. A. (2017). Systematic iteration between model and methodology: A proposed approach to evaluating unintended consequences. Evaluation and Program Planning. doi:10.1016/j.evalprogplan.2017.09.006
14.
MurrayT. A.PoleD. C.CiarioE. M.HolmesS. (2016). A nursing workforce diversity project: Strategies for recruitment, retention, graduation, and NCLEX-RN success. Nursing Education Perspectives, 37, 138–143. doi:10.5480/14-1480
15.
National Academies of Science, Engineering, and Medicine. (2011). Expanding underrepresented minority participation. Washington, DC: National Academies.
16.
National Research Council. (2004). In the nation’s compelling interest: Ensuring diversity in the health-care workforce. Washington, DC: The National Academies.
17.
OzogulG.SullivanH. (2009). Student performance and attitudes under formative evaluation by teacher, self and peer evaluators. Educational Technology Research & Development, 57, 393–410. doi:10.1007/s11423-007-9052-7
18.
PritchardT. J.PerazzoJ. D.HoltJ. A.FishbackB. P.McLaughlinM.BankstonK. D.GlazerG. (2016). Evaluation of a summer bridge: Critical component of the leadership 2.0 program. Journal of Nursing Education, 55, 196–202.
19.
RamirezL. K. B.BakerE. A.MetzlerM. (2008). Promoting health equity: A resource to help communities address social determinants of health. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention.
20.
RossiP. H.LipseyM. W.FreemanH. E. (2004). Evaluation: A systematic approach. Thousand Oaks, CA: Sage.
21.
ScrivenM. (1967). The methodology of evaluation. In TylerR. W.GagneR. M. (Eds.), Perspectives of curriculum evaluation (Vol. I, pp. 39–83). Chicago, IL: Rand McNally.
22.
ScrivenM. (1991). Beyond formative and summative evaluation. Chicago, IL: University of Chicago.
23.
SilvermanD. (2006). Interpreting qualitative data: Methods for analyzing talk, text and interaction. Thousand Oaks, CA: Sage.
24.
SwinneyJ. E.DobalM. T. (2008). Embracing the challenge: Increasing workforce diversity in nursing. Hispanic Health Care International, 6, 200–204. doi:10.1891/1540-4153.6.4.200
25.
TateS. A.BagguleyP. (2017). Building the anti-racist university: Next steps. Race Ethnicity and Education, 20, 289–299. doi:10.1080/13613324.2016.1260227
26.
U.S. Department of Education, Office of Planning, Evaluation and Policy Development, & Office of the Under Secretary. (2016). Advancing diversity and inclusion in higher education. Washington, DC: Author.
27.
USU Health Action Groups. (2013). Urban universities: An action plan for transforming the future health workforce. Washington, DC: Urban Universities for HEALTH.
28.
WestA. (2012). Formative evaluation of the transition to postgraduate study for counselling and psychotherapy training: Students’ perceptions of assignments and academic writing. Counselling & Psychotherapy Research, 12, 128–135. doi:10.1080/14733145.2011.604425
