Spaced education (SE) is a learning strategy that can improve long-term knowledge retention. Inspired by the concept of SE, we conducted a mixed methods study of a smartphone application (app) as a platform of SE. Objectives were to: (phase 1 quantitative) estimate the extent to which weekly alerts on the app can stimulate medical residents to visit the app, (connection of phases) identify participants for a second qualitative phase, and (phase 2 qualitative) describe factors, from the resident perspective, which influence sustainable participation in SE, and describe strategies for improvement of the app. Methodology and methods: phase-1 design was pre-experimental, phase-2 design was qualitative descriptive (deductive-inductive thematic analysis). Results: We observed a stimulating effect of weekly alerts for the first two months of the one-year study. Per participant, alert visits varied from 0 to 34 (mean 1.7, SD 4.5) and total page visits varied from 1 to 442 (mean 28.3, SD 61.4). Barriers and facilitators to sustainable participation in SE fell into five dimensions: user-related factors, information content factors, mobile app design factors, alert system factors, and service factors. Four strategies for app improvement were described. In conclusion, we propose five dimensions pertaining to potential predictors of sustainable participation in SE to deliver clinical information.
Allen,T.Brailovsky,C.Rainsberry,P.Lawrence,K.Crichton,T.Carpentier,M.P., & Visser,S. (2011). Defining competency-based evaluation objectives in family medicine: dimensions of competence and priority topics for assessment. Canadian Family Physician, 57(9), e331-340.
2.
Augustin,M. (2014). How to learn effectively in medical school: test yourself, learn actively, and repeat in intervals. Yale Journal of Biology and Medicine, 87(2), 207-212.
3.
Blazek,M.C.Dantz,B.Wright,M.C., & Fiedorowicz,J.G. (2016). Spaced learning using emails to integrate psychiatry into general medical curriculum: keep psychiatry in mind. Med Teach, 38(10), 1049-1055. doi: 10.3109/0142159x.2016.1150982.
4.
Carpenter,S.K.Cepeda,N.J.Rohrer,D.Kang,S.H.K., & Pashler,H. (2012). Using spacing to enhance diverse forms of learning: review of recent research and implications for instruction. Educational Psychology Review, 24(3), 369-378. doi: 10.1007/s10648-012-9205-z.
5.
Creswell,J.Plano Clark,V.Gutmann,M., & E Hanson,W. (2003). Advanced mixed methods research designs. In Tashakkori,A., & Teddlie,C., (Eds.), Handbook on Mixed Methods in the Behavioral and Social Sciences, pp. 209-240. Thousand Oaks, CA: Sage.
6.
Creswell,J.W., & Plano Clark,V.L. (2011). Designing and conducting mixed methods research. Los Angeles: SAGE Publications.
7.
Custers,E. (2010). Long-term retention of basic science knowledge: a review study. Adv Health Sci Educ Theory Pract, 15(1), 109-128. doi: 10.1007/s10459-008-9101-y.
8.
Delone,W.H., & McLean,E.R. (2003). The delone and mclean model of information systems success: a ten-year update. Journal of Management Information Systems, 19(4), 9-30.
9.
Fereday,J., & Muir-Cochrane,E. (2006). Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods International Journal of Qualitative Methods, 5(1), 80-92.
10.
Grad,R.Pluye,P.Wong,E.Brailovsky,C.Moscovici,J.L.Kaczorowski,J.Rodriguez,C.Luconi,F.Rousseau,M.Karanofsky,M.Delleman,B.Kegel,S.Mercuri,M.Kluchnyk,M., & Schabort,I. (2017). Do Weekly Alerts From a Mobile Application Influence Reading During Residency? 1. doi: 10.22454/PRiMER.2017.243866.
11.
Greb,A.E.Brennan,S.McParlane,L.Page,R., & Bridge,P.D. (2009). Retention of medical genetics knowledge and skills by medical students. Genetics in Medicine, 11(5), 365-370. doi: 10.1097/GIM.0b013e31819c6b2d.
12.
Houston,R.D. (2009). A model of compelled nonuse of information. (Doctoral dissertation), The University of Texas at Austin, Austin, TX. Retrieved from http://hdl.handle.net/2152/6896.
13.
Ivankova,N.Creswell,J., & Stick,S. (2006). Using mixed-methods sequential explanatory design: from theory to practice. Field Methods, 18(1), 3-20.
14.
Ivankova,N.V., & Stick,S.L. (2007). Students’ persistence in a distributed doctoral program in educational leadership in higher education: a mixed methods study. Research in Higher Education: Journal of the Association for Institutional Research, 48(1), 93-135.
15.
Janssen,A.Shaw,T.Bradbury,L.Moujaber,T.Norrelykke,A.M.Zerillo,J.A.LaCasce,A.Co,J.P.Robinson,T.Starr,A., & Harnett,P. (2016). A mixed methods approach to developing and evaluating oncology trainee education around minimization of adverse events and improved patient quality and safety. BMC Med Educ, 16, 91. doi: 10.1186/s12909-016-0609-1.
Kaiser,K. (2009). Protecting respondent confidentiality in qualitative research. Qualitative Health Research, 19(11), 1632-1641. doi: 10.1177/1049732309350879.
18.
Kane-Gill,S.L.O’Connor,M.F.Rothschild,J.M.Selby,N.M.McLean,B.Bonafide,C.P.Cvach,M.M.Hu,X.Konkani,A.Pelter,M.M., & Winters,B.D. (2017). Technologic distractions (Part 1): summary of approaches to manage alert quantity with intent to reduce alert fatigue and suggestions for alert fatigue metrics. Crit Care Med, 45(9), 1481-1488. doi: 10.1097/ccm.0000000000002580.
19.
Kerfoot,B.P. (2009). Learning benefits of on-line spaced education persist for 2 years. Journal of Urology, 181(6), 2671-2673. doi: 10.1016/j.juro.2009.02.024.
20.
Morgan,D.L. (1998). Practical strategies for combining qualitative and quantitative methods: applications to health research. Qualitative Health Research, 8(3), 362-376. doi: 10.1177/104973239800800307.
21.
Pereira,J.Peden,J., & Campbell,K. (2000). Instructional technology in medical education: lessons learnt. J Telemed Telecare, 6(Suppl 2), S56-58.
22.
Ramos,D. (2015). Usability Assessment of a Mobile Application: Experience and Effects among Family Medicine Residents. (Master of Arts Master’s thesis), McGill, Montreal. Retrieved from http://digitool.Library.McGill.CA:80/R/-?func=dbin-jump-full&object_id=135696&silo_library=GEN01 Available from McGill eScholarship@McGill database. (135696)
23.
Robinson,T.Janssen,A.Kirk,J.DeFazio,A.Goodwin,A.Tucker,K., & Shaw,T. (2017). New approaches to continuing medical education: a QStream (spaced education) program for research translation in ovarian cancer. J Cancer Educ, 32(3), 476-482. doi: 10.1007/s13187-015-0944-7.
24.
Sackett,P.R., & Mullen,E.J. (1993). Beyond formal experimental design: towards an expanded view of the training evaluation process. PEPS Personnel Psychology, 46(3), 613-627. doi: 10.1111/j.1744-6570.1993.tb00887.x.
25.
Salkind,N.J. (2010). Encyclopedia of research design.
26.
Sandelowski,M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18(2), 179-183.
27.
Sandelowski,M. (2000). Whatever happened to qualitative description? Research in Nursing and Health, 23(4), 334-340.
28.
Shaw,T.J.Pernar,L.I.Peyre,S.E.Helfrick,J.F.Vogelgesang,K.R.Graydon-Baker,E.Chretien,Y.Brown,E.J.Nicholson,J.C.Heit,J.J.Co,J.P., & Gandhi,T. (2012). Impact of online education on intern behaviour around joint commission national patient safety goals: a randomised trial. BMJ Qual Saf, 21(10), 819-825. doi: 10.1136/bmjqs-2011-000702.
Tong,A.Sainsbury,P., & Craig,J. (2007). Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care, 19(6), 349-357.
31.
Whiting,L.S. (2008). Semi-structured interviews: guidance for novice researchers. Nursing Standard (Royal College of Nursing (Great Britain): 1987), 22(23), 13-19.
32.
Yeh,D.D., & Park,Y.S. (2015). Improving learning efficiency of factual knowledge in medical education. Journal of Surgical Education, 72(5), 882-889. doi: 10.1016/j.jsurg.2015.03.012.
