Implementing health worker training on sepsis in South Eastern Nigeria using innovative digital strategies: an interventional study

Abstract

Background:

Sepsis is a leading cause of morbidity and mortality especially in low- and middle-income countries such as Nigeria. Training of health workers using digital platforms may improve knowledge and lead to better patient outcomes.

Objectives:

To assess the effectiveness of a digital health educational module on sepsis in improving the knowledge of medical doctors in Cross River State Nigeria on the diagnosis and management of patients presenting with sepsis.

Design:

Quasi-experimental analytical study.

Methods:

We developed and deployed a sepsis module through an innovative application (Sepsis tutorial app) to doctors in Calabar, Nigeria. We assessed quantitative pre- and post-intervention knowledge scores for those completing the tutorial on sepsis between both assessments. A user satisfaction survey evaluated the content of the tutorial and the usability of the app.

Results:

One hundred and two doctors completed the course. There were more males than females (58.8% versus 41.2%). Over half (52%) were junior doctors, a minority were general practitioners and house officers (3% and 5%, respectively), and 72.6% had practiced for periods ranging from 1 to 15 years post-qualification. Gender and age appeared to have no significant association with pre- and post-test scores. The oldest age group (61–70) had the lowest mean pre- and post-test scores, while general practitioners had higher mean pre- and post-test scores than other cadres. The majority (95%) of participants recorded higher post-test than pre-test scores with a significant overall increase in mean scores (25.5 ± 14.7%, p < 0.0001). Participants were satisfied with the content and multimodal delivery of the material and found the app usable.

Conclusion:

Digital training using context-responsive platforms is feasible and may be used to close the critical knowledge gap required to respond effectively to medical emergencies such as sepsis in low- to middle-income settings.

Plain language summary

Training health workers on sepsis using digital strategies

Sepsis occurs when the body injures itself as it attempts to fight an infection. It is now recognized as a leading cause of death especially in low- and middle-income countries such as Nigeria. Training of health workers using digital platforms may improve knowledge and lead to better patient outcomes. We assessed the effectiveness of a digital health educational course on sepsis in improving the knowledge of medical doctors in Cross River State, Nigeria on the diagnosis and management of patients presenting with sepsis. One hundred and two doctors completed the course. Most participants recorded higher post-test than pre-test scores, were generally satisfied with the content and delivery of the material, and found the app usable. We conclude that digital training using digital platforms may be useful in bridging the critical knowledge gap required to respond effectively to sepsis in low- to middle-income settings.

Keywords

health worker training implementing sepsis

Background

Sepsis, a syndrome of life-threatening organ dysfunction due to a dysregulated host response to infection,¹ is a leading cause of morbidity and mortality worldwide particularly in Africa where awareness is low and resources are limited.² Sepsis-related deaths in sub-Saharan Africa (sSA) reflect underlying political, poverty, health inequity and health system challenges across Africa.³ There are limited population incidence data for sepsis in sSA. However, data from a few available cohort studies suggest that the case fatality rates of sepsis in sSA are higher than in high-income settings.^4,5 The Global Burden of Disease 2020 study estimates 16.7 million cases of sepsis in sSA annually, accounting for up to 45% of all deaths in many of those countries.^6,7

Reduction in sepsis-related mortality linked to the implementation of the sepsis-6 care bundle has been a major priority worldwide.⁸ Sepsis-6 is a clinical care bundle comprised of six processes, namely, blood culture, full blood count and lactate estimation, intravenous (IV) fluid challenge, IV antibiotic(s) administration, urine output monitoring and oxygen therapy. When completed within the first hour of admission, these six processes have been associated with decreased mortality.⁹ Every hour of delay in antibiotic administration has been associated with an 8% increase in mortality among patients with septic shock.¹⁰ A recent systematic review with potential selection and information bias and confounding suggests that the evidence for the recommended timings is not robust.¹¹ Nonetheless, the recommendations reflect the need for heightened awareness and urgency in the initial evaluation and target-driven approach to the management of sepsis.¹² Lack of awareness of early symptoms of several other conditions and their seriousness has been linked to delayed health-seeking behaviour among people in several African contexts.^13
–15 In high-income countries, public awareness of sepsis has been increasing due to concerted efforts by several groups such as the Surviving Sepsis Campaign and a multitude of national regulators and expert bodies.^14
–16 However, in low- and middle-income country settings such as Nigeria, a low level of awareness of sepsis and its early symptoms in combination with other factors may be contributing to poorer outcomes.¹⁷

Medical personnel can introduce further delays in accessing appropriate care due to a lack of knowledge of sepsis. Although the recommendations for sepsis recognition and management have evolved over the years, it is unclear if healthcare workers especially doctors are fully aware of these changes. Presently, there are no local guidelines for the management of patients with sepsis and critically unwell patients in Cross River State (CRS) located in south-eastern Nigeria. Infection prevention and control (IPC) policies that promote practical, evidence-based approaches for preventing patients and health workers from avoidable infections are also lacking in hospitals in CRS. Consequently, the medical care of these groups of patients may not reflect evidence-based international standards.

In this study, the key research question was: ‘How effective is training delivered via context-responsive digital platforms in closing critical knowledge gaps in sepsis management among medical doctors in CRS Nigeria’? We aimed to evaluate the effectiveness of a digital health educational module on sepsis in improving the knowledge of medical doctors across CRS on the diagnosis and management of patients presenting with sepsis. This was complemented by the deployment of education, information and communication (EIC) materials for the general population to raise awareness of and reduce the likelihood of death from sepsis in CRS.

Materials and methods

Study setting

CRS is in Southeast Nigeria and has a population of approximately 3,866,300 people who are served by 3 tertiary-level facilities, 16 secondary-level facilities (general hospitals) and 909 primary-level facilities. Approximately 70% of clients in CRS access care from public facilities. The University of Calabar Teaching Hospital (UCTH) is a 500-bed hospital that has two intensive care units (adults and neonatal) and serves as the main specialist referral hospital in CRS. Between 2012 and 2017, nearly 50,000 patients were admitted. Of about 2200 deaths, 26.0% were attributable to sepsis.¹⁸ Sepsis has also been reported to cause high maternal mortality in the UCTH.^19,20

Conceptual framework

The theoretical underpinning for this study is linked to the three-layered behaviour change wheel, which comprises opportunity, capability and motivation.²¹ These factors are in turn driven by attributes such as physical, physiological, psychological and social behaviours. In addition, the delivery of interventions such as training, education and incentivization of practice enable behavioural change, which ultimately leads to outcomes such as the adoption of guidelines and better service provision.⁸ In the context of a healthcare setting, deployment of a digital training platform against the backdrop of current suboptimal behaviour in the care of sepsis patients at the UCTH should lead to a change in better IPC practices, earlier diagnosis and quick responses, as well as appropriate antibiotic prescribing for sepsis patients. Ultimately, these improvements should lead to a reduction in morbidity and mortality (Figure 1).

Figure 1.

Behaviour change framework: incorporates sources of behaviour, intervention targets and outcomes.

Design

Our group, the Foundation for Healthcare Innovation and Development (FHIND) leveraged its extensive experience in developing and deploying low-cost and effective digital training materials for health workers to address diseases such as Ebola Virus Disease and COVID-19.^22,23 This was a quasi-experimental study that involved the collection and analysis of both quantitative and semi-quantitative data. There were quantitative pre- and post-intervention knowledge assessments for the health workers who had to complete the electronic training module on sepsis between both assessments. The assessment was followed by an end-of-module user satisfaction survey that collected both quantitative and semi-quantitative data using a semi-structured instrument containing some open-ended questions after which the health workers were eligible for Continuing Professional Development (CPD) credit points issued by the Nigerian Medical Association (NMA), CRS branch. The course contents had been assessed and accredited by the NMA, CRS. To be eligible for inclusion in the study, participants had to be medical doctors practising in CRS and who could provide informed consent electronically. Doctors practising outside the state and those not able to provide electronic informed consent were excluded from the study.

Sampling

The sample size was calculated using an online sample size estimator.²⁴ The margin of error was 10%, at a 95% confidence level, the population size of 778 (the total number of registered practising doctors in the state), and the response rate of 49%.²⁵ This gave a recommended sample size of 86.

Implementation

The sepsis tutorial app was made freely available to all doctors in CRS via the NMA CRS between 24 October and 20 December 2021 to promote representativeness. The tutorial app (Figure 2) was designed using the Video training (VTR) mobile training platform by InStrat Global Health Solutions (https://instratghs.com/), an indigenous information technology company. The tutorial app was based on content developed by the FHIND (www.fhind.org) and aligned with international best practices. The app had three sections which were accessible in series namely pre-test assessment, training module completion with effort tracker and post-test assessments. The training module consisted of a voice-over slide set covering key concepts in sepsis management. These included recognition and early assessment, identifying the source of infection, diagnostic and prognostic value of blood markers for sepsis, initial treatment, monitoring and red flags for escalation of care. There were also two open-source complementary videos with animations lasting about 3 and 4 min each. Finally, the PDF version of the slide set was downloadable for offline study for participants with connection problems. Only on completion of these activities were participants able to do the post-test assessment.

Figure 2.

Sepsis tutorial app.

Upon reading the study information, interested participants completed a consent form electronically. User logins were then created and sent to the participants via WhatsApp, email and SMS by the staff of InStrat to download onto their Android mobile phones and computers. The participants then used the learning material at their convenience, and their progress was tracked automatically. We estimated that it would take the participants at least 90 min to complete the module. InStrat and FHIND staff were involved in monitoring responses, pre-test, training module completion and post-test assessments, and support was provided to the participants via WhatsApp and telephone. A standardized end-of-course evaluation survey was sent out to all participants who completed the sepsis training course using Google Forms. This survey was aimed at understanding how effective the FHIND/ARCS sepsis tutorial app was for disseminating vital information to health workers. Participants were eligible for a certificate of completion and/or CPD certificates upon scoring at least 80% in the post-test assessment and completing the course evaluation survey.

Statistical analysis

The socio-demographic characteristics of the participants were described using relative frequency. To test for homogeneity of variances and normality of distribution of the pre-test and post-test scores, we utilized Levene’s test and Shapiro–Wilk’s test, respectively. Logarithmic transformation was performed on the pre- and post-test scores to allow the use of parametric tests for analysis. Paired Student’s t test was used for comparing the mean pre- and post-test scores of the participants, while analysis of variance (ANOVA) was used to compare them between groups. A cut-off of p < 0.05 was used to determine significance. Analyses were conducted using SPSS Statistics for Windows, Version 20.0 (IBM Corp., Armonk, NY, USA).

Results

A total of 778 requests for participation were sent out via email to medical doctors across several affiliates of the Nigerian Medical Association and facilities in CRS. Of these, 110 consented to participate in the study, out of which 102 completed the study. A review of the various cadres revealed that the different cadres of doctors were represented proportionally based on the size of their groups within the larger sampling frame. There were more males than females (58.8% versus 41.2%), over half (52%) were junior doctors, a minority were general practitioners and house officers (3% and 5%, respectively) and 72.6% had practised for periods ranging from 1 to 15 years post-qualification (Table 1).

Table 1.

Characteristics of study respondents (medical doctors: n = 102).

Charateristic	Frequency n = 102	Percent (%)	Pre-test score (mean ± SD)	Post-test score (mean ± SD)
Gender
Male	60	58.8	59.5 (14.2)	83.3 (6.0)
Female	42	41.2	56.7 (11.5)	84.5 (5.3)
				83.8 (5.7)^a
Age
21–30	11	10.8	61.5 (15.6)	81.2 (3.9)
31–40	48	47.1	58.7 (14.0)	84.7 (6.1)
41–50	31	30.4	56.6 (11.8)	82.7 (4.7)
51–60	9	8.8	59.1 (12.3)	87.3 (7.7)
61–70	3	2.9	55.0 (10.1)	80.0 (0.0)
Professional level
Consultant	27	26.5	56.9 (10.8)	82.9 (4.5)
General practitioner	3	2.9	71.0 (10.1)	97.7 (4.0)
House officer^b	5	4.9	61.4 (14.5)	80.0 (0.0)
Medical officer^c	14	13.2	52.1 (8.7)	85.6 (7.2)
Resident doctor	53	52.0	59.7 (14.7)	83.1 (5.1)
Years of practice (post-qualification) (years)
1–5	15	14.7	58.9 (14.5)	83.0 (4.8)
6–10	33	32.4	59.5 (14.9)	85.7 (6.6)
11–15	26	25.5	60.0 (12.4)	81.5 (3.8)
16–20	12	11.8	44.8 (10.6)	83.2(4.4)
21–25	8	7.8	60.6 (11.4)	87.4 (8.2)
>25	8	7.8	54.8 (9.3)	82.5 (4.9)

Mean (SD) for males and females.

Newly graduated temporarily licensed doctors undergoing the mandatory 12-month internship in Nigeria.

Fully registered doctors who are not undergoing residency training of any form.

The mean pre-test score for all participants was 58.3 ± 13.2, while the mean post-test score was 83.8 ± 5.7. Age-wise and gender-wise, there were no statistically significant differences in the pre- and post-test scores of the participants. However, there was a statistically significant difference (p < 0.0001) in post-test scores across the different professional levels when compared with general practitioners (accounting for the highest post-test scores) and house officers (accounting for the lowest post-test scores). Furthermore, post-test scores also significantly differed (p = 0.037) according to the number of years of practice.

The overall mean scores showed a highly significant (p < 0.0001) improvement in the post-test scores of all the participants compared to their pre-test scores with a mean difference of 25.5 ± 14.7% between pre- and post-test scores.

Levene’s test showed equal variance for the post-test scores across the categories of age (p = 0.001), professional level (p = 0.000) and years of post-qualification practice (p = 0.001). The Shapiro–Wilk test showed that the distribution of both the pre- and post-test scores departed significantly from normality (W = 0.962, p < 0.05 for pre-test scores and W = 0.690, p < 0.05 for post-test scores). Based on this result, log transformation of the data was performed to enable the conducting of parametric tests.

A variable representing the difference between pre- and post-test scores was computed for all respondents. The mean of this variable was also computed for all the groups of participants, and ANOVA was used to compare these means across the various groups (Figure 3). The mean difference of scores of the house officers was lower than the average of all participants, while that of the general practitioners was higher than the average.

Figure 3.

Variation in the mean differences between pre- and post-test scores.

User survey

A total of 102 participants completed the user evaluation survey. Responses were obtained from open-ended questions from a semi-structured user satisfaction survey questionnaire. They responded usability, user experience and applicability of the app and the knowledge derived from it (see Supplemental File 1). Seventy (69%) of them stated they had received some training on sepsis before embarking on the FHIND/ARCS course. Most respondents (73/102, 71.2%) answered that they ‘agree strongly’ that the app had provided them with a better understanding of sepsis, and 69 (67.6%) stated that the knowledge and skills obtained would be applied in the work environment. Positive feedback on the usability of the app and user experience was obtained from 82 and 88 doctors, respectively. Sixty-nine of them stated that they would prefer e-learning courses to face-to-face training. A summary of the user evaluation survey is shown in Figure 4.

Figure 4.

User evaluation survey.

Some illustrative quotes from participants’ feedback from the user satisfaction survey are provided in the box below.

Box 1.

Illustrative quotes from user satisfaction survey.

Comments • ‘I enjoyed the session, the inclusion of different delivery methods – explanatory, audiovisuals and diagrams, aid in imprinting the knowledge.’ (Resident Doctor)• ‘This virtual training is an exceptional innovation to be promoted. Overall, my experience was top-notch.’ (Medical Officer)• ‘I wish to thank the team for this innovation. This is the first e-learning course made in a resource-limited setting like ours, I have partaken in.’ (Resident Doctor)• ‘The course is timely and has enriched my knowledge on the management of patients with sepsis/septic shock’ (Medical Officer)

Suggested areas for improvement. • ‘The video section should be like a live demonstration not just the audio. More videos. Occasional visuals of presenters may be helpful’ (Medical Officer)• ‘Text copies of the video and audio material as not all persons are complete audio-visual learners.’ (Resident Doctor)• ‘If the modules could be downloaded and stored and the training done offline before uploading it could help those with erratic internet connectivity.’ (Consultant)• ‘Navigation through the app can be improved especially when going back to the course page; it does not take someone back when one finishes a material. Fantastic job!’ (Resident doctor)• ‘Would follow-up short reminders and an assessment of its impact be done months after?’ (Resident Doctor)• ‘What about non-clinicians and clinicians as well, can there be more emphasis on integrated preventive strategies?’ (Resident Doctor)

Discussion

Our study involved the use of a digital health platform to train 102 doctors in CRS on sepsis with statistically significant improvements in their pre- and post-test scores. The majority of those trained reported an improvement in their understanding of sepsis and an acquisition of skills relevant to their work. Many also provided positive feedback on the usability of the app.

In Nigeria, conventional face-to-face residential training is the typical approach for providing CPD for doctors. However, this approach is fraught with financial and logistic challenges, including the need for resourcing persons who conduct the training, the need for travel of participants across long distances to attend the training and limited options for scalability.²⁶

The increasing availability and use of digital technologies in Nigeria offer new opportunities to support remote training. Nigeria has the fastest-growing mobile market on the African continent with current smartphone subscriptions in Nigeria estimated to be between 20 and 40 million and projected to grow to 140 million by 2025.²⁷ This increasing penetration provides a springboard for launching digital training for health workers on key topics such as sepsis as many health workers in Nigeria own smartphones.²⁸

Our previous efforts utilizing a similar app to train frontline health workers in Nigeria on Ebola Virus Disease²² and COVID²³ demonstrated the feasibility and acceptability of using digital technology to deliver training programmes in low-resource settings. The findings of our study corroborate two systematic reviews that concluded that digital training was as effective as traditional methods for improving the knowledge and skills of health workers.^29,30 Our findings align with research conducted in other African settings such as Rwanda,³¹ the Democratic Republic of the Congo,³² Uganda³³ and Ethiopia³⁴ where video conferencing was successfully utilized to improve knowledge, skills of healthcare professionals concerning maternal and child health. In these countries, VTR delivered at minimal cost led to increases in health worker knowledge and self-confidence in the management of obstetric and newborn emergencies and was feasible and acceptable to the health workers.

Our research showed that post-test scores also significantly differed according to the number of years of practice. This is not surprising as doctors who had been exposed to a longer duration of post-qualification experience and training were likely to have better knowledge of sepsis when compared to the younger and less experienced doctors. Overall, based on the behaviour change framework, improved knowledge driven by an mhealth resource is expected to engender behaviour change leading to early identification, diagnosis and clinical guideline-based treatment of sepsis and consequent improvement of outcomes.^8,21 Admittedly, these downstream outcomes will have to be evaluated in follow-up studies.

Our research findings have several implications for health policy, practice and future research. Our digital training approach holds promise for low- and middle-income country settings with minimal health budgets as it can provide high-quality, customized information to health workers in a safe, minimally disruptive and inexpensive way. We were also able to evaluate learning using the same platform, which is an added advantage.

Participants appreciated the multimodal delivery strategy of the course, which included a mix of texts, audio–visuals and diagrams. Indeed, a shift from a teacher-centric to a student-centric approach has been demonstrated to motivate learners, keep them attentive and improve learning.³⁵ Content creation for e-learning platforms such as the one deployed here would need to consider this shift for optimal learning outcomes.

Although there was a significant improvement in the post-test scores, further evaluation is needed to determine whether the newly acquired knowledge and skills will translate to behaviour change in early recognition of sepsis and prompt the institution of appropriate care. Given the pre-post-test design of the study and the use of self-reported data from a self-selecting group without control group or randomization, we are less certain about how much the use of the app has contributed to knowledge gain. However, we think that the scores are reflective of the likely trend in a comparative study.

We identified some challenges in the process of executing this intervention. First, several of the users found it challenging to download the app due to incompatibility issues. InStrat staff had to provide remote instructions to users on the WhatsApp platform to guide them through the entire process. Second, some of the users had limited storage space to run the sepsis app on their mobile devices, and they had to be directed to delete unused/seldom-used apps and old media files to create space on their devices. For subsequent deployments of this and similar training apps, it is apparent that extensive training and supportive supervision will be vital for users who are not technologically savvy. Also, power outages and poor internet connectivity³⁶ are challenges to digital training that will need to be addressed before deploying such training at a large scale.

Although the analysed sample was within the calculated size, it was not possible to obtain a proportionate representation of the different groups of doctors based on the group sizes within the population of 778 doctors. A more robust response to the survey would have been more helpful. A randomized controlled study would be an appropriate design to address this effectiveness question. However, because of time constraints due to the very tight reporting timelines for this National Institute for Health Research Global Health Research project, we opted for a quasi-experimental study using a pre-post design with the intention of conducting a future (and larger) study to guide the upscaling subsequently. Finally, as we were assessing only knowledge improvement and not skills, we considered the pre-post design appropriate.

Conclusion

In conclusion, with improved information communication infrastructure and appropriate e-health support, digital training in sSA is scalable and can sustainably close the critical knowledge gap required to respond more effectively to medical emergencies such as sepsis in low- to middle-income settings.

Footnotes

Acknowledgements

We appreciate all the doctors who participated in this study.

Declarations

ORCID iDs

Akaninyene Otu

Emmanuel Effa

References

Singer

Deutschman

Seymour

, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315: 801–810.

Otu

Elston

Nsutebu

Sepsis in Africa: practical steps to stem the tide. Pan Afr Med J 2015; 21: 323.

Yaya

Bishwajit

Okonofua

, et al. Under-five mortality patterns and associated maternal risk factors in sub-Saharan Africa: a multi-country analysis. PLoS One 2018; 13: e0205977

Jacob

Banura

Baeten

, et al. Promoting resource-limited interventions for sepsis management in Uganda study group. The impact of early monitored management on survival in hospitalized adult Ugandan patients with severe sepsis: a prospective intervention study. Crit Care Med 2012; 40: 2050–2058.

Andrews

Semler

Muchemwa

, et al. Effect of an early resuscitation protocol on in-hospital mortality among adults with sepsis and hypotension: a randomized clinical trial. JAMA 2017; 318: 1233–1240.

Rudd

Johnson

Agesa

, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the global burden of disease study. Lancet 2020; 395: 200–211.

Otu

Nsutebu

Hirst

, et al. How to close the maternal and neonatal sepsis gap in sub-Saharan Africa. BMJ Glob Health 2020; 5: e002348.

Dellinger

Levy

Rhodes

, et al. Surviving sepsis campaign: international guidelines for the management of severe sepsis and septic shock: 2012. Crit Care Med 2013; 41: 580–637.

Daniels

Nutbeam

McNamara

, et al. The sepsis six and the severe sepsis resuscitation bundle: a prospective observational cohort study. Emerg Med J 2011; 28: 507–512.

10.

Kumar

Roberts

Wood

, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006; 34: 1589–1596.

11.

Sterling

Miller

Pryor

, et al. The impact of timing of antibiotics on outcomes in severe sepsis and septic shock: a systematic review and meta-analysis. Crit Care Med 2015; 43: 1907–1915.

12.

Brent

. Meta-analysis of time to antimicrobial therapy in sepsis: confounding as well as bias. Crit Care Med 2017; 45: e242–e243.

13.

Agodirin

Aremu

Rahman

, et al. Prevalence of themes linked to delayed presentation of breast cancer in Africa: a meta-analysis of patient-reported studies. JCO Glob Oncol 2020; 6: 731–742.

14.

Sogbanmu

Goon

Obi

, et al. Sociodemographic and clinical determinants of late presentation among patients newly diagnosed with HIV in the Eastern Cape, South Africa. Medicine (Baltimore) 2019; 98: e14664.

15.

Jinga

Mongwenyana

Moolla

, et al. Reasons for late presentation for antenatal care, healthcare providers’ perspective. BMC Health Serv Res 2019; 19: 1016.

16.

National Confidential Enquiry into Patient Outcome and Death. Just say sepsis! A review of the process of care received by patients with sepsis, http://www.ncepod.org.uk/2015report2/downloads/JustSaySepsis_FullReport.pdf (2015, accessed 9 April 2021).

17.

England NHS. Improving outcomes for patients with sepsis, https://www.england.nhs.uk/wp-content/uploads/2015/08/Sepsis-Action-Plan-23.12.15-v1.pdf (2015, accessed 9 April 2021).

18.

Okoroiwu

Uchendu

Essien

RA.

Causes of morbidity and mortality among patients admitted in a tertiary hospital in southern Nigeria: a 6-year evaluation. PLoS One 2020; 15: e0237313.

19.

Ekanem

Etuk

Ekabua

, et al. Clinical presentation and complications in patients with unsafe abortions in University of Calabar Teaching Hospital, Calabar, Nigeria. Niger J Med 2009; 18: 370–374.

20.

Agan

Monjok

Akpan

, et al. Trend and causes of maternal mortality in a Nigerian tertiary hospital: a 5-year retrospective study (2010–2014) at the University of Calabar Teaching Hospital, Calabar, Nigeria. Open Access Maced J Med Sci 2018; 6: 1153–1158.

21.

Michie

van Stralen

West

The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011; 6: 42.

22.

Otu

Ebenso

Okuzu

, et al. Using a mHealth tutorial application to change knowledge and attitude of frontline health workers to Ebola virus disease in Nigeria: a before-and-after study. Hum Resour Health 2016; 14: 5.

23.

Otu

Okuzu

Effa

, et al. Training health workers at scale in Nigeria to fight COVID-19 using the InStrat COVID-19 tutorial app: an e-health interventional study. Ther Adv Infect Dis 2021; 8: 20499361211040704.

24.

Raosoft Sample size Calculator, http://www.raosoft.com/samplesize.html (2004, accessed 20 May 2021).

25.

Nucera

Esposito

Tagliani

, et al. Physicians’ and nurses’ knowledge and attitudes in management of sepsis: an Italian study. J Health Soc Sci 2018; 3: 13–26.

26.

Muke

Tugnawat

Joshi

, et al. Digital training for non-specialist health workers to deliver a brief psychological treatment for depression in primary care in India: findings from a randomized pilot study. Int J Environ Res Public Health 2020; 17: 6368.

27.

Statista. Smartphone users in Nigeria 2014–2025, https://www.statista.com/statistics/467187/forecast-of-smartphone-users-in-nigeria/ (2023, accessed 2 February 2023).

28.

Otu

Ukpeh

Okuzu

, et al. Leveraging mobile health applications to improve sexual and reproductive health services in Nigeria: implications for practice and policy. Reprod Health 2021; 18: 21.

29.

Wong

eLearning as good as traditional training for health professionals. Imperial News, Imperial College, London, 2015.

30.

Rohwer

Motaze

Rehfuess

, et al. E-learning of evidence-based health care (EBHC) to increase EBHC competencies in healthcare professionals: a systematic review. Campbell Syst Rev 2017; 13: 1–147.

31.

Ngenzi

Scott

Mars

Information and communication technology to enhance continuing professional development (CPD) and continuing medical education (CME) for Rwanda: a scoping review of reviews. BMC Med Educ 2021; 21: 245.

32.

Bolan

Sthreshley

Ngoy

, et al. mLearning in the Democratic Republic of the Congo: a mixed methods feasibility and pilot cluster randomized trial using the safe delivery app. Glob Health Sci Pract 2018; 6: 693–710.

33.

O’Donovan

Kabali

Taylor

, et al. The use of low-cost android tablets to train community health workers in Mukono, Uganda, in the recognition, treatment, and prevention of pneumonia in children under five: a pilot randomised controlled trial. Hum Resour Health 2018; 16: 49.

34.

Lund

Boas

Bedesa

, et al. Association between the safe delivery app and quality of care and perinatal survival in Ethiopia: a randomized clinical trial. JAMA Pediatr 2016; 170: 765–771.

35.

Prithishkumar

Michael

SA.

Understanding your student: using the VARK model. J Postgrad Med 2014; 60: 183–186.

36.

Dele-Olowu

Ogunrinu

Chukwu

, et al. Leveraging e-learning to train health workers during the COVID-19 pandemic in Nigeria, https://www.clintonhealthaccess.org/leveraging-e-learning-to-train-health-workers-during-the-covid-19-pandemic-in-nigeria/2020 (2020, accessed 14 January 2022).