Abstract
There is a critical shortage of trained human immunodeficiency virus (HIV) providers in resource-limited settings. To strengthen preservice HIV training for postgraduate health care providers, University of Maryland’s Institute of Human Virology and Center for Clinical Care and Research Nigeria collaborated with University of Nigeria to plan and implement a comprehensive 4-week course in HIV medicine. The first course was piloted with 30 postgraduate doctors. Mean objective structured clinical examinations (OSCE), pretest, and posttest scores score were 51%, 53%, and 75%, respectively; follow-up examination at 6 months showed mean score of 74%. In multivariate regression analysis, pretest score was positively associated with posttest score (22.03, P < .001) and OSCE (0.29, P = .04), age negatively associated with pretest score (−0.94, P = .001), and female gender positively associated with OSCE score (6.15, P = .05). Six- and 18-month online surveys revealed trainees continued to apply knowledge and skills gained. North–south university collaborations to develop practicum-based preservice curricula offer a sustainable way to strengthen preservice evidence-based HIV medicine training with long-lasting retention of skills and knowledge.
Keywords
Introduction
The early success of the US President’s Emergency Plan for AIDS Relief (PEPFAR) in implementing HIV care and treatment programs in sub-Saharan Africa unmasked the reality that many countries lacked sufficient quality and quantity of health workers to manage the HIV epidemic. 1 Nearly 10% of the 30 million people living with HIV worldwide live in Nigeria, 2 constituting a massive health, economic, and security crisis. The shortage of trained, well-prepared health workers is a major bottleneck in implementing evidence-based interventions to improve health outcomes. 3 The Nigerian health workforce (0.3/1000 population) falls far below the threshold of 2.5 health care providers per 1000 people required for effective service provision. 4 Nigeria’s Human Resource Strategic Plan identifies an absence of systemic in-service and preservice training as a significant contributor to a weak health care system and compromised patient care. 4
What Do We Already Know About This Topic?
Little is known or documented about the effectiveness of President’s Emergency Plan for AIDS Relief-funded short courses in human immunodeficiency virus (HIV) medicine in Nigeria that link in-service training (IST) with preservice education in HIV medicine as suggested by the comprehensive assessment conducted by United States Aid for International Development (USAID) in Nigeria in 2013.
How Does Your Research Contribute to the Field?
Our research contributes to the field by demonstrating short- and long-term evaluation measures of this HIV educational activity of the health care providers who completed the course and by implication on the quality of HIV health care they provide in their areas of clinical practice.
What Are Your Research’s Implications toward Theory, Practice, or Policy?
Our research implications toward theory practice and policy suggest that more investments in building institutional structures and collaborations toward increasing the quantity and quality of health care providers involved with practicing HIV medicine in resource-constrained settings are needed at the preservice level as this will ultimately contribute toward ending the HIV/acquired immuno deficiency syndrome epidemic.
Preservice education (PSE) is critical to the successful scale-up of HIV care and treatment services. 3 However, HIV training of health professionals at the preservice or in-service level is guided by very few studies in resource-limited settings (RLS). 1,5
Recent calls for reform in global health professional education have stressed the need for approaches that strengthen clinical reasoning skills. 6 While the development of these skills is critical to enhance health systems, little research has been carried out on the effectiveness of applying these strategies in the context of severe human resource shortages and complex disease presentations. 7 Training and experience of HIV/AIDS providers may impact not only individual patient outcomes but increasingly health care costs. 8 While in-service training (IST) provides an immediate response to an acute need for rapid HIV prevention, care, and treatment scale-up, training needs for ongoing HIV care and treatment have evolved over time. 9 The increased requirements for more robust training place additional pressures on IST delivery and require adaptation of training initiatives like strengthening links between preservice and in-service education. 10
In 2013, a comprehensive assessment was conducted by USAID of 39 PEPFAR-funded IST programs for all cadres of health workers in Nigeria. 10 This study noted that few courses are recognized by a training institution or provide continuing professional development credits recognized by a professional health council. 10 The study recommended that strengthening links between IST and PSE has the potential to promote sustainability and consistency in learning approaches and content between PSE and IST. Association with a training institution or health professional council encourages participation, may increase sustainability, raises the visibility of the training, and adds credibility. 10 The preservice training initiative described in this article was associated with a local tertiary institution University of Nigeria Nsukka (UNN), charged a nominal fee, provided Continuous Professional Development units from Medical and Dental Council of Nigeria, engaged faculty from UNN as tutors for the course, and used a curriculum developed from the national curricula for IST.
The University of Maryland’s Institute of Human Virology (UMB/IHV) and the Center for Clinical Care and Research Nigeria (CCCRN) implemented 2 training grants to address the Human Resources for Health (HRH) challenges in Nigeria. Here, we describe our innovative approach to strengthening preservice HIV training programs for postgraduate doctors in training. The strategy involved partnering with a federal university to develop a comprehensive HIV training course targeting doctors in training and utilizing both interactive teaching methods and clinical practicums to reinforce concepts learned in the classroom. The goal of the course was to strengthen HRH in Nigeria by enhancing the number and capacity of postgraduate doctors to provide evidence-based comprehensive HIV care, treatment, and prevention services.
The primary objective of this study was to evaluate the effectiveness of this 4-week preservice HIV management course for Nigerian postgraduate doctors on clinical skills and activities.
Methods
Course Design and Implementation
Center for Clinical Care and Research Nigeria partnered with the University of Nigeria Teaching Hospital due to its location in Enugu where its administrative offices are located, the presence of a PEPFAR-supported Comprehensive ART Clinic for clinical practicum sessions, and an MPH program at the Community Medicine department. A series of advocacy, stakeholder, and consensus-building meetings with University leadership and faculty were held to agree on the design, implementation, and evaluation of the course. We held a curriculum development workshop to identify evidence-directed core competencies and standards for HIV service delivery required by doctors in postgraduate training to practice in the community upon graduation. We developed a comprehensive framework for the course that included 9 modules with learning objectives and methods, session content outline, assessment methods, session requirements, and key supplemental resources (Online Appendix 1). We also developed a clinical practicum experience at the PEPFAR-supported comprehensive ART clinic. The UMB/IHV/CCCRN and UNN formed a joint course planning committee to facilitate planning and ownership of the course by the University.
A baseline needs assessment informed the design of an integrated HIV update/training-of-trainer workshop for 30 UNN faculty members drawn from departments of Medicine, Community Medicine, Pediatrics, Obstetrics/Gynecology to strengthen their facilitation skills based on adult learning principles using the microteaching model. 11,12 Microteaching is a faculty development technique designed to improve and develop teaching skills in a “scaled-down” or “micro” environment.
The pilot course was implemented in 2013 with 30 trainees consisting of postgraduate doctors in MPH programs and clinical residency training per the grant requirements. The course was taught jointly by faculty from UNN (21 of the 30 trained during the training-of-trainers), UMB/IHV, and CCCRN trainers using various instructional methods including interactive lectures, role-playing, case discussions, quizzes, recap sessions, and reading assignments. Recap sessions were held every morning and a surprise activity with prizes were held every week (debates, mini-dramas) and facilitate retention. Journal club sessions during the last week of the course used relevant and topical peer-reviewed journals to teach students how to critically appraise medical literature and appropriately apply to patient care. Clinical Practicum sessions took place at the PEPFAR-supported ART clinic, a 5-minute walk from the training venue. Students were grouped into 4 and had the opportunity to rotate through 8 units in the ART clinic (Adult and Pediatric ART, HIV-Exposed Infant clinic, PMTCT, M/E, Laboratory, Pharmacy, HIV Testing and Counseling clinic) twice over the course of 3 weeks for 4 hours per rotation. Students were given logbooks with learning objectives developed by UMB/IHV/CCCRN as a learning assessment tool. UNN faculty received a small stipend. The course trainees received CME credit and a certificate of successful completion signed by the university.
Course Evaluation Strategies
The course evaluation strategy used the Kirkpatrick evaluation model. 13 Level 1 (reaction) evaluations were conducted weekly during the course and at the end of the course to assess trainees’ reaction to the course logistics, content, teaching methods, teaching aids, facilitators, and clinical practicum. Level 2 evaluations were performed to assess knowledge and skills learned during the course using pre and posttests, objective structured clinical examinations (OSCE), and a clinical practicum logbook. Objective structured clinical examinations stations were provided, manned, and scored by UNN faculty. Course participants were assigned a logbook for rotation through each unit in the ART clinic. Logbooks identified key skills to be learned and were signed off by unit heads on successful completion of each rotation. Level 3 evaluations were conducted 6 and 18 months postcourse to assess whether transfer of learning actually took place and whether trainees were applying what they had learnt from the course at their workplaces.
Six months after the completion of the course, we conducted an online survey focused on relevance and usefulness of the course to personal clinical practice, strengths of the training, and overall rating. Course alumni completed telephone interviews where clinical scenarios were presented and 3 questions on clinical management assessed decision-making skills through interactive case discussions.
An 18-month postcourse electronic survey was sent to the same respondents to assess the impact of the course, that is, what percentage of course alumni continued to practice HIV medicine and the continued relevance and usefulness of the knowledge and skills learnt during the course to their clinical practice.
Data Collection
Data collection was program-based from participants in the course and included: Attendance (classroom and clinic). Pre/postcourse examination scores. Objective structured clinical examinations scores. Attendee daily and end-of-course evaluation forms. Follow-up of course alumni at 6 and 18 months using telephone interviews and SurveyMonkey electronic surveys (SurveyMonkey Inc, Palo Alto, California).
Data Analysis
Descriptive statistics were used to summarize trainee characteristics, evaluations, and course results. Univariate and multivariate linear regression was used to estimate the association between participant characteristics and course outcomes. A mixed multivariate model was used to account for repeated measurements (test scores) on subjects. All statistical analyses including data management were carried out using Stata 13.1(Stata Corporation, College Station, Texas 2009).
Results
Thirty health care workers participated in the original course, and 25(83%) were available for the 6-month postcourse follow-up. Participant characteristics are outlined in Table 1. The mean course attendance was 68% and the mean practicum attendance was 86%. The OSCE score was 51%, while the pretest mean score was 53% and the posttest was 75%. There was a 22% difference between the pretest and posttest scores (Table 1). In the subjective evaluation of the course, over half of the participants listed HIV case management and how to care for the patient properly as one of the important topics learned during the training; this was followed by Opportunistic Infections in HIV management at 31% and the new World Health Organization (WHO) guidelines at 21%. The greatest strengths of the training were the practicum sessions at 66%; these practicum sessions were also listed as the most effective presentation styles, followed by group exercises (44%) and case studies (38%). Sixty-two percent reported that the course was very useful in their daily work and none found it to be not useful. Seventy-five percent found that the course had increased their willingness to work in HIV medicine, and 86% revealed that it had increased their ability to work in HIV medicine clinical practice. In response to how their practice will change as a result of the training, 86% responded that they would be able to now manage and provide better management to HIV-positive patients (Table 2).
HIV Medicine Course Participants and Outcomes.
Abbreviation: OSCE, objective structured clinical examinations.
Participant’s Subjective Evaluation of the Course (Respondents = 29).
Linear Regression of Course Participant Characteristics on Outcome Evaluations.
Abbreviation: OSCE, objective structured clinical examinations.
In regression analysis, pretest score was strongly associated with posttest score; for every additional point scored on the pretest, the participant was likely to score 0.56 points higher on the posttest score (P < .001). Pretest was also moderately associated with OSCE; for every point on the pretest, a participant was likely to score 0.28 points higher on the OSCE (P = .04). In the multivariate mixed model, which accounted for repeated measurements on subjects, test scores increased by a mean of 22.03 points from pretest to posttest (P < .001). Age was associated with decreased test scores in the univariate analysis (−0.58, P = .02). Using multivariate mixed modeling to control for pretest score and repeated measures on the same subject, age maintained a statistically significant effect on posttest outcome; for each additional year of age, posttest scores decreased by −0.94 (P = .001). Female gender was associated scoring 6.52 points on the OSCE compared to males (P = .01), an effect that remained statistically significant in multivariate analysis (6.15, P = .05). There was no association between gender and posttest scores. Neither class nor clinical practicum attendance had any statistical association with posttest or OSCE scores.
For the 6-month postevaluation of the HIV short course, 24 (80%) of 30 course participants responded. Of all, 88% of the respondents are currently managing HIV-positive patients at their places of work. Over 70% reported that they have been able to apply knowledge and skills learnt during the course. Self-assessment of knowledge and skills related to HIV medicine increased from a mean score of 2.4 before the course to a mean score of 4.04 after the course. The HIV course was rated as outstanding by 27%, very good by 63%, and fair by 4%. The 6-month post course objective testing (N = 10) was 74% (Table 4).
Results of 6-Month Postcourse Evaluation (Respondents = 24).
Nineteen (79%) students said they would be willing to answer a specific HIV-related clinical management questionnaire administered over the phone by course faculty as part of postcourse evaluation; 7 students responded to the phone interviews and the average score was 73%. The lowest score was 46% and the highest was 93%.
The 18-month postsurvey had a 55% response rate. Over half (56%) of course participants reported continuing to practice HIV-related medicine, defined as seeing over 25% of HIV-infected patients or practicing HIV-related medicine. In their current practice, 88% still utilize the skills and knowledge from the HIV course, and 63% and 75% use the knowledge gained from the didactic portions and the supervised practicum portions of the course, respectively. There was 100% agreement that more supervised practicum experiences should be incorporated into HIV medical education (Table 5).
Results of 18-Month Postcourse Survey.
Discussion
We describe the first study in Nigeria to conduct a preservice comprehensive course in HIV medicine targeted at postgraduate doctors in training that integrated clinical practicums. This program was developed and conducted jointly by UNN, a tertiary academic institution and CCCRN and UMB/IHV, PEPFAR-funded implementing partners.
Our study has several important findings: first, we demonstrate that it is feasible to work collaboratively with a tertiary academic facility to institutionalize preservice HIV education in Nigeria. Second, including a clinical practicum added significant value to the course as demonstrated in the OSCE results, the online surveys, and telephone case discussion interviews. Finally, the inclusion of the postcourse Kirkpatrick level 3 evaluation methods provided more evidence of program success for the funders as well as an opportunity for course faculty to reinforce key clinical concepts related to HIV management and clinical decision-making.
Following course completion, participants were more likely to be confident in starting an antiretroviral (ARV) regimen either for treatment or preventing mother-to-child transmission, understanding ARV toxicities, encouraging patient adherence, diagnosing immune reconstitution syndrome, and monitoring patients on ARV medications. All participants (100%) strongly agreed/agreed that they would recommend this course to others, and most (96%) strongly agreed that they would take a course in this format again. From our evaluation data, 75% of the course alumni found that the course had increased their willingness to work in HIV medicine, and 86% stated that it had increased their ability to work in HIV medicine.
In the linear regression analysis (Table 3), we found that posttest scores increased significantly from pretest scores (mean increase of 22%) and were strongly associated with pretest scores. This is expected as participants with higher pretest scores likely have a larger baseline knowledge level that was reinforced through the course. Increasing age was significantly associated with lower posttest scores in both univariate and multivariate mixed models. The reason for this association is unclear; perhaps participants who were older were out of medical training for longer and were less exposed to HIV medicine during this training.
In the analysis of the OSCE score, we found associations between OSCE score and female gender and pretest scores in both univariate and multivariate analyses. For every point scored on the pretest, participants were likely 0.3 points higher on the OSCE, after adjusting for other covariates. This likely reflects fund of knowledge and ability to apply clinical knowledge to clinical situations. Female gender was associated with scoring 6.15 points higher on the OSCE in the multivariate model, suggesting that female participants demonstrated better clinical skills overall; it could also reflect a bias, if for example evaluators rated women higher on the OSCEs.
Evaluation strategies for most training programs typically end with level 1 and 2 Kirkpatrick evaluations, where reaction to the training and knowledge gained is measured during the training. 14 However posttraining level 3 evaluations to assess how much of the training transferred from delivery to workplace are rarely documented. 15 This additional level of evaluation provides more evidence of our program’s success, improved accountability to funders, and helped identify enablers or barriers to application.
Human immunodeficiency virus medicine is a rapidly evolving science with a short half-life of knowledge. 9 More mechanisms should be built into training programs and academic institutions in RLS to address training needs as they evolve. A practical solution to this dilemma is strengthening PSE. 9
Preservice education provides an organized and systematic approach to ensure all professionals within a given setting are inculcated with fundamental HIV knowledge and skills before entering the workforce. Preservice education in the long term is less costly than IST and avoids workplace absences. 9 Incorporating practical training with didactics as we did is an effective way to reinforce application of knowledge and skills learned in the classroom. 5 Interactive methods, such as hands-on practice sessions, case discussion, and role-play, are effective at changing physician practice and in some cases patient outcomes. 16 Didactic training methods predominate in much of sub-Saharan Africa aside from some recent clinical mentoring initiatives. 5 Other preservice HIV training initiatives have also been described, each employing unique approaches. 9
Our program had several strengths. We utilized a pragmatic approach to HIV curriculum development and evaluation, interactive teaching methods, and clinical practicums, all of which led to more reliable learning outcomes, higher learner satisfaction, and improved knowledge and skills. To ensure sustainability, majority of the course facilitators were UNN faculty. This study also had limitations. There was no comparator cohort of students, and this was the first cycle of the course. Our response rate to the 18-month survey was just over 50%, and study sample was small, limiting the generalizability of the results to all course alumni and other settings. Even though the addition of the clinical practicum to the training program was an innovative approach, mastery of the skills could not be completely ascertained, as the time allocated for the rotations was short. Outcomes could be further evaluated and measured by comparing multiple cohorts of similar participants as further cycles of the course are conducted. A systematic approach to evaluation methods and data collection is critical to improving the effectiveness of training programs in RLS. For countries with high burden of HIV disease like Nigeria, the time has come to integrate preservice HIV education initiatives into the training of health care professionals.
Conclusion
Learning through north–south University collaborations to develop practicum-based preservice HIV curricula offers a sustainable way to teach evidence-based medicine with long-lasting retention of skills and knowledge. This initiative demonstrated the capacity to strengthen specific HRH workforce in Nigeria to manage patients infected with HIV.
Such partnerships also require building relationships and establishment of trust between faculty members of each institution. Besides investments in infrastructure and equipment, empowered faculty will require ongoing technical assistance and capacity building in curriculum development and continuous medical education to keep up to date with a rapidly evolving science. The presence of UNN faculty mentors working on-site at university and ART clinics could greatly enhance the quality of teaching in both didactic and clinical settings. Further efforts and funding are needed to sustain and improve this innovative approach to strengthening HIV preservice medical education in RLS.
Supplemental Material
Appendix_1_UNN_Short_Course__Schedule_ - Effectiveness of a Comprehensive 4-Week Course in HIV Medicine for Postgraduate Doctors at University of Nigeria: A Preservice Education Initiative
Appendix_1_UNN_Short_Course__Schedule_ for Effectiveness of a Comprehensive 4-Week Course in HIV Medicine for Postgraduate Doctors at University of Nigeria: A Preservice Education Initiative by Anthea Nwandu, Cassidy Wayne Claassen, David James Riedel, Theresa Madubuko, Ayodotun Olutola, Eugene Onu, Chinwe Onyekonwu, Emmanuel Nwobi, and Chinwe Chukwuka in Journal of the International Association of Providers of AIDS Care (JIAPAC)
Footnotes
Authors’ Note
This course was developed as one of the activities designed to facilitate the achievement of the goals and objectives of a CDC grant (CDC-RFA-GH12-1229), which had the stated purpose to strengthen, expand, and develop preservice and IST programs for public health students and health care providers working in Nigeria on comprehensive HIV/AIDS prevention, care and support and treatment services provision at the facility level. We obtained permission and approval from the University of Nigeria senate to implement the course. We did not get IRB/ethical approval as it was not designed to be a research activity but rather an educational activity designed and conducted in an educational setting—the University of Nigeria. All data obtained from the participants were deidentifed. The manuscript describes the program as a quality improvement measure. Data from this study were presented in part at the 2016 Annual Consortium of Universities for Global Health (CUGH) Conference, April 9 to 11, San Francisco, California.
Acknowledgements
This publication was made possible by support from the PEPFAR through cooperative agreement CDC-RFA-GH12-1229 from the HHS/Centers for Disease Control and Prevention (CDC), Global AIDS Program. The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention. The authors wish to thank Kristen Stafford, PhD, for her insightful review of the manuscript. Finally, we wish to thank the course participants whose dedication to the treatment of HIV in Nigeria made this possible.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
