Abstract
Background and Aims:
Maintaining the quality and safety of immunization is as important as the efficacy of vaccines in vaccine-preventable diseases (VPD) programs. The aim of this study was to determine the problems associated with different stages of vaccine use and to assess the outcome of an educational intervention on safety and quality use of vaccines among health care providers.
Methods:
A pilot prospective interventional study was conducted over a period of 2 years at 271 sites in Mysuru, India. The study population was health care professionals (HCPs) involved in immunization and a sample of parents (one per site). A validated questionnaire was used as a study tool. An educational intervention on best immunization practice was conducted for the enrolled HCPs and the impact of the educational intervention was assessed using the study tool after 3 weeks.
Results:
The total number of the study population was 594 (323 HCPs and 271 parents). Of these, 41.49% were working at community health care facility and 33.13% were enrolled from primary care centers. There were statistically significant improvements in post interventional assessment of all stages of the immunization process including storage (p −0.001), transportation (p −0.001), administration (p 0.001), monitoring and reporting of adverse events following immunization (AEFIs) (p −0.001), knowledge of AEFIs (p 0.001), and HCP-parent communication (p 0.001). AEFI reporting improved by 30% in the post education phase.
Conclusion:
Continuous education and motivation can result in positive behavioral changes on best immunization practices amongst HCPs involved in immunization, which may help to improve and maintain the safety and quality use of vaccines in immunization centers irrespective of the type of facility.
Introduction
Immunization is one of the major cost-effective healthcare interventions in promoting health and protecting the individual and the community from vaccine-preventable diseases (VPDs).1,2 Globally, immunization prevents 2–3 million deaths every year, an additional 2 million deaths can be prevented annually with appropriate use of currently available vaccines.3,4
Vaccines used in National Immunization Programs are considered safe and effective when used correctly. 4 Immunization quality and safety surveillance has become as important as the efficacy of vaccines in VPD Programs.2,4 Quality use of vaccines can be described as the utilization of properly maintained vaccines for immunizing at appropriate ages for maximal benefit with minimal risk. Safe and quality use of vaccines at the immunization center includes activities such as safe transport of vaccines; maintaining proper storage conditions for vaccines; safe administration practices; educating parents about the vaccination coverage; possible adverse events following immunization (AEFIs) and the importance and process of reporting AEFIs; adequate safety monitoring by the health care team in the hospital and communication of reported AEFIs to the National Regulatory Authority (NRA); maintenance of immunization records in electronic for; and the timely, routine reporting of immunizations to local or national registries as appropriate.5,6
Many studies have highlighted operational issues in the safe and quality use of vaccines in the Expanded Program on Immunization (EPI).5–16 Researchers have suggested that the main strategy to improve safe and quality use of vaccines in the EPI was educational interventions to the health care professionals (HCPs) involved in the immunization process.15,16 The government typically conducts routine educational sessions whenever new vaccines are introduced, before any mass immunization campaigns or during periods of public concern about vaccination programs. However, there is a clear lack of continuing education for HCPs practicing in the community, even though they vaccinate the majority of the population. Community practitioners have limited access to independent research-based information on new vaccines and safety related information and medical representatives of pharma companies may be the main source of information in making their prescription decisions. 17 Routine immunization programs in India have almost 157 million beneficiaries and conduct approximately 10 million immunization sessions annually, with the support of a network of nearly 27,000 cold chain points to ensure safe delivery of vaccines. 18 However, reports suggest that at least 25% of vaccines are wasted even before reaching the immunization centers due to the lack of a good quality supply chain and logistics management system. 19
This study was conducted to determine the problems associated with different stages of vaccine use among HCPs involved in immunization and to assess the outcome of an educational intervention for HCPs on the safe and quality use of vaccines.
Methods
This was a prospective interventional study, conducted over a period of 2 years (July 2016–June 2018). Study sites were immunization centers and attached community pharmacies within the city of Mysuru, India. The study population was HCPs such as physicians, nurses, and pharmacists involved in the immunization process practicing at different study sites, and parents of children vaccinated at these study sites. HCPs involved in immunization who were willing to allow the researcher to observe the immunization process and interview any of the beneficiaries of vaccination from his/her clinic were included in the study. HCPs or beneficiaries of vaccination who were unwilling to be a part of the study were excluded.
Development of study tool
A questionnaire was developed and validated for the purpose of assessing the quality and safe use of vaccines. 20 The questionnaire was prepared based on published similar studies and best practice guidelines for immunization. Experts in the field of immunization were requested to validate the questionnaire. The questionnaire was finalized after discussions and deliberations amongst the researchers and the feedback received from the experts. An English language expert language edited the questionnaire. The questionnaire was divided into three sections.
Section I. Questions to assess the knowledge of HCPs on various steps involved in the immunization process and AEFI reporting practice. Section I had 22 questions assessing the knowledge of vaccine transportation (1), storage (2), administration (3), monitoring of AEFI (2), AEFI reporting (3), types of AEFIs and their causality assessment (7), and one question each assessing the practice of HCPs on vaccine transportation and reporting of AEFIs. This section also had a question to understand the barriers in reporting of AEFIs and a question to ascertain the HCPs preferred method of reporting AEFIs in future.
Section II: Questions in this section assessed storage of vaccines (five questions) and immunization practices (four questions) of HCPs at the immunization center.
Section III: This section comprised eight questions for the parents of the vaccinated children to understand the communication between HCPs and parents (four questions), to assess AEFI monitoring practice of HCPs (two questions), willingness of the parents to report AEFIs to the immunization center (one question) and their preferred method of reporting AEFIs (one question). The questions for parents were translated to the local language (Kannada) with the help of linguistic experts.
A total of 30 experts from related fields such as pediatrics, pharmacy, nursing, and social work contributed to validation of the questionnaire. The experts independently rated each question and answer for clarity, relevance, ambiguity, and simplicity. The scale content validity index (S-CVI) was calculated using the questionnaire rating by the experts.21–23
Study process
Eligible HCPs were enrolled into the study after explaining the importance of the study, study procedure, expectation from them, and informed consent. HCPs were asked to answer the first section of the questionnaire at baseline and again 3 weeks after the educational intervention. At the same time points the immunization clinic and the vaccine storage area were observed to assess the practice of HCPs using the second section of the questionnaire. Parents waiting at the immunization clinic (one parent per selected health care facility) for vaccination of their children were enrolled after taking written informed consent, and the third section of the questionnaire was used to assess the communication between HCPs and parents. The responses collected from HCPs and parents and observations made from the immunization clinic were assessed to identify problems associated with the immunization process. One point was awarded for each correct answer scored in the questionnaire.
Development of educational intervention
An extensive literature search was done for preparing the educational intervention. Keywords used for the literature search were: “quality and safe use of vaccine”, “vaccine transport”, “vaccine storage”, “vaccine administration”, “adverse events following immunization (AEFIs)”, “monitoring of AEFIs”, “reporting of AEFIs”, “problems associated with immunization process”, “causality assessment of AEFIs”, and “errors made during vaccination” using search engines such as Google scholar, Science Direct, PubMed/Medline, and Clinical Key. Observations made by the researcher during visits to immunization centers in the city and the knowledge gained by interacting with experts was also utilized to prepare the educational module. The module had information on the immunization process, including transport, storage and administration of vaccines, AEFIs, monitoring of AEFIs, reporting of AEFIs, problems associated with immunization process, causality assessment of AEFIs and information practically applicable to HCPs in their routine practice. Three experts in the related field were asked to validate the module. The educational material was finalized after discussions and deliberations among the researchers taking the feedback received from the experts into consideration.
Conduct of educational sessions on safe and quality use of vaccines
Educational programs (three sessions of 3 h duration each) were conducted for the enrolled HCPs from various study sites. The researcher also conducted one-to-one interactive educational sessions for enrolled HCPs who were unable to attend the educational session (separately for pediatricians, nurses, and pharmacists). The study participants’ knowledge of safe and quality use of vaccines was reassessed to assess the outcome of the implemented strategies during the next visit, which was 21 days ± 3 days following the educational intervention.
Statistical analysis
The collected data was entered into MS Excel followed by statistical analysis using SPSS software version 22. The reliability/internal consistency of each question in the questionnaire was estimated using Cronbach’s Alpha (102, 129). Comparisons of pre and post scores of the HCPs were done using paired sample t-test. p value of < 0.05 was considered statistically significant with 95% confidence interval.
Results
The overall content validity score and the item-content validity index (I-CVI) of the prepared questionnaire was 3.85 and 96.25%, respectively. The internal consistency of the questionnaire measured using Cronbach’s alpha was 0.89.
A total of 323 HCPs and 271 parents were recruited, from 229 community health care/hospital facilities and 42 primary care centers. Of these, 41.49% of HCPs were working at a community health care facility and 33.13% were enrolled from primary care centers. The HCP study population included pediatricians [121 (37.46%)], pharmacists [125 (38.72%)] and nurses [77 (23.82%)].
There were knowledge deficits amongst HCPs on various stages of the immunization process as given in the following.
Vaccine transportation
Only 32.20% (n = 104) HCPs answered that they receive the supply of vaccine with a temperature indicator in the shipment pack. The remaining HCPs had no experience with vaccine transportation, as it was not their responsibility at the practice sites. However, 92.85% (n = 323) were aware that exposure to heat shortens the half-life of the vaccine. All of those who gave an incorrect answer on vaccine transportation (n = 23) were nurses by profession. It was also observed that vaccine transportation was not the responsibility of nurses as they were enrolled from primary health centers (n = 12) and hospitals without EPI supply (n = 11).
Storage of vaccines
The vaccine storage temperature and the impact on potency of frozen vaccines that are not supposed to be frozen was not known by 3.09% (six nurses and four pharmacists) of the HCPs included in the study. Some HCPs (n = 38, 11.76%) did not have a vaccine storage facility at their work place. From the total of 285 HCPs with a vaccine storage facility, back up storage unit/back up power supply availability in case of power failure or other unforeseen events was not available in 37.89% (n = 108) of the facilities. Though all the enrolled HCPs knew that temperature monitoring is important while storing the vaccines, a calibrated thermometer with the vaccine in the refrigerator was available only in 49.84% (n = 161) of the HCPs practice facilities and all were PHCs and hospitals with EPI vaccine supply. In community facilities of 102 HCPs practices, vaccines were stored in the door of the refrigerator though the vaccine handling guidelines clearly mentions that vaccines should not be stored in this location. Expired vaccines were stored with non-expired vaccines in 10 PHCs where 28 of the enrolled HCPs practiced. Medicines were also stored along with the vaccines in 46 community clinics and all were using a domestic refrigerator to store the vaccine supply without power back up.
Administration of vaccine
Correct reconstitution method and the time period to store the reconstituted vaccines was not known by 38.69% (n = 125) of the HCPs. The HCPs with inadequate knowledge on vaccine reconstitution were mainly pharmacists (n = 98) who were responsible for vaccine storage only. A total of 11 nurses and 16 pediatricians who administered vaccine did not know the maximum time for which a reconstituted vaccine could be kept.
Monitoring of AEFI
Most of the HCPs knew the importance of first aid facilities to manage immediate adverse events following immunization. Pharmacists were not actively involved in vaccination as their role majorly involved the transportation and storage of vaccines. Among the HCPs who were actively involved in vaccination (n = 143), almost half (49.65%; n = 71) answered that they do not observe babies after vaccination for the occurrence of any immediate AEFIs. Most of the HCPs (96.95%) answered that it is important to have an emergency first aid kit to manage immediate AEFIs at the immunization clinic, but this kit was available in 133 facilities (where only 41.17% of HCPs practiced). From the 271 parents enrolled to study the communication of HCPs with parents from each facility, only 43 answered that HCPs instructed them to wait in their facility for a minimum of 30 min to observe for any kind of immediate adverse events. It was also observed that these 43 parents were enrolled for the study from hospitals.
Reporting of AEFIs
Almost 91.02% (n = 294) of the HCPs knew that AEFI monitoring is an important responsibility in the immunization process, but 73.37% (n = 237) had not reported AEFIs to a regulatory agency. Nearly half of the HCPs (n = 147; 45.65%) in this study answered that all the events following vaccination need to be reported, majorly from PHCs (n = 23), hospitals with (n = 58) or without EPI (n = 20) supply, and from community clinics (n = 49). It was also observed that, in many of the PHCs, the monthly reporting of AEFIs of mild and moderate severity was nil in many months. Lack of time or interest in reporting, non-availability of the reporting form and inadequate knowledge about the reporting process were the major reasons (38.63%) for not reporting AEFIs by the HCPs. Only 48.60% (n = 157, 115 pediatricians, and 42 nurses) were aware that the adjuvants also could cause AEFIs. None of the pharmacists were aware that the adjuvants could cause adverse events. Causality assessment algorithm availability was not known by 70.34% of the enrolled HCPs and only 17.64% (47pediatrician, 4 nurses and 6 pharmacists) were aware of the existence of spontaneous reporting of AEFIs. A total of 48.56% of HCPs were unable to answer correctly the questions on the categorization of AEFIs.
HCP communication with parents
Out of 271 parents enrolled, 38.37% (n = 104) answered that their HCPs explained the importance of vaccinating a child and about different government and non-government supplied vaccines (26.95%). Only 24.78% of the HCPs intimated the date of the next vaccination to the parents; 71.53% of HCPs did not inform parents about possible AEFIs or to come back to their facility upon the occurrence of any AEFIs. The scores of HCPs on various activities involved in safe and quality use of vaccines are presented in Table 1.
Baseline score of activities involved in safe and quality use of vaccines.
AEFI, adverse events following immunization.
A total of 239 HCPs participated in the educational session/one-to-one interactions with a response rate of 73.99%. The knowledge on best immunization practice was improved by 32.62% during post education assessment [pre education total score was 24.2 (60.5%) and the post education total score was 37.25 (93.12)]. Assessment of pre and post education knowledge and practice scores of HCPs and the statistical analysis is presented in Table 2.
Pre- and post-education knowledge and practice scores of HCPs.
AEFI, adverse events following immunization; HCP, health care provider; SD, standard deviation.
Discussion
The negative impact of using inappropriately handled vaccines include an outbreak of VPDs leading to loss of public confidence in the programme, unnecessary healthcare burden, and need for re-vaccination. Depending on the type of health care facility, the roles and responsibilities of HCPs in the immunization process vary. For example, in a hospital with EPI vaccine supply, the responsibility of the pediatrician is limited to effective communication with the parents, whereas the nurses have the responsibilities of transportation, storage of vaccine at the immunization center, reconstitution, administration of the vaccine and monitoring for any immediate AEFIs (at the immunization site for 30 min). The pharmacists have the responsibility for storage of vaccines, which are not covered under the EPI. In a hospital without EPI vaccine supply, all vaccines are stored in the pharmacy under the supervision of the pharmacist. Pediatricians have the responsibility of communicating with the parent and the nurse is usually responsible for vaccine reconstitution, administration, and monitoring of AEFIs at the immunization center for 30 min. In a community/private health care facility, pediatricians assume the responsibility of storage, reconstitution, administration of the vaccine, monitoring of AEFIs at the immunization center for 30 min and communication with the parents. In primary health centers, the pediatrician/the medical officer in-charge is responsible for effective communication with parents, the pharmacist has the responsibility of transportation and storage, and nurses have the responsibility of reconstitution, administration of the vaccine and monitoring of AEFIs at the immunization center for 30 min.
A percentage higher than 80% for I-CVI and Scale Content Validity Index (S-CVI) is considered as minimum acceptable rate for a newly prepared tool.24,25 No questions were discarded as all the questions scored CVI of more than 0.75 but a few questions were modified based on the opinion from the experts. 22
Vaccine storage and transportation
Vaccine exposure to temperatures outside the recommended ranges may decrease the potency of vaccines, leading to inadequate immune responses, loss of vaccine investments, and potentially placing children at increased risk of VPDs.26,27 Similar issues in vaccine storage and transportation have also been identified in studies conducted in different countries, irrespective of their economic status.28,29 In a recent review 37.1% of the lower income countries and 33% of the higher income countries had exposed vaccines to below recommended temperature ranges during storage and 38% of the higher income countries and 19.3% of the lower income countries had temperature problems during transportation. 25
Researchers from the Program for Appropriate Technology in Health (PATH) published another systematic review stating that 75–100% of the vaccine shipments are exposed to freezing temperatures. 30 In a descriptive analysis of the Vaccine Adverse Event Reporting System (VAERS) in the United States (US), 582 cluster incidents of incorrect storage of vaccines were identified, which affected 1715 patients for a period of 13 years (2000–2013). The study also stated that 44% of errors reported to VAERS were exposure of vaccines to inappropriate temperatures. 31 Similar to the current study, a cross-sectional study conducted at Cebu, Philippines also identified lack of availability of backup power supply/storage unit in the immunization center (90.9%) and thermometers in refrigerators (68.2%). 32 Similar findings were also observed from a study conducted by researchers from Italy. 15
In this study, we observed the presence of expired vaccines along with non-expired vaccines in the same refrigerators in 16.40% of the immunization centers. Possible reasons may include poor knowledge of guideline recommendations, unclear lines of responsibility amongst HCPs, inadequate inspection by higher authorities, inappropriate monitoring of the storage facility by the in-charge officer due to other responsibilities, involvement of many vaccinators and movement of vaccines to communities for vaccination. All these storage related problems can affect the efficacy and safety of the vaccines as described in the previously published literature. 33 In 19.56% of immunization centers, vaccines and medicines were stored together. This was observed more commonly in private sector clinics. Reasons for this may include space constraints, the expense of having two refrigerators, and a belief that there were no chances of a mix-up of any sort resulting in administration errors. Many studies identified the lack of knowledge among HCPs as the contributing factor for issues in vaccine transportation and storage. Studies suggest that it is important to address these issues by proper education and training. 26
Administration of vaccines
Knowledge on administration of vaccines was higher among pediatricians and nurses than pharmacists. Knowledge of vaccine administration was high among the nurses in our study, as seen in previous studies. 33 A nationwide population-based cluster survey from India by the IPEN study group, found that 62.9% of the injections were unsafe and that the proportion of unsafe injections was highest in the immunization clinics (74.0%). In that study, 50.1% of the immunization clinics had non-satisfactory disposal of injection waste, while the current study observed 54.48% of the immunization facilities with non-satisfactory disposal practices. Errors in vaccine administration in VAERS accounted for 15% of the total errors reported. 31
Reporting of AEFIs
The AEFI standard operating procedure of the Government of India recommends that HCPs report all serious AEFIs within 24 h using a Case Reporting Format (CRF) to the PHCs or to the District Immunization officer (DIO), and minor events to the DIO through a routine monthly reporting form. The components of AEFI surveillance system include detection, reporting, investigation, causality assessment, and risk/benefit assessment. 32 The national quality assurance standards for AEFI surveillance system clearly states that private practitioners also need to report AEFIs to the Medical officer/DIO for further processing. 34 However the majority of the HCPs in our study were unaware of the AEFI reporting process, and 68.11% had not reported any AEFIs in their practice.
Monitoring of AEFIs
Informing the beneficiaries to wait at the immunization center for half an hour to observe any AEFIs is the first responsibility listed in the immunization handbook for immunization officers. 35 Despite this, the majority of the HCPs (83.28%) did not ask parents to wait at their facility for 30 min. Most of the HCPs, especially the pediatricians, are aware of this responsibility but they did not do so due to lack of sufficient waiting area.
Communication to parents
A relationship between the HCPs involved in immunization and the parent/beneficiary is essential in maintaining confidence in vaccine efficacy and safety. The voice of healthy individuals receiving vaccines and parents needs to be considered and addressed to avoid vaccine hesitancy among the public. As the regulatory agency encourages clinical and patient reported outcome research, it is important to communicate effectively with the parent about the importance of vaccination, next vaccination due, possible AEFIs, and also encourage them to come back and report in case of occurrence of any AEFIs. 36 The World Health Organization (WHO) also gives importance to the maintenance of confidence of the public by properly responding to parent/community concerns, while increasing public and professional awareness about vaccine risks. 37 From the survey conducted in the patient population, it is very clear that there is a gap in communication between the vaccinators and the parents. The HCPs were informing parents about the next date of vaccination without additional advice such as the name or the importance of the vaccine, information about AEFIs, or the procedure to be followed in case of any AEFI occurrence.
Outcome of educational intervention on safe and quality use of vaccines
Studies have suggested the need for educational sessions to improve the knowledge and the practice of HCPs on various aspects of the immunization process,26,38,39 and our study demonstrated that educational sessions improve the knowledge level of HCPs on all aspects of safe and quality use of vaccines.
In a study conducted among HCPs practicing at primary care centers in Germany, an 80% improvement in the maintenance of vaccine storage facilities was observed following structured education. 40 Another study from Korea among private hospitals concluded that the practice of proper storage of vaccines can be improved by relevant education; however, repetitive education and changes in policy making by regulators are also required to store vaccines safely at the immunization centers. 41 The Bill Gates foundation also stated that major challenges for vaccination, such as unreliable transportation and storage facilities, could be improved by training and support from regulatory agencies. 42 Our study found an improvement in knowledge among HCPs on the storage and transportation of vaccines after education; however, to translate the improved knowledge into practice, continuous motivation and training, and a mechanism to monitor the practice of HCPs in private and public health care facilities, might be required.
Studies done earlier also identified a positive impact on vaccine administration practices following education sessions. In a study of nursing staff, an improvement in the understanding of best vaccination practices was observed following education sessions. 43 Many other studies have also identified the need for high quality education to improve AEFI reporting among HCPs.43,44 The current study has shown a significant improvement in the knowledge of AEFIs and their reporting in the study population and could help to initiate and strengthen the reporting culture among the HCPs practicing at both private and public health care facilities. There would be a need for ongoing motivational communication to HCPs for AEFI reporting to maintain this at an optimum level. Proposed possible strategies to improve the AE reporting culture among the HCPs were the organization of workshops, and continuous medical education sessions. 45
This study demonstrates that educational interventions to HCPs can significantly improve the knowledge on safe and quality use of vaccines and, thereby, their immunization practice. However, multicentric larger studies are warranted to investigate the efficacy of educational interventions in improving the best immunization practice of HCPs.
Conclusion
To promote safe and quality use of vaccines, starting from cold chain management of vaccine to AEFI reporting, appropriate ongoing educational interventions are necessary. HCP communication helps parents to gain confidence in vaccination efficacy and safety, which supports a successful local and national immunization program. Inadequate knowledge about the AEFI reporting process, and lack of time and interest were the identified challenges contributing to a low reporting of AEFIs. Targeted educational sessions can improve immunization practices of HCPs, which improve parents’ confidence in immunization and hence the vaccination coverage of the community.
Footnotes
Acknowledgements
The authors would like to thank all study participants for their active participation in the study. Also, the authors acknowledge the support of administrators of JSS College of Pharmacy, JSS Hospital and JSS AHER, Mysuru for their support in the completion of the study.
Conflict of interest statement
The authors declare that there is no conflict of interest.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Ethical approval
Ethical approval of the study was obtained from Institutional Ethical Committee of JSS College of Pharmacy, Mysuru (JSSCPM/IHEC/2015/004).
