Abstract
The objective of this study was to determine the effectiveness of an educational program for the University’s mandatory vaccination program. The study examined the relationship between level of education, gender, and/or job classification and information retention and perceptions of the mandatory vaccination program. The hypothesis was that understanding and information retention level was increased in personnel with increased levels of basic and higher education and job classification complexity. The outcomes identified will be used to revise and improve the vaccine education program and materials, as well as develop recommendations to be used as a model by other institutions that may require a similar mandatory vaccination program.
The American Association of Occupational Health Nurses (AAOHN) has defined occupational and environmental health nursing as a specialty that focuses on preventive health care and health promotion using primary, secondary, and tertiary prevention principles, counseling, health education, and training programs with adult learning approaches (AAOHN, 2012c). Combining this definition with AAOHN’s primary objective, for every occupational health and safety program, occupational health nurses assume the role of health promotion specialist (AAOHN, 2012b). A basic and necessary strategy used in health promotion is education, and a stated research priority of AAOHN is evaluating the “effectiveness of health promotion nursing intervention strategies” (AAOHN, 2011, p. 184). Therefore, occupational health nurses must be educators, evaluators, and researchers.
AAOHN has 11 standards of practice, “developed to help define and advance practice and provide a framework for evaluation” (AAOHN, 2012c, p. 97). The fifth standard states “interventions to include health teaching and health promotion” (AAOHN, 2012c, p. 99), and Standard 6 emphasizes “the evaluation and measurement of outcomes allowing for correction of deficits” (AAOHN, 2012c, p. 99). Standards 9 and 10 emphasize “research and the need for collaboration” (AAOHN, 2012c, p. 100). The research project discussed in this article focuses on knowledge retention after education about required vaccinations at a large university-based research center.
University Research Laboratory Hazards
At the University of Pittsburgh, a large teaching and research institution of higher education, the above-stated standards were tested recently. The University’s occupational health nurse assumed the AAOHN-designated expert role for the following competencies: (a) health promotion and disease prevention, (b) health and safety education, and (c) training and collaboration (AAOHN, 2012a). The University, after an extensive risk assessment and evaluation process, developed and implemented a mandatory vaccination program for researchers and their staff working with specific high risk, infectious agents including Japanese encephalitis virus, Neisseria meningitides; rabies virus; yellow fever virus; and Bacillus anthracis. The at-risk population included all employees who, in the course of their normal duties, handled cultures of or animals that had been infected with the above-mentioned high-risk infectious agents.
The University considered several elements in the risk–benefit analysis prior to implementation of its mandatory vaccination program. The safety of vaccines available for high-risk infectious agents in use at the University was the first element assessed. Because the Food and Drug Administration (FDA) requires extensive safety testing as part of its approval process, the decision was made to limit vaccines under consideration for the mandatory program to those with FDA approval.
The level of protection afforded by each FDA-approved vaccine under review was assessed for each infectious agent. The decision was made to include vaccines with data-supported efficacy for the specific agent studied in the program and to leave vaccinations, such as seasonal influenza, as a recommendation rather than a requirement because no data were available to demonstrate employee exposure to high-risk strains of influenza virus in use at the University (e.g., highly pathogenic Avian influenza strains; Bodewes et al., 2011; Bodewes et al., 2012; Bodewes, Kreijtz, & Rimmelzwaan, 2009; Park et al., 2014).
University Research Laboratory Occupational Health Program
According to the AAOHN Core Curriculum (Moore & Moore, 2013), health promotion is a necessary component of workplace illness and injury prevention programs; a necessary component of health promotion is health education. Using education as the primary prevention strategy, the University’s occupational health nurse and the biosafety officer (BSO) examined the identified knowledge gap between employees’ understanding of required vaccinations prior to implementing the new mandatory program and the understanding required to ensure a knowledgeable workforce and a safe work environment post-implementation via face-to-face lunchtime discussions and interviews with employees in appropriate job categories. Identifying the knowledge gap provided the foundation for behavioral objectives (Table 1) and development and implementation of a program to not only educate employees about the vaccinations but also alleviate apprehension about the new mandatory vaccination program. The occupational health nurse and the BSO used the Health Belief Model to develop not only the educational materials but the entire program considering employee-perceived susceptibility, severity, benefits, and barriers to the vaccination program.
Behavioral Objectives of University’s Mandatory Vaccination Education Program
The University Biohazards Committee, consisting of faculty experts in the fields of vaccine research and development and public, environmental, and occupational health sciences, performed the initial risk assessment and provided guidance regarding implementation of the mandatory vaccination program (The University of Pittsburgh Department of Environmental Health and Safety, 2011-2012). After an assessment of the employee population potentially affected by the new mandatory vaccination program, an educational program was developed. The program was designed to specifically address the wide range of educational backgrounds, ages, and years of experience within the affected employee population.
Principal investigators from the University’s infectious disease research community as well as members of the University Biohazards Committee and the Institutional Biosafety Committee reviewed the educational materials for both construct and content validity. Because this type of study had not been reported in the literature previously, it was not possible to determine external validity.
Materials were presented to affected employees in a series of informal educational group settings by a team of educators including the BSO, Director of Environmental Health and Safety, Medical Director of Employee Health Services, and the occupational health nurse. Management staff was present to answer any questions related to the work environment. In addition, representatives from the University’s Office of General Counsel and Human Resources Department were present to answer any employment questions and demonstrate the University administration’s support for the mandatory vaccination program. The educators were well known to the at-risk staff and had interacted with most of them on multiple occasions. All members of the audience were encouraged to ask questions and express their opinions concerning the new mandatory vaccination program.
Each employee received easy-to-understand written educational materials regarding the mandatory vaccinations including side effects, booster dose requirements (if applicable), and duration of action. Links to additional electronic resources pertaining to the vaccinations and the diseases caused by the infectious agents were included in the written material packet, as well as phone numbers and e-mail addresses for all presenters. The employees were encouraged to anonymously comment concerning the educational session and the overall mandatory vaccination program by sending comments and evaluations via interoffice mail. The final step in the program development and implementation process was to evaluate the efficacy of the program.
Study Design
This study was reviewed by the University’s Institutional Review Board and determined to be exempt from committee review. In an attempt to determine the effectiveness of the educational program, the researchers developed and distributed a survey to answer the following research question:
The objective of the survey was to determine the effectiveness of the educational program. Survey data were used to revise and improve the vaccine education program and materials as well as develop a model for use by other institutions that may develop a similar mandatory vaccination program.
Method
An electronic database was used to identify all employees who had received any of the mandatory vaccines since the implementation of the program in 2009 through the end of the most recent calendar year for which complete data were available (2012). The list was reviewed (51 employees were vaccinated during the study period), and terminated employees (11) were removed from the list of possible participants.
The researchers discussed use of a pre-test, but because employees who work with the identified infectious agents are required to garb completely, including but not limited to Powered Air Purifying Respirators (PAPRs), which limited employees’ time to interact with the researchers, and because the researchers had met with the employees informally during lunchtime interviews, a pre-test was not administered. The researchers considered mailing the test to the employees but felt another mailing to employees could be considered “overwhelming,” so this option was not used.
Each of the remaining employees (total = 40) received a survey, with a code to prevent a single employee from submitting more than one survey, a description of the study, and a consent form. Data collection began in January 2013 and ended in July 2013. Surveys were distributed via the U.S. Postal Service (USPS) because some of the employees in the sample population did not have daily access to a computer or personal mailboxes at their worksites. A total of three surveys (initial and two reminder surveys) were sent to employees. If a survey was returned as undeliverable, both human resources and the employee’s current supervisor were contacted to secure a current address. If the survey was returned, and neither the supervisor (e.g., employee had transferred to another department or left the University) nor Human Resources could provide an updated address, the individual was eliminated from the study (3).
Employees were directed to return the survey via e-mail, fax, University interoffice mail, or the USPS. Beginning 2 weeks after the initial surveys were distributed, employees were contacted either in person or via e-mail every week as a reminder. Reminders were distributed to all personnel who had not yet responded to maximize the response rate.
All data were collected and analyzed to determine any potential relationships between education retention and age, education, and job classification. Education retention was measured by assessing accuracy of employee responses to questions regarding known side effects and booster requirements for vaccines received (Questions 11 and 12 of Survey Instrument, appendix). Chi-square analysis was used to quantify the relationship between independent and dependent variables as well as Fisher’s Exact Test for independent samples with a significance level of .05. The analysis was conducted by an independent contractor.
Results
Three of the 40 surveys were undeliverable, even after contacting supervisors and the Department of Human Resources. Survey responses were received from 28 employees generating an overall response rate of 70%. Although a total of 6 surveys were missing answers to at least one question, due to the small size of the study population, none of the received surveys were classified as ineligible. The analysis included the data submitted by the 28 respondents to the survey.
The sample consisted of 11 (39.3%) males and 17 (60.7%) females. The sample population included animal care staff (i.e., animal husbandry personnel, veterinary technicians, and veterinarians), principal investigators (i.e., PhD- or MD-level researchers), research staff (i.e., graduate students, postdoctoral fellows, and research technical personnel), and health and safety personnel. The educational preparation of the sample population included personnel who had completed high school to terminal degrees in their fields. At the time of the study, active work with Bacillus anthracis, Neisseria meningitides, rabies virus, and yellow fever virus was ongoing. The number of employees vaccinated for each infectious agent between 2009 and 2012 is depicted in Figure 1.
Personnel vaccinated as part of mandatory vaccination program 2009-2012.
No significant associations between age or gender and feeling pressured to receive vaccination or the type of vaccination received were found (Figures 2 and 3). No significant associations between gender, education and occupation, and knowledge retention were found because due to the small sample size, no valid statistical tools were available for these computations. However, several points of interest to health educators were identified.
Associations between employee age or gender and perceived pressure to receive vaccine.
Associations between employee age or gender and type of vaccination received.
One of the original concerns voiced by some members of the University Biohazards Committee, and a common concern among institutions considering implementing mandatory vaccination programs, is whether employees will feel pressured to receive the required vaccinations (Douville, Myers, Jackson, & Lantos, 2010). However, in the study population, the majority of employees reported they did not feel pressured to receive the vaccinations (25; 89.7%). Of the three individuals who reported they did feel pressured to receive the vaccination (10.3%), only two indicated that the requirement itself was perceived as pressure. Employees indicated the perceived pressure was generated by the many precautions and rules associated with handling the selected agents. In fact, in the program thus far, all employees required to receive the mandated vaccinations have opted to be vaccinated instead of transferring to another department or facility where vaccination is not required.
The original educational materials for the mandated vaccination program were developed on a fifth-grade level to ensure that all employees, regardless of job classification or education, could easily understand the written and verbal explanations. Employee retention of information from written and verbal education was assessed by asking individuals to name three side effects of the vaccination received, whether a booster is required for the vaccination received, and when a booster dose is required for the vaccination (appendix). Study data indicated that gender, job classification, and education did not appear to influence the retention of program education in the study group.
Using the statistical conclusions and collaboration of the Environmental Health and Safety and the Occupational Health departments, the researchers are confident that (a) the University ensured successful adoption of the Mandatory Vaccination Program by including all potential stake holders during program development, (b) all at-risk employees were effectively identified by the Department of Environmental Health and Safety and appropriately screened by the Occupational Health Department, and (c) a strong positive partnership exists between the Occupational Health and Environmental Health and Safety departments. This successful educational endeavor was dependent on these three program aspects. As the research at the University focuses on new infectious agents, the mandatory vaccination program will be periodically reviewed and revised. The Occupational Health and Environmental Health and Safety departments will develop additional education activities as the mandatory vaccination program expands. The new educational programs will consider the above principles as well as these employee comments: “Thank you for making us feel safe”; “Maybe provide information on the University’s website”; “Why were certain vaccines required?” “What other vaccines will be required?”
Study Limitations
The small sample size, a significant limitation of this study, was defined by the number of personnel affected by the mandatory vaccination program. It is possible that selection bias was introduced by the individuals chosen to work with viruses that later required mandatory vaccine administration. Because the infectious agents handled by personnel in the sample are defined as high risk, it is possible that supervisors and principal investigators self-selected only highly motivated and easily trained individuals to work with these infectious agents. If this selection bias is present in the study, an effect between the selection of particular individuals and knowledge retention or demonstrated capacity to learn could be mistakenly attributed to the educational efforts described in this study. Employees were allowed to complete surveys at their own time without observation. This protocol may have provided opportunities for employees to speak to each other or to use external resources to improve their knowledge scores.
Future Studies
Enrollment in the University’s mandatory vaccination program continues to increase over time as the program is fully implemented. In future studies, a larger sample population would potentially allow research questions to be answered using parametric or nonparametric statistical tests, which was a major limitation of the present study. Future updates to University departmental information technology infrastructure could facilitate electronic distribution and analysis of future surveys, which may decrease employees’ opportunities to confer about answers and lead to a more accurate demonstration of knowledge retention. In addition, future studies could be designed to examine whether potential selection bias in regard to individuals chosen for higher-risk work has the potential to impact the educational process or employee retention of vaccine-specific knowledge. Finally, as research involving high-risk infectious agents continues to evolve, the authors plan to examine potential collaboration with other academic institutions that have implemented similar mandatory vaccination programs to increase the sample population and therefore the statistical power of the studies as well as to compare the effects of various educational strategies on employee health education.
Conclusion
This study supports the AAOHN core competencies, standards of practice, and research priorities (AAOHN, 2011, 2012a, 2012c). To achieve effective health promotion through health education, a complete and thorough assessment must be conducted prior to implementing any program modifications. A program plan should be developed and implemented, and perhaps, most importantly, the program MUST be evaluated to determine effectiveness. Did the program meet not only the goals of the presenter, but did the audience also comprehend the material presented, retain the desired knowledge, and use the knowledge to protect their health and safety at the worksite?
With significant investment of federal funding in infectious disease research as well as the construction of new high and maximum containment research facilities, the number of personnel working with and therefore potentially exposed to high-risk infectious agents will increase (Committee to Review the Health and Safety Risks of High-Biocontainment Laboratories at Fort Detrick, 2010; Jaax, 2005). As infectious disease research expands, occupational health professionals will be expected to develop comprehensive programs to protect worker health and safety (Committee to Review the Health and Safety Risks of High-Biocontainment Laboratories at Fort Detrick, 2010; Jaax, 2005; Rusnak, Kortepeter, Hawley, Anderson, et al., 2004; Rusnak, Kortepeter, Hawley, Boudreau, et al., 2004).
High Risk Infectious Agents in Use
The severity of disease caused by each infectious agent in use at the University for which an FDA-approved vaccine was available was also part of the risk assessment process for the mandatory vaccination program. Due to the potential severity of disease from laboratory exposures to the infectious agents in use (Harding & Byers, 2006; Rusnak, Kortepeter, Hawley, Anderson, et al., 2004; Rusnak, Kortepeter, Hawley, Boudreau, et al., 2004), the University decided to include the FDA-approved vaccinations for the following infectious agents in its mandatory vaccination program:
Bacillus anthracis causes moderate to severe, and sometimes fatal, disease in animals and humans (Wright, Quinn, Shadomy, Messonnier, & Centers for Disease Control and Prevention, 2010). The three main types of disease include cutaneous anthrax with an approximate 20% case fatality rate in untreated cases although deaths from cutaneous anthrax are rare if appropriate antibiotics are administered; gastrointestinal anthrax with a case fatality rate of 25% to 60% (Wright et al., 2010); inhalational anthrax, of particular concern in laboratory settings (Harding & Byers, 2006), is associated with high case fatality rates (estimated to be approximately 90%) even if identified early and appropriate antibiotics administered (Wright et al., 2010).
Japanese encephalitis virus can cause significant illness ranging from mild asymptomatic infection to fatal encephalitis. The case fatality rate is between 10% and 50%, with survivors experiencing neurological and psychiatric sequelae (Chen et al., 1999).
Meningitis, caused by Neisseria meningitides, has significant public health impact with 500,000 cases worldwide and 50,000 deaths annually. Very few medications provide effective treatment; 10% to 20% of survivors develop epilepsy, cognitive impairment, or sensorineural disease (Rosenstein, Perkins, Stephens, Popovic, & Hughes, 2001).
Rabies virus is a zoonotic microbe usually transmitted through a bite, although potential routes of laboratory exposures include percutaneous injuries from contaminated sharps and potential mucous membrane exposures. The virus causes an acute, progressive encephalomyelitis, and the disease is almost always fatal if post-exposure prophylactic treatment is not administered (Manning et al., 2008).
Yellow fever virus causes a mild to severe influenza-like disease in humans. The case fatality rate varies from less than 5% in endemic areas to 20% to 50% in non-endemic areas (Center for Disease Control and Prevention, 2011).
Applying Research to Practice
The use of an organized, well-developed and tested educational endeavor as presented in this study ensured a well-educated and non-apprehensive workforce and a safe work environment. As the research programs continue to expand, the occupational health program will grow to include other required vaccinations and safeguards with accompanying opportunities for employee education, question and answer sessions, and a supportive environment where employees feel safe.Utilizing the AAOHN model and guidelines for health promotion and injury prevention, the researchers developed appropriate educational tools, examined the effectiveness of their educational instruction, and will lead to use of these results to inform changes as the program continues to expand.
Footnotes
Appendix
Conflict of Interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the 2012 AAOHN Foundation Research Grant.
Author Biographies
Dr. Yolanda C. Lang is a practicing Occupational Health Nurse Practitioner at the University of Pittsburgh Medical Center. She has assisted in the development and implementation of the Animal Health Surveillance and BSL-3 Occupational Health programs at the University of Pittsburgh, and works closely with the Department of Environmental Health and Safety to ensure a safe workplace and a healthy workforce.
Dr. Molly Stitt-Fischer is a Certified Biological Safety Professional and is the University Biosafety Officer in the Department of Environmental Health and Safety. She manages biosafety programs at the University of Pittsburgh, including the University’s BSL-3 laboratory safety program, and works in close collaboration with the Occupational Health providers at the University of Pittsburgh Medical Center to support safe conduct of research at the University.