Abstract
Keywords
Introduction
Influenza is a contagious respiratory infection with the potential to cause serious illness and death. Although studies have shown effective strategies to increase coverage rates in recommended groups, vaccination rates for influenza remain below national goals. Despite the expanded recommendations, influenza vaccination coverage for all eligible age groups during the 2012-2013 season was 45.0%. 1
Most children and adults routinely visit physician’s offices to obtain acute care as well as immunizations and other preventive services. Frequency of visits varies greatly depending on age as well as health conditions. Most visits represent an opportunity to provide recommended vaccinations. Thus when a recommended vaccine is not provided at a visit, it is a missed opportunity (MO). MOs have been well documented for children2-5 and adults.6,7 Studies uniformly find that MOs to vaccinate are associated with suboptimal vaccination rates regardless of setting, sample or patient age. One study of influenza during the 2004-2005 season assessed 1724 children, 6 through 23 months of age seeking care at 52 primary care sites found that 62.0% had at least 1 MO during this influenza season. 5 Clinic-associated reasons for failure to vaccinate include provider knowledge and attitudes, failure to discuss vaccination, suboptimal medical records, and other operational factors.6,8 Assessing patients 65 years or older in 4 adult care settings, Nowalk et al 9 found low rates for annual influenza (24.1%), pneumococcal (49.1%), and tetanus (28.6%) with MOs to this set of vaccines ranging from 38.4% to 94.5% across these clinics. Assessing influenza vaccination in adults 50 years and older at a sample of 13 clinics, Fishbein et al 7 found MOs in 38.6% of clinic visits, with MOs more frequent in January (56.6%) compared with December or earlier (29.4%). Younger adults between 50 and 64 years experienced greater MOs compared with those 65 years and older (48.5% vs 26.8%). 7
Efforts to reduce MOs to vaccinate are an important goal of quality improvement. Systematic reviews have identified the effectiveness of many evidence-based practices to improve immunization, implementation is often difficult.10-15 Data for this study were collected as part of a cluster randomized intervention designed to determine the effectiveness of encouraging medical clinics to extend influenza vaccinations efforts into late-season defined as January through March. 16 The purpose of this report is to assess MOs to vaccinate for influenza among children 6 months to 5 years and adults 50 years and older.
Methods
Description of Medical Clinics
To ensure a variety of medical practice types and a diverse patient population, clinic participation was based on type of clinic (private practice, medical group, nonprofit community health center), medical specialty (pediatric, family medicine, internal medicine), and geographic location within San Diego County. Participating clinics agreed to have clinic managers complete detailed interviews regarding their immunization administration practices and to consider using project-promoted interventions. Details of the key informant interviews have been described elsewhere. 16 Clinics received feedback about influenza coverage, MO rates, and practical evidence-based practices to improve influenza coverage.
Random Chart Review
To be eligible for the random audits, patients had to have made a visit to the practice between April 1, 2010 and March 31, 2011, as evidenced from billing or electronic medical records. Pediatric patients must have been at least 6 months by October 1, 2010 and younger than 5 years by April 1 2011 to be included in the database. Adults were included if they were 50 years or older by October 1, 2010. A target sample of 250 charts for each practice was established for systematic review; however, if fewer records were available then all eligible records were reviewed. Data collected included: patient’s age, insurance type if available in the medical record, dates and number of visits during influenza season (from September 2010 to March 2011), date when influenza vaccination was administered, documented refusals for influenza vaccination if any, and where influenza vaccine was administered if provided outside of that clinic.
Measurement of Missed Opportunities to Vaccinate for Influenza
Most published studies measure MO as an aggregate percentage calculated by summing the number of visits for an entire clinic or for an individual provider for which no vaccine was provided dividing by the total number of visits observed for that clinic or provider; however, alternate methods for calculating MO for influenza vaccination have been reported in the literature.8,9 Here, we tabulated for each patient, 1 MO at each visit during the influenza season when the patient’s chart showed no record of having received the influenza vaccine. For children, documentation of 1 dose of influenza vaccine was considered being up to date.
Statistical Analysis and Human Subjects Approval
The total number MOs for each patient was obtained by summing the number of visits with an MO until vaccination was recorded or the influenza season ended; and the percentage of visits with MOs was obtained by dividing this total over the number of visits for each month. The clinics were of the following practice types: community health center, private practice, and medical group. The insurance categories were self-pay or private, public, including military, Medicaid (pediatric patients only), or Medicare (adult patients only), and unknown. Being up to date was defined as having received the influenza vaccine during the observation period, where the regular season spanned from September 1 to December 31, 2010, and the late season was from January 1 to March 31, 2011. While charts from children came from pediatric clinics, adult charts were from family practice and internal medicine clinics. Because of the low number of pediatric charts available in family medicine clinics, these were excluded from the study.
Timing is an important element in adequacy of vaccination. Our study calculated patient specific MO rates that account for variable numbers of visits and a variable period of time between the patient’s first visit after the start of the influenza vaccination season and the date when the influenza vaccination was recorded. We used Kaplan-Meier survival analysis to examine the timing of vaccination coverage over the influenza season.17,18 Multivariable Cox proportional hazards regression was used to identify associations of individual and clinic characteristics with influenza vaccination by the end of the season Variables included in the multivariable regression analysis were selected using a priori knowledge from the study design, the literature, and examination of bivariate regression results (not shown). Because patients were selected at the clinic level clustering was taken into account in the analysis using the robust sandwich estimate in the Cox proportional hazard models. 19 All analyses were performed using SAS (Version 9.3, SAS Institute Inc, Cary, NC). The study protocol was approved by the University of California San Diego Institutional Review Board (IRB 081790) as well as each clinic’s administration and applicable institutional reviews.
Results
Complete data on 1136 pediatric patients aged 6 months to 5 years, were available. These came from 6 pediatric clinics, including 2 community health centers, 2 private practices, and 2 medical groups. For adults 50 years and older, data on 1329 patients were available coming from 7 primary care providers, including 2 community health centers, 3 private practices, and 2 medical groups.
Figure 1 displays the cumulative influenza vaccination coverage. Influenza coverage increases most rapidly during October and November. December vaccinations increased more rapidly for children than adults. Very few vaccinations were documented after January 1, especially among the adults. Influenza vaccination rates were much higher for children than for adults.

Cumulative influenza vaccination for children 6 months to 5 years and adults aged 50 years or older, by month 2010-2011 influenza vaccination season, 13 primary care clinics, San Diego, CA.
The number of unvaccinated patients declines most rapidly during the early season (Table 1). The percentage of pediatric visits with an MO was high at 70% in September, dropped to 48% in October and again rose steadily to 92% by March. For adults, the percentage of visits with an MO was 87% in September, dropped to a low of 66% in October then rose to the highest level in February at 98%. MOs for adult influenza vaccination occurred in almost all patient visits (>97%) in January and later months.
Patient Visits and Influenza Vaccination for Children 6 Months to 5 Years, and Adults Aged 50 or Older, by Month 2010-2011 Influenza Vaccination Season, 13 Primary Care Clinics, San Diego, CA.
Table 2 displays the number of MOs for influenza vaccination by patient and clinic characteristics. Vaccination at the first visit (no MO) during the season occurred in only 41.7% of the children and 30.2% had 2 or more MO visits. Those pediatric patients with private health insurance had fewer MOs compared with those with Medicaid or with no record of insurance. Among adults, 44.9% had 2 or more visits with MOs. The 50 to 64-year age group was least likely (11.3%) to get vaccinated during the first visit compared with the other adult age groups. Fewer MOs were associated with patients 80 years and older. Medical groups and private practices had more patients vaccinated on their first visit (no MO) compared with the community health centers.
Missed Opportunities for Influenza Vaccination, by Individual and Clinic Characteristics, Children 6 Months to 5 Years, and Adults Aged 50 Years or Older, by Month 2010-2011 Influenza Vaccination Season, 13 Primary Care Clinics, San Diego, CA.
Abbreviations: MO, missed opportunity; n/a, not applicable; CHC, community health center; PP, private practice; MG, medical group; FP, family practice; IM, internal medicine.
P < .05 based on χ2 test among children.
P < .05 based on χ2 test among adults.
Medicaid insurance type applied to children only, while Medicare applied to adults only.
Restricted to patients with influenza vaccination by the end of the observation period.
The age group of the child and the number of MOs were significantly associated with influenza vaccination by the end of the season as seen in Table 3. Younger patients (6-17 months old) were 2.65 times more likely to receive vaccination compared with the older children (4-5 years old). Among adults, those attending a medical group, compared to patients seen at community health centers, were 1.97 times more likely to get vaccinated (95% confidence interval = 1.76-2.21). Among both children and adults, multiple opportunities in an extended influenza vaccination season did not lead to getting these repeat clinic visitors vaccinated (P < .001).
Risk Factors Associated With Vaccination, Children 6 Months to 5 Years and Adults Age 50 Years or Older 2010-2011 Influenza Vaccination Season, 13 Primary Care Clinics, San Diego, CA.
Abbreviations: HR, hazard ratio; CI, confidence interval; CHC, community health center; PP, private practice; MG, medical group.
Medicaid insurance type applied to children only, while Medicare applied to adults only.
Discussion
This study investigated MOs for influenza vaccination during a complete influenza season, from September 2010 to March 2011. Both in adult and pediatric patients, cumulative vaccination coverage plateaued starting in January, although the number of patients having clinic visits did not substantially decrease. The percentage of visits with an MO was higher in the month of September than in October through December. While the vaccine was made available early in September, we suspect that some clinics did not have it or that standing orders or other procedures for influenza vaccination were not in full effect in September. Although some patients only visited a clinic once during the influenza season, a large proportion of patients was seen 2 times or more but were still not vaccinated. We observed that after the first MO, the likelihood of receiving vaccination is extremely diminished.
Our findings quantify the large numbers of MOs to vaccinate for influenza that took place in a diverse sample of primary care medical settings. Also affirmed is the conventional wisdom that the vast majority of influenza vaccinations are completed prior to January 1. Overall, study clinics had poor cumulative influenza vaccination coverage during the 2010-2011 season, possibly as an effect of less vigilance in clinic’s vaccination efforts after the intense interest and media coverage during the H1N1 pandemic during the previous year. This post-H1N1 decline in seasonal influenza vaccination levels was similar to that seen by researchers in Italy. 20
Age of the patient is seen to clearly influence the number of MOs experienced. Vaccinations are a cornerstone of pediatric care and younger children in our sample were in the age range of those who are still undergoing their recommended series of initial vaccines. Children in this vaccine intensive period of their lives had the highest influenza vaccination rates. Also, the seasonal pattern of vaccination timing (Figure 1) shows that compared with adults, influenza vaccination in children continues after the first of the year, albeit at a slower rate; likely because of “routine” nature of vaccines in pediatric clinics.
In children, older age was a negative predictor of influenza vaccination, while among adults it was found to be a positive predictor. It is possible that parents of older children were wary of repeated vaccinations or that the clinic visits for the children in the older age groups do not allow time for discussion of influenza vaccination. Among adults, this observation is consistent with Fishbein et al 7 that adults 50 to 64 years old were more likely to have MOs during the season than adults aged 65+ years. This observation could also be attributed in part to the difference in insurance providers, as 58% of adults aged 50 to 64 years had self-pay or private insurance. Other patient related factors not studied may be influencing the repeated MOs such as clinical conditions, or patient/parent refusals based on their personal beliefs (or their parents). Second, patients who may desire vaccination may not request the vaccine because they are not aware that they can also get vaccinated for influenza even if the main reason for the clinic visit is for an acute problem.
It is interesting to speculate why clinics and the public continue to view the influenza vaccination season as the time prior to the start of the New Year. Some insight may be gained from the differences between children and adults as well as the relative priorities for immunization provided for these age groups. Data from our key-informant interviews of clinic managers showed that patient management processes or evidence-based practices that are known to facilitate better vaccination rates were used at less than optimal levels. 16 Many clinics, especially those serving adults, tend to prepare for influenza early in the season by having special “flu clinic” sessions or mass vaccination events. After winding down before the New Year, influenza vaccination is relegated to nonpriority status, thought about and prescribed less frequently. From a process improvement perspective, until adult vaccinations are integrated into routine clinic practices (such as prompts from electronic medical records) MOs will continue to abound. Influenza vaccination after the first of the year will be relegated to an afterthought.
Our methods employed a patient-focused unit of analysis to examine the problem of MOs allowing the examination of individual factors implicated. Previous studies have calculated MOs by aggregating the number of visits with no vaccination and dividing this number by the total patient population to calculate the percentage of visits with an MO.6,7 By using individual records, we quantify the number of patients without documented vaccination who have 1, 2, or more visits for care during the influenza season. Also, the use of Kaplan-Meier and Cox proportional hazards allowed adjustment for unequal periods of patient observation and provided better estimates in multivariable models predicting factors associated with vaccination by the end of the season.
Despite the thorough methodology, there are a few limitations to this study. The results are based on analysis of a limited number of clinics in a single large metropolitan community and as such cannot be generalized. While rigorous, our methods may underestimate the actual influenza coverage because of several factors. Clinics may not have documented influenza vaccinations given at other locations such as drug stores, worksites, or community events. Also, some patients may have been offered influenza vaccine and refused it. Our abstractors were trained to look for explicit refusals but only 52 were found. Abstracting information from both paper and electronic medical records at these diverse clinics also presented problems of standardization.
This study highlights circumstances contributing to MOs for influenza vaccination. It adds to the understanding of internal and external factors both at the clinic and the individual levels. With the annually recurring influenza vaccination season, the results of this study can help shape interventions to increase coverage rates in pediatric and older adult patients. The findings of this study could influence the timeliness and sustainment of standing orders protocols.
Conclusion
Our study quantifies the abundance of MOs to vaccinate for influenza. For most clinics, influenza vaccination ends in December. Influenza vaccination does not appear to be integrated into processes of routine medical care, rather annual influenza vaccination efforts are viewed as short-term events added on to already hectic day-to-day activities in these primary care clinics. Based on these results, to reduce MOs, clinic-based strategies are needed that encourage the use of evidence-based clinic operations such as standing orders, prompts in electronic medical records, patient reminders, and monitoring of missed influenza vaccination opportunities.
Footnotes
Acknowledgements
The authors gratefully acknowledge all clinic partners who aided in the completion of this project. We also thank the following staff for their support in completion of this project: Michelle Rubio, Adrienne Collins-Yancey, Karen Waters-Montijo, Jennifer Sterling, Everardo Aguilar, Daniel Wurm, Jae Hansen, Naomi Katzir, and Nancy Knickerbocker. We thank Pascale Wortley, MD at CDC for her oversight of the cooperative agreement.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funded in part by the U.S. Centers for Disease Control and Prevention (CDC) Cooperative agreement: 5U01IP000189.
