Sage Journals: Discover world-class research

Abstract

Objectives

To conduct a systematic review of strategies to optimize immunisation uptake within preschool children in developed countries.

Design

Systematic review.

Setting

Developed countries

Participants

Preschool children who were due, or overdue, one or more of their routine primary immunisations.

Main outcome measures

Increase in the proportion of the target population up to date with standard recommended universal vaccinations.

Results

Forty-six studies were included for analysis, published between 1980 and 2009. Twenty-six studies were randomized controlled trials, 11 were before and after trials, and nine were controlled intervention trials. Parental reminders showed a statistically significant increase in immunisation rates in 34% of included intervention arms. These effects were reported with both generic and specific reminders and with all methods of reminders and recall. Strategies aimed at immunisation providers were also shown to improve immunisation rates with a median change in immunisation rates of 7% when reminders were used, 8% when educational programmes were used and 19% when feedback programmes were used.

Conclusion

General practitioners are uniquely positioned to influence parental decisions on childhood immunisation. A variety of strategies studied in primary care settings have been shown to improve immunisation rates, including parental and healthcare provider reminders.

DECLARATIONS

Competing interestsNone declared

Ethical approvalNot applicable

GuarantorNW

Contributorship

All authors contributed equally

AcknowledgementsNone

Introduction

Childhood vaccines currently save 3 million lives a year globally and are among the most successful and cost-effective public health interventions of the 20th century.¹ Immunisation has been responsible for substantial falls in serious bacterial infectious diseases in children including tetanus, diphtheria, meningococcal serogroup C (MenC), measles and Haemophilus Influenzae B (HiB), since their inclusion in the UK primary vaccination schedule.² Immunising children not only protects individuals from infection but also contributes to population-based immunity by reducing the circulation of infectious agents leading to community-wide health gains.³

To maximize the potential population-wide benefits of routine vaccination through herd immunity, the World Health Organization (WHO) has set national targets of 95% coverage annually for each antigen in the routine immunisation schedule by 2 years of age.⁴ Yet recent figures show that these targets are not being met in many countries. In England, in the quarter between January and March 2010, vaccination coverage for the measles, mumps and rubella vaccine (MMR), pneumococcal vaccine (PCV) booster and the Hib/MenC vaccines were 88%, 88% and 90%, respectively, at 24 months of age.⁵

Coverage tends to be lowest in deprived, urban areas with mobile populations.^6–8 For example, childhood immunisation rates for London are substantially below the national average with levels in one East London borough (Bexley) of just 63%, 71% and 74% for the PCV (Pneumococcal conjugate vaccine) booster, MMR and Hib/MenC, respectively.⁵ In areas of low MMR coverage, this has resulted in outbreaks, with a high proportion of those affected being under 1 year of age, who were most likely infected by unimmunised older contacts. One consequence of this is that measles and mumps have once again become endemic in the UK, with over 1100 cases of measles and over 7000 cases of mumps reported in the UK in 2009, 14 years after local transmission of measles was halted.⁹

Barriers to immunisation can stem from parental concerns about risks, inadequate knowledge and provision by providers, and generalized systemic barriers involving the organization of the health system and access to services.¹⁰ Ninety-eight percent of infants born in the UK are registered with a UK general practitioner (GP)¹¹ and their first contact with their GP is often at the primary vaccination. Hence, GPs and practice nurses are uniquely positioned to influence a parent's decisions to have their child immunised.

Our objective was to conduct a systematic literature review aimed at providing GPs with up-to-date, evidence-based guidelines on how to improve uptake rates of primary immunisations for children registered under their care. Our research question was ‘How can primary care practitioners in developed countries improve preschool immunization uptake?'

Methods

Search strategy and data sources

We systematically searched electronic databases including MEDLINE, EMBASE, PsycInfo, Cochrane and OpenSIGL from inception to 1 June 2010 using MESH and Key Terms including immunis*, vaccin*, innoc* and rates, uptake and coverage (Appendix 1) to identify studies reporting interventions to improve preschool immunisation uptake and to evaluate their effectiveness in children in developed countries.

We hand-searched the reference lists from retrieved studies and reviews to find additional studies and contacted experts in the field. We also identified relevant grey literature such as conference papers, dissertations and government guidelines.

Study selection

Two reviewers (NW and HW) independently screened titles and abstracts of all citations for eligibility and retrieved those that met the inclusion criteria. If insufficient information was available in the abstract to decide on eligibility the whole article was retrieved for review. Discrepancies were resolved by discussion and by involving a third reviewer (SS) when necessary.

We included experimental studies reporting original research including randomized controlled trials, controlled clinical trials, before and after studies and interrupted time-series studies, published in English. Our target population was children under the age of 5 years living in developed countries. We included studies reporting our main outcome measure; the increase in the proportion of the target population who were up to date with standard recommended universal vaccinations. Outcomes could be for either single vaccinations or combinations of vaccines due. We excluded studies for which the full article was not available, and studies that did not contain any original data such as review articles, commentaries and correspondence.

Data extraction and quality assessment

The two reviewers independently extracted data from included studies on study setting, participant characteristics, clinical setting, interventions and outcomes measured. Included studies were graded for methodological quality using 26 points from the 27 point score devised by Downs and Black (Appendix 2).¹²

Results

A total of 32,624 studies were retrieved from the electronic searches of which 32,410 were excluded after reviewing their titles and abstracts. The remaining 214 papers were retrieved for full text review. Recommendations from experts in the field and reviews of reference lists identified a further 14 studies. A total of 46 papers met the inclusion criteria (Figure 1).

Figure 1

Quorom diagram

The commonest reasons for exclusion were failure to report interventions or outcomes of interest and poor study design.

The 46 included studies were published between 1980 and 2009. Twenty-six were randomized controlled trials, 11 were before and after trials and nine were controlled intervention trials. The studies originated mainly from the USA (36) and the remainder originated from the UK and Ireland (7), Australia (1), New Zealand (1) and Finland (1). The study participants ranged in age from 6 weeks to 11 years of age but the majority of studies focused on preschool children. Studies were conducted in different settings including paediatric outpatient clinics, family practices, primary care clinics, community health centres, managed care organizations and health maintenance organizations in the USA; and community clinics and general practices in the UK, New Zealand and Australia.

Studied interventions aimed to remind and/or recall parents of upcoming or overdue vaccinations; targeted providers to improve uptake through feedback, audit or chart prompts; provided simple education to parents within a general practice setting or consisted of multicomponent interventions.

Client-based interventions

Client-based interventions target the parent and child to increase the demand for immunisation services. Educating parents and communities on the benefits of vaccinating their children can empower parents to practice preventative healthcare and thereby improving immunisation uptake.

Reminder and recall

Reminders aim to advise parents of upcoming vaccinations that are due and remind parents of those children that are overdue. They vary in methodology from automated telephone calls and generic postcards to personalized letters and even home visits.

Twenty-two included papers reported on 41 intervention arms studying parental reminders and recalls.¹³ ^– ³⁴ Details of the included studies are seen in Table 1. The average score for study quality using Down and Black's quality scoring framework was 24.8 out of a potential 31 (range 21– 29.5). Fourteen (34%) of the 41 intervention arms showed a statistically significant (P < 0.05) increase in immunisation rates (Szilagyi et al.²⁹ did not report significance). Overall, these studies reported a median point change of 11% (mean 10%, range -11% to 24%). Studies comparing usual care with postal reminders alone^{17,19,20,23–26,30,33,34} reported an overall median point change of 10% (mean 8.6%, range -11% to 19%). Studies looking at telephone reminders alone^14,17,28,31 reported an overall median point change of 9.5% (mean 12%; range 3-24%). Those studies looking at the effect of the combination of postal and telephone reminders on vaccination rates^{13,15–17,21,31} reported an overall median point change of 10.5% (mean 10.8%, range 2.8-19%). One study¹⁸ compared a traditional reminder card with a card designed with health belief model in mind. Hawe reported a 12% increase in immunisation rates when the health belief model reminder card was used.

Table 1

Study characteristics of reminder and recall studies

Paper	Study period	Setting and population	Design	Quality	Intervention	Outcome
Abramson et al. ¹³	1993	Public health centre and children's hospital continuity clinic, Forsyth County, North Carolina, USA; low socioeconomic status	Randomized controlled trial	23	1. Usual care (control group) vs. 2. Postcard reminders followed by telephone reminders	Age appropriate immunisations (DTP/OPV/ Hib) at 7 months of age 1 vs. 2, net change = 19% (P <0.00001)
Alemi et al.¹⁴	1993-1994	Paediatric outpatient clinic, Cleveland, USA; children under 6 months of age at recruitment; urban; predominantly ethnic minorities; low socioeconomic status	Controlled intervention trial	22.5	1.Usual care (control group) vs. 2.Computer-generated telephone reminders and recalls	On time immunisation with complete series (DTP/OPV/ MMR/Hib): 1 vs. 2, net change = 24.4% (P = 0.0005)
Alto et al.¹⁵	1991	Family practice clinic, Colorado, USA; children between 2 months and 7 years old; low socioeconomic status	Randomized controlled trial	26	1.Usual care (control group vs. 2.Specific postcard reminders followed by telephone reminders	Up to date with DTP/OPV/ MMR/Hib vaccinations: 1 vs. 2, net change = 8% (P <0.011)
Daley et al.¹⁶	2000	Primary Care Clinic, Denver, Colorado, USA; Predominantly Medicaid and uninsured patient population; Children aged 6 weeks to 22 months	Randomized controlled trial	29.5	1.Usual care (control group) vs. 2.Reminder letter followed by telephone recall 10 days later	Immunisation with one or more doses of PCV7: 1 vs. 2, net change = 2.8% (95% confidence interval -1.8% to 7.4%)
Dini et al.¹⁷	1993-1996	County health department, Denver, USA; children aged 60-90 days who had received the first dose of DTP and/or IPV	Randomized controlled trial	25	1.Usual care (control group) vs. 2.Telephone messages vs. 3.Letter reminders vs. 4.Telephone messages followed by letters	Completed immunisation series by 24 months of age 1 vs. any intervention net change = 8.3% (RR (rate ratio) = 1.21; CI (confidence interval) = 1.01-1.44). 1 vs. 2 net change = 8.4%, 1 vs. 3 net change = 7.3%, 1 vs. 4 net change = 9%
Hawe et al.¹⁸	Not reported	Local government-operated public vaccination clinic, Ballarat, Australia, 259 children due for measles vaccination aged 12 months	Randomized controlled trial	26.5	1. Usual reminder card 2. Health belief model reminder card	Usual card group 67% vaccinated Health belief model card group 79% 12% difference (0.026)
Irigoyen et al.¹⁹	1997	Paediatric clinic, New York, USA; children aged 4-18 months	Controlled intervention trial	26.5	1.Usual care (control group) vs.z 2.Postcard reminder vs. 3.Telephone reminder vs. 4.Postcard plus telephonereminder	No significant difference in vaccination coverage by study group
Irigoyen et al.²⁰	2001	Five inner-city community paediatric practices, New York, USA; children aged 6 weeks to 15 months due or late for DTP	Randomized controlled trial	27	1.Usual care (control group) vs. 2.Continuous postcard reminders (as many as needed) vs. 3.Limited postcard reminders(up to 3)	Up to date 4:3:1:3 at 3 months post randomization 1 vs. 2 net change = 5.3% (P < 0.01), 1 vs. 3 net change = 1.5% (not significant) Up to date 4:3:1:3 at 6 months post randomization 1 versus 2 net change = 4.9% (p < 0.05), 1 vs. 3 net change = 2.8% (not significant) Multivariate analysis showed reminders had no independent effect on immunization outcomes
Kempe et al.²¹	1999	Outpatient paediatric clinic, urban teaching hospital, Denver, Colorado, USA; children aged 5-17 months; low socioeconomic status; highly transient population	Randomized controlled trial	26.5	1.Usual care (control group) vs. 2. Postcard reminder plus telephone reminders	UTD at 12 months 1 vs. 2 net change 12% (P = 0.07)
Le Baron et al.²²	1996-1998	Fulton County, Georgia, USA; children born between 1 July 1995 and 6 August 1995	Randomized controlled trial	25	1.Usual care (control group) vs. 2.Automated telephone or mail reminder recall vs. 3.In-person telephone, mail or home visit recall vs. 4.Auto-dialler with outreach backup	UTD with DTP/OPV/MMR/ Hob (4:3:4:1) at 24 months of age 1 vs. 2 net change = 6% (P < 0.05), 1 vs. 3 net change = 3% (not significant), 1 vs. 4 net change = 4% (not significant)
Lieu et al.²³	1994-1995	Managed care organization, Northern California, children aged 20-24 months old, n = 149 control group, n = 172 intervention group	Randomized controlled trial	23.5	1.Usual care (control group) vs. 2.Patient recalls via computer-generated personalized letter	MMR by age 24 months, 1 vs. 2 = 19% net change (P < 0.001)
Lieu et al.²⁴	1996-1997	Health Maintenance Organization, California, USA; under-immunised 20-month old children	Randomized controlled trial	23	1.Automated telephone reminder vs. 2.Letter reminder vs. 3.Automated telephone reminder followed by a reminder letter 1 week later vs. 4. Letter reminder followed by an automated telephone message 1 week later	Proportion of under-immunised children who received any needed vaccinations by age 24 months, 1 vs. 2 no net change; 1 vs. 4 net change 14% (P = 0.02); 2 vs. 4 14% (P = 0.01), 1 vs. 3 net change = 9% (P = 0.1), 2 vs. 3 net change 9% (P = 0.09)
Mason and Donnelly²⁵	1998-1999	Local health authority, Wales, UK; children aged 21 months who had not received MMR vaccine	Randomized controlled trial	24	1.Usual care vs. 2.Personal reminder letter and educational leaflet	Immunised with MMR at age 21-24 months of age 1 vs. 2, net change = 1.1% (95% CI -3.3-5.5); immunised with MMR at >24 months of age, 1 vs. 2 = 1.1% (95% CI -3.6-5.7)
Morgan et al.²⁶	1996	South Glamorgan, Wales, UK; children aged between 9 months and 21 months	Randomized controlled trial	27.5	1.Usual care (control group) vs. 2.Non-directive telephone call to child's health visitor vs. 3.Mailed reminder	Immunised with MMR 1 vs. 2 net change = -7%, 1 vs. 3 net change = -11%
Rodewald et al.²⁷	1994-1995	Nine primary care practices, Rochester, New York, USA; 3015 infants	Randomized controlled trial (two by two factorial design)	25.5	1.Usual care (control group) 2.Tracking/Outreach prompting- lay outreach workers using postcards, telephone calls and home visits 3.Tracking/outreach 4.Prompting- during visits to primary care office, marking charts	UTD net change 1 vs. 2 = 21%; 1 vs. 3 21%; 1 vs. 4 = 2% (none showed significance); 1 vs. 2 plus 3 P < 0.001
Stehr-Green et al.²⁸	1990	Public health clinics, Atlanta, Georgia, USA; Average age 8.7-9.2 months; n = 110 control group, n = 112 intervention group	Randomized controlled trial	24.5	1.Usual care (control group) vs. 2.Automated telephone reminder (max 9 attempts)	DTP immunisation, 1 vs. 2 = 3% net change (non-significant)
Szilagyi et al.²⁹	1993-1996	Three geographical regions of Monroe County, New York, USA; children aged up to 2 years	Randomized controlled trial	24.5	1.Usual care 1993 (control group) 2.Staged intervention of increasing intensity letters, postcards or phone calls and outreach 1999	UTD with DTP/OPV/MMR/ Hib 4:3:1:1 1 vs. 2 net gain = Monroe county 20%, suburbs 15%, inner city 29%, rest of city 17% (significance not stated)
Tollestrup and Hubbard³⁰	1987	County health department clinics, Washington, USA; children aged up to 5 years	Controlled intervention trial	23.5	1.Usual care (control group) vs. 2.Recall letter if 1 month overdue	DTP vaccination within 5 months, 1 vs. 2 net change = 18% (P < 0.01)
Vivier et al.³¹	1998	Primary care clinics, Rhode Island, USA; children enrolled in a managed care programme, up to the age of 6 years	Randomized controlled trial	24	1.Usual care (control group) vs. 2.Telephone reminder vs. 3.Postal reminder vs. 4.Sequential postal/telephone reminders	Immunisations UTD, 1 vs. 2, net change = 11% (P > 0.05); 1 vs. 3, net change = 12% (P > 0.05); 1 vs. 4, net change = 14% (P > 0.05); 1 vs. 2-4 combined P < 0.05
Wilcox et al.³²	1997	Philadelphia, Pennsylvania, USA; 1752 children aged 6-10 months	Randomized controlled trial	24.5	1.Usual care (control group) vs. 2.Outreach	One immunisation received during study period 1 vs. 2 net change = 18% (P < 0.001); UTD with immunisations 1 vs. 2 net change -1% (not significant)
Yokley and Glenwick³³	Not stated	Public health clinic, Akron, Ohio, USA; children aged under 5 years; mean age 37 months; n= 183-195 in each group	Group randomized trial by family	23.5	1.Usual care (control group) vs. 2.General postal patient reminder vs. 3.Specific postal patient reminder vs. 4.Specific postal patient reminder plus special out of hours clinics vs. 5.Specific postal patient reminder plus parent incentive lottery	Vaccinated with at least one antigen after 3 months: 1 vs.2 = 3% net change (non-significant), 1 vs.3 = 13% (non-significant),1 vs. 4 = 16% (significant),1 vs. 5 = 18% (significant)
Young et al.³⁴	1978	Ohio, USA; 6-month old high-risk children	Randomized controlled trial	21	1.Usual care (control group) vs. 2.Mailed reminder	One immunisation received during study period 1 vs. 2 net change = 16% (P = 0.02); UTD with vaccinations 1 vs. 2 net change = 12% (P = 0.06)

Parental reminders have been shown to have an overall positive effect on immunisation uptake. These effects have been reported with both generic and specific reminders and with all methods of reminders and recall.

Parental education

In the context of this review, we considered only educational programmes that could feasibly be delivered within the setting of primary care and we excluded studies reporting national or regional education programmes.

Two of the included papers reported on two intervention arms studying the effect of simple parental education programmes on immunisation uptake (Table 2).^35,36 One study assessed the impact of a promotional teddy bear featuring the address of an information website for MMR and one studied the impact of an interactive graphic card and verbal explanation on immunisation uptake. The quality of these two studies averaged at 23.8 points from a possible 31. Neither study showed a significant effect on immunisation rates and the limited number of studies precludes us from reaching an evidenced-based conclusion on the effect of these strategies on parental behaviour.

Table 2

Study characteristics of parental education studies

Paper	Study period	Setting and population	Design	Quality	Intervention	Outcome
Porter-Jones et al.³⁵	Not reported	Flintshire, Wales, UK; children eligible for their first dose of MMR	Randomized controlled trial	22	1. Usual care (control group) vs. 2. Usual care plus a promotional teddy bear displaying an MMR information website and telephone number	MMR1, 1 vs. 2 net change 0.7% (P = 0.143)
Stille et al.³⁶	1997-1998	Three paediatric primary care sites, Connecticut, USA; children under 28 days old; inner city; low socioeconomic status	Controlled intervention trial	25.5	1. Usual care (control group) vs. 2. Interactive graphic immunisation card plus verbal explanation	Rate of appropriate immunisation aged 7 months 1 vs. 2 net change 0.4% (P = 0.93); DTP1 by 3 months of age 1 vs. 2 net change 2.8% (P = 0.47)

Patient-held records

Only one study³⁷ assessing the impact of patient-held records effect on immunisation rates fulfilled the inclusion criteria (Table 3); it scored 22 from a possible 31 points on study quality. This study did not demonstrate a significant difference between usual care and a home-based record booklet. Therefore, it is not possible to come to an evidence-based conclusion on the effect of this strategy on immunisation uptake.

Table 3

Study characteristics of patient-held record studies

Paper	Study period	Setting and population	Design	Quality	Intervention	Outcome
Lakhani et al.³⁷	1980	West Lambeth Health Authority, London, UK; mothers from obstetric wards at St Thomas’ Hospital	Randomized controlled trial	22	1. Usual care (control group) vs. 2. Home-based record booklet	No significant difference in the uptake of immunisations between groups

Provider-based interventions

Provider-based interventions aim to improve vaccination rates by reducing opportunities missed by the medical professional to immunise children. These include provider reminder/recall, assessment and feedback, provider education and a combination of some or all of these strategies.

Provider reminder/recall

This strategy aims to investigate client's immunisation status either by manual searching of notes or by automatic computer notifications. Providers are then notified either with paper or computer-based chart prompts that the vaccination is due or overdue.

Five of the included papers^38–42 reported on six intervention arms studying provider reminder/ recall strategies (Table 4). The studies averaged a quality score of 23.7 from a possible 31 (range 21-27). Overall, these studies reported a median point change of 7% (mean 10%, range -2% to 33%). Both manual searching and electronic reminders were shown to have a positive effect on immunisation rates.

Table 4

Study characteristics of provider-based interventions

Paper	Study period	Setting and population	Design	Quality	Intervention	Outcome
Burns et al.³⁸	1995-1996	Two family health centres, Pittsburgh, Pennsylvania, USA; children aged up to 6 years; 448 intervention group, 529 control group	Randomized controlled trial	25	1.Usual care (control group) vs. 2.Chart prompt	1 vs. 2: Immunisation with DTP4 net change 15% (P = 0.03); Immunisation with MMR 1 net change 16% (P = 0.01); Immunisation with OPV3 net change 14%(P = 0.04); Immunisation with DTP3, DTP5, HepB3 and OPV4, no statistically significant difference 1 vs. 2
Christy et al.³⁹	1990-1991	Two hospital-based primary care centres, Rochester, New York, USA; Children aged 2 to 60 months old; Urban	Controlled intervention trial	21	1.Usual care at site 2,1 July 1991-6 December 1991(concurrent control group) vs. 2.Nursing intervention guided by an algorithm form at site 1 during same period vs. 3.Usual care at site 1,1 July 1990 and 30 November 1990(retrospective control group)	Up to date with DTP/OPV/ MMR/Hib aged 24-36 months net change 1 vs. 2 = 0.4%, 2 vs. 3 = 10% (significance not tested)
Fairbrother et al.⁴⁷	1995-1996	Family and paediatric practices in nine neighbourhoods, New York City, USA; low socioeconomic status	Randomized controlled trial	24.5	1.Usual care (control group) vs. 2.Physician bonus and feedback vs. 3.Enhanced fee for service and feedback 4.Feedback only	UTD with DTP/Hib/OPV/ MMR 8 months after baseline, 1 vs. 2 net change = 19.2% (P < 0.01), no significant difference between group 3 or 4 and control group
Fiks et al.⁴⁰	2004- 2005	Four urban primary care centres, Philadelphia, Pennsylvania, USA; children under 24 months old; >80% ethnic minorities	Before and after study	27	1.Usual care (control group) vs. 2.Electronic Health Record-based clinical reminders	1 vs. 2: Up to date for 4DTP/ 3IPV/1MMR/3Hib/ 3HepB/1varicella In well child net change 33% (95% CI 32.2-34.6), in sick child 22% (95% CI 20.6-23.1)
Franzini et al.⁴³	2003-2005	Paediatric and family health practices, Greater Houston, Texas, USA; children aged 12-23 months of age	Before and after study	20.5	1.Usual care vs. 2.1-hour peer-based educational lunch presentation	Immunisation rates 1 vs. 2 net change = 4% (not significant)
Harper et al.⁴⁸	October 1993-October 1994	Family practice clinic (intervention), community health centre (control), St Paul, Minnesota, USA; children aged 24-35 months old; predominantly white, low socioeconomic group	Controlled intervention trial	23	1.Usual care(non-equivalent control group) vs. 2.Physician chart reminder plus feedback on performance plus patient education	4:3:1 DTP/OPV/MMR respectively, 1 vs. 2= 12% net change (P <0.02)
Margolis et al.⁴⁴	Not reported	44 primary care practices, North Carolina, USA; children aged 24-30 months	Randomized controlled trial	28	1.Usual care (control group) vs. 2.Practice-based continuing medical education	Immunisation rates at 30 months post randomization 1 vs. 2 net change 0.6% (significance not reported)
Sinn et al.⁴⁹	Not reported	Ten paediatric group practices, Virginia Beach, Virginia, USA; children aged 9-30 months	Before and after trial	21	1.Usual care (control group vs. 2.‘The physician leadership model'including peer influence and review and goal setting with feedback	Up to date with immunisations at 24 months 1 vs. 2 net change 18.8% (P < 0.001)
Smith et al.⁴⁵	1994	Children under 2 years of age; low income families	Before and after study	21	1.Usual care vs. 2.Physician education plus free vaccines and assistance with billing disputes	UTD for age, 1 vs. 2, net change = 25% (P < 0.00001)
Soljak and Handford⁴¹	1985	Clinics and offices, Northland, New Zealand; all children born during study periods	Controlled intervention trial for patient reminders; before and after study for provider reminders	21	1.Retrospective control group vs. 2.GP reminder vs. 3.GP reminder plus postal reminder cards	Up to date ‘with all appropriate antigens’ at 5 months 2 vs. 3 net change 0.4% (p = 0.86); 1 vs. 2 and 3 net change 4% (P= 0.034)
Szilagyi et al.⁴²	1991-1993	Teaching hospital paediatric clinic and neighbourhood health centre, Rochester, New York, USA; children aged up to 2 years (mean ages 7-13 months); 1988 participants before randomization; urban; low socioeconomic status	Randomized controlled trial	24.5	1.Usual care (control group) 2.Provider chart reminders	UTD with DTP/OPV/MMR/ Hib, 1 vs. 2 net change = 3% at clinic (P = 0.3) and -2% at health centre (P = 0.5)
Taylor et al.⁵⁰	Not reported	Private paediatric and family clinics, King County, Washington, USA; children aged 3-19 months	Randomized controlled trial	28	1.Immunisation rate feedback and assessment of current immunisation procedures 2.As above plus peer educational programme	Immunisation rate 1 vs. 2 net change = 2.2% (P = 0.24)
Waterman⁴⁶	1992-1994	San Diego, California, USA; children aged 2-4 years; low socioeconomic status	Controlled intervention trial	20	1.Usual care (control group) 2. Walk-in clinics plus provider education plus patient reminders plus education and health promotion	DTP/OPV/MMR (4:3:1) 1 vs. 2 net change = 12% (not significant)

Provider education

Provider education strategies aim to enhance the knowledge of the immunisation provider through a variety of methods including peer support and the use of educational resources. Educational tools may be one-off sessions or part of continuing medical education.

Four of the included papers^43–46 reported on four intervention arms studying the effect of educating the provider of vaccinations on immunisation rates (Table 3). The average quality score for included studies was 22.4 (range 20-28). Overall, these studies reported a median point change of 8% (mean 10%, range 1-25%). The educational interventions varied from 1-hour peer education sessions to regular continuing medical education within the practice. Two studies looked at provider education alone and two studies looked at education in combination with other interventions such as patient reminders.

Feedback

This strategy retrospectively evaluates the performance of providers in childhood immunisations and feeds this information back to the medical practitioner. Feedback can also be combined with other strategies for example financial incentives or provider education.

Four of the included papers (Taylor, Sinn, Harper, Fairbrother) reported on six intervention arms studying the effect of provider feedback combined with other strategies on immunisation rates (Table 3). The average quality score for these papers was 24.1 (range 21–28). Overall, these studies reported a median point change of 19% (mean 17%, range 12–19%).

Multicomponent interventions

Multicomponent interventions encompass strategies that use a combination of techniques to improve immunisation uptake. These strategies include combining interventions aimed at both the client and the provider.

Eight of the included studies reported on eight intervention arms that combined a variety of interventions aimed at improving immunisation uptake (Table 5).^51–58 The average score for study quality was 20.5 from a possible 31 (range 17.5–24). Overall, these studies reported a median point change of 15% (mean 19%, range –4% to 47%). Three (38%) of the eight intervention arms reported a statistically significant difference in immunisation rate. Four (50%) studies did not report the significance level for their intervention arms. It is not possible to distinguish which component of the intervention has had the greatest effect on immunisation rates.

Table 5

Study characteristics of multicomponent intervention

Paper	Study period	Setting and population	Design	Quality	Intervention	Outcome
Carter and Jones⁵¹	1982	Fife, Scotland, UK; children aged 2 years	Before and after study	17.5	1. Usual care vs. 2. Health education programme including mass media campaign and distribution of measles immunisation guidelines and updates	Immunisation with measles vaccine 1 vs. 2 net change 13% (significance not reported)
Hicks et al.⁵²	2002	Community health centre, Northeast Colorado, USA; children aged 13-35 months old; 75% population Latino	Before and after study	21	1. Usual care vs. 2. Reminder cards plus posters in examination rooms	Completely immunised with DTP/IPV/Hib/HepB/MMR/ Varicella 1 vs. 2 net change =12.2% (P < 0.05)
Melinkovich et al.⁵³	1996 (pre) 2006 (post)	Community and school health centres, Denver, Colorado, USA; children aged 12-35 months	Before and after study	19	1. Usual care vs. 2. Postcard reminders plus standing orders and provider education	Immunization rates in 12-23-month-old cohort, 1 vs. 2 net change = 26%; in 24-35 month-old-cohort, 1 vs. 2 net change = 47% (significance not tested)
Mohr et al.⁵⁴	2000-2001	University of North Carolina paediatric clinic; children under 24 months old; ethnically diverse	Before and after study	19	1. Usual care (control group) vs. 2. Clinical improvement model including parent education and standardizing immunization records	Proportion of children UTD with DTP/Polio/MMR/Hib and HepB vaccines (4:3:1:3:3) at 24 months old; 1 vs. 2 net change 26% (P= 0.04)
Murphy et al.⁵⁵	1994-1995	General Practice, Dublin, Ireland; children aged between 6 months and 5 years; inner city	Before and after study	22	1. Usual care (control group) vs. 2. Postal reminders plus opportunistic immunisation	Vaccine uptake 1 vs. 2 net change for DTP = 27% (P < 0.0005), Hib = 43% (P < 0.0005), MMR 22% (P < 0.0005)
Oeffinger et al.⁵⁶	Not reported	Family practice residency programme in community and hospital, Texas, USA; children aged under 12 months	Controlled intervention trial	24	1. Usual care (control group) vs. 2. Clinic patient education plus non-specific reminder letter at 2 months old	UTD with 3 doses of DTP/30PV at 12 months, 1 vs. 2 = -4% net change (0 = 0.41)
Paunio et al.⁵⁷	1982-1986	Community-wide, Finland; children aged 0-11	Time series study	21.5	1. Usual care vs. 2. Registry plus mass-media reporting local on vaccination rates plus provider reminders plus parent reminders	MMR 1 vs. 2 = 8% net change (significance not tested)
Rossdale et al.58	1982-1985	Well-baby clinic, Bristol, UK; ethnically diverse; low socioeconomic status	Before and after study	20	1. Usual care prior to changes vs. 2. Open access to clinic plus immunization reminder on notes plus ‘well baby card’ for parents to keep noting due immunizations and recording those received	1 vs. 2 immunised with DTP net change = 11%; Pertussis = 16%; Measles 1% (significance not performed)

Discussion

Numerous studies have reported interventions to improve primary immunisation uptake in children. Effective interventions include parental reminders, which can increase uptake by 11% in the intervention arms. These effects were reported with both generic and specific reminders and with all methods of reminders and recall. Strategies aimed at immunisati on providers were also shown to improve immunisation rates with a median change in immunisation rates of 7% when reminders were studied, 8% when educational programmes were studied and 19% when feedback programmes were studied.

Providers who were educated by peers and who received feedback on their performance as vaccine providers were shown to have improved immunisation uptake within their practice. There was limited evidence for patient-held records and parental education alone as strategies for improving immunisation uptake.

Multicomponent strategy studies included interventions aimed at parents alone (e.g. reminder cards plus educational posters) as well as those that combined parental and healthcare provider strategies (e.g. parental reminders plus provider education). Melinkovich showed when studying community and school health centres in Denver, USA that combining parental and provider strategies led to the greatest point increase in immunisation coverage in the multicomponent strategy group of studies.⁵³ The most successful strategies appear to be those that target both the healthcare provider and those that are to be immunised.

Our search methods yielded more studies than any similar previous review. We combined studies from a variety of clinical settings and a range of socioeconomic populations in developed countries, which makes the results generalisable in this setting. Included studies were heterogenous in setting, service delivery, intervention delivery and quality which made meta-analysis difficult. We did not include studies from developing countries as the barriers to immunisation are different from those in developed countries and include financial barriers that are generally not relevant to parents and general practitioners in many developed countries that offer universal access to primary care services. Grey literature, conference papers and government documents were also included in the review as well as research published in peer-reviewed journals to ensure the broadest possible range of studies and to minimize publication bias. We excluded studies published in languages other than English, which is the main weakness of this study.

We did not examine the effect of GP financial incentives on immunisation uptake. There is a large literature of ‘pay for performance’ in healthcare and this has been a subject of previous systematic reviews, the inclusion of which is outside the scope of this paper.^59,60

Various sociodemographic factors have been shown to reduce the likelihood of a child being up to date with recommended vaccinations including being from a lone parent family^61,62 from an ethnic minority^63,64 and living in urban areas.⁶⁵ The high level of mobility seen in these populations is thought to contribute to these differences.^66,67 Interventions to increase vaccination rates have a greater effect on those who are most at risk of being under-immunised. Hence, it is important that vaccine coverage data are collected in a way that highlights differences in uptake rates between socioeconomic and ethnic groups. This would help in the implementation and evaluation of public health programmes to improve immunisation rates in the groups in the most need of intervention.

Childhood immunisation can continue to benefit the health of children. For example, the UK has seen a successful campaign to add pneu-mococcal vaccination to the primary schedule with a catch-up campaign for older children. This has resulted in a substantial reversal of previously increasing trends with falls in hospital admissions for bacterial pneumonia (20%) and empyema (22%) in the 2 years after implementation, linked with uptake rates of 80% and 98% after the first and second years of the campaign, respectively.⁶⁸

Conclusions

Maintaining high vaccine uptake rates is an essential component of the success of any vaccination programme and in improving the health status of children. Health planners and professionals must engage actively with parents and the public and invest in process measures that ensure children receive primary prevention. Our review has highlighted a number of interventions that can help improve childhood immunisation rates in developed countries. These include reminding parents and providers of upcoming and overdue immunisations and educating and providing feedback to the vaccination providers. Some additional research is required to test the cost-effectiveness of these interventions and their impact in groups with poor immunisation rates or high risks of complications from vaccine preventable diseases.

Footnotes

Appendix 1

TERM	LINKING TERM
Immunis*	OR
Vaccin*	OR
Inoculat*	OR
Tetanus	OR
Diptheria	OR
Polio*	OR
Measles	OR
Mumps	OR
Rubella	OR
Pertussis	OR
Whooping	OR
Haemophilus	OR
Pneumococcal	OR
MMR
	AND
Rates	OR
Coverage	OR
Uptake

Appendix 2

Checklist for measuring study quality: Downs S and Black N. The feasibility of creating a checklist for the assessment of the methodological quality of both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52:377–384

Reporting 1.

Is the hypothesis/aim/objective of the study clearly described?

Yes =1

No = 0

Are the main outcomes to be measured clearly described in the Introduction or Methods section?

If the main outcomes are first mentioned in the Results section, the question should be answered no.

Yes =1

No = 0

Are the characteristics of the patients included in the study clearly described?

Inclusion and/or exclusion criteria should be given.

Yes =1

No = 0

Are the interventions of interest clearly described?

Treatments and placebo (where relevant) that are to be compared should be clearly defined.

Yes =1

No = 0

Are the distributions of principal confounders in each group of subjects to be compared clearly described?

Yes =2

Partially = 1

No = 0

Are the main findings of the study clearly described?

Simple outcome data (including denominators and numerators) should be reported for all major analyses and conclusions. (This question does not cover statistical tests which are considered below).

Yes =1

No = 0

Does this study provide estimates of the random variability in the data for the main outcomes? In non normally distributed data the inter quartile range of results should be reported. In normally distributed data the standard error, standard deviation or confidence intervals should be reported. If the distribution of the data is not described, it must assume that the estimates used were appropriate and the question should be answered yes.

Yes =1

No = 0

Have all important adverse events that may be a consequence of the intervention been reported?

Not going to use this one.

Yes =1

No = 0

Have the characteristics of patients lost to follow up been described?

Yes =1

No = 0

10.

Have actual probability values been reported (eg 0.035 rather than < 0.05) for the main out comes except where the probability value is less than 0.001?

Yes =1

No = 0

References

Maciosek

, Coofield

, Edwards

, Flottemesch

, Goodman

, Solberg

. Priorities among effective clinical preventive services: results of a systematic review and analysis. Am J Prev Med 2006; 31: 52–61.

Roush

, Murphy

. Historical comparisons of morbidity and mortality for vaccine-preventable diseases in the United States. JAMA 2007; 298: 2155–63.

Fox

, Elveback

, Scott

, Gatewood

, Ackerman

. Herd immunity: basic concept and relevance to public health immunization practices. 1971. Am J Epidemiology 1995; 141: 187–97.

World Health Organization. Operational targets for EPI diseases. Geneva: WHO, 1996

HPA Quarterly Vaccine Coverage Data Tables Q09-4 Jan to Mar 2010. See http://www.hpa.org.uk/web/HPAweb&HPAwebStandard/HPAweb_C/1211441442288

Baker

, Garrow

, Shiels

. Inequalities in immunisation and breast feeding in an ethnically diverse urban area: cross-sectional study in Manchester, UK. J Epidemiol Community Health 2011; 65: 346–52.

Coupland

, Harcourt

, Vinogradova

. Inequalities in uptake of influenza vaccine by deprivation and risk group: time trend analysis. Vaccine 2007; 25: 7363–71.

Samad

, Tate

, Dezateux

, Peckham

, Butler

, Bedford

. Differences in risk factors for partial and no immunisation in the first year of life: prospective cohort study. BMJ 2006; 332: 1312–13.

Editorial team.

Measles once again endemic in the United Kingdom.

Euro Surveill 2008; 13; pii=18919. See http://www.eurosurveillance.org/ViewArticle.aspx? ArticleId=18919

10.

Kimmel

, Burns

, Wolfe

, Zimmerman

. Addressing immunization barriers, benefits and risks. J Fam Pract 2007; 56: S61–S69.

11.

Hippisley-Cox

, Vinogradova

. Trends in consultation rates in General Practice 1995/1996 to 2008/2009: Analysis of the Research database. Leeds: Information Centre for Health & Social Care, 2009. See URL: http://www.ic.nhs.uk/webfiles/publications/gp/Trends_in_Consultation_Rates_in_General_Practice_1995_96_to_2008_09.pdf

12.

Downs

, Black

. The feasibility of creating a checklist for the assessment of the methodological quality of both randomised and non-randomised studies of health care intervention. J Epidemiol Community Health 1998; 52: 377–84.

13.

Abramson

, O'Shea

, Ratledge

, Lawless

, Givner

. Development of a vaccine tracking system to improve the rate of age-appropriate primary immunization in children of lower socioeconomic status. J Pediatr 1995; 126: 583–6.

14.

Alemi

, Alemagno

. Computer reminders improve on-time immunization rates. Med Care 1996; 34 (Suppl. 10): OS45–51.

15.

Alto

, Fury

, Condo

, Doran

, Aduddell

. Improving the immunization coverage of children less than 7 years old in a family practice residency. J Am Board Fam Pract 1994; 7: 472–7.

16.

Daley

, Steiner

, Brayden

, Xu

, Morrison

, Kempe

. Immunization registry-based recall for a new vaccine. Ambul Pediatr 2002; 2: 438–43.

17.

Dini

, Linkins

, Sigafoos

. The impact of computer-generated messages on childhood immunization coverage. Am J Prev Med 2000; 19: 68–70.

18.

Hawe

, McKenzie

, Scurry

. Randomised controlled trial of the use of a modified postal reminder card on the uptake of measles vaccination. Arch Dis Child 1998; 79: 136–40.

19.

Irigoyen

, Findley

, Earle

, Stambaugh

, Vaughn

. Impact of appointment reminders on vaccination coverage at an urban clinic. Pediatrics 2000; 106 (Suppl. 4): 919–23.

20.

Irigoyen

, Findley

, Wang

. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr 2006; 6: 100–4.

21.

Kempe

, Lowery

, Pearson

. Immunization recall: Effectiveness and barriers to success in an urban teaching clinic. J Pediatr 2001; 139: 630–5.

22.

LeBaron

, Starnes

, Rask

. The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Arch Pediatric Adolesc Med 2004; 158: 255–61.

23.

Lieu

, Black

, Ray

. Computer-generated recall letters for underimmunized children: how cost-effective? Pediatr Infect Dis J 1997; 16: 28–33.

24.

Lieu

, Capra

, Makol

, Black

, Shinefield

. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics 1998; 101: e3.

25.

Mason

, Donnelly

. Targeted mailing of information to improve uptake of measles, mumps, and rubella vaccine: a randomised controlled trial. Commun Dis Public Health 2000; 3: 67–8.

26.

Morgan

, Evans

. Initiatives to improve childhood immunisation uptake: a randomised controlled trial. BMJ 1998; 316: 1569–70.

27.

Rodewald

, Szilagyi

, Humiston

, Barth

, Kraus

, Raubertas

. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics 1999; 103: 31–8.

28.

Stehr-Green

, Dini

, Lindegren

, Patriarca

. Evaluation of telephoned computer-generated reminders to improve immunization coverage at inner-city clinics. Public Health Rep 1993; 108: 426–30.

29.

Szilagyi

, Schaffer

, Shone

. Reducing geographic, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices. Pediatrics 2002; 110: e58.

30.

Tollestrup

, Hubbard

. Evaluation of a follow-up system in a county health department's immunization clinic. Am J Prev Med 1991; 7: 24–8.

31.

Vivier

, Alario

, O'Haire

, Dansereau

, Jakum

, Peter

. The impact of outreach efforts in reaching underimmunized children in a Medicaid managed care practice. Arch Pediatr Adolesc Med 2000; 154: 1243–7.

32.

Wilcox

, Koepke

, Levenson

, Thalheimer

. Registry-driven, community-based immunization outreach: a randomized controlled trial. Am J Public Health 2001; 91: 1507–11.

33.

Yokley

, Glenwick

. Increasing the immunization of preschool children; an evaluation of applied community interventions. J Appl Behav Anal 1984; 17: 313–25.

34.

Young

, Halpin

, Johnson

, Irvin

, Marks

. Effectiveness of a mailed reminder on the immunization levels of infants at high risk of failure to complete immunizations. Am J Public Health 1980; 70: 422–4.

35.

Porter-Jones

, Williams

, Powell

, Pusey

, Roberts

. Impact of a novel way to communicate information about MMR on uptake of MMR vaccine: a randomized controlled trial. Public Health 2009; 123: 78–80.

36.

Stille

, Christison-Lagay

, Bernstein

, Dworkin

. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila) 2001; 40: 365–73.

37.

Lakhani

, Avery

, Gordon

, Tait

. Evaluation of a home based health record booklet. Arch Dis Child 1984; 59: 1076–81.

38.

Burns

, Zimmerman

, Santibanez

. Effectiveness of chart prompt about immunizations in an urban health center. J Fam Pract 2002; 51: 1018

39.

Christy

, McConnochie

, Zernik

, Brzoza

. Impact of an algorithm-guided nurse intervention on the use of immunization opportunities. Arch Pediatr Adolesc Med 1997; 151: 384–91.

40.

Fiks

, Grundmeier

, Biggs

, Localio

, Alessandrini

. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics 2007; 120: 707–14.

41.

Soljak

, Handford

. Early results from the Northland immunisation register. N Z Med J 1987; 100: 244–6.

42.

Szilagyi

, Rodewald

, Humiston

. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med 1996; 150: 1193–200.

43.

Franzini

, Boom

, Nelson

. Cost-effectiveness analysis of a practice-based immunization education intervention. Ambul Pediatr 2007; 7: 167–75.

44.

Margolis

, Lannon

, Stuart

, Fried

, Keyes-Elstein

, Moore

Jr . Practice based education to improve delivery systems for prevention in primary care: randomised trial. BMJ 2004; 328: 388

45.

Smith

, Connery

, Knudsen

. A preschool immunization project to enhance immunization levels, the public-private relationship, and continuity of care. J Community Health 1999; 24: 347–58.

46.

Waterman

, Hill

, Robyn

. A model immunization demonstration for preschoolers in an inner-city barrio, San Diego, California, 1992-1994. Am J Prev Med 1996; 12 (Suppl. 4): 8–13.

47.

Fairbrother

, Hanson

, Friedman

, Butts

. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health 1999; 89: 171–5.

48.

Harper

, Madlon-Kay

, Luxenberg

, Tempest

. A clinic system to improve preschool vaccinations in a low socioeconomic status population. Arch Pediatr Adolesc Med 1997; 151: 1220–3.

49.

Sinn

, Morrow

, Finch

. Improving immunization rates in private pediatric practices through physician leadership. Arch Pediatr Adolesc Med 1999; 153: 597–603.

50.

Taylor

, Rietberg

, Greenfield

. Effectiveness of a physician peer educator in improving the quality of immunization services for young children in primary care practices. Vaccine 2008; 26: 4256–61.

51.

Carter

, Jones

. Measles immunisation: results of a local programme to increase vaccine uptake. Br Med J (Clin Res Ed) 1985; 290: 1717–19.

52.

Hicks

, Tarr

, Hicks

. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med 2007; 20: 581–6.

53.

Melinkovich

, Hammer

, Staudenmaier

, Berg

. Improving pediatric immunization rates in a safety-net delivery system. Jt Comm J Qual Patient Saf 2007; 33: 205–10.

54.

Mohr

, Randolph

, Laughon

, Schaff

. Integrating improvement competencies into residency education: a pilot project from a pediatric continuity clinic. Ambul Pediatr 2003; 3: 131–6.

55.

Murphy

, Harrington

, Bury

. Impact of a collaborative immunisation programme in an inner city practice. Ir Med J 1996; 89: 220–1.

56.

Oeffinger

, Roaten

, Hitchcock

, Oeffinger

. The effect of patient education on pediatric immunization rates. J Fam Pract 1992; 35: 288–93.

57.

Paunio

, Virtanon

, Peltola

. Increase of vaccination coverage by mass media and individual approach: intensified measles, mumps, and rubella prevention program in Finland. Am J Epidemiol 1991; 133: 1152–60.

58.

Rossdale

, Clark

, James

. Improved health care delivery in an inner-city well-baby clinic run by general practitioners. J Roy Coll Gen Pract 1986; 36: 512–13.

59.

Granerdon

, White

, Andrews

, Crowcroft

. Vaccine coverage in England: the impact of health service reorganisation. Arch Dis Child 2006; 91: 805–7.

60.

Giuffrida

, Gosden

, Forland

. Target payments in primary care: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 1999; 4: CD000531

61.

Sharland

, Atkinson

, Maguire

, Begg

. Lone parent families are an independent risk factor for lower rates of childhood immunisation in London. Commun Dis Rep CDR Rev 1997; 7: R169–172.

62.

Bardenheier

, Yusuf

, Rosenthal

. Factors associated with under immunization at 3 months of age in four medically underserved areas. Public Health Rep 2004; 119: 479–85.

63.

Marks

, Halpin

, Irvin

, Johnson

, Keller

. Risk factors associated with failure to receive vaccinations. Pediatrics 1979; 64: 304–9.

64.

Herrera

, Zhao

, Klevens

. Variation in vaccination coverage among children of Hispanic ancestry. Am J Prev Med 2001; 20 (Suppl. 4): 69–74.

65.

Kenyon

, Matuck

, Stroh

. Persistent low immunization coverage among inner-city preschool children despite access to free vaccine. Pediatrics 1998; 101: 612–16.

66.

Peckham

, Bedford

. Towards better immunisation rates. Practitioner 1989; 233: 1343–4.

67.

Pearce

, Elliman

, Bedford

, Law

. Residential mobility and uptake of childhood immunisations: findings from the UK Millennium Cohort Study. Vaccine 2008; 26: 1675–80.

68.

Koshy

, Murray

, Bottle

, Sharland

, Saxena

. Impact of the seven-valent pneumococcal conjugate vaccination (PCV7) programme on childhood hospital admissions for bacterial pneumonia and empyema in England: national time-trends study, 1997-2008. Thorax 2010; 65: 770–4.

Primary care strategies to improve childhood immunisation uptake in developed countries: systematic review

Abstract

Objectives

Design

Setting

Participants

Main outcome measures

Results

Conclusion

Introduction

Methods

Search strategy and data sources

Study selection

Data extraction and quality assessment

Results

Client-based interventions

Reminder and recall

Parental education

Patient-held records

Provider-based interventions

Provider reminder/recall

Provider education

Feedback

Multicomponent interventions

Discussion

Conclusions

Footnotes

Appendix 1

Appendix 2

References