Sage Journals: Discover world-class research

Abstract

Objectives:

The federal Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program requires grantees to demonstrate program improvement as a condition of funding. The MIECHV program monitors grantee progress in federally mandated conceptual areas (ie, benchmarks) that are further subcategorized into related sub-areas or constructs (eg, breastfeeding). Each construct has an associated performance measure that helps MIECHV collect data on program implementation and performance. In 2016, MIECHV modified the constructs and associated performance measures required of grantees. Our objective was to identify whether the constructs were supported by the home visiting literature.

Methods:

We conducted an evaluation of one of the MIECHV program’s benchmarks (Benchmark 1: Maternal and Newborn Health) for alignment of the Benchmark 1 constructs (preterm birth, breastfeeding, depression screening, well-child visit, postpartum care, and tobacco cessation referrals) with home visiting evidence. In March 2016, we searched the Home Visiting Evidence of Effectiveness database for all publicly available articles on studies conducted in the United States to determine how well the study findings aligned with the MIECHV program constructs.

Results:

Of 59 articles reviewed, only 3 of the 6 MIECHV constructs—preterm birth, breastfeeding, and well-child visits—were supported by home visiting evidence.

Conclusions:

This evaluation highlights a limited evidence base for the MIECHV Benchmark 1 constructs and a need to clarify other criteria, beyond evidence, used to choose constructs and associated performance measures. One implication of not having evidence-based performance measures is a lack of confidence that the program will drive positive outcomes. If performance measures are not evidence based, it is difficult to attribute positive outcomes to the home visiting services.

Keywords

program evaluation evaluation methodology home visits child health maternal health

Compared with other countries, the United States falls short on many maternal and child health outcomes. It has one of the highest infant mortality rates compared with other high-income peer countries.¹ In one report, the United States ranked 33rd among 179 countries in the Mothers’ Index, an assessment of the well-being of mothers and children, and 61st in maternal health, with maternal mortality rates (ie, deaths related to pregnancy complications) that were 10 times higher than rates in some peer countries.² In 2014, the preterm birth rate in the United States was 9.6 per 100 live births, and the low birth-weight rate was 8 per 100 live births, unchanged from 2007-2014.³ These factors are strongly associated with infant mortality.

Program Description

The 2010 Patient Protection and Affordable Care Act, in an effort to improve maternal–child health status in the United States, created the Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program, administered by the Health Resources and Services Administration in partnership with the Administration for Children and Families. In fiscal year 2016, the Health Resources and Services Administration awarded $344 million to support MIECHV’s implementation.⁴ Home visiting services have a long history in the United States, with some evidence of positive impacts on prenatal, post-birth, and long-term health and behavioral outcomes of mothers and children.^5
–7 Home visiting services began in the United States in the mid- to late-1880s.⁸

A 1988 survey identified more than 4500 home visiting efforts nationwide.⁸ Each home visiting program model has its own requirements and processes but shares some components, such as a focus on parenting education and comprehensive services.⁹ The US Department of Health and Human Services launched the Home Visiting Evidence of Effectiveness (HomVEE)¹⁰ project in 2009 to systematically gather and transparently assess the quality of the evidence from the home visiting literature. HomVEE includes literature on home visiting program models published since 1979 and has reviewed 45 home visiting program models (eg, Early Head Start, Parents as Teachers, Child First, and Healthy Families America). MIECHV grantees that receive federal funds must choose from the evidence-based home visiting program models⁴ that are reviewed by HomVEE. In implementing those evidence-based home visiting program models, MIECHV grantees are required to demonstrate improvement in federally mandated conceptual areas (ie, benchmarks) that are further subcategorized into related sub-areas, or constructs.

Purpose

From program inception, MIECHV grantees used 37 constructs within 6 benchmarks defined by the Health Resources and Services Administration to track program implementation, progress, and impact. In 2016, MIECHV added 7 and removed 10 constructs.¹¹ Key goals of this redesign were to reduce the annual reporting burden for grantees and increase the utility of the constructs and associated performance measures. MIECHV’s performance measures for monitoring maternal–child health improvement are crucial for evaluating the program’s success at the local and federal levels.¹² To offer the best chance of ameliorating poor US maternal–child health outcomes, the MIECHV program would ideally track progress on performance measures for proven evidence-based constructs. For example, the performance measure for the construct preterm birth is the percentage of infants (among mothers who enrolled in home visiting prenatally before 37 weeks) who are born preterm following program enrollment (Table 1). The objective of this study was to evaluate alignment between the home visiting program literature, as identified through HomVEE, and the MIECHV constructs for MIECHV Benchmark 1 (Maternal and Newborn Health).

Table 1.

Maternal, Infant, and Early Childhood Home Visiting constructs and performance measures for Benchmark 1: Maternal and Newborn Health, 2016^a

Construct	Performance Measure
1. Preterm birth	Percentage of infants (among mothers who enrolled in home visiting prenatally before 37 weeks) who are born preterm following program enrollment
2. Breastfeeding	Percentage of infants age 6-12 months (among mothers who enrolled in home visiting prenatally) who were breastfed any amount at 6 months of age
3. Depression screening	Percentage of primary caregivers enrolled in home visiting who are screened for depression using a validated tool within 3 months of enrollment (for those not enrolled prenatally) or within 3 months of delivery (for those enrolled prenatally)
4. Well-child visit	Percentage of children enrolled in home visiting who received the last recommended visit based on the American Academy of Pediatrics schedule
5. Postpartum care	Percentage of mothers enrolled in home visiting, prenatally or within 30 days after delivery, who received a postpartum visit with a health care provider within 8 weeks (56 days) of delivery
6. Tobacco cessation referrals	Percentage of primary caregivers enrolled in home visiting who reported using tobacco or cigarettes at enrollment and were referred to tobacco cessation counseling or services within 3 months of enrollment

^aThe constructs are subcategories under Benchmark 1: Maternal and Newborn Health. Data source: James Bell Associates, Inc.¹¹

Methods

In March 2016, we reviewed the literature on home visiting program models after we identified articles through a search in HomVEE.¹³ HomVEE is a comprehensive, federally sponsored database updated yearly with the research literature on home visiting. Using HomVEE’s search engines, we selected “child health” and “maternal health” in the Outcome Domain Measured tab, and we selected “high,” “moderate,” and “low” from the Rating tab. HomVEE rates articles that examine the evidence of effectiveness of a home visiting model using quantitative data and statistical analyses.¹⁰ The HomVEE search yielded 74 articles. We excluded 6 articles on studies conducted outside of the United States and 9 articles that were not publicly available and whose authors did not reply after 2 months of attempting to secure a copy of their article. We conducted searches on multiple occasions and cross-checked findings across time points.

We retrieved the full text of the remaining 59 articles through primary sources. We reviewed each article to identify those whose outcomes aligned with the 6 constructs (preterm birth, breastfeeding, depression screening, well-child visits, postpartum care, and tobacco cessation referrals) of MIECHV’s Benchmark 1: Maternal and Newborn Health (Table 1). We selected this benchmark because it is directly related to maternal–child health outcomes and because the constructs are urgent areas for improvement in the United States. We identified alignment when the article clearly studied an outcome similar to the MIECHV construct. For example, in a randomized trial, Fergusson et al¹⁴ tracked the effect of home visiting on the number of well-child visits received; Caldera et al¹⁵ did the same through a review of medical records. We considered that the outcomes in both studies aligned with the construct well-child visits and that both articles provided evidence to support these performance measures. A study could be aligned with more than 1 construct.

When we identified an alignment, we next reviewed the results pertaining to that outcome (eg, number of well-child visits) for their effect (positive effect or no effect). We noted a positive effect when the study demonstrated a significant outcome. We considered a study’s findings to have no effect when the study reported no effect of the intervention (eg, home visiting) on the outcome measured (eg, number of well-child visits). Finally, when reviewing these articles, we noted “other” outcomes identified through these methods that demonstrated a positive effect of the home visiting program model on maternal–child health. We noted these other outcomes as potential evidence-based constructs for Benchmark 1 after we cross-checked that they were not already measured by Benchmarks 2-5 (Benchmark 2: Child Injuries, Abuse, Neglect, Maltreatment and Emergency Department Visits; Benchmark 3: School Readiness and Achievement; Benchmark 4: Crime or Domestic Violence; Benchmark 5: Family Economic Self-sufficiency). Benchmark 2 tracks emergency department visits only for child injury. When we cross-checked, we considered articles that looked for all causes for emergency department visits and hospitalization for parents and children (Figure 1).

Figure 1.

Search, selection, and review process of evidence for Maternal, Infant, and Early Childhood Home Visiting Benchmark 1: Maternal and Newborn Health using the Home Visiting Evidence of Effectiveness study search tool, United States, 2016. Data sources: James Bell Associates, Inc¹¹ and US Department of Health and Human Services.¹³ Abbreviations: CAPS, Childhood Asthma Prevention Study; CBFRS, Community-Based Family Resource and Support; CPEP, Child Parent Enrichment Project; REST, Reassurance, Empathy, Support, and Time-Out.

A driver diagram is a visual tool used as a framework for measuring and monitoring progress on a goal or objective. Part of the value of a driver diagram is to facilitate communication of complex evaluation plans.¹⁶ For this study, we used a driver diagram to display constructs intended to influence the achievement of a desired benchmark. We created a driver diagram to show alignment and nonalignment between outcomes identified in our review and MIECHV constructs, on the basis of the theory of the constructs as the direct drivers to accomplish improvement in Benchmark 1 (Figure 2).

Figure 2.

Hypothetical driver diagram for Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Benchmark 1: Maternal and Newborn Health. A driver diagram is a visual tool used as a framework for measuring and monitoring progress on a goal or objective.¹⁶ The outcome is Benchmark 1: Maternal and Newborn Health as the area that the intervention is trying to improve. The drivers are the constructs (ie, subcategories of the benchmark) that will accomplish the improvement. The interventions are the home visiting program models (eg, Early Head Start, Parents as Teachers) funded by MIECHV and evaluated by the Home Visiting Evidence of Effectiveness (HomVEE) project. The HomVEE project systematically gathers and transparently assesses the quality of the evidence from the home visiting literature.

Because this study did not meet the definition of human subjects research, institutional review board approval was not required.

Results

We identified 14 articles (23.7%), of the 59 articles we reviewed, that measured outcomes aligned with Benchmark 1 constructs (Table 2). None of the articles directly measured depression screening, postpartum care, or tobacco cessation referrals (Figure 3). The literature indicated that home visiting had a positive effect on preterm birth and mixed results (positive effect or no effect) on breastfeeding and well-child visits.

Table 2.

Articles in the Home Visiting Evidence of Effectiveness database (n = 14) that assessed Maternal, Infant, and Early Childhood Home Visiting constructs for Benchmark 1: Maternal and Newborn Health, United States, 2016^a

Construct and Type of Effect^b	Article	Home Visiting Program Model
Preterm birth
No effect	Kitzman et al (1997)¹⁷	Nurse Family Partnership
Positive effect	Koniak-Griffin et al (2000),¹⁸ Koniak-Griffin et al (1999),¹⁹ Nguyen et al (2003),²⁰ Olds et al (1986)²¹	Early Intervention Program for Adolescent Mothers
		Early Intervention Program for Adolescent Mothers
		Nurse Family Partnership
		Nurse Family Partnership
Breastfeeding
No effect	Guyer et al (2003),²² Minkovitz et al (2001)²³	Healthy Steps
No effect	Guyer et al (2003),²² Minkovitz et al (2001)²³	Healthy Steps
Positive effect	Johnston et al (2006),²⁴ Johnston et al (2004),²⁵ Mitchell-Herzfeld et al (2005)²⁶	Healthy Steps
		Healthy Steps
		Healthy Families
Well-child visits
No effect	Caldera et al (2007),¹⁵ Duggan et al (1999),²⁷ Wagner et al (2002)²⁸	Healthy Families
		Healthy Start
		Parents as Teachers
Positive effect	Fergusson et al (2005),¹⁴ Johnston et al (2004)²⁵	Early Head Start
Positive effect	Fergusson et al (2005),¹⁴ Johnston et al (2004)²⁵	Healthy Steps

^aThe constructs are subcategories under Benchmark 1: Maternal and Newborn Health. Data source: James Bell Associates, Inc.¹¹

^bNo articles were found for 3 of the 6 constructs: depression screening, postpartum care, and tobacco cessation referrals.

Figure 3.

Driver diagram of Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Benchmark 1: Maternal and Newborn Health based on a review of articles on home visiting programs in the Home Visiting Evidence of Effectiveness (HomVEE) database (n = 14), United States, 2016. Data sources: HomVEE study search tool,¹³ Fergusson et al,¹⁴ Hubley,²⁹ and Kitzman et al.¹⁷ A driver diagram is a visual tool used as a framework for measuring and monitoring progress on a goal or objective.¹⁶ The outcome is Benchmark 1: Maternal and Newborn Health as the area that the intervention is trying to improve. The drivers are the constructs (ie, subcategories of the benchmark) that will accomplish the improvement. The interventions are the home visiting program models (eg, Early Head Start, Parents as Teachers) funded by MIECHV and evaluated by HomVEE. In the diagram, squares depict the MIECHV constructs and ovals depict the “other” identified constructs (ie, those not identified by MIECHV). The number of times that a tested association of the drivers (MIECHV constructs or other) with the intervention (home visiting) was identified through the review is reflected numerically within each driver in the diagram. Data source: James Bell Associates, Inc.¹¹

The articles demonstrated robust evidence (ie, 72 positive associations) that home visiting affects 16 other outcomes related to maternal–child health (Figure 3), including positive effects on subsequent births and birth spacing, maternal symptoms of depression and psychological outcomes, and child emergency department visits and hospitalizations (11 positive associations each). Although few articles mentioned the use of community health resources (n = 6 articles), birth weight (n = 5 articles), and child externalizing symptoms (n = 5 articles), findings still offered more evidence of the effectiveness of home visiting programs on these outcomes than on the MIECHV constructs.

Discussion

Jacobson and Neumann³⁰ highlighted the imbalance between the value of public health services, the funding for public health services, and the need to continuously quantify the contribution of these services to population health to maximize resources. The authors recognized that “scientific effectiveness (ie, evidence-based public health) should guide program decisions.”³⁰ Others also recognized the need for evidence-based performance measures for the quality improvement process of medical services.^31,32 Baker and Qaseem³³ addressed common unintended consequences of non–evidence-based performance measures, especially relating to changing health providers’ practices, reinforcing that “it is essential that national performance measures be based on high-quality scientific evidence.” The possibility of unintended consequences reinforces the importance of establishing standards for vetting performance measures.

Our review did not demonstrate a strong scientific evidence base for the revision to the MIECHV constructs: only 3 of the 6 constructs for Benchmark 1 had supporting evidence in the HomVEE database. Because half of the constructs (depression screening, postpartum care, and tobacco cessation referrals) did not have evidence of effectiveness, one would be challenged to attribute changes to the impact of the home visiting services funded and evaluated by MIECHV. We support and recommend the use of evidence-based constructs, particularly for nationally implemented programs.

Although HomVEE is currently the national resource for vetted evidence on the effectiveness of home visiting programs, only a small number of articles provided evidence to support a set of constructs for this large, publicly funded program (MIECHV). The MIECHV construct of preterm births had the most articles (n = 4) supporting it as an outcome, yet other outcomes related to maternal–child health had a stronger evidence base and were supported by up to 11 articles. Some of the evidence-based outcomes, such as subsequent births and birth spacing, were included as MIECHV constructs until they were removed in 2016. One construct that did not show any evidence of support in our review—postpartum care—was a recent addition to MIECHV.

We speculate that numerous potential relationships exist among the other outcomes and with the Benchmark 1 constructs. For example, hypothetically, the increase in use of community resources promoted by a home visiting program model could also positively affect access to postpartum care (MIECHV construct 5). Similarly, by maintaining a primary care provider, children served by home visiting program models might also have more well-child visits (MIECHV construct 4). On the other hand, a decrease in tobacco use could be a consequence of tobacco cessation referrals (MIECHV construct 6). Future work should examine these constructs from a systems approach to better understand these interrelationships and how the improvement in one might have a multiplicative effect on others.

The Health Resources and Services Administration’s Division of Home Visiting and Early Childhood Systems intended to reduce the reporting burden for grantees and increase the use of performance measures with its yearlong redesign of MIECHV constructs, which included collecting public feedback. However, it is unclear what criteria were used to determine which constructs to eliminate or incorporate. Our study indicates that criteria other than scientific evidence may have been prioritized. The Centers for Disease Control and Prevention³⁴ suggested that criteria such as the relevance of the outcomes (the association with the program goal), availability of resources, time to measure progress, and data collection capacity should be considered when identifying which performance measures to use to monitor program effectiveness. These considerations may have influenced the MIECHV revision.

We acknowledge the difficulty in establishing the appropriate performance measures for a nationally funded program. Federal agencies have the added challenge of attempting to align MIECHV goals with individual home visiting program goals, which may or may not always be consistent. This difficulty further underscores the importance of transparency in these decision-making practices.

Using Driver Diagrams to Strengthen Program Evaluation

Driver diagrams display a theory-of-change strategy (eg, “home visitation will improve maternal–child health”). The Medicaid Innovation Accelerator Program recommends driver diagrams as a tool to help frame performance improvement, suggesting that programs use driver diagrams to “identify measures to monitor progress.”³⁵ We used a driver diagram in our study to display alignment and lack of alignment between scientific evidence and MIECHV program constructs. MIECHV and similar programs should consider using driver diagrams not only for planning purposes but also to track and review performance and evaluation strategies.

At their core, performance measures are a strategy for continuous evaluation of programs.³⁴ The use of driver diagrams to evaluate the evidence base of a program’s performance measures can play an important role in programs’ continuous quality improvement efforts. Identifying the most evidence-based drivers will increase the likelihood of associating improvement of a construct (eg, breastfeeding) with the intervention (eg, home visits).

Challenges exist in reaching a common understanding of best practices for performance measures among public health programs.²⁹ The use of driver diagrams to assess evidence of performance measures could be a particularly useful strategy for program operations as programs mature. By visualizing the complexity of the associations, the drivers enable a focused discussion about areas of interest for decision makers.

Limitations

This review had several limitations. Although HomVEE is the federal body investigating evidence for home visiting program models, its website is limited by its inclusion and selection criteria for studies and the pace at which it reviews and posts articles with respect to the articles’ publication date (ie, a lag between the publication date and incorporation by HomVEE). Finally, articles that could be relevant to maternal–child health benchmarks but were not identified by HomVEE as “child health” or “maternal health” could have been excluded from our sample.

Conclusions

Federal programs use performance measures to inform decisions about program funding. We found that only 14 of 59 articles identified from the HomVEE database supported MIECHV constructs for Benchmark 1, and only 3 of the 6 Benchmark 1 constructs had supporting literature in the HomVEE database. We reviewed only 1 of the MIECHV benchmarks, given the need to improve national maternal–child health outcomes. Future work should apply this methodology to other benchmarks.

Currently, there is no consensus for the best strategies in choosing performance measures. Newly established reporting requirements for the MIECHV program relating to maternal–child health were in large part not supported by the available literature. A driver diagram is an innovative use of that tool to illustrate the lack of support. The need to clarify what other criteria are used to develop construct revisions, beyond evidence, is urgent.

We have recommendations for evaluators and program planners that could strengthen public health performance. First, we support implementing evidence-based practices for decision making when choosing reporting requirements for grantees. Through a “validity check” (ie, is the association between the intervention and outcome supported by evidence?), the planning process is driven by a common understanding and increases the likelihood of measuring program impact. Second, driver diagrams are a useful tool to support this shared understanding and decision making. Third, it is unclear what critical strategies and frameworks federal agencies use to select MIECHV constructs and associated performance measures (ie, which ones have a greater influence on final decisions and which strategies are replicable). Finally, the body of evidence supporting the effectiveness of home visiting program models is deficient.

The continuous increase in funding for MIECHV and similar programs and the federal government’s interest in assessing various home visiting program models (HomVEE) should stimulate the interest of scholars in replicating our methods for the various benchmarks. States might also prioritize evaluation strategies that explore performance measures when granted expansion and special funds. Without evidence-based performance measures, there is less confidence that the program will drive important outcomes, and it would not be possible to attribute changes in outcomes to the impact of home visiting services.

Footnotes

Acknowledgments

The authors are grateful to the Missouri Department of Health and Senior Services’ MIECHV program leadership and project personnel for their diligent efforts in implementing this program. The information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, the Health Resources and Services Administration, the US Department of Health and Human Services, or any other agency of the US government.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project is supported by the Health Resources and Services Administration of the US Department of Health and Human Services under Grant No. D89MC2791501-Affordable Care Act Maternal, Infant, and Early Childhood Home Visiting program in the amount of $379 024 with 0% financed with nongovernmental sources.

ORCID iD

Julie M. Kapp, PhD, MPH

References

MacDorman

Mathews

Mohangoo

Zeitlin

. International comparisons of infant mortality and related factors: United States and Europe, 2010. Natl Vital Stat Rep. 2014;63(5):1–7. https://www.cdc.gov/nchs/data/nvsr/nvsr63/nvsr63_05.pdf. Accessed December 28, 2017.

Save the Children. The Urban Disadvantage: State of the World’s Mothers 2015. Fairfield, CT: Save the Children; 2015. http://www.savethechildren.org/atf/cf/%7B9def2ebe-10ae-432c-9bd0-df91d2eba74a%7D/SOWM_2015.PDF. Accessed December 28, 2017.

Hamilton

Martin

Osterman

Curtin

Mathews

. Births: final data for 2014. Natl Vital Stat Rep. 2015;64(12):1–64.

Health Resources and Services Administration. Programs and initiatives: home visiting. https://mchb.hrsa.gov/maternal-child-health-initiatives/home-visiting-overview. Published 2017. Accessed December 28, 2017.

Schonberg

Anderson

Bays

. The role of home-visitation programs in improving health outcomes for children and families. Pediatrics. 1998;101(3):486–489.

Avellar

Supplee

. Effectiveness of home visiting in improving child health and reducing child maltreatment. Pediatrics. 2013;132(suppl 2):S90–S99.

Peacock

Konrad

Watson

Nickel

Muhajarine

. Effectiveness of home visiting programs on child outcomes: a systematic review. BMC Public Health. 2013;13:17.

National Commission to Prevent Infant Mortality. Home Visiting: Opening Doors for America’s Pregnant Women and Children. Washington, DC: National Commission to Prevent Infant Mortality; 1989.

US Department of Health and Human Services, Administration for Children and Families. Home Visiting Evidence of Effectiveness: home visiting program model effects. https://homvee.acf.hhs.gov/evidenceoverview.aspx. Accessed December 20, 2017.

10.

US Department of Health and Human Services, Administration for Children and Families. Home Visiting Evidence of Effectiveness: about us—project overview. https://homvee.acf.hhs.gov/About-Us/5/Project-Overview/20/1. Accessed May 29, 2017.

11.

James Bell Associates, Inc. The Maternal, Infant, and Early Childhood Home Visiting program: form 2 performance indicators and systems outcomes data collection & reporting manual and grantee plan. https://mchb.hrsa.gov/sites/default/files/mchb/MaternalChildHealthInitiatives/HomeVisiting/performanceresources/performancemeasurementtoolkit.pdf. Published August 2016. Updated October 2016. Accessed September 11, 2017.

12.

Lichiello

Turnock

. The Turning Point Guidebook for Performance Measurement. Seattle, WA: Turning Point; 2003.

13.

US Department of Health and Human Services, Administration for Children and Families. Home Visiting Evidence of Effectiveness: study search. https://homvee.acf.hhs.gov/Studies/Study-Search/19. Accessed May 29, 2017.

14.

Fergusson

Horwood

Ridder

Grant

. Early Start Evaluation Report. Christchurch, New Zealand: Early Start Project Ltd; 2005.

15.

Caldera

Burrell

Rodriguez

Crowne

Rohde

Duggan

. Impact of a statewide home visiting program on parenting and on child health and development. Child Abuse Negl. 2007;31(8):829–852.

16.

Svoronos

Mate

. Evaluating large-scale health programmes at a district level in resource-limited countries. Bull World Health Organ. 2011;89(11):831–837.

17.

Kitzman

Olds

Henderson

Jr . Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing. A randomized controlled trial. JAMA. 1997;278(8):644–652.

18.

Koniak-Griffin

Anderson

Verzemnieks

Brecht

. A public health nursing early intervention program for adolescent mothers: outcomes from pregnancy through 6 weeks postpartum. Nurs Res. 2000;49(3):130–138.

19.

Koniak-Griffin

Mathenge

Anderson

Verzemnieks

. An early intervention program for adolescent mothers: a nursing demonstration project. J Obstet Gynecol Neonatal Nurs. 1999;28:51–59.

20.

Nguyen

Carson

Parris

Place

. A comparison pilot study of public health field nursing home visitation program interventions for pregnant Hispanic adolescents. Public Health Nurs. 2003;20(5):412–418.

21.

Olds

Henderson

Jr Chamberlin

Tatelbaum

. Preventing child abuse and neglect: a randomized trial of nurse home visitation. Pediatrics. 1986;78(1):65–78.

22.

Guyer

Barth

Bishai

. Healthy Steps: The First Three Years. Baltimore, MD: Women’s and Children’s Health Policy Center, Department of Population and Family Health Sciences, Johns Hopkins Bloomberg School of Public Health; 2003. http://www.jhsph.edu/research/centers-and-institutes/womens-and-childrens-health-policy-center/publications/Healthy_Steps_Final_Reports/FR_COVCONT_030103FIN.pdf. Accessed May 29, 2017.

23.

Minkovitz

Strobino

Hughart

Scharfstein

Guyer

; Healthy Steps Evaluation Team. Early effects of the Healthy Steps for Young Children program. Arch Pediatr Adolesc Med. 2001;155(4):470–479.

24.

Johnston

Huebner

Anderson

Tyll

Thompson

. Healthy steps in an integrated delivery system: child and parent outcomes at 30 months. Arch Pediatr Adolesc Med. 2006;160(8):793–800.

25.

Johnston

Huebner

Tyll

Barlow

Thompson

. Expanding developmental and behavioral services for newborns in primary care: effects on parental well-being, practice, and satisfaction. Am J Prev Med. 2004;26(4):356–366.

26.

Mitchell-Herzfeld

Izzo

Greene

Lee

Lowenfels

. Evaluation of Healthy Families New York (HFNY): First Year Program Impacts. Albany, NY: Center for Human Services Research, University at Albany; 2005. http://www.ocfs.state.ny.us/main/prevention/assets/HFNY_FirstYearProgramImpacts.pdf. Accessed May 29, 2017.

27.

Duggan

McFarlane

Windham

. Evaluation of Hawaii’s Healthy Start Program. Future Child. 1999;9(1):66–90; discussion 177-178.

28.

Wagner

Spiker

Linn

. The effectiveness of the Parents as Teachers program with low-income parents and children. Top Early Child Spec Educ. 2002;22(2):67–81.

29.

Hubley

. Lessons from a project to create performance measures for public health. Eval Program Plann. 2008;31(4):410–415.

30.

Jacobson

Neumann

. A framework to measure the value of public health services. Health Serv Res. 2009;44(5 pt 2):1880–1896.

31.

Myers

Slovis

Eckstein

. Evidence-based performance measures for emergency medical services systems: a model for expanded EMS benchmarking. Prehosp Emerg Care. 2008;12(2):141–151.

32.

Stuart

Strite

. Evidence-Based Performance Measurement: Validity Issues & Avoiding Important Pitfalls. Seattle, WA: Delfini Group, LLC; 2006. http://www.delfini.org/Delfini_WhitePaper_Performance%20measures_Short.pdf. Accessed December 28, 2017.

33.

Baker

Qaseem

; American College of Physicians’ Performance Measurement Committee. Evidence-based performance measures: preventing unintended consequences of quality measurement. Ann Intern Med. 2011;155(9):638–640.

34.

Centers for Disease Control and Prevention. Introduction to program evaluation for public health programs: a self-study guide. https://www.cdc.gov/eval/guide/introduction. Published 2012. Accessed May 29, 2017.

35.

Llanos

Griffith

Griffin

. Commentary: performance improvement. https://www.medicaid.gov/state-resource-center/innovation-accelerator-program/iap-commentary/2016-iap-blog-entries/36356.html. Published 2016. Accessed May 29, 2017.

The Evidence Base for the Maternal,Infant,and Early Childhood Home Visiting Program Constructs

Abstract

Objectives:

Methods:

Results:

Conclusions:

Keywords

Program Description

Purpose

Methods

Results

Discussion

Using Driver Diagrams to Strengthen Program Evaluation

Limitations

Conclusions

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

ORCID iD

References