Abstract
The present study is one of the largest quasi-experimental studies to date on the effects of home visiting on documented child maltreatment during a child’s first two years of life. In this matched comparison group study, we compare 8796 families that participated in a home visiting program (HV families) to 8796 similar non-participating families (non-HV families) selected from birth records using Coarsened Exact Matching. Using sequential logistic regression, we identify that HV families have significantly higher odds of experiencing a child maltreatment investigation by their child’s second birthday compared to non-HV families; however, among those that were investigated, HV families have significantly lower odds of having their first investigation substantiated for maltreatment. Overall, HV families do not differ significantly from non-HV families in the odds of experiencing a substantiated investigation over 2 years. We share implications for considering surveillance bias, and we highlight the importance of including both substantiated and unsubstantiated investigations when studying the effects of home visiting on documented child maltreatment.
Evidenced-based home visiting programs served more than 277,000 US families in 2022 (National Home Visiting Resource Center, 2022 [NHVRC]). In the United States, the federal Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program funds evidence-based home visiting programs, aiming in part to prevent child maltreatment. Yet, only six of the 23 MIECHV-eligible home visiting programs have strong evidence of positive effects on maltreatment reduction according to the US Department of Health and Human Services (HHS) Home Visiting Evidence of Effectiveness (HomVEE, 2023). Some studies do not explicitly examine involvement with the child welfare system, and studies that do typically only measure substantiated reports. Examining both unsubstantiated and substantiated investigations represents an important, holistic look at child welfare involvement (Kugler et al., 2019).
To address the gaps in the existing evidence base, we use a quasi-experimental approach to examine differences in the rates of child welfare involvement (investigation and substantiated investigation) between a large sample of families that participated in home visiting (HV families) over nearly a decade and a rigorously matched group of comparison families (non-HV families). The study addresses the following research questions: (1) Are HV families less likely than non-HV families to experience a child maltreatment investigation in the 2 years following the child’s birth?; (2) Among families that are investigated for child maltreatment within 2 years of the child’s birth, are HV families less likely than non-HV families to have their first investigation substantiated?; and (3) Among all families, are HV visiting families less likely than non-HV families to experience a substantiated investigation of child maltreatment in the 2 years following the child’s birth?
Child Maltreatment and Home Visiting
From October 2020 to September 2021, 3 million children in the United States experienced an investigation for suspected abuse or neglect, and an estimated 600,000 children were substantiated or indicated victims of maltreatment (HHS et al., 2023). Children who have experienced maltreatment are at greater risk for pathology in multiple domains; they are more likely to experience internalizing and externalizing behaviors, difficulties in school, and serious psychopathology, among other outcomes (Casanueva et al., 2012; Toth & Manly, 2019). In fact, children who become involved in the child welfare system often experience negative outcomes even if their investigations are unsubstantiated (Casanueva et al., 2012; Hussey et al., 2005; Kugler et al., 2019).
Home Visiting as a Preventative Intervention
The two most widely used and evaluated home visiting programs in the United States that target child maltreatment among other outcomes are Healthy Families America (HFA) and Nurse Family Partnership (NFP), serving 66,883 and 53,045 US families respectively in 2021 (National Home Visiting Resource Center, 2022). NFP targets first-time mothers with low incomes and their children, beginning prenatally, and serving families until the child turns 2 years old (Olds et al., 1994). HFA begins either prenatally or within the first few months of life and serves families for 3–5 years (DuMont et al., 2008; HomVEE, 2020; National Home Visiting Resource Center, 2022). HFA targets families experiencing multiple challenges, including but not limited to low income, adverse childhood experiences, or parental substance abuse, though individual sites may select specific target populations to serve (HomVEE, 2020). These programs aim to improve prenatal, maternal, and child health, encourage positive child development, improve economic self-sufficiency, and prevent child maltreatment through parent education, increasing informal support systems, and linking families with service providers, among other activities (DuMont et al., 2008; Olds et al., 1994).
Home visiting programs have the potential to reduce child maltreatment risk via several pathways. Olds et al. (1986) theorized that parent education would lead to improved caregiver sensitivity and responsiveness, which would result in a secure attachment relationship between caregiver and child and reduced maltreatment risk. Parent education could also increase self-efficacy among participating parents, helping parents to feel equipped to use the parenting strategies taught by the program (Bilukha et al., 2005). Other researchers have theorized that improved economic self-sufficiency for participants, via continued education and workforce involvement, increases families’ access to financial resources, decreasing stress for parents and reducing maltreatment risks (Bilukha et al., 2005). We further theorize that home visiting families may gain access to needed resources through referrals to other community supports and experience reduced stress through the strengthening of their informal support system of family and friends.
Surveillance Bias
Although there is strong theoretical support for how home visiting may reduce child maltreatment risk, home visiting may also increase the detection of possible maltreatment, resulting in more maltreatment reports (Chaffin & Bard, 2006; Drake et al., 2017; Zielinski et al., 2009). Chaffin and Bard (2006, p. 301) defined surveillance bias as “any increased, systematic, outcome-related scrutiny that may exist for some individuals or groups but not others.” Within a home visiting framework, participants could undergo increased surveillance for child maltreatment as a direct or indirect result of participation (Chaffin & Bard, 2006).
First, home visiting families may experience surveillance directly through home visitors, who are mandatory reporters and may witness and report signs of maltreatment (Chaffin & Bard, 2006). Indirectly, home visitors often refer families to additional services as part of the home visiting program, which may in turn increase surveillance in families’ lives (Chaffin & Bard, 2006). Finally, families enrolled in home visiting may already be more likely to be involved in other social services programs that increase surveillance.
Research has indicated that surveillance bias effects are likely real, particularly during service involvement, contributing to up to 25% of maltreatment reports (Holland et al., 2024); however, the effects are small enough that they cannot fully explain a lack of positive effects for home visiting’s effect on child maltreatment reports (Chaffin & Bard, 2006; Drake et al., 2017). In these studies, surveillance reports from service providers, such as home visitors, were often not unique, which means that families with surveillance reports also often had reports from other sources and would eventually be investigated (Chaffin & Bard, 2006; Drake et al., 2017).
Unsubstantiated Child Welfare Investigations
One mechanism to explore the presence and effects of surveillance bias would be to measure unsubstantiated investigations. However, most of the existing studies on home visiting and child welfare involvement have included only substantiated investigations and excluded unsubstantiated investigations. Investigations are substantiated when there is enough evidence to conclude that abuse or neglect occurred. A common assumption is that unsubstantiated reports reflect false reports, overreporting, and/or low-risk instances that require no CPS involvement (Drake, 1996). In fact, the HomVEE (2023) review does not include unsubstantiated report outcomes, citing concerns of overreporting and surveillance bias. Although these valid concerns require careful attention by researchers and practitioners, unsubstantiated reports remain an important consideration in the study of child welfare involvement and program effectiveness.
Drake (1996) conceptualized child welfare cases along a two-dimensional continuum involving harm and evidence. Drake (1996) described substantiated reports as involving both strong evidence and actual or potential harm, whereas unsubstantiated reports can involve moderate to substantial harm with insufficient evidence, strong evidence with insufficient harm, or insufficient harm and evidence. False reports make up only a small portion of unsubstantiated investigations: investigations lacking harm and evidence (Drake, 1996). In some cases, intentionally false reports are believed to make up less than 0.3% of official dispositions (HHS et al., 2023). Even in unsubstantiated cases, harm may have occurred, risk may be present, and families may be referred to services.
Unsubstantiated investigations likely include at least some level of negative experience for families. Parents who have experienced child welfare investigations have both expressed appreciation for services resulting from investigations but also suggested that the investigation itself incited fear, stigma, distrust of the reporters, and likelihood of disengaging from service providers (Fong, 2020; Merritt, 2020). These negative experiences can be particularly salient for individuals from minoritized groups (Fong, 2020; Merritt, 2020), such as Black families, who are often overrepresented in the child welfare system (HHS et al., 2023).
In summary, unsubstantiated reports should not be automatically interpreted as unfounded maltreatment reports; rather, the existing evidence suggests that the risks and outcomes of substantiated and unsubstantiated investigations are similar, and examining unsubstantiated investigations is vital to understanding the full context of child welfare involvement (Drake et al., 2003; Hussey et al., 2005; Kohl et al., 2009; Kugler et al., 2019). However, with the understanding that a child welfare investigation can bring disruption to a family and incite fear, anxiety, and stress, we should not ignore the potential negative effects of unsubstantiated investigations.
Existing Evidence on Home Visiting and Child Maltreatment Prevention
Over several decades, researchers have evaluated many home visiting program models to assess programs’ effects on maltreatment. Most meta-analytic studies of home visiting and child maltreatment have included multiple home visiting programs and combined various measures of child maltreatment, including confirmed maltreatment, medical encounters potentially resulting from abuse or neglect, and self-reported maltreatment. These meta-analyses have found small or null effects of home visiting on maltreatment (Filene et al., 2013; Gubbels et al., 2021; MacLeod & Nelson, 2000; Sweet & Appelbaum, 2004). Some authors cited surveillance bias as a possible reason for these limited effects of home visiting (Filene et al., 2013; MacLeod & Nelson, 2000). Unsurprisingly, programs designed to prevent child maltreatment were more effective at preventing child maltreatment than programs that did not target the outcome (Sweet & Appelbaum, 2004).
A substantial portion of the literature on home visiting and child maltreatment assesses the effects of HFA and NFP, and the evidence from these programs of reducing maltreatment is limited. HomVEE’s (2023) review of the evidence from these two programs includes 27 favorable, 207 null, and one unfavorable or ambiguous effect on child maltreatment outcomes.
Researchers often measure medical encounters that may have resulted from abuse or neglect as a proxy for child maltreatment. In early NFP Randomized Controlled Trials (RCTs), NFP children had fewer medical encounters compared to control children during their first 2 years (Kitzman et al., 1997; Olds et al., 1986). In contrast, a recent quasi-experimental study from the United States found that NFP participant children actually had greater odds of experiencing any abuse-related injuries than comparison children during the first 2 years of life (Matone et al., 2018). HFA has had no documented effects on medical encounters in similar age ranges (Caldera et al., 2007; Duggan et al., 1999, 2007; Mitchell-Herzfeld et al., 2005).
Although HFA has no documented effects on medical proxies for abuse and neglect, the program has shown more promising effects on self-reported maltreatment. The vast majority of the maltreatment outcomes assessed for HFA were of self-reported maltreatment behaviors (Caldera et al., 2007; Duggan et al., 2004, 2007; DuMont et al., 2008, 2010). Although most effects on the presence or absence of these activities were null, researchers have found decreases in the frequency of various self-reported maltreatment behaviors (Caldera et al., 2007; Duggan et al., 2004, 2007; DuMont et al., 2008, 2010). The evidence is strongest for HFA’s impact on self-reported mild physical aggression, psychological aggression, and serious abuse, as HFA participants self-reported using these behaviors less frequently than control families (Duggan et al., 2007; DuMont et al., 2008, 2010).
Evidence for the impact of home visiting programs, such as NFP and HFA, on maltreatment as measured by child welfare involvement is also limited. The earliest NFP RCT found no differences in child welfare involvement for the full sample or most subsamples; however, NFP children born to teen mothers and unmarried teen mothers living in poverty experienced fewer substantiated reports of abuse or neglect than control children over a span of 2 years (p = .07; Olds et al., 1986). Studies of a Dutch adaptation of NFP (Mejdoubi et al., 2015) and an adaptation with an Aboriginal Australian sample (Segal et al., 2018) detected that participating families that met typical NFP eligibility criteria had significantly less child welfare involvement than comparison families.
Peer-reviewed published journal articles contain no claims of significant positive effects on substantiated maltreatment reports for full HFA samples (Chambliss, 1998; Duggan et al., 2007; DuMont et al., 2008; Easterbrooks et al., 2012; Green et al., 2017; Jacobs et al., 2016). Furthermore, within a Treatment-on-the-Treated subsample, HFA participants in Oregon were significantly more likely than control families to have an unsubstantiated investigation during the first 2 years after enrollment, despite having no differences in likelihood of substantiated investigations during the same timeframe (Green et al., 2017). The authors state that “these markedly different patterns for substantiated and unsubstantiated reports also suggest the critical importance of examining both types of maltreatment reporting in prevention research” (Green et al., 2017, p. 84).
The authors suspect that surveillance bias may explain the greater number of unsubstantiated investigations for Oregon HFA families and the lack of positive effects for initial and reoccurrence of maltreatment among HFA participants in New York (DuMont et al., 2008; Green et al., 2017; Lee et al., 2018). Green et al. (2017) further found that most substantiated reports occurred after families left the program, but approximately half of unsubstantiated reports occurred during program participation. Among New York mothers who self-reported serious maltreatment, HFA mothers were more likely to have had child welfare investigations than control mothers, suggesting that HFA participants may have been more likely to be reported by home visiting staff or other HFA-referred service providers (DuMont et al., 2008; Lee et al., 2018).
The evidence for NFP reducing child welfare involvement is weaker for the early years of a child’s life, during families’ active participation in the program, but positive impacts from the program are stronger as the children age (Olds et al., 1997; Zielinski et al., 2009). The same pattern appears to be true for HFA. Across a few samples, documented positive effects of HFA on child welfare involvement arose later in childhood within subsamples and specific types of maltreatment. HFA mothers in New York were significantly less likely to be confirmed as a perpetrator of sexual abuse within 7 years after enrollment (DuMont et al., 2010), and HFA families in New York and Massachusetts had significantly less reoccurrence of maltreatment reports over 7 years, with differences emerging at approximately age 3 (Easterbrooks et al., 2019; Lee et al., 2018).
In all, the existing research on home visiting and child welfare involvement suggests that home visiting may have limited effects on child maltreatment in the first years of life as measured by substantiated child welfare investigations, but the impact of home visiting may become more prominent after families have left the home visiting program. Surveillance bias may be active and responsible for small or null effects on substantiated maltreatment and some findings of greater unsubstantiated investigations amongst home visiting participants. However, the presence of a surveillance effect should not automatically be interpreted as negative. In fact, surveillance bias may be at least partially responsible for increases in effects over time because investigations could help families get services and supports they need to ameliorate risk, even with unsubstantiated investigations.
Present Study
The present study compares child welfare involvement, including all investigated and substantiated reports of child maltreatment, between a large sample of families that participated in one voluntary home visiting program in a single state and a matched comparison group of non-participating families. Although there is a need to understand which home visiting programs are effective at preventing maltreatment, this study was part of a much larger evaluation designed to examine the full context of the state’s services to families through home visiting and was not intended to be an evaluation of any specific program model. We only included one of the state’s home visiting models in the child maltreatment portion of the evaluation, and the study did not examine program model fidelity or dosage. We only included one program model because only one of the state’s program models systematically begins serving families prenatally. Focusing on families served prenatally provides a comparable timeframe (birth to 2 years) to observe all treatment and comparison families. Given these contextual factors of the study, this article does not identify the specific home visiting program. The study is the largest known study to date to compare suspected and confirmed reports of child maltreatment between families participating in home visiting and a comparison group. 1
Home visiting programs that enroll families at or before the birth of a first child are particularly well-suited for the quasi-experimental study of child welfare involvement. First, if the program serves first-time mothers, 2 there is limited concern for the confounding effects of family size, in that more children could increase the likelihood that a family would become involved in the child welfare system (Putnam-Hornstein & Needell, 2011). Second, when mothers enroll at or before birth, researchers can easily set a comparable observation period starting at birth for both home visiting and non-home visiting families, which eliminates child’s age as another potential confounder.
Finally, prenatal or at-birth enrollment of first-time mothers also facilitates the study of home visiting as a primary prevention intervention. We previously found that child welfare involvement prior to beginning home visiting is the strongest predictor of child welfare involvement after beginning home visiting for participating families across multiple home visiting models (Osborne et al., 2021). As first-time mothers, caregivers are less likely to have previously experienced a child maltreatment investigation as a perpetrator; therefore, we can study home visiting as a primary prevention strategy using almost the entire population of participating families. The home visiting program in the present study enrolls first-time mothers prenatally and provides regular home visits by a trained home visitor through the child’s second birthday.
We use Coarsened Exact Matching (CEM) to conduct a rigorous quasi-experimental study including 8,786 HV-participating families and 8,786 non-HV families from nearly a decade of births, tracking each family’s child welfare involvement for 2 years. This study additionally contributes to the literature by including both unsubstantiated and substantiated investigations of abuse and neglect, highlighting the practical implications of looking beyond just substantiated cases of maltreatment.
Methods
Population
We began with the full population of families that participated in a specific home visiting program in a large state, had their first home visit between September 1, 2009 and December 31, 2019, and had complete name and date of birth information for the participating mother and/or child (N = 15,754).
Data Sources
This study relied on three separate data sources. First, we used names, dates of birth, demographic characteristic, and location information from the home visiting program administrative data to identify HV families in the state birth records data. Second, we used vital statistics birth record data from the state health department for all births that occurred in the state from 2009 to 2019 to identify home visiting families, to select a comparison sample of non-participating families, and to gather covariate variables included in analysis. Third, we used data provided by the state’s child welfare system as outcomes for this study, including investigations extending from January 1, 2008 through March 1, 2020, and prior investigations for individuals involved in cases from that timeframe. In the final sample, we excluded families whose children were born after March 1, 2018 to be able to follow them for 2 years, and we intentionally did not include any data after March 1, 2020 to avoid data impacted by the COVID-19 pandemic.
Linking Program Administrative Data to Birth Records
Beginning with the entire program participant population (N = 15,754), we searched for participating families in the state birth records with guardian’s and child’s name and date of birth, using exact and fuzzy matching techniques (See Supplemental Materials for more details). We identified 13,558 of 15,754 (86%) home visiting families within the birth records.
Sampling
After linking the 13,558 families from home visiting administrative data to the birth record data, we excluded 4,178 HV families from our study: 201 for multiple births (e.g., twins or triplets), 66 for miscarriage/death, 555 for being enrolled in two or more state-offered home visiting programs, 119 for having given birth previously, 27 for congenital anomalies, 21 that were missing variables necessary for matching (missing mother’s education, ZIP code, or WIC use), 2 because the administrative data showed the family enrolling in home visiting after birth, and 3,187 HV families because the HV child’s birthdate was on or after March 1, 2018, which was the latest date for which we could observe children for 2 years after birth within the child welfare data. After exclusions, we had 9,380 HV families eligible for matching. We took an intent-to-treat approach, including all families with at least one home visit in the HV group.
Prior to matching, we imputed race or ethnicity for mothers in the birth records who were missing the variable using the wru R package (Imai & Khanna, 2016). This package uses a Bayesian approach to combining the Census Bureau’s Surname List with geolocation data (county) to predict individuals’ race or ethnicity. Overall, 5.6% of the final analytic sample (5.2% of non-HV, 6.1% of HV mothers) had an imputed race or ethnicity. We conducted a sensitivity check by rerunning the analyses after excluding families and their matches with imputed race, and the results were consistent with the results presented in this paper.
Next, we used CEM to construct a one-to-one comparison group of non-HV families that was as similar to the HV group as possible. CEM groups participants within strata, which are groups of participants with identical characteristics on matching variables (King et al., 2011). CEM removes strata that contain only treated or only control participants from the dataset. Researchers manually or automatically adjust the categories created from existing categorical and continuous variables to retain as large of a sample as possible while maintaining a good covariate balance.
The final selected sample was matched on the following variables: mother’s race/ethnicity, age, education, marital status, Medicaid/CHIP payment for birth, WIC use for self during pregnancy, child’s birth year group, percent poverty terciles, and county of residence at birth (See Supplemental Materials). The matching process generated matches for 9210 (98.2%) of the 9380 eligible HV families. During the matching process, we assigned each family a random number and sorted within each strata and group to create a match indicator. This assigned each HV family a specific non-HV family match for use in further exclusions. The unmatched HV mothers were more likely than matched HV mothers to be married, more highly educated, older, non-Hispanic, and born outside the US.
Sample Characteristics by HV Participation.
Note. N = 17,592 (n = 8796 for each group). The values displayed for binary variables indicate the percentage with a “yes” for that variable category. There are no statistically significant differences between the HV and Non-HV groups on the measured variables.
aWe did not match on this variable.
bFathers married to the mother at the time of birth are automatically listed on the birth certificate in this state.
Measures
Our dependent variables of interest included child welfare investigations and substantiated child welfare investigations. An investigation was defined as a family experiencing a child welfare investigation within 2 years of the child’s birth on which the mother was listed as perpetrator and/or the child was listed as victim. A substantiated investigation was defined as a family having an investigation that met the above criteria on which one or more allegations had a disposition of “reason to believe,” which indicates sufficient evidence that abuse or neglect occurred. An unsubstantiated investigation was defined as an investigation that received any disposition other than “reason to believe” (See Supplemental Materials).
For the sequential logistic regression, the outcome was an ordered variable indicating a family’s investigation status and the substantiation status of their first investigation during the child’s first 2 years of life (0 = no investigation; 1 = any investigation and first investigation was not substantiated; 2 = any investigation, and their first investigation was substantiated). For the logistic regression, the dependent variable was a binary indicator of whether a family experienced any substantiated investigation over the child’s first 2 years of life (0 = no, 1 = yes).
We selected covariates that were theoretically linked with the outcomes or that our previous but unpublished research on a similar sample showed were associated with experiencing a child welfare investigation (Osborne et al., 2021). Although our sample was well matched between the HV and non-HV groups, we included these covariates to account for any remaining heterogeneity. We included the following covariates in the analyses: mother’s race/ethnicity, mother’s age, mother’s age squared, mother’s education, mother’s nativity, mother’s marital status and father listed on birth certificate, Medicaid/CHIP payment for birth, WIC use for self during pregnancy, mother’s reported smoking status in the three months prior to pregnancy or in any trimester of pregnancy, percent of children under 18 years of age in the mother’s ZIP code of residence who are living at or below the poverty level, child’s birth year, and child’s sex assigned at birth (See Supplemental Materials). Finally, we included a fixed-effect control for each county with more than 50 families in the sample to account for unmeasured factors such as geographic-based differences in demographic characteristics and variation in local child welfare practices, which can vary across counties.
Analysis
We used several analytic approaches to address our research questions, including proportions tests, t-tests, multivariate logistic regression, and sequential multivariate logistic regression. All multivariate models attempted to control for observed heterogeneity that remained after the matching process by including the previously described covariates.
First, we used univariate descriptive and inferential statistics to provide a general overview of the sample’s child welfare involvement. We computed the: (1) prevalence of investigations and substantiations among all families and the prevalence of substantiations among families with investigations, (2) prevalence of abuse vs. neglect-only allegations, and (3) average time to the first investigation and substantiated investigation. We also used proportions tests and t-tests to compare these values between HV and non-HV families.
For our main analysis, we tested whether HV families are more or less likely than non-HV families to experience an investigation of abuse or neglect in the child’s first 2 years of life using sequential logistic regression. Sequential logistic regression simultaneously predicts logistic regression models for two sequential outcomes, accounting for the second stage relying on the result of the first stage of the model. Our model’s first stage estimates the home visiting program’s effect on the odds of having any investigation in the child’s first 2 years while controlling for additional covariates, and the second stage estimates the odds of the first investigation being substantiated within the sample that had an investigation.
Next, we used logistic regression to estimate the home visiting program’s effect on the odds of having any substantiated investigation in the child’s first 2 years of life, within the full sample of families, regardless of whether they had any investigation. This logistic regression tested the difference in substantiated investigations between all HV and non-HV families.
To explore whether the effects of the home visiting program differ by mother’s race or ethnicity, we also estimated separate models that included an HV by race/ethnicity interaction term. However, the HV by race/ethnicity interactions were not significant; therefore, we did not retain these models, nor do we report on them further.
Results
Descriptive Statistics
Investigation Outcomes by HV Participation.
aN = 17,592 (n = 8796 for each group).
bHV n = 1085; Non-HV n = 890.
cHV n = 476; Non-HV n = 419.
*p < .05.
**p < .01.
***p < .001.
HV families did not differ from non-HV families in the length of time until their first investigation (p = .234) or substantiated investigation (p = .577). On average, investigated families experienced their first investigation approximately 8.5 months after birth, and families with substantiations had their first substantiation approximately 9 months after birth.
We also used proportions tests to compare investigated families on their allegation types (neglect allegations only vs. any abuse allegations). Approximately 40% of families with investigations had at least one abuse allegation, but allegation type did not differ by HV participation (p = .956). Less than 25% of families with substantiated investigations had any abuse allegation substantiated, with no differences by HV participation (p = .889).
Multivariate Results
Sequential Logistic Regression and Logistic Regression Results.
Note. Predictors not shown: birth year category and county fixed effect. N = 17,592 (n = 8796 for each group). OR = Odds Ratio.
aWe did not match on this variable.
bFathers married to the mother at the time of birth are automatically listed on the birth certificate in this state.
cContinuous, 1 unit equal to 10 percentage point increase in poverty.
†p < .1.
*p < .05.
**p < .01.
***p < .001.
The second stage of the sequential logistic regression indicated that although HV families had significantly higher odds of experiencing an investigation overall, investigated HV families had 21% lower odds than investigated non-HV families of having their first investigation substantiated (OR = 0.79, p = .016). Furthermore, in the full sample, HV families did not differ significantly from non-HV families in the odds of experiencing any substantiated investigation during the child’s first 2 years of life (OR = 1.10, p = .186).
Discussion
Using a large, diverse sample, we examine differences in child welfare involvement rates between families that participated in one home visiting program and comparison families. Our study also goes beyond most existing studies of home visiting and child welfare involvement by including both substantiated and unsubstantiated investigations. Overall, 11% of the sampled families experienced investigations in the first 2 years of life. HV families are more likely than non-HV families to experience investigations for abuse or neglect during the child’s first 2 years of life, but investigated HV families are less likely than investigated non-HV families to have a substantiation, resulting in equivalent rates of substantiations among HV and non-HV families. This finding indicates that the difference in investigations between the two groups is comprised primarily of unsubstantiated investigations (Figure 1). Predicted Probability of Substantiated and Unsubstantiated Investigations: First Investigation. Note. Values above the bars represent predicted probabilities from the first stage of the multivariate sequential logistic regression model. Values on bars represent predicted probabilities from the second stage of the multivariate sequential logistic regression model.
Although we must use caution in assuming causality, the results are consistent with a surveillance hypothesis; HV families may experience greater surveillance than non-HV families, leading to a higher number of maltreatment reports. However, in the event of an investigation, HV families are less likely than non-HV families to have confirmed maltreatment. Surveillance could be direct through home visitors or indirect through other service providers to which home visitors refer families. Also, families enrolled in home visiting may already be involved in additional social services programs with staff that could report suspected abuse or neglect.
Readers should not assume that higher odds of unsubstantiated investigations indicate that home visiting families experience unfounded maltreatment reports (Drake, 1996). Although surveillance could lead to investigations that may be deemed unnecessary, service providers, such as home visitors, may be able to detect and report true risks within participating families’ homes. Higher rates of unsubstantiated investigations may indicate risk for harm and may lead to families being offered needed services (Drake et al., 2003; Fong, 2020; HHS et al., 2023; Kohl et al., 2009; Kugler et al., 2019). However, families receiving child welfare services may be more likely to experience rereports, even after adjusting for risk (Fuller & Nieto, 2014)
The results from this study are generally in line with previous studies. Most of the evidence that home visiting may reduce early maltreatment involved medical encounters or self-reported maltreatment, rather than investigations for abuse or neglect as the outcomes (Caldera et al., 2007; Duggan et al., 1999, 2004; Kitzman et al., 1997; Mitchell-Herzfeld et al., 2005; Olds et al., 1986). Beyond the original NFP RCT, in which Olds et al. (1986) found trend-level effects for confirmed reports of abuse and neglect among unmarried teen parents living in poverty, there has not been evidence that the widely-used home visiting models serving families prenatally lead to less involvement with the US child welfare system in the first 2 years of life.
Previous evidence suggests that home visiting may have a larger impact on child maltreatment after participation ends, when children are older (DuMont et al., 2010; Easterbrooks et al., 2019; Lee et al., 2018; Olds et al., 1997; Zielinski et al., 2009). Zielinski et al. (2009) hypothesize that surveillance related to home visiting may detect families at risk for maltreatment early and provide them with needed services to prevent later maltreatment, which is in line with Drake’s (1996) theory on unsubstantiated investigations. If the home visiting program in this study has preventative effects, those effects may not appear until beyond the observation window for the current study.
Because certain racial and ethnic groups often disproportionately experience child welfare system involvement in contrast to the racial and ethnic composition of the population (HHS et al., 2023), we tested HV by race/ethnicity interactions to determine whether effects of the home visiting program differ by race/ethnicity. The interactions were not significant, providing no evidence that program effects differ by race/ethnicity.
Limitations
Despite this study’s rigorous quasi-experimental methods, it is not without limitations. Because families are not randomly assigned to participate in home visiting in this population, selection bias may occur in the sample; we cannot rule out that families that choose to participate in home visiting may differ from non-participants in important but unmeasured ways. We address the concern of selection bias by constructing a comparison group that is statistically equivalent to the intervention group on observable characteristics that are likely to be related to our outcomes of interest, and by controlling for a wide range of covariates to account for additional heterogeneity between the HV and non-HV samples. However, unmeasured differences may remain.
We successfully identified 86% of the home visiting program population in the state birth records data prior to selecting the study sample. The remaining 14% of families we did not find may have given birth outside of the state, may have not experienced a live birth (i.e., families without child name & date of birth information), or they may have remained unidentified in the population of birth records. Therefore, some of the unidentified home visiting families could contaminate the comparison sample. However, unidentified home visiting families would have made up only a tiny fraction of the population deemed eligible for the comparison group, making it unlikely that unidentified home visiting families made it into the comparison sample.
We exclude a small number of families from the full population of home visiting participants when creating our sample. For example, we exclude families from small minority racial groups, families that participated in more than one home visiting program, families with mothers who had prior child welfare involvement, and children with congenital anomalies. Because of their rarity, the results from these groups might not be generalizable even if we did include them. The results may also not generalize to other home visiting program models or outside of the state of this study.
Our analyses may represent a somewhat conservative estimate of the true child maltreatment rates of families for the sample. This study includes only documented investigations and substantiated investigations for maltreatment, but child maltreatment can go undetected or unreported and may never lead to a formal investigation. Furthermore, the state’s maltreatment investigations data are purged regularly, and investigations that have been “ruled out” with low to moderate risk, are purged 18 months to 5 years after the investigation closes. Although we combine multiple waves of data that we received from the state over numerous years to retain as many purged investigations as possible, some families may have investigations that were purged prior to our receipt, particularly families with earlier births. Our estimate may undercount the true number of families with child welfare involvement, but we believe data loss is limited. Also, all substantiated investigations remain in the state child welfare database; therefore, we expect that we captured all substantiated investigations. Further, we have no reason to expect a difference in the rate of data loss between the HV and non-HV groups.
Finally, our analyses do not examine the source of the child maltreatment allegation(s) that lead to investigation (i.e., the reporter); this was a limitation of the data available for the study. Understanding who reports suspected child maltreatment to the state would be valuable for further exploring whether service involvement, including home visiting and other community services, leads to more reports to child welfare services on a family (i.e., surveillance bias). If service involvement is associated with more child welfare reports, reporter data can also provide insight into whether reports from service providers tend to be substantiated or unsubstantiated allegations of maltreatment. Future research should focus on addressing this limitation of the current study.
Implications
We find that HV families have greater odds of experiencing investigations and equivalent odds of experiencing substantiated investigations in the first 2 years of life compared to non-participating families. The most probable explanation for the results is that surveillance bias may occur, leading to a greater number of unsubstantiated investigations for participants.
Although the results indicate that there are no differences in rates of substantiated investigations between HV and non-HV families, unsubstantiated investigations can have important implications for families. Undergoing a child abuse and neglect investigation can be incredibly disruptive, traumatizing, and stressful for families; this means that an excess of unsubstantiated investigations may be a negative outcome for home visiting participants (Fong, 2020; Merritt, 2020).
In existing research, recidivism rates differ little (Drake et al., 2003) or not at all (Kohl et al., 2009) between unsubstantiated and substantiated cases. Other studies have found few differences in child outcomes, such as children’s anxiety, anger, post-traumatic stress, socialization, daily living skills, internalizing and externalizing behavior, teen parenthood, HIV-risk behaviors, or drug use following unsubstantiated and substantiated investigations (Hussey et al., 2005; Kugler et al., 2019).
The similarities between substantiated and unsubstantiated cases suggest that risk in unsubstantiated cases may still be high, and families may benefit from services even if maltreatment cannot be confirmed (Kohl et al., 2009). In fact, though the rates vary greatly across states, more than one-quarter of children in unsubstantiated cases in the United States receive post-investigation services (HHS et al., 2023). As Zielinski et al. (2009) posit, if home visiting families with high risk for abuse and neglect are detected early and receive the services they need, there could be long-term effects of this home visiting program on maltreatment; if so, more unsubstantiated investigations may be a positive outcome. Without further research examining risk severity, service provision, and longer-term trajectories of families in this study, we cannot determine whether excess unsubstantiated results are positive, negative, or something in between.
Various evidence-based home visiting programs have shown favorable effects in child development and school readiness, child health, family economic self-sufficiency, linkages and referrals, maternal health, and positive parenting practices, as well as reductions in juvenile delinquency, family violence, and crime (HomVEE, 2023); therefore, participating families may benefit from home visiting in multiple ways. Although determining whether unsubstantiated investigations are justified and weighing the potential benefits or harms from such investigations are beyond the scope of this study, the home visiting field may benefit from future research addressing the overall net benefits or harms from the potential effects of home visiting across multiple domains.
To reduce the possible negative impact of surveillance bias from home visiting programs, we recommend that state agencies continue to carefully train service providers, emphasizing the difference between circumstances that indicate a family needs supports (i.e., parent education, material resources) rather than indicate possible abuse or neglect. Finally, we found that most studies of home visiting and child welfare involvement focus on substantiated investigations and do not include unsubstantiated investigations in analyses. Though substantiations are likely one of the best indicators of whether abuse or neglect has occurred, including only confirmed reports of maltreatment leaves out an important piece of the story (Kugler et al., 2019). We strongly encourage researchers studying child welfare preventative programs to include unsubstantiated investigations along with substantiated investigations in their future research.
Conclusion
The home visiting program in this study is a widely used evidence-based home visiting program designed to prevent child maltreatment in addition to targeting other prenatal, maternal, and child outcomes. To our knowledge, this is the largest quasi-experimental study to date that compares investigations of abuse and neglect, both substantiated and unsubstantiated, between home visiting-participating families and non-participants. HV families are more likely than non-HV families to experience investigations for abuse or neglect during the child’s first 2 years of life, but investigated HV families are less likely than investigated non-HV families to have a substantiation; this results in equivalent rates of substantiations among HV and non-HV families overall. These results may indicate that home visiting families experience increased surveillance during their service period, with important implications for staff training and future research.
Supplemental Material
Supplemental Material - Home Visiting and Child Welfare Involvement: A Matched Comparison Group Study
Supplemental Material for Home Visiting and Child Welfare Involvement: A Matched Comparison Group Study by Hilary A. Doe, Cynthia Osborne, Jennifer Huffman, Sean M. Craig, and Mason Shero in Child Maltreatment.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for the data collection and original data analysis for this research was through FY20 MIECHV Formula Grant Funding.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
