Sage Journals: Discover world-class research

Abstract

The purpose of this study is to estimate the rate of emotional disturbance (ED) among children in foster care and assess the validity of the national foster care census data (AFCARS) measure of ED. This study used linked child protection and Medicaid records from 2014 and 2015, for the states of California and Wisconsin, as well as data from AFCARS, a federal population census of children in foster care which states are mandated to contribute to. ED is defined by AFCARS and includes an array of mental and behavioral health diagnoses. According to AFCARS, 13% of CA children in foster care and 15% of WI children in foster care had an ED, whereas Medicaid claims produce rates of 45% and 48%, respectively. Rates of ED among children in congregate care were underestimated by 43–46 percentage points, with substantial proportions having diagnoses of disruptive behavioral disorders. Despite the AFCARS ED measure being cited in congressional testimonies and its wide use in research, results from this study suggest that the AFCARS ED estimates are an unreliable metric for use in research, policy, or practice.

Keywords

adoption and foster care reporting system child welfare foster care congregate care mental health

Despite consensus that youth in foster care face elevated risks of mental illness compared with the general population (McMillen et al., 2005; Steele & Buchi, 2008), estimated rates vary considerably. Many researchers, as well as federal evaluation protocols (Administration for Children and Families, 2014), rely on the Adoption and Foster Care Reporting System (AFCARS), a federal population census of children in foster care to which states are mandated to contribute. AFCARS has a single indicator of “emotional disturbance” (ED), defined based on caseworker report of whether the child has one or more qualifying diagnoses, such as posttraumatic stress disorder or depression (AFCARS Foster Care Annual File Codebook, 2021). There have been, however, few attempts to verify the quality or reliability of the AFCARS ED measure, despite its continued relevance and use in research, evaluation, and policy (Finster & Norwalk, 2021; U.S. Children’s Bureau, 2015).

A recent example of the importance of validating estimated rates of ED comes from the Family First Prevention Services Act of 2018 (FFPSA) (Bipartisan Budget Act, 2018). Preceding the enactment of this law, Congressional hearings discussed the potential harms and overuse of congregate care. In the US, congregate care is typically used for adolescents and is supposed to be a placement option only where a family (kin or non-relative) foster home is unable to meet a child’s needs. Citing a U.S. Children’s Bureau analysis of AFCARS (U.S. Children’s Bureau, 2015), it was asserted that only 21% of children living in congregate care – the vast majority of whom are teens— had an ED (No Place to Grow up: How to Safely Reduce Reliance on Foster Care Group Homes; Senate Hearing 114-273, 2015, p. 26). This estimate is 6 percentage points higher than the general population prevalence of depression, anxiety, or ADHD among children 6–11 years (15%) and equal to the general population prevalence for children ages 12–17 (21%) (Whitney & Peterson, 2019). The suggestion that rates of ED in the general population are no different than for youth in the most restrictive form of foster care should be viewed skeptically given abundant evidence linking exposure to child maltreatment – the primary antecedent to foster care entry—with mental health problems (Norman et al., 2012) as well as studies documenting that children in foster care have elevated rates of ED (McMillen et al., 2005; Steele & Buchi, 2008; Turney & Wildeman, 2016). Nevertheless, FFPSA, as enacted, includes provisions intended to drastically reduce use of congregate care by limiting federal funding for such placements. These changes have further incentivized states to transition a greater share of foster care youth into traditional foster or kinship placements. Although it is optimal to place children in the least restrictive setting in which their needs can be met, FFPSA came with little funding to improve recruitment, retention, or training of foster homes for youth with behavioral and emotional challenges. Efforts to scale down congregate care are motivated by several factors, including concerns that lack of a family environment is developmentally harmful to youth and the high cost of congregate care. Yet, the assertion that large proportions of children in congregate settings had no clinical justification for a restrictive placement motivated support for this legislation (Larson, 2018).

An accurate understanding of children’s mental health in foster care is crucial, both for the development of health policy and appropriate allocation of resources. Thus, this study sought to: (1) estimate rates of ED as defined by the AFCARS codebook, overall and by diagnosis group, from Medicaid claims records of children in care in Wisconsin and California; (2) compare the Medicaid claims-derived ED estimates to those reported in AFCARS; and (3) compare ED estimates for youth in congregate care versus family foster care settings.

Methods

Data and Sample

We draw primarily on foster care records from state child welfare information systems and Medicaid claims data in Wisconsin and California. Wisconsin’s data were accessed via the Wisconsin Administrative Data Core (WADC) at the University of Wisconsin Madison and California’s data were accessed from the Children’s Data Network at the University of Southern California; both repositories hold data sharing agreements with the involved state entities. Data were probabilistically linked within state using available identifiers and then stripped of all direct identifiers prior to analysis. For comparison, we also draw upon the AFCARS child files, obtained through the National Data Archive on Child Abuse and Neglect (NDACAN), which are an annual national census of children in foster care.

For the CA, WI, and AFCARS datasets, we selected the following analytic samples: children in foster care between January 1, 2014 through September 30, 2015, who entered care prior to their 18^th birthday and spent at least 30 consecutive days in foster care. In AFCARS, we excluded children with missing or invalid information on birthdate or date of foster care entry. We then produced two datasets for each state: (1) linked foster care and Medicaid claims data (WI-Medicaid Linked and CA-Medicaid Linked); and (2) records submitted to the federal AFCARS child files (WI-AFCARS and CA-AFCARS). A description of these datasets is shown in Supplementary Appendix A, along with the national AFCARS.

Measures

AFCARS data files include an aggregate binary indicator of ED that is reported by the child’s caseworker. A value of 1, according to the AFCARS codebook, should be assigned if the child has diagnoses that meet the definition of an ED, based on the Diagnostic and Statistical Manual of Mental Disorders IV (American Psychiatric Association, 2000). This list was mapped onto the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9) diagnostic codes that are used in medical claims. We create a narrow version, reflecting only the diagnoses stated explicitly in the AFCARS codebook, and an extended definition, which included all listed diagnoses in addition to conceptually relevant but not listed diagnoses (see Supplementary Appendix B). ED is equal to 1 if a child received any Medicaid-billed health care services billed under a qualifying diagnosis while in foster care and during the observation period. A qualifying claim could stem from inpatient or outpatient services provided by any health professional (e.g., pediatrician, therapist, psychiatrist). Medicaid is a promising source of data on diagnoses given that it covers an estimated 99% of children in foster care (Libby et al., 2006; Medicaid and CHIP Payment and Access Commission, 2015).

Using the Medicaid-linked data, we also categorized ED conditions into eight groups (World Health Organization, 2019): (1) Adjustment disorders, including acute reaction to stress and post-traumatic stress disorder; (2) Anxiety disorders; (3) Mood disorders; (4) Attachment disorders; (5) Autism; (6) Attention Deficit disorders; (7) Conduct disorders; and (8) other. The other category includes diagnoses that are too rare to generate estimates for individually and diagnoses that do not fall into any of the other groups. Appendix B lists the diagnoses by group and indicates inclusion in the narrow definition or only the definition only.

Additionally, data from our linked state datasets are used to characterize the rates of any and specific types of ED-qualifying diagnoses for congregate care placements and non-congregate care placements. Congregate care is defined as any group home or residential facility placement and family setting is defined as any kin or non-kin foster family home, pre-adoptive or guardian placement. As congregate care placements are rare for children under 9 years of age (<2% for children 0–8 years vs. 24% for children 9–17 years), diagnostic breakdowns by placement type include only children aged 9–17 years. Placement type was indicated based on the child’s last placement within the time frame.

Descriptive data summaries are provided to illustrate and compare the percentage of children in foster care with an ED diagnosis overall and by state using both AFCARS and Medicaid derived estimates. Additional descriptive statistics are provided to outline ED estimates by age group (0–8 years; 9–17 years), type of diagnosis and placement setting.

Results

Figure 1 displays the rates of ED identified in the AFCARS data files and those estimated from Medicaid-linked data. Rates of ED in CA and WI AFCARS are 13% and 15%, respectively. In both states, rates are very similar to that reported in AFCARS for the nation overall (15%), suggesting that neither state is an outlier with respect to AFCARS reporting. There is a 32-percentage point gap for reported ED diagnoses between AFCARS and state data for CA (13% vs. 45%) and a 33-point difference between AFCARS and state data for WI (15% vs. 48%). Even when using the conservative (narrow) ED definitions there remains a three-fold difference between AFCARS and state data estimates (CA 13% vs. 39%; WI 15% vs. 45%).

Figure 1.

Rates of emotional disturbance identified in the adoption and foster care reporting system data files and Medicaid-linked data, by state.

In both states, majorities of children ages 9+ have an ED regardless of setting (Table 1). However, children in congregate care have significantly higher rates of ED compared to children in family settings. Again, rates of diagnoses for children in both family-like and congregate care settings are substantially lower in AFCARS than in Medicaid claims. Rates of disruptive disorder diagnoses are twice as high in WI and nearly three times higher in CA for children in congregate care compared to children placed in a family-like setting. In CA, one in four children in congregate care has an ADHD diagnosis, more than double the rate among children in a family-like setting, and in WI nearly half (45%) of children have an ADHD diagnosis. There is a 34-percentage point (CA) and 21 percentage point (WI) difference in proportion with mood disorder diagnoses between children in family-like versus congregate care settings. A minority of children eight and under had an ED, with Medicaid derived rates pointedly higher than those in AFCARS (Medicaid: CA 33%, WI 28%; AFCARS: CA 4%, WI 5%).

Table 1.

Emotional Disturbance Estimates for Foster Care Population for 8 and Under and 9+ by State and Final Placement Setting.

	Children 0–8 years All settings		Children 9+ Family settings		Children 9+ Congregate care
	CA	WI	CA	WI	CA	WI
AFCARS data
Caseworker recorded ED indicator	4%	5%	21%	25%	35%	35%
Medicaid-linked data
Constructed ED indicator - extended definition	33%	28%	56%	69%	78%	83%
Diagnosis group
Adjustment	19%	20%	34%	43%	39%	33%
Anxiety	3%	3%	9%	14%	16%	19%
Mood	2%	3%	23%	28%	57%	49%
Attachment	1%	2%	1%	4%	2%	6%
Autism	1%	2%	1%	3%	4%	7%
ADD/ADHD	4%	9%	12%	31%	25%	45%
Disruptive/conduct	5%	6%	12%	19%	36%	40%
Other condition	10%	3%	6%	8%	22%	13%

Note. Population of children 0–8 years in CA during study time frame was 53,034 and WI was 7,162. Population of children 9–17 years in foster family (kin or non-relative) settings in CA was 36,472 and WI was 4,365. Population of children 9–17 years in congregate care in CA was 6,003 and in WI was 1,555.

Note. CA = California. WI = Wisconsin. ED = emotional disturbance. ADD/ADHD = Attention Deficit Disorder/Attention Deficit Hyperactivity Disorder.

Discussion

AFCARS data are used for federal performance monitoring of foster care systems under the Child and Family Service Reviews (Administration for Children and Families, 2014), in annual reports to Congress (U.S. Department of Health and Human Services, 2016), and to inform policy debates (No Place to Grow up: How to Safely Reduce Reliance on Foster Care Group Homes; Senate Hearing 114-273, 2015). States have long cited problems providing adequate care for children with mental and behavioral health concerns (Government Accountability Office, 2015), but their submitted AFCARS data imply that a small minority of children in their care have an ED, broadly defined to include most psychiatric diagnoses. This study leveraged linked foster care and Medicaid records in two states to ascertain the rates and types of psychiatric diagnoses among children in care and to evaluate whether AFCARS is a reliable source of information on ED prevalence. We find state estimates using linked Medicaid claims data – which largely align with the broader research (McMillen et al., 2005; Turney & Wildeman, 2016) -- are three times higher than AFCARS estimates, signaling that AFCARS ED estimates are an unreliable metric for use in research, policy, or practice.

Low reliability of the AFCARS ED measure may have several causes. First, states have an array of medical information available to them for children in their care and need not rely on this specific data field for decision-making and evaluation. Thus, the accuracy of this specific field is inconsequential to agencies and may not be recorded at all or updated over time as children’s mental health status changes or new information becomes available. Second, “emotionally disturbed” is a pejorative label that caseworkers may be reluctant to use, especially if a child’s ED conditions are mild or adequately managed with current treatments. Although we found that most children in congregate care (78% in CA and 83% in WI) have one or more claims indicating a mental health diagnosis, it is challenging to ascertain whether their condition was severe enough to temporarily or permanently preclude placement in a family-like (less restrictive) environment. We note, however, that more than one-third of children in congregate care had a diagnosis of conduct, oppositional defiance, or impulse control disorder – conditions characterized by externalizing behaviors that are strongly predictive of placement disruption (Konijn et al., 2019). Thus, states may face serious challenges to moving children from congregate care to less-restrictive environments. Understating the prevalence of mental health needs in the congregate care population may also lead states to under-invest in the resources needed for children to safely and stably step down from or avoid congregate care.

Limitations

Despite the high numbers of children overall, and especially older children, with diagnoses and claims for mental health services, this study may nevertheless undercount the incidence of ED. First, the probabilistic linking strategies used by both states may fail to link children where there were errors in their identifying information, such that they would be incorrectly identified as having no billed services during the time period. Second, some children may not be receiving mental health care despite a need (Shin, 2005) or may be receiving mental health care that is not billed through insurance (Department of Healthcare Services, 2020). Because both limitations mean that our Medicaid-based estimates are conservative (i.e., provide a lower-bound estimate of ED), we are potentially understating how unreliable the AFCARS estimates are. Additionally, this study is limited by an inability to characterize the onset, duration, or severity of ED and the extent to which a child’s symptoms were effectively managed by the services they received. Future research may consider whether caseworkers’ awareness of children’s ED diagnoses affects the quality or intensity of services provided.

Implications

The current study underscores shortcomings of data captured in state administrative child welfare systems and the significant inaccuracy of national statistics on mental health conditions among children in foster care. The implications of using inaccurate estimates to inform policy can be profound as exemplified by the congressional hearings on FFPSA and congregate care where inaccurate data contributed to the reduction in congregate care without investment in the recruitment, retention and training of foster families equipped to address the emotional and behavioral needs to youth stepping down from restrictive care. More broadly, however, children’s mental and behavioral health both affects and is affected by many core foster care performance objectives and outcomes, including safety, stability, non-restrictive care, and permanency (Aarons et al., 2010; James et al., 2006; Rubin et al., 2007). Yet, there is currently no regular and systematic collection of this information. Prior studies have reported on rates of mental health conditions among children in foster care (McMillen et al., 2005; Steele & Buchi, 2008), but these data are typically confined to specific regions or points in time making generalizability and examinations of trends difficult. The lack of reliable information related to the mental health needs of children in foster care make it difficult to understand trends over time and accurately evaluate or compare system performance in providing quality care for children with ED, therefore, improving sound measurement of ED in administrative data is imperative. Given that all children in foster care are eligible for Medicaid, cross system information linking on a national level (AFCARS and Medicaid) could be utilized to improve measurement quality.

Supplemental Material

Supplemental Material - Rates of Emotional Disturbance Among Children in Foster Care: Comparing Federal Child Welfare Data and Medicaid Records in Two States

Supplemental Material for Rates of Emotional Disturbance Among Children in Foster Care: Comparing Federal Child Welfare Data and Medicaid Records in Two States by Lindsey Palmer, Sarah Font, Toria Herd, John Prindle and Emily Putnam-Hornstein in Child Maltreatment

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development grant R21HD091459 at the University of Wisconsin-Madison and P50HD096719 grant at the University of North Carolina Chapel Hill. Additional support was received in the form of research grant (R01HD095946), training grant (T32HD101390), and infrastructure grants (P50HD089922 and P2CHD041025) at Pennsylvania State University. Data infrastructure support for California records was provided by First 5 LA, the Conrad N. Hilton Foundation, and the Heising-Simons Foundation. The authors of this article are solely responsible for the content therein. The authors would like to thank the Wisconsin Department of Children and Families and Department of Health Services, as well as the California’s Department of Social Services and Department of Healthcare Services for the use of data for this analysis, but these agencies do not certify the accuracy of the analyses presented. All conclusions drawn are those of the authors alone and do not necessarily reflect those of any agency partner or funding entity.

ORCID iDs

Lindsey Palmer

Sarah Font

John Prindle

Supplemental Material

Supplemental material for this article is available online.

References

Aarons

G. A.

James

Monn

A. R.

Raghavan

Wells

R. S.

Leslie

L. K.

(2010). Behavior problems and placement change in a national child welfare sample: A prospective study. Journal of the American Academy of Child & Adolescent Psychiatry, 49(1), 70–80. https://doi.org/10.1016/j.jaac.2009.09.005

Administration for Children, & Families (2014). Final notice of statewide data indicators and national standards for Child and Family Services Reviews. Children’s Bureau.

AFCARS Foster Care Annual File Codebook (2021). National data archive on child Abuse and Neglect. AFCARS Foster Care Annual File Codebook. https://www.ndacan.acf.hhs.gov/datasets/pdfs_user_guides/afcars-foster-care-file-codebook.pdf

American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR. American Psychiatric Association.

Bipartisan Budget Act, Pub. L. No. 115–123 (2018). https://www.congress.gov/bill/115th-congress/house-bill/253.

Department of Healthcare Services (2020). Mental Health Services Act. Department of Healthcare Services. https://www.dhcs.ca.gov/services/mh/Pages/MH_Prop63.aspx

Finster

H. P.

Norwalk

K. E.

(2021). Characteristics, experiences, and mental health of children who re-enter foster care. Children and Youth Services Review, 129(1), Article 106165. https://doi.org/10.1016/j.childyouth.2021.106165

Government Accountability Office (2015). HHS could do more to support states’ efforts to keep children in family-based care. (GAO-16-85). https://www.gao.gov/assets/gao-16-85.pdf

James

Leslie

L. K.

Hurlburt

M. S.

Slymen

D. J.

Landsverk

Davis

Mathiesen

S. G.

Zhang

(2006). Children in out-of-home care: Entry into intensive or restrictive mental health and residential care placements. Journal of Emotional and Behavioral Disorders, 14(4), 196–208. https://doi.org/10.1177/10634266060140040301

10.

Konijn

Admiraal

Baart

van Rooij

Stams

G.-J.

Colonnesi

Lindauer

Assink

(2019). Foster care placement instability: A meta-analytic review. Children and Youth Services Review, 96(1), 483–499. https://doi.org/10.1016/j.childyouth.2018.12.002

11.

Larson

J. B.

(2018). Text - H.R.1892 - 115th congress (2017-2018. Bipartisan Budget Act of 2018 (2017/2018) [Legislation] https://www.congress.gov/bill/115th-congress/house-bill/1892/text

12.

Libby

A. M.

Kelleher

K. J.

Leslie

L. K.

O’connell

Wood

P. A.

Rolls

J. A.

Landsverk

(2006). Child welfare systems policies and practices affecting medicaid health insurance for children. Journal of Social Service Research, 33(2), 39–49. https://doi.org/10.1300/J079v33n02_04

13.

McMillen

J. C.

Zima

B. T.

Scott

L. D.

Auslander

Munson

Ollie

Spitznagel

(2005). Prevalence of psychiatric disorders among older youths in the foster care system. Journal of the American Academy of Child & Adolescent Psychiatry, 44(1), 88–95. https://doi.org/10.1097/01.chi.0000145806.24274.d2

14.

Medicaid and CHIP Payment and Access Commission (2015). Intersection of Medicaid and child welfare (p. 33). Medicaid and CHIP Payment and Access Commission.

15.

No place to grow up: How to safely reduce reliance on foster care group homes; Senate Hearing, 114-273, 114th Congress, 1(2015).

16.

Norman

R. E.

Byambaa

Butchart

Scott

Vos

(2012). The long-term health consequences of child physical abuse, emotional abuse, and neglect: A systematic review and meta-analysis. PLoS Medicine, 9(11), Article e1001349. https://doi.org/10.1371/journal.pmed.1001349

17.

Rubin

D. M.

O’Reilly

A. L.

Luan

Localio

A. R.

(2007). The impact of placement stability on behavioral well-being for children in foster care. Pediatrics, 119(2), 336–344. https://doi.org/10.1542/peds.2006-1995

18.

Shin

S. H.

(2005). Need for and actual use of mental health service by adolescents in the child welfare system. Children and Youth Services Review, 27(10), 1071–1083. https://doi.org/10.1016/j.childyouth.2004.12.027

19.

Steele

J. S.

Buchi

K. F.

(2008). Medical and mental health of children entering the Utah foster care system. Pediatrics, 122(3), e703–e709. https://doi.org/10.1542/peds.2008-0360

20.

Turney

Wildeman

(2016). Mental and physical health of children in foster care. Pediatrics, 138(5), Article e20161118. https://doi.org/10.1542/peds.2016-1118

21.

U.S. Children’s Bureau (2015). A national look at the use of congregate care in child welfare. https://www.acf.hhs.gov/cb/resource/congregate-care-brief

22.

U.S. Department of Health and Human Services (2016). Child welfare outcomes 2016: Report to congress (p. 95). Retrieved from: https://www.acf.hhs.gov/sites/default/files/cb/cwo2016.pdf#page=57

23.

Whitney

D. G.

Peterson

M. D.

(2019). US national and state-level prevalence of mental health disorders and disparities of mental health care use in children. JAMA Pediatrics, 173(4), 389–391. https://doi.org/10.1001/jamapediatrics.2018.5399

24.

World Health Organization (2019). International classification of Diseases, Ninth Revision. CDC. https://www.cdc.gov/nchs/icd/icd9.htm

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.43 MB