Abstract
Child neglect is prevalent across all societies and can cause long-lasting and significant harm for children and young people (Daniel, 2015; Daniel et al., 2010; English et al., 2005; Horwath, 2013; Moran, 2009; Stevenson, 2007). Neglect is often defined as unmet need (Daniel, 2015). However, a variety of more sophisticated definitions have been developed within professional practice and academia. Without clarity on the definition of a concept like neglect, precise and accurate measurement is difficult to achieve (Perron & Gillespie, 2015).
Neglect incorporates a variety of experiences for children and young people ranging from lack of supervision to extreme deprivation. It is widely recognized that experiencing neglect during childhood can increase the risk of negative health and of negative emotional and social outcomes later in life (Corby et al., 2012; Horwath, 2007; Howe, 2005; Radford, 2011). Neglect raises issues for the helping professions in terms of identification, assessment, and support (Brandon et al., 2009; Daniel et al., 2010; Horwath, 2007). Despite this, there has been limited research into neglect compared to other forms of child abuse (Daniel et al., 2011; Dubowitz, 2007; McSherry, 2007; Mulder et al., 2018).
A number of authors have raised concerns related to the significant limitations and imprecision of the evidence base around neglect (Barlow & Schrader-Macmillan, 2010; Horwath, 2013; Moran, 2009; Morrongiello & Cox, 2020; Mulder et al., 2018). Although evidence-based high-quality measurement tools are important for measuring abuse and neglect, there are currently no gold standards for the measurement of child neglect or abuse (Bailhache et al., 2013).
In this article, which has been co-produced with an advisory group of relevant stakeholders, we present a systematic review of neglect measurement tools for children's social work. Although we focused on measures of neglect, we recognize that risk assessments of potential neglect are also commonly undertaken in practice (De Bortoli et al., 2017; Mulder et al., 2018).
Background and Significance
Research into maltreatment has been criticized for lacking methodological rigor, imprecise definitions, and inadequate measurement strategies (Manly, 2005), issues that have been linked to its complexity and definitional challenges (Gershater-Molko et al., 2003). There has been a notable lack of research into assessing neglectful parenting, likely influencing the tendency of practitioners to rely on subjective judgements as opposed to evidence-based measures (Hines et al., 2006; Morrongiello & Cox, 2020; Stewart et al., 2015). The extent, impacts, and costs of neglect merit greater attention and scientifically rigorous research (Dubowitz et al., 2005; Horwath, 2013).
Definitions and Complexity of Child Neglect
There are clear issues in defining neglect in both scholarship and professional practice. While abuse is typically identified as an act, neglect is often correlated with omission (English et al., 2005; Moran, 2009), and as such is frequently dichotomized (Sullivan, 2000). Neglect is now understood as a heterogeneous concept and phenomenon, inclusive of a variety of (in)experiences for children and young people (Dubowitz et al., 2005; Horwath, 2013). It has been described as the most subjective of all legally recognized concepts in child welfare (Dubowitz et al., 2005; Zuravin, 1999), which occurs on a continuum with varying frequency and types (Helm, 2010; Higgins & McCabe, 2001; Mennen et al., 2010; Slack et al., 2003).
The scope of neglect, whether it should incorporate both potential and actual harm or just the latter, is debated (Horwath, 2007; Zuravin, 1999). Debates continue around whether definitions should rely on children's basic needs not being met from their perspectives or parental omissions in care (Dubowitz et al., 2005). The concept of neglect is contested and open to significant interpretation in academia and practice (Dubowitz et al., 2004). It is an expansive concept where additional dimensions could be included until it becomes too complex to effectively measure. These issues are important in light of the long-standing issues of accuracy in assessments of neglect (Daniel et al., 2011; Horwath, 2013; Taylor, 2017).
For this review, we have used the operational definition of neglect adopted by the UK government in their Working Together to Safeguard Children (2018) guidance which is “the persistent failure to meet a child's basic physical and/or psychological needs, likely to result in the serious impairment of the child's health or development” (Department for Education [DfE], 2018a, p. 105). Although operational definitions of neglect vary due to factors such as social and cultural influences, definitions in Western countries such as the United States, Australia, and Canada have distinct similarities, for example, all refer to parental failure to meet a child's needs (Horwath, 2013). The US federal legislation provides guidance on child neglect, but definitions are state specific (Horwath, 2013; Child Welfare Information Gateway, 2019).
Prevalence of Child Neglect
It is often difficult to accurately determine the prevalence of neglect (Daniel et al., 2014; Moran, 2009). One proxy for prevalence is the harm category given to cases deemed to meet the threshold of significant harm. These harm categories are neglect and physical, sexual, and emotional abuse (DfE, 2018b). As of March 31, 2021, there were 50,010 children subject to a child protection plan in England and Wales, and neglect accounted for 52% of initial child protection plans (DfE, 2021).
While international comparisons of neglect data are difficult (May-Chahal & Cawson, 2005), similarly high levels can be found in countries such as the United States, Canada, and the Netherlands (Euser et al., 2010; Stoltenborgh et al., 2015; Trocme et al., 2003). In the United States, neglect accounts for 75% of initial referrals to child protective services (CPS) and for the majority of recurrent maltreatment reports (Bae et al., 2010; Jonson-Reid et al., 2019; U.S. Department of Health & Human Services, 2021). It is important to note that the UK and US figures quoted should be compared with caution, as like-for-like data is not available.
Existing Social Work Assessments of Child Neglect
As discussed, there is currently no gold standard for the measurement of child neglect (Bailhache et al., 2013). A global systematic review for the National Institute for Health and Care Excellence (NICE) guidelines for child abuse and neglect failed to find any high-quality evidence for the predictive validity of any tools for identifying neglect (NICE, 2017).
In the absence of clear standards and effective tools, assessments can be subjective, with practitioners setting their own criteria for what is neglectful (Daniel et al., 2010; Stokes & Taylor, 2014; Sullivan, 2000). Neglectful care can be difficult to capture as a static picture within assessments due to a variety of interlocking issues, which include breakdowns in social relationships, inconsistent levels of care, variable impacts of neglect on children, and social workers acting on partial information (Horwath, 2007; Jones et al., 2006; Lacharite, 2014).
Assessment of neglect is complicated by the role of wider social and political contexts. The roles of social harms, such as poverty and insecure housing, should arguably influence what we define as neglect and where responsibility lies. Chronic neglect often involves families facing a wide range of social harms, including socioeconomic disadvantage (Chambers & Potter, 2009; Dufour, 2008). For assessments of neglect to be thorough, an evidence-based approach to systematically construct a layered social reality attentive to these interlocking issues is required (Helm, 2010; McNaughton, 2009; Sayer, 2000), while maintaining a focus on the child (Department of Health [DoH], 2000; Dyke, 2019). Further, the presence, absence, and levels of all neglect categories need to be given full consideration. As with broader social work, effective assessments should be collaborative with families (O’Brien, 2004).
The evidence from research and serious case reviews shows that social work assessments can range from good to flawed (Barlow et al., 2010; Dorsey et al., 2008; Macdonald et al., 2017). Assessments are only as good as the workers completing them and as the support they receive in terms of evidence, research, and training (Milner et al., 2015). Even with these challenges, a thorough social work assessment remains the best tool available in the field (Holland, 2010; Milner et al., 2015; Taylor, 2017), and high-quality assessments are the cornerstone of good practice (Munro, 2020). They are completed by qualified professionals in accordance with government laws and policies (Holland, 2010; Munro, 2020). This study's advisory group and existing guidelines indicated that these continue to be the best and most commonly used tools in practice (Boyd Webb, 2019; DfE, 2018a; DoH, 2000; National Association of Social Workers [NASW]; 2013). Good social work assessments can capture the child within their environment, the feasibility of change, and the support required to effect such change (Milner et al., 2015).
Measuring Neglect
The lack of clarity in defining neglect leads to challenges in how to measure or quantify it (Dubowitz et al., 2005). However, well-developed tools and frameworks can support more accurate and holistic assessments, and counter significant sources of bias within assessments and decision making (Barlow et al., 2010; Parker, 2020). Such tools can be intelligently utilized to inform sound professional judgements (Barlow et al., 2010), enabling a balance between intuitive and analytical reasoning (Munro, 2020).
For neglect, there are four main fundamental assessment elements (Daniel et al., 2011; Horwath, 2013; Jones et al., 2006):
assessment of actual neglect, including types, frequency, severity, and chronicity, assessment of family circumstances, including risk and protective factors, assessment of risk of further neglect, including prospects for change, and how best to meet the child's needs.
This systematic review focuses on tools to assess actual neglect. Predictive tools have clear limitations for neglect. They are not considered good predictors of neglect in a range of both US and UK studies (Logan-Greene & Semanchin Jones, 2018; Semanchin Jones & Logan–Greene, 2016; Taylor et al., 2008). There are multiple and fluctuating individual, familial, community, and societal risk factors for neglect that are nigh on impossible to capture effectively in a predictive tool, while some risk factors can also be consequences of abuse, leading to a further level of complexity and confusion for accurate prediction (Brandon et al., 2014; Lacharite, 2014; Mulder et al., 2018; NICE, 2017).
Existing Reviews of Neglect Measures
Extant reviews of neglect measures have not considered neglect in isolation or have only considered parent/caregiver reports (Daniel et al., 2010; Saini et al., 2019; Yoon et al., 2021a, 2021b). Research into child maltreatment measures has raised concerns about validity, reliability, and usability of tools and the quality of the research undertaken. Yoon et al. (2021a) raised concerns about the validity of child maltreatment measures, stating that the current evidence base is not sufficient. Vial et al. (2020) found that further research is required on the validity, reliability, and usability of child safety assessment instruments, and that an evidence-based approach to measure development is required. Saini et al. (2019) found significant variation in approaches to measure child abuse and in methodologies employed.
The Present Study
This systematic review's evaluation of the evidence of tools or measures for social work assessments of child neglect, therefore, helps fill a key gap. We compared and evaluated these tools against the gold standard of an assessment by a qualified children's social worker or by an assessor working within children's social work (DoH, 2000; Leveille & Chamberland, 2010). The choice of this gold standard is discussed in the section entitled “The Gold Standard for this Review”.
A revised version of Horwath's (2007) neglect typology has been used for this review in line with our advisory group's advice:
emotional, medical, physical, educational, lack of supervision and guidance, and social.
It was developed from a review of other relevant definitions, has a logical delineation into comprehensible neglect categories, and offers a comprehensive understanding of neglect. It has been adopted in the United Kingdom by organizations such as Action for Children. Social neglect has been added to the typology, which involves parents/carers failing to meet a child's social needs for close bonds and relationships, friendships, and social adaptation. It can be understood to play a role in all types of neglect (Horwath, 2013).
The adopted typology is visually represented in our theory of change diagram (Figure 1), which depicts an outline of children's basic needs, categories of neglect, risk, and protective factors. Developed from the review's neglect typology, consultation with the advisory group, and a review of literature on children's needs and neglect, it provides the framework to guide this review. Its simplicity, clarity, and focus on the range of factors influencing neglect fit well with our project's ethos and purpose, and are key features of theories of change (Taplin & Clark, 2012).
Theory of change.
Munro (2020) has proposed that in order to practice effectively, social workers need a formal knowledge base, value base, set of reasoning skills, emotional wisdom, and practice wisdom. Instead of these complicated knowledges and skills, social workers tend to rely on practice wisdom (a combination of everyday skills and wisdom enriched through experience and training) in their assessments (Crisp et al., 2007; Munro, 2020). This study addresses this gap between knowledge and practice actuality by adding to the formal knowledge base for neglect to support practitioners to undertake more focused, evidence-based, and informed assessments.
Method
This review has broadly followed guidance from the Cochrane Collaboration (Higgins et al., 2020). Compared to many medical conditions, however, neglect is a complex phenomenon (Horwath, 2013). Therefore, our approach has been adapted to the field of social work by a deeper exploration of the “target condition” and acceptance of a greater range of study types, in light of the evidence base. In line with recent methods’ developments, this study includes an emphasis on service and practitioner collaboration to mobilize social work values throughout the review (Uttley & Montgomery, 2017).
A review protocol was registered with PROSPERO by Haworth et al. (2020).
Search Strategies and Procedure
A systematic search of the national and international, clinical and academic, single index, and multidimensional measures of child neglect was undertaken. Measures were defined as those concerned with the extent, frequency, chronicity, or severity of neglect as well as those with a focus on its impact on, harm to, and significance for the child.
We searched relevant multidisciplinary and science/social science electronic databases, search engines, and grey literature. This strategy decreased publication bias (Burdett et al., 2003; McFadden et al., 2015). Searches were tailored according to the scope of each database.
The following databases and platforms were searched originally between June and August 2020, with an updating search completed between March and June 2021:
Bibliographic Databases
ProQuest ASSIA (1987-);
Ovid MEDLINE (1946-);
Ovid PsycINFO (1806-);
SCIE Social Care Online;
ProQuest Sociological Abstracts (1952-);
ProQuest Social Services Abstracts (1979-);
Web of Science: Social Science Citation Index (SSCI) (1900-);
EBSCOhost ERIC (Educational Information Resources Centre) (all available years);
EBSCOhost CINAHL (Cumulative Index Nursing and Allied Health) (1981-); and
Prospero https://www.crd.york.ac.uk/prospero/#searchadvance.
Grey Literature
OpenGrey.
Theses and Dissertation Databases
ProQuest Dissertations and Theses Global;
DART—Europe E-Theses Portal;
EThOS—the British Libraries e-theses online service;
Networked Digital Library of Theses and Dissertations (NDLTD); and
Open-Access Theses and Dissertations.
Other Resources
Key websites were searched directly (DoE, Children's Society, and NSPCC).
Handsearching was undertaken to identify additional literature. Relevant email alert services were used to identify newly published literature. New and unpublished trials were searched in ClinicalTrials.gov and through contact with key authors in the fields of measurement tools in social work and neglect.
The original search terms can be found in our registered protocol. Additional relevant keywords identified during the searches were incorporated within a modified search strategy. A list of named instruments identified through a preliminary scoping search was included, and instruments identified through the review process were appended to the search. Information on these modifications and the number of studies identified in each search can be obtained from Simon Haworth (s.p.c.haworth@bham.ac.uk).
Criteria for Considering Studies for This Review
Prior to starting the search, inclusion, and exclusion criteria were determined as follows:
Population: Children aged 0–18 years referred to children's social work services, and parents or caregivers of these children.
Dates of studies: No limits set.
Language of the studies: No restrictions applied. We only reviewed studies in English (due to resource/time constraints). Studies in other languages that may be relevant have been listed.
Tools or measures: Tools must have been designed for children aged 0–18, with suspected neglect who have been referred to children's social work services, or for the children's parents or carers. The tool had to ascertain at least one form of child neglect. Screening tests were not assessed, because the evidence, although weak, suggests unacceptably high false positives (McTavish et al., 2020). For further details on this criterion, please refer to the study protocol.
Evidence included: No restriction was set on the type of study; published and unpublished material was reviewed. For evidence of diagnostic accuracy, only cross-sectional studies, and index or test measures involving the target population with a contemporaneous (± 3 weeks) comparison of a (gold standard) social work assessment was included. All included studies were quality-assessed against this criterion.
Studies that did not meet the inclusion criteria were excluded. They fell into one of the following categories:
no comparison with the gold standard of a social work assessment, no child neglect, not a tool or measure for child neglect, wrong setting (e.g., a medical setting), wrong population (e.g., tool used retrospectively with adults), assessment of future risk not current measurement of neglect, and small sample size (less than 10 subjects).
Because of a paucity of studies, we modified our protocol to include measures with retrospective comparisons with a social work assessment.
The Gold Standard for This Review
We compared and evaluated tools against the gold standard of an assessment by a qualified children's social worker or assessor working within children's social work.
Social work assessments can be of variable quality. However, as discussed, a thorough social work assessment remains the best tool available in the field, and these assessments are completed by trained professionals, in line with government laws and policies, and subject to quality control mechanisms (Dubowitz et al., 2005; Holland, 2010; Munro, 2020). The advisory group to this study and existing guidelines indicated that these remain the best and most commonly used tools currently in practice; offering comprehensive and holistic assessments of people within their environments (Boyd Webb, 2019; DfE, 2018a; DoH, 2000; NASW, 2013).
Social workers are the lead professionals that identify and intervene in child neglect, both in the UK and North America (where the included studies were conducted) (Horwath, 2013; Stevenson, 2007). Social work is a key profession that engages with neglect in many countries globally (UNICEF, 2021; World Health Organization [WHO], 2020). In the United Kingdom and United States, undertaking high-quality informed and holistic assessment focused on people in their environments is a key competency for practice (British Association of Social Workers [BASW], 2022; NASW, 2013).
A range of established and validated measurement tools have been tested against the gold standard of clinical and professional assessments completed in the field in a range of countries and settings. This includes the Beck Depression Inventory (I and II) (Gomes-Oliveira et al., 2012; Wang & Gorenstein, 2013; Williams et al., 2021); the Child Behavior Checklist (Ebesutani et al., 2010; Nakamura et al., 2009; Skarphedinsson et al., 2021); and the Children's Depression Inventory (de la Vega et al., 2016; Sorensen et al., 2005; Wu et al., 2010).
Social work assessments have been used as a comparator for validation in other studies. This includes King et al. (2013), who compared a structured assessment tool with social work assessments, Smith et al. (2015), who used social work assessments as comparators in the development of a screening tool, and Flood et al. (2005), who used assessments by social workers as a comparison when assessing the Community Dependency Index. Further, arguments have been made that social work research underuses practice or clinical information such as assessments, closely linked as it is to practice realities and key concepts used in real-world practice (Epstein, 2001).
Review Procedure
Identified records were stored and screened on the Rayyan QCI database for systematic reviews (Ouzzani et al., 2016). The primary review author reviewed the title and abstract of each record, based on the inclusion criteria. A random sample of 25% of these records was independently reviewed by a second review author. Second review authors reviewed 100% of the first reviewers’ decisions. Full texts of all potentially relevant articles were obtained and reviewed. Duplicate records were removed through each subsequent database search and on further checking. Figure 2 presents the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of the evidence selection process.
PRISMA flowchart.
Data Collection and Analysis
Data Extraction
We extracted data from retrieved articles and studies using a piloted data extraction template designed for this review. The template consisted of (a) face validity, (b) key properties and risk of bias, (c) diagnostic accuracy testing, and (d) desired properties (developed from the recommendations of the advisory group, to ensure practice relevance).
One review author extracted the key information on the template, which was then verified by a second reviewer. Any disagreements were addressed through discussion and consensus.
Assessment of Risk of Bias/Study Quality
We adopted a multistep approach to assess study quality. First, we assessed which tools measured which domains of child neglect. Any measures with no face validity were screened out. Next, we critically appraised the evidence for measures with some relevance.
Evidence was classified into the study method employed and the relevant CASPs (Critical Appraisal Skills Programme) checklist was used to assess the risk of bias (CASP, 2018). One review author assessed and a second then validated the assessment. Any disagreements were resolved through discussion and reaching a consensus.
We distinguished between levels of evidence (type and quality of evidence available based on how well tests have been performed, on whom, in which settings, and against which other tests or assessments) and reported properties of tests including reliability, validity, accuracy, and precision; then included measures were inspected for further properties of content validity, reliability, accuracy, interpretability, and sensitivity to change. We placed importance on the concurrent validity of tools, as “concurrent validity is the most appropriate form of criterion validity to examine when the aim is to make inferences on the psychometric quality of an instrument” (Vial et al., 2020, p. 108).
We used the definitions used in the COSMIN framework for key characteristics of good measures (Mokkink et al., 2010). They are as follows:
validity, reliability, responsiveness or sensitivity to change, and interpretability. simple and easy to use, child-focused, able to be used throughout the organization (from the front door to long-term work with children and their families), and functional for different service areas, identifies the type of support needed, and designed for the needs of families and of the professional/organizational system.
Following these quality assessment steps, we sought the views of the advisory group on the following criteria:
Missing Data
The effect of missing data was assessed under risk of bias. There were two missing results in Trocme's study, and these were judged unlikely to have significantly altered the study's findings.
Collaborative Approach
This review adopted a collaborative approach with practice partners, Birmingham Children's Trust. User involvement has been in the form of advisory and stakeholder groups. For this review, “user” has been defined as a social worker, the individual using the neglect measurement tools. The advisory group consisted of nominated social work staff from Birmingham Children's Trust and gained the views of service users recruited through the Trust. This was achieved through social workers seeking their views and subsequently sharing these with the group. The stakeholder group also includes key academics in the field, services users, and social workers from other organizations. The additional element of user involvement has been used previously by other systematic reviews (Hyde et al., 2017; Oliver et al., 2014; Pollock et al., 2015).
The review has been conducted at the partnership level on Arnstein's (1969) Ladder of Citizen Participation. There were four advisory group meetings, where responsibilities, including idea generation on the quality and relevance of measures, were shared. Their involvement helped orient the review and promote the relevance of findings (Esmail et al., 2015).
Results
In total, 5,109 records were reviewed. Just four studies met our inclusion criteria (Figure 2). We were unable to access 12 studies, which are described in Table 1. For one of these references, we were able to find the data in an alternative article. We made requests through our institution and the British Library for all articles written in English.
References not Reviewed.
Description of Included Studies
Study Characteristics
There were four cohort studies reviewed and analyzed, all based in high-income countries (Canada and the United States): a study by Trocme (1996) on the Child Neglect Index (CNI) and studies by Dubowitz et al. (2005), Runyan et al. (2005), and Mennen et al. (2010) on modifications of the Modified Maltreatment Classification System (MCS or MMCS). The studies were completed between 1996 and 2010 and in total included 1,715 cases. The children in these studies were all considered at risk or having suffered harm. Table 2 sets out the key characteristics of these studies.
Characteristics of Included Studies.
Note. CWBS = Child Well-Being Scales CPS = child protection services; NIS-2 =National Incidence Study 2.
Whereas Trocme's (1996) study aimed to develop a short, valid, and reliable measurement instrument for the type and severity of neglect in Canada, Dubowitz et al.'s (2005) and Runyan et al.'s (2005) studies formed part of a larger longitudinal study in the United States examining antecedents and outcomes of child abuse and neglect. Dubowitz et al. (2005) used the MMCS to retrospectively rescore and reclassify neglect from CPS records. Runyan et al.’s (2005) companion paper compared the concordance of the main types of child maltreatment classifications defined by CPS’ official codes to two types of alternative classification systems—the MMCS and the National Incidence Study 2 (NIS-2) (the NIS-2 is a further research tool, not meeting our inclusion criteria). The outcomes measured were those reported by Dubowitz et al. (2005). Mennen et al. (2010) used the same approach to reclassify neglect and maltreatment, and to measure co-occurrence in 9- to 12-year olds in the care system.
Trocme's (1996) study was the smallest, focusing on 127 consecutive “intake” investigations. The sample population for the Dubowitz et al. (2005) and Runyan et al. (2005) studies were children and their primary carers from four research sites in eastern, southern, midwestern, and northwestern parts of the United States. These sites differed in terms of sample populations, with some children at risk for, and some having suffered maltreatment. Cases were described as routine CPS cases; these were collected and reclassified by research staff, not social workers. Mennen et al.’s (2010) study focused on 303 cases of children identified as maltreated by a public child welfare agency.
Study Designs
The four studies that met our inclusion criteria all used variations of cohort study designs. Dubowitz et al.'s (2005) and Runyan et al.'s (2005) studies included follow-up data, whereas Mennen et al.'s (2010) and Trocme's (1996) stand-alone studies did not.
Trocme's (1996) study incorporated a two-stage process. Stage one involved gaining expert views on index construction from practitioners in the child welfare field. Stage two involved field testing within a social work setting. Classification of neglect using the CNI was compared to maltreatment classifications of the NIS child protection worker survey form. Concurrent validity was assessed against 14 neglect-related scales from the Child Well-Being Scales (CWBS) (n = 125 for each scale). Test–retest reliability was assessed by workers completing the tool twice within a 2-week period.
Dubowitz et al. (2005) examined 481 CPS records to determine how the MMCS, capturing six subtypes of “failure to provide,” three of “lack of supervision,” and frequency of reports of each, compared to two CPS classifications of neglect defined as “general neglect” and “caregiver absence.’ They also examined how well the categories and subcategories of each predicted a range of child outcomes ascertained at the age of 8 years from a set of standardized measures for all 740 children. Runyan et al. (2005) attempted to answer two questions. First, how did MMCS classifications compare with CPS and NIS-2 classifications of the main child maltreatment categories of physical abuse, sexual abuse, neglect, or emotional maltreatment and second, how well did the various categories of abuse predict child problems at the age 8 years. Mennen et al. (2010) extended the MMCS tool to also include caretaker incapacity and child “at risk” from neglect and/or abuse.
All four studies met our gold standard comparison with a social work assessment through different routes: Trocme's (1996) study through the social workers completing the CNI when they completed their standard assessment reports; Dubowitz et al.'s (2005) and Mennen et al.'s (2010) studies through comparing a modified MMCS with CPS records of assessments by child protection workers; and Runyan et al.'s (2005) study through CPS data being compared to the MMCS tool and NIS-2 data, with the MMCS viewed as their gold standard. For these three studies, comparisons were made between CPS classifications decided by CPS workers and reclassifications of narrative data by trained research assistants, not social workers.
Types/Subtypes, Severity, and Chronicity of Neglect
Our review focuses on six subtypes of neglect: emotional, medical, physical, educational, social, and lack of supervision or guidance. No tool assessed all of these, but the CNI assessed medical, physical, and educational neglects, as well as lack of supervision and guidance. Emotional neglect was partially measured under “mental health care,” but social neglect was not captured. Neglect severity was measured, but chronicity was not. The tool would be easily repeatable for measuring the change in cases, but the age-weighted component would have to be disregarded.
The MMCS measured medical and physical neglect, and lack of supervision or guidance. Facets of neglect severity and chronicity were captured, but only partially, in Dubowitz et al.'s (2005) study. No data on severity or chronicity was provided in Runyan et al.'s (2005) study. Additionally, the MMCS would not be easily repeatable for measuring the change in cases, as it takes considerable time and effort to complete.
Quality of Evidence and Risk of Bias of Included Studies
There was no selection bias identified in the Trocme (1996) study: it contained 127 consecutive intake cases. Two missing results were judged unlikely to have significantly altered the findings. The tool is simply designed and there is no reason to suspect any other measurement bias. However, validity assessment against the NIS classifications was not blinded: repeat CNI and CWBS assessments were completed by the same worker up to 2 weeks later, raising the possibility of these being influenced by social work case decisions. Trocme (1996) provided no information as to whether the CNI score influenced workers’ decisions. It is possible that Trocme's results were influenced by the results of the reference tools.
There was variation in the sample population within Dubowitz et al.'s (2005) study, some with CPS records (n = 481) and some not (n = 259). However, all children used for comparison had CPS records. Sixty-five children were excluded from the sample mostly because of omissions in data, but the number of children lost to follow-up was unstated. Records were retrospectively re-coded using the MMCS by trained research assistants, but separate simultaneous coding by social workers using the MMCS independently and blind of CPS findings was not carried out. It is possible that more cases may have been classified as neglect using MMCS than through the CPS definitions. There was therefore potential for selection bias. Mennen et al. (2010) counted the number of children classified and not classified as subject to neglect by experimental and control methods, meaning that selection bias was limited only by willingness to take part in the study.
To compare CPS classifications with MMCS and NIS-2 codes, Runyan et al. (2005) re-coded CPS data for each maltreatment report into MMCS and NIS-2 codes. Only those CPS records with a single, valid CPS classification of maltreatment were included, resulting in 35% of records being excluded. This left the study open to selection bias. The number of CPS negative/MMCS positive, or CPS negative/NIS-2 positive, could not be ascertained. The exclusion of multiple maltreatment cases and manner of reclassifications into one subtype left the study open to measurement bias. Within the study, being placed lower down their hierarchy of abuse (as follows: sexual abuse, physical abuse, neglect, and emotional abuse) translated into being less likely to be classed as the predominant type of abuse.
Trocme (1996) recognized that the assessment of neglect is complex, but provided a limited discussion of potential confounding factors. The child's age was considered in the CNI, with higher scores added to the index for younger children but confounding factors such as worker issues, family issues, difficulty disaggregating neglect from poverty, or issues in the NIS and CWBS tools against which the CNI was validated were not discussed.
Dubowitz et al. (2005) also recognized the complexity of assessing neglect, but no data were provided on other types of potentially co-existing maltreatment. For example, there was no assessment of differences in outcomes between children remaining in foster care and those who returned home, which may be a confounder for children's problems. Table 3 sets out the overall quality of the evidence of the included studies.
Overall Quality of Evidence.
Note. CNI = Child Neglect Index; CPS = child protection services; CWBS = Child Well-Being Scales; MMCS = Modified Maltreatment Classification System; NIS =National Incidence Study.
Validity and Reliability of Included Studies
The results in Trocme's (1996) study indicated that the CNI has face validity. It is specific for neglect and measures neglect type and severity. The MMCS tool also appeared to have face validity. It measures neglect and Runyan et al.'s (2005) results indicated specificity for neglect.
The CNI was developed with the input of an expert panel and tested in practice. However, Trocme (1996) relied on practitioners’ substantiation and intervention criteria for neglect and Ontario's legal definition of neglect (Ontario Child and Family Services Act, 1984), meaning the understanding of neglect was context-specific. Academic and service user perspectives were not included. The MMCS or a further variation was applied retrospectively by Dubowitz et al. (2005), Runyan et al. (2005), and Mennen et al. (2010), and its development was not discussed. Cross-cultural factors were not discussed by any authors.
The CNI was tested against the NIS and CWBS which have not been evaluated as part of this review, but as with other abuse/neglect measurement tools, both have their own weaknesses and limitations. Concurrent validity scores were generally good for the CNI. The CNI correlated with the CWBS overall (inverse correlation 65%), with only the CWBS parent stimulation scale not correlating. Trocme (1996) compared the predictive validity of CNI and CWBS scores with the decision to provide ongoing child welfare services. Table 4 sets out the properties of the included measures and, as highlighted in Table 4, mean scores were higher for open than for closed cases. By comparison, the differences in the CWBS scores were very small. Structural validity is reported in Table 4.
Properties of Included Measures.
Note. MMCS = Modified Maltreatment Classification System; CPS = child protection services; CWBS = Child Well-Being Scales.
The MMCS was tested against the NIS-2. There was a kappa score of 0.743 for agreement between the MMCS and NIS-2 codes for neglect and a predictive value of 94% for the NIS-2, suggesting that the MMCS classification would also be neglect. Each MMCS neglect subtype was moderately correlated with CPS “general neglect.” There was limited or no correlation between MMCS subtypes and CPS “caregiver absence.” The findings were that the MMCS had an 83% positive predictive value for neglect. MMCS classification agreed with CPS for 82% of physical abuse, 90% of sexual abuse, 82% of neglect, and only 37% of emotional abuse cases. Structural validity of the MMCS is highlighted in Table 4.
Trocme (1996) provided partial data on reliability, as shown in Table 4. No data for the reliability of the MMCS over time were provided in either Dubowitz et al.‘s (2005) or Runyan et al.‘s (2005) studies. Dubowitz et al. (2005) and Runyan et al. (2005) found a 90% inter-rater reliability between assessors, with Runyan et al.'s score measured after training had been provided. Mennen et al.'s (2010) study provided limited data to add to our review.
No data were provided on the range of, or variation between, scores using the CNI, resulting in no data to enable assessment of precision. It was not possible to assess the precision of the MMCS from Dubowitz et al.‘s (2005) study. They provided p-values for study outcomes, but no SDs or confidence intervals. Runyan et al.'s (2005) study provided data for sensitivity, specificity, and positive predictive values. These suggested that a large number of cases recorded as neglect by CPS records were “false positives.” CPS scores were slightly more sensitive than MMCS scores in predicting child outcomes. The Runyan et al. (2005) study completed a regression analysis for outcomes measured in the Dubowitz et al. (2005) study and each of the classification systems. However, due to the methodology used, we have not rated the prospective validity of the study high enough to warrant detailed analysis.
How the Tools Perform Against our Desired Characteristics
We tested for the desired characteristics in a neglect measurement tool, based on the views of the advisory group as stated previously. Table 5 sets out the applicability of each tool for social work.
Applicability for Social Work.
Note. MCRAI (Mennen et al., 2010) is essentially the same tool as the MMCS, with two added categories. CNI = Child Neglect Index; MMCS = Modified Maltreatment Classification System.
The CNI is a short tool that appears simple to administer and comprehensible, whereas the MMCS does not meet these criteria (Table 5). Dubowitz et al. (2005) state regarding use of the MMCS that “findings in the present study do not support the considerable time and effort involved in abstracting and coding CPS records, at least for studying the frequency of reported types and subtypes of neglect” (p. 508).
Comprehensibility for social workers was not tested in any of the included studies. Trocme (1996) recognized that due to the CNI's brevity, accuracy and comprehensiveness could be questioned, but also stated with some justification that “…brevity of the CNI may simply reflect our limited knowledge of the characteristics of neglect and the lack of consensus about underlying constructs” (p. 150). The CNI performed well against the lengthier CWBS tool. The MMCS is a more detailed tool than the CNI, but covers fewer subtypes in this review's neglect typology, which raises questions about its comprehensiveness.
The CNI focuses on substantiating neglect rather than future risk. Neglect is assessed as categories ranging from adequate to seriously inadequate, with scorings applied, whereas the MMCS simply assesses neglect as present or absent. As shown in Table 5, the CNI appears to have more potential to be used across the stages of children and families social work than the MMCS.
The MMCS questions primary carers and children, while Trocme (1996) does not state who the CNI questions. None of the studies reported acceptability of the tools to children and families. Potential benefits, harms, and false positives and negatives were not reported by Dubowitz et al. (2005) or Trocme, but the CNI and MMCS are neglect-specific. Runyan et al. (2005) did report false positives, but the results were vulnerable to measurement and selection biases. Runyan et al.’s (2005) results would not be reliable for neglect if other types of maltreatment were also present.
Excluded Studies
Due to the small number of studies that met the inclusion criteria for this review, and to promote the review's rigor and transparency, we thought it important to discuss studies one might plausibly expect to find among the included studies, such as well-known neglect measurement tools, and studies that on the surface met the eligibility criteria, but on further inspection did not (Page, Cumpston, et al., 2020; Page, McKenzie, et al., 2020). Three studies were of significant interest but did not meet the inclusion criteria. They are shown in Table 6.
Excluded Studies of Interest.
Discussion and Applications to Practice
The aim of this systematic review was to examine neglect measurement tools that may be useful for children's social work. We examined the published and unpublished reports against strictly defined criteria of population, tool focus (neglect), evidence type, and comparison to a defined gold standard. We further examined the validity, reliability, and quality of the evidence base and key features of reviewed tool's usability and feasibility in practice. We synthesized the best evidence of the effectiveness of tools or measures for the assessment of child neglect.
This review revealed the limitations of the evidence base for social workers to assess child neglect. The overall evidence base for measures of child neglect can be considered weak. The most significant finding of the review is the lack of rigorous testing of potential measures for assessing child neglect. There is a paucity of high-quality evidence and robustly tested tools, with studies of “popular” tools lacking methodological rigor and robustness. This raises significant issues for social work assessments of neglect and the impact of child neglect means that the lack of valid, usable, and reliable measurement tools is a significant concern. In sum, only four studies met the inclusion criteria, with only one tool, Trocme's CNI, considered simple enough to feasibly be used in practice.
The findings suggest the need for robust testing of neglect measures in social work settings. Robust testing is important for the development of tools that can satisfy the criteria of validity, reliability, and practice/clinical utility. Child protection social workers’ time with children and families has been reduced through the COVID-19 pandemic, with in-person home visits becoming less frequent and shorter (Ferguson et al., 2020). This change in practice accentuates the need for assessments to be focused and feasible in terms of time and resources. Because of these changes and issues, it is timely to develop a new evidence-based, short, and easy-to-administer child neglect measurement tool.
Analysis revealed the gaps between the two included tools, the CNI and MMCS. The tools conceptualize and measure child neglect very differently, reflecting wider issues and imprecision around how neglect is defined and understood, but both present clear omissions and weaknesses. The CNI was designed for simplicity and brevity, while the MMCS was more complicated and cumbersome. The CNI captured a greater range of neglect subtypes identified in this review, but certainly not all of them. Notably, neither tool covered social neglect. Assessments that do not examine social neglect are not as holistic as would be preferred. The CNI captures neglect severity and recognizes that neglect should not be assessed dichotomously as present or absent, but neither tool effectively captures chronicity. Severity and chronicity are both key features of neglect for children's social work, given that children's services often become involved in situations of chronic and severe neglect (English, 1997). The CNI could at best be considered partially effective in measuring neglect. Trocme's (1996) study did not discuss cross-cultural factors and transferability to the UK context would need to be tested. The MMCS would not merit testing, due to the time and effort needed to complete it being unfeasible in practice.
As reported in the results section, there are significant concerns about the quality, validity, and reliability of the included studies. The findings of this review resonate with previous research by Yoon et al. (2021a), Vial et al. (2020), and Saini et al. (2019) into child maltreatment. As discussed earlier, their studies also found issues around validity, reliability, and usability, and suggested that the current evidence base is not sufficient.
This review has a number of strengths. It has followed Cochrane Collaboration recommendations (Higgins et al., 2020) (adapted to the field of social work), providing a rigorous and systematic approach. We systematically searched a range of multidisciplinary and science/social science electronic databases and search engines, as well as grey literature. Social work values and practice relevance have been promoted through the collaboration of an advisory group of practitioners. Development of a template specifically for this review has enabled clear and focused data extraction to answer the research questions. A multistep approach to assessing study quality has promoted rigorous analysis. Finally, the review has set out clearly what measurement issues are important and how to assess them.
Whiting et al. (2016) state that “bias occurs if systematic flaws or limitations in the design, conduct or analysis of a review distort the results” (p. 226). We undertook steps to minimize bias throughout this review, including following the protocol in full. This set clear eligibility criteria and laid out robust methods for the review, including the risk of bias of included studies being assessed by one review author and checked by a second. The adoption of a team-based approach with Birmingham Children's Trust was important for lowering bias (Uttley & Montgomery, 2017).
As with any study, this review has limitations. Setting a contemporaneous comparison to a social work assessment as a gold standard and limiting the review to studies published in English reduced the number of included studies. Further, it restricted the type of measurement instrument and excluded measures from linked professional fields such as health. Social work assessments can be of variable standards, although the advisory group and existing guidelines indicated that these remain the best tools currently available—thus we contend that our review gives greater rigor in its approach. Only 25% of identified records were independently reviewed by a second review author; however, inter-rater agreement at this stage was greater than 90% and thus we do not see this as a significant limitation.
It is reasonable that the findings from this review will be broadly generalizable to high-income countries. Although there is significant variation across time and between cultures as to what is considered abusive (Munro, 2020), there are international and cross-cultural aspects to the basic foundations of neglect as an unmet need. However, caution should be adopted in generalizing the findings to countries with significantly different economic, social, and legal contexts. Variations in definitions of neglect between countries further complicate the practice landscape and create issues for a consistent approach to neglect measurement.
Given the current evidence base for neglect measures, social workers should continue to undertake assessments based on established frameworks, such as the Framework for the Assessment of Children in Need and their Families (DoH, 2000). Current neglect measures are largely untested and should be used cautiously. Until reliable, valid, and usable measures are available, social workers should conduct full detailed assessments and not rely on measures whose validity, reliability, and neglect specificity are not robustly investigated.
Assessment of need as opposed to a singular focus on assessing risks should be adopted in practice, as neglect can be understood as an unmet need (Daniel, 2015). A risk-focused approach fails to fathom the relationship between the wider economic, social, and community contexts influential in neglect and practice, and can exclude effective assessment of needs and support for these to be met (Bilson & Hunter-Munro, 2019; Warner, 2015).
The current evidence based on measuring child neglect is too limited to effectively inform practice. The significant cost of neglect at personal, professional, community, and societal levels justifies the need for a thorough and robust research project to develop a new child neglect measurement tool. The study should be practice-informed and focussed on the development of a tool that is accessible and useable in practice Therefore, the tool should be designed with, as well as for, professionals and families. The development of an evidence-based, valid, and reliable child neglect measurement tool, rigorously tested in practice, is likely to improve the standards of social work assessments.
Any future neglect measurement tool will need to pay particular attention to validity, reliability, and relevance of the aspects measured. Further, it will need to capture neglect subtypes, severity, and chronicity. We suggest the clear neglect typology used in this review would be applicable. Trocme's (1996) study starts to demonstrate that measurement tools can be concise, but a fuller evidence base is required to have full confidence in this.
Future research should examine both needs and risks approaches for measuring child neglect to ensure a more complete evidence based on the costs and benefits of both approaches for families, practitioners, organizations, and communities.
Footnotes
Acknowledgments
The authors would like to thank Dr. Sarah Dawson for her expertise for the searches. We would like to thank Dr. Clio B. Weisman for her support with this paper, and members of our social work advisory group at the Birmingham Children's Trust for their advice and guidance. We would like to thank Birmingham Children's Trust for the partial funding of this project.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
