Red Flags for Early Referral for Infant Mental Health: Content Validation of a Tool

Abstract

Background:

There is growing evidence about the psychological distress and mental health issues experienced by infants with developmental problems. Problems related to developmental issues can be mitigated with early identification and intervention. A simple screening measure is needed to identify infants at risk for mental health issues to make informed decisions about further assessment and interventions. In this background, the present study aimed to develop a checklist to identify infants with psychological distress and at risk for mental health issues without the need to administer full-scale socioemotional tools and intended to be used by primary care professionals for early identification of risk factors followed by intervention to reduce the risk. Early identification of infants at risk in view of the limited resources would help in reducing the impact of early trajectories on the socioemotional health of any infant.

Aim:

The present study aims to develop a checklist for rapid screening. In view of the limited resources of low-income countries, early identification of infants at risk is important. This would help in reducing the impact of early trajectories on the socioemotional health of infants.

Methods:

The checklist was based on the “functional and emotional developmental approach” and a literature review of early indicators of psychological distress and mental health issues in infants. The checklist was prepared using objective terms which were further evaluated by mental health experts using standard methods. Accordingly, a content validity index (CVI) was calculated to determine the relevance and clarity of the items.

Result:

Based on the CVI, 16 items were retained in the final checklist, which was reported to have excellent relevance (CVI = 0.91) and clarity (CVI = 0.84).

Conclusion:

The checklist is concise with adequate content validity and has the potential to be used in primary care settings to screen infants at risk for psychological distress and mental health issues and make appropriate referrals.

Introduction

Socioemotional development comprises an infant’s capacity to experience, regulate emotions, and build a satisfying relationship with caregivers and others. ¹ Infant mental health, according to the World Association of Infant Mental Health ² , is the capacity to develop physically, intellectually, and socially in a way that enables a child to master the key emotional skills of early childhood without being severely disrupted by negative life events. Because infants develop in a framework of caring environments, the psychological balance of the infant-family system is important for infant mental health. This period is theorized as tremendous growth and development as the infant grows rapidly and profoundly in the first three years of life. ³

Early experiences can shape and mind, affecting lifelong health, behaviors, and learning. Common developmental difficulties that affect infants include separation anxiety, fear around strangers, problems with autonomy, difficulty controlling their emotions, and many others. Therefore, early behavioral and emotional issues (eg, inconsolable crying, sleep disturbance, aggressive or impulsive behavior, eating problems, etc) have implications for the mental health of infants. Untreated early emotional and behavioral issues can translate into difficulty relating to others, trusting significant adults, low motivation and learning, and calming themselves to tune into activities and skills necessary for overall development in later stages. ⁴

According to two Lancet series published in 2007 and 2011 on child development in developing nation, 219 million (or 39%) children under the age of 5 in low- and middle-income countries are at risk of not developing to their fullest potential. Early experiences during infancy profoundly impact brain development, affecting health, learning, and behavior. Failure to follow the expected trajectory of socioemotional development can lead to undetected mental and emotional health problems. ⁵ Adverse childhood experiences can alter development significantly. Emotional and behavioral problems affect a child’s overall development, particularly their adult academic and social outcomes. A child’s temperament, lack of adequate responsible care, poor opportunities, and health conditions can affect social and emotional development. ⁶

Infants’ socioemotional problems canbe picked up as early as a week after the child’s birth. Awareness of signs indicative of issues in your child’s social emotions willprevent long-term damage and receive therapeutic intervention as soon as possible. Despite the expert consensus supporting the need for early assessment, the specific socioemotional screening protocols at this young age are lacking. Some of the regular scalesused are the Infant Behaviour Questionnaire, ⁷ the Strengths and Difficulties Questionnaire, ⁸ the Ages and Stages Questionnaire, ⁹ and the Bayley Scale for Socio-emotional Development. ¹⁰ Many of these standardized screening tools require professional training but are not a quick measure to identify potential emotional and behavioral problems. Against this backdrop, the purpose of the study was to develop and content-validate a screening checklist to identify behavioral and emotional issues forearly identification and intervention. The checklist was created to make it easier and faster to identify children who might be eligible without the need to conduct extensive socioemotional testing, and it was created for use by main referral sources when recommending children for early intervention.

Method

This study is part of a larger, longitudinal study that focused on early intervention for infant mental health. It was approved by the Institute’s Ethics Committee on January 7, 2019. A multistep process was used to develop the checklist, which is as follows.

1. Identifying the Theoretical Model

The theoretical basis of this checklist is based on the functional and emotional developmental approach, ¹¹ which describes the critical emotional capacities that characterize development. This method postulates that affect and affective contact are what lead to emotional development. The other facets of development, such as motor, sensory, language, and cognitive functioning, are likewise organized by this method. As a result, the functional emotional development approach offers a method for defining emotional functioning as well as a method for examining how all the development-related elements interact when arranged in accordance with the designated emotional goals. It allows us to see not just the emotional abilities that are already there, such as emotional difficulties, but also the emotional abilities that aren’t there but need to be. According to this theory, emotional expression allows the entire spectrum of anticipated developmental capacities to cooperate effectively. ¹² This checklist has listed out specific emotional challenges faced by the 0 to 3 age group, which is critical for early intervention.

A draft checklist was prepared after the extensive literature was conducted to determine the early signs of infant mental health intervention needs based on information collected from the literature and interviewing the experts in the field. The literature on infant mental health and early signs of infant mental health issues are adapted based on guidelines by the centers for disease control and prevention. Draft preparation phase based on age-appropriate socioemotional development and common concerns reported by parents was listed. After multiple discussions amongst the researchers of this study, 16 items were shortlisted for further feedback by the experts.

2. Content Pooling and Preparing the Statements for a Checklist

The deductive method ¹³ describes the domain and the identification of items. This was achieved through a review of the literature, diagnostic classification, and evaluation of the scales and indicators that are currently used in that domain. The literature review also offers the theoretical framework for a thorough comprehension of problematic behaviors. Once the list of problem behaviors was prepared, the wording of the statement onthe checklist was designed to keep it simple, and unambiguous, and use minor technical terms. The focus of this preparation stage was that items on the checklist were consistently understood by professionals’ primary care workers and parents. Initially, the total number of items listed was 13. After multiple discussions with the coauthors, the total number of items was 16. After the consensus of the research scholar, guides, and joint guides, the first draft was prepared for content validation.

3. Content Validation by Experts

The methods recommended by Polit et al ¹⁴ for selecting and using content experts in the development checklist were followed. The experts were chosen based on their clinical experience and expertise. A total of 16 experts, based on their expertise, were approached via email to participate. These experts were included from a child care background, including a pediatrician, clinical psychologist, child psychiatrist, and psychiatric social workers for content validation based on relevance and clarity. Once experts gave their consent, a draft of the checklist was shared with them. Out of 16 experts, 4 gave permission to participate. Two clinical child psychologists and 2 child psychiatrists are among the experts. The red flag checklist’s entries were all submitted to content validation. The reason for inviting experts to participate was explained in a cover letter along with a checklist, which also included explicit guidelines for rating the checklist’s components. Each item was rated by experts based on its applicability and clarity of representation, as well as ideas for improving the items. To evaluate the items’ content validity, a content validity index (CVI) was computed. ¹⁴ The CVI shows the degree of consensus among experts on each component of the overall scale. Scales with an I-CVI cutoff of 0.78 and an S-CVI/Ave cutoff of 0.90 are considered to have high content validity, according to Polit et al. ¹⁴ The final checklist was used as a screener for PhD research at the Cross disability Early Intervention unit of the Composite Regional Centre at Lucknow. The checklist was circulated among the pediatrician, special-educators, physiotherapists, occupational therapists, and parents coming to the center. The total number of children aged below 3 years sent for psychological intervention was 38. Among these children, no new concerns were reported apart from what was included in the checklist. Hence, no modifications were made to the checklist.

Results

The CVI scores are shown in Table 1. All 4 raters rated all the items. Under the domain of relevance, the I-CVI ranged from 0.25 to 1, and S-CVI was 0.9. Item number 4, eventful perinatal histories with an I-CVI of 0.25, was discarded as it duplicated item number 1. Item numbers 3, 10, and 16 with an I-CVI of 0.75 was retained asthey were critical red flags. Raters suggested rewording the statements to avoid overlap with other items or to elaborate on them with more examples. In the second domain of clarity, the I-CVI ranged from 0.25 to 1, and S-CVI was 0.84. Item number 6 with an I-CVI of 0.25 and items 1, 3, 9, 10, 15, and 16 with an I-CVI of 0.75 were modified and retained as critical red flags. Raters suggested rewording the statements to avoid overlap and to elucidate with examples. Experts explained why some of the items were not relevant and clear. Fifteen items were retained out of 16 with revisions.

Table 1.

Content Validity Index (CVI) for Content Experts.

Domain	Items	I-CVI RANGE	S-CVI/AVE
Relevance	16	0.25–1	0.91
Clarity	16	0.25–1	0.84

Note: I-CVI, item-CVI; S-CVI, scale CVI; S-CVI/Ave, Scale CVI average.

Discussion

Every infant is different from every other infant, and some of the red flags may be present in infants with other developmental concerns; some signs warrant screening and evaluation of an infant’s overall development. Though there are various scales, like Ages and Stages Questionnaire: Social Emotional ¹⁵ ; Brief Infant Toddler Social Emotional Assessment ¹⁶ ; Child Behaviour Checklist List Preschool and Child Teacher Report form ¹⁷ ; Infant Toddler Social Emotional Assessment/Evaluation Measure (SEAM) and SEAM Family Profile ¹⁸ , they have measured wide domains of the developmental milestones and challenges. However, these tools are very good for a comprehensive assessment of the current level of functioning of infants, but it requires clinical skills and training in assessment. Additionally, socioemotional-behavioral issues are made up of a number of dimensions, or domains, including problem behaviors, social and emotional competence, emotional regulation, and healthy social interactions. As a result, it’s critical to have a tool that can effectively identify the relevant construct to explain children’s issues in the region. Hence, a brief checklist was needed for use by stakeholders, including frontline workers, professionals, and families of infants at risk for socioemotional problems. In this regard, the current study has identified specific red flags that could be used as a checklist for socioemotional concerns and making appropriate referrals for further evaluations and interventions (Table 2).

Table 2.

Red Flags for Children Birth to 3 Years Old.

National Institute of Mental Health and Neurosciences, Department of Clinical Psychology, Bangalore
Red Flags—For children birth to 3 years old		Tick
1.	Eventful perinatal history (eg, prenatal complications, abnormal birth details, postpartum illnesses).
2.	The child has not achieved age-appropriate developmental milestones, particularly in motor, language, and social domains.
3.	The child has a deficit in expected functional behaviors (eg, eating, toileting, dressing, poor play).
4.	Sudden loss or lack of progress in skill behaviors in the child (eg, motor skills, speech, concepts, self-help skills, etc).
5.	The child is displaying sensory issues in the form of excessively seeking or avoiding particular sensory stimuli. Child is displaying temper tantrums and excessive anger.
6.	The child is displaying repetitive behavior (eg, regurgitation, twirling, object manipulation).
7.	The child is displaying difficulty in paying attention or being overactive for the age.
8.	The caregiver has difficulty in feeding, soothing, or putting the baby to sleep.
9.	The child is not interacting with the mother or primary caregiver and not maintaining adequate eye contact with others.
10.	The child is excessively clingy to the primary caregiver or avoidant in general interactions.
11.	Playful and intimate interactions are either reportedly or observed to be inadequate.
12.	The mother or primary caregiver is either reportedly or observed to be not active in child care.
13.	The child is difficult to soothe as compared to other children of the same age.
14.	The child is either reported or observed to be engaging in too much screen time.
15.	The primary caregiver finds the baby difficult to manage.
16.	Others (as noted by the clinician).

Any psychological measure should have adequate reliability and validity, especially content validity as it is an essential step in developing a measure and it involves the judgment of experts as to whether a checklist includes appropriate content to capture the concept being assessed. ¹⁹ Generally, a minimum of 3 to a maximum of 10 experts are necessary for content validation, ²⁰ with an I-CVI of 0.78 or higher was considered a good indicator of interrater agreement. ¹⁴ Accordingly, 4 experts were considered for the content validation in the current study. The interrater agreement indicates that the checklist has good content validation, with an I-CVI of 0.91 on relevance and 0.84 on clarity.

Valid assessment is a very important component of early intervention which requires a skillful clinician in the identification of developmental deviation or any mental health issues. This may require considerable time in the form of taking a clinical history of the infants. Usually, checklists involve developmental assessment to understand the behavioral issues in a developmental context. However, it could be very resource intensive and has financial implications for the families. In this regard, the current checklist includes all the concerns generally reported by parents on a visit to the clinicians or identified by the clinicians working with children without the prerequisite of administering any developmental scale or multidisciplinary evaluation. This checklist can help professionals, caregivers, and families screen and refer children with socioemotional problems for early intervention services.

Ideally, developmental delays should be checked in all newborns and young children. However, it is not feasible in LAMI countries and low-resource situations where there is a lack of skilled labor and a great need. Further, it will be time-consuming. ²¹ Hence, screening may be considered as the first step to identifying any risk factor. ²² In this regard, it is anticipated that the checklist will be particularly useful for infants with subthreshold levels of problematic behaviors. Although issues listed cannot be seen in isolation but in relative occurrence to each other, and the context in which they are noted and reported. In this regard, the family context, especially the home environment, responsive caregiving, resources for meeting the socio-emotional needs of the infants, becomes relevant for further assessment and intervention.

Though the checklist appeared to be handy and easy to use, it has limitations However, a bigger sample that includes various child-care arrangements must be used to further analyze these findings. Second, the checklist had technical terms that may need some clarification when used with primary healthcare workers and parents. The checklist needs to be examined for internal consistency, test-retest reliability, and concurrent and criterion validity on a larger sample. Future research can explore the barriers to referrals and interventions.

Conclusion

Overall, the items in the checklist met the CVI requirement for establishing content validity. Establishing the content validity of the Red flag checklist canassist in identifying the socioemotional issues in the children age group of 1 month to 3 years old. This checklist can be used in various settings like primary health centers, Anganwadi centers, child-care institutes, child and adolescent psychiatry clinics, and pediatric clinics by multiple stakeholders.