Sage Journals: Discover world-class research

Abstract

Background

The Assessment Checklist for Adolescents Plus (ACA+) is a 30-item caregiver-report mental health and relational strengths measure, containing items that are the converse of maltreatment-related mental health and relational difficulties measured by the Assessment Checklist for Adolescents (ACA) and its short form (ACA-SF). The present study is the first to investigate the ACA+’s content validity and factor structure.

Methods

ACA+ and ACA-SF scores were obtained for 220 adolescents (50.9% boys), aged 12-18 years, living in residential out-of-home care (OOHC) in the UK. Item validity was estimated from item score distributions (item means and prevalence), and correlations with ACA+ total and ACA-SF scale scores. Factor structure was investigated using principal components factor analysis.

Results

As anticipated, all item-rest correlations were positive (r = .20 to .58), while correlations with the ACA-SF total score were all negative. However, items measuring emotion sharing and relational intimacy were positively correlated with several ACA-SF sub-scale scores, suggesting that among some adolescents these items measure maladaptive dysregulated emotion and relational difficulties. To maximise clinical utility, two sets of factor analyses were performed, yielding: a 21-item four-factor solution that can be safely interpreted by children’s agencies; and a 29-item six-factor solution that requires skilled interpretation by mental health clinicians. Factor analyses explained 53.9% and 75.1% of the score variance respectively.

Conclusions

The present analyses suggest the ACA+ has strong content and factorial validity for assessment of adolescents in residential OOHC. Further research is needed to establish its validity for use with adolescents in foster and kinship care.

Plain language summary

Background Adolescents in residential out-of-home care (OOHC) have high relationship and mental health difficulties. Clinical assessments mainly focus on symptoms rather than mental health strengths. But measuring young people’s mental health strengths can increase understanding of their treatment needs. The Assessment Checklist for Adolescents Plus (ACA+) is a list of 30 statements describing adaptive behaviours and mental health strengths that is completed by young people’s caregivers. The statements describe positive alternatives to mental health problems experienced by youth in OOHC, as measured by the Assessment Checklist for Adolescents (ACA) and its short form (ACA-SF). This study explored how well the ACA+ measures mental health strengths of adolescents in residential OOHC. Methods ACA+ and ACA-SF checklists were completed for 220 adolescents living in residential OOHC in the United Kingdom. The study investigated statistical relationships between mental health strengths and problems, and which strengths tend to co-occur with each other. Results The young people had high levels of mental health strengths, as well as difficulties. Individual strengths tended to co-occur with similar types of strengths. We found that some of the behaviours measured by the ACA+ were positive and adaptive for some young people but were possibly problematic for others. Conclusion The ACA+ checklists appear to accurately measure mental health strengths in this population. A shorter version with 21 questions and four scales (measuring ‘regulated emotion’, ‘relational security’, ‘empathy’, and ‘sociability’) can be safely used by children’s agencies without clinical oversight. A longer version with 29 questions (measuring ‘regulated emotion’, ‘relational security / contentment’, ‘empathy’, ‘relational closeness’, ‘emotional sharing/help-seeking’, and ‘sociability’) requires skilled interpretation by mental health clinicians.

Keywords

assessment checklist for adolescents plus ACA+strengths-based measure mental health measure out-of-home care residential care looked after children

Introduction

Children and adolescents residing in statutory out-of-home care (OOHC, including family-based foster care and kinship care, and residential care) constitute a highly disadvantaged and vulnerable population. Their early and chronic exposure to child maltreatment and other biopsychosocial adversities accounts for high levels and rates of mental health difficulties. Roughly half of such children have clinical-level mental health difficulties, and another quarter have difficulties approaching clinical significance (Engler et al., 2022; Oswald et al., 2010). Many manifest complex attachment- and trauma-related mental health and relational difficulties that are not well conceptualized within standard diagnostic classifications (DeJong, 2010; Tarren-Sweeney, 2021). These difficulties typically follow a long-term developmental course and have trait-like durability, such that children’s recovery tends to occur over much longer time periods than does recovery from more discrete, state-like disorders (Tarren-Sweeney & Goemans, 2019).

Identifying Early Signs of Mental Health Recovery

Most clinical and epidemiological studies carried out with this population measure their mental health using symptom checklists, notably the Achenbach Child Behavior Checklist (CBCL) and Youth Self Report (YSR) (Achenbach & Rescorla, 2001), and the Strengths and Difficulties Questionnaire (SDQ) (Goodman, 2001). These checklists measure numbers of symptoms and how often they are manifested, but not individual symptom severity. The symptom frequency timeframes are quite long – six months for the CBCL, YSR and SDQ. The lack of symptom severity measurement, and long measurement timeframes greatly constrains their ability to detect mental health changes over shorter periods – and hence, treatment sensitivity. Furthermore, measuring mental health improvement exclusively in terms of reduced symptoms and symptom frequency only tells half of the story. Recovery from maladaptive mental health difficulties is also demonstrated by increasing manifestations of adaptive behaviours, emotions, thoughts and relational styles that indicate normative psychological functioning – which can be termed mental health and relational strengths. For example, recovery from Reactive Attachment Disorder (RAD) can be measured not just from reductions in inhibited social interactions, but also from an increase in spontaneous affection towards family members. It is important to emphasize that recovery from maltreatment-related mental health difficulties does not, at least initially, involve a net replacement of maltreatment-related difficulties (symptoms) with strengths (the opposing and desired behaviours). Instead, foster carers often describe the earliest signs of recovery as their child displaying positive ‘uncharacteristic’ behaviours that surprise their carers (e.g. spontaneous, unsolicited affection), without necessarily observing a reduction in the converse problem behaviours (Tarren-Sweeney, 2014b).

Measuring Mental Health Strengths of Children and Adolescents in OOHC

While there are several measures of general psychological strengths that are potentially suitable to include in mental health and relational assessments of children and adolescents in OOHC (Day et al., 2022), very few explicitly measure mental and/or relational health. The Child and Adolescent Strengths Assessment (CASA) is a 30-item measure of psychological strengths designed to assist mental health service planning and delivery (Lyons et al., 2000). Strengths are assessed on six dimensions (family; school/vocational; psychological; peer; moral/spiritual; extracurricular). However, the domains and items don’t appear to explicitly measure positive mental health or indications of recovery. The SDQ includes a 5-item general prosocial behaviour sub-scale (along with four mental health symptom scales). The SDQ’s prosocial sub-scale appears to be the only strengths measure that is regularly included in clinical and epidemiological studies of young people in OOHC (Goemans et al., 2018; Goodman et al., 2004). The BERRI is a caregiver-report questionnaire designed to measure the mental health and well-being of children and adolescents in OOHC across five domains: Behaviour; Emotional wellbeing; Risk (to self and others); Relationships; and Indicators (of psychiatric or neurodevelopmental conditions) (Harris, 2019; Silver, 2025). Twenty strengths items were recently added to BERRI (personal communication with the developer).

The Assessment Checklist for Adolescents – Plus (ACA+) and the Assessment Checklist for Children – Plus (ACC+), are corresponding 30-item caregiver-report strengths measures for ages (12-18) and (ages 4-11) respectively. Their items were derived deductively, with a view to measuring adaptive behaviours, thoughts, emotions and relational styles that are the direct converse to maltreatment-related mental health and relational difficulties – as measured by the caregiver-report Assessment Checklist for Adolescents (ACA) and Assessment Checklist for Children (ACC) (Tarren-Sweeney, 2014a). The ACA+ and ACC+ share 27 items, with each checklist having only three unique items. The checklists employ a 4-point response scale (Never = 0; Rarely = 1; Sometimes = 2; Often = 3). The ACA+ and ACC+ have been distributed with the ACA and ACC checklists to licensed clinicians since 2016. To date however, neither the ACA+ or ACC + has been formally developed or validated.

While no research data have been published for the ACA+, two studies have published data for the ACC+. The first is a longitudinal study of mental health, attachment security and psychological strengths among children and adolescents in a therapeutic foster care programme in Ontario, Canada (N = 555) (Smith et al., 2022, 2024). The study commenced prior to development of a separate ACA+ measure, and the ACC+ was therefore used for both children and adolescents. Children in permanent foster placements had significantly higher ACC+ strengths scores than children in non-permanent placements, while there was no significant difference in ACC+ scores between adolescents in permanent versus non-permanent placements (Browne et al., 2019). Using growth curve and random intercepts cross-lagged panel models, the study found that: (1) whilst girls’ attachment security improved and boys’ attachment security worsened over time, both boys and girls ACC+ strengths increased over time; and (2) fewer insecure attachment behaviours predicted higher ACC+ strengths six months later (Smith et al., 2024).

The second study evaluated a group-based parenting programme for adoptive parents of UK children aged 2 to 8 years, who had been adopted from OOHC (Harold et al., 2017). The study measured 2-year pre-post changes in children’s mental health strengths and difficulties among an evaluation sample (N = 93), and a subsequent pilot sample (N = 157), using the ACC+, the Assessment Checklist for Children – Short Form (ACC-SF) (Tarren-Sweeney, 2014a) and the SDQ (Goodman, 2001). The study measured significant reductions in ACC-SF and SDQ total mental health difficulties scores (Cohen’s d = .43 and .21 respectively), and a significant improvement in ACC+ mental health strengths scores (d = .22) (Harold et al., 2017).

Background to the Present Study

The present study was conducted by the psychology service of the Keys Group, which is a large private provider of residential out-of-home care (OOHC) in England and Wales, in collaboration with two university researchers. Prior to this, the psychology service conducted a systematic review of strengths-based mental health measures appropriate for assessment of young people in residential OOHC (Day et al., 2022). The review concluded that the most suitable measures for routine assessment of this population were the ACA+ and the ACA/ACA-SF. Whereas the ACA and ACA-SF are formally developed measures with evidence of good content, construct and criterion-related validity, and high internal reliability (Tarren-Sweeney, 2014a), the review noted that the ACA+ “… is yet to be psychometrically researched. Further research is needed to assess the psychometric properties of the strengths-based supplementary checklist and to understand if this can be used as a standalone measure, or only in conjunction with the ACA” (Day et al., 2022). Following on from the systematic review, the provider’s psychology service included the ACA-SF and ACA+ in their standard assessment protocol, with a plan to evaluate their utility and accuracy with respect to their client group.

Given the need for validated mental health strengths measures suitable for routine assessment and monitoring of mental health recovery in adolescents living in OOHC, and the unique access to data from a large sample of adolescents living in residential OOHC in England and Wales, the current study therefore aimed to: (1) investigate the content validity and factor structure of the ACA+ strengths items among adolescents in residential OOHC; and (2) learn more about the levels of, and relationship between, mental health/relational problems and strengths in adolescents living in residential OOHC.

Method

Participants and Setting

Caregiver-reported ACA-SF and ACA+ checklist scores were routinely collected over an 8-month period (December 2022 to July 2023) for all adolescents (age 12 to 17) residing in 105 group homes in England and Wales, as part of their standard psychological assessments. Residents were young people accommodated by the statutory social care authorities and considered “children looked after”. Homes accommodated one to six young people. All young people had experienced maltreatment and other social adversity prior to entering care.

ACA+ and ACA-SF data were collected for 220 adolescents; 112 boys (50.9%) and 96 girls (43.6%), with gender not reported for 12 participants. Their mean age was 14.8 years (range 11.9 to 17.8 years), with boys’ and girls’ having similar mean ages (14.7 and 15.0 years respectively). Mean age at entry into the current OOHC event (i.e. excluding previous entries into OOHC) was 13.0 years (range 6.5 to 17.8 years).

Procedure

In accordance with the service assessment protocol, assessments of newly placed young people were carried out after they had spent at least 10 weeks in their new placement; to ensure carers had adequate time to gain enough knowledge and understanding of the child to reliably respond to the checklist items. Checklists were administered verbally by assistant psychologists in face-to-face or video interviews with the young person’s care team. The care team included those staff who routinely supported the young person within the home on a daily basis and were well known to them. The assistant psychologists entered checklist item scores on a central data spreadsheet, as well as the young person’s date of birth, date of entry into care (present care event), and gender.

ACA+ Item Pool

The ACA+ items are listed with the corresponding (i.e. opposing) ACC/ACA mental health difficulties scales in Table 1. None of the ACA+ items align with ACC/ACA ‘self-injury’, ‘suicide discourse’ or ‘sexual behaviour’ scales.

Table 1.

Unpublished ACA+ Item Pool With Corresponding ACC/ACA Mental Health Difficulties Scales

ACA+ items	ACC scale(s)	ACA scale(s)
1. Acts his age (not too grown up, or immature)	II, V	II
2. Comfortable with adults caring for him (e.g. not too independent for his age, doesn’t try to ‘parent’ his caregivers, allows adults to care for his siblings)	II, III	I, II
3. Copes well with frustrating circumstances		II, III
4. Confides in caregivers or family if he is upset or concerned about something	III	I
5. Confides in friends if he is upset or concerned about something	III	I
6. Enjoys chatting with family or friends	III	I
7. Expresses his feelings	III	I
8. He feels as though he belongs with your family, or with another family (doesn’t feel ‘alone in the world’)	II, III	I, II
9. Is close to you and/or another adult caregiver (seems attached)	II, III, IV	I
10. Is humorous, playful or light-hearted	III, V, VI	I, III, IV
11. Is kind or helpful to others	II, III	I, II
12. Is relaxed at home	III, V, VI	I, III, IV
13. Keeps friends (maintains friendships)	II, III, IV	I, II, III
14. Likes ‘hanging out’ with friends	III	I, III
15. Misses people when they’re away (e.g. if you’re away a few days, family living elsewhere)	III, IV	I, II
16. Relaxed about food (not preoccupied with supply of food)	VIII	V
17. Relaxed at bedtime	V, VI	III, IV
18. Remains calm in everyday (non-stressful) situations		II, III
19. Says when his feelings are hurt	III	I
20. Says when he is physically hurt or experiencing pain	VII
21. Seeks comfort or hugs from family or carers	III	I
22. Seems carefree (without concerns or worries)	III, V, VI	I, III, IV
23. Seems cheerful or happy	III, V, VI	I, III, IV
24. Seems genuinely sorry if he hurts someone’s feelings	II, III, IV	I, II
25. Seems secure	V, VI	III, IV
26. Sensible with strangers (appropriately wary, not overfamiliar)	IV	II
27. Shows concern for others	III, IV	I, II
28. Stands up for himself	V
29. Tolerates other people receiving attention or praise		II, III
30. Trusts the people he feels closest to (e.g. family or friends)	II, III, V, VI	I, III

ACC Scales

II. Pseudomature interpersonal behaviour.

III. Non-reciprocal interpersonal behaviour.

IV. Indiscriminate interpersonal behaviour.

V. Insecure interpersonal behaviour.

VI.Anxious – Distrustful.

VIII. Food maintenance.

ACA Scales

I. Non-reciprocal interpersonal behaviour.

II. Social instability/behavioural dysregulation.

III. Emotional dysregulation/distorted social cognition.

IV. Dissociation/trauma symptoms.

V. Food maintenance.

ACA-SF

The 37-item ACA-SF and 105-item ACA are caregiver-report mental health rating scales that measure problem behaviours, emotional states, traits, and manners of relating to others, as manifested by adolescents (ages 12 to 17) residing in various types of alternate care, as well as those adopted from care (Tarren-Sweeney, 2014a). They were designed to measure a range of maltreatment-related mental health and relational difficulties experienced by adolescents in OOHC that are not measured by generic checklists such as the CBCL and SDQ. Whereas the ACA is designed for comprehensive mental health assessments, the ACA-SF was designed for use as a brief assessment and monitoring tool. In addition to a total clinical score, the ACA-SF has six clinical sub-scales derived through factor analysis, labelled: Non-reciprocal (NR); Social Instability (SI); Emotional Dysregulation/Distorted Social Cognition (ED/DSC); Dissociation/Trauma Symptoms (D/TS); Food Maintenance (FM); and Sexual Behaviour (SB). The ACA-SF was employed in two recent studies of young people in residential OOHC (Eadie et al., 2020; Maneiro et al., 2025).

Statistical Analysis

Item prevalence rates were calculated for two dichotomous variables: item prevalence defined by a score of 3 (occurs ‘often’); and item prevalence defined by a score of 2 or 3 (occurs ‘sometimes’ or ‘often’). Item-rest correlations (Pearsons r) were calculated, with rest scores being nominal total scores (sum of 30 item scores) minus the relevant item score. Correlations were also calculated for the ACA+ items with the ACA-SF total and sub-scale scores (excluding the ‘sexual behaviour’ scale, since no ACA+ items align with it). For the purpose of item analysis, well-performing strength items should be at least moderately, negatively correlated with mental health difficulties items. Principal components factor analyses of retained ACA+ items were performed in STATA 16 (StataCorp, 2019), using the same factor analysis procedure used to develop the Assessment Checklist measures (Tarren-Sweeney, 2014a).

Ethics Approval

The study data were routinely collected within standard psychological assessments of young people residing in the Keys Group residential homes. Use of anonymised data for the current research study was approved by the Keys Group’s internal ethics committee. All procedures performed were in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Results

Level of Mental Health Difficulties

Mean ACA-SF mental health difficulties scores are reported for boys, girls and the aggregate sample in Table 2. There were no significant differences between boys’ and girls’ mean scores.

Table 2.

Mean ACA-SF Scale Scores

ACA-SF scale	Mean scores (SD)
ACA-SF scale	Boys N = 112	Girls N = 96	All N = 220
Total score	20.8 (9.0)	22.4 (8.8)	21.5 (8.9)
Non-reciprocal	4.9 (2.9)	4.9 (2.9)	4.8 (2.7)
Social instability	6.3 (2.6)	7.3 (3.0)	6.7 (2.8)
Emotional dysregulation/distorted social cognition	4.8 (3.4)	5.3 (3.5)	5.0 (3.4)
Dissociation/trauma symptoms	1.3 (1.6)	1.7 (1.8)	1.5 (1.7)
Food maintenance	2.4 (2.7)	2.1 (2.8)	2.2 (2.7)
Sexual behaviour	1.2 (2.0)	1.3 (2.0)	1.3 (2.0)

Item Analysis

Item Means and Prevalence

ACA+ item mean scores and item prevalence rates are listed in Table 3. Item prevalence as defined by a score of 3 ranged from 8.6% to 70.0%, while item prevalence as defined by a score of 2 or 3 ranged from 46.4% to 93.6%. Very high item mean scores and prevalence reduce the instrument’s measurement utility, because they do not help to differentiate between children with low and high strengths and may thus be poor indicators of mental health recovery.

Table 3.

ACA + Mean Item Scores and Item Prevalence

Item #	Mean item score	Item prevalence (% scoring 2 or 3, i.e. Sometimes or Often)	Item prevalence (% scoring 3, i.e. Often)
Item #	Mean item score	%	%
1	2.0	75.5	27.3
2	2.5	90.0	60.5
3	1.4	48.2	8.6
4	2.1	76.8	32.7
5	1.4	46.4	20.9
6	2.6	90.0	67.3
7	2.1	75.9	32.3
8	2.3	83.2	51.8
9	2.6	91.4	69.1
10	2.5	93.6	59.6
11	2.2	87.7	36.4
12	2.5	91.8	58.6
13	1.4	47.7	17.7
14	1.8	60.0	35.9
15	2.2	80.0	43.2
16	2.0	65.0	49.6
17	2.3	82.3	50.5
18	2.1	78.2	38.6
19	1.9	72.7	25.5
20	2.6	89.1	70.0
21	2.0	74.1	39.1
22	1.6	60.5	16.4
23	2.2	88.2	38.2
24	1.7	58.6	20.0
25	1.9	70.9	28.6
26	2.0	73.6	44.1
27	2.0	80.0	25.5
28	2.5	89.5	56.4
29	1.9	63.6	33.2
30	2.4	89.1	56.4

Item Correlations

Table 4 lists item-rest correlations, which were all positive and ranged from r = .20 to .58. Table 4 also lists item correlations with ACA-SF total and sub-scale scores. Small negative correlations with the ACA-SF total score (less than r = −.10) and positive correlations with ACA-SF sub-scale scores are highlighted in italics and bold font to indicate potentially problematic items.

Table 4.

Item Correlations (Pearsons r) With ACA+ Rest Scores, and With ACA-SF Total and Sub-scale Scores

Item #	ACA+ rest score^a	ACA-SF total score	NR^b	SI	ED/DSC	DTS	FM
1	.33	−.24	+.07	−.21	−.21	−.14	−.18
2	.47	−.13	−.23	+.01	−.09	−.04	+.02
3	.36	−.36	−.05	−.19	−.50	−.23	−.08
4	.43	−.02	−.35	+.25	+.08	−.01	−.05
5	.29	−.12	−.09	+.09	−.12	−.14	−.11
6	.42	−.26	−.23	.00	−.17	−.28	−.12
7	.37	−.06	−.41	+.21	+.05	.00	−.02
8	.39	−.31	−.45	−.01	−.21	−.16	−.11
9	.42	−.03	−.30	+.21	+.03	−.08	+.03
10	.50	−.20	−.20	.00	−.09	−.18	−.06
11	.58	−.20	−.29	+.07	−.10	−.16	−.08
12	.51	−.27	−.09	−.14	−.28	−.21	.00
13	.44	−.40	−.05	−.29	−.31	−.22	−. 26
14	.43	−.19	−.13	+.04	−.15	−.20	−.13
15	.32	−.03	−.14	+.13	−.01	−.05	−.01
16	.45	−.30	−.09	−.12	−.22	−.12	−.30
17	.36	−.33	−.11	−.23	−.29	−.21	−.12
18	.50	−.37	−.06	−.21	−.36	−.25	−.22
19	.24	−.01	−.34	+.21	+.17	+.06	−.07
20	.27	−.06	−.23	+.16	+.15	−.06	+.04
21	.39	−.01	−.43	+.26	+.12	−.01	−.06
22	.47	−.35	−.27	−.16	−.30	−.14	−.15
23	.57	−.36	−.43	−.05	−.22	−.27	−.09
24	.56	−.24	−.35	+.03	−.11	−.15	−.11
25	.48	−.36	−.23	−.18	−.33	−.18	−.09
26	.32	−.38	+.03	−.39	−.28	−.31	−.07
27	.49	−.11	−.24	+.15	−.05	−.09	−.09
28	.20	−.03	+.01	+.05	+.01	+.01	−.06
29	.38	−.44	−.02	−.35	−.37	−.26	−.25
30	.31	−.09	−.23	+.08	−.06	−.06	−.01

^aThe ACA+ ‘rest’ score is the nominal total score minus the item itself i.e. total of all other item scores.

^bACA-SF subscales: NR = Non-reciprocal; SI = Social instability; ED/DSC = Emotional dysregulation/distorted social cognition; DTS = Dissociation/trauma symptoms; FM = Food maintenance behaviour.

Item Retention for Factor Analyses

Item 28 ‘Stands up for him/herself’ had the lowest item-rest correlation and was positively correlated with most ACA-SF sub-scales. While this behaviour can suggest mental health recovery or adaptive coping in a residential setting for some youth, it can also measure maladaptive aggression among others. The item was therefore excluded from further analysis. Item 16 ‘relaxed about food (not preoccupied with supply of food)’ is the only item designed to measure recovery from ‘Food Maintenance’ difficulties and was therefore withheld from the factor analyses.

Emotion Expression/Emotion Sharing/Relational Intimacy Items

Of the remaining 28 items, several that measure adolescents’ expressed emotions, emotion sharing, or relational intimacy were: (1) moderately, negatively correlated with the ACA-SF NR sub-scale; (2) positively correlated with the SI and (to a lesser extent) the ED/DSC sub-scales of the ACA-SF; and (3) weakly, negatively correlated (r < −.1) with the ACA-SF total score. This suggests those items measure behaviours that reflect mental and relational health for some young people in residential care, notably those with high non-reciprocal scores. Paradoxically, they may constitute mental and relational difficulties for others, particularly those with high SI and ED/DSC scores. We considered the potential clinical significance of each of these items and decided that item 27 (‘shows concern for others’) and item 30 (‘trusts the people he/she feels closest to’) are unlikely to represent maladaptive behaviour for adolescents with high SI and/or ED/DSC scores – whereas items 4, 5, 7, 9, 15, 19, 20 and 21 may do. We considered excluding the latter eight items from the factor analysis, to ensure any derived sub-scales measure unambiguous and easy-to-interpret strengths. However, there is clinical utility in retaining items that measure mental health recovery for socially withdrawn, relationally non-reciprocal youth in OOHC. Doing so would necessitate a more complex scoring and interpretation system, and restricted licensing to skilled clinicians (as is the case for the ACA). On the other hand, excluding those items from the factor analysis would ensure sub-scales that are relatively easy to score and interpret, and that could be safely used by agencies and caseworkers without clinical training or clinical oversight. We therefore conducted two exploratory factor analyses with a view to identifying two potential versions of the ACA+: a longer version that includes the emotion expression-sharing, and relational intimacy items; and a shorter version that excludes those items.

Factor Analysis of the Larger Item Pool (27 Items)

In an initial exploratory factor analysis of the 28 retained items, item 2 ‘comfortable with adults caring for him/her’ loaded weakly across multiple factors, regardless of how many rotated factors were specified. We then performed a principal components factor analysis of 27 items (i.e. excluding item 2), with promax (oblique) rotations in STATA 16. Eight factors were initially retained after performing a scree plot test, each of which had an eigenvalue equal to or greater than 1. The best solutions were the 5-factor and 6-factor models. The 6-factor model included ‘regulated emotion’ and ‘relational security/contentment’ factors, while the 5-factor model merged those two factors. The 6-factor model provides the most clinically meaningful solution, and is also the most robust factorial model, accounting for 75.1% of the score variance. All items loaded greater than .45 on a single factor, except for items 12 and 29, which loaded weakly on multiple factors. We labelled the six factors (and their derived sub-scales) ‘regulated emotion’ (5 items), ‘relational security/contentment’ (4 items), ‘empathy’ (4 items), ‘relational closeness’ (3 items), ‘emotional sharing/help-seeking’ (5 items) and ‘sociability’ (4 items). The derived sub-scales are listed with the constituent items, item factor loadings, internal consistency (Cronbach’s Alpha) and sample mean scores and standard deviations in Table 5.

Table 5.

ACA + Factors/Sub-scales Derived From 27-Item Principal Components Factor Analysis: Sub-scale Internal Consistency and Sample Mean Scores, and Item Factor Loadings

Factors/Sub-scales/Other retained items	Factor loading	Mean sub-scale score (SD)
Factors/Sub-scales/Other retained items	Factor loading	Total N = 220	Boys N = 112	Girls N = 96	p
I. Regulated emotion (5 items, α = .68^a)		9.8 (3.1)	9.9 (2.8)	9.5 (3.2)	ns
1. Acts his/her age (not too grown up, or immature)	.57
3. Copes well with frustrating circumstances	.75
17. Relaxed at bedtime	.48
18. Remains calm in everyday (non-stressful) situations	.71
26. Sensible with strangers (appropriately wary, not overfamiliar)	.53
II. Relational security/contentment (4 items, α = .73)		8.1 (2.6)	8.1 (2.5)	8.1 (2.5)	ns
8. She/he feels as though they belong with your family, or with another family (doesn’t feel ‘alone in the world’)	.73
22. Seems carefree (without concerns or worries)	.65
23. Seems cheerful or happy	.53
25. Seems secure	.66
III. Empathy (4 items, α = .75)		8.4 (2.4)	8.1 (2.3)	8.8 (2.3)	.03
10. Is humorous, playful or light-hearted	.58
11. Is kind or helpful to others	.66
24. Seems genuinely sorry if he hurts someone’s feelings	.58
27. Shows concern for others	.80
IV. Relational closeness (3 items, α = .61)		7.2 (1.7)	7.0 (1.7)	7.5 (1.6)	.03
9. Is close to you and/or another adult caregiver	.76
15. Misses people when they are away	.49
30. Trusts the people he/she feels closest to (e.g. family or friends)	.78
V. Emotion sharing/help-seeking (5 items, α = .76)		10.6 (3.1)	9.6 (3.2)	11.7 (2.5)	<.001
4. Confides in caregivers or family if he/she is upset or concerned about something	.74
7. Expresses her/his feelings	.64
19. Says when her/his feelings are hurt	.70
20. Says when he/she is physically hurt or experiencing pain	.60
21. Seeks comfort or hugs from family or carers	.53
VI. Sociability (4 items, α = .72)		7.2 (3.0)	6.4 (3.0)	7.9 (2.9)	<.001
5. Confides in friends if he/she is upset or concerned about something	.66
6. Enjoys chatting with family or friends	.61
13. Keeps friends (maintains friendships)	.74
14. Likes ‘hanging out’ with friends	.78
Other items (4 items, excluded from factor analysis)
2. Comfortable with adults caring for him (e.g. not too independent for his age, doesn’t try to ‘parent’ his caregivers, allows adults to care for his siblings)
12. Is relaxed at home
16. Relaxed about food (not preoccupied with supply of food)
29. Tolerates other people receiving attention or praise
Total score (29 items, α = .87)		59.9 (12.0)	58.0 (11.5)	62.1 (11.9)	.005

^aCronbach’s alpha.

Factor Analysis of the Smaller Item Pool (19 Items)

The smaller item pool excluded the eight emotional expression/relational intimacy items (items 4, 5, 7, 9, 15, 19, 20 and 21). In an initial exploratory factor analysis of the 20 retained items, item 2 ‘comfortable with adults caring for him/her’ loaded weakly across multiple factors, regardless of how many rotated factors were specified. We then performed a principal components factor analysis of 19 items (i.e. excluding item 2), with Promax (oblique) rotations in STATA 16. Five factors were initially retained after performing a scree plot test, each of which had an eigenvalue equal to or greater than 1. The 4-factor model provided the most conceptually coherent solution. The 3-factor model merged ‘empathy’ and ‘relational security’ factors, while the 5-factor model was much less conceptually coherent than the 4-factor model. The 4-factor model accounted for 53.9% of the score variance. All items loaded greater than .4 on a single factor. We labelled the four factors (and their derived sub-scales) ‘regulated emotion’ (6 items), ‘relational security’ (5 items), ‘empathy’ (4 items), and ‘sociability’ (4 items). The derived sub-scales are listed with the constituent items, item factor loadings, internal consistency (Cronbach’s Alpha) and sample mean scores and standard deviations in Table 6.

Table 6.

ACA+ Factors/Sub-scales Derived From 19-Item Principal Components Factor Analysis (N = 220): Sub-scale Internal Consistency and Sample Mean Scores, and Item Factor Loadings

Factors/Sub-scales/Other retained items	Factor loading	Mean sub-scale score (SD)
Factors/Sub-scales/Other retained items	Factor loading	Total N = 220	Boys N = 112	Girls N = 96	p
I. Regulated emotion (6 items, α = .75^a)		12.1 (3.7)	12.5 (3.4)	11.7 (3.9)	ns
3. Copes well with frustrating circumstances	.69
12. Is relaxed at home	.57
17. Relaxed at bedtime	.63
18. Remains calm in everyday (non-stressful) situations	.72
26. Sensible with strangers (appropriately wary, not overfamiliar)	.52
29. Tolerates other people receiving attention or praise	.50
II. Relational security (5 items, α = .72)		10.5 (2.9)	10.5 (2.9)	10.6 (2.8)	ns
8. She/he feels as though they belong with your family, or with another family (doesn’t feel ‘alone in the world’)	.67
22. Seems carefree (without concerns or worries)	.55
23. Seems cheerful or happy	.43
25. Seems secure	.69
30. Trusts the people he/she feels closest to (e.g. family or friends)	.66
III. Empathy (4 items, α = .75)		8.4 (2.4)	8.1 (2.3)	8.8 (2.3)	.03
10. Is humorous, playful or light-hearted	.50
11. Is kind or helpful to others	.68
24. Seems genuinely sorry if he hurts someone’s feelings	.72
27. Shows concern for others	.87
IV. Sociability (4 items, α = .69)		7.7 (2.7)	7.3 (2.8)	8.1 (2.6)	.03
1. Acts his/her age (not too grown up, or immature)	.48
6. Enjoys chatting with family or friends	.63
13. Keeps friends (maintains friendships)	.75
14. Likes ‘hanging out’ with friends	.80
Other items (2 items, excluded from factor analysis)
2. Comfortable with adults caring for him (e.g. not too independent for his age, doesn’t try to ‘parent’ his caregivers, allows adults to care for his siblings)
16. Relaxed about food (not preoccupied with supply of food)
Total score (21 items, α = .87)		43.2 (9.7)	42.7 (9.4)	43.7 (9.9)	ns

^aCronbach’s alpha.

Inter-scale Correlations

Mean scores were calculated for sub-scales constructed from the derived factors. Correlations of sub-scale and total scores for the 29-item and 21-item ACA+ measures are listed in Table 7, while correlations of the 29-item and 21-item ACA+ sub-scale scores with ACA-SF sub-scale scores are listed in Table 8. Except for the ‘emotion sharing/help seeking’ score, sub-scale scores were moderately, negatively correlated with the ACA-SF total score (r = −.25 to −.55), providing evidence of construct validity. However, the ‘emotion sharing/help seeking’ score had zero correlation with the ACA-SF total score, due to contrasting negative (NR) and positive (SI and ED/DSC) correlations with the ACA-SF sub-scale scores.

Table 7.

Correlation Matrices (Pearsons r) of ACA+ Factor Scores (N = 220)

29-Item ACA+ scales (restricted version)	RS/C	E	RC	ES/HS	S	Total Score
Regulated emotion	.44	.37	.10	.05	.35	.66
Relational security/contentment (RS/C)		.51	.33	.25	.23	.72
Empathy (E)			.40	.43	.30	.76
Relational closeness (RC)				.40	.22	.54
Emotion sharing/help-seeking (ES/HS)					.26	.56
Sociability (S)						.61
21-Item ACA+ scales (unrestricted version)	RS	E	S	Total score
Regulated emotion	.50	.42	.44	.82
Relational security (RS)		.52	.25	.76
Empathy (E)			.35	.74
Sociability (S)				.65

Table 8.

Correlation Matrix (Pearsons r) of ACA+ and ACA-SF Scale Scores (N = 220)

	ACA-SF scale scores
	ACA-SF total	NR^a	SI	ED/DSC	DTS
29-Item ACA+ scales (restricted version)
ACA+ total	−.46	−.41	−.07	−.32	−.31
Regulated emotion	−.51	−.04	−.38	−.49	−.35
Relational security/contentment	−.46	−.45	−.14	−.36	−.24
Empathy	−.25	−.36	+.08	−.12	−.19
Relational closeness	−.36	−.29	+.19	−.02	−.08
Emotion sharing/help-seeking	.00	−.50	+.30	+.16	.00
Sociability	−.32	−.16	−.05	−.25	−.27
21-Item ACA+ scales (unrestricted version)
ACA+ Total	−.55	−.31	−.22	−.44	−.35
Regulated emotion	−.54	−.07	−.39	−.52	−.37
Relational security	−.43	−.46	−.10	−.34	−.23
Empathy	−.25	−.36	+.08	−.12	−.19
Sociability	−.37	−.11	−.16	−.29	−.29

^aACA-SF subscales: NR = Non-reciprocal; SI = Social instability; ED/DSC = Emotional dysregulation/distorted social cognition; DTS = Dissociation/trauma symptoms.

Discussion

The present study found that mental health strengths (as reported by caregivers) are highly prevalent in this high-risk sample. The present analyses provide initial empirical support for the item and factorial validity of the ACA+ mental health strengths items, yielding: a 21-item four-factor solution for general use by children’s agencies; and a 29-item six-factor solution that requires symptom-specific differential interpretation of sub-scale scores by mental health clinicians. Both versions demonstrate strong factorial validity, with the factor solutions explaining 53.9% and 75.1% of the score variance respectively.

The item analysis and inter-scale correlations identified a paradox, whereby some behaviours, thoughts and feelings likely indicate mental health and relational strengths for some young people, versus difficulties for others. In other words, some items are contextualised to the nature of a young person’s difficulties. In the present analysis, several ACA+ items measuring adolescents’ emotion expression, emotion sharing or relational intimacy were moderately, negatively correlated with the ACA-SF NR sub-scale, but positively correlated with the ACA-SF SI and (to a lesser extent) the ED/DSC sub-scales. This is borne out further in the inter-scale correlations. Three of the 29-item version sub-scale scores (empathy; relational closeness; emotion sharing/help-seeking), as well as the 21-item version ‘empathy’ sub-scale score, were positively correlated with the ACA-SF SI scale. Conversely, the ACA+ regulated emotion subscale was only weakly negatively correlated with the ACA-SF NR scores, but moderately negatively correlated with the remaining ACA-SF scale scores. Adolescents with high NR scores tend to be socially withdrawn and non-relational, and in some instances manifest signs of Reactive Attachment Disorder (RAD) (Tarren-Sweeney, 2013). For them, increased social-emotional expression and relational behaviours are unambiguously positive. However, whereas high ‘regulated emotion’ scores probably reflect a mental health strength for most young people, they are less likely to indicate recovery for those with high NR scores. Conversely, young people with high SI, and to a lesser extent ED/DSC, may manifest excessive and dysregulated emotions, an intensely insecure relational style, pseudomaturity, and/or signs of Disinhibited Social Engagement Disorder (DSED) (Tarren-Sweeney, 2014a). For these young people, high scores on ACA+ ‘relational closeness’ and ‘emotion sharing/help-seeking’ scales may partially reflect dysregulated emotion and/or relational difficulties. These results highlight why a 29-item version of ACA+ will require a detailed scoring and interpretation procedure that is restricted to qualified clinicians.

The present study results suggest that adolescents living in residential OOHC in the UK have considerable levels of mental health difficulties, with no differences between the sexes. The distribution of mental health difficulties is consistent with the type of care and child protection jurisdiction in which they reside – namely that residential care is largely reserved in the UK for those young people who are most difficult to place or retain in family-based care. They had roughly double the mean ACA-SF scores of the Australian CICS foster care sample (Tarren-Sweeney, 2014a), and of a ‘going well’ sub-sample of UK adolescents adopted from OOHC (Selwyn et al., 2014). Their mean ACA-SF scores were a little higher than those obtained for a sample of Spanish youth in residential care (Maneiro et al., 2025), which is consistent with a much greater proportion of Spanish OOHC youth being placed in residential care (44% in Spain, versus 11% in the UK) (Del Valle & Bravo, 2013). However, they had lower mean ACA-SF scores than that of an Australian clinic-referred sample (Eadie et al., 2020), and that of an ‘experiencing major difficulties’ sub-sample of UK adolescents adopted from OOHC (Selwyn et al., 2014).

A strength of the study design is that scores were obtained for all adolescents residing in the provider’s residential homes over an 8-month period. However, a limitation of the study is that the score data were collected within a single provider’s care homes, and the extent to which those young people are representative of the broader residential care population in England and Wales is not known. A second limitation is that the item validity and factor structures identified among the present sample’s ACA+ scores may not generalize to young people in family-based (foster and kinship) OOHC. Further research is needed to establish the extent to which these two proposed measures are valid for adolescents in foster care, or those adopted from OOHC.

In conclusion, the present analyses provide data for formal development of two versions of the ACA+ that are suitable for assessing the mental health and relational strengths of adolescents residing in residential care homes: an unrestricted 21-item checklist with four sub-scales that can be safely used and interpreted by care services; and a 29-item checklist with six sub-scales that will need to be administered and interpreted in conjunction with the ACA or ACA-SF by mental health clinicians.

Footnotes

Acknowledgments

The authors gratefully acknowledge the invaluable support and cooperation of Keys Group, a national provider of residential care in the UK, in facilitating their research. We are particularly indebted to the children’s residential clinical team at Keys Group and in particular, to Matilda Steele, Assistant Psychologist, for their crucial assistance in setting up the original database used in this study. Their dedication was fundamental to the data collection process and the successful completion of this study. Without their collaboration and generous access to their facilities and participants, this research would not have been possible.

ORCID iD

Michael Tarren-Sweeney

Ethical Considerations

The study data were routinely collected within standard psychological assessments of young people residing in Keys Group children’s residential homes. Use of anonymised data for the current research study was approved by the Keys Group’s internal ethics committee. All procedures performed were in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

There is no approval to share the study data.*

Author Biographies

Sarah Elgie is a Consultant Clinical Psychologist in the UK and specialises in supporting children looked after, and the adults who support them. My interests lie with developmental trauma and it's impact, and children living with adoptive parents, foster parents or residential care.

Astrid Røsland Seim is a child and adolescent psychiatrist working as a researcher and senior doctor in child and adolescent mental health care services. Her work focuses on understanding and supporting the mental health of children and adolescents in out-of-home care.

Michael Tarren-Sweeney is a clinical child psychologist, epidemiologist and child developmental theorist. His life’s work is focussed on understanding the psychological development and well-being of children and young people in statutory care, those who are adopted from care, as well as maltreated children who remain in their parents’ care.

References

Achenbach

Rescorla

(2001). Manual for ASEBA school-age forms and profiles. University of Vermont, Research Center for Children, Youth, & Families.

Browne

D. T.

Johnson

Beatty

Cameron

M. P.

Durham

Shlonsky

(2019). The therapeutic family care program: a 10-year community implementation of treatment foster care in Ontario, Canada. Developmental Child Welfare, 1(1), 22–40. https://doi.org/10.11772F2516103218815701

Day

Elgie

Robinson

(2022). Assessing social and emotional difficulties of children in residential care settings: A systematic review of strengths-based measures. Scottish Journal of Residential Child Care, 21(1), 2022.

DeJong

(2010). Some reflections on the use of psychiatric diagnosis in the looked after or ‘in care’ child population. Clinical Child Psychology and Psychiatry, 15(4), 589–599. https://doi.org/10.1177/F1359104510377705

Del Valle

Bravo

(2013). Current trends, figures and challenges in out of home child care: An international comparative analysis. Psychosocial Intervention, 22(3), 251–257. https://doi.org/10.5093/in2013a28

Eadie

Wegener

Bergh

(2020). Assessment of mental health difficulties in children and young people in care attending a specialist mental health service. Developmental Child Welfare, 2(4), 262–277. https://doi.org/10.1177/2516103220971296

Engler

Sarpong

Van Horne

Greeley

Keefe

(2022). A systematic review of mental health disorders of children in foster care. Trauma, Violence, & Abuse, 23(1), 255–264. https://doi.org/10.1177/1524838020941197

Goemans

Tarren-Sweeney

van Geel

Vedder

(2018). Psychosocial screening and monitoring for children in foster care: Psychometric properties of the brief assessment checklists in a Dutch population study. Clinical Child Psychology and Psychiatry, 23(1), 1–16. https://doi.org/10.1177/1359104517706527

Goodman

(2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry, 40(11), 1337–1345. https://doi.org/10.1097/00004583-200111000-00015

10.

Goodman

Ford

Corbin

Meltzer

(2004). Using the Strengths and difficulties questionnaire (SDQ) multi-informant algorithm to screen looked-after children for psychiatric disorders. European Child & Adolescent Psychiatry, 13(Suppl 2), 25–31. https://doi.org/10.1007/s00787-004-2005-3

11.

Harold

Hampden-Thompson

Rodic

Sellers

(2017). An evaluation of the AdOpt parenting programme: Department for Education.

12.

Harris

(2019). The structure and psychometric properties of the BERRI, an outcome measure for looked after children in residential care DClinPsych dissertation. University of Leicester.

13.

Lyons

Uziel-Miller

Reyes

Sokol

(2000). Strengths of children and adolescents in residential settings: Prevalence and associations with psychopathology and discharge placement. Journal of the American Academy of Child & Adolescent Psychiatry, 39(2), 176–181. https://doi.org/10.1097/00004583-200002000-00017

14.

Maneiro

Tarren-Sweeney

van Geel

Vedder

(2025). Navigating mental health complexity in residential care: Validation of the assessment checklist for adolescents – Short form. Developmental Child Welfare, 7(1), 22–44. https://doi.org/10.1177/25161032251324852

15.

Oswald

Heil

Goldbeck

(2010). History of maltreatment and mental health problems in foster children: A review of the literature. Journal of Pediatric Psychology, 35(5), 462–472. https://doi.org/10.1093/jpepsy/jsp114

16.

Selwyn

Wijedasa

Meakings

(2014). Beyond the Adoption Order: Challenges, interventions and adoption disruption: Department for Education. Retrieved from. https://www.gov.uk/government/publications/beyond-the-adoption-order-challenges-intervention-disruption

17.

Silver

(2025). Retrieved 10/4/2025 from. https://berri.org.uk/

18.

Smith

Durham

Beatty

Price-Cameron

Kartusch

Shlonsky

Browne

(2022). Trajectories of psychosocial functioning and attachment behaviors among children adopted in the Ontario child welfare system. Journal of Public Child Welfare, 16(3), 321–348. https://doi.org/10.1080/15548732.2021.1881690

19.

Smith

Durham

Price-Cameron

Sloss

Browne

(2024). The longitudinal interplay between insecure attachment behaviors and psychosocial strengths among children in child welfare services. Development and Psychopathology, 36(2), 578–588. https://doi.org/10.1017/S0954579422001419

20.

StataCorp . (2019). Stata statistical software: Release 16. College Station, TX. StataCorp LLC. [Statistical software].Version 16.0.

21.

Tarren-Sweeney

(2013). The assessment checklist for adolescents – ACA: A scale for measuring the mental health of young people in foster, kinship, residential and adoptive care. Children and Youth Services Review, 35(3), 384–393. https://doi.org/10.1016/j.childyouth.2012.12.005

22.

Tarren-Sweeney

(2014a). Clinician's guide to the Assessment checklist series: Specialized mental health measures for children in care. Routledge.

23.

Tarren-Sweeney

(2014b). Our twenty first century quest – Locating effective mental health interventions for children and young people in care, and those adopted from care. In Tarren-Sweeney

Vetere

(Eds.) Mental health services for vulnerable children and young people: Supporting children who are, or have been, in foster care, (pp. 37–57). Routledge.

24.

Tarren-Sweeney

(2021). Mental health symptom profiles of adolescents placed into foster care. Journal of Child & Adolescent Trauma, 16(2), 419–431. https://doi.org/10.1007/s40653-021-00417-2

25.

Tarren-Sweeney

Goemans

(2019). A narrative review of stability and change in the mental health of children who grow up in family-based out-of-home care. Developmental Child Welfare, 1(3), 273–294. https://doi.org/10.1177/F2516103219874810

The Assessment Checklist for Adolescents Plus (ACA+) Mental Health and Relational Strengths Measure for Adolescents in Statutory Out-Of-Home Care: Content Validity and Factor Structure for a Residential Care Sample

Abstract

Background

Methods

Results

Conclusions

Plain language summary

Keywords

Introduction

Identifying Early Signs of Mental Health Recovery

Measuring Mental Health Strengths of Children and Adolescents in OOHC

Background to the Present Study

Method

Participants and Setting

Procedure

ACA+ Item Pool

ACA-SF

Statistical Analysis

Ethics Approval

Results

Level of Mental Health Difficulties

Item Analysis

Item Means and Prevalence

Item Correlations

Item Retention for Factor Analyses

Emotion Expression/Emotion Sharing/Relational Intimacy Items

Factor Analysis of the Larger Item Pool (27 Items)

Factor Analysis of the Smaller Item Pool (19 Items)

Inter-scale Correlations

Discussion

Footnotes

Acknowledgments

ORCID iD

Ethical Considerations

Funding

Declaration of conflicting interests

Data Availability Statement

Author Biographies

References