Sage Journals: Discover world-class research

Abstract

Objective: The aim of the present study was to develop and validate an appropriate tool to assess the social and emotional well-being (SEWB) of Indigenous adolescents participating in the longitudinal Aboriginal Birth Cohort (ABC) Study.

Method: A range of tools was assessed as to the suitability of each for use in the ABC Study. Two existing tools and a newly developed one called ‘Strong Souls’ were piloted in a representative group (n = 67). Strong Souls was selected as the most appropriate for use in the ABC Study, and was completed by 361 participants. Exploratory factor analysis was used to explore construct validity. Cronbach alpha was used to assess the reliability of the latent constructs and the tool overall.

Results: Factor analysis produced a 25-item, four-factor model accounting for 34.5% of the variance. This model demonstrated sound construct validity and reliability. Factor structure was consistent with the epidemiological literature, identifying constructs of anxiety, resilience, depression and suicide risk. While these align with observations in mainstream populations, different relationships between distinct factors, and differences in symptomatology were found in this population. For example, two key findings were: feelings of sadness and low mood were linked with anxiety and not depression; and the expression of anger was verified as a unique symptom of depression for Indigenous people.

Conclusions: Strong Souls demonstrated validity, reliability and cultural appropriateness as a tool for screening for SEWB among Indigenous young people in the Northern Territory.

Keywords

Indigenous mental health psychometric evaluation screening instrument youth

The poor health status of Indigenous Australians has been well documented [1,2] and particularly their poor mental health status [3–5]. The first comprehensive collection of data on Indigenous mental health undertaken in Australia showed that the mortality for Indigenous youth was fourfold that of non-Indigenous youth; the leading cause being intentional self-harm or suicide [2]. Subsequent analysis based on these data show that increases in depression, anxiety and suicidal behaviours noted within Western cultures are mirrored in Australian Indigenous society [2,6,7].

Conceptualizations and experiences of mental health have been internationally recognized as being strongly influenced by culture [8]. The experience of disorders such as anxiety and depression are universal but the triggers, symptoms and understanding of these disorders vary among cultures [8–13].

Australian Indigenous cultures have been shown to be based on holistic concepts, with health defined in terms of the ‘… physical, emotional, cultural and spiritual wellbeing not only of the individual but of the whole community’ (p. 65) [14]. Indigenous perspectives of mental health include being in harmony with country, lawfulness, and social and kinship relationships. This belief system does not differentiate between body and mind, and is centred on an external locus of control [13–15]. To reflect these holistic concepts mental health in an Indigenous context is often referred to as social and emotional well-being (SEWB) [5,14,16]. Why individuals cope or fail to cope in adverse circumstances is complementary to SEWB assessment. Factors allied with the construct of resilience are consistently identified in the cross-cultural literature as an important feature in assessing mental health [17–20].

The increased awareness of the influence of culture has led to improvements in some aspects of health services, with practitioners being cognisant of cultural differences in their interactions with Indigenous people [12,13,15,21–25]. Mainstream mental health assessment and screening tools, or their ad hoc adaptations, continue to be used without appropriate consideration of cultural differences. This leads to test bias, inappropriate application of normative data, or depletion of test construct validity and reliability. There is a lack of tools available to validly screen, measure and assess SEWB among Indigenous people [5,26,27].

Since 1987 a life course study of an Aboriginal birth cohort (ABC) has been established in the Northern Territory [28]. At the second follow up of the cohort, the aim was to test the hypothesis that SEWB factors will predict biomarkers of chronic adult disease. The objective of the present study was to find or develop an appropriate tool to assess the SEWB of Indigenous youth participating in the ABC Study.

Method

Participants

Participants were all identified as Indigenous, representing significant cultural and linguistic diversity from a range of urban, rural and remote communities across northern Australia. Participants were mostly multilingual with English as a non-primary language and they generally had poor English literacy.

Pilot study participants (n = 67) were students from a local boarding high school in Darwin, aged between 13 and 19 years, who were demographically similar to the main cohort on factors including age, language, culture and geographical homelands.

For the main study, 460 participants of the ABC Study completed an extended battery of health measures that included the Strong Souls questionnaire. Of these, n = 93 did not complete the questionnaire and a further six participants were removed from the final dataset because they answered only a minority of the items. This left a final sample of 361 participants who completed the SEWB assessment, aged between 16 and 20.5 years (mean = 18.3 ± 1.1 years). The breakdown of the sample by gender was even between male (n = 169, 47%) and female (n = 192, 53%).

Ethics

The study was approved by the Menzies School of Health Research Ethics Committee and Aboriginal Ethics Subcommittee. All participants gave informed written consent for their participation. Participants could, at any time, decline to participate in some or all of the study.

Procedure

Tool selection and development

A literature review identified three SEWB screening tools validated for use with Indigenous Australians. These were the Strengths and Difficulties Questionnaire (SDQ) [29], the Westerman Aboriginal Symptoms Checklist for Youth (WASC-Y) [12], the Kessler Psychological Distress Scale (K10) and its abridged version the K6+ [30]. The SDQ was eliminated as an option because the age range (3–16 years), was too divergent from the ABC cohort. The (K6+) has demonstrated reliability in a young Indigenous population. The only tool developed and validated specifically for Indigenous youth was the WASC-Y. It was designed for an age range similar to that of the ABC Study, but has not been validated on populations outside of Western Australia. As such, both the (K6+) and WASC-Y were pilot tested alongside an instrument designed specifically to meet the needs of the study.

The first step in developing this new instrument was a comprehensive review of the Indigenous and general mental health literature using PUBMED and PsycINFO. This focused on diagnostic criteria and psychosocial factors relating to depression, anxiety, suicide risk and resilience. Based on this review a number of instruments and studies were used to inform the design of the new tool and a large preliminary pool of items [15,29–45]. All items were then reviewed in relation to their meaningfulness and appropriateness for the target group through a widespread consultation process with Indigenous people and Indigenous mental health experts. This consultation network was utilized during all stages of tool development with feedback used to eliminate or alter items. Through this process, the tool was named ‘Strong Souls’ in recognition that the concept of ‘soul’ encompasses a person's physical, emotional, social and spiritual being and was therefore synonymous with SEWB. Also, through this consultation process a four-choice, self-report Likert scale was decided upon as the most appropriate response format. For the mental health items participants were asked how often they felt or experienced symptoms in the past few months, that being: not much, little bit, fair bit and lots. A low score (not much) indicated minimal or no issues and a high score (fair bit or lots) indicated significant mental health issues. For resilience items, a low score indicated disagreement with the statement, indicative of lower levels of resilience, and vice versa.

Pilot testing

The WASC-Y and K6+ tools were pilot tested with the permission of the authors along with Strong Souls. Testing took place after school in a supervised study session at the school. Items were read aloud. Students under 17 years (n = 43) were asked to evaluate the tools by indicating if items made sense, or were inappropriate (providing comments as applicable). Results were used to assess the cultural and face validity of the three tools. Students over the age of 17 (n = 24) completed the tools by answering the questions. These data were used to assess discriminative power and internal reliability.

Both the Strong Souls and WASC-Y demonstrated good face validity on most items. Strong Souls demonstrated the best discriminative power with its four response choices, compared to the five choices on the WASC-Y and K6. Generally the internal reliability of the WASC-Y and K6+ were good with alphas >0.7, but not all the reliabilities for WASC-Y subscales could be established because of the small number of items, or low endorsement of some items. Strong Souls was divided into two subsets: SEWB and resilience. The SEWB subset demonstrated strong reliability (>0.7), and resilience demonstrated moderate reliability (>0.6).

The Strong Souls tool was selected because it demonstrated good initial discriminative power and reliability. Primarily, however, it was selected because it was developed for young Indigenous Territorians, with the help of these young people themselves. They provide feedback on the cultural and face validity of the questions. This feedback informed the final version of Strong Souls, which consisted of 34 items screening for problems related to depression, anxiety, suicide risk and levels of resilience.

Large-scale sampling

Strong Souls was one of several tests completed by participants in the second follow up of the ABC Study. Data were collected in diverse urban, rural and remote communities across the Northern Territory. Protocols were established for administering each test in the ABC study. Strong Souls was administered one on one, with an interpreter (usually a family member) if required. Questions were read aloud and the participant indicated their answer verbally or physically (e.g. pointing to the appropriate option). If participants answered with a high score on items relating to hopelessness and suicide ideation, they were referred for follow up to the local health clinic or mental health service (if present in the community). The need for this referral was noted in the data collected as a dichotomous variable: 0 = no follow up required, 1 = follow up required.

Data analysis

Among the 361 completed instruments all items had <5% missing data except for ‘You get used to big changes in your life fairly quickly’, indicating possible conceptual confusion relating to this item, which was therefore excluded from further analysis. After listwise exclusion of missing data the final sample size was n = 345 (final ratio was 11:1 participants to items). Outliers were not explored because the data were nominal. Data were negatively skewed with positive kurtosis for SEWB items, and the opposite pattern for the resilience items. Polychoric approaches, which allow for non-linear relationships between variables, were explored but in practice these analyses produce very similar outcomes to the more broadly utilized factor analysis. Preliminary data analysis identified no major distributional issues, and that the sample was adequate for factor analysis. Data skewedness was not considered to be problematic given the large sample size, the nature of the data collected, and the type of analysis to be undertaken. That is, assumptions of normality do not apply to certain types of exploratory factor analysis (EFA) [46,47]. Hence, EFA using the principal axis factoring extraction method was undertaken.

The symptomatology of mental health issues can often cross over so it was anticipated that there would be correlations between items. Hence, an oblique rotation was used to improve the interpretability of the factor solution. The rotation used was promax, which allowed, but did not necessitate, correlations between items to provide the optimally interpretable solution [46]. Because inter-correlations between items were allowed, the variance explained by the EFA using the promax rotation is partially shared across all the factors. Items with low correlations (≤0.3), and items with factor loadings <0.32 were eliminated. A correlation matrix for all 34 items is shown in Table 1. The factor loading level was set at 0.32 because this is identified in the literature as the minimum required before a loading is no longer contributing to the variance [48]. These criteria are consistent with the development of mainstream tools including the SDQ and the Kessler scales [49,50]. Cronbach alpha was used to assess the reliability of the emerging factors, and the tool as a whole.

Table 1.

Full correlation matrix for the oblique solution including correlations with factor and scale scores (n = 345)

	1	2	3	4	5	6	7	8	9	1	1	1	1	1	1	1	1	1	1	2	2	2	2	2	2	2	2	2	2	3	3	3	3	3	3	3	3	3	3	4	4	4	4	4
										0	1	2	3	4	5	6	7	8	9	0	1	2	3	4	5	6	7	8	9	0	1	2	3	4	5	6	7	8	9	0	1	2	3	4
1. F_score1_Anx	X
2. F_score2_Res	−0.22∗	X
3. F_score3_Dep	0.46∗	−0.21∗	X
4. F_score4_Suicide	0.46∗	−0.29∗	0.53∗	X
5. S_score_1 Anx	0.98∗	−0.2∗	0.48∗	0.43∗	X
6. S_Score_2_Res	−0.16∗	0.98∗	−0.14∗	−0.2∗	−0.14∗	X
7. S_Score_3_Dep	0.4∗	−.018	0.97∗	0.44∗	0.4	−0.12	X
8. S_Score_4_Suicide	0.52∗	−0.25∗	0.44∗	0.95∗	0.46∗	−0.16∗	0.36∗	X
9. S_Score_Mental Health	0.81∗	−0.25∗	0.88∗	0.66∗	0.81∗	−0.17∗	0.84∗	0.64∗	X
10. Suicide risk follow up	0.33∗	−0.27∗	0.53∗	0.52∗	0.32∗	−0.21∗	0.49∗	0.47∗	0.53∗	X
11. Have trouble sleeping	0.21∗	−0.08	0.62∗	0.29∗	0.23∗	−0.05	0.62∗	0.22∗	0.51∗	0.27∗	X
12. Get angry or wild real quick	0.16∗	0.05	0.61∗	0.18∗	0.17∗	0.05	0.64∗	0.14	0.48∗	0.21∗	0.28∗	X
13. Hard to focus, thinking all over the place	0.18∗	−0.05	0.51∗	0.21∗	0.18∗	−0.03	0.57∗	0.16∗	0.45∗	0.2∗	0.23∗	0.26∗	X
14. Had too many bad moods	0.21∗	−0.12	0.65∗	0.21∗	0.22∗	−0.07	0.63∗	0.18∗	0.51∗	0.26∗	0.30∗	0.34∗	0.23∗	X
15. Had really bad dreams	0.36∗	−0.11∗	0.3∗	0.26∗	0.35∗	−0.11	0.32∗	0.26∗	0.4∗	0,18∗	0.17∗	0.2∗	0.12	0.1	X
16. Felt pretty lonely most of the time	0.28∗	−0.22∗	0.59∗	0.27∗	0.28∗	−0.17∗	0.63∗	0.23∗	0.54∗	0.29∗	0.3∗	0.2∗	0.25∗	0.31∗	0.22∗	X
17. Some part of your body is always hurting	0.36∗	−0.1	0.39∗	0.2∗	0.37∗	−0.07	0.37∗	0.21∗	0.43∗	0.23∗	0.24∗	0.32∗	0.17∗	0.22∗	0.15∗	0.26∗	X
18. Felt so sad nothing could cheer you up	0.57∗	−0.06	0.51∗	0.23∗	0.65∗	−0.04	0.4∗	0.26∗	0.59∗	0.24∗	0.22∗	0.15∗	0.13	0.29∗	0.2∗	0.34∗	0.4	X
19. Felt so worried you start to shake	0.74∗	−0.18∗	0.27∗	0.32∗	0.73∗	−0.13	0.25∗	0.35∗	0.56∗	0.25∗	0.15∗	0.02	0.18∗	0.15∗	0.22∗	0.19∗	0.19∗	0.29∗	X
20. Felt so worried it was hard to breathe	0.84∗	−0.18∗	0.31∗	0.3∗	0.79∗	−0.14∗	0.27∗	0.36∗	0.6∗	0.21∗	0.12	0.1	0.15∗	0.18∗	0.2∗	0.2∗	0.29∗	0.4∗	0.56∗	X
21. Felt so worried you got really sweaty	0.62∗	−0.17∗	0.31∗	0.32∗	0.68∗	−0.11	0.26∗	0.33∗	0.54∗	0.2∗	0.16∗	0.18∗	0.08	0.16∗	0.18∗	0.12	0.26∗	0.24∗	0.44∗	0.5∗	X
22. Felt so worried you got sick in the guts	0.66∗	−0.1	0.41∗	0.31∗	0.7∗	−0.06	0.33∗	0.31∗	0.59∗	0.28∗	0.18∗	0.16∗	0.15∗	0.12	0.26∗	0.24∗	0.32∗	0.43∗	0.37∗	0.42∗	0.34∗	X
23. Felt so worried you got dizzy	0.78∗	−0.15∗	0.25∗	0.35∗	0.76∗	−0.12	0.23∗	0.37∗	0.57∗	0.18∗	0.13	0.11	0.1	0.06	0.28∗	0.13∗	0.2∗	0.37∗	0.51∗	0.52∗	0.39∗	0.47	X
24. Stopped doing things that used to be fun	0.28∗	−0.05	0.32∗	0.29∗	0.29∗	−0.01	0.3∗	0.27∗	0.37∗	0.22∗	0.19∗	0.14	0.16∗	0.16∗	0.24∗	0.19∗	0.9	0.2∗	0.21∗	0.2∗	0.26∗	0.2∗	0.17∗	X
25. Got wild and angry and stayed that way	0.38∗	−0.15∗	0.51∗	0.22∗	0.34∗	−0.12	0.55∗	0.21∗	0.53∗	0.28∗	0.22∗	0.29∗	0.19∗	0.24∗	0.27∗	0.26∗	0.24∗	0.23∗	0.17∗	0.18∗	0.19∗	0.27∗	0.21∗	0.12	X
26. Felt like everything is hard work	0.28∗	−0.05	0.32∗	0.29∗	0.29∗	−0.01	0.3∗	0.27∗	0.37∗	0.4∗	0.37∗	0.19∗	0.16∗	0.22∗	0.31∗	0.23∗	0.16∗	0.24∗	0.21∗	0.2∗	0.26∗	0.2∗	0.17∗	0.29∗	0.31∗	X
27. Been treated unfairly because you are Aboriginal	0.19∗	−0.01	0.22∗	0.12	0.17∗	0.00	0.22∗	0.15∗	0.37∗	0.12	0.13	0.94	0.13	0.11	0.17∗	0.21∗	0.09	0.15∗	0.08	0.17∗	0.08	0.18∗	0.06	0.17∗	0.17∗	0.9	X
28. Felt like you got no control	0.31∗	−0.14∗	0.38∗	0.31∗	0.32∗	−0.11	0.36∗∗	0.31	0.42∗	0.26∗	0.23∗	0.12	0.17∗	0.28∗	0.23∗	0.27∗	0.26∗	0.3∗	0.23∗	0.23∗	0.25∗	0.17∗	0.17∗	0.19∗	0.21∗	0.28∗	0.24∗	X
29. Felt like giving up	0.29∗	−0.2∗	0.63∗	0.56∗	0.29∗	−0.12	0.59∗	0.42∗	0.56	0.59	0.3∗	0.28∗	0.25∗	0.24∗	0.27∗	0.26∗	0.23∗	0.17∗	0.18∗	0.19∗	0.19	0.27∗	0.21	0.34∗	0.28∗	0.47∗	0.06	0.26∗	X
30. Wished you were dead	0.28∗	−0.26∗	0.38∗	0.92∗	0.26∗	−0.17∗	0.3∗	0.81∗	0.47∗	0.41∗	0.3∗	0.11	0.12	0.16∗	0.19∗	0.2∗	0.14∗	0.24∗	0.17∗	0.24∗	0.16	0.16∗	0.19∗	0.18∗	0.17∗	0.21∗	0.07	0.24∗	0.38∗	X
31. Felt like hurting yourself	0.6∗	−0.27∗	0.4∗	0.72∗	0.52∗	−0.19∗	0.33∗	0.84∗	0.61∗	0.43∗	0.2∗	0.13∗	0.15∗	0.19∗	0.2∗	0.2	0.31∗	0.36∗	0.44∗	0.37∗	0.34∗	0.34∗	0.41	0.18∗	0.21	0.43∗	0.13	0.24∗	0.37∗	0.51∗	X
32. Felt like killing yourself	0.36∗	−0.05	0.26∗	0.71∗	0.31∗	−.0.02	0.23∗	0.79∗	0.44∗	0.31∗	0.16∗	0.08	0.12	0.08	0.24∗	0.16∗	0.17∗	0.23∗	0.22∗	0.17∗	0.25∗	0.25∗	0.27	0.24∗	0.13	0.32∗	0.16∗	0.27∗	0.26∗	0.53∗	0.45∗	X
33. Have strong family who help each other	0.06	0.55	−0.28	−0.27	−0.27	0.52	−0.2	−0.17	−0.14	−0.2∗	−0.19∗	0.01	−0.14∗	−0.14	0.7	−0.19∗	−0.00	−0.00	0.02	−0.02	0.04	0.04	0.03	0.23∗	0.02	−0.07	0.01	−0.08	−0.02∗	−0.23∗	−0.14	−0.03	X
34. You get used to big changes in your life quickly	−0.23∗	0.23∗	−0.25∗	−0,24∗	−0.22∗	0.3∗	−0.21∗	−0.24∗	−0.28∗	−0.21∗	−0.23	−0.07	−0.1	−0.18∗	−0.07∗	−0.18∗	−0.15∗	−0.17∗	−0.14	−0.22∗	−0.13	−.14	−.16∗	−0.1	−0.09	−0.22∗	−0.03	−0.23∗	−0.09	−0.45∗	−0.23∗	−0.15∗	0.19∗	X
35. You know lots about white fella ways	−0.14∗	0.43∗	0.09	0.09	−0–09	0.53∗	0.04	0.02	−0.02	0.02	0.04	0.12	0.03	0.02	−0.14∗	−0.1	−0.03	−0.14∗	−0.17∗	−0.01	−0.01	0.12	−0.07	0.13∗	−0.06	−0.04	−0.04	−9.09	0.14	0.05	−0.01	0.03	0.13	0.05	X
36. You someone who is a really good person	−0.1	0.63∗	0.02	−0.11	−0.09	0.61∗	0.13	−0.08	−0.05	−0.09	−0.05	0.11	0.05	−0.01	−0.03	−0.04	−0.05	−0.07	−0.06	−0.06	−0.02	−0.02	−0.14∗	−0.02	−0.00	−0.05	0.09	0.01	−0.03	−0.1	−0.05	−0.05	0.27∗	0.19∗	0.13∗	X
37. You laugh and make jokes a lot	−0.13	0.52∗	−0.03	−0.13	−0.09	0.54∗	−0.04	−0.11	−0.09	−0.1	0.05	0.01	−0.03	0.02	−0.16	−0.02	−0.01	−0.09	−0.08	−0.08	−0.03	−0.03	−0.12	−0.08	−0.13	−0.10	0.03	10.00	−0.11	−0.12	−0.14	−0.02	0.17∗	0.08	0.27∗	0.22∗	X
38. You are really into something	−0.11	0.59∗	−0.06	−0.14∗	−0.07	0.57∗	−0.08	−0.13	−0.1	−0.17	0.04	−0.01	0.06	−0.07	−0.1	−0.13	0.03	−0.12	−0.07	−0.03	−0.06	−0.06	−0.06	−0.00	−0.11	−0.13	−0.03	−0.09	−0.1	−0.14∗	−0.09	−0.09	0.14	−0.02	0.27∗	0.25∗	0.31∗	X
39. You are a good son/daughter	−0.03	0.57∗	−0.2∗	−0.16∗	−0.04	0.55∗	−0.14	−0.11	−0.12	−0.17∗	−0.08	−0.04	−0.06	−0.1	−0.01	−0.1	−0.00	−0.04	−0.07	−0.07	−0.02	−0.02	−0.00	−0.03	−0.12	−0.08	0.03	−0.03	−0.09	−0.13	−0.11	−0.02	0.33∗	−0.13	0.14	0.3∗	0.25∗	0.26∗	X
40. People say you are really good at something	−0.14	0.40∗	−0.08	−0.08	−0.13	0.41∗	−0.06	−0.06	−0.01	−0.12	0.01	0.02	−0.03	−0.07	0.02	−0.04	−0.05	−0.1	−0.08	−0.14∗	−0.03	−0.13	−0.08	−0.01	−0.1	−0.05	0.1	−0.02	−0.06	−0.07	−0.09	0.02	0.12	0.29∗	0.22∗	0.32∗	0.2∗	0.23∗	0.31∗	X
41. You wish you were a different person	0.21∗	−0.15∗	0.18	0.2∗	0.21∗	−0.14∗	0.15∗	0.18∗	0.23∗	0.14∗	0.1	−0.01	0.15∗	0.12	0.1	0.1	0.04	0.16∗	0.14	0.16∗	0.13	0.13	0.2∗	0.06	0.08	0.14∗	0.08	0.18∗	0.14∗	−.15∗	0.14	0.16∗	−0.07	0.12	0.29∗	0.27∗	0.32∗	0.2∗	0.23∗	0.31∗	X
42. You got an older person looking out for you	−0.06	0.51∗	−0.16∗	−0.13	−0.07	0.53∗	−0.11	−0.09	−0.11	−0.12	−0.02	−0.05	−0.06	−0.04	−0.12	−0.09	−0.02	−0.08	0.01	−0.07	−0.05	−0.12	−0.09	−0.02	−0.08	−0.01	−0.07	−0.05	−0.09	−0.04	−0.02	−0.05	0.17∗	0.2∗	0.09	0.21∗	0.12	0.24∗	0.25∗	0.22∗	−0.18∗	X
43. You got lots of friends	−0.18∗	0.64∗	−0.1	−0.06	−0.15∗	0.58∗	−0.08	−0.07	−0.13	−0.15∗	−0.03	−0.01	−0.00	−0.05	−0.1	−0.09	−0.03	−0.16∗	−0.1	−0.19∗	−0.12	−0.05	−0.03	−0.12	−0.17	−0.13	−0.07	−0.12	−0.07	−0.02	−0.16∗	0.03	0.27∗	0.16∗	0.2∗	0.25∗	0.3∗	0.33∗	0.18∗	0.17∗	−.06	0.23∗	X
44. When you are upset you can talk to someone	−0.12	0.55∗	−0.07	−0.14∗	−0.1	0.57∗	−0.05	−0.12	−0.11	−0.14∗	−0.04	0.03	−0.01	−0.03	−0.05	−0.07	−0.01	−0.03	−0.09	−0.1	−0.07	−0.07	−0.15∗	−0.02	−0.04	−0.21∗	0.01	0.1	−0.07	−0.09	−0.14∗	−0.05	0.21∗	0.15∗	0.11	0.33∗	0.19∗	0.15∗	0.15∗	0.23∗	−0.19∗	0.39∗	0.31∗	X

F_score1_Anx, factor 1 anxiety; F_score2_Res, factor 2 resilience; F_score3_Dep, factor 3 depression; F_score4_Suicide, factor 4 suicide risk; S_Score_Mental_Health, Scale Score Mental Health

∗Correlation is significant at the 0.01 level (two-tailed).

The correlations between the resultant Strong Souls factor and scale scores, the constituent items, and need for follow up were analysed to explore construct and criterion validity. These correlations are reported in Table 1. The χ2 analyses between scale scores and the need for follow up and gender were used to explore relationships in the data and provide further evidence of criterion validity.

Results

An unrestricted EFA of the 34 Strong Souls items identified that a four-factor solution would be optimal (based on the eigenvalues and scree plot). The final four factors were consistent with the constructs of research interest, namely, anxiety, resilience, depression, and suicide risk, loading on the solution in that order. A range of extraction methods and rotations was trialled and all generated similar outcomes. The final four-factor solution included 25 items that accounted for 34.5% of the variance, shared across all four factors. The resultant factors including factor loadings and endorsement means of the composite items are reported in Table 2.

Table 2.

Item factor loadings (pattern coefficients) and endorsement means

Item	Factor 1 Anxiety	Factor 2 Resilience	Factor 3 Depression	Factor 4 Suicide Risk	Item endorsement means
Have you felt so worried it was hard to breathe?	0.78	−0.03	−0.01	−0.04	0.32
Have you felt so worried you got dizzy?	0.71	−0.001	−0.09	−0.08	0.32
Have you felt so worried you start to shake?	0.68	−0.04	−0.04	0.04	0.35
Have you felt so worried you got sick in the guts	0.55	0.04	0.15	0.01	0.37
Have you felt so worried you got really sweaty?	0.53	−0.04	0.04	−0.06	0.32
Have you felt so sad that nothing could cheer you up?	0.45	0.06	0.33	−0.09	0.40
You got lots of friends	−0.11	0.56	−0.01	−0.13	2.64
You know someone who is a really good person	−0.3	0.56	0.12	−0.01	2.44
You are really into something (like music, cars, clothes etc)	−0.02	0.51	0.05	−0.02	2.59
You are a good son or daughter to your family	0.11	0.49	−0.12	−0.02	2.55
When you are upset you can usually talk to someone about it	−0.03	0.47	0.04	−0.02	2.43
You laugh and make jokes a lot	−0.06	0.45	0.08	−0.03	2.50
You have a strong family who help each other	0.26	0.45	−0.20	−0.15	2.50
You got an older person looking our you	0.05	0.44	−0.09	0.01	2.51
You know lots about white fella ways	−0.18	0.40	0.13	0.19	1.45
Had too many bad moods?	0.01	−0.01	0.60	−0.08	0.58
Get angry or wild real quick?	−0.02	0.12	0.58	−0.05	0.73
Have trouble sleeping?	−0.01	0.02	0.53	0.04	0.54
Felt pretty lonely much of the time?	0.07	−0.11	0.48	−0.03	0.77
Hard to focus. Thinking all over the place?	0.01	0.4	0.44	−0.002	0.67
Felt like giving up – no point in trying?	−0.01	−0.03	0.40	0.31	0.37
Got angry or wild and stayed that way for a long time?	0.23	−0.5	0.38	−0.07	0.43
Have you wished you were dead?	−0.06	−0.04	−0.02	0.85	0.17
Have you felt like killing yourself?	0.14	0.12	−0.09	0.65	0.12
Felt like hurting yourself?	0.35	−0.05	0.01	0.50	0.25

Comparing the retained items with the nine items removed from the final analyses, there was no clear, consistent pattern as to why these items did not contribute to the final result. Some removed items included terms that may not have translated well linguistically such as ‘Felt like your life was getting worse and worse?’, or ‘Have you stopped doing things that used to be fun?’. Other items may not have been clear, such as ‘You wish you were a completely different person’ or ‘Had really bad dreams that make you worried or scared’. The item exploring impacts of the experience of discrimination created its own factor: ‘Been treated unfairly or discriminated against because you are Aboriginal?’

Reliability analyses performed on all 25 items, and on each factor using Cronbach alpha showed reliability coefficients ≥0.7. These results suggest good internal consistency of Strong Souls overall (0.70), and within each factor. Reliability coefficients, eigenvalues and percentage of variance for each factor are reported in Table 3. It is noted that although cut-offs for alpha are arbitrary, alpha is still useful because it proves a lower bound on test reliability given an assumption of no correlated error [51]. Given that alpha is an index of internal consistency (i.e. the average inter-item correlation), we expected to obtain high alphas within factors, with such high values acting as a further check of reliability for these factors. These results are consistent with reliabilities of mainstream tools such as the SDQ, which demonstrated reliabilities of between 0.62 and 0.82 [52].

Table 3.

Exploratory factor analysis (oblique rotation)

Factor	Eigenvalue	% of variance	Cronbach α
1. Anxiety	4.96	17.50	0.80
2. Resilience	2.67	7.80	0.71
3. Depression	1.96	5.34	0.71
4. Suicide risk	1.49	3.87	0.73

The consistent replication of the factors across different extraction methods supports Strong Souls’ construct validity, as does the consistency of the derived factors with the constructs of research interest. It is further demonstrated by strong correlations between factor scores and the composite items.

Participant outcomes were measured by summing each participant's score on the composite items for each factor, creating scale scores. Outcomes on the three mental health factors were also summed to create a mental health scale score. Inter-factor and scale score correlations are included in Table 1.

The correlations between the three mental health factor and scale scores were moderate and significant in a positive direction, indicating that the three mental health factors are distinct constructs. Negative, significant corre lations between the three mental health factors and resilience demonstrated the discriminant validity of Strong Souls. Further evidence of this, and consistent with the underpinning theoretical constructs, the mental health score was negatively correlated with resilience. High, significant correlations between the anxiety and depression factor scores and the mental health scale score indicates strong construct validity within Strong Souls. Convergent validity was demonstrated through positive, significant correlations between the need for follow up and factor and scale scores of the three mental health factors; as well as through low but significant negative correlations between the need for follow up and resilience.

The χ² analysis identified some strong relationships (<0.0001 level of significance) between the need for follow up and all the mental health scales, including the overarching mental health scale score, but not resilience. There were also highly significant relationships between the suicide and resilience scales and gender (p<0.01); and significant relationships between gender and the depression and mental health scales (p<0.05). Results of all the χ² analyses are reported in Table 4.

Table 4.

χ² analysis

Variable	Factor	χ²	df	p
ASIST	Anxiety	41.65	14	0.001
	Resilience	30.46	20	0.06
	Depression	1.104 × 100	16	0.001
	Suicide risk	1.07 × 100	8	0.001
	Mental health	1.27 × 100	28	0.001
Gender	Anxiety	18.43	14	0.19
	Resilience	42.08	20	0.003
	Depression	29.69	16	0.02
	Suicide risk	21.49	8	0.006
	Mental health	43.27	28	0.03

ASIST = whether participant required follow-up due to indentified possible suicide risk

Discussion

Strong Souls demonstrated strong construct validity, reliability and appropriateness as a tool for screening SEWB among Indigenous young people in the Northern Territory. It showed sensitivity to the SEWB constructs of anxiety, depression, suicide risk and resilience. These factors demonstrated acceptable reliability, and low to moderate correlations. The resilience factor correlated negatively with the other three factors. These factors are consistent with those identified in the limited epidemio-logical literature as prevalent SEWB issues for Indigenous Australians [4,5,53].

These results are comparable to the WASC-Y, the only broadly available Indigenous SEWB assessment tool. Variance explained by comparable WASC-Y factors is between 34% (cultural resilience subscale) and 57% (suicide). The reliability outcomes on comparable factors are also on par with the WASC-Y, which reports reliability coefficients between 0.75 (Cultural Resilience) and 0.88 (Suicide subscale) [12].

The resultant Strong Souls factor structure supports existing literature that identifies strong cross-cultural similarities between Indigenous and mainstream cultures in the symptoms of anxiety, depression and suicide risk. It also supports other research that shows distinct differences and interesting relationships between these factors [9,11–13,15].

The anxiety factor identifies somatic symptoms such as dizziness, indigestion, and breathing difficulties as relevant for Indigenous people. The experience of low mood or sadness, however, was strongly associated with anxiety in this sample, compared to its association with depression in mainstream mental health screening tools.

The suicide risk factor was composed of only three items but showed strong reliability and items were highly correlated, indicative of robustness. Interestingly, items that were expected to load on suicide risk, such as feelings of general hopelessness and hopelessness for the future, failed to load on the final factor structure, or were instead linked to depression. The data suggest that feelings of hopelessness are not necessarily linked to suicide risk. This is in accordance with previous research showing that Indigenous youth suicide behaviours are not necessarily contemplated and planned over time [11–13,15].

In accordance with previous research and key informant information, anger was explored as a unique indicator of depression for Indigenous Australians. Items that captured this concept were ‘had too many bad moods’, ‘Get angry or wild real quick?’ and ‘Got angry or wild and stayed that way for a long time?’ These items loaded on the depression factor, indicating that the externalization of anger is a significant indicator of depression for Indigenous young people [12,15,45]. Other symptomatology of depression, which is consistent with mainstream tools, includes difficulties with concentration, sleep and feelings of hopelessness and loneliness [31,34–41,44].

The presence of a resilience factor is consistent with other scales such as the WASC-Y, where positively worded resilience items load onto a separate factor. All resilience items correlated negatively with items from the other three factors, demonstrating the protective effect of resilience on SEWB. These results support previous research on constructs identified as most likely to underlie strong resilience and positive SEWB. For example, the item ‘You know someone who is a really good person’ can be linked to positive role models. The item ‘You laugh lots and make jokes’ is indicative of a positive and even temperament; and ‘You got lots of friends’ links into the ability to form peer relationships. Finally, items such as ‘You are a good son or daughter’ and ‘You have a strong family who help each other’ demonstrates the importance of good family relationships [17–20,33,53–56].

There are a number of potential limitations to this research. Because this research was undertaken as part of the larger ABC Study, the sample was not randomly selected. This is mediated, however, by the diversity of the cultural and language groups represented in the sample drawn from towns and communities across the Northern Territory. Because of the scope of the study the collection of data was an enormous logistical exercise. Data were, therefore, collected over a period of 18 months and collection took place in a variety of settings from classrooms, to clinics, to under trees. These factors could potentially have influenced the study results. These factors, however, are a fundamental part of Indigenous research and therefore any tool to be used in this context, must have utility under such a range of circumstances. Because the ABC Study collected data on a large range of health indicators that were considerably time consuming, individuals’ responses could potentially be affected by test fatigue. Finally, this study did not facilitate the exploration of convergent validity.

This paper outlines the steps in the development and preliminary validation of a tool to collect data on the SEWB of Indigenous adolescent participants of the ABC Study. These data will be used to test hypotheses relating to SEWB and biomarkers of chronic adult diseases. This study undertook EFA and, for Strong Souls to be used further, these results would need to be replicated through confirmatory factor analysis. With this and further clinical validation, Strong Souls has potential to be used as an SEWB screening tool applicable to the wider Indigenous population of Australia.

Footnotes

Acknowledgements

We would like to gratefully acknowledge our consultation network especially Top End Mental Health Services and Danila Dilba for their support and assistance with this research. We would also like to acknowledge all the participants of the ABC Study for their ongoing participation in such important longitudinal research. This research was supported by funding from the National Health and Medical Research Council of Australia (NHMRC project grant 383500).

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

References

Australian Bureau of Statistics. The health and welfare of Australia's Aboriginal and Torres Strait Islander Peoples 2008, cat. no:4704.0. Canberra: Australian Bureau of Statistics, 2008.

Australian Bureau of Statistics. National Aboriginal and Torres Strait Islander Health Survey, 2004-05, cat. no:4715.0. Canberra: Australian Bureau of Statistics, 2006.

Australian Institute of Health and Welfare. Australia's health 2008. Canberra: Australian Institute of Health and Welfare, 2008.

Swan

Raphael

. Ways forward: national Aboriginal and Torres Strait Islander mental health policy national consultancy report. Canberra: Commonwealth of Australia, 1995.

Zubrick

Silburn

Lawrence

. The Western Australian Aboriginal Child Health Survey: the social and emotional wellbeing of aboriginal children and young people. Perth: Curtin University and Telethon Institute for Child Health Research, 2005.

Australian Institute of Health and Welfare. Young Australians: their health and wellbeing 2007. Canberra: Australian Institute of Health and Welfare, 2007.

Urbis Keys Young. Evaluation of the Social and Emotional Well Being (Mental Health) Action Plan. Canberra: Office for Aboriginal and Torres Strait Islander Health, 2001.

Bird

. Epidemiology of childhood disorders in a cross-cultural context. J Child Psychol Psychiatry 1996; 37:35–49.

Allen

. Personality assessment with American Indians and Alaskan Natives: instrument considerations and service delivery style. J Pers Assess 1998; 70:17–41.

10.

Dana

. Cultural identity assessment of culturally diverse groups: 1997. J Pers Assess 1998; 70:1–16.

11.

Tatz

. Aboriginal suicide is different: a portrait of life and self-destruction. Canberra: Aboriginal Studies Press, 2001.

12.

Westerman

. The Westerman Aboriginal Symptoms Checklist–Youth: manual. Perth: Indigenous Psychology Services, 2000.

13.

Vicary

Westerman

. ‘That's just the way he is’: some implications of Aboriginal mental health beliefs. Aust eJ Adv Ment Health 2004; 3 (3).

14.

Anderson

. Aboriginal well-being. Grbich

. Health in Australia: sociological concepts and issues. NSW: Longman, 1999:53–73.

15.

Westerman

. Development of an inventory to assess the moderating effects of cultural resilience with Aboriginal youth at risk of depression, anxiety, and suicidal behaviours. PhD Thesis. Perth: School of Psychology, Curtin University, 2003.

16.

Hunter

. Ways of seeing: changing directions in the health of Indigenous populations. Clin Child Psychol Psychiatry 1998; 3:519–530.

17.

Atwool

. Attachment and resilience: implications for children in care. Child Care Pract 2006; 12:315–330.

18.

Mykota

Schwean

. Moderator factors in First Nation students at risk for psychosocial problems. Can J School Psychol 2006; 21:4–17.

19.

Ungar

. Constructionist discourse on resilience: multiple contexts, multiple realities among at-risk children and youth. Youth Soc 2004; 35:341–365.

20.

Zubrick

Robson

. Research agenda: resilience to reoffending in high risk group: focus on Aboriginal youth. Western Australia: Criminology Research Council, 2003.

21.

Vicary

Andrews

. Developing a culturally appropriate therapeutic approach with Indigenous Australians. Aust Psychol 2000; 35:181–185.

22.

Westerman

. Engagement of Indigenous clients in mental health services: what role do cultural differences play. Aust eJ Adv Ment Health 2004; 3 (3).

23.

Dudgeon

Garvey

Pickett

. Working with Indigenous Australians: a handbook for psychologists. Perth: Gunada Press, 2000.

24.

Vicary

Andrews

. A model of therapeutic intervention with Indigenous Australians. Aust N Z J Public Health 2001; 25:349–351.

25.

Vicary

Bishop

. Western psychotherapeutic practice: engaging Aboriginal people in culturally and respectful appropriate ways. Aust Psychol 2005; 40:8–19.

26.

Commonwealth Department of Health and Aged Care. National Action Plan for Promotion, Prevention and Early Intervention for Mental Health. Canberra: Commonwealth Department of Health and Aged Care, 2000.

27.

Westerman

Kowall

. Mental health outcome measures for Indigenous Australians: a review of current practice in culturally valid assessment processes. Perth: Indigenous Psychology Services and the Cooperative Research Centre for Aboriginal and Tropical Health, 2002.

28.

Sayers

Mackerras

Singh

Bucens

Flynn

Reid

. An Australian Aboriginal birth cohort: a unique resource for a life course study of an Indigenous population. A study protocol. BMC Int Health Hum Rights 2003; 3:1.

29.

Goodman

. The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. J Child Psychol Psychiatry 1999; 40:791–799.

30.

Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32:959–976

31.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders, fourth edition (DSM IV). Washington, DC: American Psychiatric Association, 1994.

32.

Zubrick

Lawrence

de Maio

Biddle

. Testing the reliability of a measure of Aboriginal children's mental health: an analysis based on the Western Australian Aboriginal Child Heath Survey. Canberra: Australian Bureau of Statistics, 2006.

33.

Alperstein

Raman

. Promoting mental health and emotional well-being among children and youth: a role for community child health? Child Care Health Dev 2003; 29:269–274.

34.

Zung

. A self-rating depression scale. Arch Gen Psychiatry 1965; 12:63–70.

35.

Grisso

Barnum

. Massachusetts Youth Screening Instrument, second version (MAYSI-2): user's manual and technical report. Worcester, MA: University of Massachusetts Medical School, 2000.

36.

Kroenke

Spitzer

Williams

JBW

. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001; 16:606–613.

37.

Manson

Beals

. American Indian Services Utilisation and Psychiatric Epidemiology Risk and Protective Factors Project (AI-SUPERPFP) interview schedule. Denver, CO: National Centre for American Indian and Alaskan Native Mental Health Research, 2005.

38.

Ware

Kosinski

Dewey

. SF-36 user's manual: how to score version two of the SF-36 Health Survey. Lincoln, RI: Quality Metric, 2000.

39.

Radolf

. The CES-D. Appl Psychol Measure 1977; 1:385–401.

40.

Beck

Steer

. Manual for the Beck Depression Inventory. San Antonio, TX: Psychological Corporation, 1996.

41.

Everson

Kaplan

Goldberg

Salonen

. Hypertension incidence is predicted by higher levels of hopelessness in Finnish men. Hypertension 2000; 35:561–567.

42.

Horsten

Erigson

Perski

Wamala

Schenck–Gustafsson

Orth–Gomér

. Psychological factors and heart rate variability in healthy women. Psychosom Med 1999; 61:49–57.

43.

Rosengren

Hawken

Ounpuu

. Association of psychosocial risk factors with acute myocardial infarction in 11119 cases and 13648 controls from 52 countries (the INTERHEART) case-control study. Lancet 2004; 364:953–962.

44.

Kristensen

Kornitzer

Alfredsson

. Social factors, work, stress and cardiovascular disease prevention in the European Union. Brighton, UK: European Heart Network (EHN), 1998.

45.

Esler

Johnston

Thomas

. The acceptability of a depression screening tool in an urban, Aboriginal community-controlled health service. Aust N Z J Public Health 2007; 31:259–263.

46.

Floyd

Widaman

. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess 1995; 7:286–299.

47.

Steger

. An illustration of issues in factor extraction and identification of dimensionality in psychological assessment data. J Pers Assess 2006; 86:263–272.

48.

Tabachnick

Fidell

. Using multivariate statistics, fifth edition. Boston, MA: Pearson Allyn and Bacon, 2007.

49.

Kessler

Andrews

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32:959–976.

50.

Palmieri

Smith

. Examining the Structural Validity of the Strengths and Difficulties Questionnaire (SDQ) in a U. S. Sample of Custodial Grandmothers. Psychol Assess 2007; 19:189–198.

51.

Rae

. Correcting coefficient alpha for correlated errors: is αk a lower bound to reliability? Appl Psychol Measure 2006; 30:56–59.

52.

Goodman

. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry 1997; 38:581–586.

53.

Kyaw

. Mental health problems among Aborigines. Ment Health Aust 1993; 5:30–36.

54.

Carlton

Goebert

Miyamoto

. Resilience, family adversity and well-being among Hawaiian and non-Hawaiian adolescents. Int J Soc Psychiatry 2006; 52:291–308.

55.

Resnick

. Resilience and protective factors in the lives of adolescents. J Adolesc Health 2000; 27:1–2.

56.

Clarke

Harnett

Atkinson

Shochet

. Enhancing resilience in Indigenous people: the integration of individual, family and community interventions. Aborig Isl Health Work J 1999; 23:6–10.

Strong Souls: Development and Validation of a Culturally Appropriate Tool for Assessment of Social and Emotional Well-Being in Indigenous Youth

Abstract

Keywords

Method

Participants

Ethics

Procedure

Tool selection and development

Pilot testing

Large-scale sampling

Data analysis

Results

Discussion

Footnotes

Acknowledgements

References