Abstract
Aboriginal children experience a disproportionately high burden of mental health related harm. Research suggests that almost a quarter of Aboriginal children meet criteria for clinically significant emotional or behavioural problems [1], and Aboriginal youths are several times more likely to commit suicide than other Australian young people [2,3]. Despite this, there little research examining the mental health of Aboriginal children, and no clinically validated psychological assessment tools are available. The establishment of a screening tool that provides culturally competent mental health assessment is a vital step in understanding, identifying and evaluating the treatment for mental health problems amongst Aboriginal young people.
To date, mental health has largely been measured amongst Aboriginal children in both clinical and research contexts using established measurement tools designed for non-Indigenous populations. While this is advantageous in terms of allowing comparison to other data sets and populations, cultural differences in the way mental health is conceptualized, the language used to speak about mental health, and norms of behaviour and emotional expression may all adversely affect the accuracy and acceptability of a test cross culturally. As many differences are known to exist between Aboriginal cultures and non-Aboriginal culture, and indeed Aboriginal and non-Aboriginal concepts of mental health [4], it is not necessarily the case that measures of mental health used in non-Aboriginal populations will be appropriate for use amongst Aboriginal children [5,6].
The Strengths and Difficulties Questionnaire (SDQ), a 25-item behavioural screening instrument appropriate for use among 4–17 year olds [7] is the most commonly used tool in studies involving Aboriginal children and, as part of the NSW Department of Health's Mental Health Outcomes and Assessment Training protocol, is routinely administered to all young people accessing mental health services in the state. The SDQ measures children's strengths (10 items) and difficulties (15 items) over the last six months. The scale comprises 5 subscales: emotional symptoms; conduct problems; hyperactivity; peer problems and prosocial behaviour. A total difficulties score ranging from 0–40 is derived from summing all subscales excluding prosocial behaviours [7].
The SDQ is considerably briefer than the Rutter and the Child Behaviours Checklist, but has comparable psychometric properties [8,9]. The SDQ takes 5 minutes to administer and is simple to use and score. Several studies have found the SDQ to be an appropriate screening tool for Australian children [10–12]. Good psychometric properties have been demonstrated in a large community sample of Australian children aged 4–9 [10] and SDQ assessments have been found to correlate to a moderate to high level with clinician diagnoses in both a community and a clinical sample [11,12].
While the suitability of the SDQ for use amongst a general sample of Australian children has been demonstrated [10–12], less is known about the appropriateness of its use amongst Aboriginal children. In the Western Australian Aboriginal Child Health Survey (WAACHS) the wording of the SDQ was modified in order to facilitate oral administration of the survey, which is typically written, and to make the language used more compatible with that of the participants. Analyses conducted by the WAACHS team suggest that this modified version had good internal validity and reliability, and overall was an appropriate measure for use amongst Aboriginal children [13]. This modified version of the SDQ has not been widely adopted, however, with the standard version still being mandated for use clinically and employed in most research programmes involving Aboriginal children. There is no information available about the reliability, validity or acceptability of the standard version of the SDQ amongst Aboriginal children.
The current study aimed to assess the acceptability of the SDQ in ACCHOs located in urban communities in NSW.
Methods
Sample
Qualitative data was collected at the three ACCHOs located within the greater Sydney region: Aboriginal Medical Service Western Sydney, Tharawal Aboriginal Corporation and the Aboriginal Medical Service Redfern. A combination of focus groups and small group interviews were used depending on the preference of the service and the individuals concerned. A total of 47 people participated in the study, 30 females and 17 males. The views of the following groups were elicited: those who may be given the SDQ to complete in a clinical or research context (Aboriginal parents, n = 15), those who have administered the SDQ to Aboriginal families in the past (Aboriginal research assistants and youth workers, n = 5) and those who may be asked to do so in the future (AMS staff and other local Aboriginal workers such as Aboriginal education officers, n = 27). It should be noted that most participating AMS staff and Aboriginal workers were also parents. The SDQ is completed by carers for children between the ages of 4–17 years; adolescents (11–17 year olds) also complete self report versions of the SDQ. It was not possible to recruit adolescents, including those who might be high scorers on the SDQ to a focus group.
Ethics
This project was approved by the Board of each participating Aboriginal community controlled health service and the ethics committees of the Aboriginal Health and Medical Research Council of New South Wales, the University of Sydney and the University of New South Wales.
Procedure
Focus groups were facilitated by the lead author in collaboration with Aboriginal research officers. A semi-structured approach was used to elicit participants’ opinions regarding factors surrounding the mental health of Aboriginal children and adolescents and how good and poor mental health is typically expressed. Participants were then given a brief introduction to the SDQ and asked to read through the survey. Initial reactions to the scale were elicited. A more detailed explanation of how the SDQ is used and scored was then relayed. Next, groups discussed each item in the SDQ in turn, paying particular attention to whether the concepts measured were important indicators of social and emotional well-being status for children in their community, whether the meaning of the question was clear and whether the language used was appropriate. After each item had been discussed, participants were again asked for their opinions on the SDQ overall. Lastly, participants were asked if there were any issues not covered by the SDQ which they felt were critical in assessing the mental health of Aboriginal children in their community.
All focus groups were audiotaped. Data were transcribed by a person with no personal or professional connection to the local communities. Audio tapes were securely transported and stored in locked filing cabinets.
Analysis
Transcribed data were analysed using NVivo qualitative data analysis software; QSR International Pty Ltd. Version 8. The data were coded independently using the constant comparative method whereby each piece of data is compared to the rest of the data to establish analytical categories and dominant themes [14]. The data was first examined as a whole to identify any dominant themes. Feedback on each item including percieved importance and the clarity and appropriateness of the language used was then considered in combination with the characteristics that participants independently nominated as being indicative of good or poor social and emotional well-being. Items for which concerns about acceptability emerged as a dominant theme are highlighted as such in the results section. The feedback given by participants on the instrument overall was also explored.
Results
Several major themes emerged from the focus groups including: general issues surrounding the use of psychological tests with Aboriginal people, general opinions regarding the SDQ, issues with each subscale and critical factors in Aboriginal child mental health not covered by the SDQ. Of note, differences in opinion on the appropriateness of the SDQ were not evident between parents, those who had administered the SDQ in the past and other Aboriginal workers.
Overall responses
Use of psychological tests with Aboriginal people
Many participants raised concerns about the appropriateness of asking Aboriginal parents or young people to answer what was perceived to be a long list of questions, particularly in a mental health setting. Many participants stated that families wanted the opportunity to tell their stories and were likely to be frustrated by being asked to communicate in such a structured format. Giving parents questionnaires to fill in by themselves was thought to be particularly culturally inappropriate, and to be problematic for those with literacy issues. Participants felt strongly that where psychological tests are used, clinicians or researchers should administer the questionnaire interview-style.
Some workers commented that if they were required to fill out a questionnaire with families they would elicit answers to the questions through general conversation rather than ask them one by one, as families would be unlikely to complete questionnaires. Many participants felt that carers would be concerned about filling in a questionnaire such as the SDQ due to fears that the information would somehow be relayed to the Department of Community Services and lead to problems for them. Similarly, several participants hypothesized that many parents would choose not to answer the questions honestly for fear of the answers reflecting badly on their parenting skills.
Opinions on the SDQ generally
Most participants initially felt that the SDQ was too long; however, after going through each question all of the groups felt that the questions asked were important and most had suggestions for additional items/issues to add. After the initial read through, some participants were worried that a response of ‘certainly true’ for any question indicated that a child had mental health problems. They were thus much more comfortable with the tool after the more detailed explanation of how it is used and scored had been explained. Overall the SDQ was felt to be acceptable; however, the wording of some questions was thought to be ambiguous with much debate arising regarding the meaning of a small number of questions. The response scale (not true, somewhat true, certainly true) was also seen as difficult to understand. In keeping with this, workers who have used the SDQ with families reported having routinely had to explain the meaning of some questions as well as the response scale. After the initial read through of the survey, some groups also noted that several of the questions seemed to be asking about the same thing; this was seen as confusing and unnecessary until the purpose behind it was explained during the more detailed explanation of the SDQ that was subsequently given. While all groups suggested modifications to the questionnaire to improve comprehension, no items were thought to be offensive and there were no questions which the majority of participants felt should be removed. After the way it is used and scored had been outlined, the SDQ was considered useful.
Responses to each subscale of the SDQ
Emotional symptoms scale
The concepts covered by the emotional symptoms scale were considered important indicators of distress by participants with item 13 ‘often unhappy, depressed or tearful’ considered the most important indicator. The language used in some questions, however, was found to be unclear. In particular, item 16 ‘nervous or clingy in new situations, easily loses confidence’ was found to be confusing. Most participants felt that this item asked about two separate things. The first, feeling nervous in new situations, was considered normal and indeed sometimes an important mechanism for assessing and avoiding dangerous situations. Easily losing confidence, on the other hand, was considered a worrying sign. Consequently, most participants felt that this question was impossible to answer in its original form. Items 8 ‘many worries or often seems worried’ and 24 ‘many fears, easily scared’ were also considered confusing by some groups as they were perceived as asking about two separate things. Of note, items 13, 16 and 8 were modified in the WAACHS version of the SDQ (Table 1), however, participants in the current study suggested different modifications than those used in the WAACHS except for item 13.
List of standard English (Australian) Strengths and Difficulties Questionnaire items for parents of 4–10 year olds, crosses indicate items for which changes were suggested (Sydney) or made (WAACHS)
Conduct problems scale
The items in the conduct problems scale were thought to be important indicators of trouble, with the exception of two questions. While participants thought it was important that children were ‘generally well behaved’ the majority of groups stated that they did want or expect their children to do what ‘adults request’ as a general rule. The primary reason given for this involved safety concerns, i.e. not wanting children to engage in unsuitable or dangerous behaviour at the request of unreliable adults. Some participants also stated that they would not expect a child to follow the directions of adults who were not considered worthy of respect. This item was also modified in the WAACHS version of the SDQ. The item concerning lying and cheating was also debated in some groups, with many participants unclear as to what constituted a lie for the purposes of the questionnaire.
Hyperactivity scale
The items listed in the hyperactivity scale were largely thought to describe the standard behaviour of healthy children. While participants recognized that these behaviours could exist at problematic levels, they were generally considered normal. Where such behaviours were problematic, participants felt that health problems, particularly otitis media-related hearing and speech difficulties, learning difficulties or hunger (e.g. no breakfast) were often the cause. Conversely, some very bright children were thought to display hyperactive behaviour at school because they were not sufficiently stimulated by their lessons. Scepticism about the increase in the number of children diagnosed with attention deficit hyperactivity disorder was often expressed, with most participants feeling that out of control hyperactive behaviour was more often due to a lack of discipline, family problems or simply a need for more exercise than to any underlying neurological condition. A wording change was suggested for item 21 ‘thinks things out before acting’; this item was also modified in the WAACHS, however the wording used was different to that suggested by Sydney participants.
Peer problems scale
The peer relationships subscale did not appear to be a good fit for Aboriginal families in Sydney as the exclusive focus on peers was not compatible with participants’ views regarding the relative importance of different interpersonal relationships. Indeed, while relationships with peers were seen as important, so were relationships with adults and with the Aboriginal community in general. Relationships with family members meanwhile were considered much more important than any of these and were mentioned frequently. Several groups stated that they would not be worried if a child did not have at least one good friend or got along better with adults than children provided that they had a good relationship with their family (family meaning extended family network). Interestingly, this view was more commonly held amongst female participants, with males more likely to endorse the importance of peer relationships. Most groups did not consider preferring to play or spend time alone problematic if it was a long standing trait of a child, but did think it was a cause for concern if this tendency suddenly emerged. Participants suggested rewording item 6, this item was also modified for the WAACHS but the wording used was different to that suggested by participants in the current study.
Prosocial scale
Prosocial behaviours were seen as very important by all groups and were often noted to be expected even of children who were generally badly behaved. Behaviours such as being kind to younger children, sharing and helping people who were sick or upset were considered to be things that children instinctively liked to do. Participants suggested changes to items 1 and 4 to improve clarity. These items were also modified in the WAACHS; however, participants in the current study suggested different modifications.
Impact supplement
An impact supplement can be added to the SDQ which determines whether the respondent perceives the child to have a problem and, if so, assesses the resultant social impairment, distress, and burden the participant perceives this problem to cause, in addition to its chronicity [7]. All groups voiced strong support for the impact supplement. This section of the SDQ was viewed as clear and straight to the point and participants agreed that the questions contained within it were very important.
Omissions
Critical factors in Aboriginal child and adolescent mental health not covered by the SDQ
While the SDQ was thought to cover many of the important aspects of child behaviour and emotions, there were several issues mentioned by all groups as being essential components of social and emotional well-being among Aboriginal children which are not covered by the questionnaire. Participants felt strongly that these factors must be explored by clinicians working with Aboriginal children if they are to obtain a complete picture of their social and emotional well-being. The majority of these factors concerned things which impact strongly on mental health: connection to or relationship with extended family, Aboriginal identity, feeling that you are accepted by and belong to an Aboriginal community, and the impact and experience of racism. One factor concerned appropriate behaviour amongst Aboriginal children, namely demonstrating appropriate levels of respect for elders.
Other issues mentioned as important which are not covered by the SDQ included: sleeping problems, anger, changes in eating habits, displays of affection and cruelty to animals.
Discussion
The SDQ is widely used in research and clinical practice with Aboriginal children and adolescents, however, no previous studies have examined the acceptability of the SDQ in an Aboriginal setting and the reliability and validity of the standard version is unknown in this context. The results of the current study highlight some important issues surrounding the use of psychological tests with Aboriginal people in general and the appropriateness of the SDQ in particular. It appears that overall the content of the SDQ is acceptable to Aboriginal people living in the Sydney region. In order to improve cultural appropriateness, however, some wording modifications may be required and additional questioning is needed to capture critical factors surrounding mental health which are of major importance to Aboriginal children and adolescents.
Participants highlighted several difficulties in using mental health questionnaires amongst Aboriginal people. As has been found previously [15], service providers in the current study noted that patients wanted to ‘tell their stories’ and often found the use of structured questionnaires off-putting and frustrating. Meanwhile many participants stated that filling out questionnaires was a pointless exercise that rarely resulted in any positive outcomes and indeed had the potential to lead to problems for families if the authorities became involved [16]. As a result of such issues most ACCHO service providers reported that they did not use questionnaires, while those who did employ such tools often reported seeking the answers to the questions through conversation rather than going through the tool systematically.
While cultural differences in language use were evident, there appeared to be a great deal of commonality in the understanding of the behaviours described in the SDQ. As such, in contrast to the results of some studies conducted with Native American communities [16,17] there were no questions which, for example, described a behaviour which was defined as problematic for the purposes of the test but was considered a positive behaviour by participants. That being said, many of the questions in the hyperactivity subscale were thought by most groups to describe behaviours which were normal, except when taken to extremes, and some of the peer relationships items described situations which were not thought to be worrying providing they were long standing and the child was able to interact well with their extended family. The items listed in the prosocial scale were strongly endorsed by participants and it may be hypothesized that prosocial behaviours are regarded as particularly important by Aboriginal people.
The meanings behind a minority of questions in the SDQ were ambiguous to participants. This was particularly evident for several items in the emotional symptoms subscale and is likely to reduce the validity of the responses of some Aboriginal parents to these questions. Disjunctive questions were found to be especially difficult to answer. The response scale was also found to be unclearly worded; however, participants were more easily able to reach agreement as to what the scale options meant. It is likely that modifications to the wording of the SDQ would significantly improve the cultural appropriateness of the SDQ for Aboriginal children in Sydney. It is not necessarily the case, however, that such modifications would improve the diagnostic utility of the tool. To that end, a detailed examination of the psychometric properties of the original and modified tools must be undertaken to determine their relative advantages. It should also be noted that as far as we are aware there have been no other qualitative validations of the SDQ conducted. It is possible that the comprehension issues highlighted by participants in the current study may be relevant more broadly.
A difference between Aboriginal and non-Aboriginal concepts of the key interpersonal relationships in the lives of children and adolescents appeared to reduce the appropriateness of the peer relationships subscale for Aboriginal families in Sydney. While peer relationships were seen to be important, relationships with extended family were seen to be significantly more so and relationships with the Aboriginal community generally were also thought to be vital. The primacy of the importance of family and kinship relationships for the social and emotional well-being of Aboriginal people has been noted in other studies [18,19]. As a result of this, many participants felt that having few friends was not problematic provided a child interacted well with their family and indeed with the Aboriginal community more broadly. An analysis of the reliability of a modified version of the SDQ amongst participants in the Western Australian Aboriginal Child Health Survey appears to support this finding with the peer relationships subscale demonstrating poor internal consistency reliability amongst their sample [13].
Participants recommended that several key issues not covered by the SDQ be considered by clinicians and researchers when attempting to evaluate the mental health of Aboriginal children. Several of these factors related to exploring the extent to which a child was proud and secure in their Aboriginal identity. Issues considered important in this regard were connection with extended family, feelings of belonging to and being accepted by an Aboriginal community and pride in oneself as an Aboriginal person. All groups emphasized the importance of a strong Aboriginal identity for good mental health. Related to this, an exploration of the extent to which a child perceived they were discriminated against because of their Aboriginality was also considered vital. Lastly, every group emphasized showing proper respect for elders as an important sign that a child was growing up well in a cultural sense. A brief set of questions exploring these issues may be a useful addition to the SDQ when used with Aboriginal participants.
The findings of this study suggest three key recommendations for clinicians when using questionnaires such as the SDQ with Aboriginal participants. Firstly, these questionnaires should be administered orally, not as pen and paper tests as is usual practice. This is important for building rapport as well as for avoiding potential embarrassment or invalid recording of answers due to literacy issues. Secondly, the purpose of the test and the way it is used and scored should be explained to participants prior to administration. Participants in the current study were much more comfortable with the SDQ when they were aware that it was a screening tool rather than a diagnostic instrument. Further, many participants assumed that if an answer was ‘certainly true’ for their child this indicated that they had mental health problems and were relieved to learn that many of the questions asked about strengths of the child, and that it was normal for children to experience some difficulties. Lastly, clinicians should be mindful of the critical issues in Aboriginal child mental health indentified in the current study including identity and culture when assessing and treating Aboriginal children, as well as of confounding factors such as physical health problems.
Further work is required to ensure that the SDQ is an accurate tool for the assessment of mental and emotional well-being in Aboriginal children. First, the validity of the standard version of the SDQ should be determined for Aboriginal people. If the recommendation to administer the SDQ orally is taken up, any future validation work should focus on oral administration of the instrument. Second, further work is needed to determine the relative psychometric properties of a modified scale. While it is always advantageous for the purposes of comparison to use the same measurement tool across all groups, this advantage only holds if respondents share a common interpretation of the questions asked and indeed the response scale. The results of the current study suggest that some of the wording in the SDQ is ambiguous and inappropriate for Aboriginal people living in Sydney, potentially compromising its validity in this context. Of note, while there were some similarities in terms of the questions presumably found to be ambiguous by those who assisted in modifying the SDQ for the WAACHS and participants in the current study, the modifications suggested by Sydney participants differ significantly from the WAACHS version of the instrument (which was designed to be used with participants living in urban, rural and remote settings), emphasizing the dangers of attempting to generalize across different Aboriginal communities. Lastly, as many of the critical issues in Aboriginal child and adolescent mental health outlined by participants in the current study are not covered by any mental health screening tool in common usage, a companion set of questions addressing issues of particular importance to Aboriginal child and adolescent mental health should be developed. This programme of work is currently underway.
The following caveats should be noted in regard to this study. The findings presented in this paper are derived from participants recruited through three ACCHOs located within the greater Sydney region. The authors make no claim as to the generalizability of these results to Aboriginal people living in other parts of Australia.
The findings of the current study suggest that overall the SDQ is an acceptable screening tool for Aboriginal children in Sydney, and is underpinned by relevant concepts of good and poor mental health. Differences are evident, however, regarding the degree of emphasis placed on peer relationships and hyperactivity and some changes to the wording may be desirable to improve cultural appropriateness and comprehension. The development of an additional subset of questions exploring issues of particular relevance for Aboriginal children to be used in conjunction with the SDQ is recommended. Further work is needed to develop and validate such a measure and to explore the relative psychometric advantage of a modified tool over the original version.
Footnotes
Acknowledgements
We thank the study participants, their communities and the staff at participating Aboriginal Community Controlled Health Organisations.