Abstract
Objective:
The Composite International Diagnostic Interview 3.0 is a standardised diagnostic interview commonly used in population-based mental health surveys, but has not been used in community-residing Indigenous Australians. This paper seeks to determine whether the Composite International Diagnostic Interview 3.0 can produce valid diagnostic information when compared with a diagnostic interview in an urban Indigenous Australian sample.
Method:
This research was conducted over 10 weeks with adult Indigenous clients of two participating Aboriginal Medical Services in South-East Queensland. Using a cross-sectional, repeated-measures design, participants were administered the Composite International Diagnostic Interview 3.0 by an Indigenous interviewer and within 2 weeks attended a second appointment with an Indigenous clinical psychologist, who produced a diagnostic summary. The Composite International Diagnostic Interview 3.0 diagnoses were compared with the diagnostic summaries and clinical concordance between the two measures was calculated.
Results:
The diagnostic accuracy of the Composite International Diagnostic Interview 3.0 differed by module. The Post-traumatic Stress Disorder and Major Depression modules had good utility in diagnosing post-traumatic stress disorder and major depressive episodes, respectively; however, the Mania module that provides diagnoses of bipolar disorder was found to be unsuitable for this population. Although there were no identified contraindications for the use of the Generalised Anxiety and Alcohol Use Disorder modules, further research on the diagnostic accuracy of these modules is warranted.
Conclusions:
The Composite International Diagnostic Interview 3.0 can accurately diagnose some common mental disorders in an Indigenous Australian population, but was found to be unsuitable for others. Given these findings, care should be taken when using the Composite International Diagnostic Interview 3.0 in epidemiological prevalence studies with Indigenous Australian populations.
Keywords
Background
Aboriginal and/or Torres Strait Islander peoples of Australia (henceforth respectfully referred to as Indigenous Australians) have disparate life expectancies (Australian Institute of Health and Welfare (AIHW), 2015), health losses (AIHW, 2016) and levels of psychological distress relative to non-Indigenous Australians. Despite this, little is known about the epidemiological prevalence of mental and substance use disorders in Indigenous Australians. There have been no representative population-based surveys and few research studies. The AIHW (2015) systematic review identified only 17 studies on mental and substance use prevalence in Indigenous Australian samples, and only 4 that used structured diagnostic interviews (Black et al., 2015). This paper will examine the diagnostic utility of the Composite International Diagnostic Interview 3.0 (CIDI 3.0) in an Indigenous Australian sample and discuss the applicability of this instrument in a prospective population-based survey.
Structured diagnostic assessments are widely used in epidemiological surveys globally, and are considered to be valid and reliable in identifying those in a population meeting diagnostic criteria for a mental disorder (Lewis et al., 1992). They are structured or semi-structured survey instruments administered by a clinician or trained interviewer. Diagnoses are usually based on criteria in established psychiatric classification systems – the
The use of structured diagnostic assessments with Indigenous Australian populations has raised concerns. Importantly, the Western diagnostic, deficit-based system of mental illness differs significantly from conceptions of mental health in Indigenous Australian populations, where it is viewed as part of a holistic concept of social and emotional wellbeing (Dudgeon et al., 2014; Vicary and Bishop, 2005). Furthermore, the Western approach may be reductionist and contribute to the objectification and stereotyping of Indigenous Australians (Dudgeon et al., 2014). As such, structured diagnostic assessments developed according to Western diagnostic models may not be appropriate or valid (Black et al., 2015), and risk producing inaccurate estimates of prevalence. There is limited data to indicate the diagnostic accuracy of such assessments in Indigenous Australian samples.
The structured diagnostic assessments that have been used in Indigenous Australian samples include the Structured Clinical Interview for the
The CIDI is a structured diagnostic assessment that has formed part of the Australian National Survey of Mental Health and Wellbeing (Andrews et al., 1999; Slade et al., 2009), used cross-culturally in World Mental Health Surveys (Kessler and Üstün, 2004), and to determine psychiatric prevalence within non-representative Indigenous Australian sub-populations (Butler et al., 2005, 2007; Heffernan et al., 2012; Nadew, 2012). Differences in cultural expressions of mental illness may lead to insensitivity of the instrument (Butler et al., 2007), and even with cultural adaptations, previous research has been unable to assess whether the CIDI under- or overestimates mental disorder prevalence without validation in an Indigenous Australian sample (Heffernan et al., 2015a, 2015b).
This paper reports on a study that tested the diagnostic accuracy of the CIDI instrument (CIDI 3.0) in a sample of community residing Indigenous Australian adults. This research forms part of a broader research project, the Queensland Urban Indigenous Mental Health Survey (QUIMHS), a population-based survey for adult Aboriginal and/or Torres Strait Islanders in South-East Queensland to determine mental disorder prevalence, the services that are currently being accessed and their effectiveness as well as barriers to care. Using data from the pilot phase of the QUIMHS project, we aimed to test the concordance of CIDI diagnoses with diagnoses made by Indigenous Clinical Psychologists. If found to produce diagnostically accurate data in Indigenous Australians, use of the CIDI offers several advantages in prospective prevalence studies relative to other structured diagnostic assessments. It is comprehensive (in that individuals can meet diagnostic criteria for more than one mental disorder), completely structured (removing the need to exercise clinical judgement and reducing interviewer bias) and can be administered by trained laypersons (leading to savings in research resources; of particular importance in large-scale epidemiological studies (Lewis et al., 1992]. In addition to this, it would enable researchers to make comparisons with the prevalence and impact of mental disorders in over 28 countries globally (Slade et al., 2009).
Method
Funding and ethics
The research presented here was conducted as part of the pilot phase of the QUIMHS research that sought to evaluate the suitability of the planned survey processes and adequacy of the instrumentation. The QUIMHS project is a collaboration between the Queensland Centre for Mental Health Research (QCMHR), the Queensland Department of Health, and the Institute for Urban Indigenous Health and its member services.
The pilot study design and protocol were approved by the Townsville Hospital and Health Service HREC, (HREC/2020/QTHS/61158) ratified by the University of Queensland HREC and were consistent with NHMRC guidelines for ethical conduct in research with Aboriginal and Torres Strait Islander Peoples and communities (National Health and Medical Research Council, 2018). The QUIMHS Steering Committee provided oversight of the research and consisted of Indigenous and non-Indigenous experts in Indigenous mental health, epidemiology and research, and representatives from the Aboriginal Community-Controlled Health Sector. The QUIMHS pilot study methods and protocols are available on request from the study authors. All participants provided written informed consent to participate.
Study design
The QUIMHS pilot study was conducted at two Aboriginal Medical Services in South-East Queensland over a 10-week period in September–November 2019. The survey was cross-sectional with a repeated-measures design. The participants were adult Indigenous (Aboriginal and/or Torres Strait Islander) Australians that were existing clients of the Aboriginal Medical Service. The survey occurred in two stages: at Stage 1, the participant attended a face-to-face interview where a trained Indigenous interviewer administered the QUIMHS survey instrument. Within 2 weeks, the participant also attended a face-to-face psychological assessment by one of two Indigenous clinical psychologists.
Trained interviewers and survey psychologists
Two Indigenous lay interviewers (one male, one female) were employed and trained to administer the survey instrument. Two Indigenous clinical psychologists (one male, one female) were employed to conduct the psychological assessment with participants and used a semi-structured intake process to gather information and generate a diagnostic summary using
Sampling strategy
The two selected Aboriginal Medical Services sit under the umbrella of the Moreton Aboriginal & Torres Strait Islander Community Health Service (MATSICHS), directly administered by IUIH. Participants were existing clients and were recruited as a convenience sample from one of two avenues: (1) clients already engaged with social health services (counselling, child and family services, etc.) were invited to take part in the research by staff and (2) promotional material was left at reception for general clients to self-refer into the research.
Survey instrument and software
The QUIMHS survey instrument was developed to collect data on (1) demographic and risk factors, (2) mental and substance use disorders in the past 12 months and (3) participant service use and barriers to care for mental and substance use disorders. The mental and substance use disorders component of the survey instrument was measured using the CIDI 3.0. The CIDI 3.0, an initiative of the World Health Organization (WHO) is a psychiatric diagnostic instrument designed specifically for use in adult epidemiological surveys and has been used in over 28 countries around the world (Andrews et al., 1999; Australian Bureau of Statistics, 2021; Kessler and Üstün, 2004; Slade et al., 2009). The CIDI 3.0, at the time of publication, is the most up-to-date version of instrument available for use, and corresponds to criteria proposed by the
Analysis
Data from the survey instrument were extracted using Blaise software (2008); analyses were conducted in R (version 3.5.3) (R Core Team, 2019). Diagnostic summaries provided by the Indigenous clinical psychologists were compiled in Excel and compared with the diagnostic output from the survey instrument. Participant responses to the CIDI 3.0 instrument were converted to corresponding
Results
By the end of data-collection, 56 participants had participated in the overarching pilot study. Of these, 42 participants had completed the CIDI 3.0 component of the QUIMHS survey instrument, of which 37 also underwent a face-to-face psychological assessment by a survey psychologist. The reappraisal for one participant was excluded from the analysis based on advice from the project psychologist, who, due to the complex presentation of the participant, was unable to make a precise diagnosis in the time allowed for the assessment, leaving 36 participants for the diagnostic concordance analyses. The demographic characteristics of the participants over both stages of the pilot study can be found in Table 1. All participants identified as Aboriginal and/or Torres Strait Islander.
Demographic data of pilot sample.
CIDI: Composite International Diagnostic Interview.
Figure 1 summarises the screening and diagnostic pathways of participants that completed the CIDI 3.0 component of the survey. In this sample, all participants screened into at least one diagnostic module, with most of the participants screening into most of the diagnostic modules.

STARD diagram to report flow of participants through the CIDI 3.0 component of the QUIMHS survey instrument.
The diagnostic modules assess for mental disorder using diagnostic criteria. If the participant meets the diagnostic threshold, the CIDI 3.0 produces a diagnosis. Based on
Table 2 summarises the concordance between the CIDI 3.0 diagnostic output and the diagnoses produced by the survey psychologists. It compares the number of cases that met diagnostic criteria for each mental and substance use disorder assessed by the CIDI 3.0 with the number of cases that were diagnosed with the same disorder by the survey psychologist.
Clinical concordance between the CIDI 3.0 and clinical reappraisals.
CIDI: Composite International Diagnostic Interview.
The interpretation of the Kappa coefficient was based on predefined thresholds in the literature (Landis and Koch, 1977). The sample size for each CIDI 3.0 module was inclusive of all participants who completed the survey, and the participants who did not finish the survey but screened into the module; diagnoses were derived using criteria from the
The data showed that there was moderate agreement between the CIDI 3.0 and survey psychologist diagnoses for PTSD. There was fair agreement between the CIDI 3.0 and survey psychologist diagnoses for mood disorders overall; however, when broken down into types of mood disorders (i.e. MDD dysthymia and bipolar disorder), concordance was lower. Survey psychologists made almost double the number of major depressive disorder diagnoses compared with the CIDI 3.0 (20 vs 12, respectively). The inverse pattern was observed for bipolar disorder, where the survey psychologists made substantially fewer diagnoses than the CIDI 3.0 (3 cases vs 15, respectively). When examined on a case-by-case basis, 8 out of the 15 cases of bipolar disorder diagnosed by the CIDI 3.0 were instead diagnosed as MDD by survey psychologists. The case numbers of dysthymia, GAD and alcohol use disorders (as identified by either the CIDI 3.0 or psychologists) were too low for clear interpretation.
Discussion
The QUIMHS pilot study found that in an Indigenous Australian sample, the CIDI 3.0 when compared with clinician diagnoses, has differing diagnostic accuracy by module. The CIDI 3.0 PTSD and Depression modules were found to have utility in making accurate diagnoses. The diagnostic accuracy of the GAD and alcohol use disorder modules were not able to be sufficiently evaluated in this research; however, no contraindications for their use were reported. The Mania module was found to be unsuitable for use within an Indigenous Australian cohort. Treating results from the clinical reappraisal as the gold standard (Heffernan et al., 2012), we have explored potential explanations for why participants flagged with a diagnosis during the clinical reappraisal were not provided with the same diagnosis by the CIDI 3.0.
PTSD
The CIDI 3.0 PTSD module was effective in diagnosing a fair proportion of cases that had been diagnosed with PTSD by psychologists. This is of interest, as previous versions of the CIDI (version 2.1) were considered unlikely to capture traumatic events experienced by Indigenous Australian people and at risk of underdiagnosing PTSD (Nadew, 2012). The PTSD module in the CIDI 3.0 has been greatly expanded relative to the CIDI v.2.1 to collect more data on duration and severity of symptoms and to assess up to three episodes of PTSD, including a comprehensive assessment of the episode self-defined by the respondent as the worst trauma (Kessler and Üstün, 2004). It is possible that the more thorough assessment of PTSD in the CIDI 3.0 has improved the ability of the module to accurately capture information required to meet diagnostic threshold for PTSD.
Mood disorders
The CIDI modules were able to identify the presence of any mood disorder (major depressive episode, dysthymia or bipolar disorder) in this sample with fair accuracy; however, when examining each subtype of mood disorder and comparing the patterns of diagnosis across CIDI 3.0 and psychologist diagnoses, it appeared that the cases psychologists diagnosed as MDD were misattributed by the CIDI 3.0 as cases of bipolar disorder. Upon consultation, the psychologists identified that symptoms such as irritability and anger are common presentations of depression among Indigenous Australian peoples (particularly men). When the eight discrepant cases and their responses in the CIDI 3.0 were examined, the majority of these participants endorsed CIDI 3.0 items related to irritability, impulsivity and anger. The literature highlights differences in cultural expressions of depression between Indigenous and non-Indigenous Australian populations. In previous research using the CIDI 2.1 to yield comparative data on the mental health status of Aboriginal and non-Aboriginal prisoners (Butler et al., 2005), MDD was counter-intuitively found to be higher in non-Aboriginal men. In exploring possible explanations for this finding the authors acknowledged that differences in cultural expressions of mental disorder symptoms could have led to insensitivity of the survey instrument (Butler et al., 2007). Similar results were found in a New Zealand research study using the CIDI 2.1 in prison populations, which found the prevalence of mental disorder across the Māori, Pacific Island and European ethnicities to be similar, with the exception of major depressive disorder, which was higher in European men. Although this specific finding was not further discussed, the authors acknowledged that the research design lacked cultural specificity that could contribute to cross-cultural error, and may have missed cultural attitudes specific to mental health issues (Simpson et al., 2004). Brown and colleagues’ qualitative research with Aboriginal men with depression identified common depressive symptoms of excessive sadness, irritability, impulsivity and self-destructive behaviours as a means of suppressing feelings of anger (Brown et al., 2012). In the evaluation of a depression screening tool – the Patient Health Questionnaire (PHQ-9) – within Aboriginal communities, Esler and colleagues found that participants often presented with symptoms of anger that the PHQ did not adequately address or capture (Esler et al., 2007).
Although the CIDI 3.0 Depressive Disorders module does contain items that relate to anger, irritability and impulsivity, the hierarchical nature of the
This suggests that the CIDI 3.0 is unable to effectively differentiate between bipolar disorder and MDD in this sample, and when used with Indigenous Australian populations may have a propensity to overestimate the number of people presenting with mania or bipolar disorder. If the mania module is excluded from the selected modules, the depression module appears to be suitable to assess major depressive episodes in Indigenous Australian populations; however, this does create limitations for cases where a diagnosis of MDD is dependent on ruling out mania.
Other
There were too few cases of dysthymia, GAD or alcohol use disorder identified in this sample to make any meaningful comment on the diagnostic accuracy of these CIDI 3.0 modules or the adapted Illicit Drug Use module; however, there were no indications by participants, psychologists or interviewers that there was an issue with the diagnostic accuracy of the corresponding modules. Indigenous mental health experts have previously advised researchers that the CIDI 2.1 was appropriate to assess depression, anxiety and substance use disorders in Queensland Indigenous adults in custody. The authors acknowledged, however, that without validation of the CIDI in an Indigenous Australian sample, they could not preclude the possibility that their findings over- or under-estimated mental disorder prevalence within their sample (Heffernan et al., 2015b). Further research on the diagnostic accuracy of the GAD and Alcohol Use Disorder modules is warranted.
Strengths and limitations
This is the first research paper to assess the diagnostic accuracy of the CIDI within an Indigenous Australian cohort. While the scope of the pilot study was to comment on the accuracy of the instrument, not to draw population-based conclusions from the diagnostic output, it is acknowledged that a small sample size and a sample drawing from a small and specific sub-population may limit generalisability of the results. The levels of consistency that were detected between CIDI responses and clinical interviews may have differed in a larger sample.
As the research engages an urban-residing Indigenous population, the findings may have limited applicability to rural and remote-residing Indigenous peoples. Although all participants were sampled from two Aboriginal Medical Services, the potential effects of sampling bias would be reduced through the within-subjects design.
The research used the most up-to-date version of the CIDI available for use at the time of data-collection, the CIDI 3.0. The CIDI 3.0 corresponds to criteria proposed by the fourth edition of the
Implications for future research
The outcomes from this paper highlight that the CIDI 3.0 can accurately diagnose some common mental disorders in an Indigenous Australian population. If used in epidemiological prevalence studies, it may be used with caution, taking the diagnostic accuracy of various modules into consideration. Although the focus of this paper was on common mental disorders relevant to the scope of the larger research project, the CIDI 3.0 contains many other diagnostic modules that cover the full range of Anxiety, Substance Use and Impulse Control disorders. Further research is needed to comment on the diagnostic validity of these.
Given that participant interpretations of survey items and their corresponding responses are influenced by social and cultural contexts, the research also highlighted that Indigenous Australian people’s cultural expressions of mental and substance use disorders are likely to impact on the diagnostic accuracy of structured diagnostic assessments if not carefully accounted for. The unsuitability of the CIDI 3.0 Mania module in this research specifically points to this. Further research to accurately measure severe mental disorders such as bipolar disorder in this population is imperative, particularly as they result in high levels of disability relative to some other mental disorders and require dedicated service provision (Begg et al., 2014; Vos et al., 2007).
More broadly, the work undertaken in this research was part of a broader scope of work undertaken for the QUIMHS project to develop and test processes for the successful administration of structured diagnostic assessments within an urban-Indigenous Australian sample. Continued efforts such as these to produce more valid and accurate population estimates of mental and substance use disorders for Indigenous Australians hold potential for improved mental health outcomes through equitable allocation of resources, investment in early screening measures and more informed service provision planning.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
