Abstract
The 2007 National Survey of Mental Health and Wellbeing found that 20% of Australians had experienced a mental disorder in the past 12 months [1]. In particular, 6.2% reported experiencing an affective disorder, and these disorders were characterized as severe, disabling and associated with a high level of service use. Research into affective disorders in Australia is required to address this high level of burden. However, to make a difference to the lives of consumers, the research must be focused on issues they believe are relevant. It has been argued that those who experience mental illness are the best placed to identify existing gaps in research and how their needs may best be met [2].
Several studies have investigated the research priorities of mental health consumers in the UK [3–6]. However, these studies have used primarily qualitative methods and none explicitly compared consumer priorities with the existing mental health research effort. In an earlier Australian study Griffiths and colleagues [7] did undertake a quantitative survey of consumer and other stakeholder research priorities for mental disorders. This study also included a comparison of existing published research with the priorities of stakeholders. However, the survey items employed were derived from the existing research base, limiting the scope to researcher-generated topics. It is possible that provided with the opportunity, consumers would prioritize research on topics, target groups and in settings not currently undertaken by academic researchers. Finally, the Griffiths et al. study focused on stakeholder priorities for mental health research in general rather than specific mental disorders.
The current project (Study of Consumer Priorities (SCOPE) for Research) extends and refines Griffiths and colleagues' work with the objective of identifying consumer priorities for research into depression and bipolar disorder specifically using consumer-derived categories. This paper reports the results of the quantitative phase of the project, comparing consumers' priorities for research with an analysis of previous research to provide a framework for research into affective disorders that is informed by a consumer perspective.
Methods
The Australian National University Human Research Ethics Committee provided ethics approval for the study.
Survey of consumers
Participants
Participants were recruited from beyondblue: the national depression initiative's e-network, an electronic database of people who have registered to receive email updates about depression and related disorders (N = 16 257) (http://www.beyondblue.org.au). The network is open to anyone with an interest in mental health but the invitation for the current project asked that only those with personal experience of depression or bipolar disorder participate in the survey. A total of 952 people completed and submitted a survey, of whom 754 self-identified as experiencing depression and 198 identified as experiencing bipolar disorder. A further 498 participants commenced but did not complete or submit the survey and were treated as having withdrawn from the study.
Overall, the profiles of participants in the depression and bipolar disorder groups were similar; there were no significant differences for demographic characteristics. The mean age of the participants with depression was 41.1 years (SD = 12.9) and the mean age for the participants with bipolar disorder was 40.8 years (SD = 11.1). Participants were predominantly female (72.7% of participants with depression; 73.6% of participants with bipolar disorder) and from urban areas (87.7% of participants with depression; 84.5% of participants with bipolar disorder). The majority had some post-secondary education: 71.2% of participants with depression and 74.7% of participants with bipolar disorder had a trade certificate or higher.
Survey
The survey consisted of 301 items in five sections and was developed from the combined data for depression and bipolar disorder research from an earlier qualitative phase of the project [Banfield M: unpublished data, 2010]. Four survey sections focused on priority setting and are relevant to the current study. The first two sections focused on the topics on which research should be conducted. This included 16 broad topic areas for research and 141 specific topics. The decision to include both broad and specific topics was influenced by two factors: (i) participants in the qualitative phase of the project defined priorities in terms of both broad areas for research and specific research topics, and (ii) previous research has found that differences in priorities are often found in the specific topics [2–4]. The third section addressed specific groups in the population that research should be targeting (81 items). The fourth section asked participants to rate the importance of conducting research in 45 different settings.
Participants were asked to rate all items on a 5-point Likert scale ranging from ‘very low priority’ to ‘very high priority’. As it was considered likely that some consumers would have no knowledge or no opinion on some topics, a ‘don't know/no opinion’ option was included to avoid artificially generated opinions [8].
The online survey was created using an open-access online survey development program, hosted on a secure server and delivered via a password-protected web portal. Participants were emailed an information sheet about the project containing the URL and access details for the survey portal page.
Two versions of the survey were created, one framed in terms of depression and the other in terms of bipolar disorder. Participants were directed to the appropriate survey based on a preliminary question regarding personal experience with depression or bipolar disorder.
Participation was anonymous and participants were required to complete the survey in one session. A complete response was defined as the participant reaching the end of the survey and clicking ‘submit’ regardless of how many questions had been answered.
Analysis
Data were analysed using SPSS version 15 (Chicago, IL). Initial examination of descriptive statistics revealed a strong negative skew on most items with participants rating most items of high or very high priority. The data were therefore dichotomized with high and very high priority ratings combined into the single category ‘important’ and the remaining options (medium, low, very low and don't know/no opinion) forming the remaining category. Primary analyses comprised the percentage of participants in the depression and bipolar disorder groups who rated an item as important. Chi-square tests for independence were used to compare ratings by people with depression to ratings by those with bipolar disorder. Due to the large number of comparisons, the significance level for all analyses was set to p < 0.01.
Content analysis of Australian research literature
Selection of abstracts
PubMed, PsycINFO and the Cochrane Library were searched using the search string: (depress∗ or dysthym∗ or melanchol∗ or mania or manic or cyclothym∗ or ((cyclic or mood or affective) AND disorder) or bipolar) AND (vic or victoria or nsw or ‘new south wales’ or qld or queensland or sa or ‘south australia’ or wa or ‘western australia’ or act or ‘australian capital territory’ or nt or ‘northern territory’ or tas or tasmania or australi∗).
The search was limited to entries with an abstract, in English, about humans and published between 1997 and 2006.
The searches yielded a combined total of 7631 results. Each abstract was assessed for relevance by one rater. Inclusion criteria were that (i) the primary purpose of the study was to investigate depression and/or bipolar disorder; and (ii) that the corresponding author was located in Australia. Studies that measured depressive or manic symptoms only as secondary outcomes without specific focus on the disorders were excluded. A total of 4734 abstracts were excluded on criterion one, 1039 were excluded on criterion two, and 33 were excluded because they lacked an abstract. The final abstract set after removal of duplicates included 1250 abstracts. To avoid including duplicates of the same research (e.g. in reviews, methodological papers or commentaries) only abstracts reporting primary research were included in the final analysis (n = 931).
Coding
The content of the abstracts was coded by the first author (M.B.) against the items comprising the consumer survey. Each abstract could be coded as addressing up to three items in each of the four survey sections. That is, up to three broad topic areas, up to three specific topics, up to three target groups for research and up to three settings in which to conduct research. Coding was as specific as possible. Within each section, where both a general and a more specific item matched an abstract, only the more specific item was coded. If the broad topic area, specific topic, target group or setting in which the research reported in the abstract was conducted were not contained in the items from the consumer survey, the details were recorded separately in the ‘other’ category. This method was based on the earlier Australian priorities study undertaken by Griffiths et al. [7]
Analysis
As an abstract could be allocated to more than one coding category for each survey section, articles allocated multiple codes were weighted so that they summed to 1.0 for each of the four sections. For example, if an article discussed both diagnosis and treatment, each of these was assigned a weight of 0.5 for the broad topic areas. All percentages discussed in this paper relate to weighted values.
Comparison of consumer ratings and published literature
Items were ranked for both the consumer ratings and literature analysis. Tied items were assigned the mean value of the tied ranks. The highest ranked consumer-rated broad topic areas, specific topics, target groups for research and settings in which to conduct research for each disorder were then compared with their overall rankings for the literature analysis.
Results
Tables present the rank and percentage of consumers rating items as important, along with the rank and percentage of literature on those items. All 16 broad topic areas are included, but to focus on the highest priorities, only the top 10 consumer-rated specific topic areas, target groups for research and settings in which to conduct research for both disorders are included. Full tables are available from the corresponding author upon request. Significant differences between the ratings of the two consumer groups are marked with an asterisk.
Broad topic areas for research
As shown in Table 1, there were large discrepancies between the highest ranked consumer research priorities and the published research for the broad topic areas. Consumers rated management as their highest priority but this topic area was ranked only 13 in the published literature for depression and 12.5 for bipolar disorder. Conversely, highly ranked topic areas in the literature such as description and characteristics were ranked 14 for consumers with both disorders. The rank orders were similar for both disorders and there were no significant differences between the groups on the broad topic areas.
Ranked percentage of consumers rating broad topic areas as important compared with ranked percentage of literature
Top 10 specific topics for research
Of the 141 individual topics for research, the topics most often endorsed by consumers with both disorders were matching the individual and treatment and effective coping strategies for living with depression (Table 2). However, there was little or no published research on these or the majority of other top consumer-rated specific topics. Only one topic for depression, suicidality and suicide prevention, and two topics for bipolar disorder, effective medications and long-term effects of medication were also amongst the highest-ranked topics in the literature.
Top consumer-rated specific topics for research compared with published research.
p > 0.01
There was a large overlap in the top-rated specific topics for research for people with depression and people with bipolar disorder: six topics were in the top 10 for both groups. One topic, education and training for police and crisis services was rated as important by significantly more people with bipolar disorder than with depression.
Top 10 target groups for research
Table 3 presents the top 10 consumer-rated target groups for research compared with the published literature targeting these groups. As for the broad topic areas and the specific topics, there was very little research on the highest ranked target group rated by people with depression, people at risk of suicide, and no research on the highest ranked target group rated by people with bipolar disorder, people at high risk. Only three groups, adolescents, people with a dual diagnosis and people with a family history of mental health problems were also highly ranked in the literature for either disorder. There was little or no research on the remaining highest-ranked target groups rated by consumers.
Top consumer-rated target groups for research compared with published research.
p < 0.01
Seven of the top-rated target groups for research were shared between participants with depression and participants with bipolar disorder. Four target groups, children who were abused, people who have experienced trauma, people who have experienced domestic violence and people in stressful occupations were rated as important by significantly more people with depression than with bipolar disorder.
Top 10 settings in which to conduct research
Compared with the three sections already described, there was greater concordance between the highest ranked consumer-rated settings in which to conduct research and the amount of published research conducted in those settings (Table 4). Community treatment teams and services, GP settings and hospitals were amongst the highest ranked items for consumers in both groups and were also some of the highest ranked settings in which the published research on both disorders was conducted. Psychiatrist settings and rural and remote areas were also similarly ranked for consumers with depression compared with the published depression research. However, there was less than 1% of published depression research and no bipolar disorder research conducted in the remaining top consumer-rated settings.
Top consumer-rated settings in which to conduct research compared with published research
Nine of the top-ranked settings in which to conduct research overlapped between participants with depression and participants with bipolar disorder and there were no significant differences in the ratings between the two groups.
Discussion
The emphasis in the priorities of both consumers with depression and consumers with bipolar disorder was on researching the areas of effective professional help and the proactive involvement of consumers in their own care, on undertaking research within a variety of professional health settings, and on research focused on population groups perceived as at greatest risk of depression or bipolar disorder. However, few of the top consumer priorities ranked highly in terms of the published literature. For example, only one specific topic rated highly by consumers with depression and two rated highly by consumers with bipolar disorder were amongst the highest ranked in the published research. By contrast, topic areas that were the focus of a substantial amount of research were not rated of high importance by consumers. For example, much of the research on both disorders focused on the description and characteristics of depression and bipolar disorder [9–11] but this was one of consumers' lowest ranked broad topic areas. Even where ratings of broad topic areas suggested concordance between consumer priorities for research and the current research effort (such as treatment) a consideration of the specific topics indicated that the research was not on the topics of greatest importance to consumers. This highlights the different perspective consumers can add to defining research priorities and the importance of unpacking the specific topics of interest.
Overall, the research priorities of consumers with depression were quite similar to the priorities of consumers with bipolar disorder. The majority of top-rated items in each section overlapped between the two groups and there were few significant differences. However, Australian research on bipolar disorder was concentrated heavily in just a few areas, and many more items ranked highly by consumers were unrepresented in the literature on bipolar disorder compared with the literature on depression. Therefore, whilst there is a disparity between consumer priorities and published research for both disorders, the disparity appears greater for research on bipolar disorder in Australia than for depression.
To our knowledge, no previous studies have directly compared consumer priorities for depression and bipolar disorder research with research output. However, one study has reported comparisons between stakeholders' (including consumers) priorities and research output for mental disorders generally [7] and another study undertook a similar comparison for the field of suicide prevention [12]. Consistent with the findings of the current study, these authors also reported many disparities between stakeholder priorities and the existing research. For example, Griffiths et al. found that the topics which were the subject of the most publications for all mental disorders in 1998 were ‘risk factors and associations,’ ‘assessment and classification’ and ‘the description of disorders’. However, stakeholders most consistently prioritized research into ‘prevention and promotion’, an area which the authors found was not well represented in the literature [7]. Robinson et al. found that whereas stakeholders prioritized intervention studies, most of the literature focused on descriptive epidemiology [12]. Thus, priority-setting exercises that include comparison of stakeholder priorities with the existing research effort consistently demonstrate significant discrepancies between the two.
An important additional finding of the SCOPE for Research project is that consideration of the specific topics of interest to consumers can reveal important differences between consumer priorities and the existing research effort that are not evident from a consideration of the broad topic areas. Davidson and colleagues [2] observed that unless consumers are involved in the articulation of the research questions, the findings may not accurately reflect the issues they consider of most concern. The results of SCOPE for Research and other studies such as those conducted by Rose et al. [4] and Owens et al. [3] are consistent with Davidson and colleagues' observation. All of these studies found that although consumers supported research on medications, the specific topics of interest focused on how medications and their effects related to the overall life experiences of mental health consumers.
Studies such as that conducted by Griffiths et al. [7] which relied on broad categories or topics developed by the researchers may therefore yield misleading results. Griffiths et al. developed their classification from existing literature for both the stakeholder survey and literature coding, the opposite approach to that taken in the SCOPE for Research project. It is possible that Griffiths et al. would have observed less concordance between the ratings of the stakeholder groups had their survey contained more detail or included items developed by consumers and the other specific stakeholder groups sampled in their survey. Similarly, greater detail in both the literature coding framework and the stakeholder survey may have revealed greater differences between stakeholder ratings and the literature.
Limitations
Participants were recruited from an existing e-mail network and were predominantly female. It is possible that the perspectives of network members differ from those of others with depression and bipolar disorder, and that the research priorities of women differ from those of men. Participants' knowledge of existing research and the decision criteria they used in prioritizing research are also not known. It is possible that the higher ratings given to topics with little recent research was influenced by knowledge of existing gaps.
It is also a limitation of the study that the reliability and validity of the questionnaire are unknown and that there was little differentiation in the ratings participants gave for many topics. Despite being asked to consider which topics were the most important in order to prioritize research, many participants still rated the majority of topics similarly.
The published research was analysed based on the abstracts of peer-reviewed studies rather than the full research articles, was restricted to a specific time period and the analysis was conducted by a single researcher. The necessary brevity of abstracts meant that occasionally the purpose, target groups and settings of the research were difficult to determine. Judgement of the strength of the research effort was also based solely on the number of papers found on each topic, with no measures of quality or possible impact of the research. It is possible that a small number of high quality studies are sufficient to address some research questions.
Finally, we acknowledge that research is a global enterprise. It is possible that research areas that were not strongly represented in the Australian literature are researched elsewhere.
Conclusion
The results of this study indicate that consumers have very specific ideas about the areas they perceive as requiring mental health research, and that at present the Australian mental health research effort is addressing few of these areas. The results highlight the importance of detail when exploring priorities as these can uncover important differences in perspective which may be obscured if items for coding or surveys are broad.
The future depression and bipolar disorder research effort in Australia should take into account the gap between the existing research effort and consumers' priorities for research. Taking this combined approach may ensure that the priority attached to some research areas that may be of lower consumer priority are re-evaluated, existing strong research into priority areas continues and important gaps are addressed.
Acknowledgements
We gratefully acknowledge the time and effort of the participants in the SCOPE for Research project. We also acknowledge beyondblue: the national depression initiative for distributing the invitation to participate.