Science needs better reporting to improve translational mental health research

To the Editor

Increasingly, results of clinical trials of psychological interventions are glossing over science in favour of a positive narrative. This practice adversely affects the establishment of basic science (i.e. the mechanisms by which interventions may work), the application of basic science to practice and patient outcomes. Practitioners and patients need accurate science-based information about likelihood of treatment effectiveness to make informed decisions. One example is antidepressants. While they are widely prescribed, they only improve symptoms in approximately 20% of patients (Institute for Quality and Efficiency in Health Care, 2006). The effectiveness for specific patient groups are needed to inform practitioner and patient decision-making is often very difficult to decipher in publications when nuanced information is omitted.

Other common reporting problems found in peer-reviewed journal papers include global statements of intervention effectiveness, using symptoms, such as anxiety and depression, as synonymous for psychiatric disorders, using parametric, rather than nonparametric tests, when statistical assumptions are violated, using p-values to imply effect size, omitting details of the proportion of the sample that achieved reliable change to demonstrate the likely real-world impact of the results, and not interpreting results within the limitations of the measures used and the positive and negative results of the study.

The identification of initial elevation bias, whereby self-report measures of internal negative mental states have elevated results at the first administration (Shrout et al., 2018), as one example, has significant implications for psychological and psychiatric research methodologies, especially in interpreting cross-sectional studies of prevalence of self-reported negative symptoms to calculate the prevalence of mental illness in population studies. Similarly, in a recent study I conducted with university students, the K10, a commonly used measure of psychological distress in clinical and epidemiological research, had an unacceptable 1 month test–retest reliability of .55. While this may have been sample-specific, population-specific, a measure characteristic, initial elevation bias or something else, it highlights the need for the limitations of self-report measures to be given more consideration when interpreting and reporting results.

While publication bias has been widely discussed as a cause for fewer published negative scientific findings (e.g. DeVito and Goldacre, 2019), in the ever-increasing quest for headlines by publishers and authors, more nuanced, factual, and balanced reporting can be counterproductive. However, better mental health outcomes for patients depend on publishers, authors and reviewers demanding higher standards. Rather than glossing over inconvenient truths, better science may beget better science and allow us to more quickly identify and disseminate effective treatments for mental illnesses.