Learning from previous disasters: Potential pitfalls of epidemiological psychosocial research in the COVID-19 environment

Abstract

It is critical to understand the psychosocial impacts of the COVID-19 pandemic on populations around the world. In this article, we highlight the key challenges associated with epidemiological psychosocial research in a disaster context and reflect on lessons learned from firsthand experience over the last decade in Christchurch, New Zealand, following the 2010/2011 Canterbury earthquakes and 2019 Mosque attacks. We make recommendations for study design to improve the quality of research evaluating the impacts of the COVID-19 pandemic, the inclusion of positive outcome measures and the need to evaluate a range of cultural contexts. We hope that highlighting these areas will improve research and result in a better understanding of the psychosocial impacts of the pandemic.

There is increasing recognition of the importance of understanding the psychosocial impacts of the COVID-19 pandemic on populations around the world (World Health Organization, 2020). This is an evolving issue arising from the morbidity and mortality of the infection and the social and economic impacts of public health restrictions. Reflecting the global nature of the pandemic, a collaborative and interdisciplinary approach has been embraced by the mental health research community with multi-centre international study designs, and the online sharing of information including ‘living’ sites where published studies (EPPI-Centre, 2020) and systematic reviews are continually updated (The DEPRESSD Project, 2020).

There are, however, challenges associated with epidemiological psychosocial research in the context of a disaster such as the pandemic. We discuss these, reflecting on lessons learned from firsthand experience over the last decade following the 2010/2011 earthquakes and 2019 Mosque attacks in Christchurch, New Zealand.

Considerations related to study design

Systematic reviews from previous disasters have consistently highlighted issues which have resulted in the variable quality of publications (Goldmann and Galea, 2014; Norris, 2006). While this may reflect the complexity of the environments in which studies are conducted, it is important to learn from this to prevent a repeat of the problems in the context of the pandemic. These issues are in the following domains.

Sampling

The extant literature is predominantly sourced from observational studies in adults from the general population. These are often convenience samples and may be more likely to include those who are highly impacted or those seeking help. Studies using convenience samples, although easier to implement, are not representative and this limits the generalisability of the findings. Youth and old age populations are typically under-represented. In previous pandemics, there has been an understandable focus on those in healthcare roles, but in the COVID-19 environment, we would suggest it is also important to assess the impacts on those working in other essential roles (e.g. in food supply, waste removal and transport). There is also a dearth of literature from vulnerable populations, for example, those with a mental illness, refugees and those from different cultural contexts. Also study samples are often small, thus fail to recruit sufficient numbers from minority groups, which is essential to allow a society wide understanding of disaster impacts.

Use of appropriate comparison

The failure to utilise a design that includes a case-control (i.e. exposed vs non-exposed population) or comparison with pre-disaster measures severely limit conclusions that can be drawn. Cross-sectional studies without either of these comparators, which for example, simply report percentages of participants with scores above cut-off thresholds, do not take into account variation between populations and typically overestimate prevalence rates. Such studies also do not allow estimation of disaster-associated morbidity in excess of baseline, which makes it impossible to differentiate between long-term factors and new impacts deriving from COVID-19.

Longitudinal assessments

As noted above, single time point assessments only provide a snapshot in time and are of limited value as compared to longitudinal assessments which are far more informative. In the context of the pandemic, these could follow change in symptoms over time (e.g. at different stages of social restrictions or with further waves) or comparison with pre-COVID-19 measures, for example, from longitudinal studies in progress before the pandemic.

Mode of delivery

Many of the public health restrictions have necessitated the rapid growth of online or other remote modes of assessment which have been helpful. However, these are also associated with limitations particularly the requirement of access to this technology which means that those without this are under-represented.

Considerations around choice of measures

Even before the pandemic, there had been calls for an alignment of agreed measures to be used across mental health research. In the context of the COVID-19 pandemic, this has been even more strongly emphasised to enable cross-cultural and other comparisons. Ideally, these would include standardised scales whose psychometric properties have been validated in different cultural contexts and for which accurate translations are available.

Measures of mental health

The key psychosocial domains likely to be impacted in the COVID-19 environment are those of psychological distress, depression, anxiety, trauma-related symptoms and loneliness (Talevi et al., 2020). We would suggest that the choice of scale needs to take into account the availability of established normative data for the context in which it is to be used, for example, if a measure has been collected previously from national survey or census data before the pandemic. This may then allow longitudinal comparisons with routinely collected outcome measures. This approach may, however, limit the degree to which measures can be common across studies in different countries.

Positive outcomes

There has been increasing recognition that positive outcomes have also been reported after disasters and that it is important to further our understanding of this. This could be facilitated by including established psychometric measures of well-being, resilience and post-traumatic growth, assessment of coping strategies and other reported positive outcomes.

COVID-19 measures

A few specific measures with narrow focus such as anxiety about COVID-19 have been validated psychometrically (Ahorsu et al., 2020; Lee, 2020). However, these only assess one, or a few aspects of the challenges posed by the pandemic rather than the multiple and overlapping impacts. Other scales have been developed with a broader focus including questions on health, economic impacts (e.g. change in financial, employment status) and social impacts (e.g. bereavement, isolation, schooling, availability of physical social space) (Covid-Minds, 2020).

There is a substantial literature suggesting that adequate assessment of exposure in a disaster context should include both the immediate impacts and the secondary life stressors that often follow. This approach was adopted by the Christchurch Health and Development Study after the earthquakes by developing a measure of exposure which was able to take into account both exposure to the immediate impacts of the earthquakes (shaking and damage) and exposure to the often lengthy period of stress and difficulty due to the adverse personal, social and economic consequences that followed (Fergusson et al., 2014). Questions were modelled on the Holmes and Rahe Social Readjustment Scale to assess exposure (Yes/No) to life events and the associated level of stress or difficulty (on a 5-point Likert-type scale) (Holmes and Rahe, 1967). From this a comprehensive measure of impact of the earthquakes could be determined. Similarly, we propose that the overall impact of the COVID-19 pandemic exposure needs to assess both exposure to COVID-19 and the secondary life events and stressors that have followed.

Contextual factors

In addition to the degree of COVID-19 exposure, there are likely a number of mediating factors that have a role in determining the extent of COVID-19-related effects and may give indications for preventive interventions in future pandemics. Some of these factors may be specific to local environments, whereas others will be influential in all settings. We suggest describing the healthcare system in which the study takes place, the political and social environment, and the level of COVID-19 infection in the community. Specific questions on fear of infection, social circumstances, financial and employment status, time for leisure, places to exercise and access to communication technology will assist in interpretation of any study findings.

Multicultural perspectives

The impacts of COVID-19 have been felt globally. However, while there is a burgeoning literature on the mental health impacts of COVID-19, most of this is from China, Europe and the United States, with very little from developing countries. This likely reflects many pre-existing challenges which have been previously described (Bhui et al., 2003). These include broad issues relating to the stigma of mental health problems and practical issues such as access to Internet and phone, which may limit study participation. It is important to recognise that disasters may be constructed differently by different cultures based on worldviews and knowledge.

It is therefore vital that culturally appropriate understandings underpin the research approaches taken through the involvement of culturally informed researchers in study design from the outset. There are also challenges relating to the validity of measures of mental health in different languages with the recognition that adapting a questionnaire for use in a different language and culture involves much more than a literal translation of words. It is important that translators are experienced in both languages, the cultural context and have cultural knowledge of expressions of mental distress. It is also recommended that translations are back-translated and cross-culturally validated in the target culture.

Concluding thoughts

While there are significant challenges ahead, the pandemic has shown that during tumultuous times, and perhaps especially at times of need, we can come up with new ways of working including increasing collaboration and sharing knowledge. The importance of understanding the psychological impacts of COVID-19 has been widely recognised, but these need to be assessed in a considered way (Table 1). This includes careful study design and a broadening of the understanding of the effects to include the possibility of positive outcomes and to take into account a range of cultural and contextual contexts. It is crucial that research is of high quality in order to understand the psychological needs, inform policy and provide effective interventions.

Table 1.

Summary of recommendations of factors to consider in research design in disaster settings.

	Consideration	Recommendations
Study design	Sampling	Sample is representative and includes less studied groups, for example, younger and older, vulnerable (e.g. those with mental illness, refugees) and different cultural contexts
		Sample size estimation is appropriate
	Comparator	Case-control or pre-post
	Number of assessments	Longitudinal assessments, that is, over time or pre-post
	Mode of assessment	Benefits but limitations associated with use of online technology
Measures	Psychological functioning	Measures are validatedPreferable to use those where population normative data is availableInclusion of positive outcome measures
	COVID-19 specific	Cover range of impacts, that is, health, social and economicExposure measure includes measure of stress from immediate and later impacts
	Contextual mediating factors	Both specific to local environments and those across all settings
Cultural understanding	Improving involvement	Culturally informed researchers involved from outset
	Translations	Use of valid and culturally appropriate measures