Abstract
The common-place quantification of humanitarian disasters enables rapid and informed crisis responses. In disaster settings, understanding feelings and perceptions regarding individuals’ experiences, livelihood disruptions and coping mechanisms can also be valuable for extending and deepening quantitative insight. This paper explores the potential for diary methods to capture extensive, nuanced data from marginalised groups during a disaster, by drawing upon a study with 100 young diarists (aged 15–29) who produced 1418 diary entries over 4 months. In particular, we share how diary-methods can be designed inclusively, through addressing themes of equitable research partnerships, supporting more vulnerable participants, ensuring data quality, data management, participatory analysis, and budgeting for collaborative research.
Introduction
At times of crisis – including pandemics and natural disasters – the tendency to rely upon quickly collected, rapidly analysed numerical data (Himelein et al., 2020; World Bank, 2020; IMF, 2020) means qualitative insights can too often be missed. Despite their sometimes longer lead times and lower participant numbers, qualitative data can play a crucial role in enriching understanding of people’s experiences, perspectives, and how social issues intersect in place (Atieno, 2009). Use of qualitative research – including during previous crises – can support policy makers to (re)conceptualize people’s situations, aspirations, and capacities; facilitating multifaceted and meaningful explanations of vulnerability and empowerment, potentially leading to novel and more inclusive policies (Terzis et al., 2022; Raven et al., 2018; Simosi et al., 2015; Heltberg, et al., 2012; Walker & Hall, 2010).
Such crises include the COVID-19 pandemic, which has set back prior gains in reducing poverty and inequality, impacting vulnerable groups most severely. Insights stemming from qualitative research are useful to deepen understanding and fortify responses for more equitable recoveries. Although qualitative data collection typically involves considerable in-person presence and coordination (Vindrola-Padros et al., 2020, p. 2193), some qualitative methods were readily adapted to social distancing requirements of the pandemic. For those with good access to computers and sufficient bandwidth, the popularisation of video calls has opened new possibilities for video interviews (Khan & MacEachen, 2022; Barford & Ahmad, 2022; Rahman et al., 2021) or telephone surveys (Cheema et al., 2021; Sanchez et al., 2022). However, this rapid digitalisation heralds another form of disconnection for more marginalised groups without digital access, reinforcing the persistent need to purposefully design-in the inclusion of marginalised groups for research and practice (Chambers, 1983, 2017; Barford & Gray, 2022). At times of disruption and disaster it is essential that marginalised people, who are often worst impacted by disasters, are understood and responded to.
Qualitative diary data, collected over months rather than weeks, could be especially valuable during crises that are protracted in nature and have long recovery periods. Months-long data collection allows for observations of changes as they occur. The COVID-19 pandemic, and associated social and economic fallout, is one such crisis. Policy responses to COVID-19 can be helpfully informed through methods that provide stronger understandings of the social dynamics of both the uneven pandemic impact and the subsequent ‘K’ shaped recovery, showing the variable rate and scale of recovery for different socio-economic groups (Bheemaiah et al., 2020). Governments, humanitarian organizations, and researchers often seek respectful engagement and equitable responses to the complex needs of diverse groups during disasters and subsequent recovery; we contribute to this agenda by sharing methodological details of a large diary-based study, conducted at a time of strict social distancing and restricted travel. This paper demonstrates the practical and knowledge-generating potential of diary-based methods conducted over distance at times of disaster, while candidly describing the obstacles and limitations that we faced.
Study Context
The ‘Youth Livelihood Diaries’ research project collected 1418 diaries from 100 young people (aged 15–29) in Indonesia and Nepal over 16 weeks during 2021 (Mueller et al., 2022). Our study focused on ‘clusters’ of young people with specific labour market vulnerabilities that were exacerbated by the COVID-19 pandemic (ILO and ADB, 2020, p. 7; Khanal, 2020; Pattinasarany, 2019; Rai, 2020; Sapkota et al., 2020). The five clusters, mirrored in both countries except for cluster five, were: (1) young mothers; (2) out-of-work tourism and trekking workers; (3) migrant waste pickers; (4) health care workers; and (5a) persons with disabilities (in Indonesia) or (5b) lesbian, gay, bi-sexual, trans, queer or intersex persons (LGBTQI+) (in Nepal). Due to the distinctive livelihood and personal characteristics of each cluster, as well as marked differences between national economies and COVID-19 policies, each cluster was analysed independently and also compared.
The research team consisted of 10 local young rapporteurs (i.e. research cluster leaders: one per cluster per country), several in-country project managers and remote advisors overseeing coordination, and two remote researchers at the University and non-profit level (Figure 1). The funders were closely involved in steering and overseeing the research. This organizational structure, with a sizeable multi-country and inter-generational team, required strong communication and good data management systems to receive, process, and analyse 100 diary entries each week (see Data Management and Analysis section). Information flows and research team structure. This organisational structure was mirrored in Nepal and Indonesia, with the same project lead and technical research team for both countries. The clusters each consisted of 10 diary writers, who submitted weekly diaries to their rapporteurs. The clusters were predefined demographic groups selected based on their exposure to the social and economic impacts of COVID-19. The rapporteurs managed each cluster, ran cluster-level discussion groups, performed cluster level data analysis, and translated diaries into English. Country coordinators managed the rapporteurs, ensured data quality, and ran rapporteur discussion groups. The technical team administered the project, oversaw research design, troubleshooting, and analysis.
Diary-Based Research Design.
aSee Figure 1 for organizational flow chart of research team across countries.
bThe Poverty Probability Index, developed by Innovation for Poverty Action, generates a baseline measure of poverty. Out of 135 candidates from Indonesia and 172 candidates from Nepal 50 Diary Writers in each country were selected based on the cluster specific primary criteria and Poverty Probability Index (PPI) as recommended by the literature review. The PPI cut-off point used was 48 for Indonesia and 59 for Nepal. Priority was given to those Diary Writers who were determined to be poor according to this scorecard. More information is available at: https://www.povertyindex.org/about-ppi.
cSee Table 2 for a list of diary questions.
Diary prompts. These seven questions are the recurring diary prompts for weekly diary entries. Several further questions were added according to the diary writer clusters, in order to gather group-specific demographic information in addition to overarching experiences.
In the following sections of this paper, we identify the design, benefits and limitations of the method of this research project, alongside wider learnings from which we hope other researchers might benefit. The Qualitative Disaster Research at a Distance section discusses the potential of diary research in disaster settings, with the COVID-19 pandemic offering an example of this implementation. The Inclusive Diary Research Design section details our efforts to design an inclusive research project, and where and how we sometimes fell short with this agenda. The Data Management and Analysis section discusses implementation, including international collaboration, data management and analysis. The conclusion summarizes transferrable takeaways for effective disaster diary research.
Qualitative Disaster Research at a Distance
Disasters are variously defined, sometimes as ‘physical happenings’, other times as ‘social constructs’ (Quarantelli, 1998, p.xiv). The COVID-19 pandemic can be considered as a disaster, according to a broad definition of disasters which includes the substantial health, social and economic impacts of pandemics. Hallmarks of disasters include sudden casualties and/or fatalities, damage to property, social disruption, significant economic cost, or psychological damage (Quarantelli, 1998, p.xiv). In such scenarios, the rapid availability of reliable data on the detailed shocks, scale and geographies of a crisis is critical for humanitarian agencies and governments to respond urgently and efficiently. At times of disaster, numerical data help establish order and knowledge, partially representing the unfolding situation by applying universal categories to establish scientific authority on vital patterns of disease morbidity and mortality, amongst other measures (Foucault, 1977; Barford & Dorling, 2016; Barford, 2014; Dorling & Barford, 2007; Smallman-Raynor et al., 2015).
Qualitative research in disaster contexts often performs less well than quantitative approaches in terms of the basic requirements for disaster response data: immediacy, cost effectiveness, and coverage of the population of interest. While quantitative data are useful in providing key indicators for humanitarian disaster targeting and rapid aid response – such as changing disease patterns and income levels – qualitative methods offer insight into experiences and opinions during and following disasters. Such data address gaps in evidence to inform sequenced and medium-term measures for recovery and renewal (Sou et al., 2021). Recovery policies and programmes which emerge from quantitative insights alone sometimes lack important “context, history and meaning” (Merry, 2021, p. 1); accessing local knowledge and detailed insight can complement and extend the quantification and standardization of basic human needs (Glasman, 2019; Merry, 2021; Lawson, 2021). In disaster situations specifically, qualitative data can offer important details concerning the complexity of unfolding scenarios, including personal experiences of livelihood struggles and changing vulnerabilities over time (Ogueji et al., 2021; Nakhaei et al., 2015; Raven et al., 2018), as well as the personal significance and sense-making surrounding these experiences. Detailed insight, captured at the time of the lived experience, can guide prioritization, targeting, design, and implementation of support for recovery and renewal from a disaster.
There has been some recent progress in using qualitative research for more immediate uptake in response strategies. Several case studies demonstrate the value of qualitative data for understanding and responding to crises and disasters. A multi-country study by the World Bank used ‘rapid qualitative assessments’ to supplement quantitative data in the years following the Global Financial Crisis, for example (Heltberg et al., 2012). Qualitative assessments extended insights from household and labour force data, enabling medium-term insight (2008–2011) on individuals’ coping strategies through the globalized market crisis within eight ‘developing economies’ (Heltberg et al., 2012). The 2014-2015 Ebola crises in Liberia is another example of quickly deployed qualitative disaster research in practice. Phone-based interviews enabled socio-economic data to be gathered ‘rapidly’, ‘at low cost’ and ‘where traditional face-to-face interviews [were] not possible’, offering quickly accessible crisis monitoring data on household welfare at a time when researchers could not visit communities (Etang & Himelein, 2020).
In the case of the COVID-19 pandemic, qualitative studies fill crucial knowledge gaps, going beyond standard health crisis data to access deeper understandings of disasters' impacts. Qualitative studies offer insights on mental health variations among unemployed and employed people (Ogueji et al., 2021); effective investment strategies for small holder farmer resilience (Goswami et al., 2021); researcher emotion during qualitative, distanced research (Scott, 2022); and more. Such information can offer depth and intersectional understanding of particular social demographics when considering how disaster research can respond immediately, and also equip policy makers for longer-term equitable recovery and renewal. Despite the promising rise of Rapid Qualitative Research and other qualitative methods to inform decision makers, their usage is still relatively rare (Luciani et al., 2021), to the extent that scholars remark upon the small proportion of qualitative findings about the COVID-19 pandemic integrated in policy and practice (Tremblay et al., 2021).
Diary methods offer particular advantages to qualitative research in disaster or crisis scenarios, especially where data across a large geography and population are required, or where physical geography forms a barrier. Diary research can access diverse groups of diary-writers, in geographically dispersed locations, over extended periods of time (Mueller et al., 2022; Rudrum, Casey and Rondinelli, 2022; Almeida, 2005) without sacrificing rich narratives of people’s unique and complicated experiences (de Lanerolle et al., 2020). Initially utilized to record the sightings of ‘natural phenomena’ in medical, astronomical and agricultural research (Hyers, 2018, p. 2), diary research has become increasingly used within the social sciences in recent decades (Filep et al., 2018; Hyers, 2018; Alia et al., 2013; Langevang, 2007; Meth, 2003). Diaries for disaster research saw a resurgence during the COVID-19 pandemic – emphasising the value of collecting insights which are both detailed and longitudinal during crisises (Mueller et al., 2022; Rudrium, Casey and Rondinelli, 2022; Scott, 2022; Saltzman et al., 2021; Terzis et al., 2022). COVID-19 diary studies have contributed detailed emotional narratives of diarists' experiences (Scott, 2022), providing fine-grained information on individuals’ ‘progression and adaptation’ through the different phases of this crisis (Terzis et al., 2022).
From immediate response to more sustained recovery and renewal, diary methods can enhance rapidly available quantitative data and offer insights for recovery with a 2-week turn-around; while also sustaining data collection in the medium- or long-term. This paper contributes details to the process of operating a cross-country diary project for disaster recovery, emphasizing the financial and human resources required to maintain timeliness, including the substantial task of translation and analysis, alongside the need for robust data organization systems (see Data Management and Analysis section). The benefits of diary methods for disaster research for recovery and renewal include: (i) social-psychological data; (ii) remote data collection; (iii) participant recall immediacy; and (iv) longitudinal insights.
Social-Psychological Data
Diary methods have considerable potential for generating complex and rich data (Scott, 2022; Filep et al., 2018; Jones et al., 2015; Thomas, 2007). Making regular diary entries over extended periods invites diarists to share detailed self-reporting on everyday situations and ongoing processes - offering accounts of their own ‘internal’ thoughts and feelings, as well as ‘external’ situations, events, and the (changing) wider context (Meth, 2003; Filep et al., 2018). Scott’s (2022) 16-month pandemic study used diary methods for their ability to capture ‘rich emotional narratives’ (1); while Eidse and Turner (2014) found diaries offered detail ‘far beyond that collected during interviews,’ (p. 246). Diary data provides insight into ongoing everyday dilemmas, coping strategies, livelihood negotiations and psychological reactions such as stress, helplessness, or hope (Filep et al., 2018; Almeida, 2005; Bakker & Daniels, 2012; de Lanerolle et al., 2020). Such information can be valuable in understanding how people are impacted by, and respond to, disasters and crises such as the COVID-19 pandemic – as interventions can be layered onto existing support mechanisms (be they local or national, informal or formal).
In our own diary study, the large volume of data (1418 diary entries from 100 diarists) offered substantial detailed personal information about the direct and indirect impacts of the COVID-19 pandemic. In response to a series of prompts (Table 2), young people wrote of a plethora of experiences and thoughts including: feelings and details about family relationships; diarists’ financial literacy, media and news becoming a trigger for mental stress; internalized feelings on movement restrictions; or young women’s adaptation domestic/economic responsibilities while home schooling children. While these data could have been captured through interviews and observations, the volume, variety, and emotional intimacy of diary entries was notably greater than the information captured by rapporteurs (research cluster leaders) through supplementary focus groups (three per cluster). We attribute this, in part, to the very personal nature of diary writing. According to their own rhythm, diarists ideally take time to think, pause, reflect, and write. The process of writing can facilitate a new level of sense making and reflection, to the extent that the writing process describes, and in part produces, diarists’ lived realities. The examples below show the type of information shared in diaries: “Once I was so depressed, not because of physical violence, but because of lack of inner peace. I don’t know what to do. Everything is out of my reach, out of my hand.” - Diary excerpt, Indonesia, Health Care Worker (10 April 2021) “I have another thing to say. I was sad. I don’t have a person to confide in. Usually, it was … my mother. I can’t do that anymore, because she passed away three months ago.” - Diary excerpt, Indonesia, Young Mother (16 April 2021)
The level of thoughtfulness, reflection and intimacy shared by the two diarists above was encouraged by the diary format – which offers diarists a space to think through and explain certain facets of their personal experiences. Thus, diaries open pathways to ‘understanding more complex relationships’ (Esquivel et al., 2008). One such insight from our own diary study is how mental health and coping among young people are strongly linked to social connection and labour market integration; yet work and social connection as sources of resilience were themselves disrupted by COVID-19 and the accompanying social and economic fallout. Findings from our study highlight the need for policy and practice to engage more directly with mental ill-health and the specific needs of domestic care workers. Note that the Inclusive Diary Research Design section addresses the safeguarding responsibilities associated with the remote intimacy and detailed knowledge of young diarists’ personal difficulties at a time of crisis.
Remote Data Collection
Written data enable remote collection, which can increase the geographical dispersion of diary writers even when working to tight timelines (Müller et al., 2015). Timely, geographically diverse collection is valuable for data which describe widespread disruptions (Himelein, 2014), as observed with the COVID-19 pandemic (Rudrum et al., 2022). Digital collection of diary entries was used where possible in our study to uphold strict health guidelines, intentionally minimizing travel and face-to-face interaction unlike most ethnographic methods (Sy et al., 2020). One caveat is that some young people without mobile phones borrowed someone else’s, hence some local face-to-face interaction did occur, even with this mainly digital approach. The resulting remote research minimized the risk of spreading disease and abated the risk of project disruptions from fluctuating mobility restrictions. Further minimizing travel limited the carbon footprint associated with our field research, thus contributing to the climate responsibility of social science researchers (Rodríguez & Martínez-Roget, 2021).
Methodological Adaptations by Cluster and Country. Though the study was designed according to the capabilities of our participants, bolded text shows further adaptations made through the study to adjust to lack of mobile phone access, writing difficulties, or lack of digital banking.
Additional adjustments helped to negate the problems of digital disconnection. Funding was provided for diarists to use internet cafes when needed, and diarists were encouraged to borrow smart technology from members of their social network where possible. The Nepali and Indonesian rapporteurs who guided diarists through the study were central to the success of the digital design, as they taught digital skills on WhatsApp and Zoom, meeting in person when allowed. To adapt to the needs of visually and hearing-impaired diarists, our study used subtitles during Zoom calls, trained diarists to use a screen reader to read out text, and ensured that instructions were clear and readable. Overall, 10% of diarists made use of some form of adjustment - people who would have been otherwise excluded from this research due to practicalities (Table 3).
Recall Immediacy
Diaries record events and experiences as close to their occurrence as possible, reducing the impact of recall bias and memory failure (Meth, 2003; Langevang, 2007; Stone et al., 2002). Given the necessity of reliable data during disaster scenarios, the immediacy of data recall offers the most precise details of experiences and needs, specificities which can fade in memory as time passes and people respond to new challenges in a fluctuating situation. In this sense, diaries differ from interviews, which often have a stronger retrospective dimension. Of course, if diarists choose to write about past events rather than the present, a form of ‘retrospective censorship’ may still occur (Meth, 2003).
Diaries submitted at the end of each week in our study captured diarists’ emotions as the pandemic unfolded. These emotions included fear as coronavirus cases rose, hope related to vaccine announcements, desperation when food markets closed, or frustration when government-mandated movement restrictions changed. Capturing these emotions and experiences within a week of their occurrence heightened understandings of how pandemic-related events impacted young people’s livelihood circumstances and mental health in real-time. We were able to observe what conditions supported diarists to cope, and when diarists struggled to adapt to hardship (see Figure 2). Detailed emotional recollections of immediate events and intersectional inequalities provide insight into the physical and psycho-social experience of diary writers on issues such as sexual orientation discrimination, food insecurity, and mental-health coping strategies.
Longitudinal Insights
The protracted nature of the COVID-19 pandemic suited extended data collection. Diary research tends to collect data over a longer period than many other qualitative methods, and this is valuable for studying changes in well-being alongside the fluctuating lives and livelihoods of young people (Scott, 2022; Rudrum et al., 2022; Almeida, 2005; Barford & Coombe, 2019). With diaries, following individuals’ stories as they emerge over time, rather than considering one-off data points, offers distinctive insights (Meth, 2003, p. 198). This is because attitudes and experiences can be cross-referenced between entries to see whether, for instance, hope for the future was well-placed. Further, unfolding situations can be better understood, such that the many factors which contribute to food (in)security, for example, can be traced over weeks, unpicked, and understood; rather than hunger being described with a less nuanced causal explanation.
Our study tracked diarists’ weekly experiences during the pandemic, allowing early insight into which pandemic impacts might be short-term (diarists recovered during the study) or longer-term (impacts with which diarists persistently struggled) (Figure 3 offers a week-on-week observation of access to informal work with changing government restrictions). Of course, many young people already faced livelihood struggles prior to COVID-19, which mediated their experiences of the pandemic (Barford et al., 2021b, 2021a, 2021c). Understanding the temporal impacts of crises – and how these intersect with pre-existing challenges – can help inform government priorities and tailor pandemic recovery plans to ensure that the worst affected sub-groups are well supported. Longitudinal analysis with a Young Mother Diarist in Nepal from Week 1 (March 27, 2021) to Week 16 (July 10, 2021) of the study. These summaries document fluctuating disruptions to informal work following changing government restrictions, as well as other disruptions such as illness and additional caring responsibilities.
For governments to benefit from the insights of diary research, financial and operational resources are required to support research and then implement policy recommendations. As resource constraints lead to certain forms of research cost cutting (e.g., reducing time, limiting possible adaptations for participants, and stretching the capacity of implementation staff), the ability for diary research to remain inclusive, or centred on harder-to-reach groups, diminishes. The next section discusses key points on designing diaries inclusively, sharing the successes, short-comings, and constraints of our own research. This is followed by a section on data management and operations, where we reflect on the potential for using diary methods effectively at scale.
Inclusive Diary Research Design
Barriers To Inclusive Research. The first six biases (in bold and italicized), are based upon Robert Chambers’ (1983, p.13–23) account of biases which threaten the inclusive representation of difficult-to-access groups. While these challenges persist, their balance has changed (Chambers, 2017). We add a seventh barrier: Digital Bias. We also suggest responses to these challenges.
Sharing Control with Partners and Participants
Establishing partnerships of equivalence is fundamental to co-producing knowledge. As an international research team we sought to reduce traditional, often racialised, education and wealth-based hierarchies of expertism (after Frediani et al., 2019; Osuteye et al., 2019). Complementary and dispersed expertise exists throughout the international team – and this project was only possible through the collaboration of diverse people with varied skills and knowledge (Proefke and Barford, forthcoming; Mueller et al., 2022). Local cultural, political, economic, linguistic, practical, and research expertise were contributed by the team working in Nepal and Indonesia; this was essential to running an international study during the COVID-19 pandemic. Further, young researchers acting as rapporteurs disrupted widespread age-based hierarchies through their critical roles in running data collection, problem solving, interpreting the data, and disseminating findings (ibid.). Strong partnerships, regular meetings, and good reliable organisation were crucial. The NGO and University partners regularly engaged the funders to co-develop the diary design, recruitment, ethical and safeguarding procedures, data collection, data analysis, and dissemination. Team meetings were often stretched across time zones from Manila in the Philippines to Chicago in the USA, as we sought to fully engage members at Restless Development Nepal, Rutgers Indonesia, the University of Cambridge, and the Asian Development Bank.
The ten rapporteurs (one per cluster per country), who were young people themselves, ensured diary questions and instructions were contextually relevant to their own groups of diarists. Rapporteurs also taught the UK and USA-based research team how much writing to expect given their close knowledge of the diarists' daily livelihoods. The continuous self-appraisal of our research progress, paired with good communication facilitated flexible adaptations of diary design throughout the project, as we learned what worked and what needed tweaking (Table 3). Our experiences in this research highlighted how erroneous the long-standing unequal power dynamics between researchers in lower- and middle- income countries with researchers based in more ‘developed’ economies often in the Northern Hemisphere. The researchers from wealthier countries had much to learn, and this was partially stalled by travel restrictions preventing in-person learning in Indonesia and Nepal.
Our flexible approach to recording diaries meant that key decisions could be made by the diarists themselves, to suit their own preferences and to best fit diary writing within their lives. Giving diarists control over the type of diary (audio, photo, paper) promoted diarist engagement, by enabling participants to shape the study to their own preferences, literacy and resource access – rather than requiring participants to adapt to a rigid study design (Table 3) (Ellard-Gray et al., 2015; Aldridge, 2014; Meth, 2003). Learning from Meth’s (2003) diary research, we found that such choices can particularly encourage marginalized groups to engage with the study as they can choose the format which suits them best. Other ways that participants exercised control over the research, also following Meth (2003), were deciding where and what time of day to make their diary entries. Like with other equitable partnerships, greater self-determination is likely to improve diarists’ overall experience and sense of empowerment in a positive way, as well as reducing diarist attrition from the study (Meth, 2003).
Distributing control over the research project does not, however, always or automatically increase participant agency (Jones et al., 2015; Langevang, 2007; Meth, 2003). As knowledge production is inherently political (Meth, 2003), we are wary of a version of ‘naive empiricism’ which neglects the ways in which participants’ voices are not necessarily their own (Buckingham, 2009). Occasionally, diarists acknowledged that a researcher would be reading their entries (see below). While this was commonly expressed in a positive way, perhaps because we would later read it, this reinforces that the researcher is never fully absent – even when physically removed from the research site (Jones et al., 2015; Pini, 2001). This creates an unavoidable tension between guidance and freedom in responses within diary-based studies, to the extent that the author of second quote below self-identifies using their cluster name assigned by our project. “In this situation, I am very happy to be able to express my sorrow in a few words through you and hope that life will take a new turn. I hope you would help us to take a step forward tomorrow … always looking forward to learning new things from you, thank you!” - Diary excerpt, Nepal, Migrant Waste Picker (27 March 2021) “Besides the things mentioned above, I would like to thank every madam who has given me the chance to write a diary as a NEET mother.” - Diary excerpt, Nepal, NEET Mother (10 July 2021)
Supplementary Methods
Diary research can miss details of the physicality, culture, activities, and movement within the location where qualitative research takes place. Diaries also miss the body language and facial expressions of participants which form part of the rapport building and a layer of communication during in-person qualitative data collection. Some critiques of diary methods stress the risk of producing overly individualized and decontextualized accounts (Meth, 2003). As such many studies triangulate methods to embed diaries within their social context and validate data (Filep et al., 2018; Rutherford, 2002). Pairing diary methods with other methods such as interviews or discussion groups supports accurate analysis and provides a chance for further exploration of themes (Langevang, 2007; Jones et al., 2015). As all methods bring their own benefits and limitations, supplementary methods can enhance understanding.
In our diary-based study, several additional points of contact negated the limitations of diaries. Rapporteurs periodically engaged with diarists by telephone (after de Lanerolle et al., 2020), to clarify questions, as well as reminding diarists of our writing guidance (Williamson et al., 2015). Rapporteurs also held digital/online focus groups discussions with diarists over Zoom, WhatsApp, and Viber, to contextualize their entries (after Eidse & Turner, 2014, p. 244; the non-digital adjustment was in-person focus groups). Three focus groups were held per country per cluster; the group size (n=10) was appropriate to give everyone time to speak and listen (after Barford, 2021). The focus groups took place at the start, middle and end of the data collection period, and were used to ask for clarification or further detail of the processes, concepts and situations diarists were reporting. The groups were also timed to boost motivation and instill a sense of belonging in the diarists involved – countering the possibly lonely experience of writing in relative isolation.
Data Quality and Participant Retention
A successful diary study is entirely contingent upon ‘quality’ diary entries. Quality can be promoted by providing clear instructions while not restricting diarists’ agency (Rudrum et al., 2022, p. 7). Our project used ‘structured diaries’, whereby prompts are used to guide diarists to write towards topics of interest (Table 2). Such prompts support the writing process, and questions were sufficiently broad for participants to choose what to share (Rudrum et al., 2022, p. 7). Weekly submissions of diary entries encourage their completion soon after events or experiences to reduce recall bias and memory failure (Meth, 2003; Langevang, 2007; Stone et al., 2002).
Ongoing quality checks of the data were crucial. A couple of weeks into data collection, it became clear that our diary instructions were not meaningful to the diarists in our study. Diarists were repeating the same stories two weeks in a row, and diary entries varied considerably in length and narrative richness – both possible signs of insufficient guidance (Williamson et al., 2015; Meth, 2003). Meth (2003) also reports some diarists filling their diary book ‘cover to cover,’ while other women wrote just 10 pages. To encourage detailed writing, promote quality, and reduce the repetition of accounts (especially when diarists were unsure of what to write), we suggested diarists write one page per question.
An important aspect of diary research design is the duration and attrition. Our study collected weekly data over 16 weeks, in two countries, with 100 diarists. In week one, some of the diarists struggled to write and were more comfortable offering oral accounts. However, by week three, diarists’ levels of comfort had increased, with some diarists especially enjoying writing (see Data Management and Analysis section). As data collection continued, there was diarist fatigue, especially as most diary prompts did not change over time, which was important for the research but possibly sometimes boring and repetitive for the diarists. As others have found, some diarists saw their daily activities as unchanging to the extent that they felt they had nothing new to contribute to the diary (Esquivel et al., 2008). In our study, this was particularly true during government lockdowns. We reassured diarists that it was still relevant to report similar events, emotions or activities (but not to copy the text from one entry to another). In addition to the methodological adjustments detailed above.
Accepting retrospective diaries for weeks when it was impossible for diarists to write that week and reliably renumerating diarists for their time and efforts also helped to retain diarists throughout the project. Our study was able to maintain 96% participation to the end of the study. Only four diarists dropped out of the study due to personal circumstances, and several weeks of submission were omitted by a small number of diarists due to a lack of phone signal. Three of four diarists who dropped out were re-recruited immediately, as the three were concentrated in one cluster. Rather than re-recruiting, some studies anticipate attrition and over-recruit initially (Rudrum et al., 2022, p. 6–7). Without the strong team of rapporteurs who worked closely with diarists in a personal way, and without the budget to compensate diarists for their time, diarist attrition would have been much higher.
Pre-emptive Care: Ethics and Safeguarding
A common concern among diary study participants is diary security, confidentiality and personal anonymity (Thomas, 2007; Meth, 2003). Diaries offer an intimate insight into people’s lives, necessitating thoughtful safeguarding and ethical responsibilities. Informed consent was crucial to this project, as were robust data management systems (Data Management and Analysis section), and the agreement of team members to protect diary writers’ identities. With our solicited (or invited) diaries, diarists were informed how their writing will be used. It was also made very clear that diarists were free to exit the study at any time and could avoid writing about difficult or private subjects while still receiving financial compensation (Table 1). Financial compensation for participation promotes inclusivity, acknowledging the precarity of the livelihoods we were studying and how time spent recording a diary was often taking away from other income generating activities. Further, everyone else working on this project was paid - not paying the youngest and most disadvantaged people involved would have perpetuated inequitable and exploitative research relationships. In response to anonymity issues, we followed Nespor (2000) to make sure that when transcription and translation occur, identifying details should be removed. This includes addresses, real names and birth dates – of both the diarist and the people they might refer to (Williamson et al., 2015).
Due to the disadvantaged and vulnerable characteristics of diarists, compounded by a crisis situation, good safeguarding was of considerable concern. Our safeguarding protocols combined Nepali and Indonesian requirements with Restless Development’s Global Safeguarding Protocol (Restless Development, 2022). Safeguarding and ethical procedures must be coordinated and co-produced with the national processes where the research takes place, backed up by a knowledge of local interventions for severe cases, such as domestic violence or self-harm. For our diary study, safeguarding concerns could be identified at two stages. Firstly, during diary translation into English by the rapporteurs; secondly during analysis and coding. All concerns were flagged to the Nepali or Indonesian project managers, who initiated a personalized safeguarding procedure between the rapporteur and the diarist, generally mobilizing counselling services as quickly as possible for diarists with self-harm concerns.
Budgets
Perhaps counter-intuitively, it is expensive to conduct research with vulnerable groups. Not only can diary studies be financially costly if the participant numbers are large, they also require up-front training and instruction to ensure participants know how to use their diary tools properly (Williamson et al., 2015). If a project requires but cannot fund participants’ access to smart technology, this has direct impacts on access to vulnerable groups (see Qualitative Disaster Research at a Distance section). Inclusive diary methods require research teams and researchers to travel to recruit geographically dispersed groups, adapt the research design to meet individuals' needs, provide financial and emotional support, and set up organizational structures to ensure diaries are distributed and collected in a timely manner. Within the context of our own diary study and its large geographic scope, resource caps impacted our access to the most marginalised groups.
Budgeting for Inclusive Diary Research. The percentages come from the project budget, the unknown costs were covered from other sources so are not expressed as percentages.
Data Management and Analysis
Data Management
A well-organised, clear, and user-friendly data management process was useful for managing our large dataset, which included raw data, translated data, cluster analyses, country reports, and focus group data. The study relied upon data management and organization techniques for file naming and storage. Nepali and Indonesian partners and rapporteurs first received the data, and uploaded it to secure digital folders to share with the wider team. Data files were named according to submission date, cluster, and diarist code (e.g. 20210327_C1_03 means 27th March 2021, Cluster 1 Young Mothers, Diarist 3), allowing organisation within large folders. Files were linked in a ‘Master Data Navigator’ Google Sheet to quickly locate diary entries by week, cluster, or diarist. Any updates were immediately available to the core team. Regular back-ups were made on secure external hard-drives. Diaries were also organized in Atlas.ti, with the Document Groups feature allowing for analysis by country, cluster, gender, age, and rural, urban or peri-urban geography. Access to online documents was restricted to the core research team to protect participants' anonymity and safety.
Translation in Cross Cultural Research
Translation from Nepali and Bahasa Indonesia into English was a major undertaking. All raw diary entries were collected in the participants’ national language, yet translation to English was vital for international analysis and wider sharing. In this process, the translator plays the role of cultural broker and meaning shaper (Clark et al., 2017, p. 1757). As such cluster rapporteurs were well placed to identify, explain and translate country- and age-specific experiences and concepts. Translators kept a list of words which were not easily translated into English for further discussion and explanation, so as not to lose more nuanced meaning. Using an Atlas.ti code for translation errors and confusing translations allowed international researchers to follow up with translators on any confusing phrases.
In practice, transcription and translation took longer than originally estimated. Time-consuming translation (up to two days for a 2000-word diary entry) led to a backlog of transcription and translation of the 100 diaries submitted each week. In response, the rapporteurs removed the time-consuming pre-translation transcription. The specific length of translation time may vary according to diary length guidelines, as well as the number of other responsibilities (i.e. initial coding, diary writer engagement) for which translators are responsible. In a future study, we would factor in more time to translate diaries, which could be established through trialling how long diary translations take. This would help plan the workflows of the project, to avoid delays project and manage the demands upon translators who performed essential intellectual labour. The value of translators is doubly important in this project, as they were also rapporteurs, and so organised, encouraged and oversaw the diary writing clusters.
Coding and Thematic Analysis
Code Groups for Atlas.ti Analysis.
Our project also used coding to review data quality, safeguarding, follow-up questions, and to learn from participants’ experiences of the diary process itself. A ‘follow-up question’ code was used to flag diary excerpts in which a researcher might ask for more depth or clarification during the next focus group discussion. Safeguarding codes similarly flagged any issues missed during translation, to swiftly engage the Nepal and Indonesia project managers. Finally, a code was used to capture instances where diarists reflected on the process of diary writing, which offered insight into experiences of the method itself. Several quotes below share the diarists’ reflections on the enjoyment and benefit they received from participating in the research, echoing the experiences of participants in other COVID-19 diary studies (Rudrum et al., 2022; Scott, 2022): “The thing that made me happy, hopeful and healthy last week was I got work to write a diary for Restless Development, Nepal.” - Diary excerpt, Nepal, NEET Mother (3 April 2021) “It is fortunate that I can join this research as a participant because I can also practice writing. I am highly motivated to do this.” - Diary excerpt, Indonesia, Person with Disabilities (10 April 2021) “Last week, my family got help from Restless Development where I have got a space to pour my internal thoughts. I am not able to say these things to anyone although I wish to do so because since childhood, I never tell anyone about my despair instead I write them to reduce my disappointment. Now I am able to say something, get some suggestions, make myself light-souled, as well as earn some money which has made the complicated situation easy.” - Diary excerpt, Nepal, NEET Mother (27 April 2021)
The diaries we collected recorded the significant challenges young people were facing during the COVID-19 pandemic (Mueller et al., 2022; Proefke and Barford, forthcoming); the quotes above show that diaries also offer something we had not expected – the personal validation of being asked about one’s experience and the at times cathartic benefits of putting pen to paper.
Conclusion
Researching disasters and crises is notoriously difficult, due to the turmoil which surrounds such occurrences. Often a numerical assessment of immediate needs is the first response, and this is vital for the rapid supply of essentials to those most in need. The COVID-19 pandemic is a protracted crisis, with vastly uneven impacts on people, exacerbating poverty and inequality, and impacting young people especially severely (Barford & Ahmad, 2021; Barford et al., 2021b; Barford, 2020; Mueller et al., 2022). In this context, longitudinal diary research offers policy makers, practitioners, and academics deeper knowledge of, and sensitisation to, some of the more acute youth experiences, challenges and outcomes from this pandemic. This can variously be applied to inform response and recovery prioritisation and resourcing.
This paper builds on a tradition of diary research, including work by leading diary researchers such as Paula Meth and Thilde Langevang, as well as more recent usages of diary methods during the COVID-19 pandemic (Scott, 2022; Rudrum et al., 2022; Rönkkö et al., 2021; Scott et al., 2021). We detail the learnings and limitations of our recent ‘Youth Livelihood Diaries’ research project to show how diary methods can be harnessed to offer a qualitative, personal, longer-term insight into the lives and livelihoods of vulnerable young people at a time of crisis. These insights build upon previous crisis responses where qualitative evidence has improved policy and programming such as the 2008 Global Financial Crisis and 2014 Ebola outbreak (Himelein, 2014; Heltberg, et al., 2012; Walker & Hall, 2010).
Diary research offers the chance to (re)conceptualize problems based on extended engagement, complementing quantitative data with nuanced, qualitative insights for medium-to-long term recovery and renewal. We argue for research which is intentionally inclusive and designed according to the needs and capabilities of participants. Inclusive and equitable research is a work in progress, especially in the current context where many financial and cultural interactions are still marked by the legacies of colonialism.
The insights, practices and limitations shared here might prove useful to future researchers seeking to run diary research in disaster settings. In practice, some elements of our work will be transferable and others not, according to the budget, partners, setting, and disaster that is being researched. Nevertheless, the transferrable takeaways for effective disaster diary research with vulnerable groups entail budgeting for inclusivity; making adaptations to avoid excluding participants; building strong, equitable, locally-rooted partnerships with good communication; personal-level interactions with diarists to avoid attrition; fair remuneration for diarists and the team; collaborative design of research and data management; and responsive flexibility to ongoing solicited feedback.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by an Asian Development Bank grant (grant number: 51332-001) to Restless Development and the University of Cambridge. Dr. Anna Barford's Prince of Wales Fellowship in Global Sustainability, funded by a philanthropic gift from Unilever to the University of Cambridge, also supported part of her role on this project. Insightful discussions with Gerard McCarthy, Chris Morris, Nadine Kotval, Jose Enrique R. Corpus, and Katarzyna Cieslik have guided the development of the ideas presented here.
