Promoting Recall and Preserving the Historical Authenticity of Data Used to Investigate Food Regulatory Policy in Australia

Abstract

This article reports on research methodology used for a case study investigating public health nutrition engagement in food regulatory policy processes in Australia. It aims to explore methods used and outline a model for how researchers can promote participant recall and accuracy. A set of key documents identified as relevant to voluntary food fortification policy (VFP) were analysed using Bacchi’s ‘What’s the problem represented to be?’ approach. The results informed a series of present-day, semi-structured, in-depth telephone interviews with experts in VFP. The major epistemological difficulty was the necessary reliance upon participant memory. Consequently, oral history methods were used to aid key informant recall. Several participants noted a benefit for their recollection of events from the tools employed. The experiences and approaches outlined in this article contribute to the toolkit needed when investigating historical events and may provide a useful model for other nutrition-related policy research.

How This Article Relates to ‘what is already known’

This article reports on the research methodology used for a case study investigating public health nutrition engagement in food regulatory policy processes in Australia. The work deals with the epistemological difficultly of a necessary reliance upon participant memory when collecting contemporary accounts of historical events.

What This Manuscript Adds to Current Understanding

The methodological knowledge gained from the approaches taken to aid recall and preserve the historical authenticity of data collected is described. This is significant because historical events can aid understanding of an issue and provide insight for managing similar situations in current settings. The experiences and approaches outlined contribute to the toolkit needed when investigating historical events and may provide a useful model for other nutrition-related policy research.

Introduction

Studying historical events can aid understanding and awareness of a particular issue and provide insights and strategies for managing similar situations in current settings and circumstances (Baugh et al., 2014). Consequently, researchers investigating topical issues may need to collect contemporary accounts of historical events. In gathering such data though, there is a necessary reliance upon research participants’ memory about past events. Thus, reticence, forgetfulness or dishonesty on the part of participants has raised concerns regarding the accuracy of data collected as persistently problematic for researchers (Cutler, 1970).

Evidence of this is found in the range of investigative methods that have been established to aid recall accuracy when examining historical events (Beimer et al., 1991; Corbally, 2011; Corbally & O’Neill, 2014; de Vaus, 2006; Dex, 1995; Fielding, 2006; Gardner, 2006; Mathiowetz & Duncan, 1998; Robertson, 2000; Sacks, 2009; Sobell et al., 1996; Sobell & Sobell, 1992; Wengraf, 2004). Such methods offer a number of approaches and tested strategies that may be of use in maximising participant recall (Colosi & Dunifon, 2006; Corbally, 2011; Dex, 1995; Ritchie, 2015; Robertson, 2000; Sobell & Sobell, 1992). While no single technique will be sufficient to minimise participant recall inaccuracies, using a combination of strategies may be beneficial.

This article reports on a case study used to investigate public health nutrition engagement in food regulatory policy processes in Australia. The focus on the research methodology provides an account of the issues faced when endeavouring to enhance and improve authenticity and fidelity of data across time and space. As such, it aims to explore the relevant methods and strategies utilised and outline a model for how researchers seeking to collect data from nutrition-related historical cases may promote participant recall and accuracy, and minimise imprecise recollection.

Background

The overarching research, which is reported elsewhere (Ashton et al., 2021), aimed to understand how a policy ‘problem’ was represented in a food regulatory setting, and the implications for public health nutrition participation in, and engagement with, policy development. Here, the term ‘problem’ represents the understanding that policy issues are formed out of framing social and political phenomena in particular ways. Thus, the investigation used a social constructionist epistemology, employing Bacchi’s (1999, 2009) ‘what’s the problem represented to be?’ (WPR) approach. Essentially, a discourse analysis method was applied to a case study of voluntary food fortification policy (VFP) developed by the Australia and New Zealand Food Regulation Ministerial Council (ANZFRMC) between July 1, 2002 and June 30, 2012.

The fortification of foods and drinks with vitamins and minerals has traditionally been used in either a mandatory or voluntary capacity to address specific dietary inadequacies linked with deficiency conditions and diseases such as goitre, rickets, beri beri and pellagra (Lawrence, 2013). The modern use of voluntary food fortification (VF), however, has been more closely associated with the commercial appeal of such products (Nestle, 2013).

The VFP case study was chosen for this research because it provided considerable rich information for in-depth study. The relevant policy was also developed over a number of years, was quite controversial, had been the subject of limited research and remains one of several food regulation policies that have particular relevance to public health nutrition and the prevention of diet-related chronic disease. There were also international parallels with the global food regulatory body, Codex Alimentarius Commission, concurrently reviewing its long-standing policy on VF during the period of interest to this research.

While the use of a single, holistic case study was risky, it was justified by the fact that the case was both longitudinal and critical (Yin, 2014). The case was longitudinal in that it was developed and observed over a long period of a time and critical in that it used the well-tested WPR approach to policy analysis in a food regulation setting. Additionally, limited attention had been given to voluntary food fortification (VF) by the government, public health nutritionists or academics, and it gave much thick data for examining the food regulation policy development process in Australia.

VFP also met all three conditions outlined by Yin (2014) as important for determining when a case study is the most appropriate research method. Firstly, this research aimed to understand how the ‘problem’ was represented in food regulation policy decisions; secondly, the investigators had no control over the policy process; and finally, the development of the policy under examination was an ongoing and modern, rather than purely a historical, event.

Ethical approval for the study was sought from and given by the Flinders University Social and Behavioural Research Ethics Committee. The project approval number was 5585. All participants in this research provided written, informed consent before taking part.

Methods

Data Collection

Two sets of data were identified as relevant to the case study. Firstly, a set of key documents used in the development of VFP were collected. These documents were read iteratively and analysed using Bacchi’s (1999, 2009) WPR method of policy analysis. Bacchi (2009, 2016) proposes that the policy ‘problem’ is a social construction and an integral part of the policy process, not something that previously existed and was waiting to be solved. Therefore, to analyse policy, a problem-questioning rather than problem-solving approach needs to be adopted. Thus, the WPR approach was used to assist in determining how the policy ‘problem’ was represented in the documents: what values, beliefs and assumptions underlay that representation and the contextual factors that supported them. The analysis then considered how the problem could have been thought about differently, and the consequences of the way in which the problem was represented.

The results from the key document data analysis were used to inform a series of present-day semi-structured, in-depth, telephone interviews with key informants. Telephone interviews were chosen for reasons of cost, time and ease of access to participants. This was because key informants were located in a number of States and Territories within Australia, as well as in New Zealand, and the cost and time of travelling to participants in all these locations would have been considerable.

While telephone interviews are not commonly used in qualitative research, largely due to concerns about data quality; Sturges and Hanrahan (2004) found that face-to-face and telephone interviews each afforded the same depth of response and amount and quality of data. Novick (2008) conducted a systematic review of the literature regarding the use of telephone interviews in qualitative research and found an apparent bias against telephone interviews. Further, there was little evidence to back up the preference for face-to-face interviews as the ‘gold standard’ for qualitative research.

Data collected during the interviews included the participant’s involvement in the development of VFP, any contextual factors believed to have impacted on the policy development, experiences of how stakeholder views were considered and incorporated into policy decision-making, the extent to which VFP was considered supportive of public health nutrition principles, and potential opportunities for advancing public health nutrition priorities in future food regulation policy development.

Key informants were defined as persons with in-depth knowledge and expertise in the development and implementation of VFP. Despite there being a time delay between when the key documents were written and used to develop the VFP (i.e. between 2002 to 2012) and when the key informant interviews were conducted for this research (2016–2017), it was important, if at all possible, to include key informants who were involved in, and, for some, possibly excluded from, the VFP process. However, because of this time lapse, recall of the policy development and decision-making process was likely to be problematic for participants. In order to address this difficulty, a number of steps were taken to enhance key informant memory.

Aiding Recall

As a first step towards minimising participant recall errors, a range of research methods designed to investigate historical events were sought and investigated for their applicability to the research. These included the Timeline Follow-Back method developed by Sobell and Sobell (1992); the Lifetime Drinking History and Cognitive Lifetime Drinking History interview-based procedure (Koenig et al., 2009; Skinner & Sheu, 1982); retrospective recall (de Vaus, 2006); post-then pre-evaluation (Colosi & Dunifon, 2006); the biographical narrative interpretive method (Corbally & O’Neill, 2014; Fielding, 2006; Wengraf, 2001, 2004); and oral history (Gardner, 2006; Robertson, 2000; Sacks, 2009). Consequently, the strengths and weaknesses of these six qualitative methods were evaluated with respect to their suitability for and compatibility with the aims and objectives of the overarching research. After consideration by the research team, oral history was deemed the most suitable method on which to draw.

As a research method, oral history aims to understand the past and the relationship of the past to the present, through the use of in-depth interviews with persons who have lived through and experienced events of interest to the researcher (Gardner, 2006). According to Sacks (2009), the character of oral history projects can be documentary, interpretive or civic. Documentary projects present and preserve information about a particular topic, while interpretive projects emphasise the meaning and significance of information collected and civic projects progress a specific community goal. For Robertson (2000), oral history consists of a number of important methods such as a recorded interview in question-and-answer format, conducted by an interviewer who has some knowledge of the subject to be discussed, with a knowledgeable interviewee speaking from personal participation on subjects of historical interest and which is made accessible to other researchers.

While examples of use of oral history methods in public health nutrition research are rare, such methods have been used in a wide range of other public health fields of research and practice (Tsui & Starecheski, 2018). Examples include assessing population education campaigns, understanding the lived experience of persons with ill-health, advocacy to improve clinical care, reflecting on efforts to address disease (Tsui & Starecheski, 2018), examining the birth of evidence-based medicine (Smith & Rennie, 2014), determining appropriate use of health promotion in indigenous communities (Fletcher & Mullett, 2016), examining changes in urban food systems (Neff et al., 2017) and documenting changes in delivery of care in public health nursing (McCray Beier, 2004).

In conducting a review of oral history use in public health, Tsui and Starecheski (2018) found a range of benefits. These included providing research that was more centred on the experiences and voices of populations affected by ill-health, developing public health programmes more engaging for relevant populations, and supplying tools for documenting and learning from past practices. While drawbacks were also noted, such as the time required to conduct oral histories, and the difficulty for some participants to discuss painful experiences, Tsui and Starecheski (2018) concluded that oral history methods have a significant contribution to make in supplementing other public health research and practice, educating practitioners and providing greater insight into public health issues.

The major epistemological difficulty faced by oral history is its necessary reliance upon memory (Gardner, 2006). However, while the validity of oral history is questioned because of possibly incorrect or fabricated memories – meaning the evidence collected may contain errors and bias – Robertson (2000) claims it is important to consider that all historical records may contain error and bias and should be used with caution. Additionally, while there is no obligation to accept the record as given, several authors (Dex, 1995; Ritchie, 2015; Robertson, 2000) assert that the majority of what is forgotten is lost soon after an event or experience, but people most accurately remember what has been particularly important or interesting to them and this is remembered for a long time. Further, information provided can be verified using other means, and oral history does not claim to be the final word in a historical record (Robertson, 2000). Consequently, despite the epistemological difficulties, oral history, particularly the interpretive oral history method, was considered a suitable method to draw on for the key informant interviews in this research.

When conducting oral history interviews, some authors note that questions are not the only means of gleaning information and aiding participant recall, suggesting the use of photographs, newspaper articles, historical timelines or other relevant items (Mould, 2009; Robertson, 2000). While such articles may create biased recollections among interviewees, by incorporating a broad range of participants and using other sources of data, this can be minimised. Further, in this case study, as in other uses of the oral history, it was the ‘how’ and ‘why’ of the policy development process that was most important, not just the factual accuracy of events (Down, 1990; Field, 2007).

The present study takes account of the literature in this area, discussed above, and applies it to the aims of the study reported here. Specifically, utilising oral history methods for this food regulatory case study, several suggested strategies were developed and instigated to aid key informants, as shown in Figure 1.

Figure 1.

Steps for aiding participant recall in historical nutrition-related case studies.

Consequently, a few days prior to their interview, key informants were emailed copies of the relevant Australia and New Zealand Food Regulatory Ministerial Council Voluntary Fortification Policies (Australia and New Zealand Food Regulation Ministerial Council, 2004, 2009), with the changes that had occurred during the research period highlighted. They were also sent a brief PowerPoint presentation (Microsoft Corporation, 2015) that provided an overview of the research, a factual timeline of key events in the development of VFP between 2002 and 2012 that was gleaned from the key document data and the first question of the interview enquiring about the key informant’s participation in the VFP development process. This gave participants the opportunity to peruse the information provided, review their records of the policy process and if desired, contact the interviewer for further information prior to the interview taking place.

In addition to these preparatory strategies, time at the beginning of the interview was dedicated to reviewing the PowerPoint presentation and VFP timeline with participants and giving them another opportunity to ask questions or revisit their records regarding the policy development process. At this point in the interview, participants were asked whether the information and timeline provided were helpful for reminding them of the time period of interest, and if not, whether anything else could be provided that would assist their recollection of events. At the end of the interview, key informants were supplied with the contact details of the interviewer in case they wished to add to their interview responses at a later date. Finally, participants were also given the opportunity to review and amend their interview transcript if desired.

Results

Analysis of 57 key documents collected that met the criteria for inclusion of content that addressed and contributed to the debate on VF led to a total of 17 key informants being invited to participate in a semi-structured, in-depth telephone interview. Three declined the invitation from the outset and one accepted but later declined, leaving a total of 13 key informants agreeing to participate. Interviews varied in length between 25 minutes and 1 hour and 34 minutes, depending on the participant’s availability and the amount and type of information elicited. The first four interviews were transcribed by one of the researchers (BA) and the remainder by a professional transcription service.

Key informants were categorised according to the four key stakeholder groups previously identified in the key documents; that is, citizens, government, industry and public health. It was noted that government stakeholders were most likely to decline the invitation to participate in an interview, citing concerns regarding the potential discussion of ‘government in confidence’ matters. Also, those from this group that did participate were concerned about their choice of words; indeed, two government interviewees were the only key informants who elected to read and approve their interview transcript before its inclusion in the data analysis. One industry key informant declined to participate because of perceived negative experiences in previous public health debates and consequent concerns regarding how their views would be interpreted by researchers. These hesitations regarding participation by some key informants may be an indication of the politicised and contested nature of the field of food regulation.

However, the major issue of interest for this article was the potential for errors in recall by key informants participating in the in-depth interviews. This was of concern because of the historical nature of the case being investigated. Despite this, one key informant claimed, ‘I well remember the decision to review the existing fortification policy’ (Rebecca). Among other key informants, four specifically noted that one or more of the oral history methods identified in Figure 1 were factors that significantly aided their recollection of relevant events. As one participant noted, ‘when I saw that (first interview) question, I thought, oh I can’t remember, but your timeline or the key event thing was very helpful because I do recall…’ (Jessica). She then went on to relate some specific details of events she remembered and had participated in that were relevant to the case study. Two other participants indicated that while their memory of the specific details of the case was unclear, recollection of broad issues was enhanced by the process of participating in the interview. A further four key informants indicated they were able to remember one or two points, but not the entirety of the case. However, the remaining two participants only commented on their difficulty in remembering relevant aspects of the case. There was no pattern observed between the length of the interviews and the opinions expressed about the recall-promoting techniques.

The reappearance of VFP in November 2015 with a reminder from ministers of the original intent of the VFP was another factor that served to refresh the issue in the minds of participants. This was the consequence of an application to Food Standards Australia New Zealand to allow the voluntary fortification of breakfast cereals with vitamin D, which the ministers considered to be inconsistent with the intent of the original policy (Australia and New Zealand Ministerial Forum on Food Regulation, 2015). Of note for the purpose of this article though was the comment by some key informants that ‘a lot of us have felt quite strongly about vitamin D and it revived the whole memory’ (Rebecca) of the original VFP process.

Discussion

The aim of this article was to explore the relevant methods and strategies utilised for a food regulatory case study and outline a model for how researchers seeking to collect data from nutrition-related historical cases may promote participant recall and accuracy. Particular attention was given to preserving the authenticity of data gathered during present-day interviews used to investigate a historical case study of VFP in Australia between 2002 and 2012.

The use of both key documents and key informant interviews enabled data triangulation and made the research design strong, but the major epistemological difficulty faced was the potential for errors and bias in the data created by problems with interview participant recall of historical events (Gardner, 2006). Even though a number of key informants in this study identified recall as an issue, the oral history methods developed and used (Figure 1) to minimise such problems were acknowledged by some participants as aiding their memory of the VFP development process. Whilst the results were not definitive, there was some suggestion of the usefulness of the techniques employed, but further testing would be required to confirm this.

According to Robertson (2000) and Ritchie (2015), people most accurately remember what has been particularly important or interesting to them about an event. Dex (1995) concurs that salient or noteworthy events are recalled more reliably. In addition, even though much of an event or experience may be forgotten soon after it occurs, what is remembered after that stage is remembered for a long time (Dex, 1995; Ritchie, 2015; Robertson, 2000). While it was not possible to completely eradicate potential errors in the data caused by recall problems for key informants in this study, it is likely that each participant remembered what was important to them and remembered that piece or those pieces of information for several years. Thus, they were able to contribute one or more significant components of the data that, along with the use of data triangulation, helped to create a complete case.

Developments in VFP that occurred in 2015, while outside the time frame selected for this research, may have also aided key informant recall. However, while the possibility of extending the time frame for this study was considered by the research team, it was determined that this development was best dealt with in the interpretation of results, rather than increasing the volume of data already collected for the case study.

Other difficulties were incurred in accessing the most information-rich key informants for the in-depth interviews. The historical nature of the case meant several persons that would have been preferred key informants were no longer working in the area and unable to be traced, or no longer living, in order to be able to extend an invitation to participate. Among those that were still working in the area and invited to participate, three declined because of political sensitivities and one because of personal concerns regarding the impartiality with which the data would be analysed. Thus, while still information-rich, in some cases, key informants that accepted the invitation to be interviewed may not have been as extensively or intimately involved in the development of VFP as those who were unable to be invited or unavailable to participate.

The strengths and limitations of the methodology used for this study correspond to those that may occur in all qualitative research. Along with the issues of recall already considered, the rigour or trustworthiness of the research and limitations of sampling needed to be taken into account (Liamputtong, 2013; Maxwell, 2013; Patton, 1999). Rigour was addressed using data triangulation, but also temporal and researcher triangulation (Johnson, 1997; Patton, 1999). Accordingly, the use of a research team with different areas of expertise and the inclusion of a range of key informants from different locations and stakeholder groups aided the triangulation. The use of a single case study as well as sampling limitations in situation, time and selection may have restricted the broader applicability of results (Patton, 1999; Yin, 2014). However, the practical need to place boundaries around the amount, type and time frame of data collected was carefully managed in order to maximise the research relevance, and the advantage of the single case was the opportunity it afforded for information-rich, in-depth study (Patton, 2002).

Conclusion

The potential for recall errors when conducting a contemporary investigation of historical events is something that will always need to be managed by qualitative researchers. Accordingly, using and reporting on approaches that can reduce such error remains important. The need to minimise recall bias was especially necessary for the single, longitudinal case study used in this investigation. As such, using a range of methods within the oral history tradition was found to be beneficial. The experiences and approaches outlined in this article, therefore, contribute to the toolkit needed when investigating historical events and may provide a novel model for using oral history in nutrition-related policy research.