Abstract
In this paper, we introduce a novel method for the synthesis of qualitative data and co-production in the development of evidence-based guidelines. The call for evidence-based practice in healthcare settings has been dominated by a focus on patient groups, overlooking the need for robust guidelines to inform the delivery of support or developmental interventions for staff members. We propose an eight-step method that brings together primary and secondary qualitative data with co-produced data. Data is synthesised at two of the eight stages. This method generates robust findings and recommendations which are well suited to informing written guidelines. We present our experience of implementing this method in the development of postvention guidelines for the support of National Health Service (NHS) staff following the death by suicide of a colleague. This worked example illustrates the application of the method to the generation of evidence-based, co-produced practice guidelines. We discuss the application of qualitative data and co-production in the development of fit for purpose guidelines, and the lack of transparent reporting of methods in existing guidelines. We recommend that guidelines should be underpinned by empirical evidence and developed in consultation with stakeholders and end-users, including those who will implement and those who will receive treatment or intervention. For transparency and to inform end-users, we conclude that written guidelines should always detail the methods employed in their production.
Introduction
In this paper, we describe a novel method for the synthesis of qualitative data and co-production in the development of evidence-based guidelines. This method originates from our application of findings from secondary and primary data analyses (Causer et al., 2022, 2024; Spiers et al., 2024) and stakeholder co-production for the development and publication of suicide postvention guidelines for NHS staff following a colleague death by suicide (Riley et al., 2023). Postvention is the provision of support to people who are exposed to, affected by, or bereaved by a death by suicide. In the UK, healthcare professionals have an increased risk of suicide, with female nurses being 23% more likely to die by suicide than their counterparts in other occupations (NCISH, 2020; NHS England, 2023). Elevated suicide rates in healthcare workers are also evident in other high-income countries (Milner et al., 2016). The impact of suicide may lead to complex physical and mental health outcomes, including depression, intrusive thoughts, hopelessness and poorer general health (Bellini et al., 2018; Shields et al., 2015; Spillane et al., 2017). People who are affected by suicide are at increased risk of occupational dropout or dying by suicide (Hill et al., 2020; Pitman et al., 2016, 2018) the provision of effective and timely postvention may reduce these risks (Jordan, 2017).
Qualitative research is particularly suited to the development of guidelines as it generates evidence from the exploration of ‘specific contexts and particular individuals’ (Brantlinger et al., 2005, p. 203). As Cardano (2018) states, qualitative procedures for data generation ‘harmonize’ (p. 3) with the context within which research is conducted. The practice of close observation (Cardano, 2018), which is integral to qualitative data collection and analysis, provides the detail and nuance that underpins good quality guidance. Furthermore, qualitative methods promote close engagement with practitioners – in our case, NHS staff – facilitates the growth of knowledge, and is therefore congruent with the notion of learning to improve (Kozleski, 2017). As such, the findings and recommendations that arise from qualitative studies are well suited to informing learning and development through evidence-based guidelines and practice (Kozleski, 2017).
Clinical treatments, health, and social care support interventions and processes for prevention, screening, and diagnosis are all underpinned by evidence-based guidelines (AGREE Next Steps Consortium, 2017). The quality of guidelines informs the quality of treatment or support; therefore, appropriate research methodologies and rigour when developing such guidelines are essential if the implementation of recommendations is to be effective (AGREE Next Steps Consortium, 2017).
There has been a consistent drive within healthcare settings to improve evidence-based guidelines that inform the provision of treatment and interventions for patients and clients (NICE, 2014). For instance, the National Institute for Health and Care Excellence (NICE) provide evidence-based guidelines for patient- and client-focused health and social care interventions and treatments. NICE guidelines are underpinned by transparent evidence and inform best practice in patient and client care (NICE, 2014). However, the NICE (2014) methods to produce guidance focus largely on the use of existing literature (such as literature reviews) rather than the generation of empirical data to answer the questions posed by guidelines. There has been a recent emphasis on co-production with patient and user groups to address the historical exclusion of patients in health research and guidelines (Ocloo et al., 2021). When undertaken meaningfully, co-production can lessen perceptions of power imbalance held by patients or service users. It can also improve engagement and contribute to recovery and well-being through positive experiences of engagement and feeling valued and heard (Harcourt & Crepaz-Keay, 2023).
Our review of published postvention guidelines indicates that similar expectations or standards are not applied to interventions aimed at staff wellbeing (Causer et al., 2022). Findings revealed limited examples of co-production and a lack of transparency in the methods and process for informing the evidence base (Causer et al., 2022). Furthermore, some sets of included guidelines were cross-referenced and cited each other, and several repeated similar ideas and frameworks, sometimes without referencing original sources (Causer et al., 2022). Another review of postvention guidelines found that only seven of 12 sets of guidelines described the methods underpinning their development, and whilst ten guidelines described their evidence base, this was most commonly references to literature, an expert advisory group, or a consensus procedure such as a Delphi study (Andriessen et al., 2019). The Delphi method, the sole source of empirical data employed to inform guidelines, engages a panel of ‘experts’ to assess and agree on items to be included in guidelines. In most instances, ‘experts’ are topic or practitioner informants rather than lived experience experts or the potential beneficiaries of the guidelines (Baker et al., 2006). Thus, empirical data generated by this method may not include the views and experiences of people who are part of the intended population.
In contrast, our review revealed that guidelines should be developed with the expressed needs and preferences of the target population in mind and that these needs differ according to contextual factors such as workplace cultures, availability of support, and professional identities (Causer et al., 2022). Thus, we concluded that, in the development of guidelines, robust empirical data is essential for identifying specific needs and how they might best be met. Additionally, that co-production methods should be engaged and that stakeholder groups should be diverse and include representatives from across all interested parties (Petkovic et al., 2020).
Co-production is recommended by Clements et al. (2023) who have proposed a model to inform the development of workplace postvention guidelines. They state that the model ‘would utilise lived experience of staff together with current postvention research and evidence to develop and integrate targeted interventions informed by, and appropriate to, the system in which they were applied’ (Clements et al., 2023, p. 11). The three-part model (Figure 1) illustrates the equitable importance of empirical data (1), co-production (2), and design and development (3) for the development of postvention guidelines. A public health approach to workplace postvention (Clements et al., 2023).
We propose that this model is widely useful as it could inform the development of practice guidelines for a range of support interventions across diverse settings. In this paper, we will illustrate how we built upon this model with examples of data and evidence from our NHS postvention study. We propose that our method could be adapted to inform the development of workplace guidelines in other areas of support or health provision as well as postvention.
A Method for the Synthesis of Evidence to Inform Guidelines
We developed an 8-step method which starts with the identification of appropriate data sources and finishes with a final output of a set of guidelines. We detail the steps here and illustrate their sequence in Figure 2 below. A method for the synthesis of evidence to inform postvention guidelines.
Step 1: Data Sources
To produce well-informed guidelines, we recommend utilising pluralistic data sources including existing evidence, taking account of gaps and limitations, capturing the views and experiences of multiple stakeholders. A flexible, open approach takes account of the capacity and needs of different research groups and diverse projects. As a baseline we recommend the following sources for data generation: 1. Published peer-reviewed literature concerning the topic and the specific cohorts who will be working with the guidelines and who will be in receipt of support or interventions subject to the guidelines. 2. Existing guidelines documents and/or policy documents pertinent to the topic and population. 3. Empirical qualitative and demographic categorical data generated with the population who will be delivering the guidelines and those who will benefit from them. We recommend that researchers explore participants’ experiences and needs as well as their preferences and ideas regarding the content and application of guidelines. 4. Consultation with the target population and key stakeholders, which should include participants from the empirical study as well as topic experts, policy makers, senior leaders and other interested parties such as union representatives, third sector experts, or patient lobby groups, for instance.
Step 2: Methods of Data Analysis
We suggest that methods of analysis be informed by the questions that you seek to address with your guidelines; by the perceived needs of the target population; and by the knowledge and expertise within your team. Some guiding questions to ask are: • Is the method suited to the kind of questions you wish to answer? • Is the method suited to the kind of data you wish to analyse? • Is the method suited to generating the kind of findings that can be translated into clear recommendations and guidelines?
Step 3: Development of Findings and Recommendations
Throughout analysis, be mindful that the purpose of findings is to inform guidelines. Therefore, ensure that your research questions reflect this and that you keep those questions central to the development of findings. Developing recommendations from findings may be an ‘add-on’ in many studies. However, when the purpose is to inform guidelines, particular care needs to be taken to ensure that recommendations are clearly anchored in your findings and align with the evidence that you have generated.
Step 4: Synthesis
Once you have a set of findings and recommendations from your literature and guidelines/policy reviews, and from your empirical study, these should be bought together and synthesised.
Pope et al. (2007) have described three definitions of synthesis: aggregative (generalization of evidence); replicative (comparison between sources); and developmental (generating theory or explanations). For generating guidelines, we suggest that a developmental model is best applied, as this is particularly useful in the development of policy ideas, theory or new interventions (Pope et al., 2007).
We propose a three-step process of data synthesis: analysis – synthesis – interpretation (adapted from Major & Savin-Baden, 2010, pp. 62–69). 1. Analysis • This is the first stage in understanding the key messages that sit within the data sources. • Extract theme headings and recommendations from data sources. • Tabulate to enable comparison and organisation which will allow identification of common themes across the data sources. 2. Synthesis • Defined by Major and Savin-Baden (2010) as taking parts and developing them into a new whole. • Combine similar themes or messages that occur across the data sources to generate composite themes. This stage may involve working iteratively – repeatedly returning to the original source to check the meaning and details of theme headings. • Generate a final group of themes and key messages and track where and how they have been informed by each of the data sources. • Translate themes into the key headings under which your recommendations will sit. 3. Interpretation • Moving away from broad ‘themes’ and common meanings, interpretation involves exploring organising principles that help to explain variation and differences across contexts and individuals. • Taking account of these variations and differences, you will now develop the recommendations that sit within the themes developed by the synthesis.
Step 5: Writing Evidence-Based Recommendations
This involves translating the results of the synthesis into a set of recommendations. Topic headings were generated at stage 2 of the synthesis, while the outcomes of stage 3 will inform the recommendations that sit within each section. We suggest that recommendations are written in language that is appropriate for your target population.
Step 6: Consultation: Stakeholder Workshop
Consultation is essential at this stage to ensure that your recommendations speak to your population; that they are clear and meet expectations. Consultation may take the form of an in-person or online workshop or a world café event, for instance. The aim is to invite discussion, critique, and a sharing of ideas and responses. All ideas generated at the workshop should be seen as data. Therefore, an online event could be recorded; paper-based exercises should be retained and collated. Tools such as Padlet can be utilised to capture ideas and suggestions (these could be either identifiable or anonymous to facilitate inclusivity); online chats and Q&A sessions can be saved and downloaded.
Step 7: Synthesis
The second stage of synthesis in this model involves the organisation of data from the consultation event and the integration of new findings into the recommendations developed at Step 5. This is likely to be a more intuitive, less structured process, but may follow these steps: 1. Review and organise the data collected from your consultation event. 2. Comparison of workshop data with the recommendations developed at Step 5. 3. Integration of new knowledge and ideas into the recommendations.
In the instance of contradictory data, we would recommend returning to the original data from all sources, to compare and consider how a contradiction can best be managed. When creating guidelines, we propose that it may be possible to acknowledge differences and present alternative recommendations to meet diverse needs.
Step 8: Generation of Guidelines
Taking the recommendations that were the output of Step 7 as a starting point, you can now populate them with clear step-by-step instructions appropriate for users and your target population. If all steps were followed when synthesising data (Steps 5 & 7), you will now have pertinent information detailed in tables or documents that were generated during the synthesis process. A well-planned and managed stakeholder event will have generated ideas around how the guidelines need to be written and presented.
Developing Evidence-Based Postvention Guidelines for NHS Staff Affected by a Colleague Death by Suicide
We will use the example of our synthesis of findings from primary and secondary data and a stakeholder workshop in the development of postvention guidelines to illustrate the application of the model.
Step 1: Data Sources
Our choice of data sources was informed by the research aims and objectives; the purpose of the guidelines that we set out to develop; and the populations of workers who would be implementing the guidelines and in receipt of support informed by the guidelines.
Research aim:
To understand the impact of a colleague’s suicide on NHS staff, and their support needs, to inform postvention guidelines.
Objectives: 1. To undertake a review of suicide impact and postvention interventions in other settings. 2. To explore the impact of colleague suicide on staff wellbeing and their grief reactions to such an event, including how staff adjust, negotiate, process, and ultimately cope with the death of their colleague. 3. To explore staff views about risk factors, relevant contextual factors, and warning signs, which may have contributed to the likelihood of suicide in their NHS colleagues. 4. To identify what helps and hinders bereaved colleagues to seek support, to characterise supportive work cultures, and to identify staff preferences for future emotional/practical support and service provision. 5. To explore how managers respond to and support their employees and colleagues following death by suicide, and to identify current postvention activity. 6. To use the findings to develop evidence-based postvention guidelines for NHS organisations and managers on how to support and respond appropriately and effectively to bereaved or affected employees.
As such, we sought to review peer-reviewed and grey literature that reported the experiences of employees following a colleague death by suicide in any profession or workplace setting as well as published sets of postvention guidelines that deliver guidelines for supporting staff following the suicide of a colleague.
Empirical data were collected via interviews with NHS staff who self-identified as being affected by the suicide of a colleague, or who had provided support to co-workers following the suicide of a colleague. Further, we facilitated an online stakeholder workshop which was attended by study participants, other NHS staff, NHS senior leaders and policy makers, interested academics, postvention leads from public and third sector organisations, and union representatives.
Step 2: Methods of Data Analysis
Integrative Review
We undertook a systematic integrative review to review the literature and guidelines. As such we undertook systematic searches using defined inclusion criteria and quality assessed all included articles. We chose this method as it is well suited to bringing together peer-reviewed and grey literature and different types of articles and reporting (Whittemore & Knafl, 2005). Other methods of review may also be useful depending on the specifics of your study, for example, a meta-synthesis (Mohammed et al., 2016) is well suited to qualitative literature, or a rapid review if a straightforward overview of findings is the desired outcome (Khangura et al., 2012). The methods and findings of this study have been published (Causer et al., 2022).
Grounded Theory
For the collection and analysis of empirical data, we used grounded theory methods. Grounded theory is useful for exploring previously unresearched topics and for the development of theory and outputs that translate well into real-world outputs such as guidelines (Glaser & Strauss, 1967; Holton & Walsh, 2017). As we had two distinct cohort of participants (affected staff and supporting staff members) we undertook two analyses and developed two theories. This gave us confidence that we had clarity about the experiences and needs of each cohort and so could develop the guidelines with the needs of both groups in mind. The methods and findings of these studies are reported in Spiers et al. (2024) and Causer et al. (2024).
Thematic Analysis
In addition to the grounded theory analyses, we applied a codebook thematic analysis (Clark & Braun, 2018) to data gathered from participant interviews in response to our specific enquiry ‘what would you like to see included in postvention guidelines?’
Step 3: Development of Findings and Recommendations
In each analytic process, we remained mindful that our findings were intended to inform the development of guidelines. This shaped the kinds of questions we posed during the analyses and how we presented findings and recommendations.
Step 4: Synthesis – Stage 1
We followed the three-stage method detailed earlier.
Analysis
We started by extracting data from our research outputs in the form of theme headings and recommendations. Figure 3 shows how we tabulated this data to find commonalities and differences across the data. We had four sources of data: the integrative review; the two grounded theories developed from our participant interviews; the thematic analysis of participant data pertaining to their ideas about what should be included in guidelines. As we created this table, the column on the left-hand side (titled ‘findings’) was blank. As we populated the table with data from each source, we engaged in a continual process of matching and re-matching data across the sources. Only when we felt that this matching process was complete did we attempt to populate the left-hand column with descriptors. We produced one table for ‘findings,’ and another for the recommendations that we had developed within these sources. Example of analysis stage of synthesis.
Synthesis
The next stage was the synthesis process. To do this, we created a further table (Figure 4). We took the headings that we had developed in the left-hand column of Figure 3 and combined those that were similar to generate composite themes, or messages, as can be seen in the ‘Message’ column of Figure 4. We then sourced the supporting data for each message from our data sources and added any notes that would help us in continuing the analytic process or writing up our guidelines. Example of synthesizing key messages.
Interpretation
At this stage, we moved away from thinking in terms of themes, and sought to identify the nuances in our data; in particular, the differences and variation across contexts and individuals. These variations and differences inform an attentive development of a set of synthesized recommendations that will sit within the themes developed at the synthesis stage.
Step 5: Writing Evidence-Based Recommendations
Following this synthesis process, we wrote a set of evidence-based recommendations, taking care to use language that was clear and relatable for our population. This gave us the output that we would use at the next stage; consultation.
Step 6: Consultation: Stakeholder Workshop
Our means of consultation was via an online stakeholder workshop. An online platform was best suited to provide accessibility to a diverse range of stakeholders, who were located across England and Wales. Our aims were as follows: • To share our key findings, our lived experience film, and postvention recommendations with stakeholders. • To invite stakeholder feedback in the form of initial responses and critique as well as new ideas for recommendations. • To provide time for questions and answers. • To provide a space for stakeholders to reflect on and discuss, in groups, their thoughts and responses to our recommendations and consider any key challenges to implementing the guidelines. • To provide an anonymous space for stakeholders to share any thoughts that they didn’t feel able to share in front of others. • To invite responses and feedback from key expert commentators such as NHS senior leaders and union representatives.
The workshop was 3 hr long, which gave ample time for stakeholder involvement and input. We disseminated invitations via key contacts, networks, and social media platforms. We also invited the study participants and study stakeholder group. The time and date of the workshop were set 4 months in advance, to give potential attendees good notice of the event. We used Zoom as a platform for this meeting, as it allowed us to facilitate breakout rooms for small group discussions. We also used Padlet, which provided an anonymous space where stakeholders could write comments, thoughts, and ideas. Sixty-eight stakeholders attended, including study participants; front line health workers; health managers and leaders; policymakers; union representatives; third-sector reps; members of professional bodies; and interested academics. We collected data (with stakeholders’ consent) by audio recording the workshop; by saving and downloading any comments written in the meeting chat or Q&A forums; by downloading comments posted on Padlet (Figure 5). Example of thematic organisation of consultation data.
Step 7: Synthesis
In line with the three steps outlined earlier, the data collected from the workshop was organised thematically. Next, we compared the key themes and evidence underpinning the themes with the recommendations we had developed at stage four. We noted new ideas, concepts, and themes, as well as similarities and affirming ideas and themes. Finally, we integrated any new knowledge and ideas into the recommendations. In this case, we did not have to remove any of our existing recommendations.
Step 8: Generation and Dissemination of Guidelines
To write the final guidelines, our themes became section headings; the key points became sub-sections; the illustrative data from the synthesis tables became the framework for the text. As we wrote we returned to the original data at times to confirm details or nuances. Our data also included ideas and requests regarding presentation of the guidelines and items for inclusion. For instance, many workshop attendees requested a checklist that could be picked up quickly and referred to in a moment. In response, we developed two checklists: one for preparation, the second for crisis response (Figure 6). The guidelines are in the form of a downloadable PDF (Riley et al., 2023) and the checklists are interactive, so they can either be used on a screen or on a paper printout. Response and review checklist from our postvention guidelines (Riley et al., 2023).
We launched the dissemination of the guidelines with an online event. We invited all stakeholders who had registered an interest in the workshop and all study participants. We also advertised the event on our social media pages and through social media groups. In launching the final output in this way, stakeholders and study participants could see how their contributions had been incorporated. Additionally, using more participatory methods leads to greater implementation and uptake (Potthoff et al., 2023).
Discussion
To generate a set of postvention guidelines for the support of NHS staff following a colleague death by suicide, we developed an eight-step method for synthesising qualitative, empirical, and co-produced evidence from a range of sources. In this paper, we have presented the method together with a worked example of how we applied it; in this case, to the development of postvention guidelines for the support of NHS staff who are affected by a colleague suicide.
We found that transparency of process and methods were missing in most postvention guidelines that we reviewed (Causer et al., 2022). This is problematic as the lack of transparency undermines the veracity of the guidelines due to the absence of underpinning evidence. As such, the reader is unable to understand where the key ideas presented in the guidelines have come from. Recommendations for writing good practice guidelines (NICE, 2014) state that underpinning evidence and methods used should be detailed. When writing our postvention guidelines (Riley et al., 2023), we included our methods in detail so that end-users have clarity around the processes involved and how the ideas and suggestions contained within the guidelines were developed.
By using qualitative methods and co-production, we were able to employ close observation of contexts, narratives, and cultural norms (Cardano, 2018) whilst also engaging closely with the nuances of participants’ and stakeholders’ experiences, perceptions, expertise, and suggestions. We can feel confident that, by underpinning our guidelines with qualitatively generated empirical evidence, the output responds specifically to the experiences and needs of the target population. Arguments for co-production in health research often focus on addressing the power imbalance between patients and health workers, or on improving engagement of patients with their treatment (Harcourt & Crepaz-Keay, 2023). We argue that these factors apply equally to the engagement of health professionals in the production of guidelines for their support. For example, our review study and our grounded theory findings highlighted that staff members perceived that they were being silenced by managers who failed to acknowledge or talk about a colleague suicide (Causer et al., 2022; Causer et al. 2024).
We drew on the three-step model developed by Clements et al. (2023), a public health approach to workplace postvention, as a framework for developing our eight-step process. At step one of Clements et al. (2023) model (review and evaluate the workplace, organisational context and staff exposure and needs), we generated data and evidence via a review study and interviews with affected NHS staff, thus taking a comprehensive overview of contexts, experiences, and needs. We then synthesised the findings and recommendations generated by both studies, the result of which was an evidence-based set of recommendations for postvention guidance. At step 2 (determine the best-fit application of postvention research and practice to this setting), we took our findings and recommendations to our stakeholders and participants for their input, inviting their further ideas and suggestions to co-produce the guidelines. At step three (design and implement targeted workplace postvention and tools and processes), we applied our method of synthesis to incorporate suggestions from our stakeholders alongside the empirical knowledge we had developed from the review and interview studies to develop a set of guidelines and tools that are evidence-based and co-produced.
The output of our research has been the first set of evidence-based, co-produced guidelines for the support of health workers internationally, following a colleague death by suicide. We propose that our eight-step process could be applied to the development of guidelines across a range of topics and practice areas in addition to postvention. The method offers flexibility of data sources and analytic methods so that researchers can use their informed knowledge of context and questions to choose the most appropriate for their area of intervention and target population. Use of this method provides complete transparency of process and methods in the production of guidelines.
We recommend that guidelines should be underpinned by empirical evidence and in consultation with stakeholders and end-users, including those who will implement and those who will receive treatment or intervention. For transparency and to inform end-users, written guidelines should always include details on the methods employed in their production.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute of Health Research (HS&DR grant number 129341).
