Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project

Abstract

Realist approaches to complex health care evaluations are increasingly used and recommended in national evaluation guidelines. However, there remains a paucity of researcher guidance on methods for elaborating and refining programme theories throughout the stages of a realist evaluation project—from prospective theory development to feasibility work, to full evaluation. We present a step by step worked example of a realist approach to elaborating a programme theory for a health care intervention during the feasibility phase of the Dementia PersonAlised care Team. We explain how multiple qualitative methods can be applied to elaborate initial theory, supporting a shift away from a hypothetical explanation, towards a theory of how the model works in practice. We reflect on what worked well, and problems encountered, attending to both processes and the impact of working. Details are provided on how this approach can help enhance the likelihood of the intervention working in practice—through the application of new insights to interventionists’ training resources. We argue that coding to a framework constructed of ’If-Then’ initial programme theory statements enabled researchers to develop a realist analytic mindset and elaborated programme theory, ready for a fuller evaluation of the D-PACT intervention.

Background

Dementia-Personalised Care Team (D-PACT) is a research programme aiming to develop and evaluate a Dementia Support Worker (DSW) intervention using a realist approach (Figure 1). In the prospective study (Phase 1a), we developed, and elaborated an initial programme theory (IPT), outlining how the intervention under development was expected to work. In the early stages of a realist evaluation, a programme theory is understood to be in its early form; an IPT, ready to be tested out and elaborated upon through subsequent study stages.

Figure 1.

D-PACT Background and Phases (Adapted From Griffiths et al., 2022)

In this paper we aim to set out a worked example of our qualitative realist approach to elaborating our IPT throughout D-PACT’s subsequent feasibility study (phase 1b, Figure 1), representing a shift away from a hypothetical explanation, towards a theory of how the model works in practice. Previous researchers have referred to ‘elaboration’ during programme theory refinement (Brand et al., 2019; Griffiths et al., 2022), which we defined as: the act of adding further detail or completely new insights to an IPT, which is still due to undergo further refinement though evaluation. Figure 2 illustrates programme theory elaboration throughout the phases of an evaluation.

Figure 2.

Programme Theory Elaboration Throughout Evaluation Phase

By sharing a detailed example of our analytic methods, we aim to contribute to the growing body of literature (e.g., Dalkin et al., 2021; Manzano, 2016) outlining practical steps for realist theory development, informing researchers embarking on similar work. We first summarise how realist theory development can occur within an evaluation, the philosophical principles underpinning realist theory development, and how D-PACT programme theory had been developed prior to the feasibility phase.

Theory Development in Realist Evaluation

Realist evaluation is a theory-driven approach to developing and evaluating complex interventions. The method recognises interventions do not work for every person, in every circumstance, but seeks to understand what works, for whom, how, and in what circumstances (Pawson & Tilley, 1997). This understanding is formulated as a programme theory, iteratively developed throughout various research stages. Theory development typically involves using multiple methods of data collection and analysis, commonly qualitative, and less commonly mixed methods (Renmans & Pleguezuelo, 2023), to identify what ‘Context, Mechanism Outcome Configuration (CMOC)’, or set of CMOCs, is at work (Dalkin et al., 2015). This sets out how contextual factors (personal, structural and organisational elements pre-existing the intervention) affect the causal mechanisms (how intervention resources, or ‘mechanism resources’ interact with the reasoning and reactions of intervention users, or ‘mechanism responses’) involved and the outcomes this context/mechanism combination is expected to generate (Pawson & Tilley, 1997; Wong et al., 2017).

The programme theory details how specific Mechanisms, triggered by a resource (such an intervention) within a certain Context, result in which specific Outcomes. By providing a causal explanation of how certain contextual factors can hinder or facilitate the outcome-generating mechanisms (Jagosh et al., 2022). The programme theory provides the intervention’s users (e.g., professionals who will deliver or commission it) with knowledge required to decide who to deliver the intervention to and when, how to deliver it and when adaptations are necessary (Fletcher et al., 2016).

Middle Range Theories (MRTs) play an important role in realist theory development (Greenhalgh et al., 2017b) and are defined by their level of abstraction. They are less overarching than ‘grand theories’ of society or human experience, such as structuralism or Marxism and less granular (more general) than the programme theory we were refining. MRTs are explanations about specific phenomenon that are ‘close enough for observed data to be incorporated in propositions that permit empirical testing’ (Merton, 1968, p. 448).

Philosophical Assumptions Underpinning Realist Theory Development

Realist evaluation is routed in the ontological assumption that there are real causal mechanisms to be observed objectively (Bhaskar, 1975; Sayer, 1992). Realists apply Bhaskar’s concept of a ‘stratified ontology’ (Bhaskar, 1989) whereby phenomena fall into three overlapping domains: the ‘empirical domain’ of observed concrete experiences and events which can be captured in data, the ‘actual domain’ of phenomena which exist whether or not people are aware of them, and the ‘real domain’ of generative mechanisms, causal powers with potential to produce events (including experiences) when triggered. Realism is situated between positivist and interpretivist paradigms in that it treats both individuals’ perspectives and their situations as real phenomena, causally interacting with each another (Maxwell & Mittapalli, 2010) and acknowledges that all knowledge is partial and grounded in perspectives (Maxwell, 2012).

To help identify underlying mechanisms, deductive and inductive reasoning can form part of a realist analysis, but taken alone do not necessarily provide the range of thinking needed to develop explanatory theories (Decoteau, 2018). Abduction can also be used, to develop inferences from the data, reframing empirical knowledge into a more abstract and general form (Fletcher, 2017), moving from an empirical finding, observed in the data e.g., that sports coaches feel ‘compelled to help [their] club in matters other than coaching’ to an inference, still grounded in data, but more theoretical, such as ‘Coaches [could/often/are likely to] feel a sense of commitment to and solidarity with their club’ (example from Wiltshire & Ronkainen, 2021, p. 171).

A ‘retroductive’ approach essential to realist analysis (Mukumbang et al., 2021), strives to identify ‘hidden causal forces that lie behind identified patterns or changes in those patterns’ (Greenhalgh et al., 2017, p. 1). It is a ‘mode of inference in which events are explained by postulating mechanisms which are capable of producing them’ (Sayer, 1992, p. 107). Analysts adopt a ‘retroductive’ mindset, which in qualitative research might start with using realist interview and analysis techniques (Manzano, 2016; Pawson, 1996) to investigate what social and psychological mechanisms could be acting as causative agents for change, e.g., enquiring of the data: What is it about this phenomenon exactly, that leads to that outcome? This analytic mindset can work in tandem with deductive, inductive, and abductive approaches, to build theory of how contexts and mechanisms interact to generate outcomes (Gilmore et al., 2019). A team approach to creative, speculative thinking, involving a range of stakeholders is crucial to this process (Jagosh et al., 2022).

Researcher Guidance

While there is a wealth of literature detailing the philosophical principles underpinning realist theory development and retroduction, there is a lack of detailed practical advice for researchers on precise methods (Fletcher, 2017). This problem is recognised by RAMESES who encouraged realist evaluators to ‘provide details on what was done, why and how – in particular with respect to the analytic processes used.’ (Wong et al., 2016, p. 16).

One type of approach to realist analysis that has been detailed more thoroughly is the use of thematic analysis (TA) to support programme theory development. TA involves ‘developing, analysing and interpreting patterns across a qualitative dataset’ (Braun & Clarke, 2022a). To facilitate this process, a coding framework is iteratively developed to support inductive and/or deductive coding. Once coding has been completed, often over several rounds, researchers start to identify core concepts or shared perspectives present in the coding, and from these, researchers generate themes (Braun & Clarke, 2022a).

Within realist studies, researchers have varied in terms of the extent to which they label their methodology as TA and how closely their methodological description aligns with TA aims and principles. The degree of alignment has perhaps been further complicated by a move to differentiate between different types of thematic analysis by researchers who initially developed the methodology (Braun & Clarke, 2022b).

Some studies have opted to use similar approaches to coding but have chosen to reinterpret, with a realist lens, what a ‘theme’ is. For instance, Halsall et al. (2022) describe their retroductive method of using an initial codebook based on constructs from existing theories, undertaking several rounds of coding, and categorising theoretical constructs as ‘contexts’, ‘mechanisms’ or ‘outcomes’ during a final refinement phase.

A different approach was taken by Wiltshire and Ronkainen (2021), who employed inductive coding to generate themes that could then be grouped based on the knowledge domain they related to. Experiential themes based on participants’ statements about their hopes, beliefs and feelings were assigned to the empirical domain and inferential themes relating to probable mechanisms at play were assigned to the actual domain. Dispositional themes (attributed to the real domain) were then developed, through retroductive reasoning regarding the mechanisms that must exist for an event or experience to be possible. All levels were combined to form explanatory theory statements.

In contrast, Gilmore et al. (2019) do not describe their realist approach as TA. They opted to code directly to IPT statements (i.e., causal strings of concepts) rather than treating contexts, mechanisms, and outcomes as separate themes. Child nodes were created as the initial theory underwent revision, allowing for clear theory refinement tracking. Any contextual or outcome information not directly linked to the existing CMOCs was captured under new parent nodes. An additional step of synthesising contexts, mechanisms and outcomes was not deemed necessary.

Similarly, Dalkin et al. (2021) coded to initial ‘If...then’ theory statements, believing that coding to contexts, mechanisms and outcomes separately would lead to disjointed themes. Both Gilmore et al. (2019) and Dalkin et al. (2021) advocate memo writing to record rationale for iterative theory refinement, thereby increasing analytic transparency.

Like many qualitative researchers using TA, some realist researchers use Computer Assisted Qualitative Data Analysis (CAQDAS) software programmes, such as NVivo, for structuring the often ‘messy process of generating, refining and testing complex programme theories when drawing on multiple data sources simultaneously.’ (Dalkin et al., 2021, p. 124). NVivo provides a way of organising and processing the data, keeping track of the analytic process, aiding transparency, and rigour.

Summary of Background

Literature detailing methods for realist programme theory development is building, but methods vary, and focus is predominantly on IPT development. There is a lack of guidance on how to elaborate IPT throughout distinct evaluation phases - from early prospective intervention development work to feasibility studies, to full evaluation.

We now set out the approach we took to elaborating our IPT in three sections. We start with the progress that had been made on theory development prior to our feasibility study, or Phase 1a, as described in Figure 1: the prospective pre-feasibility phase. This is important background for understanding the approach we later took in the feasibility study. We then move on to describe our process for theory elaboration during the feasibility phase, before presenting the resulting elaborated programme theory (EPT).

Initial Programme Theory Building in Prospective Phase (1A: See Figure 1)

D-PACT’s initial CMOCs were expressed as hypothetical ‘If…then…’ causal theory statements (Dalkin et al., 2021; Jagosh et al., 2022). At the prospective stage, our programme theory focused mainly (but not exclusively) on the relationship between mechanism-resource and mechanism-response. This is because, prior to feasibility testing, we had limited access to ‘in the field’ knowledge of how potential contexts would influence our hypothetical mechanism-resource: mechanism-response relationship and the outcomes it might generate. At this early stage, our theoretical work was also largely driven by our need to develop the D-PACT practitioner support package, consisting of an intervention manual and training resources, ready for feasibility testing (Phase 1b).

In Griffiths et al. (2022) we set out how literature reviews and knowledge of existing person-centred coaching-based interventions such as PARTNERS2 (Gwernan-Jones et al., 2020) informed the development of 21 broad ‘if-then’ theory statements which we called our IPT. The following is an example IPT statement:

‘IF the Dementia Support Worker sets up meetings with/makes referrals to/signposts to healthcare, social care and community resources AND offers support with new encounters (mechanism resources) THEN the person with dementia and carer will feel more hopeful, more confident in their own abilities, allowed to do things for themselves (carers), and will start to try out new behaviours. (mechanism responses)’ (Griffiths et al., 2022).

Through analysis of interview and focus group data, and consultation with key stakeholders, we iteratively refined the statements throughout the prospective phase.

Like Gilmore et al. (2019) we coded these data deductively and inductively to the 21 causal statements. These were grouped into four areas of overall programme strategy:

• Facilitation: statements relating to DSW training, supervision, and mechanisms for embedding DSWs in primary care settings.

• Style of Delivery: statements relating to DSW mechanisms for building trust and engagement with people with dementia and carers over time.

• Developing shared understanding and generating a plan: statements relating to the central coaching approach used by DSWs to build a mutual understanding of peoples’ priorities and values, and co-create a plan of action to address unique needs.

• Collaboration: statements relating to the way in which DSWs worked with health and social care professionals and community assets to share and enact plans of action.

In addition to coding by statement, an inductive thematic analysis (Braun and Clarke, 2019) of the same data was undertaken, specifically to elucidate relevant contexts. We opted to label this analysis as inductive TA, whereby we identified concepts, or themes, that were meaningful in the data, without aiming to combine these into a causal chain of concepts. Given the strength of TA in uncovering broad psychological and social phenomena, our aim was to explore whether there were overarching contextual elements that may affect how the intervention worked as a whole. The CMOC-related themes we developed (1–6) are represented in Figure 3. Feelings of control, hope and social connectedness, as well as an individual’s experiences of previous care, their age, nature and progression of dementia, social situation, and cultural background, all played a part in influencing peoples’ acceptance of and adaptation to dementia. This acceptance and adaptation waxes and wanes over time.

Figure 3.

Contextual Themes

From these findings we developed a logic model, which we continued to refine concurrently with the statement-based programme theory. The ‘If-then’ statements provided a detailed written account of how change occurred through individual mechanisms of the D-PACT intervention, whilst the logic model provided a visual overview of how aspects of the model were interlinked and collectively brought change. Articulating theory in both forms helped us cross-check our analysis at every stage and consider what was missing or unexplained. At the prospective phase it allowed us to:

• Visually map our contextual themes to the four programme strategies initially, hypothesising how contexts might affect intervention delivery at strategy level (see Supplemental File One).

• Having done this mapping, review whether and how contextual themes were already being addressed through the IPT. For instance, the overarching context of people with dementia and carers wanting more control over their lives, was already central to the IPT, with mechanism resources designed to address this. Less consideration had been given to how hope and trust may impact on peoples’ response to the intervention (mechanism response). As a result, there was less coverage of these topics in our practitioner support package.

Next, we considered how the IPT might be elaborated to incorporate the impact of the newly identified additional contexts on mechanisms and outcomes, and how practitioners might be supported to address them through the practitioner support package. We opted not to embed all our contextual elements into our programme theory statements at this stage, as they were not tied to specific mechanism resources or responses but were insights into factors that may affect how people with dementia and carers responded to the intervention as a whole. Therefore, we referred to them as ‘floating contexts.’

Rather than setting out with a fixed idea of existing theory that would help us make inferences from the data, MRTs gradually became identifiable as potentially relevant and were narrowed down through the process. For example, based on the TA work on relevant contexts, we became interested in literature on hope. This led us to a conceptual (MRT) model of adaptation in dementia (Górska et al., 2021), which we then drew upon to refine mechanism responses and outcomes. We were also influenced by Kitwood’s concept of how a sense of personhood (or a sense of identity and self-worth) can be enhanced in dementia through ‘positive person work’ (Kitwood, 1997).

By the end of the prospective phase, we had found coding to statements feasible and completed a first stage elaboration of the IPT into a series of 42 statements (see Supplemental File Two). We developed a prototype intervention, informed by this theory, and were ready to collect evidence of how the intervention could work in practice to elaborate our programme theory further.

Theory Elaboration During the Feasibility Phase (1b: See Figure 1)

Key elements of our approach to elaborating our theory during the D-PACT feasibility study (Figure 1: Phase 1b) are presented in this section: Feasibility data collection, Data coding and analysis, The coding framework, Coding guidance and sense checking, Use of Memos, and Analysis meetings. It is important to note that this work, in line with the realist evaluation approach, was not linear but involved iterative overlapping of data collection, programme theory elaboration, engagement with stakeholders, and drawing on existing MRT. First, we set out the aims of theory elaboration and the timeline of activities in this phase.

Aims of Theory Elaboration Were to:

(1) Add more evidence informed detail to the programme theory.

(2) Extend the breadth of the programme theory.

(3) Use EPT to finalise the D-PACT practitioner support package.

The theory elaboration included patient and public involvement and engagement (PPIE) through monthly meetings with a group of contributors with experience of dementia, their carers, and former carers. We shared and discussed our ongoing learning with this group and sought their perspectives. Figure 4 shows a timeline of feasibility phase activities.

Figure 4.

Feasibility Phase Timeline

Feasibility Data Collection

A range of qualitative data was collected during feasibility. We have reported in detail on recruitment, selection processes and participant demographics elsewhere (Wheat et al., 2023). Here we present the nature of data collected:

• Semi-structured Interviews: Seventeen people with dementia and carers (eight dyadic interviews, seven with carer only, and two with person with dementia only), eight DSWs, three GPs and practice managers, and three supervisors. Data enabled exploration into types of changes those receiving and delivering the intervention experienced and their perceptions of how, when, why and for whom this change was generated.

• 13 non-participant observations: Researchers completed unstructured fieldnotes of video recorded meetings between DSWs, people with dementia and carers, focusing on interaction during these encounters. This enabled us to examine how specific communication practices influenced how the intervention was delivered and responded to.

• 50 DSW unstructured diary reflections: Completed after DSWs met with the people they supported. These provided an opportunity for DSWs to reflect on what worked well, less well in a timely way.

• Video-stimulated Recall interviews (Nguyen et al., 2013): 3 with DSWs, 1 with carer, 1 with person living with dementia. An established method in applied health research, whereby participants are video-recorded and watch back salient extracts of recordings, to stimulate discussion. This provided important insights, for instance on how internal reasoning influenced practitioners’ approaches to intervention delivery, and the responses of intervention recipients.

Due to a delay in recruitment in one site, data collection and coding occurred in two distinct phases. Interview schedules were informed by the programme theory developed in phase 1a. Interviewers received training in realist interviewing techniques (Manzano, 2016; Pawson, 1996). Observations, while unstructured, were focused on how core intervention actions (resource mechanisms), which were interactive in nature, played out in practice. For more detail on data collection methods and decision-making regarding when to stop data collection see Wheat et al. (2023).

Data Coding and Analysis

Rather than moving linearly from coding to analysis, these processes were entwined. Splitting these processes seemed inconducive to the aims of this stage of the project, as we wanted ongoing analysis to inform subsequent data collection and coding decisions. In line with Gilmore et al. (2019), we opted to label this methodological approach as a realist analysis, informed by processes one would take in TA. We now describe how we approached coding and analysis and how they interlinked.

The Coding Framework

As coding to statements had been successful in the prospective phase as a means of theory development, we decided to adopt this approach again, developing a coding framework, structured by our 42 - ‘If...then’ formatted causal theory statements. A sample of the coding framework is shown in Supplemental File Three.

Our coding centred upon a parent node for each of the four programme strategies. Under each of these nodes, additional child nodes were created for each programme theory statement relating to that domain area. Under these child nodes, grandchild nodes were created for a), evidence that supported the statement and b), evidence that suggested refinement to the statement. At the same level as the child nodes for each parent (programme strategy) node, additional nodes were created to support coding of inductive insights (i.e., insights not captured by existing theory statements) relevant to that programme strategy.

One of these inductive nodes was originally labelled to include ‘dark logic’ (Bonell et al., 2015), to indicate that the coder should also code instances of data that suggested that the intervention had unintentionally generated harmful mechanisms and outcomes to this node, as we wanted to ensure our EPT and logic model did not naively presume that the intervention would only ever generate positive change. We felt that the addition of these nodes could also help us better understand when the intervention did not work and for whom it did not work for – potentially building further theory or refining existing theory to mitigate or prevent against these types of unintended change. We later took out this reference to ‘dark logic’ in this node’s label to avoid reinforcing racist connotations (Lingayah & Kelly, 2023) and to prevent biassing the inductive coding toward only negative unexpected changes, rather than insights pertaining to neutral, positive, or negative unexpected changes. It was agreed that if inductive coding led to new statements detailing unintended mechanisms and outcomes, they would be referred to as ‘unintended statements’ (Jabeen, 2016), linked (through statement numbering) to their counterpart ‘intended’ casual statement.

As the intervention phase had started before the pre-Covid 19 pandemic, and then continued remotely during it, the question arose of how adapting the intervention for remote (on-line) delivery impacted on how the intervention worked in practice (Wheat et al., 2023). To explore this, an inductive node for researchers to code insights specifically relating to remote delivery was also created.

In addition, a node (named ‘floating contexts’) was added to each programme strategy node sub-group to enable coding of any contextual details present in the data that were also relevant to that specific programme strategy.

Coding Guidance and Sense Checking

We created a codebook of coding instructions with space for coders to write reflections, capturing ongoing thoughts on how the theory could be elaborated (see Supplemental File Four). The deductive and inductive coding approach was carried out by SG, HW and AG initially. After an initial coding test phase, SG and HW carried out the remainder of the coding. AG (having developed experience of the coding framework), carried out two rounds of ‘sense-checking’ of the data assigned to a selection of nodes to check whether SG and HW were interpreting and applying the coding process similarly. Any discrepancies were discussed, and decisions made about what coding rules to apply, for reliability. This conformed with qualitative research reporting guidelines (Levitt et al., 2018) and stabilised the coding methods.

Use of Memos

To enhance analytic transparency (Dalkin et al., 2021) the researchers kept memos relating to all programme strategies under a separate parent node, capturing ongoing reflections about the ease of the coding process, how coding was influencing their analytic thoughts and how insights from the data was informing their analytic thinking. In these memos researchers captured ongoing ‘workings’ towards elaborated statements or developing new statements based on inductive coding. Influences from other sources such as PPIE meetings and further reviews of academic literature (including MRT) were captured. This was a way of journaling the programme theory development. Figure 5 shows an example of a researcher memo which reflects on potential changes to a causal statement based on different sources of evidence and on potential elements to include in the DSW training.

Figure 5.

Example Memo

Analysis Meetings

At agreed deadlines based on coding progress, the coders and sense checker met to analyse the coding done so far. The codebook was used as a living document to aid these meetings. Prior to each meeting, each coder would review their memos and write into the codebook suggestions for change to each programme theory statement, based on their collection of reflections. These would then be shared during the meeting, along with any suggested changes to the coding framework and process. These discussions in turn informed any changes to the ongoing realist interviews, to ensure that new insights and gaps in understanding were explored in subsequent data collection. Key points of these analytic discussions, along with examples from the data, were also shared with our PPIE group and wider members of the team (who were actively collecting data and had expertise in this field) , so they could also inform theory elaborations. Notes from PPIE discussions were drawn on in our analysis meetings to inform programme theory elaboration and in turn, the refined statements were reviewed for accuracy by the PPIE group.

After these meetings, each coder’s coding files were merged to create a master copy. Any agreed changes to the theory (and therefore coding framework) and approach to data coding were undertaken prior to further coding. We also refined the logic model and continued to use it as a way of cross-checking our analysis.

At the first analysis meeting we decided not to incorporate the coding of observational data in the feasibility analysis. These data were collected to enable exploration into how micro-level communication practices influenced how people responded to the intervention. This decision was made to keep the programme theory at a more generic and manageable level. However, the findings were core to the elaboration of the practitioner support package instead. For example, the observational data showed that practitioners, when interacting with people with dementia and carers, were sometimes asking leading questions e.g., asking the person with dementia ‘You liked that didn’t you?’ (regarding a visit to a replacement care setting). This misaligned with our coaching model. As a result, we built a session on question-design into the practitioners’ ongoing training. In this respect, we saw the practitioner support resources as a continuation of the programme theory, providing more specific detail on how the intervention resources could be delivered in order bring about the intended outcomes, in this case, creating potential for self-empowerment for people with dementia and carers in exploring their concerns.

The proposed elaborated theory was presented to the wider research team at a final analysis meeting, allowing for sense-checking, after which final refinements were made. Throughout the analytic process, the statements were elaborated three times (from 42 statements at the beginning of the feasibility phase, to 40, to 39 and finally 42).

Elaborated Programme Theory (EPT): Final Feasibility Statements

The final set of causal statements included those that had been refined, inductive statements, those relating to remote intervention delivery, those where context was more confidently stated as part of the causal chain, and statements about unintended mechanisms and outcomes. For some statements, outcomes were expressed as more proximal or more distal. The statement referred to in the memo example (Figure 3) was iteratively refined and included the importance of offering laughter/fun and adapting the style of delivery to a person’s unique context:

IF the person adopts person-centred interaction skills that promote a sense of personhood for the person with dementia and carer, finding ways of enjoying interaction with them, using appropriate self-disclosure and laughter/fun, demonstrating interest in who they are and what they have to say, and tailoring communication to fit the mode of interaction (in person or remote) and the person’s characteristics (e.g., stage and acceptance of dementia) (mechanism resources adapting to context), THEN the person with dementia and carer will feel important and valued, feel like who they are as individuals has been recognised, feel more connected to the wider world, and feel like they have a positive relationship with the DSW (mechanism responses) THEN they will talk about their situation (with the DSW and others) (immediate outcome). This will lead to improved emotional health of the person with dementia and care: reduced anxiety/depression (medium-long-term outcome).

Further examples are shown in Supplemental File Five.

By the end of the feasibility phase, we had an EPT to take forward to a full evaluation study, along with a comprehensive practitioner support package, informed through the feasibility work.

Discussion

The aims of the feasibility phase of the study, in relation to programme theory elaboration were to: (1), add more evidence informed detail to the programme theory; (2) extend the breadth of the programme theory and (3), use the EPT to finalise the D-PACT practitioner support package. We now reflect on how well our chosen approach enabled us to achieve these aims.

Data Collection: Benefits and Challenges

The breadth of qualitative data collection supported our aims, as it provided insights into how people thought it would work, and on how it did work in practice. It enabled different aspects of the theory to be elaborated to varying degrees e.g., the DSW reflections provided more insights into ‘floating contextual factors’ whereas the interview data enabled us to explore what types of hidden mechanisms were triggered for people with dementia and carers. Through observational data collection and analysis, we were able to show practitioners how the theory could work in practice, through refining the practitioner support package.

However, several challenges during data collection impeded our ability to elaborate certain aspects of our programme theory. Firstly, our ability to engage in retroductive reasoning, when analysing interview data, was affected substantially by the level of reflection interviewees were able to engage in. This varied substantially amongst practitioners. Despite scheduling interviews for people with dementia as soon as possible following support meetings, difficulties in recalling events and associated feelings, understanding and beliefs, impacted on interview depth. Given these challenges, the video observations offered an essential source of triangulation.

Our pilot use of video stimulated recall (VSR) interviews (Nguyen et al., 2013) helped mitigate recall issues with one person with dementia and carer dyad. However, the participants experienced distress due to the data highlighting how dementia symptoms had worsened since the video had been made. It also caused them discomfort when watching discussions of sensitive topics with the DSW. As a result, this data collection was discontinued. However, the limited data we did collect provided rich insights. VSR has potential to be adapted for use in dementia studies, with a person-centred approach attending to the needs of people with dementia and the avoidance of distress. This topic requires further research.

Due to Covid-19 related pressures on staff at the time of data collection, we collected little data from professionals based at participating general practices. This reduced our ability to elaborate on programme theory statements relating to the programme strategy ‘Collaboration.’

Coding and Analysis: Benefits and Challenges

Taking a context-focused thematic analysis approach alongside coding to realist theory statements enabled us to identify potential contexts during the prospective phase, with some of these becoming embedded within our programme theory.

Coding only to statements during the feasibility stage was manageable, both because we had grouped the statements into programme strategies to streamline coding and because coders were largely consistent and became highly familiar with the statements. This approach enabled us to adopt a retroductive analytic mindset as we were consistently dealing with entire causal reasoning strings in our thinking: whilst coding, whilst writing memos to rationalise our decisions, and whilst discussing theory in analytic meetings. Postulating underlying mechanisms to explain outcomes and considering the impact of context was ingrained in our thinking throughout. Like Gilmore et al. (2019) and Dalkin et al. (2021), we believe that coding to Context, Mechanism and Outcomes separately at this stage of theory development would have been a more disjointed approach, impeding understanding of the connections between different parts of the CMOC and therefore of the D-PACT intervention.

Developing a visual logic model alongside the theory statements helped us check that our statements made logical sense within the overall developing ‘picture’ of the intervention, in a way that written statements alone do not necessarily achieve. However, by the end of the feasibility stage the logic model had become too complex for dissemination purposes. We realised it had been helpful just to us in formulating our analysis of the intervention and its CMOCs, but that a much more simplified model would need to be created for dissemination.

NVivo software worked well for managing the coding process. We made the decision not to use a cloud-based group NVivo file due to reports of file crashes and/or frequent sync failure (https://delvetool.com/blog/nvivo-cloud-collaboration). We found we were easily able to merge individual coding files, and the memo option helped us journal our ‘in the moment’ analytic thinking. However, whilst NVivo enabled us to manage data and capture reflections, we did need a separate (code book) document to support and capture analytic discussion.

Due to a significant delay in recruitment and data collection in one site during feasibility, we decided to code and analyse the other site data first, while data collection continued at the former. This staggered approach meant we were not looking at data from the different geographical settings concurrently, which potentially limited our ability to elaborate on existing ‘floating contexts’ using such insights. Also, by the time we started coding the data from the second site, one of the two main coders had left the study. The two main coders had, over time, developed an in-depth knowledge and understanding of the developing theory. This presented a challenge to the remaining coder who undertook the final stage of analysis without the same degree of interactive analysis that had come before. We argue that realist coding and analysis requires an in-depth knowledge and understanding of the ongoing programme theory development, built up over time. This requires personnel to remain throughout a project. Short-term research contracts can make that difficult.

Finally, our methods were purely qualitative in nature, a feature highlighted by Renmans & Pleguezuelo (2023) as common in realist research. These authors also propose that to strengthen realist evaluations, there is a need to explore a broader range of methods, including quantitative surveys. This is an area we did not address, but one that is ripe for future research.

Conclusions

We have provided a detailed, worked example of how we elaborated our D-PACT programme theory using a realist approach. We hope that sharing this worked example and our reflections on decisions along the way, and what worked well/less well, will help other researchers. We also hope this account goes some way towards demystifying realist evaluation and retroductive thinking; areas that can be intimidating for new readers (as they were for us initially). Researchers who are suitably trained and immersed in realist evaluation develop realist analytic mindsets, but consideration needs to be given to how retain them in their research roles so as to capitalise on their skills and knowledge. We have presented how we arrived at a final EPT at the end of a feasibility study. The next step was full evaluation, where rolling out the intervention across a wider range of settings would enable us to elaborate contextual and collaboration elements of theory in more depth. Findings from that evaluation will be reported elsewhere.

Supplemental Material

Supplemental Material - Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project

Supplemental Material for Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project by Sarah Griffiths, Hannah Wheat, Sarah Morgan-Trimmer, Lauren Weston, Alex Gude, Tomasina M. Oh, Rod Sheaff and Richard Byng in International Journal of Qualitative Methods

Supplemental Material

Supplemental Material - Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project

Supplemental Material for Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project by Sarah Griffiths, Hannah Wheat, Sarah Morgan-Trimmer, Lauren Weston, Alex Gude, Tomasina M. Oh, Rod Sheaff and Richard Byng in International Journal of Qualitative Methods

Supplemental Material

Supplemental Material - Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project

Supplemental Material for Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project by Sarah Griffiths, Hannah Wheat, Sarah Morgan-Trimmer, Lauren Weston, Alex Gude, Tomasina M. Oh, Rod Sheaff and Richard Byng in International Journal of Qualitative Methods

Supplemental Material

Supplemental Material - Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project

Supplemental Material for Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project by Sarah Griffiths, Hannah Wheat, Sarah Morgan-Trimmer, Lauren Weston, Alex Gude, Tomasina M. Oh, Rod Sheaff and Richard Byng in International Journal of Qualitative Methods

Supplemental Material

Supplemental Material - Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project

Supplemental Material for Using a Realist Informed Qualitative Approach to Elaborate Programme Theory: Experiences From the Feasibility Phase of the D-PACT Project by Sarah Griffiths, Hannah Wheat, Sarah Morgan-Trimmer, Lauren Weston, Alex Gude, Tomasina M. Oh, Rod Sheaff and Richard Byng in International Journal of Qualitative Methods