Sage Journals: Discover world-class research

Abstract

This article discusses the alignment of framework analysis with a critical realist philosophy in qualitative analysis. Whilst both are used in social sciences and prioritise meaning making of complex phenomena, the two approaches have not been combined prior to the study outlined in this article. The article describes critical realist ontology and epistemology, the implications of this philosophy for data analysis and the structured processes of framework analysis. Using an example from a study of youth programmes in Aotearoa/New Zealand, the article demonstrates the value of framework analysis for critical realist studies. Emphasis is placed on the capacity of framework analysis to support varied foci on data, theory, and different modes of inferencing which are used in critical realist studies to provide an explanatory account of data.

Keywords

critical realism framework analysis qualitative data analysis youth programmes positive youth development Aotearoa/New Zealand

Introduction

This article illustrates how critical realism and framework analysis can be used together to enhance qualitative analysis. Analytic processes in qualitative research have been criticised for lacking rigour and reliability due to their intuitive and not easily replicable nature (Kiernan & Hill, 2018). Equally, the application of critical realist philosophy is not always clearly visible in methodological accounts, which obscures its potential to generate meaning from social data (Fletcher, 2017; Oliver, 2012; Stutchbury, 2022).

In combination, framework analysis and critical realism can address some of these critiques. Framework analysis provides a rigorous analytical methodology via systematic steps, and the ability to ‘push beyond’ identifying themes to the intuitive leaps that comprise qualitative research (Goldsmith, 2021, p. 2062). Furthermore, explicit use of critical realism’s abductive and retroductive analysis assists researchers to develop explanatory accounts, thereby enhancing the rigour of qualitative studies (Danermark, 2002).

The study described in this article examined whether outcomes, defined as the gains made by young people, were perceived similarly by youth and staff in different youth development programmes in Aotearoa/New Zealand and whether a shared outcomes framework could be articulated. It was conducted for the lead author’s PhD study. Framework analysis identified nine key outcomes which were common to most settings, and critical realist analysis identified potential generative mechanisms behind these outcomes.

The article begins by outlining critical realism and framework analysis and describing the ways they can combine to support data analysis in qualitative studies. It then describes the analytic processes used in the study, with a particular focus on how framework analysis supported the application of critical realist philosophy.

Critical Realism

Critical realism is a research philosophy that addresses both ontological and epistemological concerns. Originating in the work of Bhaskar (1975), critical realism draws a distinction between the world, what is real, and the way we experience it (Longhofer & Floersch, 2012). What is real shapes the world, including the social domain, irrespective of our recognition of this. Furthermore, our understanding of the world is always conditional; shaped by our perception and experiences (Groff, 2004). So, social structures, systems, and the theories we hold about the world are all shaped by our experiences. Critical realism, therefore, is ontologically realist and epistemologically relativist. That is, reality exists whether or not we perceive it. At the same time our perception and experience of reality is shaped by our understanding of the world (Groff, 2004).

To reconcile ontological realism and epistemological relativism, critical realism understands reality as divided into three linked domains: the real, the actual and the empirical (Longhofer & Floersch, 2012). This categorisation includes both the physical and the social world.

The real domain includes physical objects, social structures such as racism, and psychological processes (Levers, 2013; Longhofer & Floersch, 2012). At the real level, different elements interact to create observable effects, and thus are said to have causal (or generative) powers (Longhofer & Floersch, 2012). Due to the many different variables and the complexity and contingency of the systems in which different elements exist, these interactions are not always present or predictable and this is why we may see similar or different results in different situations (Deforge & Shaw, 2012; Longhofer & Floersch, 2012; Roberts, 2014; Seal, 2016).

At the actual level of reality, we see events, behaviours, and outcomes. These are the observable, although not always observed, results of interactions between generative mechanisms at the real level (Deforge & Shaw, 2012; Longhofer & Floersch, 2012). What is observed and measured comprises the empirical level, which may be a small subset of the actual. However, the empirical is important, as it is through this that we hypothesise about events at the real level (Fletcher, 2017). Thus, our understandings of the real are relative and therefore fallible because they are filtered through human perception and may never accurately, fully, or objectively describe the real (Longhofer & Floersch, 2012; Seal, 2016). Figure 1 describes the three nested domains in critical realist ontology.

Figure 1.

Critical realism’s three domains.

Critical realism is also interested in the interaction of structure and agency (Deforge & Shaw, 2012; Seal, 2016), as both have generative powers. Structures and systems, such as racism are created by humans but have influence at the actual and empirical levels and are therefore real. They have the power to enhance or limit agency. Equally humans can generate or change structures (Fletcher, 2017; Scott, 2005; Seal, 2016). Thus, structure and agency are separate but connected (Stutchbury, 2022).

Critical Realist Research

Critical realism is becoming a popular tool in social science research where phenomena occur in dynamic systems and where there is interest in understanding complex interactions, such as how agency and structure combine to impact on people’s lives (Fletcher, 2017).

A critical realist approach has several methodological implications, particularly for data analysis. Firstly, the goal of critical realism is to describe tendencies, rather than to determine causal relationships (Fletcher, 2017). This is because it is not possible for everything contained at the real level to be understood at the empirical level. In qualitative studies, tendencies can be observed via coding methods, whilst the identification of generative (also called causal) mechanisms and how they interact can help to explain these tendencies (Fletcher, 2017). In describing both tendencies and generative mechanisms, critical realist research seeks to uncover pieces of the actual and the real (Seal, 2016). For example, the study of youth programmes in this article was interested in both the outcomes of programmes (their tendencies at the actual level) and interaction of the structural influences that could both support and constrain outcomes (generative mechanisms).

Secondly, theory is central to critical realist research because theories represent our explanations of what occurs at the real level. These may change and are recognised as embedded in worldviews, cultures, times, and understandings (Longhofer & Floersch, 2012; Seal, 2016). As such, analysis through theory and subsequent support, rejection, or modification of these is an important part of critical realist research (Danermark, 2002; Danermark et al., 2019; Fletcher, 2017; Meyer & Lunnay, 2013). The study described in this article was embedded within positive youth development (PYD) theory, which is both a theory of human development and an approach to providing programmes and interventions for young people (Hamilton et al., 2004). Data were analysed through this PYD lens, with other theories added to provide deeper explanation where necessary.

Due to the interest in both generative mechanisms and theory, critical realist studies are more interpretive than descriptive (Danermark, 2002; Stutchbury, 2022). This invites researchers to ask ‘why’ questions about what is happening behind the empirical level, and how structure and agency interact in context (Stutchbury, 2022). At the same time, critical realist research invites researcher reflexivity, by recognising that our interpretations and theories represent particular research lenses and worldviews and are not the only possible explanation for phenomena (Longhofer & Floersch, 2012). To help answer ‘why’ questions, two key reasoning processes are used: abductive and retroductive analysis.

Abductive reasoning involves considering data through the original theoretical premise of the study, through additional theories as indicated by data analysis and, where relevant, to propose new theories and explanations for what is known (Danermark, 2002; Meyer & Lunnay, 2013). Abductive reasoning looks beyond participant experiences to explore possible generative mechanisms (Danermark et al., 2019; Fletcher, 2017; Vincent & O’Mahoney, 2019). Abductive processes support critical realism’s relativist approach to knowledge; that more than one explanation is possible, and as such, data that do not fit existing theories are important. Abductive reasoning may be particularly relevant in cross cultural settings, such as the bicultural setting of Aotearoa/New Zealand, where the study reported in this article took place. In Aotearoa, a critical realist approach sees both Māori (the indigenous people of Aotearoa) and other worldviews as real and valid (Martel et al., 2022) and draws on theories from all of these. For example, the study used Western, Māori, and Pacific nation’s theories to help understand the diversity of youth experiences in programmes.

Retroductive reasoning involves assessing conditions which must be present or absent for something to exist or be possible (Danermark et al., 2019; Eastwood et al., 2014). This can include looking for patterns as well as extreme or negative cases for comparison, and to help separate universal conditions from contextual ones (Danermark et al., 2019). Understanding such conditions can help to identify generative mechanisms. For example, the study described in this article, showed that conditions such as young people navigating to programmes to meet needs, creating a positive environment for each other, and leaders seeing and responding to the skills, interests, identities, and cultures of young people were important for outcomes. These conditions acted as signposts for generative mechanisms.

Both of these analytic processes are researcher driven, thus critical realist research clearly acknowledges the positioning and authorship of the researcher. It is not merely an observational account or a rich description, but rather an interpretive process, which sees research as the product of both participants and researcher, recognising that neither hold a full or accurate picture of reality (Clark, 2008; Groff, 2004). The interplay between participant experiences and researcher interpretations can be clearly explored through framework analysis, which is described in the next section.

Framework Analysis

Framework analysis is a qualitative analysis method that takes a structured, five stage approach. Framework analysis has its origins in social policy research but is increasingly applied in health and social services research because of its capacity to make analysis transparent (Parkinson et al., 2016; Ritchie & Spencer, 1994). It is an epistemologically neutral approach that can be used with a range of methodologies and inductive or deductive coding (Gale et al., 2013). Framework analysis is careful to include all data and remain close to raw data throughout the five steps. The structured stages ensure that data can easily be referenced, retrieved, and compared at any point (Ritchie & Spencer, 1994). It is these processes which serve to make the analysis more transparent (Kiernan & Hill, 2018). The stages of framework analysis are described below.

Familiarisation

Familiarisation involves initial review and coding of either the full dataset or a subset of data, depending on the size (Kiernan & Hill, 2018). If a selection is chosen, a good range of data is selected based on data collection methods, sample diversity, data collection times or the number of researchers (Ritchie & Spencer, 1994).

Framework Development

Following familiarisation, an initial coding framework is developed based on both emergent codes and a priori concerns, such as research questions (Parkinson et al., 2016; Ritchie & Spencer, 1994). This framework is considered a ‘rough’ draft and includes broad and sub-themes (Kiernan & Hill, 2018; Ritchie & Spencer, 1994).

Indexing

The framework is applied to all data and through this process the framework is further developed (Kiernan & Hill, 2018). New codes may be generated, some codes removed, and overlaps identified (Ritchie & Spencer, 1994). What is important at this stage is remaining close to raw data and ensuring all excerpts are indexed, including things that may at first appear irrelevant (Furber, 2010). This process, which is acknowledged as analytic and intuitive, continues until all data have been indexed (Kiernan & Hill, 2018). Kiernan and Hill (2018) emphasise the importance of documenting these processes to demonstrate researcher thinking and subjectivity. In critical realist studies, this documentation shows more clearly how participant experiences of reality are interpreted and can signpost thinking about generative mechanisms.

Charting

Charting involves summarising and grouping indexed data by extracting material from transcripts and grouping them into themes (Ritchie & Spencer, 1994). Grouping may be based on the framework, research questions or how the research will be written, and presented by case or by theme (Ritchie & Spencer, 1994). Data pieces are summarised, and relevant quotes included (Gale et al., 2013). By summarising and presenting data in order, charting allows data to be easily viewed, compared, and referenced back to raw data. During charting, the framework may be further modified, data re-indexed and larger themes identified (Kiernan & Hill, 2018).

Mapping and Interpretation

Mapping and interpretation draws together the previous phases to make sense of data, identify patterns and answer the research questions, which is an intuitive and researcher driven process. Presentation of the previous phases helps to highlight this subjectivity (Kiernan & Hill, 2018; Parkinson et al., 2016; Ritchie & Spencer, 1994). This subjectivity is consistent with a critical realist standpoint that all knowledge is conditional (Seal, 2016). These subjective decisions are made more explicit through the indexing and charting processes (Kiernan & Hill, 2018). Figure 2 shows the framework analysis process.

Figure 2.

The framework analysis process.

Alignment Between Framework Analysis and Critical Realism

Framework analysis and critical realism align because both are interested in the descriptions of phenomena and building understanding of what is occurring behind these descriptions (Gale et al., 2013; Longhofer & Floersch, 2012; Ritchie & Spencer, 1994). The structured and transparent process in framework analysis can indicate where abductive and retroductive reasoning are used, because each stage is clearly recorded and explained.

In critical realist studies, analysis needs to move between the concrete and the abstract. Abstract hypotheses about mechanisms are generated from data; then descriptions from data about how mechanisms might operate are proposed (Fletcher, 2017; Raduescu & Vessey, 2009; Roberts, 2014).

Framework analysis is ideally placed to support this movement by keeping close to raw data and ensuring that no important data are missed. In this way, patterns and extreme or negative cases can be easily identified, and retroductive inferencing processes are signposted.

Critical realism balances tensions between participant and researcher perceptions of reality and the theoretical descriptions we use to make sense of these (Groff, 2004). As such, analysis includes a combination of data driven, researcher driven, and theory driven processes. The stages in framework analysis enable these distinctions of focus to be more clearly identified.

The ability to support and signpost data, theory and researcher driven processes is an advantage of framework analysis to critical realist studies over other approaches, which may prioritise one position over another. For example, Fletcher (2017) argues that deductive, researcher driven analysis best fits critical realist studies because of the researcher’s role in abduction and retroduction. Conversely, Oliver (2012) suggests that data driven, grounded theory approaches align with exploring the interaction of structure and agency, and the fallibility of explanations. Framework analysis supports both, depending on the data and the research question. For example, in the study described below, the research question was explicitly interested in participant perception and the similarities and differences within this, suggesting a need for both data driven and researcher driven analysis. Where and how these processes occurred are described below.

The Process in Action

The following paragraphs describe how framework analysis was applied in a study of youth programmes in Aotearoa/New Zealand and how this process was informed by critical realist analysis and supported the identification of generative mechanisms. Analysis was undertaken by the lead author as part of a PhD study and discussed with the other authors of this article who were PhD supervisors.

The study explored outcome perception in 14 different positive youth development programmes in Aotearoa/New Zealand. A qualitative, exploratory methodology was deployed, examining how outcomes were defined, understood and experienced by youth; how they were operationalised, and prioritised in these programmes; the similarities and differences in outcome perception and whether or not there was enough similarity to form an outcomes framework. Ethics approval was granted for the study and participants provided written consent.

Interviews and focus groups included 110 youth and 17 staff. Nine key outcomes were identified in most programmes, suggesting there was sufficient similarity to form an outcomes framework. These were grouped into three themes: (1) Behavioural Outcomes, including outcomes of skills, achievement, giving back and agency; (2) Internal Outcomes, including outcomes of confidence, future focus and positive affect; and (3) Relational Outcomes including connection and positive identity. Three programme processes: a Youth Centric Space, an Accepting Atmosphere, and Leader/Adult Behaviours supported outcomes. Generative mechanisms of agency and recognition were identified and together they formed ‘the magic of programmes’ which helped youth to meet their various needs and goals.

Both critical realism and framework analysis were relevant to the study topic and aims. As noted above, the study sat within PYD theory, which considers developmental processes in youth, approaches that support development, and interventions that use these approaches (Hamilton et al., 2004). PYD asserts that all young people can develop positively but that there are numerous possible paths to this development (Lerner, 2005). As an approach, PYD aims to support positive development for youth, whilst PYD programmes provide opportunities for young people to develop positively.

There is a tension in PYD between the many possible individual developmental pathways young people can take, and the use of a common set of broad approaches that support such development. A critical realist philosophy provides a good fit for these tensions as looking for tendencies and generative mechanisms supports both general and contextually specific explanations for phenomena. The structured approach offered by framework analysis means all data are useful when exploring similarities and differences in participant perception and identifying broad themes and contextual nuances in the data. Critical realism’s interest in the interaction between structure and agency is also relevant to PYD theory, which asserts that young people are active agents in their own development and are both constrained by structures and are able to shape them (Lerner, 2004; Lerner & Chase, 2019).

Familiarisation

Ten transcripts were familiarised by the lead author including a mixture of individual interviews and focus groups. Familiarisation was done manually on transcripts and via spreadsheet. Emergent coding (Ayres, 2008) was used, which identified descriptive labels for data excerpts. Inductive codes described the outcomes identified by participants and aspects of behaviour and emotion. Examples of early emergent codes included confidence, skills, belonging, freedom and overcoming barriers.

During familiarisation researcher reflections and points of interest were noted and discussed with supervisors. For example, programmes were perceived by participants as safe spaces to explore identity and young people navigated themselves to programmes to meet individual needs. Codes such as ‘I can be me here’ captured these programme aspects. These observations indicated early retroductive thinking about the conditions needed for phenomena to occur (Danermark et al., 2019) and foreshadowed potential generative mechanisms. They were noted for later analysis. Table 1 Shows an excerpt of familiarisation coding from one focus group.

Table 1.

Familiarisation Coding Extract, Focus Group 7.

Excerpt	Code 1	Code 2	Code 3
P7FG: And when you have those skills you’re more comfortable as [name] said, but you also have more confidence and I think that makes it easier for us to talk to other people about it, talk to other youth about it, because when you’re confident that confidence will portray onto other students or other staff members or other people in general. It’s all about that confidence we learn in here and take into other parts of your life.	Skills lead to confidence	Social confidence to approach people in group and outside	Skills help and influence others
JM: Confidence came up in both things. What is confidence? How would you describe it? It’s a real tricky question.
P7FG: I Think confidence is when you’re able to step out of your comfort zone and engage with others you don’t normally talk to, just being comfortable to share your personal experiences with people who you don’t know that well.	Confidence	Social confidence	Try something new/Push comfort zones
JM: Cool. Anything else?
P7FG: I Think it’s being comfortable in who you are as a person, and what you believe in, and knowing that no matter what people say about you or what happens to you, that you’re always gonna stand for what you said.	Confidence	Sense of self/Identity	Values -based living

Developing the Initial Framework

Familiarisation generated 266 unique codes which were grouped into broader themes, with sub-themes identified to develop an initial framework. Similar codes were grouped or collapsed, whilst codes that only appeared once were re-reviewed to see if they could be coded elsewhere or represented important differences. Nine outcomes and three programme processes which supported outcomes were described in the initial framework and these broad themes remained throughout the analysis. Table 2 shows a selection of the codes and sub-codes in the initial framework, showing how ideas were grouped. These were entered into NVivo and used for indexing.

Table 2.

A Selection of Codes From the Initial Framework.

Group	Code	Subcode
Outcomes	Giving back	Making a difference
		Contribution
	Agency	Independence
		Living life/Getting on with it
		Transition/Exit
		Overcoming barriers
		Overcoming stigma
		Changing mindsets
		Taken seriously
	Confidence
	Future focus	Preparing for future
		Hope
	Positive affect	Fun
		Excitement
		Happiness
		Managing feelings
	Connection	Peer connection/Friendship
		Belonging
		Support
		Connections with community
		Connections with organisation
		Relationships with adults/leaders
		Family/whānau connection
		Ownership
	Positive identity	Know who I am
		Pursue passions
		I can be me
		Cultural expression
		Failure/vulnerability ok
		Self-worth
		Develop values and attitudes

Indexing

NVivo was used to index data. Familiarised data were indexed first, by uploading transcripts to NVivo and allocating codes to excerpts. Familiarisation coding was revisited if excerpts proved difficult to index, as recommended by Kiernan and Hill (2018). Through this back and forwards process, the framework was further modified, with new codes added and some removed to ensure the fit of the framework to the data. The framework was then applied to remaining data and no data were excluded.

As additional data were indexed, no new codes were added, but some were further refined. Whilst inevitably researcher driven, these first three stages of framework analysis were largely inductive and guided by what was present in the data. Table 3 shows some examples of codes added during indexing familiarised data, whilst Table 4 shows an example of some of the refinements made as additional data were indexed. These changes highlight some of the ways data were grouped and understood. Notes were kept of all of these changes and the development of the framework was tracked to show the researcher’s thinking process. Framework development and subsequent changes were discussed with the lead author’s supervisors. Documentation and discussion of these analysis processes supported rigour by helping make the interpretation of data more visible (Kiernan & Hill, 2018).

Table 3.

Example of Codes Added During Indexing.

Node	Child Node	Description and Rationale
Unintended outcomes		This new node was added to reflect challenges related to some outcomes, such as anxiety about leadership.
Skills	Personal skills	The new child node of personal skills included resilience/adaptability, where this related to skills about navigating challenges or change. Where resilience related to attitudes related to drive and overcoming challenges this was coded under attitudes and values development. Personal skills also included managing feelings which was initially under positive affect. Managing feelings was more skill related, whilst other codes under positive affect were about feelings. Personal skills also included the ability to set goals, which was different from achieving goals and to manage oneself and expected life tasks.
Achievement	Completion of activities	This new child node included completion of activities or goal achievement, such as gaining a badge, or running and event.
	Achievement	This child node included post-programme and general achievements such as graduating the programme or gaining employment. This data had previously sat under achievement broadly.

Table 4.

Example of Codes Modified From Indexing.

Node	Child Node Merged	Description and Rationale
Agency	Living life/getting on with it	This child-node was particularly important for one group and was re-coded under overcoming barriers because it was about living life in spite of barriers and did not appear elsewhere. The experiences of this group and the particular value they placed on agency was noted.
	Changing mindsetsOvercoming barriersOvercoming stigma	Throughout indexing these codes were refined into two child nodes. The first, challenging stereotypes, was related to overcoming stereotypes and changing views of self or others. The second having a voice, related to speaking up and being empowered to play a role in the community.
	Taken seriously	As indexing occurred, the child node of taken seriously was renamed as having something to offer, which was about being valuable to others and society through actions.
Positive affect	Pride	The child node of pride was moved from achievement to positive affect. Pride was not just related to achievement but may be about pride in other areas such as giving back.

Charting

Data were charted first by theme (outcomes) and then by case (programme). During thematic charting, each code in NVivo was carefully reviewed and due to the size of the dataset, data were summarised manually on a spreadsheet. This process facilitated cross referencing of each item, checking back to raw data, and moving items if necessary. The manual chart was further modified by the addition of notes about links to other codes and researcher reflections. Table 5 shows an excerpt from these early charts from the theme of connection, with researcher notes and reflections added.

Table 5.

Chart Excerpt From the Theme of Connection.

Interview/Group Number	Instance number	Quotation Summary	Links to Other Nodes	Reflective Ideas
P7FG:	3	Friendship
	4	Friendship
	5	Connect with people wouldn’t have known otherwise. Nice to have support and know they’ve got your back - people who care outside of family. Support in group.	Accepting atmosphere (supportive group environment)	People different from me, support. For this group connections family like. Also identified P10, P3, P14
	6	Comfortable with each other makes it safe to be vulnerable.	Accepting atmosphere (OK to be myself)	Comfortable, safe connections gained but also the nature of these shapes how the group is
	7	Grow close - makes a safe environment to talk about anything.	Accepting atmosphere (OK to be myself)
	8	Connecting important - learn skills and all need to be on the same page.	Accepting atmosphere (shared goals and motivation)	Connection supports skill development
	9	Everyone accepting non-judgemental- ‘roast each other sometimes’ but know how to deal with each other and different moods.	Accepting atmosphere (supportive group environment)

The manual charting process helped to make the analysis and the interpretation and shaping of meaning more explicit (Kiernan & Hill, 2018). In critical realist studies, the additional modification of adding researcher reflective notes to charting can help to identify both abductive and retroductive thinking, whilst linking inferencing to empirical observations. These notes offer insight into the mapping and interpretation phase, by showing where links are made. The modification also serves to show how the charting process was influenced by researcher perceptions. For example, in Table 5, the notes identify how this group and others linked programmes to being ‘family like’, highlighting the researcher’s interest in this idea.

Charting supported deeper understanding of data and enabled further refinements to the framework. Charts also noted where particular focus groups or interviews did not discuss a theme. This process identified patterns or negative cases, to support later retroductive inferencing.

Charting helped to identify overlaps between themes. Whilst in some instances this resulted in changes to coding or the framework, in many cases they indicated contingent conditions, where the presence of one outcome could support others. For example, in the chart sample in Table 5, participants commented how being comfortable with each other made it safe to be vulnerable. This was coded to both the outcome of connection and the programme process of an accepting atmosphere, because the comfort in the group supported deeper connection. Noting these overlaps could help point to possible generative mechanisms, in this instance, the way that recognition from others could support deep connections with peers.

The second round of charting further summarised all data onto one master chart, organised by programme with themes identified. In this way, whole transcripts could be reviewed and compared. Researcher notes and reflections were also included in this chart. Re-charting the data in this way helped to confirm the presence of the nine outcomes and three programme processes in most programmes, with notes identifying key themes and what was seen as significant or unique in each context. Table 6 shows an excerpt from the master chart:

Table 6.

Master Chart Excerpt.

Skills	Achievement	Giving Back	Agency	Confidence	Researcher Reflections
Task, Leadership, People, Personal	Achievement, Completion, Celebration	Helping and contributing, Benefits	Independence, Having a voice, Challenging stereotypes, Something to offer	Sources, Expressions, Impacts
Task: Education skills, financial. Leadership: Leadership, set examples and be a person others can rely on. Show leadership, tackle any situation and get things done. People: Outgoing, communication skills, empathy relating to others, teamwork, social and approachable. Social skills used at home and in community. Contribution opportunities build skills to work with others, useful for jobs and school. Community interaction creates communication skills and teamwork skills. Personal: Maturity, smarter life choices, more productive, life skills.	Achievement: Academic achievement. Get good at things from repetition.	Helping and contributing: Help the community. Volunteer and work experience. Example of making a mural for the community. Keeping others safe. Have a role in family, make good choices, do things without being asked and good role model to siblings. Benefits: Feel blessed, successful, and competent from helping.	Independence: Able to think for self not relying on others. Having a voice: Use their voice, be heard, and right to speak freely. Able to share ideas and be comfortable and empowered. Feel part of the community. Having something to offer: Feels good to do something for the community. Can influence friends and family - 'positive ripple effects’.	Sources: Programme creates confidence. Knowing who you are and what you want leads to confidence. Expressions: Ready to take on the world. Confident in front of a group, not doubting self, self-belief. Assertive tone of voice - how you hold yourself around others. Not feeling thrown in new situations. Open minded. Impacts: Be a role model others can depend on them.	The idea of the group as a safe space to learn skills and gain confidence seemed important. The group seemed very respectful of different personalities and had learned how to work well together. They were aware of their shared goals and each person could be themselves. This was a key part of the programme. A couple of participants referred to challenges outside or socially and the group provided an important place to learn social skills and a respite from challenges outside.

The charting process allows for the framework to be reduced by combining options, or expanded as larger patterns are identified (Kiernan & Hill, 2018). In this case, when considering how the outcomes fitted together, three larger themes emerged from the charting process. These groupings included Behavioural Outcomes (outcomes related to observable behaviours, such as skills and giving back); Internal Outcomes (related to young people’s internal experiences, such as developing a future focus) and Relational Outcomes (outcomes which had significant overlap with young people’s relational experiences of programmes). The identification of these larger themes was an intuitive and researcher driven process which helped make sense of the data as a whole in preparation for writing. In this way, the development of super-ordinate themes was both part of charting and of mapping and interpretation (Kiernan & Hill, 2018; Parkinson et al., 2016).

Mapping and Interpretation and Generative Mechanisms

The relationships between themes and ideas identified during charting, were tested in the data by reviewing the notes made in the master chart and running queries in NVivo. This process helped to provide concrete evidence of the abstract interpretations made by the researcher. For example, the initial observation that programmes were safe spaces to explore identity was tested by reviewing participant comments and coding overlaps between the programme process of an accepting atmosphere and the outcome of positive identity.

Themes and ideas from the data were then reviewed while considering theories, a process that occurred during the writing of the thesis. These abductive processes included both the original theoretical premise (PYD) and moved beyond these which is consistent with a critical realist approach (Meyer & Lunnay, 2013). Abductive reasoning helped to anchor the themes identified in charting and supported them as valid descriptions of the data. Widening of the theoretical lens could support the identification of generative mechanisms and consider data beyond the empirical level (Danermark, 2002), whilst charts with modified notes helped to signpost where additional theories could provide explanation. For example, the theme of positive identity and young people’s relational experiences of self within programmes was more fully understood by considering Māori and Pacific world views, where identity is seen as firmly nested within contexts and relationships, spanning both time and place (Macfarlane, 2004; Mila-Schaaf, 2006; Rameka, 2018; Tamasese, 2002; Toso, 2011; Tuagalu, 2008).

Broadening the theoretical lens highlights the role of theory as a transitive vehicle to provide explanation and supports critical realism’s view that knowledge cannot fully represent reality (Longhofer & Floersch, 2012). It also provides a more culturally safe place for researchers to engage with and interpret data from diverse communities by recognising that the theoretical descriptions provided may be one of many possible interpretations.

Retroductive inferencing also occurred during mapping and interpretation, supported by earlier processes in framework analysis. Whilst writing, questions and reflective notes on the data were revisited, such as the roles young people played in navigating to programmes and the experiences of being able to explore identity. These questions, were investigated further through revisiting charts and original transcripts, again reflecting movement between the abstract and the concrete.

For example, as noted in the thesis, fieldnotes from one focus group conducted early in the study identified a strong sense that some of the young people had joined the programme to fill a need or a gap, both personally and in relation to their role in society. Evidence of this sense of gap and the ways programmes could fill perceived gaps was sought and found in other programmes. Programmes could provide necessary skills for the adult world, or vehicles for young people to prove themselves and counter stereotypes about youth, or particular groups. For example, some young people identified navigating to programmes to gain skills that would give them an edge in the workforce, some wanted to develop social skills in a friendly environment and some sought programmes as avenues to shape decision making and show that young people could be valuable. Young people also brought their values into programmes, such as a desire to give back to others. These motivations kept young people engaged in programmes and they used their goals and values to help ensure that programmes met their needs.

These clues about young people’s role in navigating to and shaping programmes led to a hypothesis that youth agency may act as a generative mechanism that supports outcomes. As Danermark and colleagues (2019) suggested, this retroductive inferencing, where underlying conditions for phenomena were considered, was a somewhat intuitive process. The idea of youth agency as a generative mechanism was supported by other clues in the data, such as the active role young people played in maintaining a group environment that supported their own and other’s outcome development. Charts helped to identify these instances and linkages between ideas. Theories within PYD also supported agency as a generative mechanism. Other studies showed how young people intentionally sought out experiences and opportunities that supported their development (Benson et al., 2007; Chauveron et al., 2015) and selected goals, used resources and adapted to changes in order to meet their goals and achieve positive outcomes (Gestsdóttir & Lerner, 2008). These theories were applied back to the data to help explain how young people used programmes to achieve their individual goals whilst participating in a shared group experience.

This example shows how the back and forwards process from concrete, to abstract and back again in critical realist analysis occurred. Data (concrete) pointed to clues that led to a hypothesised mechanism (abstract) that was then re-tested in the data and with theory to identify how the mechanism occurred (concrete) (Fletcher, 2017).

The other proposed generative mechanism of recognition was developed through a similar process. Clues from the data about the importance of being seen and valued by others, as an individual, as a contributor, and culturally were hypothesised as a mechanism, labelled recognition. This hypothesis was tested in the data and reviewed through theory, drawing upon PYD theory, (the original theoretical premise) and new theory via Honneth’s (1995) Recognition Theory and Māori and Pacific cultural frameworks. These ideas were applied back to the data to help explain the mechanism. In this case, recognition of the skills, culture, and contribution of young people, helped them to see themselves and their achievements and value more clearly.

The way these mechanisms interacted was described as ‘the magic of programmes’. Here young people used agency in their navigation to and use of programmes to attain goals and this was combined with recognition from programmes, leaders and peers of youth goals, values, capabilities, and identities to support outcomes and positive development. This ‘magic’ helped to explain how programmes could support both broad and contextually specific outcomes, thus also providing one way to understand the tensions in PYD theory between specificity and generalisability (Tolan, 2014).

Inferencing processes and identified mechanisms were discussed with supervisors and cultural advisors to the research. These discussions helped to enhance understanding of the mechanisms, for example, seeing where the concept of recognition could fit within Māori and Pacific world views. The findings were also acknowledged as based in the understandings of the lead author as a Pākehā (New Zealand European) researcher in a diverse setting, reflecting the fallibility of explanations in critical realist research (Seal, 2016).

At a practice level, these mechanisms can help programmes consider the ways they encourage agency and provide recognition, and how mechanisms may operate differently in different settings or for different groups of youth.

On Combining Framework Analysis and Critical Realism

The identification and description of generative mechanisms was a natural progression of the mapping and interpretation phase and maintained a critical realist focus on looking beyond what is empirically observed. In reflecting on the combination of these two approaches, framework analysis supported the use of a critical realist lens in several ways. Through the intensive five step process, the data became very well-known which helped to surface the questions and clues that led to generative mechanisms. Tracking the framework development throughout helped to show the thinking processes behind these clues, making the analysis process open to inspection by others (Kiernan & Hill, 2018). Charts helped to explore ideas further, by quickly referencing back to data, whilst modifications to the charting process, through researcher notes, again helped to elucidate researcher thinking and track the development of ideas. These structured steps enhanced transparency and efficiency in the back and forwards movements between concrete and abstract thinking.

Such an explicitly structured and detailed process can also support clarification of the different modes of inferencing in critical realist studies and whether this is data, researcher or theory driven. In this instance the first three stages of familiarisation, framework development and indexing were largely driven by the data and used inductive inferencing, where observations are used to develop generalisations (Danermark, 2002). These inductive processes generated the codes and the framework. These codes helped to describe some of the tendencies in the data, in this case the perception of outcomes. Charting, with the addition of researcher notes helped to refine inductive thinking and supported retroductive inferencing with ideas about what processes may be behind the data. This was a researcher driven process, and charts could help to show this thinking clearly. Mapping and interpretation included two distinct stages of inferencing: the first abductive in considering data through theory, and the second, a continuation of the retroductive process begun in earlier stages, leading to the hypothesis and testing of generative mechanisms.

In the study described in this article, the combination of framework analysis and critical realism helped to generate a deeper understanding of programme outcome perception and the ways in which programmes can use mechanisms of agency and recognition as levers to support positive outcomes for young people. More widely, the study provides a concrete example of how the two approaches can be used together to support qualitative data analysis.

Conclusion

This article has shown that framework analysis in critical realist research enables the research philosophy to flow throughout data analysis and provides a clearly documented way to track the identification of generative mechanisms. Framework analysis can support easy transition between concrete and abstract thinking through closeness to the raw data and the transparency provided by the framework. The modification of adding researcher reflections to the charting process tracks the development of generative mechanisms and intuitive, retroductive thinking processes. Combined, the two approaches provide a clear structure to use data and theory to facilitate thinking and may be useful for other researchers looking to use critical realism in qualitative projects.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the PhD Leaders Programme, Unitec Institute of Technology, New Zealand; Massey University, Graduate Research Fund.

ORCID iDs

Jayne Mercier

Jackie Sanders

References

Ayres

(2008). Thematic coding and analysis. In Given

L. M.

(Ed.), The Sage encyclopaedia of qualitative research methods (p. 811). Sage. https://dx.doi.org/10.4135/9781412963909

Benson

P. L.

Scales

P. C.

Hamilton

S. F.

Sesma

(2007). Positive youth development: Theory, research, and applications. In Damon

Lerner

R. M.

Lerner

R. M.

(Eds.), Handbook of child psychology (pp. 894–941). https://doi.org/10.1002/9780470147658.chpsy0116

Bhaskar

(1975). A realist theory of science. Routledge.

Chauveron

L. M.

Linver

M. R.

Urban

J. B.

(2015). Intentional self-regulation and positive youth development: Implications for youth development programs. Journal of Youth Development, 10(3), 89–101. https://doi.org/10.5195/jyd.2015.10

Clark

A. M.

(2008). Critical realism. In Given

L. M.

(Ed.), The Sage encyclopaedia of qualitative research methods (pp. 168–170). Sage. https://dx.doi.org/10.4135/9781412963909

Danermark

(2002). Interdisciplinary research and critical realism: The example of disability research. Alethia, 5(1), 56–64. https://doi.org/10.1558/aleth.v5i1.56

Danermark

Karlsson

J. C.

Ekström

(2019). Explaining society: Critical realism in the social sciences (2nd ed.). Routledge.

Deforge

Shaw

(2012). Back- and fore-grounding ontology: Exploring the linkages between critical realism, pragmatism, and methodologies in health and rehabilitation sciences. Nursing Inquiry, 19(1), 83–95. https://doi.org/10.1111/j.1440-1800.2011.00550.x

Eastwood

J. G.

Jalaludin

B. B.

Kemp

L. A.

(2014). Realist explanatory theory building method for social epidemiology: A protocol for a mixed method multilevel study of neighbourhood context and postnatal depression. SpringerPlus, 3(1), 12. https://doi.org/10.1186/2193-1801-3-12

10.

Fletcher

A. J.

(2017). Applying critical realism in qualitative research: Methodology meets method. International Journal of Social Research Methodology, 20(2), 181–194. https://doi.org/10.1080/13645579.2016.1144401

11.

Furber

(2010). Framework analysis: A method for analysing qualitative data. African Journal of Midwifery and Women's Health, 4(2), 97–100. https://doi.org/10.12968/ajmw.2010.4.2.47612

12.

Gale

N. K.

Heath

Cameron

Rashid

Redwood

(2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 117. https://doi.org/10.1186/1471-2288-13-117

13.

Gestsdóttir

Lerner

R. M.

(2008). Positive development in adolescence: The development and role of intentional self-regulation. Human Development, 51(3), 202–224. https://doi.org/10.1159/000135757

14.

Goldsmith

L. J.

(2021). Using framework analysis in applied qualitative research. Qualitative Report, 26(6), 2061–2076. https://doi.org/10.46743/2160-3715/2021.5011

15.

Groff

(2004). Critical realism, post positivism and the possibility of knowledge (1st ed.). Taylor and Francis. https://doi.org/10.4324/9780203417270

16.

Hamilton

S. F.

Hamilton

M. A.

Pittman

(2004). Principles for youth development. In Hamilton

S. F.

Hamilton

M. A.

(Eds.), The youth development handbook: Coming of age in American communities (pp. 3–22). Sage. https://doi.org/10.4135/9781452232560.n1

17.

Honneth

(1995). The struggle for recognition: The moral grammar of social conflicts. Polity Press.

18.

Kiernan

M. D.

Hill

(2018). Framework analysis: A whole paradigm approach. Qualitative Research Journal, 18(3), 248–261. https://doi.org/10.1108/QRJ-D-17-00008

19.

Lerner

R. M.

(2004). Liberty: Thriving and civic engagement among America’s youth. Sage. https://doi.org/10.4135/9781452233581

20.

Lerner

R. M.

(2005). Promoting positive youth development: Theoretical and empirical basis [White paper]. National Academy of Sciences.

21.

Lerner

R. M.

Chase

P. A.

(2019). Enhancing theory and methodology in the international study of positive youth development: A commentary. Child and Youth Care Forum, 48(2), 269–277. https://doi.org/10.1007/s10566-018-9485-7

22.

Levers

M. J. D.

(2013). Philosophical paradigms, grounded theory, and perspectives on emergence. Sage Open, 3(4), 1–6. https://doi.org/10.1177/2158244013517243

23.

Longhofer

Floersch

(2012). The coming crisis in social work: Some thoughts on social work and science. Research on Social Work Practice, 22(5), 499–519. https://doi.org/10.1177/1049731512445509

24.

Macfarlane

(2004). The value of Māori ecologies in the study of human development. In Drewery

Bird

(Eds.), Human development in Aotearoa: A journey through life (2nd ed., pp. 38–42). McGraw-Hill.

25.

Martel

Shepherd

Goodyear-Smith

(2022). He awa whiria—a “braided river”: An indigenous Māori approach to mixed methods research. Journal of Mixed Methods Research, 16(1), 17–33. https://doi.org/10.1177/1558689820984028

26.

Meyer

S. B.

Lunnay

(2013). The application of abductive and retroductive inference for the design and analysis of theory-driven sociological research. Sociological Research Online, 18(1), 86–96. https://doi.org/10.5153/sro.2819

27.

Mila-Schaaf

(2006). Vā-centred social work: Possibilities for a Pacific approach to social work practice. Social Work Review, 18(1), 8–13.

28.

Oliver

(2012). Critical realist grounded theory: A new approach for social work research. British Journal of Social Work, 42(2), 371–387. https://doi.org/10.1093/bjsw/bcr064

29.

Parkinson

Eatough

Holmes

Stapley

Midgley

(2016). Framework analysis: A worked example of a study exploring young people’s experiences of depression. Qualitative Research in Psychology, 13(2), 109–129. https://doi.org/10.1080/14780887.2015.1119228

30.

Raduescu

Vessey

(2009). Methodology in critical realist research: The mediating role of domain specific theory. AMCIS 2009 Proceedings, San Francisco, CA, USA, August 6-9, 2009. https://aisel.aisnet.org/amcis2009/433

31.

Rameka

(2018). A Māori perspective of being and belonging. Contemporary Issues in Early Childhood, 19(4), 367–378. https://doi.org/10.1177/1463949118808099

32.

Ritchie

Spencer

(1994). Qualitative data analysis for applied policy research. In Bryman

Burgess

(Eds.), Analyzing qualitative data (pp. 173–194). Taylor and Francis. https://doi.org/10.4324/9780203413081

33.

Roberts

J. M.

(2014). Critical realism, dialectics, and qualitative research methods. Journal for the Theory of Social Behaviour, 44(1), 1–23. https://doi.org/10.1111/jtsb.12056

34.

Scott

(2005). Critical realism and empirical research methods in education. Journal of Philosophy of Education, 39(4), 633–646. https://doi.org/10.1111/j.1467-9752.2005.00460.x

35.

Seal

(2016). Critical realism’s potential contribution to critical pedagogy and youth and community work: Human nature, agency and praxis revisited. Journal of Critical Realism, 15(3), 263–276. https://doi.org/10.1080/14767430.2016.1169721

36.

Stutchbury

(2022). Critical realism: An explanatory framework for small-scale qualitative studies or an ‘unhelpful edifice. International Journal of Research and Method in Education, 45(2), 113–128. https://doi.org/10.1080/1743727X.2021.1966623

37.

Tamasese

K. I.

(2002). Honouring Samoan ways and understandings: Towards culturally appropriate mental health services. International Journal of Narrative Therapy and Community Work, 2002(2), 64–71. https://dulwichcentre.com.au/wp-content/uploads/2016/12/11-kiwi.pdf

38.

Tolan

(2014). Future directions for positive development intervention research. Journal of Clinical Child and Adolescent Psychology, 43(4), 686–694. https://doi.org/10.1080/15374416.2014.936604

39.

Toso

(2011). Reconceptualising spirituality as philosophy of practice for Pasifika early childhood education in New Zealand: A Samoan perspective. Pacific-Asian Education, 23(2), 129–138. https://natlib.govt.nz/records/32309545

40.

Tuagalu

(2008). Heuristics of the vā. AlterNative: An International Journal of Indigenous Peoples, 4(1), 107–126. https://doi.org/10.1177/117718010800400110

41.

Vincent

O’Mahoney

(2019). Critical realism and qualitative research: An introductory overview. In Cassel

Cunliffe

A. L.

Grandy

(Eds.), The Sage handbook of qualitative business and management research methods: History and traditions (pp. 201–216). Sage. https://dx.doi.org/10.4135/9781526430212

Fine Companions: Critical Realism and Framework Analysis

Abstract

Keywords

Introduction

Critical Realism

Critical Realist Research

Framework Analysis

Familiarisation

Framework Development

Indexing

Charting

Mapping and Interpretation

Alignment Between Framework Analysis and Critical Realism

The Process in Action

Familiarisation

Developing the Initial Framework

Indexing

Charting

Mapping and Interpretation and Generative Mechanisms

On Combining Framework Analysis and Critical Realism

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

References