Establishing Trustworthiness in Health Care Process Modelling: A Practical Guide to Quality Enhancement in Studies Using the Functional Resonance Analysis Method

Abstract

The Functional Resonance Analysis Method (FRAM) is a novel healthcare research methodology that has increasingly been applied in the healthcare domain. The method has an ability to map and model everyday healthcare activities and their interdependencies, as well as, demonstrate how variability can emerge and impact system outcomes. To build a FRAM model, researchers gather data from key stakeholders, such as health care workers and patients and their families using qualitative data collection methods. An important consideration for researchers using the FRAM is how they will establish trustworthiness in their study findings given the data used to build and analyze a FRAM model can be subjective. In the spirit of advancing the quality of qualitative research, the aim of this paper is to provide practical guidance to researchers on how to employ quality enhancement criteria and strategies in their qualitative research efforts so that the resulting FRAM models and insights afforded by them are trustworthy.

Keywords

the functional resonance analysis method functional resonance analysis method healthcare system modelling trustworthiness quality enhancement

The Functional Resonance Analysis Method (FRAM) is a novel research methodology that has increasingly been applied in the health care domain due to its ability to map and model everyday healthcare activities, identify how individual activities are connected and interdependent, and demonstrate the variability that can emerge and impact system outcomes. A strength of the FRAM is that the essential and interacting activities, as well as the complexity of a healthcare process, are visually represented in a non-linear model. Often this complexity is challenging to appreciate using traditional or linear methods of analysis (McGill et al., 2021). Using the FRAM to gain an in depth understanding of how work is accomplished in the context of a complex health care process has been the focus of several studies in recent years (McGill et al., 2022). With its wide scope of application, the FRAM has been used to examine processes such as hospital discharge in older adults (Buikstra et al., 2020; Laugaland et al., 2014; O’Hara et al., 2020; Salehi et al., 2021), safety in medication administration (Oduyale et al., 2020; Schutijser et al., 2019), and how care guidelines can differ from clinical practice (Clay-Williams et al., 2015; van Dijk et al., 2022). To build a FRAM model, researchers rely on contextual data gathered from health care workers, patients, and their families using qualitative data collection methods, such as interviews, focus groups, and observations (Hollnagel & Slater, 2022). An important consideration for researchers using the FRAM is how they will establish trustworthiness in their study findings given the data used to build and analyze a FRAM model can be subjective. The aim of this paper is to provide practical guidance to researchers on how to employ quality enhancement criteria and strategies in their qualitative research efforts so that the resulting FRAM models and insights afforded by them are trustworthy. This paper will begin by providing an overview of the FRAM and how qualitative data collection is essential to conducting a FRAM analysis. Lincoln and Guba’s trustworthiness criteria will then be presented along with several quality enhancement strategies researchers using the FRAM can apply (Guba & Lincoln, 1994; Lincoln & Guba, 1985).

Additionally, illustrative examples and tables depicting how these quality enhancement strategies have been used to establish trustworthiness criteria in select healthcare studies using the FRAM will be presented.

The FRAM

The following sections will provide rationale for the use of the FRAM in healthcare research, introduce principles of the method, define key terminology, and list the steps of the FRAM. Providing a full methodological review of the FRAM is beyond the scope of this paper. Before using the FRAM to examine and analyze a complex healthcare process, researchers need to familiarize themselves with the principles of the method, the rationale for its use, and its stepwise approach. This information can be found in greater depth in several fundamental FRAM publications (Hollnagel, 2012; Hollnagel et al., 2014; Hollnagel & Slater, 2022). There is also a website https://functionalresonance.com/practice.html created by the method’s founder, dedicated to providing guidance on using the method (“Functional Resonance,” n.d). Functional resonance analysis method modelling software used to build and edit FRAM models can also be accessed on this website, with literature provided that can assist researchers in becoming proficient in its use (Hollnagel & Hill, 2020). Literature reviews on the FRAM have also been conducted and provide an overview the method’s development, methodological aspects, and domains of application (Patriarca et al., 2020; Salehi et al., 2020). A scoping review of the literature on the FRAM as a healthcare research methodology by McGill et al. (2022) is an additional publication that provides healthcare researchers with an overview of how the method has been applied specifically in the healthcare domain.

Purpose of the FRAM

Hollnagel and Slater (2022) state “the purpose of the FRAM is to analyse activities – how something has happened, how something happens, or how something could happen – in order to produce a representation of the activity in a systematic manner using a well-defined format” (p. 2).

A FRAM model is a representation of the individual and interdependent activities that are essential in a process to produce an outcome. Healthcare has been described as a complex sociotechnical system that is dependent on the interactions between humans, technology, and organizations delivered in and across multiple sectors that are often loosely connected (Carayon, et al., 2011). Traditional research paradigms and methodologies fail to gain an understanding of the complexities of everyday healthcare work. Recommendations are often characterized by fragmentation and standardization with knowledge dispersed over different professional groups and research communities (Aase & Waring, 2020). Acknowledging and confronting complexity in the health care system requires a clear description and level of understanding of the specifications and activities of health care delivery under dynamic conditions (Hollnagel, 2012). The FRAM is an approach that has been shown to achieve this level of understanding and with this knowledge can assist in identifying potential problem areas and areas of success within a process or system (Smith et al., 2020).

Principles of the FRAM

There are four underlying principles of the FRAM that can explain the outcome of a process or how something happens. These four principles are outlined below as explained by Hollnagel and Slater (2022).

• The principle of equivalence of successes and failures assumes that things can go right and go wrong in much the same way, different consequences don’t necessarily require different types of explanations. Outcomes, both acceptable and unacceptable, occur because individuals, groups, and organizations have an ability to adjust what they do in expected and unexpected conditions.

• The principle of approximate adjustments assumes that work is continuously adjusted in relation to the existing conditions (time, information, conflicts, interruptions, resources, tools). Adjustments are not precise, they are approximate and are made by individuals, groups, and organizations. This principle explains why most often things go right but can also explain why occasionally things can go wrong.

• The principle of emergence assumes that not all outcomes or results can be explained by an identifiable cause. The variability from multiple functions can combine in unexpected ways (producing a non-linear effect) and lead to outcomes that are disproportionally large. These outcomes are emergent rather than resultant.

• The principle of resonance assumes that in some situations multiple approximate adjustments in work can coincide and mutually influence one another in unintended ways resulting in the emergence of noticeable variability. This variability can lead to outcomes (negative and positive) that may not have been anticipated. This is known as functional resonance.

FRAM Terminology

Functions and Aspects

The FRAM refers to activities in a process as “functions” (Hollnagel & Slater, 2022). Functions are carried out continuously in a healthcare process, and can be human, organizational, or technological (Ross et al., 2018). The functions of a healthcare process can be described or characterized by six aspects: Input (I), Output (O), Resources (R), Time (T), Control (C), and Preconditions (P), these are best explained below by Clay-Williams et al. (2015).

1. The Input is what the function acts on or changes (what is used to start the function).

2. The Output is what emerges from the function (an outcome or state change).

3. A Precondition is a condition that must be satisfied for a function to happen.

4. The Resources are materials or people needed to execute a function.

5. The Control is how the function is regulated or controlled (guidelines, protocols).

6. The Time refers to any temporal requirements of the function.

The data gathered to identify functions and then describe them in terms of their aspects should come from information-rich sources who are stakeholders in the process, such as healthcare professionals, patients, and their families. The FRAM uses qualitative data collection methods, such as interviews and focus groups, to gather accounts from stakeholders who can accurately describe the activities in a process and how the activities are connected and mutually dependent. Hollnagel et al. (2014) found document review of clinical guidelines, protocols, and procedures can be helpful in preparation for interviews as it provides the researcher with insight into the practice setting and how work is intended to be carried out. Researchers have also used observations to gather data necessary to build a FRAM model. Observing work as it is performed in context can potentially contribute rich data regarding the nuances of how a health care process or system operates (Weston et al., 2022). In A FRAM model, a function is diagramed visually as a hexagon with aspects branching from the corners of the hexagon. Figure 1 is an example of a function (adapted from Hollnagel et al., 2014).

Figure 1.

Functional Resonance Analysis Method (FRAM) function hexagon.

Couplings

When the functions in a process are identified and described in terms of their aspects, a graphical FRAM model can be built using the FRAM Model Visualizer (FMV) software programming (Hollnagel & Hill, 2020). One of the objectives of the FRAM is to identify how functions are connected/interdependent, also known as “coupling” of functions (Hollnagel, 2012). Functions are coupled through mutually shared aspects and can impact and one another in a variety of ways. For instance, when there is variability in how a function is carried out, functions can absorb any variability by mutual dampening each other, they can also amplify the variability by mutually reinforcing each other (Salehi et al., 2021). Hollnagel (2012) explains the variability and adjustments of functions is an essential and normal part of work and can lead to both successful and unsuccessful outcomes. Using the FMV software, researchers can input functions and aspects to create a model depicting all the functions of a healthcare process and how they are coupled (Sujan et al., 2022).

Figure 2 depicts three functions involved in the fracture care process in an Emergency Department. The figure shows how functions are coupled through mutually shared aspects. The function <Xray affected limb> has an Output “Interpret Xray” which is also the Input or what starts the function <Diagnose fracture>. The Output “Interpret Xray” is also the Precondition of the function <Apply cast>.

Figure 2.

Process of fracture care in an emergency department: FRAM function hexagons with mutually shared aspects.

Potential Variability

Potential variability refers to what could happen under different conditions in a health care process (Hollnagel et al., 2014). The FRAM is concerned with the variability that can potentially occur in the outputs of functions within a process rather than the variability of the function itself. Hollnagel et al., (2014) describes three ways the output of functions can vary:

1. The uniqueness of the function (endogenous or internal variability)

2. The conditions in which the function is performed (exogenous or external variability)

3. The variability from the output of functions that occur earlier in the process (functional upstream-downstream coupling)

When a FRAM model is built, the model is a representation of all “the potential or possible relationships or dependencies within a process without referring to a specific situation” (Hollnagel et al., 2014, p. 32). For instance, if the FRAM was used to examine the process of fracture care in an emergency department, data would be collected to identify functions of the fracture care process and describe them in terms of their aspects. Stakeholders in the process would additionally provide data on how the outputs of functions in the process of fracture care vary. This variability is usually described in two ways - time, and precision. Hollnagel et al. (2014) explains, the variability in the output of a function related to time can be described as too early, on time, or too late and for precision the variability in the output of the function can be described as imprecise, acceptable, and precise.

Actual Variability

Actual variability in the process of fracture care can only be determined by monitoring a specific situation or scenario within a process. For instance, a patient presents to the emergency department with a fractured arm. The scenario in which they navigate the fracture care process can be monitored to identify what functions were activated and what mutual dependencies or resonance occurred in the process. Hollnagel and Slater (2022) refer to these specific scenarios as instantiations. If several patients presented with fractures, those instantiations can be analyzed to determine the actual variability that occurred in each instantiation of the process of fracture care. Recommendations could then be made regarding how the variability could be managed by enhancing positive variability and dampening negative variability.

Steps of the FRAM

Describing the purpose of the FRAM, the principles underlying the method, and the description and definition of key FRAM terminology have been provided to ensure readers grasp a basic understanding of the method. The steps of the method can be challenging to understand without this knowledge. Below are the four steps for building and analyzing a FRAM model: (Hollnagel & Slater, 2022)

1. Identify and characterize the essential functions in a process. Together, the identified functions constitute a FRAM model.

2. Describe the potential variability of the functions in the FRAM model, as well as the actual variability in one or more instantiations of the model.

3. Determine the possibility of resonance based on the interdependence among functions given their potential/actual variability.

4. Develop recommendations on how to monitor and manage the variability, either by dampening negative variability or enhancing positive variability.

The goal of any study using the FRAM is to produce a model that is an accurate depiction of the everyday activities and interdependencies of a complex healthcare process. Accurate modelling is dependent on the quality of the data gathered from stakeholders and study design decisions made by the researcher(s). The following sections will aim to provide guidance to researchers on how to use quality enhancement criteria and strategies in their qualitative research efforts so that the resulting FRAM models and insights can produce trustworthy findings and practice and policy recommendations.

Quality Enhancement

When designing a quantitative health care study, researchers move through methodical steps that exert control over the study with a goal of ensuring validity, reliability, objectivity, and generalizability in their study findings (Polit & Beck, 2020). These standards of quantitative research are scientifically rigorous and afford end users, such as health care administrators and clinicians, with some assurance that the evidence they are including in their decision making is of high quality. Adler (2022) explains the same assurance of quality is not as easily demonstrated in qualitative research studies.

Adler (2022) goes on to explain qualitative research as a paradigm has had to contend with a reputation for being less credible when compared to quantitative research because “finding relationships between numbers is assumed to be more rigorous than deriving meaning from words and actions” (p. 1). In qualitative research, the researchers themselves are the principal research instrument (Dodgson, 2019). This makes the appraisal of quality in qualitative research challenging. The personal characteristics of researchers cannot be appraised but the means researchers use to enhance the quality of their research can be (Adler, 2022). There is debate regarding the essential criteria for evaluating quality in qualitative research (Polit & Beck, 2020). Braun and Clarke (2013) explain qualitative research is not about comparisons, testing hypotheses, or replication, rather it is about gathering accounts of knowledge and information that are “produced in particular contexts by participants who come from and are located within, specific contexts” (p. 33). In the spirit of advancing the quality of qualitative research, researchers who employ the FRAM should ensure they take the steps necessary to build trustworthy models that can reliably inform practice and policy recommendations aimed at improving the quality, efficiency, and safety of health care delivery.

Achieving Trustworthiness in Qualitative Research

Reicher (2000) suggests it may not be possible to develop one set of quality criteria guidelines that can be applied to the broad range of theoretical approaches across the field of qualitative research. Polit and Beck (2020) explain the lack of consensus in the literature makes it challenging to provide guidance on quality standards. While there are different approaches to enhancing the quality of qualitative research, a comparison of approaches is beyond the scope of this paper. For the purpose of this paper, Lincoln & Guba’s trustworthiness criteria will be used (Guba & Lincoln, 1994; Lincoln & Guba, 1985). Trustworthiness is perhaps the most widely recognized measure of quality in qualitative research and refers to the level of confidence in data, interpretation, and methods used to uphold the quality of a study (Polit & Beck, 2020).

Lincoln & Guba’s trustworthiness criteria – credibility, confirmability, transferability, authenticity, and dependability – are viewed as the gold standard guide to quality enhancement in qualitative research and have been deemed as parallel to the rigor of the well-established quantitative quality criteria (Lincoln & Guba, 1985; Polit & Beck, 2020, p. 398).

Employing qualitative research methods for building a FRAM model without considering and presenting quality enhancement criteria and strategies will be met with challenges from end users, funders of health care research, as well as health care publications that disseminate health care research. These decision makers will be seeking how they can assess the quality of a FRAM model and ultimately the trustworthiness of FRAM research findings.

The remainder of this paper is dedicated to describing Lincoln and Guba’s trustworthiness criteria (See Table 1) and presenting examples of quality enhancement strategies employed by researchers who have used the FRAM in a health care research context. This paper is not meant to provide an exhaustive review of the literature on the use of the FRAM in the health care domain, rather the authors are highlighting select studies as examples where researchers have employed quality enhancement criteria and strategies. We believe researchers using the FRAM may find Lincoln & Guba’s trustworthiness criteria and the practical strategies for enhancing research quality to be useful when planning their research. As applications of the FRAM to health care settings tend to be focused on relatively specific circumstances, such as medication administration and safety in an intensive care unit (Furniss et al., 2020; Kaya et al., 2019; Oduyale et al., 2020), or the variability that exists in blood collection (Clay-Williams et al., 2020; Pickup et al., 2017) researchers will have to decide which criteria and strategies are most relevant to building trustworthiness into their particular FRAM model.

Table 1.

Lincoln & Guba’s Trustworthiness Criteria Defined (Guba & Lincoln, 1994; Lincoln & Guba, 1985; Polit & Beck, 2020).

Trustworthiness criteria	Definitions
Credibility	The confidence in the truthfulness of the study and its findings. There are two elements to credibility
	1. Carrying out qualitative inquiry in a manner that establishes the veracity of the findings
	2. Taking the necessary steps to demonstrate credibility to readers
Confirmability	Mainly concerned with showing objectivity and demonstrating how the interpretation of data is corroborated and is representative of the information that participants provided to researchers
Transferability	An effort to provide enough information and descriptive data for readers to determine how relevant study findings could potentially be to other settings and contexts
Dependability	Concerned with demonstrating how stable the study findings are over time and conditions and determining if the findings of the study could be repeated if similar participants and a similar context were used
Authenticity	When researchers convey to readers a sense of context that provide an awareness of the range of the different realities experienced by the participants

Credibility

Credibility is described by Polit and Beck (2020) as the confidence in the truthfulness of the study and its findings. Lincoln and Guba (1985) equate credibility to internal validity in quantitative research. Member checking, triangulation, reflexivity, persistent observation, and prolonged engagement are specific strategies that have been used by researchers using the FRAM method to establish credibility. Each of these strategies is discussed below, along with illustrative examples from the FRAM literature.

Member Checking

Member checking is the most crucial strategy to establish credibility and is also know as participant/respondent validation (Lincoln & Guba, 1985). It explores the credibility of study results by returning data or findings to participants to obtain feedback and check for accuracy. This ensures the findings are an accurate reflection of their experiences (Birt et al., 2016). In the present context of research that employs the FRAM, a common example of member checking is validation of the FRAM model. After constructing an initial model, researchers take further steps to ensure the model is an accurate representation of the process or system under examination. Several studies have described their approach to FRAM model validation to ensure accuracy of their respective models (Buikstra et al., 2020; Schreurs et al. 2021; Sujan et al., 2022; Thude et al., 2021; van Dijk et al., 2022). Buikstra et al. (2020) aimed to ensure their interpretations of participants’ responses were accurate in their examination of variability in the hospital discharge process of older adults. Throughout interviews and focus groups, study participants were given opportunities to correct notes and offer additional information to researchers.

When the model was constructed, it was presented to participants for validation. Schreurs et al. (2021) similarly presented their FRAM model for validation to stakeholders at a meeting, then adjusted the model based on feedback, and subsequently presented the revised model to stakeholders prior to model finalization. Salehi et al., 2021 used data from focus groups to improve a constructed model of the hospital to home transition process. The focus group data added more functions and new couplings, highlighting the importance of validation prior to finalization of a model.

Triangulation

When researchers seek patterns of convergence to advance or substantiate their interpretation of research findings, they are using the well-known quality enhancement strategy of triangulation (Mays & Pope, 2000). Triangulation can be achieved by using different methods of data collection, multiple data sources, and multiple investigators (Korstjens & Moser, 2018). The rationale for triangulation is that using multiple methods is superior to a single method in that it can potentially facilitate a deeper understanding and shed more light on a phenomenon (Lincoln & Guba, 1985). The following sections describe three types of triangulation (method, data source, investigator) with examples from the FRAM literature.

Method Triangulation

Method triangulation is an important strategy that researchers employing the FRAM can implement in the initial stages of study design. Method triangulation is a purposeful approach where multiple data collection methods (interviews, focus groups, document review, and/or observations) are used to inform a conclusion about the truth, and can enhance the credibility of a qualitative study (Polit & Beck, 2020). Murray & Chamberlain, 1999 explains the importance of multi-method research and likens it to a jigsaw puzzle by stating “it is only when we put the different pieces of the jigsaw together that we see a broader picture and gain some insight into the complexity of our research” (p. 49).

Salehi et al., 2021 used the FRAM to model the hospital to home transition process for frail older adults. The authors used six different methods of data collection (observations, interviews, focus groups, textual review, home observations, and questionnaires) to enhance study findings. When using the FRAM, researchers should take the time to consider what methods of data collection and combinations of data collection methods can best answer their research question.

Data Source Triangulation

A convergence of information about a phenomenon from different times, spaces, and/or persons is known as data source triangulation (Carter et al., 2014). In this instance, researchers using the FRAM to examine and analyze a healthcare process would enrich study findings by seeking the perspectives of all those central to everyday work. This may include patients and their families, as well as health care providers from a variety of professional backgrounds. Researchers may also seek out these perspectives in different locations of work (also known as cross site consistency) and over a range of times of the day/week to ensure an accurate description of the phenomenon is captured. van Dijk et al. (2022) used the FRAM to gain a better understanding of hospital worker compliance with standards for medication reconciliation. The authors interviewed 63 participants from a variety of health care professions (nursing, medicine, and pharmacy). Additionally, the authors conducted these interviews in nine different cardiology and orthopedic wards.

The use of data source triangulation related to space and person was identified as a study strength by the authors due to the diversity of the study population, which allowed the authors to gain a better overall view of the process (van Dijk et al., 2022). To strengthen the credibility of the study findings further, the authors could have also conducted interviews at different times and on different shifts to ensure data gathered reflected the work being always conducted on a hospital ward. Laugaland et al. (2014) conducted observations of the performance shaping factors of the discharge process of older adults and noted the lack of time triangulation as a study limitation.

Due to practical and resource-based issues, the authors were only able to conduct observations during regular working hours; they conceded that other performance shaping issues related to the discharge process of older adults may be influential at other times of the day.

Investigator Triangulation

Investigator triangulation uses more than one investigator in a study to gather data and/or make decisions on coding, analysis, and interpretation (Korstjens & Moser, 2018). Carter et al. (2014) describes it as an opportunity to convey alternate perspectives as well as confirm findings. Collaboration between investigators can also potentially reduce bias in decision making and analysis among investigators (Polit & Beck, 2020). Thude et al. (2021) conducted a FRAM study to build a model of the referring routines from hospital to community rehabilitation for patients with Chronic Obstructive Pulmonary Disease (COPD). A team of four researchers was formed to code the data from interviews and observation notes and to analyze the data identifying the functions and aspects. Any discrepancies identified were discussed to consensus (Thude et al., 2021). In their study examining fluoride varnish application for children attending general dental practices, Ross et al. (2018) had two members of the research team independently code a representative sample of data to identify the functions of the process. This exercise was done to provide readers with a reliability assessment using the qualitative strategy of Inter Coder Reliability (ICR). O’Connor and Joffe (2020) define ICR as “a numerical measure of the agreement between different coders regarding how the same data should be coded” (p. 2). This can be a useful exercise for researchers to undertake when using the FRAM because it can also establish the trustworthiness criterion of confirmability. This criterion is concerned with objectivity and demonstrating how the interpretation of data is not imagined by the researcher, rather truly represents the information that participants provided (Polit & Beck, 2020). Kurasaki (2000) found the reporting of ICR can potentially make the argument that the analysis was performed “conscientiously and consistently” (p. 8). This can establish credibility and confirmability, and ultimately enhance the trustworthiness of a FRAM model and its accuracy in representing the process of interest.

Reflexivity

Polit and Beck (2020) define reflexivity as the awareness a researcher has about their own unique background, set of values, and professional identity and how those characteristics can potentially impact the research process. Researchers can demonstrate reflexivity by keeping a personal journal documenting their own reactions, feelings, insights, and potential biases regarding data collection, analysis, and interpretation (Yin, 2016). An example of this is often demonstrated in observational research with the use of reflective field notes. Polit and Beck (2020) explain reflective field notes are often the personal reflections and experiences of the observer in the field and can provide guidance on how future observations can be made. Tresfon et al. (2022) examined the use of restraints on a nursing unit by conducting observations over 10 days. The researcher conducting observations kept field notes as well as a research diary in which they reflected on their role and influence as an observer (Tresfon et al., 2022). Presentation of reflexivity to readers often occurs in the presentation of study findings. Reflexivity statements can provide the reader with reassurance that potential for researcher bias was considered and steps to avoid bias were taken (Polit & Beck, 2020).

When reporting study limitations on the discharge process of older adults, Laugaland et al. (2014) described “possible observer bias” and attributed this to observations being conducted by a single researcher with a nursing background (p. 13). The researchers acknowledged the observer’s nursing background could potentially enhance the findings of study but may also affect the accuracy of observations. The researchers recognized this limitation and presented alternative ways the observations could have been conducted and presented how they attempted to control this potential bias. Specifically, the researchers had weekly meetings with the larger research team, who had diverse health care backgrounds, to discuss preliminary impressions of the observations made. These meetings with the larger research team demonstrate an additional quality enhancement strategy known as peer debriefing, which aims to further explore elements of the research process that “might otherwise remain only implicit within the inquirer’s mind” (Lincoln & Guba, 1985, p. 308). This can be achieved through probing by the debriefer to explore any omissions, perspectives, assumptions, and potential biases not self-identified by the researchers (Lincoln & Guba, 1985).

Additionally, a peer debriefer can provide support, challenge assumptions, and can ask tough questions about interpretations (Lincoln & Guba, 1985). When using the FRAM, researchers should consider reflexivity at all stages of the research process; if reflexive efforts have been made to identify bias or reduce bias, these should be communicated to readers. The presentation of this information is a strategy that can establish study credibility and enhance the trustworthiness of study findings.

Persistent Observation and Prolonged Engagement

When choosing observation as a method of data collection, one way to enhance credibility in the study findings is to ensure the study design allows for prolonged engagement and opportunities for persistent observation (Polit & Beck, 2020). With prolonged engagement, the researcher ensures enough time is invested to become familiar with the participants, setting, culture, and context of a study (Korstjens & Moser, 2018); such knowledge cannot be obtained in hurried fieldwork (Polit & Beck).

Persistent observation is a strategy that aims to gain an in-depth understanding of the phenomenon in its context by identifying and focusing on the qualities and components of the phenomenon that are most relevant to the problem or issue being studied (Polit & Beck). Lincoln and Guba (1985) state “if prolonged engagement provides scope, persistent observation provides depth” (p. 304). Laugaland et al. (2014) successfully demonstrated prolonged engagement and persistent observation in their study on hospital discharge of older adults. Over the course of 8 months, 173 conversations were had with patients, next of kin, and health care providers, as well 90 hours of moderate participant observations. Laugaland et al. explains in moderate participant observation, the researcher does not have a role in the social setting as a participant but is present and identifiable which allows for a high level of involvement while maintaining a level of detachment. The information provided to readers regarding the time spent observing the discharge process and the specifics and depth of what the researchers were observing was well described for readers.

Laugaland et al. recognized that their prolonged engagement and persistent observation of the discharge process for older adults was a significant study strength and documented this approach clearly for readers. As previously described, credibility is carrying out qualitative inquiry in a manner that establishes the veracity of study findings and takes the necessary steps to demonstrate these efforts to readers. Credibility is a crucial trustworthiness criterion with several strategies that can be exercised to demonstrate its presence in a study. Table 2 provides an overview of quality enhancement strategies used to establish credibility in select studies that used the FRAM to examine a healthcare process.

Table 2.

Strategies to Establish Credibility and Examples Where Strategies Were Used in the FRAM Healthcare Literature.

Trustworthiness criterion	Strategy	Examples from the FRAM Healthcare Literature
Credibility - the confidence in the truthfulness of the study and its findings	Member checking	Buikstra et al. (2020) Schreurs et al. (2021) Sujan et al. (2022)
	Member checking	Van Dijk et al. (2022) Thude et al. (2021)
	Data source triangulation	van Dijk et al. (2022)
	Investigator triangulation	Thude et al. (2021)
	Investigator triangulation	Ross et al. (2018)
	Method triangulation	Salehi et al., 2021
	Reflexivity	Laugaland et al. (2014), Trefson et al. (2022)
	Prolonged engagement	Laugaland et al. (2014)
	Persistent observation	Laugaland et al. (2014)
	Peer debriefing	Laugaland et al. (2014)

Confirmability

Confirmability is mainly concerned with objectivity and demonstrating how the interpretation of data is not imagined by the researcher and truly represents the information that participants provided (Polit & Beck, 2020). Korstjens and Moser (2018) defined confirmability as “the degree to which the findings of the research study could be confirmed by other researchers” (p. 121). The strategies of member checking and ICR were already presented as means for researchers to establish credibility when using the FRAM in healthcare research. Each of these strategies can also be used to establish confirmability.

Keeping an audit trail is an additional strategy researchers can take to establish confirmability. An audit trail can be achieved by recording and transcribing interviews and focus groups, keeping detailed field notes, methodological or reflexive notes, and/or a study journal. An audit trail allows for other researchers to access the data, recreate the steps taken, and draw similar conclusions (Polit & Beck). Salehi et al., 2021 provides readers with information about how data was collected using multiple methods, and then reviewed by the research team.

Salehi et al., 2021 goes on to describe the use of audio recordings and transcriptions of semi-structured interviews and focus groups with health care workers and patients, as well as field notes taken at meetings, and email exchanges between the research team and health care workers.

A decision trail is another strategy that could be implemented by researchers when using the FRAM. This strategy describes the researcher’s decision rules for how data and analytic interpretations are categorized and allows for readers to evaluate the reliability of the decisions being made (Polit & Beck, 2020). To date this strategy has not been used to establish confirmability in a study using the FRAM. A decision trail could be used to provide readers with more guidance when it comes to identifying the functions and aspects that are the core of a FRAM model. Several FRAM studies describe the process of identifying functions and aspects as an “iterative process”, rather than providing a clear description of how their research team finalized their functions and models (Alm & Woljter, 2010; Furniss et al., 2020, Kaya et al., 2019; Laugaland et al., 2014; Oduyale et al., 2020; O'Hara et al., 2020; Schreurs et al., 2021; Tresfon et al., 2022).

Without a clear description of this process, readers are left to wonder how final decisions were made and if they accurately reflect the experiences of participants. Some of these FRAM studies overcame this ambiguity by using member checking with participants to validate FRAM models (Kaya et al., 2019; Oduyale et al., 2020; O'Hara et al., 2020; Schreurs et al., 2021; Tresfon et al., 2022). Polit and Beck (2020) found that for confirmability to be established, study findings need to represent the information provided by the study participants and make a concerted effort to avoid any researcher biases. Table 3 provides an overview of quality enhancement strategies used to establish confirmability in select studies that used the FRAM.

Table 3.

Strategies to Establish Confirmability and Examples from the FRAM Healthcare Literature.

Trustworthiness criterion	Strategy	Example from FRAM literature
Confirmability	Audit trail	Salehi et al., 2021
Demonstrating how the interpretation of data is not imagined by the researcher and is representative of the information that participants provided to researchers	Decision trail	N/A
	Intercoder reliability checks	Ross et al. (2018)
	Member checking	Buikstra et al. (2020)
		Schreurs et al. (2021)
		Sujan et al. (2022)
		van Dijk et al. (2022)
		Thude et al. (2021)

Transferability

It is the researcher’s responsibility to provide enough information and descriptive data for readers to determine how relevant study findings could potentially be to other settings and contexts (Lincoln & Guba, 1985). This is usually achieved by providing a “thorough, and vivid description of the research context, study participants, and events and experiences observed during the inquiry” (p. 408).

Oduyale et al. (2020) presented a full description of the context related to the administration of multiple medications using a single intravenous lumen to intensive care unit patients. The potential for physiochemical reactions and harm to the patient are well described. The challenges of determining the safety of administration of multiple medications and the work arounds to ensure safety are also well described for the reader using verbatim quotes from ICU nurses (Oduyale et al., 2020). The quotes from the nurses provided a sense of frustration with some of the daily challenges they face in this process, and conveyed the decisions they make to adapt to these challenges.

When using the FRAM, researchers may choose to include such descriptions for readers to decide related to transferability of study findings. Because of the highly contextualized nature of studies using the FRAM, readers may find the transferability of study findings to be limited. Buikstra et al. (2020) found that the FRAM favours a depth of understanding about a specific workplace over generalizability, so any conclusions drawn from a FRAM study would be largely limited to the specific study setting. Researchers can potentially address challenges in transferability by offering an alternative, such as functional transferability.

Several FRAM studies have examined care transitions in the elderly, specifically hospital to home transitions (Buikstra et al., 2020; Laugaland et al., 2014; O’Hara et al., 2020; Salehi et al., 2021). Although the FRAM models are specific to each work context, many of the functions in the care transitions models are similar and could constitute a type of functional library that other researchers could use to assist in developing new FRAM models specific to their study settings. Table 4 provides an overview of quality enhancement strategies used to establish transferability in select studies that used the FRAM.

Table 4.

Strategy of Vivid Description Used to Establish Transferability and an Example of the Use of Vivid Description in the FRAM Healthcare Literature.

Trustworthiness criterion	Strategy	Example from FRAM literature
Transferability	Vivid description	Oduyale et al. (2020)
An effort to provide enough information and descriptive data for readers to determine how relevant study findings could potentially be to other settings and contexts		O'Hara et al. (2020)
		Salehi et al., 2021
		Laugaland et al. (2014)

Authenticity

A study can demonstrate authenticity if it “invites readers into a vicarious experience of the lives being described and enables readers to develop a heightened sensitivity to the issues being depicted” (Polit & Beck, 2020, p. 400). There are several ways that researchers can establish authenticity when using the FRAM. Strategies previously described include vivid description, audit trails, prolonged engagement, persistent observation, reflexivity, and impactful and evocative writing (Polit & Beck, 2020). Many of these strategies are used to establish other types of trustworthiness criteria and have previously been described.

Impactful and evocative writing is another strategy to establish authenticity and aims to bring strong images or feelings to the mind of the reader. In their study using the FRAM to examine transitional care using multistakeholder perspectives, O'Hara et al. (2020) reported that in cases where patients were encouraged to retain independence and involvement in their own care while in hospital, they overcame discharge challenges in the home more readily.

One of this study’s key findings was how impactful the perspectives and experiences of older adults were in identifying functions key to the success of the discharge process for this population. The researchers highlighted key findings of their study by providing examples using ‘illustrative cases” from the data. Key functions identified in the transitional care process were described for readers, along with how those functions varied and impacted post discharge outcomes. Exemplar cases were presented describing the experiences of older adults. Providing this impactful description allows the reader to see how the knowledge and information gathered from patients and their families can make a significant contribution to the findings of a FRAM study. Salehi et al., 2021 included quotes from healthcare professionals in their study on hospital to home transitions in older adults to emphasize the importance of follow up in the community after discharge. The healthcare providers communicate that the success of a discharge depends on the availability and access to appropriate community based follow up and if not in place can result in readmission. Table 5 provides an overview of quality enhancement strategies used to establish authenticity in select studies that used the FRAM.

Table 5.

Strategies to Establish Authenticity and Examples Where Strategies to Establish Authenticity Were Used in the FRAM Healthcare Literature.

Trustworthiness criterion	Strategy	Example from FRAM literature
Authenticity	Evocative writing	O'Hara et al. (2020)
When researchers convey to readers a sense of context that provide an awareness of the range of the different realities experienced by the participants	Vivid description	Oduyale et al. (2020)
	Audit trails	Salehi et al., 2021
	Prolonged engagement	Laugaland et al. (2014)
	Persistent observation	Laugaland et al. (2014)
	Reflexivity	Laugaland et al. (2014)

Dependability

Dependability is the final trustworthiness criteria to be reviewed. It is concerned with how stable the data from a study is over time and over the conditions of a study (Polit & Beck, 2020). Dependability in a study shows that the findings are consistent and could be repeated (Korstjens & Moser, 2018). Maintaining an audit trail is a key strategy technique for establishing dependability. The study field notes, transcriptions, and/or recordings can then be examined by an external reviewer for the purposes of additional strategies, such as peer debriefing (previously described) and inquiry audits. Inquiry audits involve having an external researcher not involved in the research process examine the entire research process, including data collection, analysis, and interpretation to evaluate if the findings, interpretations, and conclusions are supported by the data (Braun & Clarke, 2013).

According to Polit and Beck (2020) relatively few comprehensive inquiry audits have been reported in the literature. There are no reported inquiry audits in the FRAM health care literature to date. Similar to transferability, this may be due to the highly contextual nature of studies using the FRAM to examine a specific process. Researchers using the FRAM should aim to keep an organized account of the research process to facilitate producing an audit trail of sufficient detail that it can support peer debriefing or an inquiry audit that may be undertaken to establish dependability and trustworthiness in the study findings. Table 6 provides an overview of quality enhancement strategies used to establish dependability in select studies that used the FRAM.

Table 6.

Strategies to Establish Dependability and Examples Where Strategies Were Used in the FRAM Healthcare Literature.

Trustworthiness criterion	Strategy	Example from FRAM literature
Dependability	Inquiry audits	N/A
Concerned with demonstrating how stable the study findings are over time and conditions. Can the findings of the study be repeated if similar participants and a similar context were used?	Audit trail	Salehi et al., 2021
	Peer debriefing	Laugaland et al. (2014)

Conclusion

Studies using the FRAM to examine and analyze healthcare processes are becoming increasingly prevalent (McGill et al., 2022). Designing and conducting a trustworthy study using the FRAM begins with developing a sound understanding of the method itself, as well as how to plan and gather qualitative data while simultaneously threading quality enhancement strategies throughout all phases of the research process. This paper presented several quality enhancement criteria and strategies with examples researchers have used in their respective studies. By using Lincoln & Guba’s criteria for trustworthiness, researchers can build FRAM models that are trustworthy and can produce findings that can assist end users and decision makers in identifying distinct points where vulnerabilities exist or where improvements can be made in healthcare processes. Building a model that truly represents how healthcare is actually delivered and received is noteworthy and can provide new insight into the complexities of a healthcare process that may not otherwise be realized.

Using that model to identify how variability emerges and impacts system outcomes has the potential to make significant contributions to improving healthcare quality, efficiency, safety, and design. The FRAM has great potential to expand its reach further in the healthcare domain, but those conducting the research need to do so in a way that instills confidence that the findings and recommendations are trustworthy.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research funding was provided to AM by the Ocean Frontier Institute through an award from the Canada First Research Excellence Fund. The funders have not had any role in content development.

ORCID iD

Alexis McGill

References

Aase

Waring

(2020). Crossing boundaries: Establishing a framework for researching quality and safety in care transitions. Applied Ergonomics, 89, 103228. https://doi.org/10.1016/j.apergo.2020.103228

Adler

R. H.

(2022). Trustworthiness in qualitative research. Journal of Human Lactation, 38(4), 598–602. https://doi.org/10.1177/08903344221116620

Alm

Woltjer

(2010). Patient safety investigation through the lens of FRAM. In Human factors: A system view of human, technology, and organisation (pp. 153–165). Shaker Publishing.

Birt

Scott

Cavers

Campbell

Walter

(2016). Member checking: A tool to enhance trustworthiness or merely a nod to validation? Qualitative Health Research, 26(13), 1802–1811. https://doi.org/10.1177/1049732316654870

Braun

Clarke

(2013). Successful qualitative research: A practical guide for beginners (pp. 1–400). Successful Qualitative Research.

Buikstra

Strivens

Clay-Williams

(2020). Understanding variability in discharge planning processes for the older person. Safety Science, 121, 137–146. https://doi.org/10.1016/j.ssci.2019.08.026

Carayon

Bass

E. J.

Bellandi

Gurses

A. P.

Hallbeck

M. S.

Mollo

(2011). Sociotechnical systems analysis in health care: A research agenda. IIE Transactions on Healthcare Systems Engineering, 1(1), 145–160. https://doi.org/10.1080/19488300.2011.619158

Carter

Bryant-Lukosius

DiCenso

Blythe

Neville

A. J.

(2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41(5), 545–547. https://doi.org/10.1188/14.ONF.545-547

Clay-Williams

Austin

Braithwaite

Hollnagel

(2020). Qualitative assessment to improve everyday activities: Work-as-imagined and work-as-done. In Transforming healthcare with qualitative research (pp. 71–82). Routledge.

10.

Clay-Williams

Hounsgaard

Hollnagel

(2015). Where the rubber meets the road: Using FRAM to align work-as-imagined with work-as-done when implementing clinical guidelines. Implementation Science, 10(1), 125. https://doi.org/10.1186/s13012-015-0317-y

11.

Dodgson

J. E.

(2019). Reflexivity in qualitative research. Journal of Human Lactation, 35(2), 220–222. https://doi.org/10.1177/0890334419830990

12.

Functional Resonance Analysis Method . (n.d.). Functional resonance analysis method. https://functionalresonance.com/

13.

Furniss

Nelson

Habli

White

Elliott

Reynolds

Sujan

(2020). Using FRAM to explore sources of performance variability in intravenous infusion administration in ICU: A non-normative approach to systems contradictions. Applied Ergonomics, 86, 103113. https://doi.org/10.1016/j.apergo.2020.103113

14.

Guba

E. G.

Lincoln

Y. S.

(1994). Competing paradigms in qualitative research. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), Handbook of qualitative research (pp. 105–117). Sage Publications, Inc.

15.

Hollnagel

(2012). FRAM: The functional resonance analysis method: Modelling complex socio-technical systems. CRC Press.

16.

Hollnagel

Hill

(2020). FRAM model visualiser instructions. Retrieved November 11, 2022 from: https://functionalresonance.com/onewebmedia/FMV_instructions_2.1.pdf

17.

Hollnagel

Hounsgaard

Colligan

(2014). FRAM – the Functional Resonance Analysis Method – a handbook for the practical use of the method. Centre for Quality www.centerforkvalitet.dk/framhandbook

18.

Hollnagel

Slater

(2022). FRAMSYNT: A FRAM handbook. Retrieved November 17, 2022 from: https://www.researchgate.net/profile/David-Slater/publication/364959115_A_FRAM_HANDBOOK/links/63611d2f6e0d367d91e7b7e2/A-FRAM-HANDBOOK.pdf

19.

Kaya

G. K.

Ovali

H. F.

Ozturk

(2019). Using the functional resonance analysis method on the drug administration process to assess performance variability. Safety Science, 118, 835–840. https://doi.org/10.1016/j.ssci.2019.06.020

20.

Korstjens

Moser

(2018). Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. European Journal of General Practice, 24(1), 120–124. https://doi.org/10.1080/13814788.2017.1375092

21.

Kurasaki

K. S.

(2000). Intercoder reliability for validating conclusions drawn from open- ended interview data. Field Methods, 12(3), 179–194. https://doi.org/10.1177/1525822x0001200301

22.

Laugaland

Aase

Waring

(2014). Hospital discharge of the elderly-an observational case study of functions, variability, and performance-shaping factors. BMC Health Services Research, 14(1), 365. https://doi.org/10.1186/1472-6963-14-365

23.

Lincoln

Y. S.

Guba

E. G.

(1985). Naturalistic inquiry. Sage.

24.

Mays

Pope

(2000). Qualitative research in health care: Assessing quality in qualitative research. British Medical Journal, 320(7226), 50–52. https://doi.org/10.1136/bmj.320.7226.50

25.

McGill

Smith

McCloskey

Morris

Goudreau

Veitch

(2021). The functional resonance analysis method as a health care research methodology: A scoping review protocol. JBI Evidence Synthesis, 19(3), 734–740. https://doi.org/10.11124/JBIES-20-00237

26.

McGill

Smith

McCloskey

Morris

Goudreau

Veitch

(2022). The functional resonance analysis method as a health care research methodology: A scoping review. JBI Evidence Synthesis, 20(4), 1074–1097. https://doi:10.11124/JBIES-20-00237

27.

Murray

Chamberlain

(Eds) (1999). Qualitative Health Psychology: Theories and Methods, SAGE Publications Ltd, 98–114. DOI:https://doi.org/10.4135/9781446217870.

28.

O’Connor

Joffe

(2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 160940691989922. https://doi.org/10.1177/1609406919899220

29.

Oduyale

Patel

Borthwick

Claus

(2020). Co-administration of multiple intravenous medicines: Intensive care nurses’ views and perspectives. Nursing in Critical Care, 25(3), 156–164. https://doi.org/10.1111/nicc.12497

30.

O'Hara

J. K.

Baxter

Hardicre

(2020). ‘Handing over to the patient’: A FRAM analysis of transitional care combining multiple stakeholder perspectives. Applied Ergonomics, 85, 103060. https://doi.org/10.1016/j.apergo.2020.103060

31.

Patriarca

Di Gravio

Woltjer

Costantino

Praetorius

Ferreira

Hollnagel

(2020). Framing the FRAM: A literature review on the functional resonance analysis method. Safety Science, 129, 104827. https://doi.org/10.1016/j.ssci.2020.104827

32.

Pickup

Atkinson

Hollnagel

Bowie

Gray

Rawlinson

Forrester

(2017). Blood sampling-Two sides to the story. Applied Ergonomics, 59(Pt A), 234–242. https://doi.org/10.1016/j.apergo.2016.08.027

33.

Polit

Beck

(2020). Essentials of nursing research: Appraising evidence for nursing practice. Lippincott Williams & Wilkins.

34.

Reicher

(2000). Against methodolatry: Some comments on Elliott, Fischer, and Rennie. British Journal of Clinical Psychology, 39, 1–6. https://doi.org/10.1348/014466500163031

35.

Ross

Sherriff

Kidd

Gnich

Anderson

Deas

Macpherson

(2018). A systems approach using the functional resonance analysis method to support fluoride varnish application for children attending general dental practice. Applied Ergonomics, 68, 294–303. https://doi.org/10.1016/j.apergo.2017.12.005

36.

Salehi

Smith

Veitch

Hanson

(2021). A dynamic version of the FRAM for capturing variability in complex operations. Methods X, 8, 101333. https://doi.org/10.1016/j.mex.2021.101333

37.

Salehi

Veitch

Smith

(2020). Modeling complex socio-technical systems using the FRAM: A literature review. Human Factors and Ergonomics in Manufacturing, 31(1), 118–142. https://doi.org/10.1002/hfm.20874

38.

Salehi

Veitch

Smith

(2021). Modeling complex socio-technical systems using the FRAM: A literature review. Human Factors and Ergonomics in Manufacturing & Service Industries, 31(1), 118–142. https://doi.org/10.1002/hfm.20874

39.

Schreurs

R. H. P.

Joore

M. A.

Ten Cate

ten Cate-Hoek

A. J.

(2021). Using the functional resonance analysis method to explore how elastic compression therapy is organised and could be improved from a multistakeholder perspective. BMJ Open, 11(10), Article e048331. http://dx.doi.org/10.1136/bmjopen-2020-048331

40.

Schutijser

B. C. F. M.

Jongerden

I. P.

Klopotowska

J. E.

Portegijs

de Bruijne

M. C.

Wagner

(2019). Double checking injectable medication administration: Does the protocol fit clinical practice? Safety Science, 118, 853–860. https://doi.org/10.1016/j.ssci.2019.06.026

41.

Smith

Veitch

Khan

Taylor

(2020). Integration of resilience and FRAM for safety management. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering, 6(2), 1–7. https://doi.org/10.1061/ajrua6.0001044.

42.

Sujan

Bilbro

Ross

Earl

Ibrahim

Bond-Smith

Ghaferi

Pickup

McCulloch

(2022). Failure to rescue following emergency surgery: A FRAM analysis of the management of the deteriorating patient. Applied Ergonomics, 98, 103608. https://doi.org/10.1016/j.apergo.2021.103608

43.

Thude

B. R.

Brink

Hansen

M. S.

Morsø

(2021). How to ensure referral and uptake for COPD rehabilitation—Part 1: Disentangling factors in the cross-sectorial workflow of patients with COPD to understand why most patients are not referred to rehabilitation. International Journal of Integrated Care, 21(1), 10. https://doi.org/10.5334/ijic.5503

44.

Tresfon

Brunsveld-Reinders

A. H.

van Valkenburg

Langeveld

Hamming

J. F.

(2022). Aligning work-as-imagined and work-as-done using FRAM on a hospital ward: A roadmap. BMJ Open Quality, 11(4), Article e001992. https://doi.org/10.1136/bmjoq-2022-001992

45.

van Dijk

L. M.

van Eikenhorst

Wagner

(2022). Daily practice performance (Work-as-Done) compared to guidelines (Work-as-Imagined) of medication reconciliation at discharge: Outcomes of a FRAM study. Safety Science, 155, 105871. https://doi.org/10.1016/j.ssci.2022.105871

46.

Weston

Krein

Harrod

(2022). Using observation to better understand the healthcare context. Qualitative Research in Medicine and Healthcare, 5(3), 1–14. https://doi.org/10.4081/qrmh.2021.9821.

47.

Yin

R. K.

(2016). Qualitative research from start to finish. Guilford Publications.