Ensuring Excellence: A Systematic Scoping Review of Quality Assurance in Biomedical Qualitative Research Paradigm

Abstract

Background: Ensuring the quality of qualitative research has led to the development of various approaches and criteria, but has also sparked challenges and disagreements. This study aims to review existing knowledge on quality assurance in biomedical qualitative research. Method: This systematic scoping review study was conducted using the Arksey and O’Malley approach in five stages. Relevant articles were accessed through keywords including “Trustworthiness”، “Research quality”، “Research integrity”، “Responsible research”، “Rigor”، “Qualitative research”، “Qualitative study”، “Qualitative paradigm”، “Robustness”، “Soundness” in PubMed، Embase، Scopus، Web of Science، CINAHL databases. The search was completed through the Google Scholar search engine, and highly relevant articles and key journals were referenced. Full-text English-language biomedical articles relevant to the research objective were included and findings were presented based on specific research questions. Result: The review highlights that the quality of qualitative research remains a significant concern for researchers. However, efforts to address these concerns have been inconsistent, leading to conflicts in terminology, definitions, and strategy implementations. Criticisms and challenges have often overshadowed proposed solutions and recommendations, affecting both paradigmatic and methodological aspects. While some strategies have been critiqued and discussed, researchers have also encountered specific situational issues that have been addressed accordingly. Conclusion: Our review indicates that ensuring the quality of qualitative research has been contentious since this methodology’s inception. Given the significant impact of these studies’ findings, it is essential for future investigations to address these conflicts and propose effective solutions. It is recommended that further investigations be conducted, encouraging researchers to utilize the findings of this study to refine methods for evaluating and enhancing the quality of qualitative research. These efforts should align with the naturalistic paradigm and take into account the unique characteristics of each methodology.

Introduction

Issues related to research quality have persisted in various forms since the emergence of modern science (Langfeldt et al., 2020). Related concepts such as open science, research integrity, and responsible research have prioritized the enhancement of quality, applicability, and ethical considerations of research findings globally (Shaw and Satalkar, 2018; Tijdink et al., 2021; Vicente-Saez and Martinez-Fuentes, 2018). Concerns about the quality and robustness of research findings have a longstanding history in both quantitative and qualitative research paradigm (Hamedanchi et al., 2021; Khankeh et al., 2024).

Research quality refers to the degree to which research meets standards of methodological rigor, relevance to stakeholders, ethical integrity, and reproducibility. High-quality research demonstrates reliability, validity, transparency, and the ability to produce meaningful and impactful findings that advance knowledge or inform decision-making (Guba, 1989). In the quantitative paradigm, the inherent nature of research, data types, data collection methods, and data analysis techniques allow for fixed and predetermined methods to evaluate the credibility of findings and rigor of the study (Khankeh et al., 2024; Speziale et al., 2011). However, in the qualitative paradigm, the flexible nature of this type of research, the non-numerical data, the diversity of analysis methods, and the role of the researcher as an instrument and co-creator of knowledge have consistently drawn criticism regarding the quality of the study findings (Khankeh et al., 2024; Speziale et al., 2011). The diversity of basic epistemological assumptions in qualitative research, ranging from positivism to constructivism, reflect differences in how reality and knowledge are understood. Positivism focuses on objective real reality and empirical data, while constructivism emphasizes subjective experiences and the social construction of reality. This variety of paradigms contributes to ambiguity in defining and assessing and defining research quality, as different approaches have distinct evaluation criteria (Spencer et al., 2014).

In this context, various scholars such as Lincoln and Guba, Morse et al., Glaser, Strauss, and Corbin have proposed criteria for evaluating the quality of qualitative research findings (Corbin and Strauss, 2008; Guba, 1981; Guba and Lincoln, 1981; Speziale et al., 2011). While numerous qualitative researchers have utilized these criteria over time, the efforts have resulted in disagreements regarding the criteria, definition, and assurance of research quality (Cohen and Crabtree, 2008; Hamedanchi et al., 2021; Kornbluh, 2015).

On the other hand, some experts argue that the proposed criteria for qualitative research are derived from the validity and reliability criteria of quantitative research, and thus, do not align with the naturalistic paradigm and the inherent nature of qualitative research (FitzPatrick, 2019; Johnson et al., 2020; Murphy and Yielder, 2010). Additionally, researchers who have implemented various strategies to ensure the quality of qualitative studies have encountered significant challenges. They believe that the application of some of these strategies is neither easy nor practical, and in many cases, the results have not been effective (Hamilton, 2020; Johnson et al., 2020). It is also important to note that the proposed criteria do not focus solely on the quality of the findings. Failure to pay attention to these criteria can also affect ethical issues and threaten the applicability and effectiveness of the findings (Hamedanchi et al., 2021; Khankeh et al., 2024; Kornbluh, 2015).

Another challenge in ensuring research quality is that different dimensions of research (such as the field and space of research) can lead to variations in definitions and criteria for study quality (Langfeldt et al., 2020). Consequently, the proposed criteria, often originating from the perspectives of scholars in the humanities and social sciences, may not be entirely suitable for other research domains, such as biomedical research (Corbin and Strauss, 2008; Guba and Lincoln, 1981; Langfeldt et al., 2020).

In biomedical studies, qualitative research provides an in-depth understanding of healthcare providers, researchers, and patients’ experiences, perceptions, behaviors, and social contexts, informing evidence-based practices and enhancing the real-world applicability of findings (Hamedanchi et al., 2021; Holloway & Galvin, 2023; Khankeh et al., 2024; Vlassoff, 2012). In this field, qualitative research plays a vital role in knowledge production by offering comprehensive insights for decision-making across macro and micro levels. These insights are crucial for planning and informed actions in education, clinical practice, management, policymaking, and research (Hamedanchi et al., 2021; Holloway & Galvin, 2023; Khankeh et al., 2024; Vlassoff, 2012). Therefore, the quality of qualitative research is essential for ensuring that the evidence generated is robust, credible, and useful.

In this regard, the first step toward achieving effective and high-quality research appears to be recognizing the existing body of knowledge on the quality of qualitative studies. The authors of this study believe that a systematic scoping review, which can identify existing evidence and gaps, can achieve a comprehensive understanding of all aspects of this subject. This study aims to identify and define the criteria and strategies for ensuring the quality of biomedical qualitative studies, examine the associated criticisms and challenges, and the solutions/recommendations that have been proposed to address these issues.

Method

Aim of the Study

This study aims to identify and define the criteria and strategies for ensuring the quality of biomedical qualitative studies, examine the associated criticisms and challenges, and the solutions and recommendations that have been proposed to address these issues.

Design of Study

This systematic scoping review was conducted using the approach proposed by Arksey and O’Malley (2005), and the PRISMA extension for scoping reviews which includes five stages:

1 Identifying the Research Questions: Defining the main questions that the review seeks to answer.

2 Identifying Relevant Studies: Conduct comprehensive searches of relevant databases and sources to gather studies pertinent to the research questions.

3 Study Selection: Applying inclusion and exclusion criteria to select studies for the review.

4 Charting the Data: Systematically extracting and organizing data from the selected studies.

5 Collating, Summarizing, and Reporting the Results: Analyzing the data and presenting the findings in a structured manner (Arksey and O’Malley (2005)).

Research Questions

In the first step, the research questions were identified by the research team based on the study’s aim. These questions were reviewed throughout the study and ultimately refined to the following:

1 How is research quality defined in qualitative research?

2 What criteria and strategies are proposed in the literature to ensure the quality of qualitative research?

3 How are the criteria and strategies defined in the literature?

4 What are the criticisms/challenges associated with the criteria for ensuring the quality of qualitative research?

5 What solutions/recommendations have been proposed in the literature to address these criticisms/challenges?

6 Is there any conceptual framework or model for helping to understand the quality of qualitative research presented in the literature?

These questions formed the foundation for the search, extraction, analysis, and presentation of the findings.

Identifying Relevant Studies

To identify literature relevant to the study’s aim, searches were conducted in databases including PubMed, Embase, Scopus, and CINAHL using English keywords such as “trustworthiness,” “research quality,” “research integrity,” “responsible research,” “rigor,” and “qualitative research” up until May 2024. Hand searches of articles were also performed using Google Scholar, key journals, and reference lists of highly relevant articles. The management of the literature was conducted using EndNote, version 7.

Study Selection

The study selection process adhered to predefined inclusion and exclusion criteria and was conducted in two stages. The inclusion criteria were: (1) peer-reviewed, full-text articles in English; (2) a primary focus on quality assurance, trustworthiness, or rigor in qualitative research; and (3) a clear context within biomedical research. The exclusion criteria were: (1) commentaries, editorials, or conference abstracts lacking original methodological discussion; (2) articles not pertaining to qualitative research methodology (National Library of Medicine, 2003). In the first stage, two researchers independently screened the titles and abstracts. In the second stage, the full texts of the shortlisted articles were independently assessed by the same researchers for final inclusion. Any disagreements at either stage were resolved through consensus within the research team (Arksey and O’Malley, 2005).

Data Extraction

Data extraction was conducted based on the study’s aim and research questions. Tables were designed by the research team, and two members of the team simultaneously reviewed five articles and extracted the findings according to these tables. The findings were then discussed with the research team, and the tables were finalized. After data extraction, the tables created by the two researchers were compared and integrated (Arksey and O’Malley (2005)). Given the diversity of quality assurance approaches in the literature, an organizing framework was essential for a coherent synthesis. We adopted Lincoln and Guba’s trustworthiness criteria—credibility, transferability, dependability, and confirmability—as our primary analytical framework (Guba, 1981). While this framework guided the initial data extraction and categorization, our analysis also remained open to emergent, inductively derived concepts that fell outside its boundaries (Bradley et al., 2007).

To ensure a common understanding within the research team, key terms were defined prior to data extraction. For this study, biomedical research is defined as “the broad area of science that looks for ways to prevent and treat diseases that cause illness and death in people and in animals” (California Biomedical Research Association, 2025; National Library of Medicine, 2003). The quality of a study refers to designing and implementing research precisely and comprehensively, in accordance with correct methodology, while adhering to ethical principles. The criteria for quality were based on the four aspects of trustworthiness proposed by Lincoln and Guba—credibility, transferability, dependability, and confirmability—to assess the accuracy and robustness of qualitative research. A strategy was defined as a set of actions (e.g., member checks, audit trails) taken to meet these criteria. Criticisms/challenges refer to problems or issues raised in the literature regarding quality assessment, while solutions/recommendations denote proposed tips to address these problems. Finally, a conceptual framework or model refers to any structured, integrated set of suggestions for ensuring qualitative research quality.

Data Analysis and Reporting Results

The findings were reported based on the research questions. Efforts were made to present the similarities, disagreements, gaps, and existing evidence in a way that facilitates the reader’s understanding of the subject under study (Arksey and O’Malley, 2005). The qualitative content analysis recommended by Graneheim and Lundman (2004) has been used for data analysis (Graneheim and Lundman, 2004).

Ethical Considerations

This study was approved by the ethics committee of Committee of the University of Social Welfare and Rehabilitation Sciences, Tehran, Iran (IR.USWR.REC.1403.193). The researchers committed to methodological rigor and precision to conduct a high-quality study (Khankeh et al., 2024).

Result

Following a comprehensive hand search and database search, we initially extracted 831 articles. After removing duplicates, 627 articles remained for the first screening. A review of the titles and abstracts led to the selection of 134 articles for secondary screening. Ultimately, based on the inclusion criteria, 58 studies were included in the final report (Figure 1).

Figure 1.

Prisma flowchart

Based on the findings, the first article was published in 1993, with a noticeable increase in publications over the years, particularly in the last decade. This trend suggests a growing recognition of qualitative research and the need for rigorous standards. Figure 2 shows the process of publishing articles related to the issue of quality assurance of qualitative studies.

Figure 2.

Publication trend of articles on quality assurance of studies

The final report encompassed all articles related to quality assurance in qualitative research. Out of the 58 articles reported, 28 primarily focused on rigor and trustworthiness, despite variations in the spelling of “rigor” (Rigour) across studies. The remaining studies addressed specific strategies or issues such as large-scale studies with big data or small sample sizes.

The findings revealed that, despite some studies’ emphasis on the qualitative paradigm, terms commonly associated with quantitative research, such as “validity” and “reliability,” remained more prevalent in the reported articles. Figure 3 illustrates the frequency of these terms in published articles over different years. The frequency did not follow a specific pattern, and the usage of quantitative research terms did not decrease over time.

Figure 3.

Frequency of terms related to the quality of qualitative studies

The majority of the studies (45 articles) did not follow a specific study design. Among those that did, the majority were review studies (6 articles). Based on the analysis of the reported studies, the number of studies mentioning each specific focus area is as follows: description of criteria (38 studies), defining strategy (42 studies), criticisms/challenges (33 studies), solutions/recommendations (24 studies), and framework (7 studies). Figure 4 illustrates the proportion of each sub-area of focus within the study findings concerning the total number of reported articles.

Figure 4.

Proportion of sub-areas of focus within study findings relative to total reported articles

Based on the findings of this study, no consensus exists regarding the definitions and conceptualization of quality assessment in qualitative research. The literature reveals significant diversity and differences in how criteria are defined. This review specifically examined how the concepts of rigor and trustworthiness have been defined. To provide a clear foundational understanding, we have first synthesized the core similarities in how the major criteria are commonly defined.

Based on the authors’ analysis of the included studies, the overarching concepts of rigor and trustworthiness can be understood as follows:

Trustworthiness

Lincoln and Guba’s seminal work in the 1980s introduced the concept of “trustworthiness” to replace traditional notions of reliability and validity in qualitative research. They proposed trustworthiness as a central criterion for assessing the rigor of qualitative studies, encompassing quality, authenticity, and truthfulness of findings. Trustworthiness is achieved through credibility, transferability, dependability, and confirmability, which parallel the conventional criteria of validity and reliability. Trustworthiness is both a goal and a measure throughout the research process, involving specific activities to meet criteria related to truth value, applicability, consistency, and neutrality.

Rigor

Lincoln and Guba (1985) established criteria for judging the rigor of a naturalistic inquiry, which they call trustworthiness, providing four different criteria for its evaluation. Rigor in qualitative research means conducting precise and high-quality research, ensuring that the design, data collection, data analysis, publication of findings, and implementation of ethical principles are transparent, comprehensive, systematic, accurate, and open to critique while being free from bias.

Regarding the specific criteria proposed for evaluating trustworthiness and rigor, the definitions and strategies for achieving them were examined across the literature. In this context, the researchers have identified and synthesized the convergent definitions and their associated strategies, presenting them in Table 1. This table is designed to assist readers by mapping each criterion to its common definition and the primary strategies proposed to achieve it. This allows for a critical reflection on the extent to which the proposed strategies logically align with and fulfill the expectations embedded within the definition of each criterion. The references provided in the table list the articles from which these definitions and strategies were extracted.

Table 1.

Quality Assurance Criteria, Definitions, and Associated Strategies

Criteria	Definition	Strategies of each criterion	Definition
Credibility	Credibility involves the researcher providing evidence to the reader that the findings accurately reflect the study question/s It refers to how believable and appropriate the research account is, especially concerning the participants’ narratives and the researcher’s interpretations	Prolonged engagement	Prolonged engagement involves dedicating substantial time to the field, interacting with participants, and observing their activities to gain deep insights. This method helps researchers build trust and rapport, understand the culture and context, and reflect on potential biases
		Persistent observation	Persistent observation involves maintaining an open-minded attitude, recognizing biases, and practicing self-reflection, while promptly recording detailed notes to enhance data accuracy and analysis. The goal is to identify and focus on the most relevant elements of the study, continuously refining the analysis to ensure depth and thorough understanding
		Triangulation	Triangulation is a technique that offers researchers various strategies to identify patterns of agreement by using multiple methods, observers, or data sources in data collection and analysis. Triangulation can be done in different ways, including methodological triangulation, data triangulation, investigator triangulation, theoretical triangulation, and environmental triangulation
		Member checks	Member checks are defined as the process of verifying the findings of a study (such as recordings, transcripts, interpretations, categories, and subcategories) with the participants, which can be done either formally or informally
		Peer debriefing	Peer debriefing involves validating content shared by study participants through a third-party evaluator, seeking feedback from peers or experts, inviting an independent researcher to analyze the data, and communicating with participants at the end of the study to explain results and reinforce understandings
		Negative case analysis	Negative case analysis is a strategy that involves actively seeking and examining data that contradicts or does not support the researcher’s initial interpretations, focusing on exceptions or variations that do not fit into existing categories
Transferability	Transferability refers to the extent to which the findings of qualitative research can be applied to other contexts or settings It allows readers to determine if the results are relevant to their own situations, considering the applicability of the findings to different environments and groups of respondents	Thick description	Thick descriptions provide detailed accounts of the research context, participants, and methods, including settings, inclusion/exclusion criteria, sample characteristics, and data collection and analysis methods, enabling readers to assess the relevance and applicability of findings to their contexts
Dependability	Dependability ensures that research findings are consistent and reliable over time, with similar results obtainable if the study is repeated. This criterion of trustworthiness highlights the importance of clearly identifying research design and operations, contributing to the enduring nature of the findings. Consistency relates to the notion of dependability, emphasizing the enduring and unwavering nature of research findings across time	Audit trail	An audit trail is a transparent record of all research decisions, changes, and processes, ensuring traceability and transparency throughout the study. It provides evidence of the researcher’s choices and methodologies from the start of the project to the reporting of findings
Confirmability	Confirmability refers to the impartiality and objectivity of research findings, ensuring they are derived from the data and not influenced by the researcher’s biases or preferences It requires the researcher to demonstrate how conclusions and interpretations have been reached, ensuring the findings reflect the participants’ views	Audit trail	An audit trail is a transparent record of all research decisions, changes, and processes, ensuring traceability and transparency throughout the study. It provides evidence of the researcher’s choices and methodologies from the start of the project to the reporting of findings
References: Cruz Neto et al. (Cruz Neto et al., 2024), Ahmed (Ahmed, 2024), Gulati et al. (Verma et al., 2023), Peddle (Peddle, 2022), Johnson et al. (Johnson et al., 2020), Hamilton (Hamilton, 2020), Amin et al. (Amin et al., 2020), FitzPatrick (FitzPatrick, 2019), Williams et al. (Williams et al., 2022), Squires and Dorsen (Squires and Dorsen, 2018), Rettke et al. (Rettke et al., 2018), Korstjensa and Moser (Korstjens and Moser, 2018), Nowell et al. (Nowell et al., 2017), Cypress (Cypress, 2017), Hadi and Closs (Hadi and José Closs, 2016), Birt et al. (Birt et al., 2016), Amankwaa (Amankwaa, 2016), Rodham et al. (Rodham et al., 2015), Noble and Smith (Noble and Smith, 2015), Baillie (Baillie, 2015), Mabuza et al. (Mabuza et al., 2014), Elo et al. (Elo et al., 2014), Konradsen et al. (Konradsen et al., 2013), Goldblatt et al. (Goldblatt et al., 2011), Murphy and Yielder (Murphy and Yielder, 2010), McGloin (McGloin, 2008), Cohen and Crabtree (Cohen and Crabtree, 2008), Roberts et al. (Roberts and Priest, 2006), Tuckett (Tuckett, 2005), Milne and Oberle (Milne and Oberle, 2005), Jacelon and O’Dell (Jacelon and O’Dell, 2005), Tobin and Begley (Tobin and Begley, 2004), Graneheim and Lundman (Graneheim and Lundman, 2004), Cutcliffe and McKenna (Cutcliffe and McKenna, 2004), Wolf (Wolf, 2003), Horsburgh (Horsburgh, 2003), Richards (Richards, 2002), Clayton and Thorne (Clayton and Thorne, 2000), Mays and Pope (Mays and Pope, 1995), Gliner (Gliner, 1994), Shmerling et al. (Shmerling et al., 1993), Rodgers and Cowles (Rodgers and Cowles, 1993)

Another noteworthy point is that the strategy of reflexivity has been presented in the literature both as a shared strategy across multiple criteria and, at times, as an independent approach for assessing rigor and trustworthiness. The definition derived from the texts is as follows:

Reflexivity involves acknowledging personal biases and preconceptions, questioning our understanding of reality and knowledge based on our perceptions, and examining our relationship.

Table 2 presents findings from studies related to criticisms/challenges concerning paradigmatic and general points, quality across different methodologies in qualitative research, issues with employing various strategies to achieve each criterion, and specific issues such as linguistic differences in studies.

Table 2.

Criticisms/Challenges Related to Quality Assurance in Qualitative Research

Categories	Criticisms/challenges
Paradigmatic/General	⁃ Rooting quality assurance criteria in positivist thinking ⁃ Lack of studies related to the different approaches of qualitative researchers in choosing strategies to ensure the quality of qualitative studies and how to use them ⁃ Disagreement about the appropriate time to apply quality measures during the study ⁃ Lack of studies on the importance of the usefulness and appropriateness of quality assurance measures ⁃ Lack of studies on the importance of the usefulness and appropriateness of quality assurance measures ⁃ Lack of studies on how to apply research quality assurance criteria ⁃ The complexity and contradiction of studies related to the quality assurance criteria of studies ⁃ Using terms related to the quantitative paradigm in qualitative research (such as validity and reliability) ⁃ Using quality assurance criteria in the form of a checklist
Methodologic	⁃ Incompatibility and inapplicability of criteria and strategies with the specific characteristics of each methodology ⁃ Lack of agreement on quality assurance criteria in different qualitative methodologies ⁃ No published recommendations on how the trustworthiness should be checked if the inductive content analysis ⁃ Small sample potentially affecting the transferability or applicability in a case study ⁃ The confusing criteria and terminology in qualitative descriptive studies (case study) ⁃ Use of similar terms and criteria for reliability and validity in both qualitative and quantitative content analysis ⁃
Strategic	⁃ Non-compliance of strategy with the qualitative paradigm ⁃ Non-compliance of strategy with all types of qualitative methodology ⁃ Complexity and difficulty of applying strategy ⁃ Feasibility of implementing the strategy ⁃ Time-consuming implementation of strategy ⁃ Necessity of the prerequisites for application ⁃ Impracticality of strategy in specific situations ⁃ Ethical concerns and issues in applying strategy ⁃ Lack of sufficient guidance for the application of strategy ⁃ Concerns about proper implementation ⁃ Disagreement about how to apply the strategy ⁃ Concerns about the usefulness of the strategy for examining trustworthiness
Specific	⁃ Challenges in accurately conveying participants’ intended meanings across languages, potentially affecting data validity ⁃ Methodological and epistemological issues affecting data trustworthiness due to language differences ⁃ Ethnographic research managing translation difficulties by focusing on shared experiences ⁃ Translation issues hindering trustworthiness in phenomenological studies requiring precise word usage ⁃ Language barriers limiting immersion in data, affecting depth of analysis and interpretation ⁃ Decreased rigor due to the imposition of participants in small-size studies ⁃ Little practical guidance available to devise efficient strategies for maximizing recruitment in small-size studies ⁃ More number of interviewers and analysts reduces rigor in a large data set ⁃ Limited studies on how to manage big data ⁃ The inappropriateness of some strategies in marginalized groups
References: Ahmed (Ahmed, 2024), Gulati et al. (Verma et al., 2023), Peddle (Peddle, 2022), Johnson et al. (Johnson et al., 2020), Hamilton (Hamilton, 2020), Amin et al. (Amin et al., 2020), FitzPatrick (FitzPatrick, 2019), Williams et al. (Williams et al., 2022), Squires and Dorsen (Squires and Dorsen, 2018), Rettke et al. (Rettke et al., 2018), Korstjensa and Moser (Korstjens and Moser, 2018), Nowell et al. (Nowell et al., 2017), Cypress (Cypress, 2017), Hadi and Closs (Hadi and José Closs, 2016), Birt et al. (Birt et al., 2016), Noble and Smith (Noble and Smith, 2015), Baillie (Baillie, 2015), Al-Amer et al. (Al-Amer et al., 2015), Mabuza et al. (Mabuza et al., 2014), Elo et al. (Elo et al., 2014), Konradsen et al. (Konradsen et al., 2013), Jessiman (Jessiman, 2013), Whiting and Sines (Whiting and Sines, 2012), White et al. (White et al., 2012), Goldblatt et al. (Goldblatt et al., 2011), Murphy and Yielder (Murphy and Yielder, 2010), Miyata and Kai (Miyata et al., 2011), McGloin (McGloin, 2008), Cohen and Crabtree (Cohen and Crabtree, 2008), Høye and Severinsson (Høye and Severinsson, 2007), Ferrari (Ferrari, 2006), Tuckett (Tuckett, 2005), Milne and Oberle (Milne and Oberle, 2005), Tobin and Begley (Tobin and Begley, 2004), Draper (Draper, 2004), Wolf (Wolf, 2003), Van Teijlingen (van Teijlingen, 2003), Mill and Ogilvie (Mill and Ogilvie, 2003), Horsburgh (Horsburgh, 2003), Clayton and Thorne (Clayton and Thorne, 2000), Mays and Pope (Mays and Pope, 1995), Gliner (Gliner, 1994), Rodgers and Cowles (Rodgers and Cowles, 1993)

Table 3 presents the criticisms/challenges of implementing quality standards through various strategies. As illustrated, most of the criticisms/challenges focus on the member check and audit strategies.

Table 3.

Criticisms/Challenges Related to Strategies

Criticisms/challenges	Reflexivity	Member check	Audit trail	Peer check	Persistent observation	Prolong engagement	Triangulation	Negative case analysis
Non-compliance of strategy with the qualitative paradigm		✓		✓
Non-compliance and application of strategy in all types of qualitative methodology		✓
Complexity and difficulty of applying strategy		✓	✓	✓	✓	✓	✓
Feasibility of implementing the strategy		✓	✓			✓
Disagreement about how to apply the strategy	✓	✓
Time-consuming implementation of strategy	✓		✓		✓	✓	✓	✓
Necessity of the prerequisites for application	✓	✓	✓		✓	✓	✓	✓
Impracticality of strategy in specific situations		✓	✓
Ethical concerns and issues in applying strategy		✓
Lack of sufficient guidance for the application of strategy			✓
Concerns about proper implementation		✓
Concerns about the usefulness of the strategy for examining trustworthiness		✓		✓	✓	✓

Table 4 presents paradigmatic and methodological solutions and recommendations for ensuring the quality of qualitative research that has been explored in the reviewed articles. The table highlights the use of diverse terminology, unclear protocols, checklists, recommendations, vague strategies, and ambiguous implementation processes for quality assurance strategies across various methodologies and paradigms.

Table 4.

Proposed Solutions/Recommendations Related to Quality Assurance of Qualitative Studies

Categories	Solutions/Recommendations
Paradigmatic/General	⁃ Use rigor instead of trustworthiness⁃ Not using criteria as a uniform checklist⁃ Considering a separate section for the quality assurance report in the study report (article)⁃ Using trustworthiness for quality assurance instead of positivist terms such as ‘reliability’ and ‘validity’⁃ Balance between preserving creativity and achieving a rigorous approach⁃ Replace the term rigor with “goodness criteria” or “trustworthiness criteria”⁃ Focusing on recognizing and embracing emerging criteria rather than seeking consensus⁃ Adapting the concept of rigor to fit the epistemology of qualitative research⁃ Encourage debate to enhance clarity, understanding, and transparency⁃ Avoid simplistic views of concepts like robustness and triangulation as mere confirmation⁃ Demonstrate a logical understanding of actions, methods, and reasons behind them⁃ Shift from narrow methods of ensuring rigor derived mainly from the positivist tradition to a pluralistic approach to legitimize naturalistic inquiry⁃ Considering the quality assurance criteria from the beginning of the research and as part of the process of conducting the study⁃ Applying validity processual approach (ensuring validity through a process-based approach)⁃ Using alternative criteria such as truth value, consistency, neutrality, and applicability instead of validity and reliability⁃ Establishing a formalized evaluation framework⁃ Not using quality criteria and strategy as a fixed set
Methodologic	⁃ The discussion of the trustworthiness of the study should be evaluated using a clear and logical set of criteria⁃ Authors should clearly define their validation terms, as there are various types of qualitative validation, such as trustworthiness, verification, and authenticity⁃ Paying attention to the epistemological basis of each methodology to apply the criteria⁃ Choosing strategies based on the specific characteristics of each methodology⁃ Applying strategies considering the research problem and question⁃ Being flexible in response to the variety of interpretive perspectives and methodologies⁃ Involvement of all researchers in the analysis and listening to the audio recordings to reduce bias in phenomenology studies⁃ Keep audio recordings for a longer duration in case secondary analysis is required in phenomenology studies⁃ One researcher leads the analysis while others monitor the process and categorization⁃ Later meeting of all researchers to discuss and resolve any differing opinions, similar to the pilot phase (18)⁃ Providing a checklist to improve the trustworthiness of a content analysis study⁃ Being systematic and well-organized to enhance the trustworthiness in content analysis⁃ Documentation of all phases of content analysis⁃ Design a matrix for the sampling procedure
References: Gulati et al. (Verma et al., 2023), Hayashi Jr et al. (Hayashi et al., 2021), Amin et al.(Amin et al., 2020), FitzPatrick (FitzPatrick, 2019), Williams et al. (Williams et al., 2022), Squires and Dorsen (Squires and Dorsen, 2018), Nowell et al. (Nowell et al., 2017), Cypress (Cypress, 2017), Hadi and Closs (Hadi and José Closs, 2016), Birt et al. (Birt et al., 2016), Amankwaa (Amankwaa, 2016), Rodham et al. (Rodham et al., 2015), Noble and Smith (Noble and Smith, 2015), Baillie (Baillie, 2015), Al-Amer et al. (Al-Amer et al., 2015), Elo et al. (Elo et al., 2014), Konradsen et al. (Konradsen et al., 2013), Jessiman (Jessiman, 2013), Whiting and Sines (Whiting and Sines, 2012), White et al. (White et al., 2012), Goldblatt et al. (Goldblatt et al., 2011), Murphy and Yielder (Murphy and Yielder, 2010), McGloin (McGloin, 2008), Cohen and Crabtree (Cohen and Crabtree, 2008), Ferrari (Ferrari, 2006), Tobin and Begley (Tobin and Begley, 2004), Cutcliffe and McKenna (Cutcliffe and McKenna, 2004), Wolf (Wolf, 2003), Mill and Ogilvie (Mill and Ogilvie, 2003), Horsburgh (Horsburgh, 2003), Richards (Richards, 2002), Clayton and Thorne (Clayton and Thorne, 2000), Rodgers and Cowles (Rodgers and Cowles, 1993)

Table 5 presents solutions and some applied recommendations for managing the criticisms/challenges raised regarding the use of various strategies in different studies.

Table 5.

Solutions/Recommendations to Ensure the Quality of Qualitative Studies According to Each Strategy

Strategy	Solutions/Recommendations
Reflexivity	⁃ Utilizing an online questionnaire with concise and precise questions, to be filled out monthly by the research team
Member check	⁃ Encourage open dialogue during member checking sessions⁃ Should not be viewed as a verification tool⁃ Recognizability as a strategy for assessing external validity and facilitating knowledge transfer in qualitative research, introduced as a framework to evaluate how well research findings resonate with broader contexts beyond the original study participants⁃ Third-party member checks for increased objectivity⁃ Exercise caution in using the strategy⁃ Employ other strategies in case of potential ethical violations⁃ Use triangulation as an alternative⁃ Maintain anonymity⁃ Conduct member checks in neutral settings⁃ Utilize synthesized member checking⁃ Ensure participant comfort and understanding⁃ Use mind maps
Audit trail	⁃ Record compulsively⁃ Consider writing sessions and summarizing notes⁃ Ensure ease in choosing the type of note⁃ Use a recording system throughout the study⁃ Avoid leaving writing tasks to the future⁃ Record thoughts and feelings and then discuss them⁃ Record the date and time on each note taken during the study⁃ Synchronize notes as much as possible with study data⁃ Write out the abbreviations used⁃ Structured documentation⁃ Use of technology⁃ Peer review engagement
Persistent observation	⁃ Practice of healthy skepticism⁃ Maintain a systematic approach
Triangulation	⁃ Explore and utilize fewer common types of triangulations⁃ Carefully consider and clearly articulate the decision to use triangulation throughout the research process⁃ Recognize the epistemological foundations of the approaches to ensure a genuine mixture of perspectives⁃ Embrace the concept of crystallization to capture the multifaceted reality better
References: Gulati et al. (Verma et al., 2023), Hayashi Jr et al. (Hayashi et al., 2021), Amin et al.(Amin et al., 2020), FitzPatrick (FitzPatrick, 2019), Williams et al. (Williams et al., 2022), Squires and Dorsen (Squires and Dorsen, 2018), Nowell et al. (Nowell et al., 2017), Cypress (Cypress, 2017), Hadi and Closs (Hadi and José Closs, 2016), Birt et al. (Birt et al., 2016), Amankwaa (Amankwaa, 2016), Rodham et al. (Rodham et al., 2015), Noble and Smith (Noble and Smith, 2015), Baillie (Baillie, 2015), Al-Amer et al. (Al-Amer et al., 2015), Elo et al. (Elo et al., 2014), Konradsen et al. (Konradsen et al., 2013), Jessiman (Jessiman, 2013), Whiting and Sines (Whiting and Sines, 2012), White et al. (White et al., 2012), Goldblatt et al. (Goldblatt et al., 2011), Murphy and Yielder (Murphy and Yielder, 2010), McGloin (McGloin, 2008), Cohen and Crabtree (Cohen and Crabtree, 2008), Ferrari (Ferrari, 2006), Tobin and Begley (Tobin and Begley, 2004), Cutcliffe and McKenna (Cutcliffe and McKenna, 2004), Wolf (Wolf, 2003), Mill and Ogilvie (Mill and Ogilvie, 2003), Horsburgh (Horsburgh, 2003), Richards (Richards, 2002), Clayton and Thorne (Clayton and Thorne, 2000), Rodgers and Cowles (Rodgers and Cowles, 1993)

In certain specific situations, solutions/recommendations have been provided to ensure the quality of qualitative studies. These are summarized in Table 6.

Table 6.

Solutions/Recommendations for Ensuring the Quality of Qualitative Studies in Specific Situations

Specific situations	Solutions/Recommendations
The difference between the language of the researcher and the participants	⁃ Allowing participants to provide feedback on the data and confirm the researcher’s interpretations⁃ Making decisions with consideration for cultural perspectives⁃ Developing cultural competence to address various questions⁃ Ensuring accuracy from the selection of the research question to the publication of the study⁃ Collaborating with experts familiar with the specific context⁃ Using a professional interpreter for accurate communication⁃ Implementing a multilingual coding process⁃ Performing independent verification of the translation and coding process⁃ Applying reflexivity to ensure unbiased research⁃ Identifying the positionality of the researcher and interpreter⁃ Forming multilingual research teams with both insiders and outsiders⁃ Learning key phrases in the participants’ native language⁃ Showing empathy to build trust with participants⁃ Creating conceptual or word equivalence in translations⁃ Utilizing multiple translators, triangulating respondents and methods, and involving independent bilingual reviewers⁃ Developing an analysis plan from the outset using various means to ensure credibility⁃ Appropriately budgeting for the study⁃ Collecting data in participants' native language⁃ Having a clear rationale and process for translating qualitative data⁃ Transparently reporting the translation process⁃ Assembling a team of bilingual and bicultural members with expertise in qualitative research⁃ Choosing translators based on clear criteria, including shared origin, language, culture, and beliefs with participants⁃ Ensuring translators are proficient in medical terminology⁃ Prioritizing semantic and content equivalence over literal translation⁃ Considering the overall meaning in translations, especially when there are no direct equivalents⁃ Using a single translator to avoid discrepancies and reduce variation in translated data⁃ Advocating for standardized guidelines for translation in qualitative healthcare research⁃ Creating a record of atypical words not found in dictionaries
Participants selected in small-size studies	⁃ Employ various strategies for participant selection⁃ Build trust with potential contributors⁃ Consider offering incentives⁃ Select participants through face-to-face interactions⁃ Acknowledge the contributions of participants⁃ Provide a detailed description of the study procedure
Large data set	⁃ Strict adherence to specific guidelines⁃ Organizing the study by an individual⁃ Comprehensive documentation of data collection and analysis processes⁃ Maintaining attention to the schedule⁃ Utilizing an iterative process for data collection and analysis⁃ Conducting internal and external audits⁃ Ensuring regular communication among the research team⁃ Allocating sufficient resources for timely project completion⁃ Allowing time for reflection and breaks⁃ Using recording systems or software
Marginalized groups	⁃ Avoiding the use or modification of strategies for specific groups
Computing in qualitative method	⁃ Comparing coding patterns is crucial for clarifying concepts, training teams, and ensuring the reliability of coding⁃ Exporting tables from qualitative analysis into a statistical software package enables further statistical exploration⁃ Utilizing software to merge multiple qualitative projects facilitates comparative or collaborative work, enhancing detailed alignment and integration⁃ Implementing a comprehensive audit system involves tracking the emergence of categories, dating memos or documents, and archiving images of the analysis at each stage
References: Gulati et al. (Verma et al., 2023), Hayashi Jr. et al. (Hayashi et al., 2021), Amin et al. (Amin et al., 2020), FitzPatrick (FitzPatrick, 2019), Williams et al. (Williams et al., 2022), Squires and Dorsen (Squires and Dorsen, 2018), Nowell et al. (Nowell et al., 2017), Cypress (Cypress, 2017), Hadi and Closs (Hadi and José Closs, 2016), Birt et al. (Birt et al., 2016), Amankwaa (Amankwaa, 2016), Rodham et al. (Rodham et al., 2015), Noble and Smith (Noble and Smith, 2015), Baillie (Baillie, 2015), Al-Amer et al. (Al-Amer et al., 2015), Elo et al. (Elo et al., 2014), Konradsen et al. (Konradsen et al., 2013), Jessiman (Jessiman, 2013), Whiting and Sines (Whiting and Sines, 2012), White et al. (White et al., 2012), Goldblatt et al. (Goldblatt et al., 2011), Murphy and Yielder (Murphy and Yielder, 2010), McGloin (McGloin, 2008), Cohen and Crabtree (Cohen and Crabtree, 2008), Ferrari (Ferrari, 2006), Tobin and Begley (Tobin and Begley, 2004), Cutcliffe and McKenna (Cutcliffe and McKenna, 2004), Wolf (Wolf, 2003), Mill and Ogilvie (Mill and Ogilvie, 2003), Horsburgh (Horsburgh, 2003), Richards (Richards, 2002), Clayton and Thorne (Clayton and Thorne, 2000), Rodgers and Cowles (Rodgers and Cowles, 1993)

The overall review of the findings from the studies indicates that the quality of qualitative research has consistently been a concern for researchers. However, the actions taken have not always been consistent, and there have been conflicts of opinion in various areas, including terminology, definitions, and the implementation of strategies. Criticisms and challenges have been more prominent than solutions and recommendations, spanning both paradigmatic and methodological aspects. While some strategies have been proposed and debated, researchers have also encountered specific challenges that have been addressed in particular contexts. Most studies have examined research quality in general, with less emphasis on individual methodologies. Presenting findings across different areas and focusing on study results have helped identify existing evidence and highlight gaps in the literature.

Discussion

This study aims to identify and define the criteria and strategies for ensuring the quality of biomedical qualitative research, explore the associated criticisms and challenges, and examine the proposed solutions and recommendations for addressing these issues. Given the broad scope of findings in the field of quality assurance in qualitative research, the discussion section focuses on key points that highlight the most significant gaps and challenges, clarify future directions, and assist researchers in planning future studies. This approach aims to address these challenges, respond to criticisms, propose solutions, and harmonize existing literature.

The first point to note when addressing quality in qualitative research is the existence of multiple terms in the literature and the lack of agreement on their use, such as validity, reliability, trustworthiness, rigor, credibility, and quality (Cypress, 2017; Elo et al., 2014; FitzPatrick, 2019; McGloin, 2008; Mill and Ogilvie, 2003; Murphy and Yielder, 2010). This lack of consensus has persisted over time and has not diminished in intensity, as evidenced by our findings. The inconsistency in terminology has created several major challenges. Primarily, it has caused confusion among researchers in applying appropriate terms and made it difficult to search for relevant research in this area (Cohen and Crabtree, 2008; Elo et al., 2014; Hadi and José Closs, 2016; Miyata et al., 2011; Murphy and Yielder, 2010; Tobin and Begley, 2004). To address these challenges, some literature has suggested the use of specific terms, such as “rigor” instead of “trustworthiness,” and vice versa which not only seems to have had no effect in solving these challenges but has also created more confusion in the literature (Mill and Ogilvie, 2003).

The next important issue is that, regardless of the terms used to discuss the quality of qualitative studies, the incompatibility of the proposed criteria with the nature of the qualitative paradigm (such as the flexibility of the research design, the nature of the data, or the belief in multiple realities) poses a more serious criticism (Baillie, 2015; Clayton and Thorne, 2000; Cohen and Crabtree, 2008; FitzPatrick, 2019; Hadi and José Closs, 2016; Horsburgh, 2003; Murphy and Yielder, 2010; Rettke et al., 2018; Williams et al., 2022). The diversity of epistemological assumptions in qualitative research, ranging from positivism to constructivism, leads to ambiguity in assessing research quality (Spencer et al., 2014). For instance, in the early work of Lincoln and Guba, they highlighted various aspects of trustworthiness in quantitative research and proposed corresponding criteria for qualitative research (Lincoln and Guba, 1985). To achieve these criteria, they also introduced specific strategies. However, these criteria and strategies, which various researchers later adopted, were criticized for not being practical, effective, or appropriate for qualitative research, as they were derived from quantitative research principles (Konradsen et al., 2013; Tuckett, 2005).

It is important to contextualize the contribution of Guba and Lincoln. While discussions about the validity and rigor of qualitative research had been ongoing, their work in the 1980s provided a pivotal and systematic framework for conceptualizing trustworthiness in naturalistic inquiry (Guba, 1981; Guba and Lincoln, 1981; Lincoln and Guba, 1985; Lockyer, 2008). Their seminal work, which organized criteria into the domains of credibility, transferability, dependability, and confirmability, offered a structured response to criticisms regarding the validity of qualitative findings and became a foundational benchmark for the field. Accordingly, it can be said that some criticisms, such as the incompatibility of strategies with various methodologies (such as participant checks in grounded theory research), were raised regardless of the context in which these criteria were proposed (Konradsen et al., 2013; Whiting and Sines, 2012). It seems that with the development of various methodologies, the criteria and strategies needed to be revised or modified; similar to the action taken by Corbin in 2008 to introduce quality assurance criteria for grounded theory research (Corbin and Strauss, 2008).

The criticisms and challenges also target researchers using quality assurance criteria. The aim of experts like Lincoln and Guba, Corbin and Strauss, or Morse et al. in presenting these criteria has always been to guide researchers in improving the quality of their research right from the outset (Corbin and Strauss, 2008; Guba, 1981; Guba and Lincoln, 1981; Morse et al., 2002). However, some researchers may have merely used these criteria as a tool to demonstrate the trustworthiness of their research in final reports or for external assessment of qualitative research quality (FitzPatrick, 2019; Johnson et al., 2020; Tobin and Begley, 2004). As a result, these criteria have been presented in some texts as evaluation standards and used as checklists or tools. To achieve this, they must make a conscientious effort to conduct their research with integrity and report their findings honestly and accurately, allowing the audience to make informed judgments. (Hamilton, 2020; Murphy and Yielder, 2010, Cohen and Crabtree, 2008). Rather than merely accepting stated implementations as indicators of study quality, evaluators and readers should carefully assess and document how these criteria have been met. (Hamilton, 2020; FitzPatrick, 2019; Amin et al., 2020; Johnson et al., 2020, Cohen and Crabtree, 2008).

Another important aspect of ensuring the quality of qualitative research is that the solutions and recommendations proposed by different researchers in response to criticisms and challenges are not entirely consistent and often contain contradictions. This underscores the need for further research and expert consensus. Despite disagreements, all experts agree on the importance of a researcher’s commitment to methodology. Proper implementation and adherence to study protocols are universally recognized as fundamental to conducting high-quality, ethical research. (Corbin and Strauss, 2008; FitzPatrick, 2019; Guba, 1981; Guba and Lincoln, 1981; Morse et al., 2002). Based on the findings of this review, the numerous contradictions, generalities, and impracticalities prevalent in existing recommendations for ensuring qualitative research quality highlight a critical need for future efforts to focus on developing more coherent, context-sensitive, and actionable frameworks.

Finally, the issue of quality in qualitative research has been a subject of considerable criticism and challenge, reflecting its importance and the attention it has garnered from experts and researchers over time. The vital role of qualitative research in knowledge development and the necessity to align with new approaches, such as open science and responsible research, emphasize the need to address these challenges and implement effective solutions. Based on the findings of this study, the authors suggest that future research should focus on developing a framework for ensuring the quality of qualitative research. This framework should incorporate the criticisms/challenges raised while leveraging existing solutions and the experiences of researchers to enhance the quality of qualitative research.

Limitation of Study

One notable limitation of this study is that it includes only materials and data in the English language.

Conclusion

This study highlights the importance of quality assurance in biomedical qualitative research. Despite the growing recognition of qualitative methods, challenges persist, particularly regarding inconsistent terminology and the dominance of quantitative concepts like “validity” and “reliability.” The lack of a universal definition of quality complicates research efforts, underscoring the need for standardization.

Existing criteria, often rooted in quantitative paradigms, may not align with qualitative research’s flexibility and multiplicity of realities. Early frameworks, such as those by Lincoln and Guba, have been criticized for their applicability to biomedical studies, prompting calls for context-specific adaptations. However, there is unanimous agreement on the necessity of methodological rigor, transparent reporting, and ethical research practices.

To advance quality standards, a comprehensive framework is needed—one that addresses criticisms integrates effective solutions, and reflects diverse research experiences. Embracing open science and responsible research will further strengthen qualitative studies. Given ongoing gaps and disagreements, a large-scale Delphi study is recommended to gather expert consensus and develop clear guidelines for biomedical researchers.