Sage Journals: Discover world-class research

Abstract

Objective: Framework Analysis (FA) and Applied Thematic Analysis (ATA) are qualitative methods that have not been as widely used/cited compared to content analysis or grounded theory. This paper compares methods of FA with ATA for mobile health (mHealth) research. The same qualitative data were analyzed separately using each methodology. The methods, utility, and results of each are compared, and recommendations made for their effective use. Methods: Formative qualitative data were collected in eight focus group discussions with physicians and nurses from three hospitals in Bangladesh. Focus groups were conducted via video conference in the local language, Bangla, and audio recorded. Audio recordings were used to complete a FA of participants’ opinions about key features of novel mHealth application (app) designed to support clinical management in patients with acute diarrhea, called FluidCalc: Rehydration Calculator for Acute Diarrhea. The resulting framework analysis was shared with the app design team and used to guide iterative development of the product for a validation study of the app. Subsequently, focus group audio recordings were transcribed in Bangla then translated into English for ATA; transcripts and codes were entered into NVivo qualitative analysis software. Code summaries and thematic memos explored the clinical utility of FluidCalc including clinicians’ attitudes about using this decision support tool. Results: Each of the two methods contributes differently to the research goal and have different implications for an mHealth research timeline. Recommendations for the effective use of each method in app development include: using FA for data reduction where specific outcomes are needed to make programming and design decisions and using ATA to capture the more nuanced issues that guide use, product implementation, training, and workflow. Conclusions: By describing how both analytical methods were used in this context, this paper provides guidance and an illustration for use of these two methods, specifically in mHealth design.

Keywords

framework matrix thematic analysis qualitative data analysis mHealth

Introduction

Qualitative methods, long a key approach in the social sciences, has expanded into the field of medical research over the last two decades, becoming an essential component of intervention and public health research. Multi-disciplinary and mixed methods studies are increasingly used in applied health research with several advocating for the use of both qualitative and quantitative methods (Barbour, 1999; Molina-Azorin, 2016; Morgan, 2015; Poses & Isen, 1998; Regnault et al., 2017). While less common in mobile health (mHealth) research, qualitative methods contribute much needed understanding of how users interact with mobile products and can play an important role in the development and design of mHealth systems. Qualitative data provide insight into how mobile and digital health systems and tools are used and can address shortcomings that occur when intervention and product design rely solely on quantitative methods (Barbour, 1999; Molina-Azorin, 2016; Morgan, 2015; Regnault et al., 2017).

There are a variety of qualitative methods a clinical researcher can incorporate into their research design, including content analysis, narrative analysis, and thematic analysis. Choosing the most appropriate research methodology for a project depends on several factors including the research timeline and how the data analysis will contribute to research aims. Linking the correct analytical strategy to the research task is therefore an essential component of qualitative method design and implementation. Framework Analysis (FA) and Applied Thematic Analysis (ATA) and are two more recently articulated analytic qualitative methods. A review of qualitative data analysis methods of studies published in Human Resource Development Quarterly between 1990 and 2019 and found that out of 59 qualitative studies, more than half used content analysis or grounded theory as their qualitative analysis approach with only 5.1% applying an ATA approach (Lester et al., 2020). For both of these forms of qualitative analysis, there is no one programmatic process that exactly guides all analyses (Braun & Clarke, 2006; Collaço et al., 2021; Gale et al., 2013; Ritchie & Spencer, 1994; Ritchie et al., 2007; Ward et al., 2013). The comparative lack of literature on FA, and the relatively recent articulation of each of these methods – specifically as compared to older approaches such as grounded theory, narrative analysis, or content analysis for example – may cause novice researchers to be unsure of how or when to incorporate these approaches into their research design (Braun & Clarke, 2006; Collaço et al., 2021; Gale et al., 2013; Nowell et al., 2017; Ritchie & Spencer, 1994; Ritchie et al., 2007; Ward et al., 2013).

Framework analysis, also known as framework matrix analysis, is a highly structured approach for analyzing qualitative data developed by Jane Ritchie and Liz Spencer in the United Kingdom for use in social policy research in the late 1980s (Ritchie & Spencer, 1994). A distinguishing feature that differentiates FA from other qualitative methods is the charting of data into a matrix; a spreadsheet which organizes participants by rows and thematic codes in columns. Codes in this process often closely align with questions asked in the qualitative topic guide used to collect data (Chakrapani et al., 2017). In the matrix charting process, participant comments are summarized in spreadsheet cells (Collaço et al., 2021; Pope et al., 2000; Ritchie & Spencer, 1994; Ritchie et al., 2007; Srivastava et al., 2009; Ward et al., 2013). By creating a matrix, researchers can easily compare data across cases and within individual cases, with ease, providing a visually straightforward method to recognize patterns (Collaço et al., 2021; Gale et al., 2013; Ritchie et al., 2007; Srivastava & Thomson, 2009). This both reduces the amount of data for final review and provides researchers with a comprehensive perspective in which differences and similarities within the data can be easily identified. This flexibility adds more depth and understanding of the phenomenon under study (Collaço et al., 2021; Gale et al., 2013; Ritchie et al., 2007; Srivastava et al., 2009; Ward et al., 2013). Matrix development is not limited to interview data, as focus group participant comments can also be charted into matrix cells. FA provides a systematic approach without requiring special software or theoretical knowledge, and is especially useful when multiple researchers from multi-disciplinary and decentralized teams are working on a project (Gale et al., 2013; Ward et al., 2013). It is particularly well-suited to mHealth research, as it synthesizes large amounts of data into a format that can be easily reviewed by mobile application (app) software developers to guide product development.

Applied thematic analysis is a flexible approach to qualitative data analysis as it can be used within different theoretical frameworks and modified for a study’s needs (Braun & Clarke, 2006; Braun et al., 2019; Smith & Firth, 2011; Vaismoradi et al., 2013). This flexibility allows researchers across a range of disciplines to apply various theories and perspectives when conducting a thematic analysis, resulting in meaningful themes and rich insight (Lester et al., 2020). Similar to other qualitative analysis approaches, ATA is especially useful rich and detailed datasets, allowing researchers to identify, analyze, and report patterns and themes (Braun & Clarke, 2006; Lester et al., 2020). Though sometimes criticized for lacking depth and transparency in how themes are developed, researchers who are relatively unfamiliar with qualitative analysis may find this approach particularly appealing as it does not require detailed technical knowledge, unlike other methods (Attride-Stirling, 2001; Braun & Clarke, 2006; Braun et al., 2019; Nowell et al., 2017). Aside from its accessibility, thematic analysis forces the researcher to take a well-structured approach to handling data, helping to produce a clear and organized final report (Braun & Clarke, 2006; Nowell et al., 2017).

Applied thematic analysis is a particularly pragmatic research application of thematic analysis which is useful in public health and other applied qualitative research contexts. ATA uses many of the tools of grounded theory, phenomenology, and other qualitative approaches (Guest et al., 2011). Articulated as a single methodological perspective by Guest et al. (2011), this analytical methodology provides a systematic approach for coding textual data and identifying themes from qualitative data (Guest et al., 2011). First, both deductive and inductive codes are used to sort or segment the data into key topics, and the content of these coding topics is then read in aggregate. The analyst next searches for patterns in the data by attending to repeated content, and whether certain types of participants are linked with particular concepts (Braun & Clarke, 2013, 2022).

The aim of this paper is to describe and compare the strengths and application of FA and ATA methods using qualitative data from a study that developed a novel mHealth clinical decision support tool (CDST) app (FluidCalc: Rehydration Calculator for Acute Diarrhea) for clinical management of dehydration in patients with acute diarrhea in resource-limited settings. Each type of analysis contributes differently to the research goal and each has different implications for the research timeline. We provide guidance for researchers interested in qualitative research for mHealth design, including an illustration of the practical use of these two methods, in order to make recommendations for their optimal use.

Background and Research Context

FluidCalc Development

The “Novel Innovative Research for Understanding Dehydration in Adults and Kids” (NIRUDAK) is an ongoing research study developing a novel mHealth CDST app for use in the treatment and assessment of dehydration severity in patients with acute diarrhea in resource-limited settings. The FluidCalc app uses several clinical diagnostic models developed by this research team for assessment of dehydration severity in patients with acute diarrhea over 5 years of age and in children under 5 years of age (Lee et al., 2021; Levine et al., 2015, 2021). This research team previously incorporated the clinical models’ algorithms into an mHealth prototype, derived from a mHealth CDST (‘Rehydration Calculator’) that adapted paper-based World Health Organization (WHO) guidelines to an mHealth platform (Garbern et al., 2022; Haque et al., 2017). The prototype version of the app allows clinicians to enter a patient’s symptoms into an input screen, and displays the patient’s dehydration severity level and specific treatment recommendations on the output screen. Once validated, FluidCalc will enable dehydration severity level assessment (none, some or severe) and improve the management of patients with acute diarrhea in resource-limited settings.

Study Design and Setting

To develop FluidCalc from the prototype, formative qualitative data were collected in a series of focus group discussions (FGDs) from November to December 2020 among clinicians working at three different types of hospitals in Bangladesh: (1) the International Centre for Diarrhoeal Disease Research, Bangladesh’s (icddr,b) Dhaka Hospital (a private specialty hospital), (2) Narayanganj General Victoria District Hospital (a public referral hospital), and (3) Shaheed Ahsan Ullah Master General Hospital (also known as Tongi Upazilla/Subdistrict Hospital, a public community level hospital). The focus groups solicited feedback from nurses and physicians at each hospital on the following: anticipated clinical utility of FluidCalc, current use of mHealth and other CDS tools, factors important to clinicians when treating diarrheal patients, feedback on preliminary app design and content. The aim of qualitative data collection was to seek feedback on the app for further development prior to a pilot test and subsequent use in a validation study.

Due to travel restrictions imposed by the COVID-19 pandemic, all data were gathered virtually, via Zoom, and facilitated by a member of the Bangladesh-based research team. The Bangladesh team included a physician, an anthropologist, and two research assistants, one of whom had experience teaching and translating English. They were supported by the US-based project coordinator and co-investigators (a clinician and a medical anthropologist). The US and Bangladesh teams adapted the focus group agenda and translated it into Bangla. Several weeks of qualitative training, including facilitation practice and remote data collection protocol development took place before the first focus group. A total of eight FGDs were conducted; two FGDs were conducted at each of the district and subdistrict hospitals (one with nurses and the other with physicians) and four additional FGDs were conducted with specialty providers (two with nurses and two with physicians). The number of participants per focus group was kept deliberately low, each consisting of two to four participants, in keeping with best practices for remote FGDs (Archibald et al., 2019; Gray et al., 2020).

Data Collection

Facilitators used a written focus group agenda (also called a topic guide) to direct the discussion which began by asking participants about their current use of mHealth tools, and then presented a standardized diarrhea patient case. The patient case vignette was used to generate discussion about essential information clinicians use when managing diarrheal patients. Specific feedback solicited included: how participants would treat the case (including when and how to provide fluids), whether to treat the patient in the hospital versus outpatient, and when antibiotic use should be considered. After the patient case, a 2-minute video of the prototype app was shown to participants which demonstrated key app features, patient data entry into the input screens, and interpretation of the recommended treatment and rehydration from the output screens.

We collected general feedback on the app based on the video demonstration and by directing participants to information about specific screens using a PowerPoint presentation displaying still screen shots, known as “cards”. Feedback was solicited on the input screen, format for entry of the chief complaint, dehydration assessment and medical danger signs, inclusion of medications and allergies, presentation of the dehydration and fluid deficit information, and additional treatment recommendations including guidance on the administration of zinc, vitamin A, and antibiotics. In this conversation, we asked about the availability of two specific resources: mid-upper arm circumference (MUAC) and systolic blood pressure (SBP). Since these predictors are used by one of the app algorithms to calculate dehydration severity and treatment recommendations, the team needed to know if these tools were readily available and currently in use in each of the treatment contexts. We then showed side-by-side comparisons of two different versions of the input screen, dehydration assessment, and output screen fluid deficit cards and asked participants which they preferred (Appendix 1). Finally, feedback on participants’ expectations of the use of the app in clinical care, including during outbreak situations (e.g., cholera outbreak), was obtained. This discussion also considered whether other healthcare workers such as pharmacists or community health workers (CHWs) could use the app, and in what contexts. Physicians were also asked to evaluate the likelihood of correct treatment, overtreatment, and undertreatment using the WHO's Integrated Management of Adolescent and Adults Illnesses (IMAI) algorithm versus the NIRUDAK models by comparing three possible prediction cut points for classifying patients with severe dehydration within FluidCalc (Appendix 2) (Rosen et al., 2022; WHO, 2011). Each discussion was audio recorded for transcription and translation by the local research team.

Data Analysis

Framework Analysis

FA was conducted using Excel spreadsheets and review of the FGD audio recordings, which were in Bangla. Eight spreadsheets were initially created, one for each focus group. In each spreadsheet, the rows were participant IDs (one per row) and the columns were key elements the app developers wanted feedback on, and based on the qualitative agenda questions. There were also columns that recorded choices between two different layouts or presentation of the dehydration assessment, input and output/fluid deficit screens. Matrices for the physician groups included a column for preferences regarding over- versus under-treatment (i.e., fluid under- or over-resuscitation). A total of fourteen columns were used for elements including “medication/allergies”, “rehydration bar”, “MUAC available”, “SBP available”.

The Bangladesh based team listened to each audio in Bangla and then summarized participant comments in the corresponding cells. This resulted in a matrix in which reading across a column one can see all of one participant’s comments. Reading down a column, one can see – and, importantly, easily compare – what all of the participants thought of a particular feature, or what choice they made when asked to select between different sample screens. A brief summary was written in the final cell of each column. For example, in one summary it was noted that three of four participants in a focus group chose layout A (rather than layout B) for the input screen. Similar summary cells were written for each column allowing us to easily track trends in the participant’s choices and recommendations. For example, stating: “Three out of six participants believed the rehydration bar would be helpful. The other three participants had no comment”. An overview of the FA process is summarized in Figure 1.

Figure 1.

Summary of framework analysis process.

Applied Thematic Analysis

The transcription and translation process required several steps. Audio recordings were first transcribed into Bangla transcripts, which were then translated into English by a research team member proficient in written and spoken English. English transcripts were reviewed for completeness and accuracy by a third team member in Bangladesh. US-based research team members then read the English transcripts, notating any areas for which they needed clarification, to ensure that meanings and idiomatic speech forms were well understood. The Bangladesh-based research team reviewed and resolved all translation clarification requests. Once these were addressed, the de-identified English transcripts were considered finalized and used in the coding and ATA process.

Deductive codes were created using the qualitative research agenda. Codes were created for each of the key questions and sub-questions in the agenda, including: current experience with mobile medical apps, presentations essential to treating diarrhea, responses to the case scenario and specifics comments about app design. We created a top level (parent) code for each of the app screens, with second level (child) codes for specific elements of the screens (Appendix 3A shows a sample of the coding structure for the app design).

The initial codes were used by two analysts, who independently coded the first focus group transcript. The coders met to review and compare their individual coding. In this process, the two coders came to an agreement on the codes. After coding the initial transcripts, inductive codes were added that captured comments about the importance of clinical judgement in treating diarrhea as well as codes that focused on the practical aspects of using the app, including comorbid illness, treating patients of different ages, and barriers and facilitators of app use. The finalized code structure was used on each of the eight transcripts (initial transcripts were re-coded to apply subsequent inductive codes). Two coders independently coded each transcript, then met to review their codes. Finalized, agreed-upon codes were entered into NVivo software for analysis (QSR International Pty Ltd, 2012).

When all the data were coded, the analysts wrote summaries of key codes (Appendix 3B). To do so they reviewed, in aggregate, all of the transcript passages assigned to a given code and wrote a prose summary of the participant comments in those passages. Each code summary followed the same organization, presenting data separately for specialty hospital physicians, specialty hospital nurses, district/subdistrict hospital physicians, and district/subdistrict hospital nurses. This structure allowed comparison of comments on each app component or use topic by clinician and hospital-type. A total of 15 summaries were written by the analysts then shared with the principal investigator (PI) and app developers. Summaries were then used to write several thematic memos (Appendix 3C). Memos gathered data from several code summaries and were used to write the team’s first qualitative paper which focused on designing FluidCalc (Rosen et al., 2022). The final ATA documents were written summaries of selected codes, which allowed the researchers to focus on the logic and explanation behind many of those same choices. For example, a specialty hospital physician said: “many times…digital blood pressure for children are not available. In that case, getting these two things [MUAC, SBC] accurately will be a bit difficult if I want to use if for mass population. I think the things [MUAC and SBC] are good, but to use in mass population is a bit difficult.” An overview of the ATA process is summarized in Figure 2.

Figure 2.

Summary of applied thematic analysis process.

Results

Framework Analysis

The framework matrices were presented to the research team and used to guide decisions for development of the app. Despite the significant reduction from the transcript data that the matrices provide, it was found that eight matrices (one for each focus group) were not the ideal organization for presenting data to the research team. Therefore, matrices were combined to create a total of four: one for the specialty hospital physicians, one for the specialty hospital nurses, one for district/subdistrict hospital physicians, and one for district/subdistrict hospital nurses. This organization made it easier to look at patterns in the data and app preferences by user category. Finally, the matrices were further reorganized in a single document containing all the feedback in the summary cells compared by both hospital type and clinician role (Table 1).

Table 1.

Framework Analysis Matrix and App Development Decisions.

FA summaries for each participant group were combined into a final matrix. The last column illustrates the final decision that was made by researchers based on the participant’s feedback.

The combined matrices were shared with the app development team in a meeting in which the qualitative analysts presented each matrix, summarizing the participants’ opinions, noting when there was agreement, if that was uniform, and where there were differences of opinion. The lead app designer and the PI then identified changes to be made in the app based on the summary results. In this discussion, the qualitative research team represented the opinions of the participants, explaining the participants’ logic and reasoning. The project coordinator and app designer created summary documents that tracked the changes to be made. Table 1 provides a summary of the framework matrix and app development decisions made based upon it.

Applied Thematic Analysis

Several key themes emerged using ATA, including: details of participants’ current experiences with other CDSTs as well as their overall perception of the app’s clinical utility, including barriers and facilitators of app use. ATA included detailed, specific consideration of the use of guidelines for diarrheal disease management, including the possibility of fluid over- and under-treatment. This specific focus was developed partly because of the research teams’ interest in dehydration prediction model development overall, and because of the desire to refine how the app used guidelines for the algorithms that drive treatment recommendations. After participant feedback on key app features were summarized using FA, we also developed thematic codes and wrote summaries to look in detail at a few specific features, including age, danger signs, and dehydration assessment. Details of the thematic analysis have been previously published (Rosen et al., 2022). That paper uses direct participant quotes for illustration and as evidence for participants’ thought processes and the decision-making behind their opinions and choices (Rosen et al., 2022). It provides context about clinicians’ willingness to use FluidCalc, suggestions for needed training, options about which types of clinicians could and should use the app, and details about the role of clinician experience versus treatment models for guiding patient care. A timeline of data collection, data analyis, and app updates is summarized in Figure 3.

Figure 3.

Timeline of data analysis.

Discussion

Qualitative analysis has become an essential tool in global clinical and public health research. The increase in its application and the articulation of a variety of data collection and analysis methods presents researchers with many choices for analysing qualitative research data. As qualitative research can be notoriously time consuming, a challenge is matching the qualitative analysis methods to the needs of the research project and overall project timeline. The efficiency of qualitative data analysis is a particular concern in formative qualitative research because accomplishing subsequent project aims depends on the timely completion of the qualitative analysis. Researchers therefore need to be aware of the various qualitative analysis methods available in order to decide which approach is most suited to the needs and the timeline of the research project.

Framework analysis uses focused, brief summaries created by analysts which track key information needed to guide decision-making, rather than working in narrative text (i.e., transcripts using the participants’ own words). In this project, the app development team needed a summary of the focus group participants’ reactions to specific components of the app within two months of the completion of the FGDs to stay on track for the project’s timeline, necessitating a quick turnaround of data. The use of audio review and a simple excel spreadsheet format allowed the Bangladesh-based team to quickly produce the needed results. It also provided a format that was easily shared with the PI and app developers who would not have been able to either listen to audios in Bangla or to read eight focus group transcripts, even if there had been time to transcribe and translate them. Though FA can be time intensive especially if transcription of audio files is required to complete the matrices or when working with large datasets. In contrast, while ATA is a pragmatic approach to qualitative analysis, it is still a time-consuming process. Its strengths include attending to the nuances of participants' discussion and being able to work in verbatim transcripts and use direct quotes. Challenges for ATA use includes the time it takes to complete the analysis, including to translate, transcribe, clean and deidentify data, and then to double code, reconcile coding, enter data, write code summaries and thematic memos for analysis.

Charting data in a FA requires significant data reduction. Participant’s own words are rarely used. Instead, several paragraphs or pages of discussion are reduced to the key relevant points. Matrix cells capture what choice a participant makes, but not all the nuances in a discussion of why that choice might be made. This was appropriate for app development. The process was also mostly limited to deductive data – that is, data that came from the questions the researchers asked, and the answers participants gave. While inductive, or emergent, data certainly could have become matrix columns, we did not find that was the case in our project. In contrast, inductive codes and an emergent topic became a focus of the thematic memos in our ATA process. Perhaps because working directly with transcripts, and therefore in the participant’s own words, several inductive codes were developed. In fact, one of them, the importance of clinician experience in patient care, became a key focus of our first paper.

While qualitative research is not statistically representative, and does not usually quantify data or present frequencies, in some opinion or consensus-based work, it is necessary to poll opinions and track preferences (Pope et al., 2000). This was the case in these focus groups, and the resulting FA matrix, and summaries made this easy to do and simple to track.

Framework analysis is more targeted, and the column headers (an analog to ATA codes) are usually based on questions from the qualitative topic guides used to facilitate the focus groups (Charkrapani et al., 2017). FA is therefore more deductive-leaning, while ATA more inductive-leaning (Pope et al., 2000). Charted FA cell content focuses on the outcomes of participants’ choices whereas ATA focuses on the reasoning and discussions that informed those choices. FA is data reduction, and analysis proceeds from summaries of what participants have said. ATA uses participants own words. As a result, FA may streamline collaboration between qualitative researchers and design teams or other non-qualitative researchers because there is less data for these collaborators to review (Ward et al., 2013). Counting participant responses is easy to do and relatively simple to track and compare across types of participants with in an FA. Key differences between the use of FA and ATA are also summarized in Table 2 below.

Table 2.

Key differences Between FA and ATA.

	Framework analysis	Applied thematic analysis
Input	Audio recordings or text (i.e., transcripts)	Narrative text (i.e., transcripts)
Output	Brief summaries	Detailed summaries and memos
Timeline	Quicker turnaround especially if not transcribing	Time consuming as it requires transcribing (and sometimes translating) audio recording, double and reconcile coding
Software needed	Any spreadsheet software (i.e., Microsoft excel, iOS numbers, NVivo)	Qualitative analysis software (i.e., NVivo)
Level of detail	Limited/streamlined as FA is data reduction-based analysis	Very detailed as ATA uses participant’s own words
Type of feedback	Brief summaries allow for less detailed feedback and overall summary of how many participants support recommendations	Detailed summaries provide better understanding and logic of participants decisions/recommendations. Able to use more direct quotes to support findings
Analytical reasoning	Deductive-leaning	Inductive-leaning

Taken together, the two approaches allowed the analysis team to provide a thorough assessment of how many participants endorsed particular app responses and recommendations (FA), the changes the PI and app development team made based on those recommendations (FA summary documents), and the logic and reasoning behind those choices including how they might impact product use (ATA analysis).

Concerns that guided the team’s choice of which analysis method to use for which analysis task include timeline, deductive and inductive data needs, and practicalities such as software access, transcription, translation, and the decentralized study team. The FA analysis was conducted within six weeks of completing the focus groups. Transcription, translation, and transcript verification could not have been accomplished in that time. Audio review of the Bangla focus groups was used, and an excel spreadsheet rather than a formal framework matrix software tool, such as the one build into NVivo, was also used because Excel did not take additional time to learn. These choices allowed the Bangladesh team to work independently and efficiently. Weekly meetings of the full qualitative team were held to review data as it was charted in the matrices, and then both US and Bangladesh researchers participated in final review, summarization, and presentation of the data to the PI, Co-Investigators, and app development team.

Limitations and Strengths

The COVID-19 pandemic necessitated the use of Zoom for these remote focus groups, and poor internet connectivity limited participants to those who were able to connect at the scheduled time. These qualitative data were collected in Bangla and translated into English for use by this international global health team. Translation always presents challenges and limitations include the possibility of having meaning lost in the translation process. In this project, the Bangladesh based team, which included a physician, an anthropologist, and an English teacher, was a particular strength. While traditionally most framework analyses use transcripts, the matrices were created using a review of the Bangla audios directly. Though a departure from other uses of FA, conducting the analysis using review of responses in the actual language of the discussion, may in fact have been a strength, particularly given the expertise of the Bangladesh based team and the need for a quick turnaround of data to the app development team.

Conclusion

Framework analysis and ATA were used on the same qualitative data for comparison of these two analytical methods as part of a research study developing a new mHealth tool for diarrhea management in resource-limited settings. FA using Bangla audio recordings allowed for more focused, rapid analysis, important for immediate use by an app development team to guide app development. A subsequent ATA using English transcripts, focused on the context of app use, including experiences with other apps and clinical decision support tools, the overall utility of the app in clinical care, and barriers and facilitators of the app use. Each analysis method was complementary and suited the needs and timeline of this project.

Supplemental Material

Supplemental Material - Use of Framework Matrix and Thematic Coding Methods in Qualitative Analysis for mHealth: The FluidCalc app

Supplemental Material for Use of Framework Matrix and Thematic Coding Methods in Qualitative Analysis for mHealth: NIRUDAK Study Data Rochelle K. Rosen, Monique Gainey, Sabiha Nasrin, Stephanie C. Garbern, Ryan Lantini, Nour Elshabassi, Sufia Sultana, Tahmida Hasnin, Nur H. Alam, Eric J. Nelson, and Adam C. Levine in International Journal of Qualitative Methods

Footnotes

Acknowledgments

The authors thank all study participants and the study staff at icddr,b Dhaka Hospital who were instrumental in collecting the data used in this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided through grants from the National Institute for Health (NIH) National Institute for Diabetes and Diarrheal and Kidney Diseases (NIDDK), (PI Levine, 1R01DK116163-01A1). This research was supported in part by NIH/NIAID R25AI140490.

Ethical Approval and Consent to Participate

Ethical approval for the formative phase of the NIRUDAK Study was obtained from the International Center for Diarrhoeal Disease Research, Bangladesh (icddr,b) and the Lifespan (Rhode Island Hospital) Institutional Review Boards.

ORCID iD

Monique Gainey

Supplemental Material

Supplemental material for this article is available online.

References

Archibald

M. M.

Ambagtsheer

R. C.

Casey

M. G.

Lawless

(2019). Using Zoom videoconferencing for qualitative data collection: Perceptions and experiences of researchers and participants. International Journal of Qualitative Methods, 18, 160940691987459. https://doi.org/10.1177/1609406919874596

Attride-Stirling

(2001). Thematic networks: An analytic tool for qualitative research. Qualitative Research, 1(3), 385–405. https://doi.org/10.1177/146879410100100307

Barbour

R. S.

(1999). The case for combining qualitative and quantitative approaches in health services research. Journal of Health Services Research Policy, 4(1), 39–43. https://doi.org/10.1177/135581969900400110

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun

Clarke

(2013). Successful qualitative analysis: A practical guide for beginners. Sage Publications. Accessed from: https://us.sagepub.com/en-us/nam/successful-qualitative-research/book233059

Braun

Clarke

(2022). Thematic analysis: A practical guide. Sage Publications. Accessed from: https://us.sagepub.com/en-us/nam/thematic-analysis/book248481

Braun

Clarke

Hayfield

Terry

(2019). Thematic analysis. In Liamputtong

(Ed.), Handbook of research methods in health social sciences (pp. 843–860). Sage Publications.

Chakrapani

Newman

P. A.

Shunmugam

Mengle

Nelson

Rubincam

Kumar

(2017). “Like holding an umbrella before it rains”: Acceptability of future rectal microbicides among men who have sex with men in India-A modified technology acceptance model. Qualitative Health Research, 27(8), 1236–1248. https://doi.org/10.1177/1049732317697947

Collaço

Wagland

Alexis

Gavin

Glaser

Watson

E. K.

(2021). Using the framework method for the analysis of qualitative dyadic data in health research. Qualitative Health Research, 31(8), 1555–1564. https://doi.org/10.1177/10497323211011599

10.

Gale

N. K.

Heath

Cameron

Rashid

Redwood

(2013). Using the Framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 117. https://doi.org/10.1186/1471-2288-13-117

11.

Garbern

S. C.

Nelson

E. J.

Nasrin

Keita

A. M.

Brintz

B. J.

Gainey

Badji

Nasrin

Howard

Taniuchi

Platts-Mills

J. A.

Kotloff

K. L.

Haque

Levine

A. C.

Sow

S. O.

Alam

N. H.

Leung

D. T.

(2022). External validation of a mobile clinical decision support system for diarrhea etiology prediction in children: A multicenter study in Bangladesh and Mali. eLife, 11, Article e72294. https://doi.org/10.7554/eLife.72294

12.

Gray

Wong-Wylie

Rempel

Cook

(2020). Expanding qualitative research interviewing strategies: Zoom video communications. The Qualitative Report, 25(5), 1292–1301. https://doi.org/10.46743/2160-3715/2020.4212

13.

Guest

MacQueen

K. M.

Namey

E. E.

(2011). Applied thematic analysis. Sage Publications.

14.

Haque

Ball

R. L.

Khatun

Ahmed

Kache

Chisti

M. J.

Sarker

S. A.

Maples

S. D.

Pieri

Vardhan Korrapati

Sarnquist

Federspiel

Rahman

M. W.

Andrews

J. R.

Rahman

Nelson

E. J.

(2017). Evaluation of a smartphone decision-support tool for diarrheal disease management in a resource-limited setting. PLoS Neglected Tropical Diseases, 11(1), Article e0005290. https://doi.org/10.1371/journal.pntd.0005290

15.

Lee

J. A.

Gainey

Kanekar

S. S.

Barry

M. A.

Nasrin

Alam

N. H.

Schmid

C. H.

Levine

A. C.

(2021). Continuous diagnostic models for volume deficit in patients with acute diarrhea. Tropical Medicine and Health, 49(1), 70. https://doi.org/10.1186/s41182-021-00361-9

16.

Lester

J. N.

Cho

Lochmiller

C. R.

(2020). Learning to do qualitative data analysis: A starting point. Human Resource Development Review, 19(1), 94–106. https://doi.org/10.1177/1534484320903890

17.

Levine

A. C.

Barry

M. A.

Gainey

Nasrin

Schmid

C. H.

Nelson

E. J.

Garbern

S. C.

Monjory

Rosen

Alam

N. H.

(2021). Derivation of the first clinical diagnostic models for dehydration severity in patients over five years with acute diarrhea. PLoS Neglected Tropical Diseases, 15(3), Article e0009266. https://doi.org/10.1371/journal.pntd.0009266

18.

Levine

A. C.

Glavis-Bloom

Modi

Nasrin

Rege

Chu

Schmid

C. H.

Alam

N. H.

(2015). Empirically derived dehydration scoring and decision tree models for children with diarrhea: Assessment and internal validation in a prospective cohort study in Dhaka, Bangladesh. Global Health, Science and Practice, 3(3), 405–418. https://doi.org/10.9745/GHSP-D-15-00097

19.

Molina-Azorin

(2016). Mixed methods research: An opportunity to improve our studies and our research skills. European Journal of Management and Business Economics, 25(2), 37–38. https://doi.org/10.1016/j.redeen.2016.05.001

20.

Morgan

D. L.

(2015). From themes to hypotheses: Following up with quantitative methods. Qualitative Health Research, 25(6), 789–793. https://doi.org/10.1177/1049732315580110

21.

Nowell

L. S.

Norris

J. M.

White

D. E.

Moules

N. J.

(2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 160940691773384. https://doi.org/10.1177/1609406917733847

22.

Pope

Ziebland

Mays

(2000). Qualitative research in health care. Analysing qualitative data. BMJ Clinical Research, 320(7227), 114–116. https://doi.org/10.1136/bmj.320.7227.114

23.

Poses

R. M.

Isen

A. M.

(1998). Qualitative research in medicine and health care: Questions and controversy. Journal of General Internal Medicine, 13(1), 32–38. https://doi.org/10.1046/j.1525-1497.1998.00005.x

24.

QSR International Pty Ltd (2012). NVivo qualitative data analysis Software. Version 10.

25.

Regnault

Willgoss

Barbic

International Society for Quality of Life Research (ISOQOL) Mixed Methods Special Interest Group (SIG) (2017). Towards the use of mixed methods inquiry as best practice in health outcomes research. Journal of Patient-Reported Outcomes, 2(1), 19. https://doi.org/10.1186/s41687-018-0043-8

26.

Ritchie

Spencer

(1994). Qualitative data analysis for applied policy research. In Bryman

Burgess

R. G.

(Eds.), Analysing qualitative data (pp. 173–194). Routledge Publications.

27.

Ritchie

Spencer

O’Connor

(2007). Carrying out qualitative analysis. In Ritchie

Lewis

(Eds.), Qualitative research practice: A guide for social science students and researchers. Sage Publications. https://doi.org/10.4135/9781452230108

28.

Rosen

R. K.

Garbern

S. C.

Gainey

Lantini

Nasrin

Nelson

E. J.

Elshabassi

Alam

N. H.

Sultana

Hasnin

Schmid

C. H.

Levine

A. C.

(2022). Designing a novel clinician decision support tool for the management of acute diarrhea in Bangladesh: Formative qualitative study. JMIR Human Factors, 9(1), Article e33325. https://doi.org/10.2196/33325

29.

Smith

Firth

(2011). Qualitative data analysis: The framework approach. Nurse Researcher, 18(2), 52–62. https://doi.org/10.7748/nr2011.01.18.2.52.c8284

30.

Srivastava

Thomson

S. B.

(2009). Framework analysis: A qualitative methodology for applied policy research. Journal of Administration and Governance, 4(2), 72–79. https://ssrn.com/abstract=2760705

31.

Vaismoradi

Turunen

Bondas

(2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences, 15(3), 398–405. https://doi.org/10.1111/nhs.12048

32.

Ward

D. J.

Furber

Tierney

Swallow

(2013). Using framework analysis in nursing research: A worked example. Journal of Advanced Nursing, 69(11), 2423–2431. https://doi.org/10.1111/jan.12127

33.

World Health Organization (WHO) (2011). IMAI district clinician manual: Hospital care for adolescents and Adults. Available from: http://apps.who.int/iris/bitstream/10665/77751/3/9789241548290_Vol2_eng.pdf?ua=1

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.54 MB