Exploring the Role of Patient Preferences in Hepatocellular Carcinoma Treatment Decisions: A Qualitative Study

Abstract

Background. Hepatocellular carcinoma (HCC) treatment decisions are becoming increasingly complex as new treatment options emerge. Improved understanding of tradeoffs and patient preferences in treatment decisions will enhance patient–provider discussions, improve treatment development, and inform HCC treatment guidelines. We performed a qualitative study involving patients with HCC and medical providers to assess the role of patient preferences in HCC treatment choices. Methods. Patient participants included those with HCC seen within a single tertiary care center. Provider participants involved physicians and advanced practice providers who cared for patients with HCC from a single center. Baseline and posttreatment patient interviews were conducted by trained qualitative research experts, informed by semi-structured interview guides, and analyzed using thematic analysis with pilot-tested codebooks. Summaries included a narrative description of the themes and subthemes that emerged related to each code, and illustrative quotes were used to highlight each theme. Results. The baseline interview involved 30 patients with HCC (22 of whom participated in follow-up interviews) and 10 providers who cared for patients with HCC. Patients identified factors considered when making treatment decisions included provider confidence and experience, patient prior cancer experiences, other health issues, and faith. Providers primarily discussed the role of Barcelona Clinic Liver Cancer stage, liver function, performance status, and eligibility of liver transplantation in making treatment recommendations. There was general agreement among providers that there is a need to better understand the role of patient values to improve care for HCC. Limitations. Qualitative interviews were limited to patients and providers from a single center. Conclusions. This qualitative study provided information on the variety of values considered by both patients and providers in HCC treatment decisions and the importance of considering tradeoffs of efficacy, toxicity, and inconvenience/costs.

Highlights

Hepatocellular carcinoma (HCC) treatment decisions are often complex and may become increasingly so as new treatment options emerge.

Improved understanding of tradeoffs and patient preferences in treatment decisions will enhance patient–provider discussions, facilitate patient-centered trials to develop new treatments, and inform HCC treatment guidelines.

This qualitative study of patients and providers provided information on the values considered in HCC treatment decisions and the importance of considering the tradeoffs of efficacy, toxicity, and inconvenience/costs.

These insights can be used to develop preference elicitation tools, perform large-scale preference elicitation surveys, and systematically assess and incorporate patient preferences into treatment decisions.

Keywords

liver cancer values choice

Introduction

Hepatocellular carcinoma (HCC) was one of the few cancers with increasing deaths in both men and women in the United States from 1999 to 2018.¹ However, since 2019, deaths have begun to plateau, in part due to improvements in HCC treatments.² For patients with nonmetastatic HCC, there are several locoregional therapies including thermal ablation, transarterial chemoembolization (TACE), transarterial radioembolization, and stereotactic body radiation therapy that provide excellent local control of HCC.^3–5 For intermediate-stage HCC not amenable to locoregional therapy or HCC with major vascular invasion or metastases, immune checkpoint inhibitors plus antiangiogenic therapies have been shown to significantly improve overall survival compared with the prior standard of care, sorafenib.^4–7 There are several ongoing studies assessing combination locoregional plus systemic therapies.⁸ An interim analysis of the EMERALD-1 study demonstrated significantly improved progression-free survival with immunotherapy plus TACE compared with TACE alone but was associated with increased risk of treatment-related hepatic adverse events.⁹ Considering that there are likely to be emerging HCC therapies with significant tradeoffs of efficacy, toxicity, and inconvenience/costs, there is a need to better understand the role of patient values and preferences in treatment decisions for HCC.

Improved understanding of patient preferences and values in HCC treatment decisions could improve care in several ways. The Food and Drug Administration has identified a list of patient preference–sensitive priority areas within oncology device treatments and therapies, which include balancing quality of life with survival and assessing how uncertainty affects risk–benefit considerations.^10,11 Tradeoffs are likely unique in HCC given that patients have underlying liver disease that influences the tolerability of therapy. The comprehension of patient values will help inform patient-centered treatment development, finding unmet needs in therapy development and identifying the most important benefits and risks from a patient perspective. Furthermore, eliciting patient preferences in clinical practice could help inform treatment choice and reduce decisional conflict for patients and providers. Despite these potential benefits, there is a lack of contemporary studies investigating patient preferences in the context of today’s expanded treatment landscape.¹² Given the lack of research to date, qualitative research methods are well suited to understanding decision-making using patients’ and providers’ own words to articulate key considerations that influence treatment decision making for HCC.

In light of the emerging therapies for HCC and the importance of patient values in treatment decisions, we performed a qualitative study to understand both patient and provider perspectives on patient preferences and goals when considering HCC treatment options and how patient preferences could be incorporated into HCC treatment decisions.

Methods

Study Design

This was a qualitative study using a narrative research approach conducted at a single tertiary care center. Qualitative interviews and analyses were carried out by the University of North Carolina (UNC) Connected Health Applications and Interventions (CHAI) Core, a National Institutes of Health (NIH)–funded core that conducts qualitative research, with a specific focus on oncology populations. Our team included qualitative experts, qualitative analysts, and medical providers from multiple specialties. All research was conducted in accordance with the Declaration of Helsinki, and institutional review board (IRB) approval was provided by the UNC IRB (#21-3051). Patients provided written consent prior to the initiation of any study procedures.

Study Sample

For patient participants, we included adult, English-speaking patients with HCC based on American Association for the Study of Liver Diseases (AASLD) guidelines.⁴ To improve generalizability and elicit opinions from a wide spectrum of patients with HCC, there were no requirements based on HCC stage or treatment status. Patients were identified through the multidisciplinary clinic and other subspecialty clinics, and attempts were made to enroll all consecutive patients who met eligibility criteria.

Provider participants included physicians and advanced practice providers who cared for patients with HCC in the fields of hepatology, oncology, interventional radiology, radiation oncology, or transplant surgery.

Data Collection Procedures

A total of 62 in-depth interviews were conducted (30 baseline interviews with patients, 22 follow-up interviews with patients, 10 interviews with providers). Interviews were conducted by 3 trained qualitative researchers through CHAI Core, who are experts on the technical and methodological aspects of qualitative research and collaborate with investigators who are experts in the topical or content areas in which they practice.

Baseline interviews were performed after clinic visits but prior to treatment receipt, and follow-up interviews were performed after patients had received HCC treatment. Baseline interviews were completed a median of 12.5 d (interquartile range [IQR] 10–16 d) after the initial clinic visit (i.e., treatment decision). Eight of 30 patients were not involved in follow-up interviews due to death, transition to hospice, or loss to follow-up.

Interviews were completed over the telephone, lasted approximately 30 to 60 min, and were digitally recorded. The interviewers followed 3 semi-structured interview guides for the baseline patient, follow-up patient, and provider interviews (Supplementary Materials). Interview guides were developed through collaboration with A.M.M. and the CHAI Core qualitative research team. Interview guides were pilot tested among qualitative interviewers. While interview guides were not formally tested with HCC patients or providers prior to data collection, iterative changes were made to the semi-structured interview guides based on feedback from interviews and qualitative expert input. Participants received a $50 gift card upon completing the baseline interview and a $50 dollar gift card after completing the follow-up interview.

Sociodemographic and clinical characteristics were extracted from the electronic medical record. Investigators with a background in hepatology and HCC determined the etiology of liver disease, Child–Pugh score, and Barcelona Clinic Liver Cancer (BCLC) staging for HCC (BCLC 0-D for very early, early, intermediate, advanced, and terminal stages, respectively).

Data Analysis

All interviews were digitally recorded, deidentified, and transcribed by a professional transcription service. To facilitate analysis, interview transcripts were imported into a qualitative research software tool (Dedoose version 9.0.86, Los Angeles, CA, USA, www.dedoose.com). Codebooks were developed based on the research aims and feedback obtained during the data collection process. The initial codebooks were pilot tested by independently coding several transcripts, which led to fine-tuning concept definitions and revising decision rules on when to apply codes. This process continued until replicability occurred across coders. The final versions of the 3 codebooks can be found in the Supplementary Materials.

Interview transcripts were coded and compiled across 62 interviews. Standard consensus coding guidelines were followed where any emerging theme or discrepancy was captured and reconciled through discussion and consensus. Once the coded transcripts were reconciled, code reports were generated for each code, and narrative summaries were written. These code summaries included a narrative description of the themes and subthemes that emerged related to each code. Illustrative quotes were used to highlight each theme.

A priori, we determined that interviews with 30 patients and 10 providers would be sufficient to achieve thematic saturation based on prior studies.¹³ If thematic saturation was not achieved after interviews with these participants, we planned to recruit additional participants. However, qualitative experts determined that the number of code counts in successive transcripts was diminishing, consistent with achieving thematic saturation.

We adhered to the Standards for Reporting Qualitative Research.¹⁴ We followed a number of steps to increase the credibility of research findings including the use of experienced interviewers, investigator triangulation, use of an approved interview guide, audio recording and transcribing interviews, use of qualitative software to manage the coding of transcripts, use of piloted codebooks for all interviews, use of multiple coders, use of summary writing by multiple analysts, use of more than 1 source of data including clinical data, and persistent observation, which involved the development and refining of codes.¹⁵

Results

Characteristics of Study Participants

The baseline characteristics and treatments received by patient participants can be found in Table 1. HCC stage at the time of the interview was as follows: 10% BCLC-0, 27% BCLC-A, 7% BCLC-B, 50% BCLC-C, and 7% BCLC-D. When reclassifying patients who were ECOG 1 (i.e., modified BCLC staging), there were 13% BCLC-0, 40% BCLC-A, 33% BCLC-B, 10% BCLC-C, and 3% BCLC-D. For the purposes of our qualitative analysis, we defined patients based on modified BCLC stage.

Table 1

Baseline Characteristics of Patient Participants

Characteristic		n (%)/Median/Mean
Age, y (mean, range)		70 (67–74)
Male sex (%)		19 (63)
Race (%)	Asian	0 (0)
	Black	6 (20)
	White	24 (80)
	Other	0 (0)
Ethnicity (%)	Hispanic	1 (3)
Ethnicity (%)	Non-Hispanic	29 (97)
County of residence (%)	Rural	12 (40)
County of residence (%)	Urban	18 (60)
Marital status (%)	Living with domestic partner	1 (3)
	Married	13 (43)
	Single	16 (53)
Time from treatment decision to interview, days (median, IQR)		12.5 (10-16)
Treatment type (%)	Thermal ablation	10 (33)
	TACE	8 (27)
	TARE	1 (3)
	SBRT	1 (3)
	Systemic	7 (23)
	No therapy	3 (10)
Liver disease etiology (%)	ALD	3 (10)
	ALD/HCV	4 (13)
	HBV	1 (3)
	HCV	6 (20)
	MASLD	8 (27)
	Met-ALD	5 (17)
	Multiple/Other	3 (10)
Child–Pugh score (%)	5A	12 (40)
	6A	6 (20)
	7B	2 (7)
	8B	2 (7)
	9B	5 (17)
	C	3 (10)
ECOG (%)	0	15 (50)
	1	10 (33)
	2	4 (13)
	3	1 (3)
Modified BCLC stage (%)	0	3 (10)
	A	12 (40)
	B	10 (33)
	C	3 (10)
	D	1 (3)

ALD, alcohol-associated liver disease; ECOG, Eastern Cooperative Oncology Group; HBV, hepatitis B virus; HCV, hepatitis C virus; IQR, interquartile range; MASLD, metabolic dysfunction associated steatotic liver disease; Met-ALD, patients with MASLD who consume greater amounts of alcohol per week; Modified BCLC, Barcelona Clinic Liver Cancer staging system; SBRT, stereotactic body radiation therapy; TACE, transarterial chemoembolization; TARE, transarterial radioembolization.

Follow-up interviews were conducted with 22 patient participants. Eight patients did not complete follow-up interviews due to a lack of response to requests to schedule an interview (n = 3), were deceased (n = 2), transitioned to hospice care (n = 2), or had moved and could not be contacted (n = 1).

Ten providers participated in the qualitative interviews. The provider characteristics are included in Table 2.

Table 2

Characteristics of Provider Participants

Characteristic		n (%)/Median (IQR)
Female sex		3 (30)
Provider type	Physician	9 (90)
Provider type	Nurse practitioner	1 (10)
Specialty	Medical oncology	5 (50)
	Transplant hepatology	2 (20)
	Interventional radiology	1 (10)
	Radiation oncology	1 (10)
	Transplant surgery	1 (10)
Years of experience (median, IQR)		6.5 (5–17)
No. of HCC patients seen per week	≥6	3 (30)
	3–5	6 (60)
	≤2	1 (10)

HCC, hepatocellular carcinoma; IQR, interquartile range;

Patient Perspectives

Factors considered in the treatment decision—pretreatment

Patient participants were asked to discuss the most important things they considered when deciding on an HCC treatment plan. In these conversations, patients described general factors that influenced their decision making, how their personal preferences and values tended to affect their decisions, and questions or concerns they had for their provider about treatment options they discussed.

General factors that were considered when making treatment decisions included 1) provider confidence in treatment recommendation and provider experience performing the treatment procedure, 2) personal experience with cancer treatment (e.g., patient or loved ones treated for liver cancer or other cancers), 3) other health issues, and 4) their faith. Factors that increased patient confidence included the discussion of their treatment plan in a multidisciplinary tumor board. Representative quotations describing these factors are included in Table 3.

Table 3

General Factors Considered in the Treatment Decision

Topic	Representative Quotations
Provider experience and patient confidence in recommendations	He did tell me that they do a lot of them up there, and . . . I have confidence in my doctors, and they’ve been great through this whole thing (Intermediate stage, TACE) No, they did a very good job of describing each one in detail so the one that they choose for me would be the one that I would choose for myself. (Early stage, ablation) All I know is that I think he knows what he’s doing, he knows what’s best, because I don’t know nothing about it . . . I’m in his hands. (Intermediate stage, percutaneous ablation) He went and talked to his powerful minds out there that are on his team, and he came back and told me about it, and I feel pretty confident. (Early stage, percutaneous ablation)
Prior liver cancer treatment experiences of patient or loved ones	We’ve been through cancer in our family and we’re no stranger to chemo and radiation. (Early stage, TACE) I heard the radiation, chemo is really bad, because my ex-wife had it, and she had cancer a year or two ago. And I could tell the chemo was bad on her. (Intermediate stage, percutaneous ablation)
Faith	I’ve always put my whole life in God’s hands. So, as far as putting my life in his hands, it takes it out of mine. So, this is just God giving me the opportunity to have the knowledge to do what I need. And to me, it’s all positive, I don’t know what the outcome is . . . I’m not afraid of it at all. (Early stage, TACE)

TACE, transarterial chemoembolization.

Patients with BCLC-0 (very early stage), BCLC-A (early stage), and BCLC-B (intermediate stage) reported similar preferences for treatment outcomes: effectiveness of treatment to cure/remove cancer, deciding on a treatment that would allow for the continuation of normal activities and/or the chance to maintain their current quality of life, quickest start date for treatment, travel time to receive treatment, recovery time/expectations for treatment, treatment with the lowest burden of care for loved ones, treatment with minimal risk factors, treatment that would maintain transplant eligibility, and cost of treatment. Patients with BCLC-C (advanced stage) HCC identified the following factors: continuing with their normal routines and/or maintaining a certain quality of life, effectiveness of a treatment to control the cancer from spreading, treatment that would prolong their life, and recovery time/expectations for treatment. Representative quotations are found in Table 4.

Table 4

Specific Attributes Considered by Patients in the Treatment Decision

Topic		Representative Quotations
Positive treatment outcomes	Prolonging survival	I just thought about my kids and my grandkids, and like I said I’m not ready to give up. I’m not ready to leave them. (Early stage, percutaneous ablation)
		Well maybe give me 6 more months of life left. (Intermediate stage, TACE)
		After the diagnosis, the main thing you’re thinking is, “I just want to live. I want to survive as long as possible.” And if we did nothing and just let the cancer take its course, I would feel guilty if I didn’t try everything I could to try and stick around a little longer. (Intermediate stage, no treatment)
	Effectiveness of the treatment to remove the cancer	You know my goal is that no more lesions pop up . . . I’m at the point right now where I want to go ahead and try the radiation and get rid of this last lesion, go back to doing the MRIs every 3 months, and hope and pray that no more show up. (Very early stage, ablation)
		Let’s get rid of it. Let’s get rid of the mass and go from there. (Early stage, no treatment)
		Well naturally I would want whatever they think would be the most effective. (Intermediate stage, percutaneous ablation)
		The most important thing was to get rid of the tumor. I mean, I don’t think it matters how many treatments or procedures you go through. (Intermediate stage, TACE)
		I want the one that will knock it out. I would prefer the smoothest, easiest way. If there’s none, I’ll take what it takes to get rid of the cancer. (Advanced stage, systemic therapy)
	Improvement in symptoms	You know, I would hope that whatever I do or whatever they suggest would help me feel a little better. (Advanced stage, systemic therapy)
	Treatment that will improve liver transplant candidacy	I want to do whatever I have to do until they can give me a liver. Mean I know that is the only way. (Early stage, percutaneous ablation)
Treatment toxicity, recovery and caregiver burden	Minimizing recovery time	Recovery time—that was very important to me. And they told me, recovery wouldn’t take that long. (Very early stage, ablation)
		I need to know that I can come back to work right away. Although I have an incredible owner of the company that I work for, I still have to work. (Very early stage, SBRT)
		I came to understand that I would have to be out of work maybe for about a week or so while I recover and . . . that’s just not something which I wanted to do. I don’t believe I got a full picture initially of what the process was going to require. I have to go to work. If I don’t work, I don’t eat. (Intermediate stage, systemic therapy)
	Maintenance of normal routine and quality of life	I’ve worked all my life, and I just don’t think I can be one of these “sit in the house” people. (Early stage, percutaneous ablation)
		One of the first questions is will this really make me sick and not be able to do daily chores. And also help my wife because she needs full-time care. (Early stage, TACE)
		Initially, I wasn’t able to walk, sit up, do any of the things that I do or needed to do to be able to care for myself. And I had been independent up to that point, and it was important to me to regain my independence as much as possible. (Intermediate stage, systemic therapy)
		I really can’t afford to be away from home even for a day because my wife is here by herself . . . and I have to be with her. (Advanced stage, systemic therapy)
	Lowest burden for caregivers	The fact that my sister can drop me off at the hospital, go back to the hotel and wait for the surgery to be done and then come pick me up when I’m ready to go, that’s very important to me. (Very early stage, ablation)
		A big factor for me is my independence. I would sooner not live if I can’t live and be independent. (Early stage, ablation)
		The only thing I can think of is when I can get back to normal, I won’t have to put anybody out of their way. (Intermediate stage, TACE)
	Avoidance of side effects and toxicity	I thought that radiation was probably least dangerous because they aren’t cutting into your body at all. (Very early stage, ablation)
		Then the side effects, the possible complications, the recovery from it, those are all concerns that run through your mind. (Intermediate stage, TACE)
		I told them I don’t want to live that way. After I received [the treatment], I experienced flu aches. I can handle that for a week or so. But 5 or 6 weeks, no. (Intermediate stage, systemic therapy)
		I was mainly concerned with how much worse it can make me feel after it’s done, of course. You know, get up and go, no gusto. (Advanced stage, systemic therapy)
	Avoiding visits and hospitalization	I don’t like to spend a lot of time in hospitals, doctor’s offices or anything like that. I just wanted to minimize the amount of time. It just interferes with your life. (Intermediate stage, systemic therapy)
Procedure-related and logistic factors	Quickest start date	I have experienced chemo, ablation, and radiation. I knew that I had not had any adverse reactions to any of them so I just thought ablation might be the quickest and most effective. (Very early stage, ablation)
		One of my main concerns. “How fast could we do this?” If I can have surgery tomorrow I would. (Early stage, ablation)
		I wanted to get this thing out of there as soon as possible. So that’s the only thing I said. (Early stage, ablation)
	Decreased travel time	Well, one consideration is that we are an hour’s drive from [the facility]. SBRT would require significantly more treatments than a one-time ablation or one-time chemo treatment. So that’s a big consideration. (Very early stage, ablation)
	Costs	When you talk fears, because that’s a major fear is expense. I’m retired, I’m living, I don’t have that much in savings. (Very early stage, ablation)
		I haven’t talked to the finance people, but yeah it’s going to be an extreme burden, even with insurance. There is not extra money to be paying phone bills—I mean it’s just not there. (Early stage, TACE)
		If this doesn’t work, I can tell you that a liver transplant’s probably not going to happen. I don’t have the insurance and stuff to cover any of that, and I got a limited income. (Early stage, ablation)

SBRT, stereotactic body radiation therapy; TACE, transarterial chemoembolization.

Factors considered in the treatment decision—posttreatment

In the follow-up interview, 22 patient participants were asked to describe the preferences for treatment attributes. Most patients identified the same factors as being important in their pre- and posttreatment interviews. One patient stated, “I don’t see where anything’s changed. It’s the road I got to go down and I got to be positive and happy. Negativity never works” (early stage, TACE). Five of 22 patients (23%) indicated that they had significant changes in their primary goal(s) of care in the follow-up interviews. This included the following transitions: (1) from “minimally invasive” to “effectiveness of treatment,” (2) “minimally invasive” and “decrease travel time to hospital” to “effectiveness of treatment,” (3) “effectiveness of treatment” and “live for family” to “follow provider treatment recommendation,” (4) “effectiveness of treatment” to “avoid pain and discomfort,” (5) “follow provider treatment recommendation” to “minimally invasive,” and (6) “no treatment goal” to “maintain normal activities.” These are summarized with representative quotations in Table 5.

Table 5

Patient Participants with a Change in Pre- Compared with Posttreatment Goals

Patient Characteristics	Baseline Interview Goal	Representative Quotation from Baseline Interview	Follow-up Interview Goal	Representative Quotation from Follow-up Interview
Early stage, ablation	Minimally invasive	My primary goal is minimal invasion, or invasive, and . . . less pain. Quick, in and out, so when I go in I have it, and when I leave I don’t, and we just keep an eye on it, every 3 months.	Effectiveness of treatment	The primary goal of doing this treatment was so I didn’t have to get a liver transplant . . . and to take care of it and get rid of it.
Very early stage, ablation	Minimally invasiveDecrease travel time to hospital	And then there’s the risk factor, how dangerous is one treatment versus another? And of course the radiation is not invasive at all. Well, one consideration is that we are an hour’s drive from [the facility]. If [external beam radiation] was one of the options, that would require significantly more treatments than a one-time ablation or one-time chemo treatment. So that’s a big consideration because of the distance in the time that we have to travel.	Effectiveness of treatment	To eliminate the cancer all together.
Intermediate stage, TACE	Effectiveness of treatmentLive for family	I hope it’ll kill [the liver tumor]. I decided to do it so I can see my grandbabies grown. I love my grandbabies . . . they’re my heartbeat. They’re the ones that keep me living.	Follow provider treatment recommendation	Oh boy. I’ll tell you, right now? I think it was Dr. [NAME] that brought it up, I don’t know.
Early stage, ablation	Effectiveness of treatment	Well, my goal of treatment with the liver cancer is to remove the tumor, and Dr. [NAME] said this tumor could be cured. He did inform me though that given the condition of my liver there is a good possibility I will experience tumors in the future.	Avoid pain and discomfort	Probably to avoid physical pain. I don’t know that I made the decision in an effort to prolong my life, but . . . I want my end of life to be as pain-free as possible.
Intermediate stage, TACE	Follow provider treatment recommendation	After the MRI showed multifocal lesions, we went back and I saw Dr. [Name] . . . chemo or radiation wasn’t an option and the infusion therapy was the way that they recommended and they made it very clear that this was not a cure.	Minimally invasive	The one tumor was being aggressive and growing, and the doctors were concerned about it. The MRI showed that it had grown considerably in the 3 months between MRIs. And they suggested this procedure, and it sounded like a good option to me. And it wasn’t terrible, it was an outpatient procedure, so I agreed to it, and we had good results from it, so I’m happy with it.
Early stage, TACE	No treatment goal	No, I didn’t have a primary goal because I wasn’t sure what it was . . . how severe it was.	Normal activities	Well I’ll never be the same what it was before and that, but I’d like to come back to possibly just being a bit more active, more normal in a way with some restrictions. I understand. But try to get my strength back up.

MRI, magnetic resonance imaging; TACE, transarterial chemoembolization.

How decision-making conversations are held—patient perspectives

Patients were asked their thoughts and experiences on HCC treatment decision conversations with medical providers. There were some patients who described a scenario in which their medical provider recommended a treatment plan, and they followed their recommendation. They indicated they trusted the expertise of their provider and/or the provider made a good case for why it was the best treatment option. One participant said, “I’m happy with the decision. He did kind of push for that, but . . . not too hard. I’m not going to say he tried to really influence, but I could tell that that’s kind of one he wanted to do, and he is supposed to be the expert” (intermediate stage, TACE). Other patient participants noted that they made the final treatment decision together with their medical provider. Of note, in some of these examples, a recommendation may have been made by the provider that the patient followed, but the patient described the decision as one made “mutually” or made “together.” One stated, “Well, he asked me which one I wanted to do, and I told him, and [he] said, ‘Well good, I concur with you’” (early stage, percutaneous ablation). Two patient participants stated that they made the decision independently, and one delegated treatment decisions to their family members. One of these patients stated, “Dr. [NAME] said, ‘We can do this, that, or the other, and do you have a preference?’ And I said, ‘I don’t want to make multiple trips up here if I don’t need to. Let’s go the ablation route’” (very early stage, ablation). Another stated, “I think the doctors got together, and they might have asked my girls something, because I told them, ‘Whatever my girls want me to have done.’ My life is in their hands, let’s go for it” (advanced stage, systemic therapy).

Patients were asked to comment on how their medical provider asked them about their treatment preferences and/or what was important or valuable to them when deciding on a treatment plan. In response, participants shared that while some medical providers asked explicitly about patient preferences, others did not but did provide patients with an opportunity to participate in treatment decision making either by explaining options and letting patients choose or by making a recommendation and then checking in with the patient. Some patients reported they shared information about their values and preferences with their medical provider without being asked. Other patients indicated they were not asked about their preferences and did not offer this information to their provider, describing varying levels of comfort with it. Representative quotations from these groups are found in Table 6.

Table 6

Patient Perspectives on Discussions of Patient Preferences in the Context of HCC Treatment Decisions

Topic	Representative Quotations
Provider did ask patient preferences	Yes, we talked about it. And I told him, I said, “What’s important is more time.” (Early stage, TACE)
	He asked me what was important and what did I expect out of the treatments and stuff and I told him to become healed, be here with my family as long as I can. And they told me they couldn’t cure the cancer, the liver, but they could treat the side effects. (Early stage, TACE)
	Yes, we did talk about that. I had been independent up to that point, and it was important to me to regain my independence as much as possible. (Intermediate stage, systemic)
Provider explained treatment options and let patient choose	Yeah, I’m the one that decided it so I mean yeah. Yes he gave me the options, and told me it was up to me, and he leaned towards this. (Early stage, TACE)
Provider explained treatment options and let patient choose	He just wanted to leave it up to me . . . he knew what treatments I’ve had in the past, so he said, “I’m comfortable with whichever direction you want to go.” And I said, “Well my preference is ablation.” (Very early stage, ablation)
Provider made a recommendation and checked to make sure the patient agreed	And he said that I’m not eligible for liver transplant. And he said we’re going to try something if you want to. We’re going to do those seeds. And I said, “Well, this is about the best I can do so I’m going to do it.” (Intermediate stage, TACE)
	He talked to his colleagues, and came back. “We just need to go in there and get rid of it, burn it. And so how are your feelings on that? Do you want to go with that?” (Early stage, ablation)
Patient expressed a preference without being asked	Well, he didn’t push that, because I had already voiced my lack of desire to have an ablation again. And he seemed to agree with me. (Very early stage, SBRT)
Patient expressed a preference without being asked	Well, actually they didn’t ask. I just told them. (Intermediate stage, systemic therapy)
Patient was not asked about preferences and did not share them	They didn’t [ask about what was most important or valuable to me] . . . no, it [my goals for treatment] never came up. I never thought about it. Guess they didn’t either. (Early stage, ablation)
	No [they did not ask about what was important for me, but] . . . I feel comfortable enough that if I had any other concerns or changes that I can talk to him and we can relate back and forth. We have a good rapport. (Early stage, TACE)
	Not really [they did not ask me what was important to me]. I mean because to me, it’s longevity versus quality of life . . . quality of life [is my priority] . . . no [I did not express this to the doctor]. I mean, it just didn’t come up in the course of the conversation. (Early stage, ablation)

SBRT, stereotactic body radiation therapy; TACE, transarterial chemoembolization.

Provider Perspectives

Factors considered in the treatment decision

Providers primarily discussed the role of BCLC stage, liver function, performance status, and eligibility of liver transplantation in making treatment recommendations. The individual approach to treatment recommendations was strongly influenced by the providers’ specialty (i.e., transplant hepatology, oncology, surgery), but all providers identified key underlying factors used to determine treatment recommendations, including 1) burden of disease, 2) comorbidities, 3) tumor board discussion, 4) treatment preferences and goals of a patient, and 5) social determinants of health. Attributes identified by patients and providers is summarized in Figure 1.

Figure 1.

Treatment decision determinants for patient and provider participants.

The primary drivers of treatment decision making identified by providers included organizational factors, clinical and technical factors, and patient-centered factors (Table 7). Organizational factors that drove decision making included the specific composition of the multidisciplinary tumor board and multidisciplinary HCC clinic. Clinical and technical factors limited what treatment options may be available and whether a patient would be eligible for “potentially curative” treatments.

Table 7

Perspectives of Providers on Factors Considered in Treatment Decision

Topic	Representative Quotations
Organizational factors	I would say at the holistic level, the big thing for me is that we’re providing multiple treatment options as a program so that we can individualize the care of the patient versus having a limited set of options that drives our decision making. (Transplant surgery)
	When it hasn’t spread, this is where it’s complicated because now there’s a lot more options and a lot less data and this is really where that multidisciplinary discussion . . . comes to fruition. (Medical oncology)
	One of the problems of care outside of a multidisciplinary team or often what happens outside of an academic environment is people come at it with a very narrow lens. . . . The hepatologist isn’t providing any of those treatments . . . so I think that we can be a little bit more objective about what is best for the patient. (Transplant hepatology)
Clinical and technical factors	The first thing that I think about is are those patients eligible for liver-directed therapy. Particularly, are they surgical candidate or liver transplant candidate? Because those two treatments are the only two treatments that are considered curative. (Medical oncology)
	If the tumor invades major blood vessels, then you can’t cut out the tumor. Or if the tumor is in multiple parts of the liver where you couldn’t resect them all, then you wouldn’t be able to make the person cancer-free. That is one type of presentation we see, but I would say the more common and more challenging one is the moderately advanced tumor doesn’t really involve major blood vessels, but the patient’s underlying liver disease has progressed to the point that it limits what options you can offer them. (Transplant surgery)
	I guess my first question that I’ll ask myself is curative intent. Is there a procedure or surgery or sometimes it’s a transplant that could cure this patient of this cancer is one of the first things I think about when I’m making a treatment decision. If it’s not, then I start thinking about the other potential treatment options that are available for a patient and I’ll try to put those in the context of patient safety a little bit. What are the risks of decompensation, side effects, symptoms that could happen afterwards? (Transplant hepatology)
	I think whether there are comorbidities . . . particularly their liver function status. Do they have compensated cirrhosis or decompensated cirrhosis? (Medical oncology)
Patient-centered factors	I think patient treatment preference has to come into play . . . and sometimes in our tumor board discussions, we’ll have a couple of options and we can present a patient with those options and let them choose. I also think about distance from the medical center, availability of these treatments elsewhere, the number of visits that are required. A patient may choose one therapy over another. (Transplant hepatology)
	I think as I’m explaining through all of that, I’m trying to get a sense of what’s important to them. There are some patients that come in and say, “I have this particular reason that I want to live. I have my wife or my kids or my job or whatever and being alive is the most important thing to me.” And in those patients, assuming that it’s safe to do, I’m more likely to offer them as much as possible. And then there are people they come in and say, “I really don’t wanna get treatment for my cancer.” This is something I tell every one of my patients who has either localized or advanced disease that, “My goal as your cancer doctor is two things: number one is to try to treat your cancer and then number two is to make sure that I am doing right by you.” (Medical oncology)
	Their performance status, their live value, their comorbidities, and then . . . you know talking to them about those options and what their goals of care are in the sense of risks from treatment and side effects from treatment. (Medical oncology)
	The support system around them . . . that they’re not going through this alone. So seeing who’s around them, who can help take care of them. (Transplant hepatology)

Providers were asked what factors mattered to patients. They identified many of the same factors identified by patients including effectiveness of the treatment (i.e., likelihood of cure and impact on prognosis), effect of the treatment on liver disease and other health conditions, side effects of treatment, accessibility issues (e.g., cost, transportation, time off work), start date of treatment, and provider recommendations. Some provider participants noted that socioeconomics, culture, race, ethnicity, and age often influenced treatment preferences and recommendations. One medical oncology provider stated, “I often am the person that’s telling people that we can’t cure their cancer. And older patients tend to be more receptive to those conversations in my opinion.” An interventional radiology provider said,

Patients are usually just focused on “when am I gonna die?” So you do really have to drag them away from that to try to figure out what realistically is the most important thing and not just about having 5 months left? Do I have 10 months left? Do I have 4 years left?

How decision-making conversations are held

Provider participants were asked how they approached conversations with patients to elicit their treatment preferences or how they learned what was important to patients when deciding on an HCC treatment plan. They were also asked if there were clinical situations in which they would not ask patients about their treatment preferences. In response, a few providers indicated they did not explicitly ask patients about their treatment preferences. Reasons cited for not asking about preferences included that it was difficult for patients to answer and there were time constraints during clinic visits. However, these providers noted they did provide patients with information about the various options and involved them in treatment decision making. One medical oncology provider stated,

I would let them know that we don’t have data for head to head comparison to say which one is better. You are eligible for both A and B. These are the pros and cons, and this is your cancer, this is your body, you are going to be the one who experiences the side effects and complications. Once you provide them with more details for each treatment, they can then have something to focus on and have something to consider.

Some providers reported they did routinely ask patients about their treatment preferences and described various methods or strategies for how they did it including providing an overview of treatment options and discussing pros and cons, starting by asking open-ended questions and then prompting with more specific questions, acknowledging that some patients may not want treatment and giving patients permission to choose no treatment and following the patient’s lead. One medical oncology provider said,

I usually just ask, “How are you doing?” And the first thing that they’ll tell me, they’ll say, “I’m doing okay, but I want to spend as much time as I can with my family. I want to go to my daughter’s wedding.” . . . And they volunteer this information pretty much immediately.

Providers described a few situations in which they would not solicit patient treatment preferences including time constraints, the availability of only 1 treatment option, patients who had already decided on a treatment plan, and patients who did not have capacity to make a treatment decision.

There were mixed responses to the imagined scenario of receiving quantitative data on an individual patient’s preferences for treatment prior to making a treatment recommendation, with half the providers indicating this type of patient data would be potentially useful in determining a treatment recommendation and the other half sharing they felt this type of patient data would not be useful or were unsure how useful the data would be. One transplant hepatology provider who thought this information may be useful for informing treatment decisions stated,

Sometimes we come to tumor board after having met a patient once and there will be two relatively equivalent options. It would be probably easier to say, “Well, given these two equivalent options, I know that the patient is going to want this.”

Conversely, one medical oncology provider was uncertain how this information would be useful: “I think in an ideal world our patients can give you a clear answer to those questions and I get that information beforehand. But, in reality, the vast majority of patients won’t be able to give you that answer.” Providers had mixed responses on when information on patient preferences or values would be most impactful. Answers included at the time of referral (n = 3), before meeting a patient for the first time (n = 1), in clinic (n = 1), at tumor board (n = 1), toward the end of the treatment decision (n = 1), and at any point (n = 1).

Discussion

This qualitative study involving patients with HCC and medical providers from multiple subspecialties provided several important insights with the potential to improve care. First, patients identified several attributes beyond overall survival and toxicity that are important in treatment decisions. These attributes should be considered in treatment discussions and decisions. Second, providers identified most but not all of these attributes that are important to patient treatment decisions, demonstrating the potential need for formalized assessment of patient values and preferences. Furthermore, both patients and providers identified several attributes that are not considered in major treatment guidelines, including start time, travel time, and accessibility.^4,5,16 Third, there was a wide range of perspectives on the importance of systematically eliciting values and preferences and incorporating these into treatment discussions and choices, suggesting the need for a tailored approach. Lastly, there were mixed opinions among HCC providers regarding the role for systematically assessing preferences as part of clinical care but general agreement that there is a need to better understand the role of patient values to improve care for HCC.

The emergence of several new HCC treatment options, including combination therapies, highlights the need for improved understanding of patient preferences in treatment decisions. As demonstrated by a recent scoping review from our group, most studies evaluating the role of preference elicitation tools have been focused on early or advanced HCC.¹² The only preference elicitation study focused on patients with intermediate-stage HCC included outdated treatment options.¹⁷ According to the International Society of Pharmacoeconomics and Outcomes Research, qualitative research helps identify a full set of attributes and levels for inclusion in such preference elicitation tools.¹⁸ Therefore, findings from this study can be used to inform attributes for inclusion in discrete choice experiments or other preference elicitation tools to elicit preferences for treatment choice in intermediate-stage HCC.

One potential clinical application of such preference elicitation tools would be facilitating treatment discussions in multidisciplinary tumor boards or clinics. As outlined above, there may be scenarios in which patient preferences have minimal influence on treatment choice. However, systematic assessment of patient preferences using such tools may ensure that patient preferences are considered in an unbiased way by HCC providers. This has the potential to improve the care of patients with HCC. Existing data suggest that implicit clarification of patient values decreases decisional conflict and the frequency of values-incongruent choices.¹⁹

Wider use of such preference elicitation tools also has the potential to identify preference heterogeneity among larger populations of patients with HCC. For instance, there may be subsets of patients who prioritize the minimization of side effects, optimization of physical functioning, or avoidance of caregiver burden. Such information has the potential to guide patient-centered treatment development, which has been prioritized by the US Food and Drug Administration.^10,11 Furthermore, such data have the potential to inform the development of treatment guidelines in a few ways. Qualitative data have the potential to inform guideline scope, evidence-to-decision frameworks, and the implementation processes. Furthermore, the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group approach, which has been adopted by many guideline development organizations, states that balancing the magnitude of desirable and undesirable outcomes requires an understanding of patients’ values and preferences.²⁰ Large-scale preference elicitation studies are therefore needed to adequately inform these assessments for HCC treatment guidelines.

This study is strengthened by its qualitative assessment of patients’ and providers’ viewpoints on the role of patient preferences in HCC treatment decisions. However, it must be interpreted in light of its limitations. First, this was a single-center study limited to primarily non-Hispanic English speakers and therefore may not be generalizable to Hispanic, European, and Asian populations. Second, our patient sample involved only a few patients who did not pursue therapy. Further study on patients who opted not to receive HCC therapy may help identify important treatment preferences among this specific subpopulation. Third, providers involved in the survey all come from a single center, and responses were likely influenced by institutional practices. More generally, qualitative studies generate results that may be influenced by small sample sizes and the specific clinical context. However, qualitative interviews can generate rich sources of data that can be used for subsequent research. For example, attributes identified in this study can be used to inform the design of preference elicitation surveys that may subsequently be used for larger-scale survey studies to elucidate the role of patient choice in HCC treatment decisions.

In conclusion, this qualitative study of patients with HCC and providers provided important information on decision-making determinants, treatment discussions, and the incorporation of preferences into HCC treatment decisions. These findings will inform the development of preference elicitation tools for use in clinical practice, for large-scale studies to assess preference heterogeneity in HCC, and to inform treatment guidelines (e.g., National Comprehensive Cancer Network, American Association for the Study of Liver Diseases, European Association for the Study of the Liver, Barcelona Clinic Liver Cancer).

Supplemental Material

sj-docx-1-mpp-10.1177_23814683251340055 – Supplemental material for Exploring the Role of Patient Preferences in Hepatocellular Carcinoma Treatment Decisions: A Qualitative Study

Supplemental material, sj-docx-1-mpp-10.1177_23814683251340055 for Exploring the Role of Patient Preferences in Hepatocellular Carcinoma Treatment Decisions: A Qualitative Study by Andrew M. Moon, Daniel Richardson, Gabriel V. Lupu, Donna M. Evon, Hanna K. Sanoff, Jessica Carda-Auten, Randall Teal, Myra Waheed, Ethan Basch, David M. Mauro, Ted K. Yanagihara, David A. Gerber, Neil D. Shah, Oren K. Fix, Hersh Shroff, Tammy Triglianos, Jonathan D. Sorah, Jingquan Jia, Ashwin Somasundaram, Lynne I. Wagner, Michael D. Kappelman, Matthew Schooler, Julia R. Phillips, Hiwot A. Ekuban, Ariel E. Sanderford and A. Sidney Barritt IV in MDM Policy & Practice

Footnotes

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: AMM is a consultant for TARGET RWE, Intercept Pharmaceuticals, Eisai, and AstraZeneca. Research funding (to the institution) was received from DCN Diagnostics. EB received research funds to the institution from the National Cancer Institute and the Patient-Centered Outcomes Research Institute. He has received personal fees as a scientific advisor to AstraZeneca, Resilience, Verily, and Navigating Cancer. TT served on Pfizer’s ad board. JJ served as a consultant or advisor to AstraZeneca and Cardinal Health and received research funding or contracted research for Roche, Genentech, and AstraZeneca. ASB is a consultant for Target-RWE, Madrigal, Life-Edit, Boehringer Ingelheim, and Merck. All other authors declared no additional disclosures or potential conflicts of interest. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: financial support for this study was provided in part by an AASLD Clinical, Translational and Outcomes Award from the AASLD Foundation (AMM). This research was supported, in part, by a grant from the NIH T32 DK 007634 (to GVL). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Author Contributions

All authors approved the final version of this manuscript. Andrew M. Moon is the guarantor of this paper. Moon: study concept and design, data acquisition, interpretation of data, drafting of manuscript, critical revision of the manuscript; Richardson: study design, interpretation of data, critical revision of the manuscript; Lupu: data acquisition, interpretation of data, critical revision of the manuscript; Evon: study design, interpretation of data, critical revision of the manuscript; Sanoff: study design, interpretation of data, critical revision of the manuscript; Carda-Auten: study design, data acquisition, interpretation of data, critical revision of the manuscript; Teal: study design, data acquisition, interpretation of data, critical revision of the manuscript; Waheed: study design, data acquisition, interpretation of data, critical revision of the manuscript; Basch: study design, interpretation of data, critical revision of the manuscript; Mauro: study design, interpretation of data, critical revision of manuscript; Yanagihara: study design, interpretation of data, critical revision of the manuscript; Gerber: study design, interpretation of data, critical revision of the manuscript; Shah: study design, interpretation of data, critical revision of the manuscript; Fix: study design, interpretation of data, critical revision of the manuscript; Shroff: study design, interpretation of data, critical revision of the manuscript; Triglianos: study design, interpretation of data, critical revision of the manuscript; Sorah: study design, interpretation of data, critical revision of the manuscript; Jia: study design, interpretation of data, critical revision of the manuscript; Somasundaram: study design, interpretation of data, critical revision of the manuscript; Wagner: study design, interpretation of data, critical revision of the manuscript; Kappelman: study design, interpretation of data, critical revision of the manuscript; Phillips: data acquisition, interpretation of data, critical revision of the manuscript; Ekuban: data acquisition, interpretation of data, critical revision of the manuscript; Sanderford: data acquisition, interpretation of data, critical revision of the manuscript; Barritt: study design, interpretation of data, critical revision of the manuscript.

Ethical Considerations

All research was conducted in accordance with the Declaration of Helsinki. Institutional review board (IRB) approval was provided by the UNC IRB (#21-3051), and patients provided written consent prior to the initiation of any study procedures.

Consent to Participate

Patients provided written consent to participate in this study prior to the initiation of any study procedures. We have not published any identifiable information as part of this study.

Consent for Publication

Patients provided written consent to have the data published as part of this study prior to the initiation of any study procedures.

ORCID iDs

Andrew M. Moon

Gabriel V. Lupu

Ethan Basch

Data Availability

The raw and analysis data sets contain protected health information and/or personally identifiable information. Based on ethical considerations and the need to protect the confidentiality of participants, we are unable to provide the full transcripts of qualitative interviews and have limited data sharing to what is included in the manuscript: deidentified and aggregate clinical data, summaries of themes and subthemes, selected quotations from interviews, the full interview guides, and finalized codebooks.

References

Moon

Singal

Tapper

. Contemporary epidemiology of chronic liver disease and cirrhosis. Clin Gastroenterol Hepatol. 2020;18(12):2650–66. DOI:10.1016/j.cgh.2019.07.060

GBD US Health Disparities Collaborators. Burden of liver cancer mortality by county, race, and ethnicity in the USA, 2000–19: a systematic analysis of health disparities. Lancet Public Health. 2024;9(3):e186–98. DOI:10.1016/S2468-2667(24)00002-1

Kwong

Ghaziani

Yao

Sze

Mannalithara

Mehta

. National trends and waitlist outcomes of locoregional therapy among liver transplant candidates with hepatocellular carcinoma in the United States. Clin Gastroenterol Hepatol. 2022;20(5):1142–50.e4. DOI:10.1016/j.cgh.2021.07.048

Singal

Llovet

Yarchoan

, et al. AASLD practice guidance on prevention, diagnosis, and treatment of hepatocellular carcinoma. Hepatology. 2023;78(6):1922–65. DOI:10.1097/HEP.0000000000000466

National Comprehensive Cancer Network. Hepatocellular carcinoma, version 1.2024, NCCN clinical practice guidelines in oncology. Available from: https://www.nccn.org/professionals/physician_gls/pdf/hcc.pdf. [Accessed 25 May, 2024].

Finn

Qin

Ikeda

, et al. Atezolizumab plus bevacizumab in unresectable hepatocellular carcinoma. N Engl J Med. 2020;382(20):1894–905. DOI:10.1056/NEJMoa1915745

Abou-Alfa

Lau

Kudo

, et al. Tremelimumab plus durvalumab in unresectable hepatocellular carcinoma. NEJM Evid. 2022;1(8):EVIDoa2100070. DOI:10.1056/EVIDoa2100070

Farsad

Nabavizadeh

Kardosh

Jou

Naugler

Kolbeck

. Combined locoregional and systemic therapy for advanced hepatocellular carcinoma: finally, the future is obscure. Ann Transl Med. 2020;8(24):1700. DOI:10.21037/atm-20-4164

AstraZeneca. IMFINZI® (durvalumab) plus bevacizumab met primary endpoint for progression-free survival in liver cancer eligible for embolization in EMERALD-1 Phase III trial. Available from: https://www.astrazeneca-us.com/media/press-releases/2023/imfinzi-durvalumab-plus-bevacizumab-met-primary-endpoint-for-progression-free-survival-in-liver-cancer-eligible-for-embolization-in-emerald-1-phase-iii-trial-11092023.html. [Accessed 12 August, 2024].

10.

Kluetz

Bhatnagar

. The FDA’s patient-focused drug development initiative. Clin Adv Hematol Oncol. 2021;19(2):70–2.

11.

US Food and Drug Administration. Patient preference information (PPI) in medical device decision making. Available from: https://www.fda.gov/about-fda/division-patient-centered-development/patient-preference-information-ppi-medical-device-decision-making. [Accessed 12 August, 2024].

12.

Ritaccio

Barritt Iv

Conklin

, et al. Scoping review of values elicitation tools for treatment decisions in hepatocellular carcinoma. BMC Gastroenterol. 2024;24(1):90. DOI:10.1186/s12876-024-03167-1

13.

Vasileiou

Barnett

Thorpe

Young

. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol. 2018;18(1):148. DOI:10.1186/s12874-018-0594-7

14.

O’Brien

Harris

Beckman

Reed

Cook

. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51. DOI:10.1097/ACM.0000000000000388

15.

Korstjens

Moser

. Series: practical guidance to qualitative research. Part 4: trustworthiness and publishing. Eur J Gen Pract. 2018;24(1):120–4. DOI:10.1080/13814788.2017.1375092

16.

European Association for the Study of the Liver. EASL clinical practice guidelines: management of hepatocellular carcinoma. J Hepatol. 2018;69(1):182–236. DOI:10.1016/j.jhep.2018.03.019

17.

Chiba

Hiraoka

Mikami

, et al. Japanese patient preferences regarding intermediate to advanced hepatocellular carcinoma treatments. Patient Prefer Adherence. 2019;13:637–47. DOI:10.2147/PPA.S198363

18.

Bridges

Hauber

Marshall

, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14(4):403–13. DOI:10.1016/j.jval.2010.11.013

19.

Witteman

Ndjaboue

Vaisson

, et al. Clarifying values: an updated and expanded systematic review and meta-analysis. Med Decis Making. 2021;41(7):801–20. DOI:10.1177/0272989X211037946

20.

Schünemann

Brożek

Guyatt

Oxman

. GRADE handbook. Available from: https://gdt.gradepro.org/app/handbook/handbook.html. [Accessed 12 August, 2024].

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB