Layperson Views about the Design and Evaluation of Decision Aids: A Public Deliberation

Abstract

Purpose

We carried out the first public deliberation to elicit lay input regarding guidelines for the design and evaluation of decision aids, focusing on the example of colorectal (“colon”) cancer screening.

Methods

A random, demographically stratified sample of 28 laypeople convened for 4 days, during which they were informed about key issues regarding colon cancer, screening tests, risk communication, and decision aids. Participants then deliberated in small and large group sessions about the following: 1) What information should be included in all decision aids for colon screening? 2) What risk information should be in a decision aid and how should risk information be presented? 3) What makes a screening decision a good one (reasonable or legitimate)? 4) What makes a decision aid and the advice it provides trustworthy? With the help of a trained facilitator, the deliberants formulated recommendations, and a vote was held on each to identify support and alternative views.

Results

Twenty-one recommendations (“deliberative conclusions”) were strongly supported. Some conclusions matched current recommendations, such as that decision aids should be available for use with and without providers present (conclusions 1–4) and should support informed choice (conclusion 9). Some conclusions differed from current recommendations, at least in emphasis—for example, that decision aids should disclose cost of screening (conclusion 11) and should be kept simple and understandable (conclusion 14). Deliberants recommended that decision aids should disclose the baseline risk of getting colon cancer (conclusions 15, 17).

Limitations

Single location and medical decision.

Conclusions

Guidelines for design of decision aids should consider putting a greater focus on disclosing cost and keeping decision aids simple, and they possibly should recommend disclosing less extensive amounts of quantitative information than currently recommended.

Keywords

colorectal cancer screening decision aids public deliberation risk communication shared decision-making

Introduction

Background

Decision aids aim to support individuals making “preference-sensitive” decisions (i.e., ones where there are multiple reasonable options that individuals may select based on their personal weighting of the value of potential harms and benefits). Research shows that decision aids increase patient knowledge, improve patient-provider communication, and increase uptake of recommended interventions, compared to usual care.¹ There are crucial questions, however, about how decision aids should be designed. Guidelines vary and rely on few randomized trials comparing different types of decision aids.^1,2 Any recommendations for their design and use involve ethical and conceptual questions about the goals of disclosure and discussion. Lay input could help answer such questions, but existing guidelines have been developed with limited involvement of laypeople. Our project gathered lay input using a public deliberation focused on questions about the design and use of decision aids.

The earliest guidelines—the Ottawa Decision Support Framework^3–5—were developed by experts without any cited lay input. Patients were included as one of the stakeholder groups in the development of the first version of the International Patient Decision Aids Standards, along with researchers, practitioners, and policy makers,⁶ but were not explicitly included in the recent update.⁷ The National Quality Forum guidelines were produced by an expert panel, with laypeople only involved during the open comment period.⁸

Developing guidelines for the evaluation of decision aids is becoming an increasingly important policy issue, as guidelines start being used to determine which decision aids are covered by health insurance. Washington State’s legislature implemented incentives to encourage orthopedists to show their patients an approved decision aid before knee or hip replacement surgery.^9,10 The Affordable Care Act proposed reimbursing the use of certified decision aids, and the National Quality Forum and others have developed criteria for the certification process.^8,11 Lay input is especially important when guidelines about decision aids become criteria for reimbursement.

Two areas where guidelines vary, and where lay input would be particularly valuable, involve what types of information decision aids should present and what value clarification activities should be included. The International Patient Decision Aids Standards (IPDAS)^6,7 recommend that decision aids include detailed quantitative probabilistic information. For preventive interventions, for instance, the IPDAS guidelines recommend disclosure of baseline risk, risk reduction, and frequency of complications and negative outcomes.^6,12 The National Quality Forum states that decision aids regarding screening or diagnostic tests should include information about the test’s positive predictive value and negative predictive value.⁸ Other guidelines and expert opinion similarly support disclosure of a range of quantitative information.^6,8,12 The rationale behind such suggestions is to provide patients with relevant information to make informed choices. Verbal descriptions of probability are inherently vague: studies show that people assign very different meanings to terms such as “rare” and “common” and may overestimate the frequency of side effects when described verbally.^13–16

Some commentators have argued against the assumption that decision aids should disclose certain types of quantitative information to all patients.^17–19 First, many people may have difficulty understanding frequencies and probability, due to low numeracy.^20,21 Charts and figures could improve understanding but can also be misunderstood, and many people have low “graphical” literacy.^22,23 Second, quantitative risk data may be misused or misapplied due to common heuristics and biases of human thought, such as denominator neglect.^24,25 Research shows that behavior is often guided by a “gist” impression of risk and benefit, rather than calculations using probabilistic information.^26,27 Finally, limited empirical research exists to assess whether quantitative information improves or harms decision making overall.^1,17,19

Those who oppose disclosing detailed quantitative information point to writers in bioethics who have argued that more information or disclosure is not always better.^28–30 Prominent theories in health communication support providing limited amounts of information initially, then providing additional information if requested.^31–33 Decision aids could take this approach, simply offering quantitative information as an option for patients to view. This would maximize patient choice about what information to receive, but it can be critiqued since it means that some patients will fail to see information that is deemed essential by some experts.³⁴ The question of what information to disclose to all patients thus raises deep questions about the goals of disclosure in decision making. Lay input can help define those goals.

A similar debate regards what sort of values clarification methods decision aids should include. Some decision aids lead patients through explicit values clarification, involving exercises such as ranking and rating options to help patients specify their relevant values and apply them to the choice at hand.^35–37 Some commentators, however, have criticized this approach since studies suggest that such detailed processes for decision making often result in choices that do not maximize happiness or welfare.^38–40 The alternative is to rely on patients’ identifying their values “implicitly,” as they learn more about the options and their pros and cons.^35,37 Choosing implicit or explicit approaches to values clarification again depends on deeper questions about what counts as adequate reflection before decision making. And, once again, lay input could help address these questions.

Rationale

Our project was designed to obtain input of laypeople about the design and use of decision aids. Some commonly used methods for identifying lay opinions, such as surveys and focus groups, have limited usefulness in this case. Surveys generally do not allow individuals to become fully informed or to reflect carefully about the complex questions we raised above, which require tradeoffs between goals. Focus groups allow more education and discussion, but these generally identify themes for further investigation rather than producing actionable advice of the sort we are seeking in this case.⁴¹

We chose to use the method of “public deliberation,” which involves 3 steps. First, a relatively small but diverse group of individuals is selected using a mix of purposive and random processes and is invited to participate. Second, participants are informed about the technical issues and the range of conflicting perspectives. The participants then discuss the issues with the help of a skilled facilitator and are asked to develop collective positions that can be conveyed to decision makers. Ideally, participants work toward consensus, although description of the causes for persistent disagreement can be valuable for decision makers as well.^42,43

Public deliberation has been used in multiple areas to develop public input into health policy decisions, especially when there are complex tradeoffs involving priority setting and ethical issues.^44,45 The method is especially appropriate for cases that require understanding of technical issues and require decisions about how to best serve the public good.⁴⁴ There is growing appreciation of the utility of public deliberation in complex science and technology,⁴⁶ health,⁴⁷ and ethical norms.⁴⁸ In numerous cases, the outcome of the public deliberation was translated into specific changes in research practice, especially regarding biobank policies and governance.^49,50

Public deliberation has not been used to address questions about design and evaluation of decision aids but has been used for evaluating patient information regarding mammography.^51,52 In this setting, public deliberation can be seen as applying bioethics principles such as the Reasonable Person Standard (i.e., the requirement that patients should be given all information that a “reasonable person” would want to consider).⁵³ While psychological studies can identify what information patients want and how different types of information affect opinions or choices, public deliberation can identify what information laypeople decide should be treated as essential.

In this project, we convened 28 members of the lay public over 2 weekends to consider the issues and come up with recommendations about a range of issues regarding the design and use of decision aids.

Our Focus: Colorectal Cancer Screening

Our deliberation focused on colorectal cancer screening, an area of study for our research group.^54,55 Colorectal cancer (CRC) screening is recommended for people ages 50 to 75 years old, and several tests are endorsed for those at average risk, resulting in a preference-sensitive choice.^56,57 Colonoscopy provides the most complete examination of the colon but is an invasive procedure that involves a 1- or 2-day preparation to clean out the colon, intravenous (IV) sedation, and the need to take a day off of work and arrange a ride home.

A commonly used alternative is annual stool blood testing (e.g., the fecal immunochemical test [FIT]), which is easy to do and is performed at home.^56–58 A single application of FIT often fails to identify polyps and may miss a cancer, however, and a positive stool test requires colonoscopic evaluation. Despite its lower sensitivity, annual FIT with colonoscopy for any positive test produces a lifetime reduction in mortality from CRC that approaches or equals that produced by colonoscopy, according to several comparative effectiveness analyses.^59,60

Several decision aids have been created and tested to guide decisions about colorectal cancer screening, and these decision aids have differed from each other in significant ways.^1,54,61–63 Some of these decision aids have been shown to support the informed selection of one of the approved screening tests, a truly preference-sensitive choice since all the approved tests are endorsed as reasonable options and the individual should select the one that best fits their values and preferences. The broader decision to be screened v. not, rather than choice of test, can be seen as not preference sensitive, since screening is recommended by all guidelines, and studies have the goal of increasing uptake of screening. From a different perspective, however, the decision to be screened in this and other cases where screening is recommended can be seen as preference sensitive, since people may rationally choose not to be screened, based on their valuation of the options.⁶⁴

Methods

Study Setting

The study was conducted over 2 nonconsecutive weekends in May 2017 on the campus of Indiana University–Purdue University Indianapolis (IUPUI). The study was approved by the Indiana University Institutional Review Board.

Inclusion/Exclusion Criteria

Participants were residents living within a 25-mile radius of the project office who were at least 18 years old. Participants were eligible if they were a member of the household that received a recruitment letter and were able to attend the entire event. We excluded people who did not speak and read English.

Recruitment Process

The goal of recruitment was to constitute a randomly selected, demographically stratified sample of 25 to 32 adults. Selection of participants occurred via a double randomization process and according to stratification across 6 demographic filters (gender, race, ethnicity, age, location/geography, and education). First, household addresses were purchased from a company with expertise in mailing list use and analysis. From the 625,000 addresses in the caption area, 10,001 were randomly selected and letters of invitation were mailed on the same day, approximately 2 months before the first day of the event. The letters explained the nature of the study, including the general topic, the time requirements for participating, and the renumeration ($125 per day). The letter invited interested recipients to contact the study office by phone, by email, or through the project’s website. Study staff followed up all phone calls, emails, or website contacts to further explain the project, answer all questions, assess interest, and collect demographic information.

A summary of the recruitment can be found in Figure 1. Of the 381 individuals who contacted the study office expressing interest, 335 were eligible. Study staff selected names at random until 32 people were found who approximated a predetermined range of age, gender, race, ethnicity, geography, and education. For each name selected, a first and second alternate with similar demographic characteristics were also selected.

Figure 1

Public deliberation recruitment summary.

For individuals selected, study staff confirmed continued interest and mailed the study information sheet along with a welcome letter and map of the event location.

Information Provision

An important component of public deliberation is providing participants with sufficient, unbiased information to serve as a foundation for meaningful deliberation.

An information booklet (see Supplementary Appendices—Briefing Book) was written specifically for this project and was provided to participants 2 wk before the event, with additional copies made available at the event. The entire research team, a patient advisory board, and a community advisory board read and discussed multiple drafts with the goal of ensuring that the information presented was complete, balanced, and understandable to a lay audience. The research team included experts in health communication, colorectal cancer screening, and bioethics; the stakeholder advisory board included health care providers and leadership from our partner health care institutions and nonprofit organizations; and the patient advisory board included individuals who had participated in previous studies of CRC screening. The booklet had 28 pages of text, introducing the participants to public deliberation and their role, and covering the following topics:

Decision Making and Decision Aids: Informed health care decisions, role of decision aids, questions about what information decision aids should include

Colon Cancer Screening: Introduction to cancer screening and colon cancer, benefits and harms of colonoscopy and FIT, challenges in increasing uptake and informing patients, quantitative information regarding comparative effectiveness

Patient Decisions and Decision Aids: Questions regarding which information and how much information to include in decision aids about colon cancer

In addition, a website was constructed to provide information about the planned deliberation, the topic, and the research team. A section of the website included the event schedules and the booklet. Potential participants were directed to the website in the initial recruitment letter.

During the first day of the deliberation event, the participants viewed expert presentations on the following topics:

Informed health care decisions and role of decision aids (“Introduction to Decision Aids and Improving Decisions”)

Colorectal cancer screening (“Colon Cancer Screening—The Basics”)

Quantitative information regarding screening (“Quantitative Information”)

Presentation of risk information (“How Much Information Is Enough?”)

On the first day of the deliberation, there was a 30-minute patient panel comprising 2 people who had experience with choosing a CRC screening test with the support of a decision aid (1 chose to get colonoscopy, 1 chose to do FIT) and 1 person who was undergoing treatment for CRC.

Deliberation Event

The public deliberation occurred over 2 noncontiguous weekends (4 days total), with a weekend between them. The activities took place between approximately 9 a.m. and 5 p.m. with frequent breaks. The deliberation was guided by 4 deliberative questions that were formulated and presented in the information booklet and repeated at various points during the event:

What information should be included in all decision aids for colon screening?

How should risk information be presented?

What makes a screening decision a good one (reasonable or legitimate)?

What makes a decision aid and the advice it provides trustworthy?

Day 1 of the event was designed to provide participants with key information about colorectal cancer screening and decision aids and to explain the activities and goals involved in public deliberation (presentations and panel as described above). Days 2 and 3 involved most of the deliberation, both in small and large groups. On day 4, participants completed their deliberation and engaged with a panel of experts and were asked to revisit and review each of their recommendations and make any changes or clarifications they felt necessary (full agenda available on request).

Each of the 4 deliberation questions was initially discussed within small groups and then by the large group led by 2 individuals with extensive experience moderating public deliberations (MB and CB).

Each small group had a facilitator and note taker and consisted of 7 to 8 participants chosen to reflect the demographic diversity of the large group. Heterogeneity in groups allows people to share a variety of experiences and opinions, as well as helps individual participants to gain a broader perspective of the issues under consideration. This is an important step toward building intersubjectivity in the group as a whole, which is necessary for the collective development of recommendations, which occurs subsequently in the large group. These groups were kept consistent throughout both weekends so that participants could build trust and familiarity with one another. Discussion in small groups encouraged expression of opinions, reflection on personal experience, and participation by all members. A representative from each small group presented discussion points to the large group.

During large group discussion, participants proposed recommendations (“deliberative conclusions”) to be considered by the entire group. The moderator at times suggested the construction of a deliberative conclusion when it seemed that discussion was coalescing on one. Possible deliberative conclusions were edited by the group; if there were disagreements about the wording of a deliberative statement, multiple versions could be constructed. After construction of a conclusion, participants voted for, against, or abstained by placing a colored index card on the desk in front of them. Each individual who abstained or dissented was encouraged by the facilitator to state the reasons for their position.

Data Collection and Sources

Research assistants recorded all proposed deliberative conclusions as the participants formulated and edited them. Proposed conclusions were displayed on a large screen. The event was audio recorded and transcribed.

Analytical and Statistical Approaches

This article presents the deliberative conclusions and summarizes prominent reasons (pros and cons) offered in large group discussion. The authors categorized the deliberative conclusions into 5 categories based on content and selected examples of reasons and quotations from a transcript of the large group discussions. This approach is intended to foreground what participants said is important to them. Future articles will provide a deeper thematic analysis of the discussion.

Results

Demographics

Twenty-nine individuals participated in the first weekend of deliberation and 28 individuals participated in the second weekend. See Table 1 for demographic information. Approximately half were male and half female, and most were white and non-Hispanic (14% black, 7% Hispanic). Almost all had graduated from high school (96%), and 50% were college graduates. On the short version of the Subjective Numeracy Scale,^65–67 participants had a mean (SD) score of 15.2 (2.52) (range, 3–18). This is higher than the mean score on these questions for a group of people with high numeracy in a recent study⁶⁸ and more than any of 7 clinical groups in another study,⁶⁷ indicating that our deliberants had relatively high numeracy.

Table 1

Demographics of Public Deliberation Participants (n = 28)

Characteristic	n (%)
Gender
Male	13 (46)
Female	15 (54)
Location
Rural	1 (3)
Suburbs	15 (54)
Indianapolis	12 (43)
Race
Nonwhite and nonblack	2 (7)
Black	4 (14)
White	22 (79)
Hispanic
Yes	2 (7)
No	26 (93)
Age, y
18–29	3 (11)
30–39	2 (7)
40–49	4 (14)
50–59	6 (21)
60–69	8 (29)
70–79	4 (14)
≥80	1 (4)
Education
Less than high school graduate	1 (4)
High school graduate/GED	3 (11)
Associate’s degree/some college/ technical or trade school	10 (36)
College graduate	8 (29)
Professional or graduate degree	6 (21)
Subjective income
Are comfortable	19 (68)
Have just enough to make ends meet	5 (18)
Do not have enough to make ends meet	4 (14)
Subjective numeracy^a
<13	4 (14)
13–15	9 (32)
>15	15 (54)

Subjective numeracy was assessed using the 3-item Subjective Numeracy Scale. Each item has 6 response options (values 1–6) and possible scores range from 3 to 18, where the higher score indicates higher subjective numeracy.

Deliberative Conclusions

The participants formulated and voted on 23 potential conclusions, and a significant majority supported 22 of them; 1 was replaced. Table 2 lists all 23 recommendations, grouped by topic: use of decision aids, content, and trustworthiness. We here summarize key issues that came up during large group discussion, along with representative quotations.

Table 2

Deliberative Conclusions Categorized by Theme

Recommendation	Strongly Supported?^a
Use of decision aids
1. A decision aid should be a part of a physician’s conversation with the patient about colon cancer screening.	Yes (28, 0, 0)
2. Decision aids should be available outside appointments with health care professionals.	Yes (26, 0, 2)
3. Decision aids should be used across appropriate health care professionals to ensure validity and consistency of information.	Yes (28, 0, 0)
4. Certain decision aids should be designed to be used without the involvement of health care professionals.	Yes (23, 4, 1)
Content: Encouraging screening and supporting informed choice
5. There should be a presumption in favor of screening for colon cancer.	Yes (29, 0, 0)
6. The decision aid should include an option not to be screened.	Yes (20, 8, 1)
7. Decision aids should disclose the potential complications like bleeding and tears of colonoscopy.	Yes (21, 3, 4)
8. A decision aid must have alternatives to colonoscopy as screening options.	Yes (26, 1, 1)
9. Health care providers must make available the information required to make an informed decision about colon cancer screening.	Yes (27, 1, 0)
10. Decision aids should state that the cost to patients is variable and there may be direct costs to the patient. Patients should be encouraged to clarify their coverage by talking to their insurance carrier. Cost may vary between providers.	Yes, but replaced by #11 (23, 2, 3)
11. Decision aids should include that cost to patients may vary based on insurance coverage and provider fees.	Yes (28, 0, 0)
12. All decision aids should include the following facts:• People aged 50–75 can reduce their risk of dying of colon cancer by screening• Colon cancer is the second largest cause of death by cancer• There are several kinds of screening tests• Colon cancer can be prevented	Yes (23, 1, 4)
Content: Complexity and tailoring
13. Decision aids need to be tailored to the intended audience.	Yes (26, 0, 2)
14. It is more important that a decision aid is easy for the relevant information to be understood than it is to include all possible information.	Yes (25, 0, 3)
Content: Quantitative information
15. Decision aids must include numerical risk of cancer.	Yes (23, 2, 4)
16. Decision aids should include that colon cancer is the second cause of death by cancer.	Yes (29, 0, 0)
17. Baseline risk^b (60 out of 1000) should be a part of the decision aid on colon cancer screening.	Yes (26, 2, 0)
18. The case in favor of colon cancer screening should be made before presenting baseline risk.	Yes (26, 1, 1)
Trustworthiness
19. Decision aids recommended by health professionals are more likely to be trusted.	Yes (22, 3, 3)
20. Decision aids should refer to the source documents, although the decision aid should remain as simple as possible.	Yes (27, 1, 0)
21. The reputation of the organization who is producing and/or endorsing the decision aid impacts trustworthiness.	Yes (26, 2, 0)
22. Conflict of interest is not a major concern as long as the decision aid is produced or endorsed by a trusted organization.	Yes (24, 1, 3)
Not supported
23. Decision aids should contain narratives of patients’ and others’ experiences of colon cancer screening.	No (10, 10, 8)

Numbers given in parentheses are the vote totals in the following order: yes, no, abstain.

“Sixty out of 1000 people will get colon cancer without screening.”

Use of Decision Aids

Some deliberative conclusions supported decision aids being used by patients with their providers (conclusions 1 and 3), while others recommended that at least some decision aids should be designed to be used without a provider present and should be available outside of clinic appointments (conclusions 2 and 4). One conclusion recommended that a variety of health professionals should use aids to provide valid and consistent information to patients (conclusion 3).

Content: Encouraging Screening and Supporting Informed Choice

The participants strongly endorsed the idea that decision aids regarding colorectal cancer screening should encourage screening (conclusion 5). One deliberant said,

I guess our main topic was communicating the need for testing. So, that was the biggest takeaway. We want to make sure everything that we, um, put in the decision aid points to that you need to get tested. (Participant 9403; large group, day 2)

The participants also supported multiple deliberative conclusions stating that providers and decision aids should provide information to support informed choice (conclusion 9). The recommendation that decision aids disclose potential complications from colonoscopy (conclusion 7) was endorsed by some participants because it supports informed choice, even though it might reduce screening uptake. One deliberant said,

I don’t think you need to have an exhaustive list of everything because that could go on for pages. But the common, the bleeding, the tears, um, certainly those top three or four things, list them, because that was part of being informed. (Participant 9088; large group, day 2)

On multiple occasions, participants attempted to balance encouraging uptake with supporting fully informed choice. There was vigorous support for the idea that decision aids should disclose the cost of screening (conclusions 10 and 11), even though such disclosure could dissuade some patients from being screened. One deliberant said,

I think there should be an inclusion of a statement that the cost may be, because of the diversity and range of cost, it is something they need to pay attention to. (Participant 721; large group, day 3)

The question of how to balance encouraging screening and supporting informed choice also arose in discussion of deliberative conclusion 6: some thought that decision aids should describe reasons for a decision not to be screened, as a way of supporting informed choice, while others opposed this proposal, since they felt that it would imply that not being screened is as good an option as being screened and thus might discourage screening.

Sometimes the goals of encouraging screening and supporting informed choice were seen as synergistic, not antagonistic, for instance, in discussion of conclusion 8. Some participants felt that letting people know that there are noninvasive options for screening such as stool blood testing would inform patients about their options and also encourage screening, especially for people who are not ready to have a colonoscopy. Multiple participants reported that they had not been told about the option of stool testing and were upset about this. One deliberant said,

When I turned 50, my doctor said you need to have a colonoscopy. Well, I wasn’t informed about the other options. That was the only option I was given. . . . My daughter just had hers three months ago. She was not told or made aware that there were other options. So my problem is that people need to be informed. (Participant 9147; large group, day 2)

Content: Complexity and Tailoring

At multiple points in the discussion, participants broadly agreed that decision aids should be kept simple and understandable (conclusion 14). Multiple participants argued that decision aids should be tailored (e.g., conclusion 13), with some decision aids presenting a limited amount of information and additional information available if patients desired it. One deliberant said,

So, basically our analogy is two decision aids. One small one, one big one. Um, the small one, basic information, gearing to get tested, but at the same time telling you some of the key points, like it is the second most common killer, . . . but then the big book has everything else, all the data that you might want, all the numbers, all the statistics, so you have, you know, basically the headlines and then you have the rest of the story. (Participant 8977; large group, day 2)

Content: Quantitative Information

Participants generally supported the idea that decision aids should present some quantitative information, specifically the baseline risk of getting colorectal cancer (conclusions 15 and 17). Some participants expressed a concern that disclosing the low frequency of colorectal cancer (6%, or 60 per 1000) would dissuade patients from being screened. One deliberant said,

[If] you’re only 60 out of 1,000, then that doesn’t encourage you to go get a test. Uh, I think it should be that the baseline risk is minimal . . . I don’t think the data should be divulged. (Participant 401; large group, day 3)

Others thought that presenting risk data could support patients being screened, as one deliberant said,

The numbers should definitely be presented, um, but I think it depends on how they are presented as I said earlier. And quantitative information is going to be a lot more trustworthy in a decision aid because if you just tell me, you know, people don’t die or people will die, it’s kind of like, okay, so, what then? (Participant 418; large group, day 3)

Some participants tried to lessen the danger that quantitative information would reduce uptake by specifying that such data should be presented after a clear argument for being screened (conclusion 18).

Participants strongly endorsed decision aids’ highlighting the fact that colon cancer is the second leading cause of death by cancer in the United States (conclusions 12 and 16). Participants said that they thought this fact would provide strong motivation for patients to be screened. One deliberant said,

A good example would be what you have talked about, out of all cancers, it is number two. So, if people saw a pie chart, and now colon cancer took a big piece of that pie, they go, “Ah.” Well now it’s more important. (Participant 815; large group, day 3)

Trustworthiness

Participants strongly emphasized that patients will feel more trusting of decision aids that were recommended personally by a health professional (conclusion 19) or by a respected organization (conclusion 21). They felt that patients would not have important concerns about conflict of interest as long as the decision aid came from a trusted organization (conclusion 22). Participants said that decision aids should refer to sources for claims to support trustworthiness (conclusion 20).

Discussion

This was the first public deliberation carried out to gather laypersons’ recommendations for the design and evaluation of decision aids. Some of the deliberative conclusions directly support current recommendations. For instance, participants recommended that decision aids should be used both during conversations between providers and patients and also should be available without a provider present.^6,8,69 Participants supported the idea that decision aids should support informed choice by patients, a central goal of decision aids.^1,6,70 Other deliberative conclusions extend or challenge current guidelines, and our discussion will focus on these:

Disclosure of cost

Disclosure of limited amounts of quantitative information

Keeping decision aids simple and comprehensible

Disclosure of Cost

Participants unanimously recommended that decision aids should include explanation of cost and its variation depending on insurers and providers (conclusion 11). Some experts have described out-of-pocket costs as an important “side effect” of testing or treatment that should be disclosed to patients during decision making,⁷¹ but previous guidelines for design and evaluation of decision aids have said little or nothing about disclosing cost information.^6,8 This deliberative conclusion should be considered in future guidelines, especially for decision aids that will be used in the United States, where the health insurance system results in such costs. The wide variety of rules regarding copay, deductible, and coverage by health insurance policies in the United States, however, makes it difficult to provide a simple message about the cost for a specific patient for any procedure. In response to this challenge, participants emphasized the importance of at least highlighting the issue and encouraging patients to talk to their insurance company.

Quantitative Information

The participants were instructed about multiple types of quantitative risk and benefit information and presentation using numbers or graphs, including incidence and risk of mortality, absolute risk reduction, sensitivity and specificity of screening tests, and frequency of negative outcomes such as complications and false positives. Participants received this education about these types of information and framing, as well as their pros and cons for risk communication, in the briefing book, presentations on the first day and at the beginning of the second weekend, and in small group discussions.

But after considering all these ways of presenting quantitative measures of risk and benefit, the participants recommended disclosure of only baseline risk of CRC (conclusions 15 and 17) and the fact that colorectal cancer is the second largest cancer killer in the United States (conclusion 16). The lack of recommendations for disclosure of other types of quantitative information differs from the recommendations of current guidelines, such as the International Patient Decision Aids Standards and criteria issued by the National Quality Forum, which recommend disclosure of absolute risk reduction, frequency of negative outcomes, and positive and negative predictive value.^6,8,72

The absence of deliberative conclusions supporting disclosure of additional quantitative information may have reflected lack of interest or engagement by participants. More likely, we believe that the lack of deliberative conclusions recommending disclosure of additional quantitative measures of risk or benefit reflects a decision or sense by participants that such information is not essential to decision making and thus does not need to be disclosed to all patients by decision aids. This is a particularly striking finding since our group had relatively high numeracy and high levels of educational attainment (50% college graduates, compared to 36% for the US adult population).⁷³

In discussion, participants expressed concern that detailed quantitative information would distract patients from the key message that they should be screened and that this information could make the decision aid too complex. Participants supported disclosure of the fact that colorectal cancer is the second largest cancer killer in the United States (conclusions 12 and 16), largely as a way to motivate people to get screened. This sort of quantitative information—a categorical ranking of risk—is not recommended for disclosure by decision aids^6,8,72 or for risk communication in general,^74,75 in part due to concerns that it does not provide useful information for comparing risk and benefit.

Keeping Decision Aids Simple

The participants repeatedly emphasized the goal of keeping decision aids as simple as possible. Although it is a central principle of effective communication to keep messages short and easy to understand, this principle has been recognized only in some of the literature on decision aids. The International Patient Decision Aids Standards and National Quality Forum guidelines, for instance, recommend that decision aids present a substantial amount of information, and these guidelines provide limited acknowledgment of the danger that decision aids could become overly complex and unwieldly.^6,8,76 One approach that participants recommended repeatedly was that decision aids should present a limited amount of information initially and should allow individuals to choose to receive additional information if they desire.

Study Limitations

First, the research team took every effort to present relevant technical information and a range of positions to the participants, as well as to avoid directing them toward one position or another. In addition, we allowed the participants to direct the discussion and to generate deliberative statements themselves, with nondirective assistance from the facilitator. But it is also inevitable that the way that information was presented, questions were framed, and deliberation was conducted influenced the specific outcomes of the discussion. Second, despite 2 steps of randomization in the recruitment process, the participants were not a random sample from the population, since they initially chose to respond to the letter of invitation, and thus they may have reflected certain biases, such as being more proscreening than other members of society. The representation of African Americans and Hispanics among our participants (14% and 7%, respectively) roughly matched that of Indiana (9.9% and 7.3%, respectively), as planned. The percentage of African Americans in our study also approximated the percentage of African Americans in the United States overall (13.4%), but the percentage of Hispanics fell below the percentage in the United States overall (18.5%).⁷⁷ Finally, the deliberants’ discussion primarily addressed decision aids and decisions about colorectal cancer screening, so caution is necessary before extrapolating the results to decision aids in other areas. As discussed above, patients face a preference-based decision among the approved CRC screening modalities, since no one test is recommended over the others and choice of test depends on the values of the patient. The choice of whether to be screened at all, with some test, however, is not clearly preference sensitive, since there is a strong recommendation for screening with at least some test for all people ages 50 to 75 years who have average risk for CRC. Participants in our exercise focused largely on the goal of getting more patients screened with some test, which affected their discussion of how to support patients making a preference-sensitive choice among the approved tests. Furthermore, even when deliberants recommended disclosing information to support an informed choice, it is possible they were considering the decision about whether to be screened rather than choosing among the approved tests. Finally, the participants may have conceived of the challenge for patients as being a single decision, selecting from 3 options (no screening, colonoscopy, or FIT), which would mix preference-sensitive and non-preference-sensitive options. Therefore, the recommendation made by our participants may not be directly applicable to choices that are purely preference sensitive. In addition, a group that was not so focused on increasing uptake of colorectal cancer screening might have made different recommendations regarding decision aids on this topic.

Conclusion

Laypeople in the public deliberation endorsed key aspects of current guidelines for the design and evaluation of decision aids but did not endorse the current recommendation that decision aids should disclose extensive amounts of quantitative information to patients facing preference-sensitive decisions. Participants emphasized the importance of keeping decision aids relatively short and simple, to encourage engagement and understanding by patients, and supported tailoring decision aids for different users. These results should be taken into account in the future development of guidelines for the design and evaluation of decision aids.

Supplemental Material

sj-pdf-1-mdm-10.1177_0272989X21998980 – Supplemental material for Layperson Views about the Design and Evaluation of Decision Aids: A Public Deliberation

Supplemental material, sj-pdf-1-mdm-10.1177_0272989X21998980 for Layperson Views about the Design and Evaluation of Decision Aids: A Public Deliberation by Peter H. Schwartz, Kieran C. O’Doherty, Colene Bentley, Karen K. Schmidt and Michael M. Burgess in Medical Decision Making

Footnotes

Acknowledgements

The authors thank the following persons for their contributions to the successful completion of this project: our patient and community advisory boards, which provided insightful and valuable help; Sabrina Cordon, Ariane Thomas, and Amy Price for their recruitment efforts and project support; Apurv Chauhan, Joban Dhanoa, Kristie Serota, and Sarah Wiebe for facilitating the small deliberation groups; Michael Barry, MD, Bridget Gaglio, PhD, MPH, Paul K. J. Han, MD, MPH, Thomas F. Imperiale, MD, Susan M. Rawl, RN, PhD, and Brian J. Zikmund-Fisher, PhD, for assistance on the preparation of the briefing book and for presenting at the public deliberation; and all our public deliberation participants.

This article was an oral presentation at the 39th Annual Meeting of the Society for Medical Decision Making; October 25, 2017; Pittsburgh, PA.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided entirely by a contract with the Patient-Centered Outcomes Research Institute (PCORI)® (CDR-1403-11040). The statements presented in this publication are solely the responsibility of the author(s) and do not represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors, or Methodology Committee. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

ORCID iD

Peter H. Schwartz

Supplemental Material

Supplementary material for this article is available on the Medical Decision Making website at .

References

Stacey

Legare

Lewis

, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2017;4:CD001431.

Syrowatka

Kromker

Meguerditchian

Tamblyn

Features of computer-based decision aids: systematic review, thematic synthesis, and meta-analyses. Journal of Medical Internet Research. 2016;18(1):e20.

O’Connor

Tugwell

Wells

, et al. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns. 1998;33(3):267–79.

O’Connor

AM.

From imitation to creation: the evolution of a research program in decision support. In: Edwards

Roelofs

, eds. Developing a Program of Research: An Essential Process for Successful Research Career. Vancouver, BC, Canada: Bright Wing Media.

O’Connor

Drake

Fiset

Graham

Laupacis

Tugwell

The Ottawa patient decision aids. Eff Clin Pract. 1999;2(4):163–70.

Elwyn

O’Connor

Stacey

, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006; 333(7565):417.

International Decision Aid Standards (IPDAS). Collaboration—resources. Available from: http://ipdas.ohri.ca/resources.html

National Quality Forum. National standards for the certification of patient decision aids: final report. Available from: http://www.qualityforum.org/Publications/2016/12/National_Standards_for_the_Certification_of_Patient_Decision_Aids.aspx

Pope

TM.

Certified patient decision aids: solving persistent problems with informed consent law. J Law Med Ethics. 2017;45(1):12–40.

10.

Washington Health Care Authority. Patient decision aids (PDAs). Available from: https://www.hca.wa.gov/about-hca/healthier-washington/patient-decision-aids-pdas

11.

Lee

Emanuel

EJ.

Shared decision making to improve care and reduce costs. N Engl J Med. 2012;368(1):6–8.

12.

Trevena

Zikmund-Fisher

Edwards

, et al. Presenting quantitative information about decision outcomes: A risk communication primer for patient decision aid developers. BMC Med Inform Decis Mak 2013; 13(Supplement 2): S7.

13.

Berry

DC.

Communicating risk of medication side effects: an empirical evaluation of EU recommended terminology. Psychol Health Med. 2003;8(3):13.

14.

Peters

Hart

Tusler

Fraenkel

Numbers matter to informed patient choices: a randomized design across age and numeracy levels. Med Decis Making. 2014;34(4):430–42.

15.

Webster

Weinman

Rubin

GJ.

How does the side-effect information in patient information leaflets influence peoples’ side-effect expectations? A cross-sectional national survey of 18- to 65-year-olds in England. Health Expect. 2017;20(6):1411–20.

16.

Smith

Webster

Rubin

GJ.

A systematic review of factors associated with side-effect expectations from medical interventions. Health Expect. 2020;23(4):731–58.

17.

Schwartz

PH.

Questioning the quantitative imperative decision aids, prevention, and the ethics of disclosure. Hastings Cent Rep. 2011;41(2):30–9.

18.

Zikmund-Fisher

BJ.

The right tool is what they need, not what we have: a taxonomy of appropriate levels of precision in patient risk communication. Med Care Res Rev. 2013;70(1, Suppl):37S–49S.

19.

McDonald

Charles

Gafni

Assessing the conceptual clarity and evidence base of quality criteria/standards developed for evaluating decision aids. Health Expectations. 2014;17(2):232–43.

20.

Reyna

Nelson

Han

Dieckmann

NF.

How numeracy influences risk comprehension and medical decision making. Psychol Bull. 2009;135(6):943–73.

21.

Nelson

Reyna

Fagerlin

Lipkus

Peters

Clinical implications of numeracy: theory and practice. Ann Behav Med. 2008;35(3):261–74.

22.

Nayak

Hartzler

Macleod

Izard

Dalkin

Gore

JL.

Relevance of graph literacy in the development of patient-centered communication tools. Patient Educ Counsel. 2016;99(3):448–54.

23.

Ancker

Senathirajah

Kukafka

Starren

JB.

Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc. 2006;13(6):608–18.

24.

Reyna

VF.

How people make decisions that involve risk: a dual-processes approach. Curr Directions Psychol Sci. 2004;13(2):60–6.

25.

Gigerenzer

Why heuristics work. Perspectives Psychol Sci. 2008;3(1):20–9.

26.

Reyna

VF.

A theory of medical decision making and health: fuzzy trace theory. Med Decis Making. 2008;28(6):850–65.

27.

Blalock

Reyna

VF.

Using fuzzy-trace theory to understand and improve health judgments, decisions, and behaviors: a literature review. Health Psychol. 2016;35(8):781–92.

28.

Schneider

CE.

The Practice of Autonomy: Patients, Doctors, and Medical Decisions. New York: Oxford University Press; 1998.

29.

Manson

O’Neill

Rethinking Informed Consent in Bioethics. Cambridge, UK: Cambridge University Press; 2007.

30.

O’Neill

Autonomy and Trust in Bioethics. Cambridge, UK: Cambridge University Press; 2002.

31.

Epstein

Alper

Quill

TE.

Communicating evidence for participatory decision making. JAMA. 2004;291(19):2359–66.

32.

Epstein

Korones

Quill

TE.

Withholding information from patients—when less is more. N Engl J Med. 2010;362(5):380–1.

33.

Gaster

Edwards

Trinidad

Gallagher

Braddock

III . Patient-centered discussions about prostate cancer screening: a real-world approach. Ann Intern Med. 2010;153(10):661–5.

34.

Schwartz

Meslin

EM.

The ethics of information: absolute risk reduction and patient understanding of screening. J Gen Intern Med. 2008;23(6):867–70.

35.

Fagerlin

Pignone

Abhyankar

, et al. Clarifying values: an updated review. BMC Med Inform Decis. 2013;13(Suppl 2):S8.

36.

Witteman

Gavaruzzi

Scherer

, et al. Effects of design features of explicit values clarification methods: a systematic review. Med Decis Making. 2016;36(6):760–76.

37.

Witteman

Scherer

Gavaruzzi

, et al. Design features of explicit values clarification methods: a systematic review. Med Decis Making. 2016;36(4):453–71.

38.

Nelson

Han

Fagerlin

Stefanek

Ubel

PA.

Rethinking the objectives of decision aids: a call for conceptual clarity. Med Decis Making. 2007;27(5):609–18.

39.

Wilson

Lisle

Schooler

Hodges

Klaaren

LaFleur

SJ.

Introspecting about reasons can reduce post-choice satisfaction. Pers Soc Psychol Bull. 1993;19(3):331–9.

40.

Wilson

Schooler

JW.

Thinking too much: introspection can reduce the quality of preferences and decisions. J Pers Soc Psychol. 1991;60(2):181–92.

41.

O’Doherty

KC.

Public Engagement and Emerging Technologies. Vancouver: UBS Press; 2012.

42.

O’Doherty

Gauvin

F-P

Grogan

Friedman

Implementing a public deliberative forum. Hastings Cent Rep. 2012;42(2):20–3.

43.

Gastil

Designing public deliberation at the intersection of science and public policy. In: The Oxford handbook of the Science of Science Communication. 2017. p 233–42. New York: Oxford University Press.

44.

Blacksher

Diebel

Forest

P-G

Goold

Abelson

What is public deliberation?

Hastings Cent Rep. 2012;42(2):14–17.

45.

Street

Duszynski

Krawczyk

Braunack-Mayer

The use of citizens’ juries in health policy decision-making: a systematic review. Soc Sci Med. 2014;109:1–9.

46.

Hamlett

PW.

Technology theory and deliberative democracy. Sci Technol Hum Val. 2003;28(1):112–40.

47.

Abelson

Blacksher

Boesveld

Goold

Public deliberation in health policy and bioethics: mapping an emerging, interdisciplinary field. J Public Deliberation. 2013;9(1):jdd157.

48.

Burgess

MM.

Public consultation in ethics: an experiment in representative ethics. J Bioethical Inquiry. 2004;1(1):4–13.

49.

Burgess

MM.

From ‘trust us’ to participatory governance: deliberative publics and science policy. Public Underst Sci. 2014;23(1):48–52.

50.

Burgess

Longstaff

O’Doherty

Assessing deliberative design of public input on british columbia biobanks. In: Dodds

Ankeny

, eds. Big Picture Bioethics: Developing Democratic Policy in Contested Domains. Cham, Switzerland: Springer International; 2016. p 263–78.

51.

Hawkes

“Citizens’ jury” disagrees over whether screening leaflet should put reassurance before accuracy. BMJ. 2012;345:e8047.

52.

Baena-Canada

Luque-Ribelles

Quilez-Cutillas

, et al. How a deliberative approach includes women in the decisions of screening mammography: a citizens’ jury feasibility study in Andalusia, Spain. BMJ Open. 2018;8(5):e019852.

53.

Beauchamp

Childress

JF.

Principles of Biomedical Ethics. 8th ed. Oxford, UK: Oxford University Press; 2019.

54.

Schwartz

Imperiale

Perkins

Schmidt

Althouse

Rawl

SM.

Impact of including quantitative information in a decision aid for colorectal cancer screening: a randomized controlled trial. Patient Educ Couns. 2019;102(4):726–34.

55.

Schwartz

Perkins

Schmidt

Muriello

Althouse

Rawl

SM.

Providing quantitative information and a nudge to undergo stool testing in a colorectal cancer screening decision aid: a randomized clinical trial. Med Decis Making. 2017;37(6):688–702.

56.

Bibbins-Domingo

Grossman

, et al. Screening for colorectal cancer: US Preventive Services Task Force recommendation statement. JAMA. 2016;315(23):2564–75.

57.

Rex

Boland

Dominitz

, et al. Colorectal cancer screening: recommendations for physicians and patients from the U.S. Multi-Society Task Force on Colorectal Cancer. Gastroenterology. 2017;153(1):307–23.

58.

Klabunde

Joseph

King

White

Plescia

Vital signs: colorectal cancer screening test use—United States, 2012. Morb Mortal Wkly Rep. 2013;62(44):881–8.

59.

Zauber

Lansdorp-Vogelaar

Knudsen

Wilschut

van Ballegooijen

Kuntz

KM.

Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med. 2008;149(9):659–69.

60.

Heitman

Hilsden

Dowden

Manns

BJ.

Colorectal cancer screening for average-risk North Americans: an economic evaluation. PLoS Med. 2010;7(11):e1000370.

61.

Schwartz

Imperiale

Perkins

, et al. Using numbers in a decision aid to describe risks and benefits of colorectal cancer screening options. Patient-Centered Outcomes Research Institute (PCORI). 2020. https://www.pcori.org/sites/default/files/Schwartz242-Final-Research-Report.pdf

62.

Schroy

III Emmons

Peters

, et al. The impact of a novel computer-based decision aid on shared decision making for colorectal cancer screening: a randomized trial. Med Decis Making. 2011;31(1):93–107.

63.

Smith

Trevena

Simpson

Barratt

Nutbeam

McCaffery

KJ.

A decision aid to support informed choices about bowel cancer screening among adults with low education: randomised controlled trial. BMJ. 2010;341:c5370.

64.

Eddy

Designing a practice policy: standards, guidelines, and options. JAMA 1990;263(22):3077, 3081, 3084.

65.

Fagerlin

Zikmund-Fisher

Ubel

Jankovic

Derry

Smith

DM.

Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Med Decis Making. 2007;27(5):672–80.

66.

Zikmund-Fisher

Smith

Ubel

Fagerlin

Validation of the Subjective Numeracy Scale: effects of low numeracy on comprehension of risk communications and utility elicitations. Med Decis Making. 2007;27(5):663–71.

67.

McNaughton

Cavanaugh

Kripalani

Rothman

Wallston

KA.

Validation of a short, 3-item version of the Subjective Numeracy Scale. Med Decis Making. 2015;35(8):932–6.

68.

Galesic

Garcia-Retamero

Statistical numeracy for health: a cross-cultural comparison with probabilistic national samples. Arch Intern Med. 2010;170(5):462–8.

69.

Elwyn

Miron-Shatz

Deliberation before determination: the definition and evaluation of good decision making. Health Expect. 2010;13(2):139–47.

70.

Volk

Llewellyn-Thomas

Stacey

Elwyn

Ten years of the International Patient Decision Aid Standards Collaboration: evolution of the core dimensions for assessing the quality of patient decision aids. BMC Med Inform Decis Mak. 2013;13(Suppl 2):S1.

71.

Ubel

Abernethy

Zafar

SY.

Full disclosure—out-of-pocket costs as side effects. N Engl J Med. 2013;369(16):1484–6.

72.

Trevena

Zikmund-Fisher

Edwards

, et al. Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers. BMC Med Inform Decis Mak. 2013;13(Suppl 2):S7.

73.

US Census Bureau. U.S. Census Bureau releases new educational attainment data. Release number CB20-TPS.09. Available from: https://www.census.gov/newsroom/press-releases/2020/educational-attainment.html

74.

Gigerenzer

Edwards

Simple tools for understanding risks: from innumeracy to insight. BMJ. 2003;327(7417):741–4.

75.

Paling

Strategies to help patients understand risks. BMJ. 2003;327(7417):745–8.

76.

Feldman-Stewart

O’Brien

Clayman

, et al. Providing information about options in patient decision aids. BMC Med Inform Decis Mak. 2013;13(Suppl 2):S4.

77.

US Census Bureau. Quick Facts: Indiana, US. Available from: https://www.census.gov/quickfacts/fact/table/IN,US/PST045219

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.95 MB