Abstract
Highlights
Communicating relative risk differences as opposed to absolute risk differences, using numerator-only instead of part-to-whole graphics, and including anecdotes or information about others’ decisions will all increase intentions to engage in a behavior.
Relative risks (rather than absolute risk differences) and numerator-only graphics (rather than part-to-whole) will also increase felt and perceived effectiveness.
To illustrate probability differences, people tend to prefer bar charts over icon arrays and graphics with labels over those without.
All findings regarding the impact of different presentation formats for probability differences on trust produced insufficient evidence.
Numerical information is essential for patients to understand probabilities of health and disease. In particular, differences between 2 probabilities are used to express the effects of risk factors or therapies on the probability of disease, health, or other outcomes. The patient may have to assess the size of the effect to select therapeutic options, make behavior changes to reduce risk, or make other health-related decisions. Such differences may be formatted in a number of ways: as 2 individual probabilities (e.g., a change from a 3% risk to a 4.5% risk), the relative difference between those probabilities (a 50% relative increase, a relative risk of 1.5), the absolute difference between them (a 1.5-percentage-point absolute increase), some combination of these, or some other format. Although all these formats express the same information, it is well established that the format can influence the reader’s perceptions and decisions.
To develop evidence about the effects of different formats on patients and the public, we conducted a large systematic literature review on the communication of numbers in health, across data types and across different data presentation formats.1,2 We organized the literature according to a conceptual model of communication in which a reader views a stimulus (consisting of
Findings of the systematic review were too large to present in a single article and thus are being presented as a series (Table A). The current article presents the evidence pertaining to probability data, difference tasks, and the following 5 outcomes: 1) effectiveness perception (perception of the size of a probability difference) or feeling (perception assessed on a scale of worry, concern, or other affective response), 2) behavioral intentions (intended or planned behavior) or health behavior, 3) trust (perceived credibility of the information as presented), 4) preference or positive perception of the data presentation format (e.g., perceived helpfulness and other related constructs), and 5) discrimination between probability differences of different sizes. A separate article labeled “Part 1” presents 5 additional outcomes regarding communication of probability differences.
Current Article’s Scope within the Making Numbers Meaningful Systematic Review
This standardized numbering system has been used for results subheadings in this article and across all Making Numbers Meaningful results articles to ensure that readers can find comparable information in all articles. Gray cells represent combinations that are not possible according to the definitions presented in Ancker et al. 1
Methods
As described in more detail in our companion methodology article, 1 the literature review sought experimental (randomized) and quasi-experimental (prospectively collected questionnaires or surveys that lacked random assignment) research comparing 2 or more ways of presenting quantitative health-related data to lay, nonmedical audiences. We followed systematic methods for the literature search, screening, risk-of-bias evaluation, data extraction, credibility evaluation of findings, and organization into evidence tables. 1 The search was performed on MEDLINE, Embase, CINAHL, the Cochrane Library, PsycINFO, ERIC, ACM Digital Library, and we conducted hand searches of tables of contents of Medical Decision Making, Patient Education and Counseling, Risk Analysis, and Journal of Health Communication. All instruments used (search strategy, data extraction instrument, and study risk of bias or S-ROB rubric) are available at the Making Numbers Meaningful Project at the Open Science Framework site (https://osf.io/rvxf2/).
We assigned each included study a study risk of bias (S-ROB) score according to a rubric developed for this project, which considered sample representativeness, randomization, protocol deviations, presence/absence of demographic and covariate information, missing data, and other potential biases. We created a brief free-text description of sample demographics, which is available with the rest of the data at the OSF repository (https://osf.io/rvxf2/). Within each included study, we extracted information about task, stimulus (data and data presentation format), and outcome. The outcomes were informed by behavioral and risk communication theory (behavior or behavioral intention, effectiveness perceptions or feelings, recall) or empirically on the basis of what was frequently measured by the research included in our review (trust, preference for a format), particularly measures used to measure comprehension (identification, contrast, computation, categorization, discrimination). Although we categorized the outcomes in this way, it is important to recognize that the actual measures used in the different studies varied.
During data extraction, each unique combination of task, format comparison, and outcome from a single study was termed a finding. For example, a single study could provide 1 finding about the effect of relative versus absolute risk difference on perceived effectiveness and another finding about the effect of relative versus absolute risk difference on health behavior. Each finding was rated for credibility on a scale from 1 to 10 by pairs of authors (N.C.B., J.S.A., B.J.Z.-F.) holistically assessing a list of finding and study characteristics (sample size, statistical methods, validity of stimulus design, comparison, outcome measures, and covariates, plus the S-ROB for the study from which the finding came). Findings from the same study often varied in credibility; for example, primary analyses with good statistical power might have high credibility, and unplanned secondary or subset analyses with small samples might have lower credibility. A credibility of 7 or higher was considered high, 4.5 to 6.5 moderate, and 4 or below low. Consistency was considered high if all findings were significant in the same direction or if a large majority were significant in one direction with a few lacking in significance, moderate if findings showed a small majority of significant effects in one direction with the remainder lacking significance, and low if the findings showed significant effects in different directions. After grouping relevant findings together, we graded the strength of evidence as follows:
For the current article, it is important to explain a few terms. Among probabilities, we distinguished between rates formatted as 1 in X (such as “1 in 5”) and those formatted as a rate per 10n (such as “20 in 100”). We used the term natural frequency only to describe a series of joint probabilities and conditional probabilities computed from the same pool of patients, in the context of Bayes’ theorem.3,4 By doing so, we intended to match the original definition of the term 4 and distinguish between findings that otherwise might appear contradictory. 3
The literature review identified 316 articles addressing probability communication, of which 84 (representing 101 unique studies) involved difference tasks with probability data and 1 or more of the outcomes for this article.
Results
As outlined in Table B, each “Results” subsection summarizes evidence on the following comparisons in order: comparisons among number formats, among graphics formats, between number and graphic formats, between number and verbal formats, between different types of contextual elements, between different framings, effect of manipulations of denominators, effect of animation or interactivity, and manipulations of time frames. No articles examined the effect of representations of uncertainty. Within subsections, evidence is arranged from strongest to weakest.
Section Headings for Each Subset of Outcome Evidence Included in This Article and the Number of Included Findings
No findings pertained to the representation of uncertainty (row G).
The subhead numbering system in Table B is standard across all Making Numbers Meaningful results articles. The standard numbers ensure that, for example, studies of the effects of gain-loss framing manipulations on health behavior are always placed in a subhead labeled section 7F (whether or not that article contains sections 1 through 6). The goal is for readers to be able to use this system to locate similar sections across articles.
Effects of Different Formats for Probability Differences on Effectiveness Perceptions and Feelings: Section 6
The way that a probability difference (the representation of the effect of a risk factor or therapy) is formatted has the potential to influence the perception of how large or important that effect is. When the perception was measured on a scale from small to large, we considered it a measure of “effectiveness perception,” and when it was measured on an affective scale such as “not at all concerned” to “very concerned,” we considered it “effectiveness feelings.” Although these 2 outcomes were extracted separately, they are similar and so we group them into section 6.
It is important to note that we report throughout whether a format was associated with increases or decreases in perceived effectiveness, not whether it was associated with increases or decreases in accuracy. The concept of accuracy—that is, the ability to correctly identify or recall a specific number or quantity—is covered in our companion part 1 article, where it appears under section 1, the so-called identification/recall outcome.
Comparisons between numerical formats for probability differences in their effect on effectiveness perceptions and feelings (subsection 6A)
Evidence-Based Guidance for Effects of Numerical Formats for Probability Differences on Effectiveness Perceptions and Feelings
EFFECT OF SHOWING RELATIVE DIFFERENCES: A high-credibility finding showed that perceived effectiveness was higher when a relative risk reduction as a percentage was added to a pair of pre and post percentages. 5 Similarly, a moderate credibility finding 6 found higher perceived effectiveness with relative outcome rates than absolute probabilities.
INCREMENTAL RISK PLUS BASELINE RISK VERSUS PRE/POST ABSOLUTE RATES: High- and moderate-credibility findings (Zikmund-Fisher et al. 7 substudy 1, Zikmund-Fisher et al. 8 ) showed less concern about side effects when the effects were presented as baseline rates per 10n plus absolute difference than when they were shown as a pair of pre/post absolute rates per 10n.
TABLES VERSUS TEXT: A high-credibility finding by Tait et al. 9 showed no difference in effectiveness perception when rate per 10n numbers were presented in tables or in text.
EFFECT OF SHOWING ARITHMETIC DIFFERENCES: A moderate-credibility finding 10 showed effectiveness perception was higher when arithmetic differences were accompanied by absolute rates than when the differences were presented alone; however, the effect was only in a high-benefit condition but not for a similarly designed low-benefit condition.
PERCENTAGE PROBABILITY VERSUS LIFE EXPECTANCY: A moderate-credibility finding by Galesic and Garcia-Retamero 11 showed perceived differences caused by a risk factor to be larger (as measured by lower perceived desirability) when post–risk factor estimates plus absolute difference statistics were framed in life expectancy terms rather than percentages.
Two low-credibility findings were not summarized (Blalock et al. 10 due to inconsistent findings across conditions with high and low benefit, and Wadhwa and Zhang 12 due to a small sample size, which reduced confidence in nonsignificant findings).
Comparisons between graphic formats for probability differences in their effect on effectiveness perceptions and feelings (subsection 6B)
Evidence-Based Guidance for Effects of Graphical Formats for Probability Differences on Effectiveness Perceptions and Feelings
PART-WHOLE VERSUS FOREGROUND-ONLY GRAPHICS: Five high and moderate-credibility findings involve comparing part-to-whole graphics to numerator-only graphics, consistently finding that people perceived larger probability differences with numerator-only formats. A part-to-whole graphic portrays both the numerator and denominator of a probability, such as an icon array representing a 10% risk with 10 colored icons and 90 white icons. A numerator-only graphic shows only the numerator (e.g., 10 affected individuals alone). These include research on icon arrays (graphics that portray the probability of an event with colored human figures or abstract icons; Okan et al. 13 substudy 2, Stone et al. 14 substudy 2), bar charts (Okan et al., 15 Stone et al. 16 substudy 2), and a comparison between numerator-only icon arrays and pie charts (Stone et al. 16 substudy 1). The latter 2 findings showed the same impact on effectiveness feelings.
BASELINE PLUS ABSOLUTE DIFFERENCE VERSUS PRE/POST ABSOLUTE RATES PORTRAYED AS ICON ARRAYS: High- and moderate-credibility findings (Zikmund-Fisher et al. 7 substudy 1, Zikmund-Fisher et al. 8 ) showed less concern about side effects when the effects were shown in icon array graphics as baseline probability plus absolute difference than when they were shown as a pair of pre/post absolute probabilities.
USE OF NUMBER LABELS: A moderate credibility finding 15 found lower perceived effectiveness of a drug therapy when rate per 10n data labels was added to bar charts. However, a moderate-credibility finding 17 showed that adding number labels to vertical number lines with interpretive labels did not affect the perception of the effectiveness of exercise.
ADDING POSTINTERVENTION PROBABILITY TO BASELINE PROBABILITY: Janssen et al. 17 also included a moderate-credibility finding that adding postintervention probabilities (thereby visually illustrating the absolute risk difference) to vertical number lines did not affect perception of the effectiveness of exercise.
ABSOLUTE RISK DIFFERENCES VERSUS RISK RATIOS: A moderate-credibility finding 18 showed greater changes in effectiveness feelings for policies to reduce population-level risk disparities when grouped bar charts showed risk ratios instead of absolute differences.
Comparisons between numerical and graphical formats for probability differences in their effect on effectiveness perceptions and feelings (subsection 6C)
Evidence-Based Guidance for Contrasts between Numerical and Graphical Formats for Probability Differences, and Combinations of Numerical and Graphical Formats, on Effectiveness Perceptions and Feelings
COMBINING GRAPHICS WITH PAIRS OF PROBABILITIES (PERCENTAGE OR RATE PER 10n): A high-credibility finding 5 showed no effect of adding part-to-whole icon arrays to pre-post percentage with or without RRR as percentage. Both a high-credibility finding (Zikmund-Fisher et al. 7 substudy 1) and a moderate-credibility finding 8 showed no effect on concern about a set of side effects of adding part-to-whole icon arrays to pairs of rate per 10n numbers, regardless of whether the information was pre/post absolute rates or baseline values plus arithmetic differences. Similarly, a moderate-credibility finding (Dragicevic and Jansen 19 substudy 2) suggested no differences between risk difference presented as pair of percentages with or without a part-to-whole bar chart. Also, Martin et al. 20 (a moderate-credibility finding) demonstrated no differences between presenting risk difference as arithmetic risk difference percentage alone or with the addition of icon arrays, speedometer graphics, or photos.
However, a high-credibility finding 7 demonstrated that in risk-benefit communication with pairs of pre-post probabilities (rate per 10n and percent) plus arithmetic differences, effectiveness perception of the drug was lowest with a data table alone and highest when the numbers were supplemented with integrated icon arrays for each harm and benefit.
A moderate-credibility finding 21 showed that the effects of varying denominators when presenting treatment effectiveness as rates per 10n could be significantly reduced by adding pairs of part-to-whole icon arrays.
NUMERATOR-ONLY GRAPHICS VERSUS ABSOLUTE PROBABILITIES OR PROBABILITY DIFFERENCES (RATE PER 10n): In high- and moderate-credibility findings (Stone et al. 14 substudy 1 and Stone et al. 14 substudy 2), each communicating benefit from improved product as pair of pre-post rate per 10n, a combined measure of effectiveness perception and probability of harm was higher when benefit was shown as a pair of foreground-only icon arrays than with the rate per 10n only. (Note that the outcome measure combines perception of the difference between probabilities with perception of individual probabilities, reducing confidence in conclusions about either alone.) A moderate-credibility finding also demonstrated that foreground/numerator-only icon arrays resulted in higher effectiveness feelings than pairs of rate per 10n numbers (Stone et al. 16 substudy 1 perceptions outcome), although effectiveness perception did not differ significantly (Stone et al. 16 substudy 1 feelings outcome). A moderate-credibility finding similarly demonstrated that foreground/numerator-only bar charts led to higher effectiveness perception and effectiveness feelings as compared with rate per 10n (Stone et al. 16 substudy 2 perception outcome and Stone et al. 16 substudy 2 feelings outcome). These findings assessed the effect only for relatively rare events.
PART-TO-WHOLE GRAPHICS VERSUS ABSOLUTE PROBABILITY DIFFERENCES (RATE PER 10 n): Two moderate-credibility findings from a single article compared rate per 10n numeric formats to graphical formats. Stone et al. 16 substudy 1 (perception and feelings outcomes) showed that pairs of rates per 10n resulted in higher perceived and effectiveness feelings than pie charts did, while Stone et al. 16 substudy 2 demonstrated the same effects of rate per 10n compared with horizontal part-to-whole bar charts (perception and feelings outcomes). Both findings assessed the effect only for relatively rare events, decreasing confidence in the generalizability of this finding for larger probabilities. However, a small moderate-credibility finding 20 showed that in communicating the effect of a hypothetical anti–rheumatoid arthritis drug, there was no difference in effectiveness perception by format (arithmetic difference in percentage, arithmetic difference in percentage plus graphic photos, pair of icon arrays, or pair of speedometer illustrations).
OTHER COMBINATIONS: One moderate-credibility finding (Vogt and Marteau 22 substudy 1) demonstrated that presenting a risk difference as rate per 10n plus icon array was associated with higher effectiveness perceptions compared with 1-in-X number formats plus relative risk reduction. Effectiveness feelings were not assessed.
Two low-credibility findings (Dragicevic and Jansen 19 substudies 1 and 3) were not summarized due to small sample sizes and in 1 case ceiling effects.
Comparisons of elements added for context on effectiveness perceptions and feelings (subsection 6D)
Evidence-Based Guidance on Effects of Contextual Elements for Probability Differences on Effectiveness Perceptions and Feelings.
VISUAL EMPHASIS ON A DENOMINATOR: A high-credibility finding (Stone et al. 14 substudy 1) demonstrated no difference in perception of differences when a foreground-only icon array accompanied by the numerical denominator had salience increased by making the denominator number bold and red.
ADDING AVERAGE CHANCE TO INDIVIDUAL CHANCE: A high-credibility finding 23 demonstrated that people rated the effectiveness of a drug as more important when they were told that they were at higher-than-average risk than when they were told they were at lower-than-average risk (when the actual quantitative risk was held constant). However, a moderate-credibility finding 17 demonstrated that when effects of exercise on risk were shown in a risk ladder, effectiveness perceptions were not affected by adding a second vertical number line showing the “average person’s risk.”
Comparisons of larger versus smaller denominators for probability differences on effectiveness perceptions and feelings (subsection 6H)
Evidence-Based Guidance on Effect of Manipulating Denominators of Probability Differences on Effectiveness Perceptions and Feelings
LARGER VERSUS SMALLER DENOMINATORS: One moderate-credibility finding 21 demonstrated that denominator neglect in perceived effectiveness was reduced by supplementing rates per 10n with different denominators with pairs of icon arrays. The attenuation of denominator neglect was particularly marked for those who were reading information not in their native language. A high-credibility finding (Zikmund-Fisher et al.) 7 found reduced concern about rarer side effects when presented as rates per 10n with a denominator of 100 versus 1,000, regardless of whether icon array displays were added. Furthermore, in a moderate-credibility finding (Zikmund-Fisher et al.), 8 concern about side effects was not affected by varying the denominator of rates per 10n and/or icon arrays between 100 and 1,000.
Comparisons of longer versus shorter time period for probability differences on effectiveness perceptions and feelings (subsection 6J)
Evidence-Based Guidance on Effect of Varying the Time Period for Probability Differences on Effectiveness Perceptions and Feelings
LONGER VERSUS SHORTER TIME PERIOD: One moderate-credibility finding (Vogt and Marteau 22 substudy 2) demonstrated that effectiveness perceptions were higher for effects presented over a 1-mo time frame than over a 12-mo time frame (rates per 10n plus icon arrays). However, in this case of a smoking cessation intervention, the chance of relapse accumulated over time so that the stated probability of being smoke free was lower in the long term than the short term, which likely affected perceptions.
Effects of Different Formats for Probability Differences on Health Behaviors and Behavioral Intentions: Section 7
Promoting specific health behaviors is an important goal of health communication. In many studies we found, behavioral intentions were measured rather than actual behaviors. Because of the relative rarity of communication studies tracking actual behavior, and because of the theoretical grounding (in the health belief model and other health behavior theories) suggesting that intentions are correlated with behaviors, we have grouped intention and behavior in this section.
Comparisons between number formats for probability differences on health behavior and behavioral intention (subsection 7A)
Evidence-Based Guidance for Effects of Numerical Formats for Probability Differences on Health Behavior and Behavioral Intention
RELATIVE RISK DIFFERENCE VERSUS PAIRS OF PRE-POST PROBABILITIES: A large majority of findings including many high-credibility ones shows that presenting a risk difference as a relative risk difference alone or in combination with other statistics prompts stronger behavioral intention than presenting the difference as pairs of pre-post values or the arithmetic difference between them. Fourteen findings showed an effect (Goodyear-Smith et al. 24 substudy 1, Gyrd-Hanson et al., 25 Berglund et al., 26 Malenka et al. 27 substudy 1, Misselbrook and Armstrong, 28 Stovring et al., 29 Sinsky et al., 30 Sarfati et al., 31 Heard et al. 32 substudy 1, Carling et al., 33 Stone et al. 34 substudy 1 and 2, Berry et al., 35 Hux and Naylor 36 ). However, 3 findings37–39 did not show a statistically significant effect.
NNT VERSUS PROBABILITY FORMATS: Four high-credibility (Goodyear-Smith et al. 24 substudy 1, Misselbrook and Armstrong, 28 Stovring et al., 29 Sarfati et al. 31 ) and 2 moderate-credibility33,39 findings show that the number needed to treat (NNT) format is less effective at prompting behavioral intention than most other probability comparison formats. However, 1 high-credibility finding, Gyrd-Hansen et al., 38 did not find this effect.
EFFECTIVE AGE VERSUS PROBABILITY FORMATS: Three findings about “heart age” or “effective age” as an alternative to probabilities do not strongly support the use of this concept to promote behavioral intention. In 2 high-credibility findings, Heard et al. 32 substudies 1 and 2 demonstrated that “effective age” prompted weaker behavioral intentions than relative risk reduction or as time lost from life (years or hours).
A single high-credibility finding, Bonner et al., 40 showed no difference in behavioral intention between heart age and pairs of pre-post percentage.
DELAY OF EVENT VERSUS PROBABILITY FORMATS: Five high-credibility findings examined the related concepts of delay of event or prolongation/shortening of life. Although Berglund et al. 26 (high credibility) demonstrated that conveying a risk difference as years of delay of event led to behavioral intentions nearly as strong as with relative risk reduction, Stovring et al. 29 (high credibility) demonstrated that supplementing visuals with prolongation of life information was less effective in changing behavioral intention than adding relative risk reduction. Harmsen et al. 41 (high credibility) demonstrated that prolongation of life led to lower behavioral intentions than the pre-post probabilities or arithmetic difference between them. Heard et al. 32 substudy 1 (high credibility) demonstrated that “years of life lost” or “hours per day lost” resulted in stronger behavioral intentions than “effective age” but was no different from the percentage format. Yet 1 high-credibility finding, Gyrd-Hansen et al., 38 did not find any differences in in willingness to accept treatment between prolongation of life and pairs of rates per 10n, relative risk reduction, or NNT formats.
ROUND NUMBERS: Several moderate-credibility findings (Wadhwa and Zhang 12 substudies 1, 3 [behavior outcome], and 4) found small and borderline statistically significant increases in behavioral intention/behavior when relative risk reduction was presented in round numbers instead of nonround numbers. However, all numbers were presented to a precision of the nearest 100th (e.g., 33.00 was considered a “round” number and 33.33 a “nonround” number).
OTHER FORMATS: Strong conclusions about the exact ranking of other probability or probability comparison formats (e.g., posttreatment probabilities, pairs of pre-post probabilities, arithmetic difference between them) do not appear possible from the existing evidence.
A high-credibility finding 42 demonstrated stronger behavioral intentions associated with 1 in X than with pairs of percentages. In 2 moderate-credibility findings, Ruiz et al. 43 demonstrated no difference in behavioral intention or self-reported behavior change whether risk-reduction information was shown as pairs of rate per 10n or pairs of percentages. In a moderate-credibility finding, 44 willingness to pay for an improved drug was higher with pairs of rate per 10n than for pairs of decimals but only for 1 of 2 probability levels tested.
Lower-credibility findings (Wadhwa and Zhang 12 substudy 3 [behavioral intentions outcome] and Hembroff et al. 45 ) are not synthesized due to the small sample size and lack of statistical significance testing, respectively.
Comparisons between graphical formats for probability differences in effect on health behavior and behavioral intention (subsection 7B)
Evidence-Based Guidance for Effectiveness of Graphical Formats for Probability Differences on Health Behavior and Behavioral Intention
NUMERATOR-ONLY VERSUS PART-TO-WHOLE GRAPHICS: One high-credibility finding (Stone et al. 14 substudy 2) and 2 moderate-credibility findings (Okan et al. 13 substudy 1 and 2) demonstrated that behavioral intention around a risk-reducing action was stronger when benefit was shown as a numerator-only icon array rather than a part-to-whole icon array. One high-credibility finding 15 demonstrated a similar part-to-whole graphic effect but using bar charts with truncated y-axes and full y-axes.
DIFFERENT GRAPHICAL FORMATS: Two high-credibility findings46,47 contrasted icon arrays with other graphical formats and demonstrated no effect on behavioral intentions. A moderate-credibility finding 48 similarly compared icon arrays and bar charts, finding small differences in behavioral intention (likely not significant, but no specific testing was reported). Adarkwah et al. 49 similarly demonstrated no effect of different graphical formats on actual adherence to lifestyle change or medication at 3 mo. In a moderate-credibility finding, Stone et al. 50 substudy 2 showed no difference in willingness to pay for “improved” products by whether the improvement was depicted in foreground-only icon arrays or a pair of bars in a bar chart. Another moderate credibility finding 51 demonstrated no difference between risk reductions presented in consistent denominator part-whole icon arrays versus an NNT format with separate different denominator arrays for those treated or not treated. However, 1 high-credibility finding 52 demonstrated that showing the benefit of a drug as an integrated icon array showing the incremental risk difference led to lower intention to take a drug than a pair of bar charts did. Two of these findings47,49 also demonstrated no difference between number lines, bar charts, icon arrays, or percentages.
NUMERICAL LABELS: A moderate-credibility finding 17 found no change to behavioral intentions when number were labeled with rate per 10n in addition to category labels. Similarly, Okan et al. 15 (moderate credibility) also did not find any differences in behavioral intention using bar charts with and without number labels in rate per 10n format.
RANDOM VERSUS GROUPED ICONS IN ICON ARRAY: A moderate-credibility finding 53 showed no difference in intention to quit smoking by whether icons colored to represent the event were randomly arranged throughout the array or grouped in a block.
ABSOLUTE DIFFERENCES VERSUS RISK RATIOS: A moderate-credibility finding 18 demonstrated no consistent difference in support for policies to reduce population-level risk disparities when grouped bar charts showed absolute differences versus risk ratios.
POSTINTERVENTION PROBABILITY DATA: One study used a visual indicator to portray the probability of disease without treatment on a vertical number line as well as a second indicator showing the probability of disease with treatment. A moderate-credibility finding found that adding the second indicator did not affect behavioral intentions. 17
Comparisons between numerical and graphic formats for probability differences in effect on health behavior and behavioral intention (subsection 7C)
Evidence-Based Guidance for Comparisons of Numerical and Graphical Formats for Probability Differences, and Combinations of Numerical and Graphical Formats, on Health Behavior and Behavioral Intention
NUMERATOR-ONLY ICON ARRAYS VERSUS NUMBERS ALONE: One high- (Stone et al. 14 substudy 2) and 5 moderate-credibility findings (Stone et al. 14 substudy 1, Stone et al. 50 substudies 1 and 2, and 2 findings from Schirillo and Stone 54 substudies 1 and 2) demonstrated that numerator-only icon arrays lead to higher willingness to pay than pairs of rate per 10n.
PART-TO-WHOLE ICON ARRAYS VERSUS NUMBERS ALONE: Two high-credibility findings and 1 moderate-credibility finding suggest that icon array pairs alone or in combination with numbers appear to have stronger effects on behavioral intention than numbers alone (Cox et al., 55 Vogt and Marteau 22 substudy 1, Timmermans et al. 56 in certain scenarios). Similarly, in a high-credibility finding, Chen et al. 42 demonstrated that an icon array pair led to stronger intention than a percentage but had weaker effects than 1 in X. A high-credibility finding and 2 moderate-credibility findings demonstrated no difference between a pair of icon arrays and numerical formats, but both studies had some limitations (small sample size for Ruiz et al. 43 [both behavioral intentions and behavior] and ceiling effects in a setting with high behavioral intention at baseline for Cameron et al. 57 ). Another moderate-credibility finding 48 compared percentage with icon arrays and also did not show any large effects, although there were no statistical tests of this comparison.
BAR CHARTS VERSUS NUMBERS ALONE: One high 52 - and 3 moderate-credibility48,58,59 findings have no clear evidence of effect on behavioral intention.
INCREMENTAL-RISK ICON ARRAYS VERSUS NUMBERS ALONE: Two findings compared numbers to part-to-whole icon arrays that displayed incremental risk, with no clear consensus. A high-credibility finding 52 showed that an incremental icon array led to lower intention to take a drug than a pair of bar charts or percentage. However, a moderate-credibility finding 60 used a variation of the incremental icon array (an image of a football stadium with dots representing people), finding that it did not increase intention above just presenting the numbers of people affected.
A lower-credibility finding by Goodyear-Smith et al. 39 was not synthesized due to concerns about small sample size.
Comparisons between numerical and verbal formats for probability differences in effect on health behavior and behavioral intention (subsection 7D)
Evidence-Based Guidance for Comparisons of Numerical and Verbal Formats for Probability Differences on Health Behavior and Behavioral Intention
NUMBERS VERSUS VERBAL PROBABILITIES: Schwartz et al. 61 had 1 finding of no difference to colorectal cancer screening uptake within 6 mo (high-credibility finding) and a second finding for overall colorectal cancer screening intention (moderate-credibility finding) with quantitative risk and benefit information (percentage and rate per 10n) versus ordinal verbal probabilities. However, with this second finding, there was a higher intention to pursue fecal immunochemical testing specifically.
Comparisons of elements added for context to probability differences on health behavior and behavioral intention (subsection 7E)
Evidence-Based Guidance for Effects of Context for Probability Differences on Health Behavior and Behavioral Intention
ANECDOTES: Two high-credibility findings looked at the impact of anecdotes, both showing that adding anecdotes to numerical information affected behavioral intention. Ubel et al. 62 study 1 showed that additional anecdotes describing 1 of 2 therapeutic options increased intention to choose that option. Ubel et al. 62 study 2 showed that with no anecdotes, people tended to choose the option with the highest chance of success and that adding anecdotes reduced the intention to choose the most effective option.
NORMATIVE MESSAGE: Two high-credibility findings examined the impact of providing a normative message (what people “like you” chose), both finding an effect under different circumstances. Zikmund-Fisher et al. 63 substudies 1 and 2 showed that the normative message influenced behavioral intention when it was a large numerical proportion but not when it was a small one and when the proportion was not numerical but qualitative (“most women”).
VERBAL EXPLANATION OF CAUSALITY, INTERPRETIVE LABELS, VISUAL EMPHASIS: Three moderate-credibility findings examined a variety of different types of information meant to provide context: verbal explanation of causality, 57 interpretive labels, 17 and visual emphasis including color coding and boldface font (Stone et al. 14 substudy 1). All of these showed no impact, but credibility is reduced by modest sample sizes.
Comparisons of frames (gain, loss, comparison) for probability differences on health behavior and behavioral intention (subsection 7F)
Evidence-Based Guidance for Framing of Probability Differences on Health Behavior and Behavioral Intention
Gain-framing or loss-framing the same probability information strongly affects behavioral intention and intended decisions.
LOSS FRAME VERSUS GAIN FRAME FOR BENEFITS: When given information about a medical therapy, a high-credibility finding 64 showed the therapy was more attractive when the benefit outcome was in the loss frame (chance of getting disease) than in the gain frame (chance of avoiding disease). Similarly, in a moderate-credibility finding, 58 when given the choice between 2 products, the safer product was more attractive when the improvement was presented as a pair of negatively framed numbers (chance of disease). However, a moderate-credibility finding 51 showed no effect of framing on intent to take a medication, although the small sample size lowers confidence in this negative result.
LOSS FRAME VERSUS GAIN FRAME FOR SURE AND RISKY OPTIONS: One high-credibility finding (Wilson et al. 65 study 1) looked at choices between a sure option and a risky option, finding that the risky option was more attractive in the gain (survival) frame. In a related moderate-credibility finding (Wilson et al. 65 study 3), when given the option to remove a patient in special care to the regular floor to make room for another needy patient, the status quo (presumably seen as a positive thing for the current patient) was more attractive when probabilities were loss framed (mortality).
A lower-credibility finding by Goodyear-Smith et al. 39 is not summarized here due to a small sample size and lack of statistical testing.
7J. Effect of varying the time period on health behavior and behavioral intention
Evidence-Based Guidance on Effect of Varying the Time Period for Probability Differences on Health Behavior and Behavioral Intention
One moderate-credibility finding (Vogt and Marteau 22 study 2) compared the effect of different time frames for probability difference information on behavioral intention, finding no effect, but internal inconsistencies between different outcome measures reduce ability to draw broader conclusions.
Effects of Different Formats on Trust in the Message: Section 8
We found multiple instances of researchers assessing whether format affected the perceived credibility or trustworthiness of the information, under the assumption that high-credibility messages were more likely to promote changes in beliefs or behaviors.
Comparisons between number formats for probability differences in effect on trust (subsection 8A)
Evidence-Based Guidance on the Effect of Numerical Formats for Probability Differences on Trust
HEART AGE OR LIFE EXPECTANCY VERSUS PROBABILITY: One high-credibility finding 40 showed that showing a probability difference with pre/post percentages was associated with higher trust than showing it as “heart age” plus the arithmetic difference between heart age and actual age. However, a moderate-credibility finding suggested a different effect: Galesic and Garcia-Retamero 11 showed that the “imaginability” of the outcome was higher for life expectancy (plus the difference in life expectancy (“. . . 73 years, or 60 months shorter than average person . . .”) than percentage probability (plus a difference in percentage from the average person).
Comparisons between numerical and graphical formats for probability differences in their effects on trust (subsection 8C)
One potentially relevant finding is not summarized due to problems with the graphic design of the graphic comparator (a bar chart depicting relative risks as bars with a baseline of 0, rather than in comparison to a baseline of 1). 66
Comparisons of animated or interactive formats for probability differences on trust (subsection 8I)
Evidence-Based Guidance on the Effect of Animated or Interactive Formats for Probability Differences on Trust
One moderate-credibility finding 67 showed no effect of various forms of animation in icon arrays on credibility of information.
Preferences about Formats for Portraying Probability Differences (Section 9)
Much research has examined the information recipient’s perceptions of the attractiveness of or helpfulness of a format for probability difference as a primary or secondary measure because it seems possible that a patient might be more attentive to or receptive of information presented in a preferred format.
Preferences for different number formats for probability differences (subsection 9A)
Evidence-Based Guidance for Preferences for Numerical Formats for Probability Differences
NUMBER FORMATS: There is no coherent message from the available findings comparing different numerical formats for communicating probability differences. A high-credibility finding 68 showed a preference for pairs of 1 in X over pairs of percentages for very small risk differences associated with genetic testing. A moderate-credibility finding (Goodyear-Smith et al. 24 substudy 1) showed relative risk reduction as percentage was preferred to a pair of pre/post percentages. However, 4 additional moderate- and high-credibility findings showed no differences between different numerical risk reduction formats.5,26,33,41
YEARS OF LIFE LOST VERSUS RELATIVE RISK REDUCTION: In a moderate-credibility finding, people preferred years lost of life to relative risk reduction as percentage or change in effective age (Heard et al. 32 substudy 1). However, relative risk reduction as percentage was preferred to effective age in another moderate-credibility finding in the same study (Heard et al. 32 substudy 2)
PRESENCE OF BASELINE PROBABILITY: In a moderate-credibility finding, Berry et al. 35 showed that in communication of probability differences, people preferred the presence of baseline probability to its absence.
Findings from Miron-Shatz et al. 69 were considered lower credibility because of the small sample and differences between the denominators in the stimuli. Both Goodyear-Smith et al. 39 and Hill et al. 70 had small samples and lack of statistical testing, and Selinger et al. 71 had a very small sample.
Preferences for different graphic formats for portraying probability differences (subsection 9B)
Evidence-Based Guidance for Preferences for Graphical Formats for Probability Differences
BAR CHARTS VERSUS ICON ARRAYS: Eight findings compared the portrayal of probability differences in bar charts and icon arrays,47,72–78 with 7 of these (2 high-credibility and 5 moderate) showing various degrees of preference for bar charts.
DATA LABELS: One high-credibility finding 15 showed bar charts with data labels (rate per 10n format) were preferred to those that did not. A moderate-credibility finding 17 similarly showed that number lines were rated as easier to understand when labeled with rate per 10n in addition to category labels. Similarly, another moderate-credibility finding 79 determined that icon arrays were evaluated more positively when supplemented with rate per 10n labels.
PIE CHARTS VERSUS BAR CHARTS: A high-credibility and a moderate-credibility finding73,74 compared pie charts to bar charts, in both cases finding a preference for pie charts. Comparisons to number lines, flow charts, and other graphic formats produced less consistent findings.
ICON ARRAYS VERSUS OTHER GRAPHICS: Both a high-credibility finding 80 and a moderate-credibility finding 20 showed no differences in preference between different graphical formats, including icon arrays versus time-to-event bars 80 and icon arrays versus speedometer graphics. 20
PART-TO-WHOLE VERSUS FOREGROUND-ONLY GRAPHICS: A high-credibility finding showed preference for side-by-side part-to-whole icon arrays rather than foreground-only icon arrays (Okan et al. 13 substudy 2). However, 2 similarly designed, high-credibility findings15,81 showed no difference in preference for part-to-whole horizontal bars versus foreground-only horizontal bars.
ICON ARRAY HIGHLIGHTING ONE VERSUS MULTIPLE OUTCOMES: A moderate-credibility finding showed preference for an integrated multioutcome icon array that highlighted risk differences over pairs of icon arrays. 82 The same finding showed a preference for icon arrays with background shading versus those without shading.
POSTINTERVENTION PROBABILITY DATA: An additional moderate-credibility finding found lower perceived understanding/believability when postintervention probabilities (thereby visually illustrating the absolute risk difference) were added to vertical number lines. 17
Two low-credibility findings were not included in the synthesis (Dolan et al. 83 and Feldman-Stewart et al. 84 substudy 1) due to small sample sizes. Another relevant finding from Emmons et al. 85 was considered lower credibility because of the inconsistency between 2 measures of preference.
Preferences for numerical versus graphical formats for probability differences (subsection 9C)
Evidence-Based Guidance for Preferences for Numerical versus Graphical Formats for Probability Differences
NUMBERS PLUS ICON ARRAYS VERSUS NUMBERS ALONE: A high-credibility finding 9 showed that rates per 10n plus icon arrays were preferred over number-only formats. However, a high-credibility finding from a very similar study using a different population 86 showed that a pair of rates per 10n in table format was preferred to a pair of rates per 10n plus icon arrays. Also, 2 moderate-credibility findings showed that people preferred pairs of numbers supplemented with icon arrays to numbers alone (pre-post rates per 10n for Cox et al., 60 pre-post percentages for Garcia-Retamero et al. 5 ). However, another finding showed no difference between rate per 10n numbers alone and numbers supplemented with icon arrays. 55
NUMBERS ALONE VERSUS GRAPHICS ALONE: A moderate-credibility finding 77 showed that numbers (rate per 10n) or a number-based flowchart were preferred to icon arrays, number lines, or bar charts. van Weert et al. 74 (moderate credibility) showed a preference for pie charts with radial clocklike labels over a table containing rate per 10n numbers (and other graphics including icon arrays), and another moderate-credibility finding comparing multiple formats showed no preference difference by format. 20
Lower-credibility findings from Goodyear-Smith et al., Hill et al., Sellinger et al., and Feldman-Stewart et al. were not included in the synthesis due to small sample sizes and lack of statistical testing.39,70,71,84
Preferences for added elements for context for probability differences (subsection 9E)
Preferences for Added Context for Probability Differences
SOCIAL COMPARISON: A moderate-credibility finding 17 showed adding social comparison information (e.g., “somewhat higher than average”) actually reduced ease of understanding/believability.
Preferences for gain-loss framing of probability differences (subsection 9F)
One finding from Goodyear-Smith et al. 39 was relevant but is not summarized due to the small sample size.
Preferences for animation or interactivity for probability differences (subsection 9I)
Preferences about Animation and Interactivity
Two findings, both moderate credibility and with different interventions, are available examining the effects of animation or interactivity on preferences for communication of probability differences.
ANIMATION: Okan et al. 79 showed that animated icon arrays were evaluated more positively than static ones.
INTERACTIVITY: Okan also showed that prompting interaction with the information via reflective questions lead to higher evaluations but only among less graph-literate respondents.
Preferences for long versus short time periods for probability differences (subsection 9J)
Preferences about Time Period for Probability Differences
LONG VERSUS SHORT TIME PERIODS: A moderate-credibility finding 78 studying communication of treatment effects showed a preference for lifetime risk over 10-y or 20-y risk statistics.
Discrimination of Probability Differences: Section 10
Discrimination is assessed by determining whether the format enables small differences in the probability difference to influence responses to the message.
Comparison between numerical formats for probability differences in effects on discrimination (subsection 10A)
Evidence-Based Guidance about the Effects of Numerical Formats for Probability Differences on Discrimination
PERCENTAGES VERSUS RATES PER 10n: Only 1 moderate-credibility finding (Wolfe et al. 87 substudy 1) looked at the effect of format on people’s judgments of probabilities as “approximately equal.” Women were more likely to judge a probability difference as “approximately equal” when it was portrayed as 2 percentages than as rates per 10n, suggesting that discrimination was higher with rate per 10n.
Comparison between numerical and graphical formats for probability differences in effects on discrimination (subsection 10C)
Evidence-Based Guidance about Effects of Numerical and Graphical Formats on Discrimination
BAR CHARTS PLUS NUMBERS VERSUS NUMBERS ALONE: Only 1 moderate-credibility finding (Wolfe et al. 87 study 2) examined the impact of numbers versus graphics on people’s ability to make a distinction between 2 probabilities. The finding showed that adding a bar chart to the numbers made no difference. No other graphical formats were assessed, making it difficult to draw any broad conclusions about graphics.
Comparisons of elements added for context on sensitivity to deviation (subsection 10E)
Evidence-Based Guidance about Effects of Context on Discrimination
CATEGORY LABELS: A high-credibility finding 88 showed that adding an interpretive label of “positive/abnormal” or “negative/normal” to a rates per 10n produced an interaction such that people had stronger reactions to whether genetic test results had increased or lowered their risk. This increased discrimination but in fact created a bias not found with numbers-only communications.
EXPLANATIONS AND EXAMPLES: A moderate-credibility finding (Wolfe et al. 87 study 2) showed that providing interpretive instructions that provided examples of small differences that were nevertheless “about equal” reduced sensitivity to small differences between numbers.
Summary of Evidence
Relative differences (with or without the baseline and post–risk factor risks) increase perceptions of effectiveness.
Numerator-only graphics (foreground-only design) make a difference in probability appear larger than part-to-whole graphics.
Baseline probability plus an absolute difference reduces concern compared with presenting it as a pair of absolute rates.
Adding part-to-whole graphics (icon array, bar chart) to pairs of probabilities (rate per 10n, percentage) may not affect perceptions of probability differences.
Showing a foreground-only risk graphic instead of pairs of rates per 10n increases effectiveness perceptions.
Relative differences (alone or in combination with other statistics) have a greater effect on behavioral intention than pairs of pre-post probabilities or the arithmetic difference between them.
Showing a probability difference in familiar formats (such as rates per 10n, 1 in X, or percentage) is more effective at prompting behavioral intentions than showing it as NNT.
Relative risk reductions have stronger effects on behavioral intention than “heart age” or “effective age” formats.
Showing a probability difference effect as a numerator-only icon array or bar chart, rather than as a part-to-whole graphic, is associated with stronger intention to act.
No specific form of probability difference graphic appears to consistently result in stronger or weaker behavioral intentions.
Numerator-only icon arrays to illustrate probability differences lead to stronger intentions to act than numbers alone do.
A pair of side-by-side, part-to-whole icon arrays to illustrate probability differences leads to stronger intentions to act than numbers alone do.
Bar charts to illustrate probability differences do not appear to lead to any increase or decrease in behavioral intention compared with numbers alone.
Probability differences related to screening tests as pairs of numbers versus verbal terms does not appear to affect screening behavioral intentions/behavior.
A numerical probability difference message about a therapeutic option is associated with stronger intention to act if it is supplemented with anecdotes about people who have experienced a benefit from it.
A numerical probability difference message about options is associated with stronger intention to act if it is supplemented with a normative message about what most others have chosen.
With a communication about the effect of a potential therapy, more people will choose it when the probability difference information is loss-framed (stated in terms of the effect on the chance of negative outcome) than when it is gain-framed information (stated in terms of the effect on the chance of a beneficial outcome).
In risky choice situations with multiple options, each with differing chances of success, the framing of the probability difference associated with the “sure” option (the one with either 0% or 100% likelihood) is particularly important. When an uncertain therapy is compared with a certain negative outcome, the therapy is more attractive in the gain frame. However, when an uncertain therapy is compared with a certain positive outcome, the therapy is more attractive in the loss frame.
People prefer bar charts over icon arrays for presenting probability differences.
Graphics with data labels (e.g., rate per 10n numbers) may be preferred to unlabeled graphics for communicating probability differences.
We are unable to present anything but insufficient evidence about the effects of probability difference formats on trust or discrimination outcomes.
Discussion
The current article focuses on the important task of helping patients evaluate probability differences, such as the information used to express the effect of a risk factor, therapy, or exposure. Such probability difference information is key to making informed judgments about the selection of therapies, the choice of whether to undergo disease screening, and the decision to avoid certain behaviors or exposures. This article presents the evidence on the effects of numerical and graphical formats for probability differences on 5 important outcomes: effectiveness perceptions/feelings, behaviors/behavioral intentions, trust, preference, and discrimination. (The remaining outcomes of identification/recall, contrast, categorization, and computation appear in the part 1 article.)
Research on difference tasks has focused heavily on behaviors and behavioral intentions (94 findings). Also, a large amount of work has focused on peoples’ preferences for graphical and numerical formats (53 findings) and perceptions of and feelings about effectiveness (50 findings). Very little work has been done on how different formats affect trust or discrimination. Almost all the available research has compared 2 or more numerical formats for communicating probability differences (66 findings), 2 or more graphics (52 findings), or graphics versus numbers (56 findings), rather than other sorts of features.
Unlike several of the other review articles from this systematic review project, the current article presents a relatively large amount of strong or moderately strong evidence about the effects of formats for communicating probability differences on perceptions and feelings about effectiveness and behavioral intention. The evidence confirms that relative risk numbers have a greater impact on perceived/felt effectiveness and on behavioral intentions than pairs of absolute probabilities, that numerator-only graphics have a greater impact on these outcomes than part-to-whole graphics, and that side-by-side icon arrays have a greater impact than numbers alone. Adding personal anecdotes or social norm information about the choices others have made similarly increases the impact on perceptions and behavioral intentions. Conversely, the impact on perceptions is smaller with the incremental difference (e.g., a 2-percentage-point increase from the baseline 5% chance) rather than the pre and post probabilities (e.g., an increase from 5% to 7%). Somewhat less familiar formats including the NNT and heart age or effective age have less effect on behavioral intention than relative risk reductions. Framing influences behavioral intention, specifically in that people are more likely to choose a therapy when its effect is loss framed or expressed in terms of its impact on the chance of a negative outcome than when it is gain framed or expressed in terms of its impact on the chance of a good outcome.
We do not draw conclusions here about whether one format is “better” or “worse” than another overall but rather recommend that communicators take these effects into consideration when they determine which format to select. (We also reiterate that the set of evidence presented here is about whether effectiveness is perceived as larger or smaller; the question of whether participants could accurately identify or recall a specific probability difference is covered in our companion part 1 article.)
This review, unlike some others in our series, revealed some moderate and strong evidence about preferences. People preferred bar charts to icon arrays, and graphics with labels to unlabeled graphics when evaluating probability differences. However, it is worth noting that part 1 of this review uncovered weak evidence that icon arrays or pie charts instead of bar charts may help people contrast probability differences to identify the treatment with the greatest effect. These contrasting findings provide yet another example of the fact that no probability communication format is optimal for all outcomes.
Although it seems plausible that trust in the information about a probability difference could affect the acceptance of it, there is little research on trust as an outcome. There is also little evidence about discrimination, that is, the extent to which people can distinguish between small differences in the size of probability differences. Finally, most of the research relevant to behavior measured behavioral intention rather than actual behavior, reducing the strength of the evidence relevant to health behavior.
This article is limited by the same factors that are limitations for the Making Numbers Meaningful project as a whole, including potential missing studies. Another limitation is the use of a small group of experts to evaluate finding credibility according to a holistic appraisal of a list of study and finding characteristics rather than a weighted quantitative checklist. Also, we conducted highly granular data extraction that enabled narrow comparisons of very similar studies (studies with the same task, outcome, and format comparison) rather than more global assessments. Although this approach enables more precise comparisons, it also prevents us from providing simple answers to very broad questions such as, “Are icon arrays better in general than bar charts?” Also, the studies did not collect the same sets of participant sociodemographics, use the same instruments to collect them, or use them in similar ways. That is, many studies that reported demographics did not use them in the analysis, while others used demographic variables to guide sampling, others controlled for them in the analysis, and a very few stratified analyses to obtain stratum-specific estimates. To handle this heterogeneity, we created free-text summaries of demographic characteristics rather than structured abstractions, and as a result, we were not able to confirm that samples were sufficiently diverse to generalize to all potential populations. However, we recognize that the effects of data presentation format might differ by participant sociodemographic and cultural characteristics, and we encourage future researchers to extend this work by designing studies to assess these impacts among diverse populations. Future evidence synthesis work could then begin to systematically analyze findings by education level, age, numeracy, culture, or other potentially important factors.
In addition, we note that our decision to separate research findings related to point tasks from those related to difference tasks means that we do not present evidence on whether people prefer point versus difference tasks. That means we considered out of scope a research question such as, “Do people prefer to evaluate information about their absolute risk of disease or information about their risk relative to the average person’s risk?” In the framework we presented above, assessing a single probability (such as absolute disease risk) is a very different task from assessing a relative difference between probabilities and involves different types of information. Such comparisons did not meet our inclusion criteria of the same information presented in different formats (see the “Methods” section). As a result, we excluded such comparisons from our review. An example is found in Emmons et al. 85 ; many of the comparisons in this article did meet our inclusion criteria, but we excluded the comparison of preferences about current risk (a point task) and about current risk plus postintervention risk (a difference task).
In conclusion, a number of different formats have been demonstrated to affect perceived and felt effectiveness and behavioral intentions, and there is strong evidence for certain preferences for formats among patients and lay people. Evidence is weaker or lacking for the effect of format on behavior (rather than behavioral intention), trust in the information, and ability to discriminate between small differences in effects.
Footnotes
Acknowledgements
We thank the Numeracy Expert Panel for contributions to conceptualizing the MNM project (Cynthia Baur, Sara Cjaza, Angela Fagerlin, Carolyn Petersen, Rima Rudd, Michael Wolf, and Steven Woloshin). We are grateful to Marianne Sharko, MD, MS, Andrew Z. Liu, MPH, and Lisa Grossman Liu, MD, PhD, for contributions to article screening and risk-of-bias assessment. We also thank Jordan Brutus for assisting with data management.
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided entirely by a grant from the National Library of Medicine (R01 LM012964, Ancker PI). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the reports.
Data Availability
Abstracted data are freely available in the online appendix referenced in the “Results” section of this article. Other methods and materials are available to other researchers upon request to Jessica S. Ancker.