People often learn of new scientific findings from brief news reports, and may discount or ignore prior research, potentially contributing to misunderstanding of findings. In this preregistered study, we investigated how people interpret a brief news report on a new drug for weight loss. Participants read an article that either highlighted the importance of prior research when judging the drug’s effectiveness, or made no mention of this issue. For articles describing no prior research, mean confidence in the drug was 62%. For articles that noted prior research was conducted, confidence increased as the proportion of studies with positive findings increased. When prior research was highlighted, confidence decreased by a small amount, even when it should have increased (i.e., even when most of the evidence supported the drug’s effectiveness). Thus, people’s judgements were more sceptical, but not necessarily more accurate. Judgements were not affected by education level, statistics experience, or personal relevance of the research topic.
AdamsR. C.SumnerP.Vivian-GriffithsS.BarringtonA.WilliamsA.BoivinJ.ChambersC. D.BottL. (2017). How readers understand causal and correlational expressions used in news headlines. Journal of Experimental Psychology: Applied, 23(1), 1–14. https://doi.org/10.1037/xap0000100
2.
BarbeyA. K.SlomanS. A. (2007). Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30(3), 241–254.
3.
BottL.BrattonL.DiaconuB.AdamsR. C.ChallengerA.BoivinJ.WilliamsA.SumnerP. (2019). Caveats in science-based news stories communicate caution without lowering interest. Journal of Experimental Psychology: Applied, 25(4), 517–542. https://doi.org/10.1037/xap0000232
4.
BrommeR.GoldmanS. R. (2014). The public’s bounded understanding of science. Educational Psychologist, 49(2), 59–69.
5.
CasscellsW.SchoenbergerA.GraboysT. B. (1978). Interpretation by physicians of clinical laboratory results. New England Journal of Medicine, 299(18), 999–1001.
6.
CornerA.HahnU. (2009). Evaluating science arguments: Evidence, uncertainty, and argument strength. Journal of Experimental Psychology: Applied, 15(3), 199–212. https://doi.org/10.1037/a0016533
7.
DruckmanJ. N.McGrathM. C. (2019). The evidence for motivated reasoning in climate change preference formation. Nature Climate Change, 9(2), 111–119.
8.
EbersoleC. R.AxtJ. R.NosekB. A. (2016). Scientists’ reputations are based on getting it right, not being right. PLOS Biology, 14(5), e1002460. https://doi.org/10.1371/journal.pbio.1002460
9.
FieldA.MilesJ.FieldZ. (2012). Discovering statistics using R. Sage.
GigerenzerG.SeltenR. (2002). Bounded rationality: The adaptive toolbox. MIT Press.
12.
GormallyC.BrickmanP.LutzM. (2012). Developing a test of scientific literacy skills (TOSLS): Measuring undergraduates’ evaluation of scientific information and arguments. CBE—Life Sciences Education, 11(4), 364–377.
13.
HendriksF.KienhuesD.BrommeR. (2015). Measuring laypeople’s trust in experts in a digital age: The Muenster Epistemic Trustworthiness Inventory (METI). PLOS ONE, 10(10), Article e0139309. https://doi.org/10.1371/journal.pone.0139309
14.
HendriksF.KienhuesD.BrommeR. (2020). Replication crisis = trust crisis? The effect of successful vs failed replications on laypeople’s trust in researchers and research. Public Understanding of Science, 29(3), 270–288. https://doi.org/10.1177/0963662520902383
15.
HendriksF.Mayweg-PausE.FeltonM.IordanouK.JucksR.ZimmermannM. (2020). Constraints and affordances of online engagement with scientific information—A literature review. Frontiers in Psychology, 11, Article 572744. https://www.frontiersin.org/article/10.3389/fpsyg.2020.572744
KahnemanD.FrederickS. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In Heuristics and biases: The psychology of intuitive judgment (pp. 49–81). https://doi.org/10.1017/CBO9780511808098.004
18.
KahnemanD.TverskyA. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251.
KitchenP. J.KerrG.SchultzD. E.McCollR.PalsH. (2014). The elaboration likelihood model: Review, critique and research agenda. European Journal of Marketing, 48(11/12), 2033–2050. https://doi.org/10.1108/EJM-12-2011-0776
21.
LewandowskyS.GriffithsT. L.KalishM. L. (2009). The wisdom of individuals: Exploring people’s knowledge about everyday events using iterated learning. Cognitive Science, 33(6), 969–998. https://doi.org/10.1111/j.1551-6709.2009.01045.x
22.
LewandowskyS.van der LindenS. (2021). Countering misinformation and fake news through inoculation and prebunking. European Review of Social Psychology, 32(2), 348–384. https://doi.org/10.1080/10463283.2021.1876983
23.
LoombaS.de FigueiredoA.PiatekS. J.de GraafK.LarsonH. J. (2021). Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nature Human Behaviour, 5(3), Article 3. https://doi.org/10.1038/s41562-021-01056-1
24.
ManraiA. K.BhatiaG.StrymishJ.KohaneI. S.JainS. H. (2014). Medicine’s uncomfortable relationship with math: Calculating positive predictive value. JAMA Internal Medicine, 174(6), 991–993.
25.
McDowellM.JacobsP. (2017). Meta-analysis of the effect of natural frequencies on Bayesian reasoning. Psychological Bulletin, 143(12), 1273–1312.
26.
MeyerD.ZeileisA.HornikK. (2007). The strucplot framework: Visualizing multi-way contingency tables with vcd. Journal of Statistical Software, 17, 1–48. https://doi.org/10.18637/jss.v017.i03
27.
PennycookG.TrippasD.HandleyS. J.ThompsonV. A. (2014). Base rates: Both neglected and intuitive. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(2), 544–554.
28.
RubergS. J.HarrellF. E.Gamalo-SiebersM.LaVangeL.Jack LeeJ.PriceK.PeckC. (2019). Inference and decision making for 21st-century drug development and approval. The American Statistician, 73(Suppl. 1), 319–327. https://doi.org/10.1080/00031305.2019.1566091
29.
SchoenfeldJ. D.IoannidisJ. P. (2013). Is everything we eat associated with cancer? A systematic cookbook review. The American Journal of Clinical Nutrition, 97(1), 127–134. https://doi.org/10.3945/ajcn.112.047142
30.
SedlmeierP.GigerenzerG. (2001). Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General, 130(3), 380.
31.
SperberD.ClémentF.HeintzC.MascaroO.MercierH.OriggiG.WilsonD. (2010). Epistemic vigilance. Mind & Language, 25(4), 359–393.
32.
StovitzS. D. (2020). The inability to calculate predictive values: An old problem that has not gone away. Medical Science Educator, 30, 685–688.
33.
SumnerP.Vivian-GriffithsS.BoivinJ.WilliamsA.BottL.AdamsR.VenetisC. A.WhelanL.HughesB.ChambersC. D. (2016). Exaggerations and caveats in press releases and health-related science news. PLOS ONE, 11(12), Article e0168217.
34.
ThompsonV. A.PennycookG.TrippasD.EvansJ. S. B. (2018). Do smart people have better intuitions?Journal of Experimental Psychology: General, 147(7), 945–961.
35.
ThompsonW. B. (2019). Alpha is not the false alarm rate: An activity to dispel a common statistical misconception. Teaching of Psychology, 46(1), 72–79. https://doi.org/10.1177/0098628318816156
36.
ThompsonW. B.GarryA.TaylorJ.RadellM. L. (2020). Is one study as good as three? College graduates seem to think so, even if they took statistics classes. Psychology Learning & Teaching, 19, 143–160. https://doi.org/10.1177/1475725719877590
37.
ThompsonW. B.RadellM. L. (2021). Acceptance of anomalous research findings: Explaining treatment implausibility reduces belief in far-fetched results. PeerJ,9, e12532. https://doi.org/10.7717/peerj.12532
38.
TrabergC. S.RoozenbeekJ.van der LindenS. (2022). Psychological inoculation against misinformation: Current evidence and future directions. The ANNALS of the American Academy of Political and Social Science, 700(1), 136–151. https://doi.org/10.1177/00027162221087936
39.
WelshM. B.NavarroD. J. (2012). Seeing is believing: Priors, trust, and base rate neglect. Organizational Behavior and Human Decision Processes, 119(1), 1–14. https://doi.org/10.1016/j.obhdp.2012.04.001
40.
WestfallP. H.JohnsonW. O.UttsJ. M. (1997). A Bayesian perspective on the Bonferroni adjustment. Biometrika, 84(2), 419–427.
41.
ZeileisA.MeyerD.HornikK. (2007). Residual-based shadings for visualizing (conditional) independence. Journal of Computational and Graphical Statistics, 16(3), 507–525.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
