Sage Journals: Discover world-class research

Abstract

Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition to p value–based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d, while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general, Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g. differences or similarities).

Keywords

Cohen’s U3 effect size scientific communication statistical communication statistical significance

Get full access to this article

View all access options for this article.

References

Allen

CPG

Mehler

DMA

(2018) Open science challenges, benefits and tips in early career and beyond. Psyarxiv Preprints. Available at: https://doi.org/10.31234/osf.io/3czyt

Bain

Hornsey

Bongiorno

Jeffries

(2012) Promoting pro-environmental action in climate change deniers. Nature Climate Change 2(8): 600–603.

Baker

Ware

Schweitzer

Risko

(2017) Making sense of research on the neuroimage bias. Public Understanding of Science 26(2): 251–258.

Bonney

Shirk

Phillips

Wiggins

Ballard

Miller-Rushing

et al . (2014) Next steps for citizen science. Science 343(6178): 1436–1437.

Bowley

SAL

(1915) An Elementary Manual of Statistics. London: P.S. King & Son.

Brown University Science Center (2014) Quick guide to science communication. Retrieved from https://www.brown.edu/academics/science-center/sites/brown.edu.academics.science-center/files/uploads/Quick_Guide_to_Science_Communication_0.pdf

Cohen

(1988) Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NY: Erlbaum.

Cooper

BEJ

Lee

Goldacre

Sanders

TAB

(2012) The quality of the evidence for dietary advice given in UK national newspapers. Public Understanding of Science 21(6): 664–673.

Cumming

(2014) The new statistics: Why and how. Psychological Science 25(1): 7–29.

10.

Faul

Erdfelder

Buchner

Lang

(2009) Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods 41(4): 1149–1160.

11.

Feingold

(1994) Gender differences in personality: A meta-analysis. Psychological Bulletin 116(3): 429–456.

12.

Gardiner

Sullivan

Grand

(2018) Who are you writing for? Differences in response to blog design between scientists and nonscientists. Science Communication 40(1): 109–123.

13.

Haller

Krauss

(2002) Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research Online 7(12). Available at: http://www.metheval.uni-jena.de/lehre/0405-ws/evaluationuebung/haller.pdf

14.

Hanel

PHP

Maio

Manstead

ASR

(2018) A new way to look at the data: Similarities between groups of people are large and important. Journal of Personality and Social Psychology. Epub ahead of print 31 December. DOI: 10.1037/pspi0000154.

15.

Hoekstra

Morey

Rouder

Wagenmakers

(2014) Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review 21(5): 1157–1164.

16.

Hoffrage

Gigerenzer

(1998) Using natural frequencies to improve diagnostic inferences. Journal of the Association of American Medical Colleges 73(5): 538–540.

17.

Varma

(2017) Extending the seductive allure of neuroscience explanations effect to popular articles about educational topics. British Journal of Educational Psychology 87: 518–534.

18.

Inman

Bradley

(1989) The overlapping coefficient as a measure of agreement between probability distributions and point estimation of the overlap of two normal densities. Communications in Statistics—Theory and Methods 18(10): 3851–3874.

19.

Jaffee

Hyde

(2000) Gender differences in moral orientation: A meta-analysis. Psychological Bulletin 126(5): 703–726.

20.

Kahan

Peters

Dawson

Slovic

(2013) Motivated numeracy and enlightened self-government (Cultural Cognition Project Working Paper No 116). Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2319992

21.

Lakens

(2013) Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology 4; 863.

22.

Lakens

(2017) Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science 8: 355–362.

23.

Lazard

Atkinson

(2015) Putting environmental infographics center stage: The role of visuals at the elaboration likelihood model’s critical point of persuasion. Science Communication 37(1): 6–33.

24.

Lichtenstein

Slovic

Fischhoff

Layman

Combs

(1978) Judged frequency of lethal events. Journal of Experimental Psychology: Human Learning and Memory 4(6): 551.

25.

McGraw

Wong

(1992) A common language effect size statistic. Psychological Bulletin 111: 361–365.

26.

Marshall

Wolfe

CDA

McKevitt

(2012) Lay perspectives on hypertension and drug adherence: Systematic review of qualitative research. BMJ 345: e3953.

27.

Miguel

Camerer

Casey

Cohen

Esterling

Gerber

et al . (2014) Promoting transparency in Social Science Research. Science 343(6166): 30–31.

28.

Montoya

Horton

Vevea

Citkowicz

Lauber

(2017) A re-examination of the mere exposure effect: The influence of repeated exposure on recognition, familiarity, and liking. Psychological Bulletin 143(5): 459–498.

29.

Morey

Chambers

Etchells

Harris

Hoekstra

Lakens

et al . (2016) The peer reviewers’ openness initiative. Royal Society Open Science. Available at: http://www.research.ed.ac.uk/portal/en/publications/the-peer-reviewers-openness-initiative(4c5dfd3a-fa82-4fb9-994b-bfadb09f711b).html

30.

Nakagawa

Cuthill

(2007) Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews 82(4): 591–605.

31.

National Institutes of Health (2017) A checklist for communicating science and health research to the public. Available at: https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/science-health-public-trust/checklist-communicating-science-health-research-public

32.

Nicoll

Laukamm-josten

Mwizarubi

Mayala

Mkuye

Nyembela

et al . (1993) Lay health beliefs concerning HIV and AIDS—a barrier for control programmes. AIDS Care 5(2): 231–241.

33.

Norman

Avolio

Luthans

(2010) The impact of positivity and transparency on trust in leaders and their perceived effectiveness. The Leadership Quarterly 21(3): 350–364.

34.

Nurse

(2006) US biomedical research under siege. Cell 124(1): 9–12.

35.

Plavén-Sigray

Matheson

Schiffler

Thompson

(2017) Research: The readability of scientific texts is decreasing over time. Elife 6: e27725.

36.

Posavac

Sinacore

(1984) Improving the understanding of statistical significance: Reporting Effect Size. Knowledge 5(4): 503–508.

37.

Research Councils UK (n.d.) Public engagement with research strategy. Available at: http://www.rcuk.ac.uk/documents/publications/rcukperstrategy-pdf/

38.

Richard

Bond

Jr Stokes-Zoota

(2003) One hundred years of social psychology quantitatively described. Review of General Psychology 7(4): 331–363.

39.

Ruscio

(2008) A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods 13(1): 19–30.

40.

Samuelson

Zeckhauser

(1988) Status quo bias in decision making. Journal of Risk and Uncertainty 1(1): 7–59.

41.

Schindler

Thomas

(1993) The structure of interpersonal trust in the workplace. Psychological Reports 73(2): 563–573.

42.

Science Media Center (2012) 10 best practice guidelines for reporting science & health stories. Available at: http://www.sciencemediacentre.org/wp-content/uploads/2012/09/10-best-practice-guidelines-for-science-and-health-reporting.pdf

43.

Sedgwick

(2014) Clinical significance versus statistical significance. BMJ 348: g2130.

44.

Spence

Poortinga

Butler

et al . (2011) Perceptions of climate change and willingness to save energy related to flood experience. Nature Climate Change 1(1): 46–49.

45.

Sullivan

Feinn

(2012) Using effect size—or why the p value is not enough. Journal of Graduate Medical Education 4(3): 279–282.

46.

Sumner

Vivian-Griffiths

Boivin

Pidgeon

(2014) The association between exaggeration in health related science news and academic press releases: Retrospective observational study. BMJ 349: g7015.

47.

Szucs

Ioannidis

JPA

(2017) Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLOS Biology 15(3): e2000797.

48.

Thompson

(2002) What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher 31(3): 25–32.

49.

Thomson

Önkal

Avcioğlu

Goodwin

(2004) Aviation risk perception: A comparison between experts and novices. Risk Analysis 24(6): 1585–1595.

50.

Tromovitch

(2015) The lay public’s misinterpretation of the meaning of ‘significant’: A call for simple yet significant changes in scientific reporting. Journal of Research Practice 11(1): P1.

51.

Tversky

Kahneman

(1973) Availability: A heuristic for judging frequency and probability. Cognitive Psychology 5(2): 207–232.

52.

Zajonc

(1968) Attitudinal effects of mere exposure. Journal of Personality and Social Psychology 9: 1–27.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB

Beyond reporting statistical significance: Identifying informative effect sizes to improve scientific communication

Abstract

Keywords

Get full access to this article

References

Supplementary Material