Sage Journals: Discover world-class research

Abstract

As the science community has recognized the vital role of communicating to the public, science communication training has proliferated. The development of rigorous, comparable approaches to assessment of training has not kept pace. We conducted a fully controlled experiment using a semester-long science communication course, and audience assessment of communicator performance. Evaluators scored the communication competence of trainees and their matched, untrained controls, before and after training. Bayesian analysis of the data showed very small gains in communication skills of trainees, and no difference from untrained controls. High variance in scores suggests little agreement on what constitutes “good” communication.

Keywords

science communication graduate training assessment evaluation evidence

Get full access to this article

View all access options for this article.

References

Appelman

Sundar

S. S.

(2016). Measuring message credibility. Journalism & Mass Communication Quarterly, 93(1), 59-79. https://doi.org/10.1177/1077699015606057

Baram-Tsabari

Lewenstein

B. V.

(2013). An instrument for assessing scientists’ written skills in public communication of science. Science Communication, 35(1), 56-85. https://doi.org/10.1177/1075547012440634

Baram-Tsabari

Lewenstein

B. V.

(2017). Science communication training: What are we trying to teach? International Journal of Science Education, Part B, 7, 285-300. https://doi.org/10.1080/21548455.2017.1303756

Baron

(2010). Escape from the ivory tower, a guide to making your science matter. Island Press.

Besley

J. C.

(2015). What do scientists think about the public and does it matter to their online engagement? Science and Public Policy, 42(2), 201-214. https://doi.org/10.1093/scipol/scu042

Besley

J. C.

Dudo

Storksdieck

(2015). Scientists’ views about communication training. Journal of Research in Science Teaching, 52 (2), 199–220. https://doi.org/10.1002/tea.21186

Bray

France

Gilbert

J. K.

(2012). Identifying the essential elements of effective science communication: What do the experts say? International Journal of Science Education, Part B, 2(1), 23-41. https://doi.org/10.1080/21548455.2011.611627

Bürkner

P.-C.

(2017). brms : An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1). https://doi.org/10.18637/jss.v080.i01

Bürkner

P.-C.

Vuorre

(2019). Ordinal regression models in psychology: A tutorial. Advances in Methods and Practices in Psychological Science, 2(1), 77-101. https://doi.org/10.1177/2515245918823199

10.

COMPASS Science Communication, Inc. (2017). The message box workbook. https://www.compassscicomm.org/

11.

Dahlstrom

M. F.

(2014). Using narratives and storytelling to communicate science with nonexpert audiences. Proceedings of the National Academy of Sciences, 111(Suppl. 4), 13614-13620. https://doi.org/10.1073/pnas.1320645111

12.

Davis

J. J.

(1995). The effects of message framing on response to environmental communications. Journalism & Mass Communication Quarterly, 72(2), 285-299. https://doi.org/10.1177/107769909507200203

13.

Dean

(2009). Am I making myself clear? A scientist’s guide to talking to the public. Harvard University Press.

14.

Druschke

C. G.

Reynolds

Morton-Aiken

Lofgren

I. E.

Karraker

N. E.

McWilliams

S. R.

(2018). Better science through rhetoric: A new model and pilot program for training graduate student science writers. Technical Communication Quarterly, 27(2), 175-190. https://doi.org/10.1080/10572252.2018.1425735

15.

Dudo

Besley

J. C.

(2016). Scientists’ prioritization of communication objectives for public engagement. PLoS ONE, 11(2), 1–18. https://doi:10.1371/journal.pone.0148867

16.

Dunning

Heath

Suls

J. M.

(2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5(3), 69-106. https://doi.org/10.1111/j.1529-1006.2004.00018.x

17.

Duran

R. L.

Zakahi

W. R.

(1987). Communication performance and communication satisfaction: What do we teach our students? Communication Education, 36(1), 13-22. https://doi.org/10.1080/03634528709378637

18.

Ericsson

K. A.

Krampe

R. T.

Tesch-Römer

(1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363-406. https://doi.org/10.1037/0033-295X.100.3.363

19.

Eva

K. W.

Cunnington

J. P. W.

Reiter

H. I.

Keane

D. R.

Norman

G. R.

(2004). How can I know what I don’t know? Poor self assessment in a well-defined domain. Advances in Health Sciences Education: Theory and Practice, 9, 211–224.

20.

Falchikov

Boud

(1989). Student self-assessment in higher education: A meta-analysis. Review of Educational Research, 59(4), 395-430. https://doi.org/10.3102/00346543059004395

21.

Fischhoff

Scheufele

D. A.

(2013). The science of science communication. Proceedings of the National Academy of Sciences of the U S A, 110(Suppl. 3), 14031-14032. https://doi.org/10.1073/pnas.1312080110

22.

Fischhoff

Scheufele

D. A.

(2014). The science of science communication II. Proceedings of the National Academy of Sciences of the U S A, 111(Suppl. 4), 13583-13584. https://doi.org/10.1073/pnas.1414635111

23.

Fiske

S. T.

Dupree

(2014). Gaining trust as well as respect in communicating to motivated audiences about science topics. Proceedings of the National Academy of Sciences of the U S A, 111(Suppl. 4), 13593-13597. https://doi.org/10.1073/pnas.1317505111

24.

Flanagin

A. J.

Metzger

M. J.

(2000). Perceptions of internet information credibility. Journalism & Mass Communication Quarterly, 77(3), 515-540. https://doi.org/10.1177/107769900007700304

25.

Flores

Tse

Boyce

(2019). ColourblindR: An R package that creates themes that make plots accessible for people with colour blindness. https://ubc-mds.github.io/ColourblindR/

26.

Gruppen

L. D.

Garcia

Grum

C. M.

Fitzgerald

J. T.

White

C. A.

Dicken

Sisson

J. C.

Zweifler

(1997). Medical students’ self-assessment accuracy in communication skills. Academic Medicine, 72(10, Suppl. 1), S57-S59. https://doi.org/10.1097/00001888-199710001-00020 (Published erratum appears in Academic Medicine, 1997, 72(12), 1126)

27.

Hansford

B. C.

Hattie

J. A.

(1982). The relationship between self and achievement/performance measures. Review of Educational Research, 52(1), 123-142. https://doi.org/10.3102/00346543052001123

28.

Kay

(2019). tidybayes: tidy data and geoms for Bayesian models. https://zenodo.org/record/3238563

29.

Kuehne

L. M.

Twardochleb

L. A.

Fritschie

K. J.

Mims

M. C.

Lawrence

D. J.

Gibson

P. P.

Stewart-Coaster

Olden

J. D.

(2014). Practical science communication strategies for graduate students. Conservation Biology, 28(5), 1225-1235. https://doi.org/10.1111/cobi.12305

30.

Liddell

T. M.

Kruschke

J. K.

(2018). Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology, 79, 328-348. https://doi.org/10.1016/j.jesp.2018.08.009

31.

Lynch

M. P.

(2017). Teaching humility in an age of arrogance. Chronicle of Higher Education, 64. https://www.chronicle.com/article/Teaching-Humility-in-an-Age-of/240266

32.

Lynch

M. P.

Johnson

C. R.

Sheff

Gunn

(2016). Intellectual humility in public discourse. IHPD Literature Review. https://humilityandconviction.uconn.edu/wp-content/uploads/sites/1877/2016/09/IHPD-Literature-Review-revised.pdf

33.

Mayer

R. C.

Davis

J. H.

Schoorman

F. D.

(1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709-734. https://doi.org/10.2307/258792

34.

Mayer

R. C.

Davis

J. H.

Schoorman

F. D.

(2007). An integrative model of organizational trust: Past, present, and future. Academy of Management Review, 32(2), 344-354. https://doi.org/10.5465/amr.2007.24348410

35.

McCroskey

J. C.

McCroskey

L. L.

(1988). Self-report as an approach to measuring communication competence. Communication Research Reports, 5(2), 108-113. https://doi.org/10.1080/08824098809359810

36.

Menninger

Gropp

(2008). Communicating Science: A primer for working with the media. American Institute of Biological Sciences.

37.

Mort

J. R.

Hansen

D. J.

(2010). First-year pharmacy students’ self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5), 78. https://doi.org/10.5688/aj740578

38.

Morton

T. A.

Rabinovich

Marshall

Bretschneider

(2011). The future that may (or may not) come: How framing changes responses to uncertainty in climate change communications. Global Environmental Change, 21(1), 103-109. https://doi.org/10.1016/j.gloenvcha.2010.09.013

39.

National Communication Association. (n.d). Learning outcomes & assessment. https://www.natcom.org/academic-professional-resources/teaching-and-learning/learning-outcomes-assessment

40.

Olson

(2009). Don’t be such a scientist: Talking substance in an age of style. Island Press.

41.

Peterman

Evia

J. R.

Cloyd

Besley

J. C.

(2017). Assessing public engagement outcomes by the use of an outcome expecations scale for scientists. Science Communication, 39(6), 782–797. https://doi:10.1177/1075547017738018

42.

Rodgers

Wang

Maras

(2018). Decoding science: Development and evaluation of a science communication training program using a triangulated framework. Science Communication, 40(1), 3-32. https://journals.sagepub.com/doi/pdf/10.1177/1075547017747285

43.

Rodgers

Wang

Schultz

J. C.

(2020). A scale to measure science communication training effectiveness. Science Communication, 42(1), 90-111. https://doi.org/10.1177/1075547020903057

44.

Rowe

Frewer

L. J.

(2005). A typology of public engagement mechanisms. Science, Technology and Human Values, 30(2), 251-290. https://doi:10.1177/0162243904271724

45.

Schemer

Kühne

Matthes

(2014). The role of measurement invariance in comparative communication research. In Canel

M. J.

Voltmer

(Eds.), Comparing political communication across time and space (pp. 31-46). https://doi.org/10.1057/9781137366474_3

46.

Smith

Baron

English

Galindo

Goldman

McLeod

, et al. (2013). COMPASS: Navigating the rules of scientific engagement. PLoS Biol, 11(4): e1001552. https://doi.org/10.1371/journal.pbio.1001552

47.

Uttl

White

C. A.

Gonzalez

D. W.

(2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42. https://doi.org/10.1016/j.stueduc.2016.08.007

48.

Wickham

(2016). ggplot2: Elegant graphics for data analysis. Springer-Verlag.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB

Assessment by Audiences Shows Little Effect of Science Communication Training

Abstract

Keywords

Get full access to this article

References

Supplementary Material