Skin in the Game: Personal Accountability and Journal Peer Review

Abstract

Two preregistered studies explored the likelihood paper reviewers would request clarification from authors regarding potential questionable research practices (QRPs). Study 1 participants were instructed to imagine reviewing a journal manuscript as either a coauthor or peer reviewer and rate the extent to which they would request clarification from the author when encountering potential QRPs. Participants reported greater likelihood of requesting clarification when assigned to the coauthor relative to the peer reviewer role. Study 2 participants were assigned to either an anonymous or open-review condition and rated the extent to which they would seek clarification from an author regarding potential QRPs. Men (but not women) in the open review condition reported greater likelihood of seeking clarification about potential QRPs than men in the blind review condition. Results provide tentative evidence that motivational factors influence the peer review process, and suggestions are made for improving peer review practices.

Keywords

self peer review research accountability questionable research practices

Get full access to this article

View all access options for this article.

References

Barclay

(2010). Altruism as a courtship display: Some effects of third-party generosity on audience perceptions. British Journal of Psychology, 101, 123–135.

Bohannon

(2013). Who’s afraid of peer review? Science, 342, 60–65.

Brown

C. M.

McConnell

A. R.

(2011). Discrepancy-based and anticipated emotions in behavioral self-regulation. Emotion, 11, 1091–1095.

Bruton

S. V.

Brown

Sacco

D. F.

(2020). Ethical consistency and experience: An attempt to influence researcher attitudes toward questionable research practices through reading prompts. Journal of Empirical Research on Human Research Ethics, 15, 216–226.

Bryan

C. J.

Adams

G. S.

Monin

(2013). When cheating would make you a cheater: Implicating the self prevents unethical behavior. Journal of Experimental Psychology: General, 142, 1001–1005.

Budden

A. E.

Tregenza

Aarssen

L. W.

Koricheva

Leimu

Lortie

C. J.

(2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23, 4–6.

García

J. A.

Rodriguez-Sánchez

Fdez-Valdivia

(2015). The principal-agent problem in peer review. Journal of the Association for Information Science and Technology, 66, 297–308.

Godlee

(2002). Making reviewers visible: Openness, accountability, and credit. Journal of the American Medical Association, 287, 2762–2765.

Griskevicius

Tybur

J. M.

Sundie

J. M.

Cialdini

R. B.

Miller

G. F.

Kenrick

D. T.

(2007). Blatant benevolence and conspicuous consumption: When romantic motives elicit strategic costly signals. Journal of Personality and Social Psychology, 93, 85–102.

10.

Hadi

M. A.

(2016). Fake peer-review in research publication: Revisiting research purpose and academic integrity. International Journal of Pharmacy Practice, 24, 309–310.

11.

Haug

C. J.

(2015). Peer-review fraud—Hacking the scientific publication process. New England Journal of Medicine, 373, 2393–2395.

12.

Helmer

Schottdorf

Neef

Battaglia

(2017). Gender bias in scholarly peer review. Elife, 6, Article e21718.

13.

Higgins

E. T.

Roney

C. J. R.

Crowe

Hymes

(1994). Ideal versus ought predilections for approach and avoidance distinct self-regulatory systems. Journal of Personality and Social Psychology, 66, 276–286.

14.

Hojat

Gonnella

J. S.

Caelleigh

A. S.

(2003). Impartial judgment by the “gatekeepers” of science: Fallibility and accountability in the peer review process. Advances in Health Sciences Education, 8, 75–96.

15.

Horbach

S. P. J. M.

Halffman

(2018). The changing forms and expectations of peer review. Research Integrity and Peer Review, 3, Article 8. https://doi.org/10.1186/s41073-018-0051-5

16.

John

L. K.

Loewenstein

Prelec

(2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524–532.

17.

Kerr

N. L.

(1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196–217.

18.

Kettle

K. L.

Häubl

(2011). The signature effect: Signing influences consumption-related behavior by priming self-identity. Journal of Consumer Research, 38, 474–489.

19.

Kowalczuk

M. K.

Dudbridge

Nanda

Harriman

S. L.

Patel

Moylan

E. C.

(2015). Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models. BMJ Open, 5, Article e008707.

20.

Kriegeskorte

(2012). Open evaluation: A vision for entirely transparent post-publication peer review and rating for science. Frontiers in Computational Neuroscience, 6, 1–18.

21.

Latané

Williams

Harkins

(1979). Many hands make light the work: The causes and consequences of social loafing. Journal of Personality and Social Psychology, 37, 822–832.

22.

Mahoney

M. J.

(1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1, 161–175.

23.

Mazar

Amir

Ariely

(2008). The dishonesty of honest people: A theory of self-concept maintenance. Journal of Marketing Research, 45, 633–644.

24.

Mehta

(2019, November 5). RSC report finds publishing pipeline hinders women. Chemistry World. https://www.chemistryworld.com/news/rsc-report-finds-publishing-pipeline-hinders-women/4010608.article

25.

National Academies of Sciences, Engineering, and Medicine. (2017). Fostering integrity in research. National Academies Press.

26.

Paulhus

D. L.

(1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 598–609.

27.

Peloza

White

Shang

(2013). Good and guilt-free: The role of self-accountability in influencing preferences for products with ethical attributes. Journal of Marketing, 77, 104–119.

28.

Retraction Watch. (2015). Diederik Stapel retraction count updated to 57. https://retractionwatch.com/2015/10/20/diederik-stapel-retraction-count-updated-to-57/

29.

Rosenblatt

Kirk

S. A.

(1981). Recognition of authors in blind review of manuscripts. Journal of Social Service Research, 3, 383–394.

30.

Ross-Hellauer

Deppe

Schmidt

(2017). Survey on open peer review: Attitudes and experience amongst editors, authors, and reviewers. PLoS ONE, 12, Article e0189311.

31.

Sacco

D. F.

Brown

(2019). Assessing the efficacy of a training intervention to reduce acceptance of questionable research practices in psychology graduate students. Journal of Empirical Research on Human Research, 14, 209–2018.

32.

Sacco

D. F.

Brown

Bruton

S. V.

(2019). Grounds for ambiguity: Justifiable bases for engaging in questionable research practices. Science and Engineering Ethics, 25(5), 1321–1337.

33.

Sacco

D. F.

Bruton

S. V.

Brown

(2018). In defense of the questionable: Defining the basis of research scientists’ engagement in questionable research practices. Journal of Empirical Research on Human Research Ethics, 13, 101–110.

34.

Schroter

Tite

Hutchings

Black

(2006). Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. JAMA, 295, 314–317.

35.

Schuessler

(2016). Hoaxsters slip breastaurants and dog-park sex into journals. New York Times. https://www.nytimes.com/2018/10/04/arts/academic-journals-hoax.html

36.

Segado-Boj

Martín-Quevedo

Prieto-Gutierrez

J. J.

(2019). Attitudes toward open access, open peer review, and Altmetrics among contributors to Spanish scholarly journals. Journal of Scholarly Publishing, 50, 48–70.

37.

Shiue

Y. C.

Chiu

C. M.

Chang

C. C.

(2010). Exploring and mitigating social loafing in online communities. Computers in Human Behavior, 26, 768–777.

38.

Shu

L. L.

Mazar

Gino

Ariely

Bazerman

M. H.

(2012). Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proceedings of the National Academy of Sciences, 109, 15197–15200.

39.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.

40.

Smith

(2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99, 178–182.

41.

Sundie

J. M.

Kenrick

D. T.

Griskevicius

Tybur

J. M.

Vohs

K. D.

Beal

D. J.

(2011). Peacocks, Porsches, and Thorstein Veblen: Conspicuous consumption as a sexual signaling system. Journal of Personality and Social Psychology, 100, 664–680.

42.

Teixeira de Silva

J. A.

(2019). Challenges to open peer review. Online Information Review, 43, 197–200.

43.

Tennant

J. P.

Dugan

J. M.

Graziotin

Jacques

D. C.

Waldner

Mietchen

… Selvaraju

(2017). A multi-disciplinary perspective on emergent and future innovations in peer review. F1000 Research. https://doi.org/10.12688/f1000research.12037.3

44.

Transparent peer review one year on. (2016). Nature Communications, 7, Article 13626.

45.

Van Vugt

Iredale

(2013). Men behaving nicely: Public goods as peacock tails. British Journal of Psychology, 104, 3–13.

46.

Ware

(2008). Peer review; Benefits, perceptions and alternatives. Publishing Research Consortium, Summary Paper, 4, 1–20.

47.

Whittaker

R. J.

(2008). Journal review and gender equality: A critical comment on Budden et al. Trends in Ecology & Evolution, 23, 478–479.

48.

Williams

Harkins

S. G.

Latané

(1981). Identifiability as a deterrent to social loafing: Two cheering experiments. Journal of Personality and Social Psychology, 40, 303–311.