Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

Abstract

Cases of clear scientific misconduct have received significant media attention recently, but less flagrantly questionable research practices may be more prevalent and, ultimately, more damaging to the academic enterprise. Using an anonymous elicitation format supplemented by incentives for honest reporting, we surveyed over 2,000 psychologists about their involvement in questionable research practices. The impact of truth-telling incentives on self-admissions of questionable research practices was positive, and this impact was greater for practices that respondents judged to be less defensible. Combining three different estimation methods, we found that the percentage of respondents who have engaged in questionable practices was surprisingly high. This finding suggests that some questionable practices may constitute the prevailing research norm.

Keywords

professional standards judgment disclosure methodology

Get full access to this article

View all access options for this article.

References

Altman

L. K.

(2006, May 2). For science gatekeepers, a credibility gap. The New York Times. Retrieved from http://www.nytimes.com/2006/05/02/health/02docs.html?pagewanted=all

Bower

G. H.

Mayer

J. D.

(1985). Failure to replicate mood-dependent retrieval. Bulletin of the Psychonomic Society, 23, 39–42.

Bradburn

Sudman

(1979). Improving interview method and questionnaire design: Response effects to threatening questions in survey research. San Francisco, CA: Jossey-Bass.

Crabbe

J. C.

Wahlsten

Dudek

B. C.

(1999). Genetics of mouse behavior: Interactions with laboratory environment. Science, 284, 1670–1672.

Crocker

(2011). The road to fraud starts with a single step. Nature, 479, 151.

Deer

(2011). How the case against the MMR vaccine was fixed. British Medical Journal, 342, 77–82.

de Jong

M. G.

Pieters

Fox

J.-P

. (2010). Reducing social desirability bias through item randomized response: An application to measure underreported desires. Journal of Marketing Research, 47, 14–27.

Doyen

Klein

Pichon

C.-L.

Cleeremans

(2012). Behavioral priming: It’s all in the mind, but whose mind? PLoS ONE, 7(1), e29081. Retrieved from http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0029081

Enserink

(1999). Fickle mice highlight test problems. Science, 284, 1599–1600.

10.

Fanelli

(2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), e5738. Retrieved from http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0005738

11.

Galak

LeBoeuf

R. A.

Nelson

L. D.

Simmons

J. P.

(2012). Correcting the past: Failures to replicate psi. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2001721

12.

Gardner

Lidz

C. W.

Hartwig

K. C.

(2005). Authors’ reports about research integrity problems in clinical trials. Contemporary Clinical Trials, 26, 244–251.

13.

Geggie

(2001). A survey of newly appointed consultants’ attitudes towards research fraud. Journal of Medical Ethics, 27, 344–346.

14.

Guttman

L. L.

(1974). The basis for scalogram analysis. In Maranell

G. M.

(Ed.), Scaling: A sourcebook for behavioral scientists (pp. 142–171). New Brunswick, NJ: Transaction.

15.

Henry

D. A.

Kerridge

I. H.

Hill

S. R.

McNeill

P. M.

Doran

Newby

D. A.

. . . Day

R. O.

(2005). Medical specialists and pharmaceutical industry-sponsored research: A survey of the Australian experience. Medical Journal of Australia, 182, 557–560.

16.

Ioannidis

J. P. A.

(2005a). Contradicted and initially stronger effects in highly cited clinical research. Journal of the American Medical Association, 294, 218–228.

17.

Ioannidis

J. P. A.

(2005b). Why most published research findings are false. PLoS Medicine, 2(8), e124. Retrieved from http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

18.

Kunda

(1990). The case for motivated reasoning. Psychological Bulletin, 108, 480–498.

19.

Lacetera

Zirulia

(2011). The economics of scientific misconduct. The Journal of Law, Economics, & Organization, 27, 568–603.

20.

Lehrer

(2010, December 13). The truth wears off. The New Yorker. Retrieved from http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer

21.

Lensvelt-Mulders

G. J. L. M.

Hox

J. J.

van der Heijden

P. G. M.

Maas

C. J. M.

(2005). Meta-analysis of randomized response research: Thirty-five years of validation. Sociological Methods & Research, 33, 319–348.

22.

List

Bailey

Euzent

Martin

(2001). Academic economists behaving badly? A survey on three areas of unethical behavior. Economic Inquiry, 39, 162–170.

23.

Marshall

(2000). How prevalent is fraud? That’s a million-dollar question. Science, 290, 1662–1663.

24.

Martinson

B. C.

Anderson

M. S.

de Vries

(2005). Scientists behaving badly. Nature, 435, 737–738.

25.

Palmer

A. R.

(2000). Quasireplication and the contract of error: Lessons from sex ratios, heritabilities, and fluctuating asymmetry. Annual Review of Ecology and Systematics, 31, 441–480.

26.

Prelec

(2004). A Bayesian truth serum for subjective data. Science, 306, 462–466.

27.

Schooler

(2011). Unpublished results hide the decline effect. Nature, 470, 437.

28.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.

29.

Sovacool

(2008). Exploring scientific misconduct: Isolated individuals, impure institutions, or an inevitable idiom of modern science? Journal of Bioethical Inquiry, 5, 271–282.

30.

Steele

K. M.

Bass

K. E.

Crook

M. D.

(1999). The mystery of the Mozart effect: Failure to replicate. Psychological Science, 10, 366–369.

31.

Steneck

N. H.

(2002). Assessing the integrity of publicly supported research. In Steneck

N. H.

Scheetz

M. D.

(Eds.), Investigating Research Integrity: Proceedings of the First ORI Research Conference on Research Integrity (pp. 1–16). Washington, DC: Office of Research Integrity.

32.

Steneck

N. H.

(2006). Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics, 12, 53–74.

33.

Sterba

S. K.

(2006). Misconduct in the analysis and reporting of data: Bridging methodological and ethical agendas for change. Ethics & Behavior, 16, 305–318.

34.

Swazey

J. P.

Anderson

M. S.

Louis

K. S.

(1993). Ethical problems in academic research. American Scientist, 81, 542–553.

35.

Tourangeau

Yan

(2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859–883.

36.

Turner

C. F.

Rogers

S. M.

Lindberg

L. D.

Pleck

J. H.

Sonenstein

F. L.

(1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873.

37.

Warner

S. L.

(1965). Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60, 63–69.

38.

Wicherts

(2011). Psychology must learn a lesson from fraud case. Nature, 480, 7.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.49 MB