The discussion regarding questionable research practices (QRPs) in management as well as the broader natural and social sciences has increased substantially in recent years. Despite the attention, questions remain regarding research norms and the implications for both theoretical and practical advancements. The aim of the current article is to address these issues in a question-and-answer format while drawing upon both past research and the results of a series of new studies conducted using a mixed-methods design. Our goal is to encourage a systematic, collegial, and constructive dialogue regarding QRPs in management research.
AguinisH.GottfredsonR. K.JooH.2013. Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16: 270-301.
AndersonM. S.MartinsonB. C.De VriesR.2007. Normative dissonance in science: Results from a national survey of US scientists. Journal of Empirical Research on Human Research Ethics, 2: 3-14.
4.
AndersonM. S.RonningE. A.De VriesR.MartinsonB. C.2007. The perverse effects of competition on scientists’ work and relationships. Science and Engineering Ethics, 13: 437-461.
5.
AndraszewiczS.ScheibehenneB.RieskampJ.GrasmanR.VerhagenJ.WagenmakersE.-J.2014. An introduction to Bayesian hypothesis testing for management research. Journal of Management, 41: 521-543.
6.
BakkerM.van DijkA.WichertsJ. M.2012. The rules of the game called psychological science. Perspectives on Psychological Science, 7: 543-554.
7.
BakkerM.WichertsJ. M.2011. The (mis)reporting of statistical results in psychology journals. Behavior Research, 43: 666-678. doi:10.3758/s13428-011-0089-5
BanksG. C.KepesS.McDanielM. A.2012. Publication bias: A call for improved meta-analytic practice in the organizational sciences. International Journal of Selection and Assessment, 20: 182-196. doi:10.1111/j.1468-2389.2012.00591.x
10.
BanksG. C.KepesS.McDanielM. A.2015. Publication bias: Understand the myths concerning threats to the advancement of science. In LanceC. E.VandenbergR. J. (Eds.), More statistical and methodological myths and urban legends:36-64. New York: Routledge.
11.
BanksG. C.McDanielM. A.2011. The kryptonite of evidence-based I-O psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4: 40-44. doi:10.1111/j.1754-9434.2010.01292.x
12.
BanksG. C.O’BoyleE. H.Jr.2013. Why we need industrial-organizational psychology to fix industrial-organizational psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 6: 291-294.
13.
BedeianA. G.2003. The manuscript review process: The proper roles of authors, referees, and editors. Journal of Management Inquiry, 12: 331-338. doi:10.1177/1056492603258974
14.
BedeianA. G.TaylorS. G.MillerA. N.2010. Management science on the credibility bubble: Cardinal sins and various misdemeanors. Academy of Management Learning & Education, 9: 715-725. doi:10.5465/amle.2010.56659889
15.
BoscoF. A.AguinisH.FieldJ. G.PierceC. A.DaltonD. R.in press. HARKing’s threat to organizational research: Evidence from primary and meta-analytic sources. Personnel Psychology. doi:10.1111/peps.12111
16.
CortinaJ. M.2015. A revolution with a solution. Opening plenary presented at the meeting of the Society for Industrial/Organizational Psychology, Philadelphia.
17.
De VriesR.AndersonM. S.MartinsonB. C. 2006. Normal misbehavior: Scientists talk about the ethics of research. Journal of Empirical Research on Human Research Ethics, 1: 43-50.
18.
EichE.2014. Business not as usual. Psychological Science, 25: 3-6.
EmersonG. B.WarmeW. J.WolfF. M.HeckmanJ. D.BrandR. A.LeopoldS. S.2010. Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170: 1934-1939. doi:10.1001/archinternmed.2010.406
FanelliD.2013. Redefine misconduct as distorted reporting. Nature, 494: 149.
23.
FangF. C.CasadevallA.2011. Retracted science and the retraction index. Infection and Immunity, 79: 3855-3859.
24.
FangF. C.CasadevallA.2015. Competitive science: Is competition ruining science?Infection and Immunity, 83: 1229-1233.
25.
FinkelE. J.EastwickP. W.ReisH. T.2015. Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science. Journal of Personality and Social Psychology, 108: 275-297.
JohnL. K.LoewensteinG.PrelecD.2012. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23: 524-532. doi:10.1177/0956797611430953
33.
KepesS.BanksG. C.McDanielM. A.WhetzelD. L.2012. Publication bias in the organizational sciences. Organizational Research Methods, 15: 624-662. doi:10.1177/1094428112452760
34.
KepesS.McDanielM. A.2013. How trustworthy is the scientific literature in I-O psychology?Industrial and Organizational Psychology: Perspectives on Science and Practice, 6: 252-268.
35.
KerrN. L.1998. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2: 196-217. doi:10.1207/s15327957pspr0203_4
36.
LeekJ. T.PengR. D.2015. P values are just the tip of the iceberg. Nature, 520: 612.
37.
LeungK.2011. Presenting post hoc hypotheses as a priori: Ethical and theoretical issues. Management and Organizational Review, 7: 471-479.
38.
LockeE. A.2007. The case for inductive theory building. Journal of Management, 33: 867-890.
39.
MartinsonB. C.AndersonM. S.CrainA. L.De VriesR.2006. Scientists’ perceptions of organizational justice and self-reported misbehaviors. Journal of Empirical Research on Human Research Ethics, 1: 51-66.
MazzolaJ. J.DeulingJ. K.2013. Forgetting what we learned as graduate students: HARKing and selective outcome reporting in I-O journal articles. Industrial and Organizational Psychology: Perspectives on Science and Practice, 6: 279-284.
42.
NosekB. A.AlterG.BanksG. C.BorsboomD.BowmanS. D.BrecklerS.BuckS.ChambersC.ChinG.ChristensenG.ContestabileM.DafoeA.EichE.FreeseJ.GlennersterR.GoroffD.GreenD. P.HeeseB.HumphreysM.IshiyamaJ.KarlanD.KrautA.LupiaA.MarbryP.MadonT.MalhotraN.WilsonE. M.McNuttM.MiguelE.PaluckE. L.SimonsohnU.SoderbergC.SpellmanB. A.TornowJ.TurittoJ.VandenBosG. R.VazireS.WagenmakersE. J.WilsonR.YarkoniT.2015. Promoting an open research culture: Author guidelines for journals to promote transparency, openness, and reproducibility. Science, 348: 1422-1425.
NosekB. A.SpiesJ. R.MotylM.2012. Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7: 615-631. doi:10.1177/1745691612459058
45.
NuijtenM. B.HartgerinkC. H. J.Van AssenM. A. L. M.EpskampS.WichertsJ. M.in press. The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods. doi:10.3758/s13428-015-0664-2
46.
O’BoyleE. H.BanksG. C.CarterK.WalterS.YuanZ.2015. A 20-year review of outcome reporting bias in moderated multiple regression. Paper presented at the annual meeting of the Academy of Management, Vancouver.
47.
O’BoyleE. H.Jr.BanksG. C.Gonzalez-MuleE. in press. The chrysalis effect: How ugly initial results metamorphosize into beautiful articles. Journal of Management. doi:10.1177/0149206314527133
48.
Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science, 349: 6251. doi:10.1126/science.aac4716
RileyJ. W.1958. Proceedings of the thirteenth conference on public opinion research. Public Opinion Quarterly, 22: 169-216.
51.
SchmidtF. L.HunterJ. E.2015. Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage.
52.
SijtsmaK.VeldkampC. L.WichertsJ. M.in press. Improving the conduct and reporting of statistical analysis in psychology. Psychometrika. doi:10.1007/s11336-015-9444-2
53.
SimmonsJ. P.NelsonL. D.SimonsohnU.2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22: 1359-1366. doi:10.1177/0956797611417632
54.
WagenmakersE.-J.WetzelsR.BorsboomD.Van Der MaasH. L.2011. Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100: 426-432.
55.
WagenmakersE.-J.WetzelsR.BorsboomD.Van Der MaasH. L.KievitR. A.2012. An agenda for purely confirmatory research. Perspectives on Psychological Science, 7: 632-638.
56.
WichertsJ. M.BorsboomD.KatsJ.MolenaarD.2006. The poor availability of psychological research data for reanalysis. American Psychologist, 61: 726-728.
57.
WieseC. W.FullickJ. M.2014. The fantastic four years: Recommendations for industrial–organizational programs. Industrial and Organizational Psychology: Perspectives on Science and Practice, 7: 21-26.
58.
WigboldusD. H.DotschR.in press. Encourage playing with data and discourage questionable reporting practices. Psychometrika. doi:10.1007/s11336-015-9445-1
59.
ZyphurM. J.OswaldF. L.2015. Bayesian estimation and inference: A user’s guide. Journal of Management, 41: 390-420.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
