It has been reported that many empirical papers published in prestigious journals in economics, psychology, and medicine prioritize statistical significance over practical importance. We investigate whether the same is true of articles published in the MIS Quarterly, a top-tier information systems journal.
AmrheinVGreenlandSMcShaneB (2019) Scientists rise up against statistical significance. Nature567(7748): 305–307.
2.
APA (2001) Publication Manual of the American Psychological Association. 5th Edition. Washington, DC: American Psychological Association.
3.
APA (2020) Publication Manual of the American Psychological Association. 7th Edition. Washington, DC: American Psychological Association.
4.
BeraPSofferPParsonsJ (2019) Using eye tracking to expose cognitive processes in understanding conceptual models. MIS Quarterly43(4): 1105–1126.
5.
CohenJ (1988) Statistical Power Analysis for the Behavioral Sciences. 2nd edition. Hillsdale, NJ: Lawrence Erlbaum Associates.
6.
DeanDLLowryPBHumpherysS (2011) Profiling the research productivity of tenured information systems faculty at U.S. institutions. MIS Quarterly35(1): 1–15.
7.
DiehlHSBakerABCowanDW (1938) Cold Vaccines. Journal of the American Medical Association111: 1168–1173.
8.
DimokaAHongYPavlouPA (2012) On product uncertainty in online markets: theory and evidence. MIS Quarterly36(2): 395–426.
9.
FangYQureshiIQureshiI, et al. (2014) Trust, satisfaction, and online repurchase intention: the moderating role of perceived effectiveness of e-commerce institutional mechanisms. MIS Quarterly38(2): 407–427.
10.
FarajSKudaravalliSKudaravalliS, et al. (2015) Leading collaboration in online communities. MIS Quarterly39(2): 393–412.
11.
FidlerFThomasonNCummingG, et al. (2004) Editors can lead researchers to confidence intervals, but can't make them think. Psychological Science15(2): 119–126.
12.
FisherRA (1926) The arrangement of field experiments. Journal of the Ministry of Agriculture of Great Britain33: 503–513.
13.
International Committee of Medical Journal (1988) Uniform requirements for manuscripts submitted to biomedical journals. international committee of medical journal editors. European Journal of Haematology41: 98–108.
14.
IversonGRLongcorWHMostellerF, et al. (1971) Bias and runs in dice throwing and recording: a few million throws. Psychometrika36(1): 1–19.
15.
JiangQTanCSiaCL, et al. (2019) Followership in an open-source software project and its significance in code reuse. MIS Quarterly43(4): 1303–1319.
16.
KankanhalliAYeHYeH, et al. (2015) Comparing potential and actual innovators: an empirical study of mobile data services innovation. MIS Quarterly39(3): 667–682.
17.
KimADennisARDennisAR (2019) Says who? The effects of presentation format and source rating on fake news in social media. MIS Quarterly43(3): 1025–1039.
18.
LowryPBMoodyGDMoodyGD, et al. (2013) Evaluating journal quality and the association for information systems senior scholars' journal basket via bibliometric measures: do expert journal assessments add value?. MIS Quarterly37(4): 993–1012.
19.
LukyanenkoRParsonsJParsonsJ, et al. (2019) Expecting the unexpected: effects of data collection design choices on the quality of crowdsourced user-generated content. MIS Quarterly43(2): 623–647.
20.
McCloskeyDN (1985) The Rhetoric of Economics. Madison, WI: University of Wisconsin Press.
21.
McCloskeyDN (1999) Other Things equal: cassandra’s open letter to her economist colleagues. Eastern Economic Journal25(3): 357–363.
22.
McCloskeyDN (2002) The Secret Sins of Economics. Chicago, IL: Prickly Paradigm Press.
23.
McCloskeyDNZiliakST (1996) The standard error of regressions. Journal of Economic Literature34(1): 97–114.
24.
MertensWReckerJReckerJ (2020) New guidelines for null hypothesis significance testing in hypothetico-deductive is research. Journal of the Association for Information Systems21(4): 1072–1102.
25.
MohajeriKMesgariMLeeAS (2020) When statistical significance is not enough: investigating relevance, practical significance, and statistical significance. MIS Quarterly44(2): 525–559.
26.
NishantRSrivastavaSCSrivastavaSC, et al. (2019) Using polynomial modeling to understand service quality in e-government websites. MIS Quarterly43(3): 807–826.
27.
PanYHuangPHuangP, et al. (2018) Board independence and firm performance in the it industry: the moderating role of new entry threats. MIS Quarterly42(3): 979–1000.
28.
PearsonES (1939) “Student” as statistician. Biometrika30(3/4Jan): 210–250.
29.
RaiAArikanIArikanI, et al. (2015) Fit and misfit of plural sourcing strategies and IT-enabled process integration capabilities: consequences of firm performance in the U.S. electric utility industry. MIS Quarterly39(4): 865–885.
30.
RamasubbuNBharadwajABharadwajA, et al. (2015) Software process diversity: conceptualization, measurement, and analysis of impact on project performance. MIS Quarterly39(4): 787–807.
31.
SavitzDAToloK-APooleC (1994) Statistical significance testing in the American journal of epidemiology, 1970-1990. American Journal of Epidemiology139(10): 1047–1052.
32.
SiponenMVanceA (2010) Neutralization: New insights into the problem of employee information systems security policy violations. MIS Quarterly34(3): 487–502.
33.
SivanLSmithMDTelangR (2019) Do search engines influence media piracy? Evidence from a randomized study. MIS Quarterly43(4): 1143–1154.
34.
SmithG (2020) Data mining fool's gold. Journal of Information Technology35(3): 182–194.
35.
SmithGCordesJ (2019) The 9 Pitfalls of Data Science. Oxford, UK: Oxford University Press.
36.
TianFXuSXXuSX (2015) How do enterprise resource planning systems affect firm risk? post-implementation impact. MIS Quarterly39(1): 39–60.
37.
Vacha-HaaseTNilssonJEReetzDR, et al. (2000) Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology10(3): 413–425.
38.
WangJGuptaMGuptaM, et al. (2015) Insider threats in a financial institution: analysis of attack-proneness of information systems applications. MIS Quarterly39(12): 91–112.
39.
WangJShanZGuptaM, et al. (2019) A longitudinal study of unauthorized access attempts on information systems: the role of opportunity contexts. MIS Quarterly43(2): 601–622
40.
WassersteinRLLazarNA (2016) The ASA statement on p-Values: context, process, and purpose. The American Statistician70(2): 129–133. DOI: 10.1080/00031305.2016.1154108.
41.
WuJHuangLHuangL, et al. (2019) Operationalizing regulatory focus in the digital age: evidence from an e-commerce context. MIS Quarterly43(3): 745–764.
42.
WunderlichP (2021) e-Mail Correspondence. October 22.
43.
WunderlichPVeitDJVeitDJ, et al. (2019) Adoption of Sustainable Technologies: A Mixed-Methods Study of German Households. MIS Quarterly43(2): 673–691.
44.
YueWTWangQWangQ-H, et al. (2019) See No Evil, Hear No Evil? Dissecting the Impact of Online Hacker Forums. MIS Quarterly43(1): 73–95.
45.
ZiliakSTMcCloskeyDN (2004) Size matters: the standard error of regressions in the american economic review. The Journal of Socio-Economics33(5): 527–546.
46.
ZiliakSTMcCloskeyDN (2008) The Cult of Statistical Significance. Ann Arbor, MI: The University of Michigan Press.
