Writing a useful empirical journal article

Abstract

The impact of research depends on how well readers can understand and use published reports, yet as researchers we often fail to consider what our readers need. Instead, we structure reports to match a socially accepted, but often inappropriate, standard design – the “hourglass”. In this essay, I suggest that when writing a report, we should adjust our reporting style to suit our target readers. I outline four reporting templates for quantitative empirical studies that are suited to differing reader needs.

Keywords

reporting epistemology uncertainty model selection.

Get full access to this article

View all access options for this article.

References

Adner

Polos

Ryall

Sorenson

2009. The case for formal theory. Academy of Management Review, 34(2): 201–208. https//doi.org/10.5465/amr.2009.36982613

Alcock

2011. Response to Bem’s comments. Skeptical Inquirer. https://skepticalinquirer.org/exclusive/response-to-bems-comments/.

Andrews

Shapiro

J. M.

2021. A model of scientific communication. Econometrica, 89(5): 2117–2142. https://doi.org/10.3982/ECTA18155

Banerjee

A. V.

Chassang

Snowberg

2017. Decision theoretic approaches to experiment design and external validity. In Handbook of Economic Field Experiments (Vol. 1, pp. 141–174). Amsterdam, Netherlands: Elsevier. https//doi.org/10.1016/bs.hefe.2016.08.005

Bem

1987. Writing the empirical journal article. In Zanna

U. M.

Darley

(Eds.), The complete academic: A practical guide for the beginning social scientist (pp. 185–219). New York: Random House.

Bem

2011. Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100(3): 407. https://doi.org/ 10.1037/a0021524

Berchicci

King

A. A.

2022a. Corporate Sustainability: A Model Uncertainty Analysis of Materiality. Journal of Financial Reporting, 7(2), 43–74. https://doi.org/10.2308/JFR-2021-022

Berchicci

King

A. A.

2022b. Building knowledge by mapping model uncertainty in six studies of social and financial performance. Strategic Management Journal, 43(7): 1319–1346. https://doi.org/10.1002/smj.3374

Bloomfield

Rennekamp

Steenhoven

2018. No system is perfect: Understanding how registration-based editorial processes affect reproducibility and investment in research quality. Journal of Accounting Research, 56(2): 313–362. https://doi.org/10.1111/1475-679X.12208

10.

Carney

D. R.

Cuddy

A. J.

Yap

A. J.

2010. Power posing: Brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychological Science, 21(10): 1363–1368. https://doi.org/10.1177/0956797610383437

11.

Cohen-Cole

Durlauf

Fagan

Nagin

2008. Model uncertainty and the deterrent effect of capital punishment. American Law and Economics Review, 11(2): 335–369. https://doi.org/10.1093/aler/ahn001

12.

Colling

L. J.

Szűcs

2021. Statistical inference and the replication crisis. Review of Philosophy and Psychology, 12: 121–147. https://doi.org/ 10.1177/09567976221140828

13.

Credé, M., & Phillips, L. A. (2017). Revisiting the power pose effect: How robust are the results reported by Carney, Cuddy, and Yap (2010) to data analytic decisions? Social Psychological and Personality Science, 8(5), 493–499.

14.

Cuddy

A. J.

Schultz

S. J.

Fosse

N. E.

2018. P-curving a more comprehensive body of research on postural feedback reveals clear evidential value for power-posing effects: Reply to Simmons and Simonsohn (2017). Psychological Science, 29(4): 656–666. https://doi.org/ 10.1177/0956797617746749

15.

Durlauf

S. N.

Navarro

Rivers

D. A.

2016. Model uncertainty and the effect of shall-issue right-to-carry laws on crime. European Economic Review, 81: 32–67. https://doi.org/ 10.1016/j.euroecorev.2015.07.020

16.

Engber

2017. Daryl Bem proved ESP is real: Which means science is broken. Slate, May, 17.

17.

Fisher

1926. The arrangement of field experiments. Journal of the Ministry of Agriculture of Great Britain, 33: 503–513. https://doi.org/10.23637/rothamsted.8v61q

18.

Fricker

1994. Against gullibility. In Knowing from Words (pp. 125–161). New York, NY: Springer.

19.

Fricker

2004. Testimony: Knowing through being told. In Handbook of Epistemology (pp. 109–130). Dordrecht, Netherlands: Springer. https://doi.org/ 10.1007/978-1-4020-1986-9_3

20.

Galak, J., LeBoeuf, R. A., Nelson, L. D., & Simmons, J. P. (2012). Correcting the past: Failures to replicate psi. Journal of Personality and Social Psychology, 103(6), 933–948.

21.

Gelman

. Haig, B., Hennig, C., Owen, A. B., Cousins R., Young, S., Robert, C., Yanofsky, Corey., Wagenmakers, E. J., Kenett, R., Lakeland, D. 2019. Many perspectives on Deborah Mayo's “Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars”. Unpublished manuscript: www.stat.columbia.edu/∼gelman/research/ unpublished/mayo_reviews_2.pdf.

22.

Gelman

Loken

2013. The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University, 348: 1–17. https://doi.org/10.5465/amr.2009.36982613

23.

Gernsbacher

M. A.

2018. Writing empirical articles: Transparency, reproducibility, clarity, and memorability. Advances in Methods and Practices in Psychological Science, 1(3): 403–414. https://doi.org/ 10.1177/2515245918754485

24.

Goldfarb

King

A. A.

2016. Scientific apophenia in strategic management research: Significance tests & mistaken inference. Strategic Management Journal, 37(1): 167–176. https://doi.org/10.1002/smj.2459

25.

Goldfarb, B., & Yan, L. (2021). Revisiting Zuckerman's (1999) categorical imperative: An application of epistemic maps for replication. Strategic Management Journal, 42(11), 1963–1992.

26.

Hacking

1965. The Logic of Statistical Inference. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9781316534960.

27.

Hacking

2001. An Introduction to probability and Inductive Logic Desk Examination Edition. Cambridge UK: Cambridge Univ press. https://doi.org/ 10.5465/amr.2009.36982613.

28.

Heckman

J. J.

2008. Econometric causality. International Statistical Review, 76(1): 1–27. https://doi.org/ 10.1257/aer.p20171118

29.

Heckman

J. J.

Singer

2017. Abducting economics. American Economic Review, 107(5): 298–302. https://doi.org/10.1257/aer.p20171118

30.

Herndon

Ash

Pollin

2014. Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Cambridge Journal of Economics, 38(2): 257–279. https://doi.org/10.1093/cje/bet075

31.

Holland

P. W.

1986. Statistics and causal inference. Journal of the American Statistical Association, 8(396), 945–960. https://doi.org/10.2307/2289064

32.

Hume

1748/1993. An enquiry concerning human understanding. Indianapolis/Cambridge: Hacket Publishing Co.

33.

Hume

1740/1985. A Treatise of Human Nature. In Mossner

E. C.

(Ed.) Penguin Classics. London, England: Penguin Classics.

34.

Ichikawa

J. J.

Steup

2018. The Analysis of Knowledge. In The Stanford Encyclopedia of Philosophy (Summer Edition). Stanford, CA: The Metaphysics Research Lab at Stanford University.

35.

King

Goldfarb

Simcoe

2021. Learning from testimony on quantitative research in management. Academy of Management Review, 46(3): 465–488. https://doi.org/10.5465/amr.2018.0421

36.

Kraimer

M. L.

2023. Introducing Journal of Management Scientific Reports. Journal of Management Scientific Reports, 1(1), 3–7.

37.

Lee

S. M.

2018. “Sliced And Diced: The Inside Story Of How An Ivy League Food Scientist Turned Shoddy Data Into Viral Studies”. BuzzFeed. Feb 28.

38.

Le Guin

U. K.

2004. The Wave in the Mind: Talks and Essays on the Writer, the Reader, and the Imagination. Boston, Massachusetts: Shambhala Publications. https://doi.org/ 10.1007/978-1-137-07261-9.

39.

Lehmann

E. L.

1993. The Fisher, Neyman–Pearson theories of testing hypotheses: One theory or two? Journal of the American Statistical Association, 88(424): 1242–1249. https://doi.org/ 10.2307/2291263

40.

Madigan

Raftery

A. E.

1994. Model selection and accounting for model uncertainty in graphical models using Occam's window. Journal of the American Statistical Association, 89(428): 1535–1546. https://doi.org/ 10.2307/2291017

41.

Maron

D. J.

Hochman

J. S.

Reynolds

H. R.

Bangalore

O’Brien

S. M.

Boden

W. E.

, … Alexander

K. P.

2020. Initial invasive or conservative strategy for stable coronary disease. New England Journal of Medicine, 382(15): 1395–1407. https://doi.org/ 10.1056/NEJMoa1915922

42.

Mayo

D. G.

2018. Statistical inference as severe testing: How to get beyond the statistics wars. Cambridge, UK: Cambridge University Press.

43.

Miller

K. D.

Tsang

E. W.

2011. Testing management theories: Critical realist philosophy and research methods. Strategic Management Journal, 32(2): 139–158. https://doi.org/10.1002/smj.868

44.

Neyman

Pearson

E. S.

1933. IX. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231(694–706): 289–337.

45.

Popper

K. R.

1971. Conjectural knowledge: My solution of the problem of induction. Revue Internationale de Philosophie, 25(95/96), 167–197.

46.

Ranehill

Dreber

Johannesson

Leiberg

Sul

Weber

R. A.

2015. Assessing the robustness of power posing: No effect on hormones and risk tolerance in a large sample of men and women. Psychological Science, 26(5): 653–656. https://doi.org/ 10.1177/0956797614553946

47.

Salmon

W. C.

2017. The foundations of scientific inference. Pittsburgh, PA: University of Pittsburgh Press.

48.

Schwab

Abrahamson

Starbuck

W. H.

Fidler

2011. Perspective—researchers should make thoughtful assessments instead of null-hypothesis significance tests. Organization Science, 22(4): 1105–1120. https://doi.org/10.1287/orsc.1100.0557

49.

Simmons

J. P.

Nelson

L. D.

Simonsohn

2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11): 1359–1366. https://journals.sagepub.com/doi/10.1177/0956797611417632.

50.

Simonsohn

Simmons

J. P.

Nelson

L. D.

2020. Specification curve analysis. Nature Human Behaviour, 4(11): 1208–1214. https://doi.org/10.1037/a0033242

51.

Sine

W. D.

Lee

B. H.

2009. Tilting at windmills? The environmental movement and the emergence of the US wind energy sector. Administrative Science Quarterly, 54(1): 123–155. https://doi.org/ 10.2189/asqu.2009.54.1

52.

Spanos

2010. Is frequentist testing vulnerable to the base-rate fallacy? Philosophy of Science, 77(4): 565–583. https://doi.org/10.1086/656009

53.

Zuckerman

E. W.

1999. The categorical imperative: Securities analysts and the illegitimacy discount. American Journal of Sociology, 104(5): 1398–1438. https://doi.org/10.1086/210178