Sage Journals: Discover world-class research

Abstract

In this article, we offer a primer on “classical” frequentist statistics. In doing so, we aim to (1) provide social workers with a nuanced overview of common statistical concepts and tools, (2) clarify ways in which these ideas have oft been misused or misinterpreted in research and practice, and (3) help social workers better understand what frequentist statistics can and cannot offer. We begin broadly, starting with foundational issues in the philosophy of statistics. Then, we outline the Fisherian and Neyman–Pearson approaches to statistical inference and the practice of null hypothesis significance testing. We then discuss key statistical concepts including α, power, p values, effect sizes, and confidence intervals, exploring several common misconceptions about their use and interpretation. We close by considering some limitations of frequentist statistics and by offering an opinionated discussion on how social workers may promote more fruitful, responsible, and thoughtful statistical practice.

Keywords

social work research statistical inference classical statistics frequentist statistics Bayesian statistics

Get full access to this article

View all access options for this article.

References

Altman

D. G.

(1980). Misuse of statistics is unethical. British Medical Journal, 281, 1182–1184. https://dx.doi.org/10.1136%2Fbmj.281.6249.1182

Altman

D. G.

(1994). The scandal of poor medical research. British Medical Journal, 308, 283–284. https://doi.org/10.1136/bmj.308.6924.283

Amrhein

Greenland

McShane

(2019). Retire statistical significance. Nature, 567. https://doi.org/10.1038/d41586-019-00857-9

Amrhein

Trafimow

Greenland

(2019). Inferential statistics as descriptive statistics: There is no replication crisis if we don’t expect replication. The American Statistician, 73(S1), 262–270. https://doi.org/10.1080/00031305.2018.1543137

Aschwanden

(2015, November 24). Not even scientists can easily explain p-values. FiveThirtyEight. https://fivethirtyeight.com/features/not-even-scientists-can-easily-explain-p-values/

Bandyopadhyay

P. S.

Forster

M. R.

(2011). Philosophy of statistics: An introduction. In Bandyopadhyay

P. S.

Forster

M. R.

(Eds.), Philosophy of statistics (pp. 1–50). North Holland.

Benjamin

D. J.

Berger

J. O.

(2019). Three recommendations for improving the use of p-values. The American Statistician, 73, 186–191. https://doi.org/10.1080/00031305.2018.1543135

Benjamin

D. J.

Berger

J. O.

Johannesson

Nosek

B. A.

Wagenmakers

E-J.

Berk

R. A.

Bollen

K. A.

Brembs

Brown

Camerer

Cesarini

Chambers

C. D.

Clyde

Cook

T. D.

De Boeck

Dienes

Dreber

Easwaran

Efferson

… Johnson

V. E.

(2018). Redefine statistical significance. Nature Human Behavior, 2(1), 6–10. https://doi.org/10.1038/s41562-017-0189-z

Berger

J. O.

(2003). Could Fisher, Jeffreys, and Neyman have agreed on testing? Statistical Science, 18(1), 1–32. https://www.jstor.org/stable/3182859

10.

Berger

J. O.

Wolpert

R. L.

(1984). The likelihood principle (2nd ed.). Institute of Mathematical Statistics.

11.

Berk

Brown

Buja

George

Pitkin

Zhang

Zhao

(2014). Misspecified mean function regression: Making good use of regression models that are wrong. Sociological Methods & Research, 43(3), 422–451. https://doi.org/10.1177%2F0049124114526375

12.

Berk

R. A.

(1977). Discretionary methodological decisions in applied research. Sociological Methods & Research, 5(3), 317–334. https://doi.org/10.1177%2F004912417700500303

13.

Berk

R. A.

(2004). Regression analysis: A constructive critique. Sage.

14.

Berk

R. A.

Brown

George

Pitkin

Traskin

Zhang

Zhao

(2013). What you can learn from wrong causal models. In Morgan

(Ed.), Handbook of causal analysis for social research (pp. 403–424). Springer

15.

Berk

R. A.

Freedman

D. A.

(2003). Statistical assumptions as empirical commitments. In Blomberg

T. G.

Cohen

(Eds.), Law, punishment, and social control: Essays in honor of Sheldon Messinger (pp. 235–254). Aoldine de Gruyter.

16.

Berk

R. A.

Western

Weiss

R. E.

(1995). Statistical inference for apparent populations. Sociological Methodology, 25, 421–458. https://doi.org/10.2307/271073

17.

Borsboom

(2013, November 20). Theoretical amnesia. Center for Open Science. http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/

18.

Bushway

S. D.

Sweeten

Wilson

D. B.

(2006). Size matters: Standard errors in the application of null hypothesis significance testing in criminology and criminal justice. Journal of Experimental Criminology, 2(1), 1–22. https://doi.org/10.1007/s11292-005-5129-7

19.

Cassidy

S. A.

Dimova

Giguère

Spence

J. R.

Stanley

D. J.

(2019). Failing grade: 89% of introduction-to-psychology textbooks that define or explain statistical significance do so incorrectly. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245919858072

20.

Chambers

(2017). The 7 deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton University Press.

21.

Chen

D.-G.

Ansong

(2019). Bayesian modeling of space and time dynamics: A practical demonstration in social and health science research. Journal of the Society for Social Work and Research, 10(2), 275–299. https://doi.org/10.1086/703444

22.

Chen

D.-G.

Fraser

M. W.

(2017). A Bayesian perspective on intervention research: Using prior information in the development of social and health programs. Journal of the Society for Social Work and Research, 8(3), 441–456. https://doi.org/10.1086/693432

23.

Chen

D.-G.

Fraser

M. W.

Cuddeback

G. S.

(2018). Assurance in intervention research: A Bayesian perspective on statistical power. Journal of the Society for Social Work and Research, 9(1), 159–173. https://doi.org/10.1086/696239

24.

Chen

D.-G.

Testa

M. F.

Ansong

Brevard

K. C.

(2020). Evidence building and information accumulation: Using the Bayesian paradigm to advance child welfare intervention research. Journal of the Society for Social Work and Research, 11(3), 483–498. https://doi.org/10.1086/711376

25.

Cohen

(1962). The statistical power of abnormal-social psychological research. Journal of Abnormal and Social Psychology, 65(3), 145–153. https://doi.org/10.1037/h0045186

26.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

27.

Cohen

(1990). Things I have learned (so far). American Psychologist, 45(12), 1304–1312. https://doi.org/10.1037/0003-066X.45.12.1304

28.

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037//0033-2909.112.1.155

29.

Cohen

(1994). The Earth is round (p < .05). American Psychologist, 49(12), 997–1003. https://doi.org/10.1037/0003-066X.49.12.997

30.

Colling

L. J.

Szűcs

(2018). Statistical inference and the replication crisis. Review of Philosophy and Psychology. https://doi.org/10.1007/s13164-018-0421-4

31.

Correll

Mellinger

McClelland

G. H.

Judd

C. M.

(2020). Avoid Cohen’s ‘small’, ‘medium’, and ‘large’ for power analysis. Trends in Cognitive Science, 24(3), 200–207. https://doi.org/10.1016/j.tics.2019.12.009

32.

Cowger

C. D.

(1984). Statistical significance tests: Scientific ritualism or scientific method? Social Service Review, 58(3), 358–372. https://doi.org/10.1086/644212

33.

Cumming

Finch

(2005). Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60(2), 170–180. https://doi.org/10.1037/0003-066x.60.2.170

34.

Dienes

(2008). Understanding psychology as a science: An introduction to scientific and statistical inference. Palgrave Macmillan.

35.

Dienes

(2011). Bayesian versus orthodox statistics: Which side are you on? Perspectives on Psychological Science, 6(3), 274–290. https://doi.org/10.1177/1745691611406920

36.

Dienes

(2016). How Bayes factors chance scientific practice. Journal of Mathematical Psychology, 72, 78–89. https://doi.org/10.1016/j.jmp.2015.10.003

37.

Dienes

Mclatchie

(2018). Four reasons to prefer Bayesian analyses over significance testing. Psychonomic Bulletin & Review, 25(1), 207–218. https://doi.org/10.3758/s13423-017-1266-z

38.

Dunleavy

D. J.

(2020a). Appraising contemporary social work research: Meta-research on statistical reporting, statistical power, and evidential value [Unpublished dissertation]. Florida State University. https://doi.org/10.31237/osf.io/f2c7b

39.

Dunleavy

D. J.

(2020b). Coronavirus as impetus for a lasting change in research culture. SocArXiv. https://doi.org/10.31235/osf.io/2ryt3

40.

Dunleavy

D. J.

(2020c). Statistical significance under low power: A Gettier case? Journal of Brief Ideas. https://doi.org/10.5281/zenodo.4281535

41.

Dunleavy

D. J.

(2021). Social work and the necessity of open access. Zenodo. https://doi.org/10.5281/zenodo.4447502

42.

Dunleavy

D. J.

(2020). The cultivation of social work knowledge: Towards a more robust system of peer review. Families in Society: The Journal of Contemporary Social Services. https://doi.org/10.31235/osf.io/rvqze

43.

Dunleavy

D. J.

Lacasse

J. R.

(2020). Reproducible social work research [Course Syllabus]. https://doi.org/10.17605/OSF.IO/PX62B

44.

Dunleavy

D. J.

Tennant

Francuzik

(2020, June 2). Scientific practice in the time of COVID-19: It’s time to commit to change. Zenodo. https://doi.org/10.5281/zenodo.3873631

45.

Etz

Vandekerckhove

(2018). Introduction to Bayesian inference for psychology. Psychonomic Bulletin & Review, 25(1), 5–34. https://doi.org/10.3758/s13423-017-1262-3

46.

Ferguson

C. J.

(2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532–538. https://doi.apa.org/doi/10.1037/a0015808

47.

Fidler

Thomason

Cumming

Finch

Leeman

(2004). Editors can lead researchers to confidence intervals, but can’t make them think: Statistical reform lessons from medicine. Psychological Science, 15(2), 119–126. https://doi.org/10.1111/j.0963-7214.2004.01502008.x

48.

Field

S. A.

Tyre

A. J.

Jonzén

Rhodes

J. R.

Possingham

H. P.

(2004). Minimizing the cost of environmental management decisions by optimizing statistical thresholds. Ecology Letters, 7(8), 669–675. https://doi.org/10.1111/j.1461-0248.2004.00625.x

49.

Fisher

R. A.

(1947). Development of the theory of experimental design. Proceedings of the International Statistical Conferences, 3, 434–439. http://hdl.handle.net/2440/15254

50.

Fisher

R. A.

(1956). Statistical methods and scientific inferences. Hafner Publishing.

51.

Food and Drug Administration. (2010). Guidance for the use of Bayesian statistics in medical device clinical trials. U.S. Department of Health and Human Services. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials

52.

Freedman

(1995). Some issues in the foundation of statistics. Foundations of Science, 1, 19–39. https://doi.org/10.1007/BF00208723

53.

Freedman

Pisani

Purves

(1978). Statistics. W. W. Norton.

54.

Gambrill

(1997, April 18). Thinking about knowledge and how to get it [Paper presentation]. Ninth Symposium on Doctoral Research in Social Work, College of Social Work, The Ohio State University, Columbus, OH, United States.

55.

Gambrill

(1999). Evidence-based practice: An antidote to authority-based practice. Families in Society: The Journal of Contemporary Social Services, 80(4), 341–350. https://doi.org/10.1606%2F1044-3894.1214

56.

Gardner

M. J.

Altman

D. G.

(1986). Confidence intervals rather than p values: Estimation rather than hypothesis testing. The British Medical Journal, 292, 746–750. https://dx.doi.org/10.1136%2Fbmj.292.6522.746

57.

Garland Campbell

(2018). Errors in statistical decision making. In Glaz

Yeater

K. M.

(Eds.), Applied statistics in agricultural, biological, and environmental sciences. American Society of Agronomy. https://doi.org/10.2134/appliedstatistics.2016.0007.c1

58.

Gelman

(2014). Beyond power calculations: Assessing type S (sign) and type M (magnitude) errors. Perspectives on Psychological Science, 9(6), 641–651. https://doi.org/10.1177/1745691614551642

59.

Gelman

(2016). The problems with p-values are not just with p-values. The American Statistician, 70 (supplemental material), 1–2. https://doi.org/10.1080/00031305.2016.1154108

60.

Gelman

(2017, December 13). Yes, you can do statistical inference from nonrandom samples. Which is a good thing, considering that nonrandom samples are pretty much all we’ve got [Blog]. Statistical Modeling, Causal Inference, and Social Science. www.statmodeling.stat.columbia.edu/2017/12/13/yes-can-statistical-inference-nonrandom-samples-good-thing-considering-nonrandom-samples-pretty-much-weve-got/

61.

Gigerenzer

(1993). The superego, the ego, and the id in statistical reasoning. In Karen

Lewis

(Eds.), A handbook for data analysis in the behavioral sciences: Methodological issues. Hove & London.

62.

Gigerenzer

(2004). Mindless statistics. The Journal of Socio-Economics, 33, 587–606. https://doi.org/10.1016/j.socec.2004.09.033

63.

Gigerenzer

(2015). Surrogate science: The idol of a universal method for scientific inference. Journal of Management, 41(2), 421–440. https://doi.org/10.1177%2F0149206314547522

64.

Gigerenzer

(2018). Statistical rituals: The replication delusion and how we got there. Advances in Methods and Practices in Psychological Science, 1(2), 198–218. https://doi.org/10.1177%2F2515245918771329

65.

Gigerenzer

Krauss

Vitouch

(2004). The null ritual: What you always wanted to know about significance testing but were afraid to ask. In Kaplan

(Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 391–408). Sage.

66.

Gigerenzer

Swijink

Porter

Daston

Beatty

Kruger

(1989). The empire of chance: How probability changed science and everyday life. Cambridge University.

67.

Giles

(2011). An overly confidence (future) Nobel laureate [Blog post]. Econometrics Beat. https://davegiles.blogspot.com/2011/08/overly-confident-future-nobel-laureate.html

68.

Giner-Sorolla

Schoemann

A. M.

Montoya

A. K.

Conrique

B. G.

Carpenter

T. P.

Reifman

Lewis

N. A.

Jr Aberson

C. L.

B. W.

Soderberg

Bostyn

D. H.

(2019). Power to detect what? Considerations for planning and evaluating sample size [Unpublished manuscript]. https://osf.io/jnmya/

69.

Glass

D. J.

(2010). A critique of the hypothesis, and a defense of the question, as a framework for experimentation. Clinical Chemistry, 56(7), 1080–1085. https://doi.org/10.1373/clinchem.2010.144477

70.

Glaszoiu

Altman

D. G.

Bossuyt

Boutron

Clarke

Julious

Michie

Moher

Wager

(2014). Reducing waste from incomplete or unusable reports of biomedical research. Lancet, 383, 267–276.

71.

Glover

Dixon

(2004). Likelihood ratios: A simple and flexible statistic for empirical psychologists. Psychonomic Bulletin & Review, 11(5), 791–806. https://doi.org/10.3758/BF03196706

72.

Gomory

(2001a). A fallibilistic response to Thyer’s theory of theory-free empirical research in social work practice. Journal of Social Work Education, 37(1), 26–50. https://doi.org/10.1080/10437797.2001.10779035

73.

Gomory

(2001b). Critical rationalism (Gomory’s blurry theory) or positivism (Thyer’s theoretical myopia): Which is the prescription for social work research. Journal of Social Work Education, 37(1), 67–78. https://doi.org/10.1080/10437797.2001.10779037

74.

Goodman

(2008). A dirty dozen: Twelve p-value misconceptions. Seminars in Hematology, 45, 135–140. https://doi.org/10.1053/j.seminhematol.2008.04.003

75.

Gorard

White

(2017). Still against inferential statistics: Rejoinder to Nicholson and Ridgway. Statistics Education Research Journal, 16(1), 74–79. http://iase-web.org/documents/SERJ/SERJ16(1)_Gorard.pdf

76.

Gøtzsche

P. C.

(2020). Vaccines: Truth, lies, and controversy. People’s Press.

77.

Greenland

(2019). Valid p-values behave exactly as they should: Some misleading criticisms of p-values and their resolution with s-values. The American Statistician, 73(supplement 1), 106–114. https://doi.org/10.1080/00031305.2018.1529625

78.

Greenland

Senn

S. J.

Rothman

K. J.

Carlin

J. B.

Poole

Goodman

S. N.

Altman

D. G.

(2016). Statistical tests, p values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3

79.

Hacking

(1965[1976]). Logic of statistical inference. Cambridge University Press.

80.

Hacking

(2001). Probability and inductive logic. Cambridge University Press.

81.

Hájek

(2008). A philosopher’s guide to probability. In Bammer

Smithson

(Eds.), Uncertainty and risk: Multidisciplinary perspectives (pp. 91–104). Earthscan.

82.

Haller

Krauss

(2002). Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research Online, 7, 1–20. https://www.dgps.de/fachgruppen/methoden/mpr-online/issue16/art1/haller.pdf

83.

Hand

D. J.

(1994). Deconstructing statistical questions. Journal of the Royal Statistical Society, 157(3), 317–356. https://doi.org/10.2307/2983526

84.

Harlow

L. L.

Mulaik

S. A.

Steiger

J. H.

(Eds.). (1997). What if there were no significance tests? Lawrence Erlbaum.

85.

Henkel

R. E.

(1976). Tests of significance. Sage.

86.

Hirschauer

Grüner

Mußhoff

Becker

(2019). Twenty steps towards an adequate inferential interpretation of p-values in econometrics. Journal of Economics and Statistics, 239(4), 703–721. https://doi.org/10.1515/jbnst-2018-0069

87.

Hirschauer

Grüner

Mußhoff

Becker

Jantsch

(2020). Can p-values be meaningfully interpreted without random sampling? Statistics Surveys, 14, 71–91. https://doi.org/10.1214/20-SS129

88.

Hoekstra

Morey

R. D.

Rouder

J. N.

Wagenmakers

E.-J.

, (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 21(5), 1157–1164. https://doi.org/10.3758/s13423-013-0572-3

89.

Howard

M. H.

Garland

E. L.

(2015). Social work research: 2044. Journal of the Society for Social Work and Research, 6(2), 173–200. https://doi.org/10.1086/681099

90.

Howson

Urbach

(1993). Scientific reasoning: The Bayesian approach (2nd ed.). Open Court Publishing.

91.

Hung

H. M.

O’Neill

R. T.

Bauer

Köhne

(1997). The behavior of the p-value when the alternative hypothesis is true. Biometrics, 53(1), 11–22. https://doi.org/10.2307/2533093

92.

Ioannidis

J. P. A.

(2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

93.

Ioannidis

J. P. A.

(2012). Why science is not necessarily self-correcting. Perspective on Psychological Science, 7(6), 645–654. https://doi.org/10.1177%2F1745691612464056

94.

Ioannidis

J. P. A.

(2016). Evidence based medicine has been hijacked: A report to David Sackett. Journal of Clinical Epidemiology, 73, 82–86. https://doi.org/10.1016/j.jclinepi.2016.02.012

95.

Ioannidis

J. P. A.

(2019). Options for publishing research without any p-values. European Heart Journal, 40(31), 2555–2556. https://doi.org/10.1093/eurheartj/ehz556

96.

Kanyongo

G. Y

Brook

G. P.

Kyei-Blankson

Gocmen

(2007). Reliability and statistical power: How measurement fallibility affects power and required sample sizes for several parametric and nonparametric statistics. Journal of Modern Applied Statistical Methods, 6(1), 81–90. http://doi.org/10.22237/jmasm/1177992480

97.

Kass

R. E.

(2011). Statistical inference: The big picture. Statistical Science, 26(1), 1–9. https://doi.org/10.1214/10-sts337

98.

Kass

R. E.

Raftery

A. E.

(1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. https://www.jstor.org/stable/2291091

99.

Kirk

R. E.

(1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56(5), 746–759. https://doi.org/10.1177%2F0013164496056005002

100.

Kruschke

J. K.

(2011). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan. Academic Press.

101.

Kuhn

T. S.

(1977). The essential tension: Selected studies in scientific tradition and change. University of Chicago Press.

102.

La Caze

(2016). Frequentism. In Hájek

Hitchcock

(Eds.), The Oxford handbook of probability and philosophy. http://doi.org/10.1093/oxfordhb/9780199607617.013.17

103.

Lacasse

J. R.

(2008). Patient-level demographic and clinical variables associated with sustained antipsychotic polypharmacy: A naturalistic study of state hospital inpatients [Unpublished dissertation]. Florida State University.

104.

Lacasse

J. R.

(2011). Reanalyzing a randomized controlled trial of combination antidepressant treatment with mirtazapine: Confidence intervals suggest substantial uncertainty. Ethical Human Psychology and Psychiatry, 13(2), 149–154. http://doi.org/10.1891/1559-4343.13.2.149

105.

Lakatos

(1968). Criticism and the methodology of scientific research programmes. Proceedings of the Aristotelian Society, 69, 149–186. https://www.jstor.org/stable/4544774

106.

Lakens

(2017, May 11). How a power analysis implicitly reveals the smallest effect size you care about [Blog]. The 20% Statistician. http://daniellakens.blogspot.com/2017/05/how-power-analysis-implicitly-reveals.html

107.

Lakens

Adolfi

F. G.

Albers

C. J.

Anvari

Apps

M. A. J.

Argamon

S. E.

Baguley

Becker

R. B.

Benning

S. D.

Bradford

D. E.

Buchanan

E. M.

Caldwell

A. R.

Calster

B. V.

Carlsson

Chen

S.-C.

Chung

Colling

L. J.

Collins

G. S.

Crook

… Zwaan

R. A.

(2018). Justify your alpha: A response to “redefine statistical significance.” Nature Human Behavior, 2, 168–171. https://doi.org/10.1038/s41562-018-0311-x

108.

Lazic

S. E.

(2018). Four simple ways to increase power without increasing sample size. Laboratory Animals, 52(6), 621–629. https://doi.org/10.1177%2F0023677218767478

109.

LeCroy

C. W.

(1985). Methodological issues in the evaluation of social work practice. Social Service Review, 59(3), 345–357. https://doi.org/10.1086/644304

110.

Lewis

M. A.

(2019). Social workers count: Numbers and social issues. Oxford University Press.

111.

Lindley

D. V.

(1993). The analysis of experimental data: The appreciation of tea and wine. Teaching Statistics, 15, 22–25. https://doi.org/10.1111/j.1467-9639.1993.tb00252.x

112.

Machery

(2012). Power and negative results. Philosophy of Science, 79(5), 808–820. https://doi.org/10.1086/667877

113.

Machery

(2019). The alpha war. Review of Philosophy and Psychology, 1–25. https://doi.org/10.1007/s13164-019-00440-1

114.

Mayo

D. G.

(2003). Could fisher, Jeffreys and Neyman have agreed? Commentary on J. Berger’s fisher address. Statistical Science, 18, 19–24. https://www.jstor.org/stable/3182864

115.

Mayo

D. G.

(2018). Statistical inference as severe testing. Cambridge University Press.

116.

Mayo

D. G.

Spanos

(2010). Error statistics. In Bandyopadhyay

P. S.

Forster

M. R.

(Eds.), Philosophy of statistics (pp. 153–198). North Holland.

117.

McCloskey

D. N.

(1985). The loss function has been mislaid: The rhetoric of significance tests. The American Economic Review, 75(2), 201–205. https://www.jstor.org/stable/1805596

118.

McCloskey

D. N.

(1995). The analytical economist: The insignificance of statistical significance. Scientific American, 272(4), 32–33. https://www.scientificamerican.com/article/the-insignificance-of-statistical-s/

119.

McElreath

(2015). Statistical rethinking: A Bayesian course with examples in R and Stan. CRC Press.

120.

McShane

B. B.

(2017). Statistical significance and the dichotomization of evidence. Journal of the American Statistical Association, 112(519), 885–908. https://doi.org/10.1080/01621459.2017.1289846

121.

McShane

B. B.

Gal

Gelman

Robert

Tackett

J. L.

(2019). Abandon statistical significance. The American Statistician, 73(S1), 235–245. https://doi.org/10.1080/00031305.2018.1527253

122.

Meehl

P. E.

(1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834. https://doi.org/10.1037/0022-006X.46.4.806

123.

Morey

R. D.

(2017). When the statistical tail wags the scientific dog. Should we “redefine” statistical significance? [Blog]. Medium. https://medium.com/@richarddmorey/when-the-statistical-tailwags-the-scientific-dogd09a9f1a7c63

124.

Morey

R. D.

(2018). Redefining statistical significance: The statistical arguments [Blog]. Medium. https://medium.com/@richarddmorey/redefining-statisticalsignificance-the-statistical-arguments-ae9007bc1f91

125.

Morey

R. D.

Hoekstra

Rouder

J. N.

Lee

M. D.

Wagenmakers

E-J.

(2016). The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin & Review, 23(1), 103–123. https://doi.org/10.3758/s13423-015-0947-8

126.

Morey

R. D.

Lakens

(2016). Why most of psychology is statistically unfalsifiable. Zenodo. https://doi.org/10.5281/zenodo.838684

127.

Morey

R. D.

Romeijn

J.-W.

Rouder

J. N.

(2016). The philosophy of Bayes factors and the quantification of statistical evidence. Journal of Mathematical Psychology, 72, 6–18. https://doi.org/10.1016/j.jmp.2015.11.001

128.

Morrison

D. E.

Henkel

R. E.

(1970). The significance test controversy. Aldine.

129.

Motulsky

(2016). Essential biostatistics: A nonmathematical approach. Oxford University Press.

130.

Mudge

J. F.

Baker

L. F.

Edge

C. B.

Houlahan

J. E.

(2012). Setting an optimal α that minimizes errors in null hypothesis significance tests. PLoS One, 7(2), e32734. https://doi.org/10.1371/journal.pone.0032734

131.

Mulaik

S. A.

Raju

N. S.

Harshman

R. A.

(1997). There is a time and a place for significance testing. In Harlow

L. L.

Mulaik

S. A.

Steiger

J. H.

(Eds.), What if there were no significance tests? (pp. 65–115). Lawrence Erlbaum.

132.

Muthukrishna

Henrich

(2019). A problem in theory. Nature Human Behavior, 3(3), 221–229. https://doi.org/10.1038/s41562-018-0522-1

133.

Naimi

A. I.

Whitcomb

B. W.

(2020). Can confidence intervals be interpreted? American Journal of Epidemiology, 1–8. https://doi.org/10.1093/aje/kwaa004

134.

National Association of Social Workers. (2017). NASW code of ethics (Rev. ed.). https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-ofEthics-English

135.

Neyman

(1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 236, 333–380. https://doi.org/10.1098/rsta.1937.0005

136.

Neyman

(1957). “Inductive behavior” as a basic concept of philosophy of science. Revue de l’Institut International de Statistique/Review of the International Statistical Institute, 25, 7–22. https://doi.org/10.2307/1401671

137.

Neyman

Pearson

(1933). On the problem of the most efficient tests of statistical hypothesis. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231, 289–337. https://www.jstor.org/stable/91247

138.

Nickerson

R. S.

(2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5(2), 241–301. https://doi.org/10.1037/1082-989x.5.2.241

139.

Nugent

W. R.

(1987). Use and evaluation of theories. Social Work Research & Abstracts, 23(1), 14–19. https://doi.org/10.1093/swra/23.1.14

140.

Oakes

(1986). Statistical inference: A commentary for the social and behavioural sciences. John Wiley.

141.

Opolka

J. L.

Rascata

K. L.

Brown

C. M.

Gibson

P. J.

(2004). Ethnicity and prescription patterns for haloperidol, risperidone, and olanzapine. Psychiatric Services, 55(5), 151–156. https://doi.org/10.1176/appi.ps.55.2.151

142.

Orme

J. G.

Combs-Orme

T. D.

(1986). Statistical power and type II error in social work research. Social Work Research & Abstracts, 22(3), 3–10. https://doi.org/10.1093/swra/22.3.3

143.

Parsons

(2018, February 5). Ignoring measurement reliability is a real-life horror story [Blog]. Medium. www.medium.com/@Sam_D_Parsons/ignoring-measurement-reliability-is-a-real-life-horror-story-b98a2517db26

144.

Perezgonzalez

J. D.

(2015). Fisher, Neyman–Pearson or NHST? A tutorial for teaching data testing. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2015.00223

145.

Poole

(1987). Beyond the confidence interval. American Journal of Public Health, 77(2), 195–199. https://dx.doi.org/10.2105%2Fajph.77.2.195

146.

Popper

K. R.

(1979). Objective knowledge. Oxford University Press.

147.

Potocky-Tripodi

Tripodi

(2003). Twelve steps for quantitative analysts anonymous: On recovery from dependence on statistical significance testing, or (effect) size matters. Journal of Social Work Research and Evaluation, 4(2), 139–144.

148.

Quintana

D. S.

Eriksen

D. R.

(2017, April 8). Bayesian alternatives for common null-hypothesis significance tests in psychiatry: A non-technical guide using JASP. https://doi.org/10.1186/s12888-018-1761-4

149.

Rasnick

(2017, July 31). What a nerdy debate about p-values shows about science—And how to fix it. Vox. https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005

150.

Reichenbach

(1951). The rise of scientific philosophy. University of California Press.

151.

Ringland

Lewis

M. A.

Dunleavy

D. J.

(2020). Beyond the p-value: Bayesian statistics and causation. Journal of Evidence-Based Social Work. https://doi.org/10.1080/26408066.2020.1832011

152.

Robinson

G. K.

(2019). What properties might statistical inferences reasonably be expected to have?—Crisis and resolution in statistical inference. The American Statistician, 73(3), 243–252. https://doi.org/10.1080/00031305.2017.1415971

153.

Rosen

Proctor

E. K.

Staudt

M. M.

(1999). Social work research and the quest for effective practice. Social Work Research, 23(1), 4–14. https://doi.org/10.1093/swr/23.1.4

154.

Rosenthal

(1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638–641. https://doi.org/10.1037/0033-2909.86.3.638

155.

Rothman

K. J.

(1978). A show of confidence. The New England Journal of Medicine, 299(24), 1362–1363. https://doi.org/10.1056/nejm197812142992410

156.

Rothman

K. J.

(1998). Writing for epidemiology. Epidemiology, 9(3), 333–337. https://doi.org/10.1097/00001648-199805000-00019

157.

Royall

(1997). Statistical evidence: A likelihood paradigm. Chapman & Hall/CRC.

158.

Rubin

(2019). What type of type I error? Contrasting the Neyman–Pearson and Fisherian approaches in the context of exact and direct replications. Synthese. https://doi.org/10.1007/s11229-019-02433-0

159.

Schwab

Abrahamson

Starbuck

W. H.

Fidler

(2011). Researchers should make thoughtful assessments instead of null-hypothesis significance tests. Organization Science, 22(4), 1105–1120. https://doi.org/10.1287/orsc.1100.0557

160.

Shireman

T. I.

Olson

B. M.

Dewan

N. A.

(2002). Patterns of antidepressant use among children and adolescents. Psychiatric Services, 53(11), 1444–1450. https://doi.org/10.1176/appi.ps.53.11.1444

161.

Shlonsky

D’Andrade

Brookhart

M. A.

(2002). JSWE submission suggestions for statistical methods. Journal of Social Work Education, 38(1), 5–13. https://doi.org/10.1080/10437797.2002.10779079

162.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177%2F0956797611417632

163.

Smaldino

P. E.

McElreath

(2016). The natural selection of bad science. Royal Society Open Science, 3. https://doi.org/10.1098/rsos.160384

164.

Sober

(2008). Evidence and evolution: The logic behind the science. Cambridge University Press.

165.

Spielmans

G. I.

Thielges

S. A.

Dent

A. L.

Greenberg

R. P.

(2008). The accuracy of psychiatric medication advertisements in medical journals. Journal of Nervous and Mental Disease, 196(4), 267–273. https://doi.org/10.1097/NMD.0b013e31816a436b

166.

Stamey

Sherr

M. E.

Williams

N. J.

(2011). Bayesian analysis for evidence-based practice in social work. Social Work Research, 35(1), 46–52. https://doi.org/10.1093/swr/35.1.46

167.

Stang

Poole

Kuss

(2010). The ongoing tyranny of statistical significance testing in biomedical research. European Journal of Epidemiology, 25, 225–230. https://doi.org/10.1007/s10654-010-9440-x

168.

Szűcs

Ioannidis

J. P. A.

(2017a). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biology, 15(3), e2000797. https://doi.org/10.1371/journal.pbio.2000797

169.

Szűcs

Ioannidis

J. P. A.

(2017b). When null hypothesis significance testing is unsuitable for research: A reassessment. Frontiers in Human Neuroscience, 11, 390. https://.doi.org/10.3389%2Ffnhum.2017.00390

170.

Thyer

B. A.

(1989). First principles of practice research. British Journal of Social Work, 19, 309–323. https://doi.org/10.1093/oxfordjournals.bjsw.a055542

171.

Thyer

B. A.

(2017). Suggested reporting guidelines to improve health-related social work research. Health & Social Work, 42(4), 1–4. https://doi.org/10.1093/hsw/hlx035

172.

Thyer

B. A.

Pignotti

(2015). Science and pseudoscience in social work practice. Springer. http://doi.org/10.1891/9780826177698

173.

Trafimow

Marks

(2015). Editorial. Basic and Applied Social Psychology, 37(1), 1–2. https://doi.org/10.1080/01973533.2015.1012991

174.

Tripodi

(1974). Uses and abuses of social research in social work. Columbia University Press.

175.

van Dongen

Sprenger

Wagenmakers

E-J.

(2020). A Bayesian perspective on severity: Risky predictions and specific hypotheses. PsyArXiv. https://doi.org/10.31234/osf.io/4et65

176.

Wagenmakers

E.-J.

(2004). A Practical solution to the pervasive problem of p-values. Psychonomic Bulletin & Review, 14(5), 779–804. https://doi.org/10.3758/BF03194105

177.

Wagenmakers

E.-J.

Lee

Lodeqtckx

Iverson

G. J.

(2008). Bayesian versus frequentist inference. In Hoijtink

Klugkist

Boelen

P. A.

(Eds.), Statistics for social and behavioral sciences. Bayesian evaluation of informative hypotheses (pp. 181–207). Springer. https://doi.org/10.1007/978-0-387-09612-4_9

178.

Wagenmakers

E.-J.

Verhagen

Bakker

Lee

M. D.

Matzke

Rouder

J. N.

Morey

R. D.

(2015). A power fallacy. Behavior Research Methods, 47(4), 913–917. https://doi.org/10.3758/s13428-014-0517-4

179.

Wagenmakers

E.-J.

Wetzels

Borsboom

van der Maas

H. L. J.

(2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100(3), 426–432. https://doi.org/10.1037/a0022790

180.

Wasserstein

R. L.

Lazar

N. A.

(2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108

181.

Wasserstein

R. L.

Schirm

A. L.

Lazar

N. A.

(2019). Moving to a world beyond “p < 0.05.” The American Statistician, 73(supp. 1), 1–19. https://doi.org/10.1080/00031305.2019.1583913

182.

Wetzels

Matzke

Lee

M. D.

Rouder

J. N.

Iverson

G. J.

Wagenmakers

E-J.

(2011). Statistical evidence in experimental psychology: An empirical comparison using 855 t tests. Perspectives on Psychological Science, 6(3), 291–298. https://doi.org/10.1177/1745691611406923

183.

White

Gorard

. (2017). Against inferential statistics: How and why current statistics teaching gets it wrong. Statistics Education Research Journal, 16(1), 55–65. http://iase-web.org/documents/SERJ/SERJ16(1)_White.pdf

184.

Wolf-Branigin

Duke

(2007). Spiritual involvement as a predictor to completing a Salvation Army Substance Abuse Treatment Program. Research on Social Work Practice, 17(2), 239–245. https://doi.org/10.1177%2F1049731506294373

185.

Ziliak

S. T.

McCloskey

D. N.

(2008). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. University of Michigan.

The Use and Misuse of Classical Statistics: A Primer for Social Workers

Abstract

Keywords

Get full access to this article

References