Sage Journals: Discover world-class research

Abstract

The reporting effect sizes (ESs) and confidence intervals (CIs) of ESs has become recommended practice in the social sciences; however, these values are frequently omitted by authors in manuscripts submitted for publication. Consequently, the meaningfulness and clinical relevance of their findings go unaddressed. As a result, a growing number of scholarly journals now require researchers to incorporate findings of clinical significance in their reporting of results. In this article, we review the most common conventions used for estimating and reporting ESs and CIs of ESs and illustrate how researchers can compute and interpret these measures of practical significance.

Keywords

effect size confidence intervals practical significance outcome research

Get full access to this article

View all access options for this article.

References

Akobeng

A. K.

(2008). Confidence intervals and p-values in clinical decision making. Acta Paediatrica, 97, 1004–1007. doi:10.111/j.1651-2227.2008.00836.x

American Counseling Association. (2014). ACA code of ethics. Retrieved from http://www.counseling.org/docs/ethics/2014-aca-code-of-ethics.pdf?sfvrsn=4

American Psychological Association. (2010). Publication manual of the APA (6th ed.). Washington, DC: Author.

Balkin

R. S.

Miller

Ricard

Garcia

Lancaster

(2011). Assessing factors in adolescent adjustment as precursors to recidivism in court-referred youth. Measurement and Evaluation in Counseling and Development, 44, 52–59. doi:10.1177/0748175610391611

Burchinal

M. R.

(2008). How measurement error affects the interpretation and understanding of effect sizes. Child Development Perspectives, 2, 178–180.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Cohen

(1990). Things I have learned (so far). American Psychologist, 45, 1304–1312. doi:10.1037/0003-066x.45.12.1304

Cohen

(1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.

Creswell

J. W.

(2014). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

10.

Cumming

Fidler

Kalinowski

Lai

(2012). The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes, confidence intervals, and meta-analysis. Australian Journal of Psychology, 64, 138–146. doi:10.111/j.1742-9536.2011.00037.x

11.

Ellis

P. D.

(2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge, England: Cambridge University Press.

12.

Erford

B. T.

Savin-Murphy

J. A.

Butler

(2010). Conducting a meta-analysis of counseling outcome research: Twelve steps and practical procedures. Counseling Outcome Research and Evaluation, 1, 19–43. doi:10.1177/2150137809356682

13.

Fan

Konold

T, R.

(2010). Statistical significance versus effect size. In Peterson

Baker

McGaw

(Eds.), International encyclopedia of education (Vol. 7, pp. 444–450). Oxford, England: Elsevier.

14.

Ferguson

C. J.

(2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology, 40, 532–538. doi:10.1037/a0015808

15.

Finch

Cumming

(2009). Putting research in context: Understanding confidence intervals from one or more studies. Journal of Pediatric Psychiatry, 34, 903–916. doi:10.1093/jpepsy/jsn118

16.

Flewelling

R. L.

Pashcall

M. J.

Ringwalt

C. L.

(1993). SAGE baseline survey. Research Triangle Park, NC: Research Triangle Institute.

17.

Fritz

C. O.

Morris

P. E.

Richler

(2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141, 2–18. doi:10.1037/a0024338

18.

Gravetter

F. J.

Wallnau

L. B.

(2012). Statistics for the behavioral sciences (9th ed.). Belmont, CA: Wadsworth.

19.

Hager

(2013). The statistical theories of Fisher and of Neyman and Pearson: A methodological perspective. Theoretical Psychology, 23, 251–270. doi:10.1177/0959354312465483

20.

Hammond

(1996). The objections to null hypothesis testing as a means of analyzing psychological data. Australian Journal of Psychology, 48, 104–106. doi:10.1080/00049539608259513

21.

Hedges

L. V.

(2008). What are effect sizes and why do we need them? Child Development Perspectives, 2, 167–171.

22.

Howard

G. S.

Maxwell

S. E.

Fleming

K. J.

(2000). The proof of the pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis. Psychological Methods, 5, 315–332.

23.

Kuhberger

Fritz

Lermer

Scherndl

(2015). The significance fallacy in inferential statistics. BMC Research Notes, 8, 1–9. doi:10.1186/s13104-015-1020-4

24.

Lakens

(2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 1–12. doi:10.3389/fpsyg.2013.00863

25.

Lancaster

Lenz

A. S.

Meadows

Brown

K. C.

(2013). Evaluation of a conflict resolution program for urban adolescent girls. Journal for Specialists in Group Work, 38, 225–240. doi:10.1080/01933922.2013.804897

26.

Levine

T. R.

Weber

Hullett

C. R.

Park

H. S.

Lindsey

(2008). A critical assessment of null hypothesis significance testing in quantitative communication research. Human Communication Research, 34, 171–187.

27.

Lindquist

E. F.

(1940). Statistical analysis in educational research. Boston, MA: Houghton Mifflin.

28.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

29.

Liu

X. S.

Loudermilk

Simpson

(2014). Introduction to sample size choice for confidence intervals based on t statistics. Measurement in Physical Education and Exercise Science, 18, 91–1000. doi:10.1080/1091367X.2013.864657

30.

Lockett

McWilliams

Van Fleet

D. D.

(2014). Reordering our priorities by putting phenomena before design: Escaping the straitjacket of null hypothesis significance testing. British Journal of Management, 25, 863–873. doi:10.1111/1467-8551.12063

31.

McCormack

Vandermeer

Allan

G. M.

(2013). How confidence intervals become confusion intervals. BMC Medical Research Methodology, 13, 134. Retrieved from http://dx.doi.org/10.1186/1471-2288-13-134

32.

McMillan

J. H.

Foley

(2011). Reporting and discussing effect size: Still the road less traveled? Practical Assessment, Research & Evaluation, 16. Retrieved from http://pareonline.net/getvn.asp?v=16&n=14

33.

Nickerson

R. S.

(2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301. doi:10.1037/1082989X.5.2.241

34.

Nix

T. W.

Barnette

J. J.

(1998). The data analysis dilemma: Ban or Abandon. A review of null hypothesis significance testing. Research in the School, 5, 3–14.

35.

Norris

J. M.

(2015). Statistical significance testing in second language research: Basic problems and suggestions for reform. Language Learning, 65, 97–126. doi:10.1111/lang.12114

36.

Page

(2014). Beyond statistical significance: Clinical interpretation of rehabilitation research literature. International Journal of Sports Physical Therapy, 9, 726–736.

37.

Stratford

(2010). The added value of confidence intervals. Physical Therapy, 90, 333–335. doi:10.2522/ptj.2010.90.3.333

38.

Thompson

(1996). AERA editorial policies regarding statistical significance testing: Three suggested forms. Educational Researchers, 25, 26–30. doi:10.2307/1176337

39.

Thompson

(1999). Why “encouraging” effect size reporting is not working: The etiology of researcher resistance to changing practices. Journal of Psychology, 133, 133–140. doi:10.1080/00223989909599728

40.

Thompson

(2002). “Statistical,” “Practical,” and “Clinical”: How many kinds of significance do counselors need to consider? Journal of Counseling & Development, 80, 64–71. doi:10.1002/j.1556-6678.2002.tb00167.x

41.

Thompson

(2006). Role of effect sizes in contemporary research in counseling. Counseling and Values, 50, 176–186. doi:10.1002/j.2161-007X.2006.tb00054.x

42.

Thompson

(2007). Effect sizes, confidence intervals, and confidence intervals of effect sizes. Psychology in the Schools, 44, 423–432. doi:10.1002/pits.20234

43.

Trafimow

Marks

(2015). Editorial. Basic and Applied Social Psychology, 37, 1–2. doi:10.1080/01973533.2015.1012991

44.

Trusty

Thompson

Petrocelli

J. V.

(2004). Practical guide to reporting effect size in quantitative research in the Journal of Counseling and Development, 82, 107–110.

45.

Valentine

J. C.

Aloe

A. M.

Lau

T. S.

(2015). Life after NHST: How to describe your data without “p-ing” everywhere. Basic and Applied Social Psychology, 37, 260–273. doi:10.1080/01973533.2015.1060240

46.

Wester

K. L.

Borders

L. D.

Boul

Horton

(2013). Research quality: Critique of quantitative articles in the Journal of Counseling and Development. Journal of Counseling and Development, 91, 280–290. doi:10.1002/j.1556-6676.2013.00096.x.

47.

Wilkinson

(2014). Distinguishing between statistical significance and practical/clinical meaningfulness using statistical inference. Sports Medicine, 44, 295–301. doi:10.1007/s40279-013-0125-y

Calculating and Reporting Estimates of Effect Size in Counseling Outcome Research

Abstract

Keywords

Get full access to this article

References