Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage

Abstract

Eta-squared (η²) and partial eta-squared (η_p²) are effect sizes that express the amount of variance accounted for by one or more independent variables. These indices are generally used in conjunction with ANOVA, the most commonly used statistical test in second language (L2) research (Plonsky, 2013). Consequently, it is critical that these effect sizes are applied and interpreted appropriately. The present study examined the use of these two effect sizes in L2 research. We begin by outlining the statistical and conceptual foundation of and distinction between η² and η_p². We then review the use of these indices in a sample of published L2 research (K = 156). Among other results, we show that η_p² values are frequently being mislabeled as η². We interpret and discuss potential causes and consequences related to the confusion surrounding these related but distinct indices. Within the context of reform efforts in quantitative L2 research, the current study seeks to respond to the recent, pointed calls for improving study quality (Plonsky, 2014) and statistical literacy (Loewen et al., 2014) in the field.

Keywords

effect size eta squared partial eta squared statistics quantitative L2 research

Get full access to this article

View all access options for this article.

References

Bakeman

(2005) Recommended effect size statistics for repeated measures designs. Behavior Research Methods 37: 379–84.

Brown

(2015) Why bother learning advanced quantitative methods in L2 research. In: L Plonsky (ed.) Advancing quantitative methods in second language research. New York: Routledge, pp. 9–20.

Cohen

(1965) Some statistical issues in psychological research. In B. B. Wolman (Ed.), Handbook of clinical psychology. New York: McGraw-Hill, pp. 95–121.

Cohen

(1973) Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement 33: 107–12.

Cohen

(1988) Statistical power analysis for the behavioral sciences. 2nd edition. Hillsdale, NJ: Erlbaum.

Dörnyei

(2007) Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford: Oxford University Press.

Egbert

(2007) Quality analysis of journals in TESOL and applied linguistics. TESOL Quarterly 41: 157–71.

Gass

(2009) A historical survey of SLA research. In: Bhatia

Ritchie

(eds) The new handbook of second language acquisition. Bingley: Emerald, pp. 3–27.

Grissom

Kim

(2012) Effect sizes for research: Univariate and multivariate applications. 2nd edition. New York: Routledge.

10.

Haase

(1983) Classical and partial eta square in multifactor ANOVA designs. Educational and Psychological Measurement 43: 35–39.

11.

Kelley

(2015) MBESS (Version 4.0.0 and higher) [computer software and manual]. Available from http://cran.r-project.org (accessed December 2016).

12.

Kerlinger

(1964) Foundations of behavioral research. New York: Holt, Rinehart and Winston.

13.

Lakens

(2013) Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology 4: 1–12.

14.

Larson-Hall

(2012) How to run statistical analyses. In: Mackey

Gass

(eds) Research methods in second language acquisition: A practical guide. Oxford: Blackwell, pp. 245–74.

15.

Larson-Hall

(2016) A guide to doing statistics in second language research using SPSS and R. 2nd edition. New York: Routledge.

16.

Larson-Hall

Plonsky

(2015) Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning 65: supplement 1: 127–59.

17.

Lazaraton

(2005) Quantitative research methods. In: Hinkel

(ed.) Handbook of research in second language teaching and learning. Mahwah, NJ: Erlbaum, pp. 209–24.

18.

Lazaraton

(2009) The use of statistics in SLA: A response to Loewen and Gass (2009). Language Teaching 42: 415–16.

19.

Lazaraton

Riggenbach

Ediger

(1987) Forming a discipline: Applied linguists’ literacy in research methodology and statistics. TESOL Quarterly 21: 263–77.

20.

Levine

Hullett

(2002) Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research 28: 612–25.

21.

Loewen

Gass

(2009) The use of statistics in L2 acquisition research. Language Teaching 42: 181–96.

22.

Loewen

Lavolette

Spino

. (2014) Statistical literacy among applied linguists and second language acquisition researchers. TESOL Quarterly 48: 360–88.

23.

Mizumoto

Plonsky

(2016) R as a lingua franca: Advantages of using R for quantitative research in applied linguistics. Applied Linguistics 37: 284–91.

24.

Norris

(2015) Statistical significance testing in second language research: Basic problems and suggestions for reform. Language Learning 65: 97–126.

25.

Norris

Ortega

(2000) Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language learning 50: 417–528.

26.

Olejnik

Algina

(2000) Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology 25: 241–86.

27.

Pedhazur

(1997) Multiple regression in behavioral research: Explanation and prediction. 3rd edition. Fort Worth, TX: Harcourt Brace.

28.

Perry

(2011) Research in applied linguistics: Becoming a discerning consumer. 2nd edition. New York: Routledge.

29.

Pierce

Block

Aguinis

(2004) Cautionary note on reporting eta-squared values from multifactor ANOVA designs. Educational and Psychological Measurement 64: 916–24.

30.

Phakiti

(2014) Research methods in linguistics: Experimental research methods in language learning. London: Bloomsbury Academic.

31.

Plonsky

(2013) Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition 35: 655–87.

32.

Plonsky

(2014) Study quality in quantitative L2 research (1990–2010): A methodological synthesis and call for reform. Modern Language Journal 98: 450–70.

33.

Plonsky

(2015) Statistical power, p values, descriptive statistics, and effect sizes: A ‘back-to-basics’ approach to advancing quantitative methods in L2 research. In: Plonsky

(ed.) Advancing quantitative methods in second language research. New York: Routledge, pp. 23–45.

34.

Plonsky

Oswald

(2014) How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning 64: 878–912.

35.

Plonsky

Oswald

(2015) Meta-analyzing second language research. In: Plonsky

(ed.) Advancing quantitative methods in second language research. New York: Routledge, pp. 106–28.

36.

Richardson

(2011) Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review 6: 135–47.

37.

Thompson

(2006) Foundations of behavioral statistics: An insight-based approach. New York: Guilford Press.

38.

VanPatten

Williams

(2002) Research criteria for tenure in second language acquisition: Results from a survey of the field. Unpublished manuscript, The University of Illinois at Chicago, IL, USA.