This commentary focuses on three methodological concerns regarding Bryfonski and McKay’s (2019) meta-analysis of existing works on the effectiveness of task-based language teaching (TBLT). These are (1) loose inclusion criteria, (2) oversimplified outcome effect size calculation, and (3) neglect of influence of moderators. To rectify these issues, we conducted a new meta-analysis to build on the strengths of their work while correcting some flaws by aiming to afford more rigorous and valid insights for education practitioners, researchers, and decision makers. While Bryfonski and McKay (BM) reported a strong positive effect (d = 0.93), our results point to a more modest effect of TBLT on second language achievement (g = 0.61). Different from BM’s claimed four moderators of the main effect (i.e. program region, institution type, needs analysis, and cycles of implementation), our meta-regression results indicate that only sample size and teacher-as-researcher suffice to account for the heterogeneity of the effect sizes. Given the reduced number of acceptable primary studies, our revised results should be interpreted with caution.
AdamsR.NewtonJ. (2009). TBLT in Asia: Constraints and opportunities. Asian Journal of English Language Teaching, 19, 1–17.
2.
BoersF.BryfonskiL.FaezF.McKayT. (2021). A call for cautious interpretation of meta-analytic reviews. Studies in Second Language Acquisition, 43(1), 2–24. https://doi.org/10.1017/S0272263120000327
3.
BorensteinM.HedgesL.V.HigginsJ.P.T.RothsteinH.R. (2013). Comprehensive meta-analysis version 3. Biostat.
4.
BorensteinM.HedgesL.V.HigginsJ.P.T.RothsteinH.R. (2021). Introduction to meta-analysis. John Wiley.
5.
BryfonskiL.McKayT.H. (2019). TBLT implementation and evaluation: A meta-analysis. Language Teaching Research, 23, 603–632.
6.
ButlerY.G. (2011). The implementation of communicative and task-based language teaching in the Asia-pacific region. Annual Review of Applied Linguistics, 31, 36–57.
7.
CheungA.C.K.SlavinR.E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.
8.
CheungA.C.K.SlavinR.E. (2016). How methodological features affect effect sizes in education. Educational Researcher, 45, 283–292.
9.
CohenJ. (1977). Statistical power analysis for the behavioral sciences. Routledge Academic.
10.
CronbachL.J.AmbronS.R.DornbuschS.M., et al. (1980). Toward reform of program evaluation: Aims, methods, and institutional arrangements. Jossey-Bass.
11.
CrossK.P. (1991). Every teacher a researcher, every classroom a laboratory. Tribal College: Journal of American Indian Higher Education, 2, 7.
12.
CummingG. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge / Taylor & Francis.
13.
De KeyserR. (2010). Practice for second language learning: Don’t throw out the baby with the bathwater. International Journal of English Studies, 10, 155–165.
14.
De RidderI.VangehuchtenL.GomezM. S. (2007). Enhancing automaticity through task-based language learning. Applied Linguistics, 28, 309–315.
15.
DuvalS.TweedieR. (2000). A nonparametric ‘trim and fill’ method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89–98.
16.
EllisR. (1997). SLA research and language teaching. Oxford University Press.
17.
EllisR. (2018). Reflections on task-based language teaching. Multilingual Matters.
18.
EllisR.ShintaniN. (2013). Exploring language pedagogy through second language acquisition research. Routledge.
19.
EllisR.SkehanP.LiS.ShintaniN.LambertC. (2019). Task-based language teaching: Theory and practice. Cambridge University Press.
20.
FunderD.C.OzerD.J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2, 156–168.
21.
GelmanA.CarlinJ. (2014). Beyond power calculations: Assessing type S (Sign) and type M (Magnitude) errors. Perspectives on Psychological Science, 9, 641–651.
22.
GelmanA.TuerlinckxF. (2000). Type S error rates for classical and Bayesian single and multiple comparison procedures. Computational Statistics, 15, 373–390.
23.
HedgesL.V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107–128.
24.
HedgesL.V.OlkinI. (1985). Statistical methods for meta-analysis. Academic Press.
25.
HenningG. (1986). Quantitative methods in language acquisition research. TESOL Quarterly, 20, 701–708.
26.
KlineR.B. (2004). Beyond significance testing: Reforming data analysis methods in behavioral research. American Psychological Association.
27.
LakensD. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.
28.
Larson-HallJ. (2010). A guide to doing statistics in second language research using SPSS and R. Routledge.
29.
LiG.NiX. (2013). Effects of a technology-enriched, task-based language teaching curriculum on Chinese elementary students’ achievement in English as a foreign language. International Journal of Computer-Assisted Language Learning and Teaching, 3, 33–49.
30.
LiaoY.K.C. (1999). Effects of hypermedia on students’ achievement: a meta-analysis. Journal of Educational Multimedia and Hypermedia, 8, 255–277.
31.
LindstrombergS. (2016). Inferential statistics in language teaching research: A review and ways forward. Language Teaching Research, 20, 741–768.
LongM. (2015). Second language acquisition and task-based language teaching. Wiley-Blackwell.
34.
NunanD. (1990). The teacher as researcher. In BrumfitC.MitchellR. (Eds.), Research in the language classroom (pp. 16–32). British Council.
35.
NyeB.KonstantopoulosS.HedgesL.V. (2004). How large are teacher effects?Educational Evaluation and Policy Analysis, 26, 237–257.
36.
OrwinR.G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159.
37.
PastorD.A.LazowskiR.A. (2018). On the multilevel nature of meta-analysis: A tutorial, comparison of software programs, and discussion of analytic choices. Multivariate Behavioral Research, 53, 74–89.
38.
PearsonP.D.FerdigR.E.BlomeyerR.L.MoranJ. (2005). The effects of technology on reading performance in the middle-school grades: A meta-analysis with recommendation for policy. Learning Point Associates.
39.
PigottT.D.PolaninJ.R. (2019). Methodological guidance paper: High-quality meta-analysis in a systematic review. Review of Educational Research, 90, 24–46.
40.
PlonskyL. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655–687.
41.
PlonskyL.OswaldF.L. (2014). How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning, 64, 878–912.
42.
PolaninJ.R.PigottT.D. (2015). The use of meta-analytic statistical significance testing. Research Synthesis Methods, 6, 63–73.
43.
SchönbrodtF.D.PeruginiM. (2013). At what sample size do correlations stabilize?Journal of Research in Personality, 47, 609–612.
44.
SedlmeierP.GigerenzerG. (1989). Do studies of statistical power have an effect on the power of studies?Psychological Bulletin, 105, 309–316.
45.
ShadishW.R.CookT.D.CampbellD.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton, Mifflin and Company.
46.
SkehanP. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30, 510–532.
47.
SlavinR.SmithD. (2009). The relationship between sample sizes and effect sizes in systematic reviews in education. Educational Evaluation and Policy Analysis, 31, 500–506.
48.
SongM.HermanR. (2010). Critical issues and common pitfalls in designing and conducting impact studies in education: Lessons learned from the What Works Clearinghouse. Educational Evaluation and Policy Analysis, 32, 351–371.
49.
SterneJ.A.BeckerB.J.EggerM. (2005). The funnel plot. In RothsteinH.R.SuttonA.J.BorensteinM. (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 75–98). John Wiley.
50.
SwanM. (2005). Legislation by hypothesis: The case of task-based instruction. Applied Linguistics, 26, 376–401.
51.
Van den BrandenK. (2016). Task-based language teaching. In HallG. (Ed.), The Routledge handbook of English language teaching (pp. 238–251). Routledge.
52.
VogtW.P.JohnsonB. (2011). Dictionary of statistics and methodology: A nontechnical guide for the social sciences. Sage.
53.
WhiteI.R.BarrettJ.K.JacksonD.HigginsJ.P. (2012). Consistency and inconsistency in network meta-analysis: Model estimation using multivariate meta-regression. Research Synthesis Methods, 3, 111–125.
54.
WiddowsonH.G. (2003). Defining issues in English language teaching. Oxford University Press.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
