High-quality systematic literature reviews provide a systematic process for identifying, synthesizing, and critiquing multiple studies and, in turn, inform theory, research, practice, and policy. With a focus on special education systematic reviews, we propose four core principles (i.e., coherence, contextualization, generativity, and transparency) to guide scholars in conducting meta-analyses, systematic narrative reviews, and qualitative meta-syntheses. Specifically, we articulate how scholars can promote each of these principles at each stage of the review process to enhance the rigor, relevance, and credibility of their systematic review: (a) framing the problem and research questions; (b) planning procedures to identify, analyze, and synthesize studies; and (c) presenting and interpreting results. Last, we discuss future directions in light of issues of equity and approaches to bridge synthesis findings and practice and policy.
Get full access to this article
View all access options for this article.
References
1.
AlexanderP. A. (2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90(1), 6–23. https://doi.org/10.3102/0034654319854352
2.
AlzahraniT.LekoM. (2018). The effects of peer tutoring on the reading comprehension performance of secondary students with disabilities: A systematic review. Reading and Writing Quarterly, 34(1), 1–17. https://doi.org/10.1080/10573569.2017.1302372
3.
AnnammaA. A.FerriB. A.ConnorD. J. (2018). Disability critical race theory: Exploring the intersectional lineage, emergence, and potential futures of DisCrit in education. Review of Research in Education, 42(1), 46–71. https://doi.org/10.3102/0091732X18759041
4.
BalA. A.TrainorA. A. (2016). Culturally responsive experimental intervention studies: The development of a rubric for paradigm expansion. Review of Educational Research, 86(2), 319–359. https://doi.org/10.3102/0034654315585004
5.
BarnesT. N.SmithS. W.MillerM. D. (2014). School-based cognitive-behavioral interventions in the treatment of aggression in the United States: A meta-analysis. Aggression and Violent Behavior, 19(4), 311–321. https://doi.org/10.1016/j.avb.2014.04.013
6.
BettiniE.CrockettJ.BrownellM. T.MerrillK. (2016). Relationships between working conditions and special educators’ instruction. Journal of Special Education, 50(3), 178–190. https://doi.org/10.1177/0022466916644425
7.
BillingsleyB.BettiniE. (2019). Special education teacher attrition and retention: A review of the literature. Review of Educational Research, 89(5), 697–744. https://doi.org/10.3102/0034654319862495
8.
BorensteinM.HedgesL. V.HigginsJ. P.RothsteinH. R. (2021). Introduction to meta-analysis. Wiley.
9.
BovedaM.BattacharyaK. (2019). Love as de/colonial onto-epistemology: A post-oppositional approach to contextualized research ethics. Urban Review, 51(1), 5–25. https://doi.org/10.1007/s11256-018-00493-z
10.
BrockM. E.Cannella-MaloneH. I.SeamanR. L.AndzikN. R.SchaeferJ. M.PageE. J.BarczakM. A.DuekerS. A. (2017). Findings across practitioner training studies in special education: A comprehensive review and meta-analysis. Exceptional Children, 84(1), 7–26. https://doi.org/10.1177/0014402917698008
11.
BrunstingN. C.SreckovicM. A.LaneK. L. (2014). Special education teacher burnout: A synthesis of research from 1979 to 2013. Education and Treatment of Children, 37(4), 681–711. https://doi.org/10.1353/etc.2014.0032
12.
CastilloW.GillbornD. (2022). How to “QuantCrit:” Practices and questions for education data researchers and users (Working Paper No. 22–546). https://doi.org/10.26300/v5kh-dd65
13.
ChoiJ.LeeM. (2021). Score manipulation, density continuity, and intent-to-treat effect for regression discontinuity. Pacific Economic Review. Advance online publication. https://doi.org/10.1111/1468-0106.12380
14.
ChowJ. C. (2021). Using quasi-experimental designs to establish causal inference in special education. In HottB.BrighamF. J.PeltierC. (Eds.), Special education research design and methodological approaches (pp. 149–164). SLACK Incorporated Group.
15.
ChowJ. C.EkholmE. (2018). Do published studies yield larger effect sizes than unpublished studies in education and special education?: A meta-review. Educational Psychology Review, 30, 727–744. https://doi.org/10.1007/s10648-018-9437-7
16.
ChowJ. C.SjogrenA. L.ZhaoH. (2021). Reporting and reproducibility of meta-analysis in speech, language, and hearing research. Journal of Speech, Language, and Hearing Research, 64(7), 2786–2793. https://doi.org/10.1044/2021_JSLHR-21-00047
17.
ChowJ. C.WallaceE. S.SenterR.KummS.MasonC. Q. (2022). A systematic review and meta-analysis of the language skills of youth offenders. Journal of Speech, Language, and Hearing Research, 65(3), 1166–1182. https://doi.org/10.1044/2021_JSLHR-20-00308
18.
CookB. G.LloydJ. W.MellorD.NosekB. A.TherrienW. J. (2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104–118. https://doi.org/10.1177/0014402918793138
19.
CooperH.HedgesL. V.ValentineJ. C. (Eds.). (2019). The handbook of research synthesis and meta-analysis (3rd ed.). Russell Sage.
20.
CrenshawK. (1991). Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43(6), 1241–1300. https://doi.org/10.2307/1229039
21.
CrottyM. (1998). The foundations of social research: Meaning and perspective in the research process. Sage.
22.
CummingM. M.BettiniE.PhamA. V.ParkJ. (2020). School-, classroom-, and dyadic-level experiences: A literature review of their relationship with students’ executive functioning development. Review of Educational Research, 90(1), 47–94. https://doi.org/10.3102/0034654319891400
23.
DonnellyC. A.BoydI.CampbellP.CraigC.VallanceP.WalportM.WhittyC. J.WoodsE.WormaldC. (2018). Four principles for synthesizing evidence. Nature, 588, 361–364. https://www.nature.com/articles/d41586-018-05414-4
24.
DouglasS. N.MeadanH.SchutheissH. (2022). A meta-synthesis of caregivers’ experiences transitioning from early intervention to early childhood special education. Early Childhood Education Journal, 50, 371–383. https://doi.org/10.1007/s10643-021-01165-6
25.
FreedleA.WahmanC. L. (2021). Understanding families’ experiences raising young children with challenging behavior through an ecological lens: A qualitative meta-synthesis. Topics in Early Childhood Special Education. https://doi.org/10.1177/02711214211036924
26.
GanannR.CiliskaD.ThomasH. (2010). Expediting systematic reviews: Methods and implications of rapid reviews. Implementation Science, 5(5), 1–12. https://doi.org/10.1186/1748-5908-5-5
27.
GeselS. A.LeJeuneL. M.ChowJ. C.SinclairA. C.LemonsC. J. (2021). A meta-analysis of the impact of professional development on teachers’ knowledge, skill, and self-efficacy in data-based decision-making. Journal of Learning Disabilities, 54(4), 269–283. https://doi.org/10.1177/0022219420970196
GoldmanS. E.BurkeM. M. (2017). The effectiveness of interventions to increase parent involvement in special education: A systematic literature review and meta-analysis. Exceptionality, 25(2), 97–115. https://doi.org/10.1080/09362835.2016.1196444
30.
HavenT. L.ErringtonT. M.GleditschK. S.van GrootelL.JacobsA. M.KernF. G.PiñeiroR.RosenblattF.MokkinkL. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406920976417
31.
Hernández-SacaD. I.Gutmann KahnL.CannonM. A. (2018). Intersectionality dis/ability research: How dis/ability research in education engages intersectionality to uncover the multidimensional construction of dis/abled experiences. Review of Research in Education, 42(1), 286–311. https://doi.org/10.3102/0091732X18762439
32.
HigginsJ. P.ThomasJ.ChandlerJ.CumpstonM.LiT.PageM. J.WelchV. A. (Eds.). (2019). Cochrane handbook for systematic reviews of interventions. Wiley.
33.
HiranoK.RoweD.LindstromL.ChanP. (2018). Systemic barriers to family involvement in transition planning for youth with disabilities: A qualitative metasynthesis. Journal of Child and Family Studies, 27, 3440–3456. https://doi.org/10.1007/s10826-018-1189-y
JonesN. D.BettiniE.BrownellM. T. (2019). Competing strands of educational reform policy: Can collaborative school reform and teacher evaluation reform be reconciled?Elementary School Journal, 113(3), 468–486. https://doi.org/10.1086/701706
36.
KilgusS.EklundK.MagginD.TaylorC.AllenA. (2018). The Student Risk Screening Scale: A reliability and validity generalization meta-analysis. Journal of Emotional and Behavioral Disorders, 26(3), 143–155. https://doi.org/10.1177/1063426617710207
37.
KingT.FazelM. (2019). Examining the mental health outcomes of peer-led school-based interventions on young people aged between 4 and 18 years old: A systematic review protocol. Systematic Reviews, 8(1), 1–6. https://doi.org/10.1186/s13643-019-1027-3
38.
KingT.FazelM. (2021). Examining the mental health outcomes of school-based peer-led interventions on young people: A scoping review of range and a systematic review of effectiveness. PLOS ONE, 16(4), Article e0249553. https://doi.org/10.1371/journal.pone.0249553
39.
KoD.MaweneD.RobertsK.HongJ. J. (2021). A systematic review of boundary-crossing partnerships in designing equity-oriented special education services for culturally and linguistically diverse students with disabilities. Remedial and Special Education, 42(6), 412–425. https://doi.org/10.1177/0741932520983474
40.
Koro-LjungbergM.Yendol-HoppeyD.SmithJ. J.HayesS. B. (2009). (E)pistemological awareness, instantiation of methods, and uninformed methodological ambiguity in qualitative research projects. Educational Researcher, 38(9), 687–699. https://doi.org/10.3102/0013189X09351980
41.
KulkarniS. (2017). Disproportionate representation of Asian students in special education: A systematic review of the literature. Multiple Voices for Ethnically Diverse Exceptional Learners, 17(2), 19–33. https://doi.org/10.5555/2158-396X.17.2.19
42.
MagginD. M.TalbottE.Van AckerE. Y.KummS. (2017). Quality indicators for systematic reviews in behavioral disorders. Behavioral Disorders, 42(2), 52–64. https://doi.org/10.1177/0198742916688653
43.
MerrillK. L.SmithS. W.CummingM. M.DaunicA. P. (2017). A review of social problem-solving interventions: Past findings, current status, and future directions. Review of Educational Research, 87(1), 71–102. https://doi.org/10.3102/0034654316652943
44.
MilnerH. R. (2007). Race, culture, and researcher positionality: Working through dangers seen, unseen, and unforeseen. Educational Researcher, 36(7), 388–400. https://doi.org/10.3102/0013189X07309471
45.
NguyenT. D.PhamL. D.CrouchM.SpringerM. G. (2020). The correlates of teacher turnover: An updated and expanded meta-analysis of the literature. Educational Research Review, 31, Article 100355. https://doi.org/10.1016/j.edurev.2020.100355
46.
PageM. J.MoherD.BossuytP. M.BoutronI.HoffmannT. C.MulrowC. D.ShamseerL.TetzlaffJ. M.AklE. A.BrennanS. E.ChouR.GlanvilleJ.GrimshawJ. M.HróbjartssonA.LaluM. M.LiT.LoderE. W.McDonaldS.McGuinnessL. A.StewartL. A.McKenzieJ. E. (2021). PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. BMJ, 372, Article n160. https://doi.org/10.1136/bmj.n160
47.
PengP.LinX.ÜnalZ. E.LeeK.NamkungJ.ChowJ.SalesA. (2020). Examining the mutual relations between language and mathematics: A meta-analysis. Psychological Bulletin, 146(7), 595.
48.
PeurachD. J.RussellJ. L.Cohen-VogelL.PenuelW. R. (2022). The foundational handbook on improvement research in education. Rowman & Littlefield.
49.
PolisenaJ.GarrittyC.KamelC.StevensA.Abou-SettaA. M. (2015). Rapid review programs to support health care and policy decision making: A descriptive analysis of processes and methods. Systematic Reviews, 4(26), 1–7. https://doi.org/10.1186/s13643-015-0022-6
50.
ReichowB.ServiliC.YasamyM. T.BarbuiC.SaxenaS. (2013). Non-specialist psychosocial interventions for children and adolescents with intellectual disability or lower-functioning autism spectrum disorders: A systematic review. PLoS Medicine, 10(12), Article e1001572. https://doi.org/10.1371/journal.pmed.1001572
51.
RodgersW. J.Morris-MathewsH.RomigJ. E.BettiniE. (2022). Observation studies in special education: A synthesis of validity evidence for observation systems. Review of Educational Research, 92(1), 3–45. https://doi.org/10.3102/00346543211042419
52.
RothJ.Sant’AnnaP. H. C.BilinskiA.PoeJ. (2022). What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. https://arxiv.org/abs/2201.01194.
53.
ScruggsT. E.MastropieriM. A.McDuffieK. A. (2007). Co-teaching in inclusive classrooms: A metasynthesis of qualitative research. Exceptional Children, 74(4), 392–416. https://doi.org/10.1177/001440290707300401
54.
ShamseerL.MoherD.ClarkeM.GhersiD.LiberatiA.PetticrewM.ShekelleP.StewartL. A., & Prisma-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. BMJ, 350, Article g7647. https://doi.org/10.1136/bmj.g7647
55.
ShapiroA. (2022). Over diagnosed or over looked? The effect of age at time of school entry on students receiving special education services. Exceptional Children. https://doi.org/10.1177/00144029221108735
ShpancerN. (2010). The good psychologist. Henry Holt.
58.
SiddawayA. P.WoodA. M.HedgesL. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70, 747–770. https://doi.org/10.1146/annurev-psych-010418-102803
59.
StageF. K.WellsR. S. (2014). Critical quantitative inquiry in context. New Directions for Institutional Research, 158, 1–7. https://doi.org/10.1002/ir.20041
60.
TalbottE.MagginD. M.Van AckerE. Y.KummS. (2018). Quality indicators for reviews of research in special education. Exceptionality, 26(4), 245–265. https://doi.org/10.1080/09362835.2017.1283625
61.
TrainorA. A.GraueE. (2014). Evaluating rigor in qualitative methodology and research dissemination. Remedial and Special Education, 35(5), 267–274. https://doi.org/10.1177/0741932514528100
62.
TriccoA. C.LillieE.ZarinW.O'BrienK. K.ColquhounH.LevacD.MoherD.PetersM.HorsleyT.WeeksL.HempelS.AklE. A.ChangC.McGowanJ.StewartL.HartlingL.AldcroftA.WilsonM. G.GarrittyC.LewinS.StrausS. E. (2018). PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850
WilsonS. M.AnagnostopoulosD. (2021). Methodological guidance paper: The craft of conducting a qualitative review. Review of Educational Research, 91(5), 651–670. https://doi.org/10.3102/00346543211012755
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
