Sage Journals: Discover world-class research

Abstract

We conducted a systematic review of missing data handling and reporting in a 1-year cross-section (2020) of group design special education intervention research. We identified 121 intervention studies. Of these 72 (60%) reported missing data at the case, variable, or item level. Few studies examined the mechanism underlying their missing data and most handled the missingness with either complete case analysis (listwise) or pairwise deletion. We discuss the implications of these findings and offer recommendations for improved reporting and handling methods.

Keywords

missing data group experimental research intervention research systematic

Get full access to this article

View all access options for this article.

References

Afkanpour

Hosseinzadeh

Tabesh

(2024). Identify the most appropriate imputation method for handling missing values in clinical structured datasets: A systematic review. BMC Medical Research Methodology, 24, Article 188. https://doi.org/10.1186/s12874-024-02310-6

Allison

P. D.

(2002). Missing data. Sage Publications. https://doi.org/10.1016/j.ssaho.2025.101451

American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40. https://doi.org/10.35925/j.multi.2023.2.20

American Psychological Association. (2010). Publication Manual of the American Psychological Association (6th ed.). American Psychological Association. https://doi.org/10.1002/jee.70019

American Psychological Association. (2020). Publication Manual of the American Psychological Association (7th ed.). American Psychological Association. https://doi.org/10.1016/j.compedu.2024.104999

American Psychological Association. (2024). APA Style Journal Article Reporting Standards. Retrieved from https://apastyle.apa.org/jars/quantitative https://doi.org/10.1002/jee.20588

Arbuckle

J. L.

(1996). Full information estimation in the presence of incomplete data. In Marcoulides

G. A.

Schumacker

R. E.

(Eds.), Advanced structural equation modeling: Issues and techniques (pp. 243–277). Psychology Press.

Bordini

Paula

C. S.

Cunha

G. R.

Caetano

S. C.

Bagaiolo

L. F.

Ribeiro

T. C.

Martone

M. C. C.

Portolese

Moya

A. C.

Brunoni

Bosa

Brentani

Cogo-Moreira

Jesus Mari

(2020). A randomised clinical pilot trial to test the effectiveness of parent training with video modelling to improve functioning and symptoms in children with autism spectrum disorders and intellectual disability. Journal of Intellectual Disability Research, 64(8), 629–643. https://doi.org/10.1111/jir.12759

Chow

Zhao

Sandbank

Bottema-Beutel

Woynaroski

(2023). Empirically-derived effect size distributions of interventions for young children on the autism spectrum. Journal of Clinical Child & Adolescent Psychology, 52(2), 271–283. https://doi.org/10.1080/15374416.2021.2007485

10.

Chung

Cai

(2019). Alternative multiple imputation inference for categorical structural equation modeling. Multivariate Behavioral Research, 54(3), 323–337. https://doi.org/10.1080/00273171.2018.1523000

11.

Collins

L. M.

Schafer

J. L.

Kam

C.-M

. (n.d.). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330–351. https://doi.org/10.1037/1082-989X.6.4.330

12.

Cook

B. G.

Lloyd

J. W.

Mellor

Nosek

B. A.

Therrien

W. J.

(2018). Promoting open science to increate the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104–118. https://doi.org/10.1177/0014402918793138

13.

Cook

B. G.

Therrien

W. J.

Fleming

J. I.

Espinas

McLucas

A. S.

Hodges

Fan

(2025). Null effects in the special education research base. Remedial and Special Education. Advanced online publication. https://doi.org/10.1177/07419325251361907

14.

Cook

B. G.

Wong

V. C.

Fleming

J. I.

Solari

E. J.

(2022). Preregistration of randomized controlled trials. Research on Social Work:, 35(3), 277–286. https://doi.org/10.1177/10497315221121117

15.

Corr

Love

Snodgrass

M. R.

Kern

J. L.

Chudzik

(2022). Methodological training in special education doctoral programs: A mixed-methods exploration. Teacher Education and Special Education, 46(2), 108–126. https://doi.org/10.1177/08884064221103902

16.

Crocker

Algina

(1986). Introduction to classifical and modern test theory. Harcourt Brace Jovanovich College Publishers.

17.

Daniels

M. J.

Hogan

J. W.

(2008). Missing data in longitudinal studies: Strategies for Bayesian modeling and sensitivity analysis. Chapman Hall/CRC. https://doi.org/10.1002/j.2168-9830.2006.tb00884.x

18.

Dempster

A. P.

Laird

N. M.

Rubin

D. B.

(1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1), 1–38. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

19.

Duncan

S. C.

Duncan

T. E.

(1994). Modeling incomplete longitudinal substance use data using latent variable growth curve methodology. Multivariate Behavioral Research, 29(4), 313–338. https://doi.org/10.1207/s15327906mbr2904_1

20.

Dziura

J. D.

Post

L. A.

Zhao

Fuz

Peduzzi

(2013). Strategies for dealing with missing data in clinical trials: From design to analysis. Yale Journal of Biology and Medicine, 86(3), 343–358.

21.

Eekhout

de Boer

R. M.

Twisk

J. W. R.

de Vet

H. C. W.

Heymans

M. W.

(2012). Missing data: A systematic review of how they are reported and handled. Epidemiology, 23(5), 729–732. https://doi.org/10.1097/EDE.0b013e3182576cdb

22.

Enders

C. K.

(2022). Applied missing data analysis (2nd ed.). Guilford Press. https://doi.org/10.1016/j.ifacol.2024.07.134

23.

Enders

C. K.

(2023). Missing data: An update on the state of the art. Psychological Methods, 30(2), 322–339. https://doi.org/10.1037/met0000563

24.

Enders

C. K.

Baraldi

A. N.

Cham

(2014). Estimating interaction effects with incomplete predictor variables. Psychological Methods, 19(1), 39–55. https://doi.org/10.1037/a0035314

25.

Enders

C. K.

Dietz

Montague

Dixon

(2006). Modern alternatives for dealing with missing data in special education research. Advances in Learning and Behavioral Disabilities, 19, 101–129. https://doi.org/10.1016/S0734-004X(06)19005-9

26.

Enders

C. K.

Hayes

(2022). Missing data handling for multilevel data. In O’Connell

McCoach

D. B.

Bell

(Eds.), Multilevel modeling methods with introductory and advanced methods. Information Age Publishing. https://doi.org/10.1007/s10639-025-13368-0

27.

Enders

C. K.

Hayes

(2018). A comparison of multilevel imputation schemes for random coefficient models: Fully conditional specification and joint model imputation with random covariance matrices. Multivariate Behavioral Research, 53(5), 695–713. https://doi.org/10 .1080/00273171.2018.1477040

28.

Enders

C. K.

Keller

B. T.

Levy

(2018). A fully conditional specification approach to multilevel imputation of categorical and continuous variables. Psychological Methods, 23(2), 298–317. https://doi.org/10 .1037/met0000148

29.

Enders

C. K.

Mistler

S. A.

Keller

B. T.

(2016). Multilevel multiple imputation: A review and evaluation of joint modeling and chained equations imputation. Psychological Methods, 21(2), 222–240. https://doi.org/10.1037/met0000063

30.

Enders

C. L.

(2004). The impact of missing data on sample reliability estimates: Implications for reliability reporting practices. Educational and Psychological Measurement, 64(3), 419–436. https://doi.org/10.1177/0013164403261050

31.

Enders

C. L.

(2010). Applied missing data analysis. Guilford Press.

32.

Enders

C. L.

(2011). Missing not at random models for latent growth curve analyses. Psychological Methods, 16(1), 1–16. https://doi.org/10.1037/a0022640

33.

Feng

Hancock

G. R.

(2021). Oh no! They cut my funding! using “post hoc” planned missing designs to salvage longitudinal research. Child Development, 92(3), 1199–1216. https://doi.org/10.1111/cdev.13501

34.

Fleming

J. I.

Wilson

S. E.

Espinas

van Dijk

Cook

B. G.

(2024). Special education researchers’ knowledge, attitudes, and reported use of open practices. Remedial and Special Education, 46(5), 359–370. Advanced online publication. https://doi.org/10.1177/07419325241237268

35.

Garbacz

S. A.

McIntyre

L. L.

Stormshak

E. A.

Kosty

D. B.

(2020). The efficacy of the family check-up on children’s emotional and behavior problems in early elementary school. Journal of Emotional and Behavioral Disorders, 28(2), 67–79. https://doi.org/10.1177/1063426618806258

36.

Gersten

Fuchs

L. S.

Compton

Coyne

Greenwood

Innocenti

M. S.

(2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71(2), 149–164. https://doi.org/10.1177/001440290507100202

37.

Goldstein

Carpenter

J. R.

Browne

W. J.

(2014). Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non-linear terms. Journal of the Royal Statistical Society: Series A: Statistics in Society, 177(2), 553–564. https://doi.org/10.1111/rssa.12022

38.

Gomer

Yuan

K.-H.

(2021). Subtypes of the missing not at random missing data mechanism. Psychological Methods, 26(5), 559–598. https://doi.org/10.1037/met0000377

39.

Gomer

Yuan

K.-H.

(2023). A realistic evaluation of methods for handling missing data when there is a mixture of MCAR, MAR, and MNAR mechanisms in the same dataset. Multivariate Behavioral Research, 58(5), 988–1013. https://doi.org/10.1080/00273171.2022.2158776

40.

Graham

J. W.

(2003). Adding missing-data-relevant variables to FIML-based structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 10(1), 80–100. https://doi.org/10.1207/S15328007SEM1001_4

41.

Graham

J. W.

(2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60(1), 549–576. https://doi.org/10.1146/annurev.psych.58.110405.085530

42.

Graham

J. W.

Hofer

S. M.

MacKinnon

D. P.

(1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures. Multivariate Behavioral Research, 31(2), 197–218. https://doi.org/10.1207/s15327906mbr3102_3

43.

Hayes

Enders

C. K.

(2023). Maximum likelihood and multiple imputation missing data handling: How they work, and how to make them work in practice. In Hooper

(Ed.), APA Handbook of research methods in psychology: Vol. 3. Data analysis and research publication (2nd ed., pp. 27–51). American Psychological Association. https://doi.org/10.1037/0000320002

44.

Hunt

Kozleski

Lee

Mortier

Fleming

Hicks

Balasubramanian

Leu

Bross

L. A.

Munandar

Dunlap

Stepaniuk

Aramburo

(2020). Implementing comprehensive literacy instruction for students with severe disabilities in general education classrooms. Exceptional Children, 86(3), 330–347. https://doi.org/10.1177/0014402919880156

45.

Ibrahim

J. G.

(1990). Incomplete data in generalized linear models. Journal of the American Statistical Association, 85(411), 765–769. https://doi.org/ 10.1080/01621459.1990.10474938

46.

Jeličić

Phelps

Lerner

R. M.

(2010). Why missing data matter in the longitudinal study of adolescent development: Using the 4-h study to understand the uses of different missing data methods. Journal of Youth and Adolescence, 39(7), 816–835. https://doi.org/10.1007/s10964-010-9542-5

47.

Moore

E. W.

Kinai

Crowe

K. S.

Schoemann

A. M.

Little

T. D.

(2014). Planned missing data designs with small sample sizes: How small is too small? International Journal of Behavioral Development, 38(5), 435–452. https://doi.org/10.1177/0165025414531095

48.

(2022). The use of multiple imputation to handle missing data in secondary datasets: Suggested approaches when missing data results from the survey structure. INQUIRY: The Journal of Health Care Organization, Provision, and Financing, 59. https://doi.org/10.1177/00469580221088627

49.

Kaçak

Kılıç

A. F.

(2024). The effects of missing data handling methods on reliability coefficients: A Monte Carlo simulation study. Journal of Measurement and Evaluation in Education and Psychology, 15(2), 166–182. https://doi.org/10.21031/epod.1485482

50.

Kenward

M. G.

Molenberghs

(1998). Likelihood based frequentist inference when data are missing at random. Statistical Science, 13(3), 236–247. https://doi.org/10.1214/ss/1028905886

51.

Köhler

Pohl

Carstensen

C. H.

(2017). Dealing with item nonresponse in large-scale cognitive assessments: The impact of missing data methods on estimated explanatory relationships: Impact of missing data on estimated relationships. Journal of Educational Measurement, 54(4), 397–419. https://doi.org/10.1111/jedm.12154

52.

Kraft

M. A.

(2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. https://doi.org/10.3102/0013189X20912798

53.

Lee

J. H.

Huber

J. C.

(2021). Evaluation of multiple imputation with large proportions of missing data: How much is too much? Iranian Journal of Public Health, 50(7), 1372–1380. https://doi.org/10.18503/ijph.v50i7.6626

54.

Lee

Leite

W. L.

(2024). A comparison of random forest-based missing imputation methods for covariates in propensity score analysis. Psychological Methods. Advanced online publication. https://doi.org/10.1037/met0000676

55.

Lipsitz

S. R.

Ibrahim

J. G.

(1996). A conditional model for incomplete covariates in parametric regression models. Biometrika, 83(4), 916–922. https://doi.org/10.1093/biomet/83.4.916

56.

Little

R. J.

Carpenter

J. R.

Lee

K. J.

(2022). A comparison of three popular methods for handling missing data: Complete-case analysis, inverse probability weighting, and multiple imputation. Sociological Methods & Research, 53(3), 1105–1135. https://doi.org/10.1177/00491241221113873

57.

Little

R. J. A.

(1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.1080/01621459.1988.10478722

58.

Little

R. J. A.

(2021). Missing data assumptions. Annual Review of Statistics and Its Application, 8, 89–107. https://doi.org/10.1146/annurev-statistics-040720031104

59.

Little

R. J. A.

Rubin

D. B.

(2020). Statistical analysis with missing data (3rd ed.). John Wiley & Sons.

60.

Little

T. D.

Rhemtulla

(2013). Planned missing data designs for developmental researchers. Child Development Perspectives, 7(4), 199–204. https://doi.org/10.1111/cdep.12043

61.

Lüdtke

Robitzsch

Grun

(2017). Multiple imputation of missing data in multilevel designs: A comparison of different strategies. Psychological Methods, 22(1), 141–165. https://doi.org/10.1037/met0000096

62.

Chen

(2018). Bayesian Methods for dealing with missing data problems. Journal of the Korean Statistical Society, 47, 297–313. https://doi.org/10.1016/j.jkss.2018.03.0021226-3192

63.

Manly

C. A.

Wells

R. S.

(2015). Reporting the use of multiple imputation for missing data in higher education research. Research in Higher Education, 56(4), 397–409. https://doi.org/10.1007/s11162-014-9344-9

64.

Marini

M. M.

Olsen

A. R.

Rubin

D. B.

(1980). Maximum-likelihood estimation in panel studies with missing data. Sociological Methodology, 11, 314–357. https://doi.org/10.2307/270868

65.

Mazza

G. L.

Enders

C. K.

Ruehlman

L. S.

(2015). Addressing item-level missing data: A comparison of proration and full information maximum likelihood estimation. Multivariate Behavioral Research, 50(5), 504–519. https://doi.org/10.1080/00273171.2015.1068157

66.

McGregor

H. A.

Sanner

C. M.

Neece

C. L.

(2020). Effects of MBSR parent intervention on internalizing problems in children: ASD status as a moderator. Journal of Mental Health Research in Intellectual Disabilities, 13(4), 343–363. https://doi.org/10.1080/19315864.2020.1815913

67.

McNeish

(2017). Missing data methods for arbitrary missingness with small samples. Journal of Applied Statistics, 44(1), 24–29. https://dx.https://doi.org/10.1080/02664763.2016.1158246

68.

Mealli

Rubin

D. B.

(2016). Clarifying missing at random and related definitions, and implications when coupled with exchangeability. Biometrika, 103(2), 491–491. https://doi.org/10.1093/biomet/asw017

69.

Mikolajczyk

Jankowicz-Szymańska

(2020). The effect of kinesio taping on the balance and foot arching in children with intellectual disability. Journal of Intellectual & Developmental Disability, 45(1), 46–53. https://doi.org/10.3109/13668250.2018.1478805

70.

Mohan

Pearl

(2021). Graphical models for processing missing data. Journal of the American Statistical Association, 116(534), 1023–1037. https://doi.org/10.1080/01621459.2021.1874961

71.

Murane

R. J.

Willett

J. B.

(2011). Methods matter: Improving causal inference in educational and social science research. Oxford University Press.

72.

Muthén

Kaplan

Hollis

(1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52, 431–462. https://doi.org/10.1007/BF02294365

73.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education.

74.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Shamseer

Tetzlaff

J. M.

Akl

E. A.

Brennan

S. E.

Chou

Glanville

Grimshaw

J. M.

Hróbjartsson

Lalu

M. M.

Loder

E. W.

Mayo-Wilson

McDonald

… Moher

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

75.

Peugh

J. L.

Enders

C. K.

(2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74(4), 525–556. https://doi.org/10.3102/00346543074004525

76.

Peugh

J. L.

Toland

M. D.

Harrison

(2023). A tutorial for handling suspected missing not at random data in longitudinal clinical trials. The Quantitative Methods for Psychology, 19(4), 347–367. https://doi.org/10.20982/tqmp.19.4.p347

77.

Raykov

(2011). On testability of missing data mechanisms in incomplete data sets. Structural Equation Modeling: A Multidisciplinary Journal, 18(3), 419–429. https://doi.org/10.1080/10705511.2011.582396

78.

Rhemtulla

Hancock

G. R.

(2016). Planned missing data designs in educational psychology research. Educational Psychologist, 51, 305–316. https://doi.org/10.1080/00461520.2016.1208094

79.

Rubin

D. B.

(1976). Inference and missing data. Biometrika, 63(3), 581–592. https://doi.org/10.1093/biomet/63.3.581

80.

Rubin

D. B.

(1987). Multiple imputation for nonresponse in surveys. John Wiley. https://doi.org/10.1002/9780470316696

81.

Schafer

J. L.

Graham

J. W.

(2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. https://doi.org/10.1037/1082-989X.7.2.147

82.

Schulz

K. F.

Altman

D. G.

Moher

, & the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. Annals of Internal Medicine, 152(11), 726–732. https://doi.org/10.7326/0003-4819-152-11-201006010-00232

83.

Seaman

S. R.

White

I. R.

(2011). Review of inverse probability weighting for dealing with missing data. Statistical Methods in Medical Research, 22(3), 278–295. https://doi.org/10.1177/0962280210395740

84.

Shin

Davidson

M. L.

Long

J. D.

(2017). Maximum likelihood versus multiple imputation for missing data in small longitudinal samples with nonnormality. Psychological Methods, 22(3), 426–449. https://doi.org/10.1037/met0000094

85.

Sijtsma

van der Ark

L. A.

(2003). Investigation and treatment of missing item scores in test and questionnaire data. Multivariate Behavioral Research, 38(4), 505–528. https://doi.org/10.1207/s15327906mbr3804_4

86.

Smith

Toms

Constantin

Johnson

Harding

Brosnan

(2020). Piloting a digitally-mediated social story intervention for autistic children led by teachers within naturalistic school settings. Research in Autism Spectrum Disorders, 75, 101533. https://doi.org/10.1016/j.rasd.2020.101533

87.

Stavseth

M. R.

Clausen

Røislien

(2019). How handling missing data may impact conclusions: A comparison of six different imputation methods for categorical questionnaire data. SAGE Open Medicine, 7. https://doi.org/10.1177/2050312118822912

88.

Stevens

E. A.

Vaughn

Swanson

Scammacca

(2020). Examining the effects of a tier 2 Reading comprehension intervention aligned to tier 1 instruction for fourth-grade struggling readers. Exceptional Children, 86(4), 430–448. https://doi.org/10.1177/0014402919893710

89.

Toste

J. R.

Logan

J. A. R.

Shogren

K. A.

Boyd

B. A.

(2023). The next generation of quality indicators for group design research in special education. Exceptional Children, 89(4), 359–378, 001440292211508. https://doi.org/10.1177/00144029221150801

90.

Toste

J. R.

Talbott

Cumming

M. M.

(2023). Special issue preview: Introducing the next generation of quality indicators for research in special education. Exceptional Children, 89(4), 357–358. https://doi.org/10.1177/00144029231174106

91.

Van Ginkel

J. R.

Sijtsma

Van Der Ark

L. A.

Vermunt

J. K.

(2010). Incidence of missing item scores in personality measurement, and simple item-score imputation. Methodology, 6(1), 17–30. https://doi.org/10.1027/1614-2241/a000003

92.

van Ginkel

J. R.

van der Ark

L. A.

Sijtsma

(2007). Multiple imputation of item scores in test and questionnaire data, and influence on psychometric results. Multivariate Behavioral Research, 42(2), 387–414. https://doi.org/10.1080/00273170701360803

93.

What Works Clearinghouse. (2022). What Works Clearinghouse procedures and standards handbook, version 5.0. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance (NCEE). https://ies.ed.gov/ncee/wwc/Handbooks

94.

Widaman

K. F.

(2006). Best practices in quantitative methods for developmentalists: III: Missing data: What to do with or without them. Monographs for the Society for Research in Child Development, 71(3), 42–64. https://doi.org/10.1111/j.1540-5834.2006.00404.x

95.

Williams

K. J.

Vaughn

(2020). Effects of an intensive reading intervention for ninth-grade English learners with learning disabilities. Learning Disability Quarterly, 43(3), 154–166. https://doi.org/10.1177/0731948719851745

96.

Woods

A. D.

Gerasimova

Van Dusen

Nissen

Bainter

Uzdavines

Davis-Kean

P. E.

Halvorson

King

K. M.

Logan

J. A. R.

Vasilev

M. R.

Clay

J. M.

Moreau

Joyal-Desmarais

Cruz

R. A.

Brown

D. M. Y.

Schmidt

Elsherif

M. M.

(2023). Best practices for addressing missing data through multiple imputation. Infant and Child Development, e2407. https://doi.org/10.1002/icd.2407

97.

Logan

J. A. R.

(2021). Treatment effects in longitudinal two-method measurement planned missingness designs: An application and tutorial. Journal of Research on Educational Effectiveness, 14(2), 501–522. https://doi.org/10.1080/19345747.2021.1875528

Missing Data Reporting and Handling in Special Education Group Intervention Research

Abstract

Keywords

Get full access to this article

References