Outcomes of Students With Disabilities After Exiting From High School: A Study of Education Data Use and Predictive Analytics

Abstract

We conducted a study of predictive analytics (PA) applied to state data on post-school outcomes (PSO) of exited high-school students with disabilities (SWD). Data analyses with machine learning Random Forest algorithm and multilevel Bayesian ordered logistic regression produced two key findings. One, Random Forest models were accurate in predicting PSO. Two, Bayesian models found high-school graduation was the strongest predictor of higher education and reliably predicted the specific type of outcome relative to other outcomes. Limitations of this study are the data source and small number of predictors. Implications of the study for researchers and educators are discussed in conclusion.

Keywords

post-school outcomes high-school students with disabilities predictive analytics machine learning Bayesian statistics

Get full access to this article

View all access options for this article.

References

Abbasi

Lau

R. Y. K.

Brown

D. E.

(2015). Predicting behavior. IEEE Intelligent Systems, 30(3), 35–43. https://doi.org/10.1109/mis.2015.19

Agresti

(2007). Introduction to Categorical Analysis (2nd ed.). Hoboken, NJ: John C. Wiley and Sons.

Aud

Fox

KewalRamani

(2010). Status and trends in the education of racial and ethnic groups (NCES 2010–015). U.S. Department of Education, National Center for Education Statistics. U.S. Government Printing Office.

Austin

P. C.,

Merlo

(2017). Intermediate and advanced topics in multilevel logistic regression analysis. Statistics in Medicine, 36(20), 3257–3277. https://doi.org/10.1002/sim.7336

Banerjee

(2018). Linear regression: Moneyball – Part 1. Towards Data Science. https://towardsdatascience.com/linear-regression-moneyball-part-1-b93b3b9f5b53

Bauer

D. J.,

Sterba

S. K.

(2011). Fitting multilevel models with ordinal outcomes: Performance of alternative specifications and methods of estimation. Psychological Methods, 16(4), 373–390. https://doi.org/10.1037/a0025813

Blackorby

J.,

Wagner

(1996). Longitudinal postschool outcomes of youth with disabilities: Findings from the national longitudinal transition study. Exceptional Children, 62(5), 399–413. https://doi.org/10.1177/001440299606200502

Bolstad

W. M.

(2007). Introduction to Bayesian statistics (2nd ed.). Wiley.

Breiman

(1996). Bagging predictors. Machine Learning, 24(2), 123–140. https://doi.org/10.1007/bf00058655

10.

Breiman

(2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/a:1010933404324

11.

Brown

D. E.

Abbasi

Lau

R. Y. K.

(2015). Predictive analytics: Predictive modeling at the micro level. IEEE Intelligent Systems, 30(3), 6–8. https://doi.org/10.1109/mis.2015.50

12.

Bureau of Labor Statistics (2020). Employment projections: Employment in STEM occupations. https://www.bls.gov/emp/tables/stem-employment.htm

13.

Burnes

J. J.

Martin

J. E.

Terry

McConnell

A. E.

Hennessey

M. N.

(2018). Predicting post-secondary education and employment outcomes using results from the transition assessment and goal generator. Career Development and Transition for Exceptional Individuals, 41(2), 111–121. https://doi.org/10.1177%2F2165143417705353

14.

Carter

E. W.

Harvey

M. N.

Taylor

J. L.

Gotham

(2013). Connecting youth and young adults with autism spectrum disorders to community life. Psychology in the Schools, 50(9), 888–898. https://doi.org/10.1002/pits

15.

Chin-Yee

B.,

Upshur

(2017). Clinical judgement in the era of big data and predictive analytics. Journal of Evaluation in Clinical Practice, 24(3), 638–645. https://doi.org/10.1111/jep.12852

16.

Couronné

Probst

Boulesteix

A-L.

(2018). Random forest versus logistic regression: A large-scale benchmark experiment. BMC Bioinformatics, 19(270), 1–14. https://doi.org/10.1186/s12859-018-2264-5

17.

Cutler

D. R.

Stevens

J. R.

(2011). Random forests (pp. 1–20). Machine Learning.

18.

DeCoster

Iselin

A. M. R.

Gallucci

(2009). A conceptual and empirical examination of justifications for dichotomization. Psychological Methods, 14(4), 349–366. https://doi.org/10.1037/a0016956

19.

Desai

R. J.

Wang

S. V.

Vaduganathan

Evers

Schneeweiss

(2020). Comparison of machine learning methods with traditional models for use of administrative claims with electronic medical records to predict heart failure outcomes. JAMA Network Open, 3(1), e1918962. http://jamanetwork.com/article.aspx?doi=10.1001/jamanetworkopen.2019.18962

20.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2014). Internet, phone, mail and mixed-mode surveys: The tailored design method (4th ed.). John Wiley & Sons, Inc.

21.

Eaves

R. C.

Rabren

Hall

(2012). The post-school outcomes transition survey: A tool for effective decision making? Career Development for Exceptional Individuals, 38(1), 30–39. https://doi.org/10.1177/1534508412452751

22.

Falkenheim

Burke

Muhlberger

Hale

(2017). Women, minorities, and persons with disabilities in science and engineering: 2017. Special report NSF 17-310. National Science Foundation, National Center for Science and Engineering Statistics. https://www.nsf.gov/statistics/wmpd/

23.

Flauzino

Colado Simão

A. N. C.

de Carvalho Jennings Pereira

W. L.

Alfieri

D. F.

Oliveira

S. R.

Kallaur

A. P.

Lozovoy

M. A. B.

Kaimen-Maciel

D. R.

Maes

Reiche

E. M. V.

(2019). Disability in multiple sclerosis is associated with age and inflammatory, metabolic and oxidative/nitrosative stress biomarkers: Results of multivariate and machine learning procedures. Metabolic Brain Disease, 34(5), 1401–1413. https://doi.org/10.1007/s11011-019-00456-7

24.

Gelman

(2007). Comment: Bayesian checking of the second levels of hierarchical models. Statistical Science, 22(3), 349–352.

25.

Gelman

Carlin

J. B.

Stern

H. S.

Rubin

D. B.

(2004). Bayesian data analysis (2nd ed.). Chapman & Hall/CRC.

26.

Gelman

A.,

Hill

(2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.

27.

Gingerich

J. A.,

Crane

(2021). Transition linkage tool: A systems approach to enhance post-school outcomes. Mathematica. Research funded by U.S. Department of Labor Office of Disability Employment Policy. Contract #1605DC-18-A0020.

28.

Gong

L.,

Flegal

J. M.

(2015). A practical sequential stopping rule for high-dimensional Markov chain Monte Carlo. Journal of Computational and Graphical Statistics, 25(3), 684–700. https://doi.org/10.1080/10618600.2015.1044092

29.

Grilli

L.,

Rampichini

(2012). Multilevel models for ordinal data. In Kennet

Salini

(Eds.), Modern analysis of customer surveys: With applications using R. Wiley.

30.

Haber

M. G.

Mazzotti

V. L.

Mustian

A. L.

Rowe

D. A.

Bartholomew

A. L.

Test

D. W.

Fowler

C. H.

(2016). What works, when, for whom, and with whom: A meta-analytic review of predictors of postsecondary success for students with disabilities. Review of Educational Research, 86(1), 123–162. https://doi.org/10.3102/0034654315583135

31.

Hastings

W. K.

(1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. https://doi.org/10.1093/biomet/57.1.97

32.

Hedeker

(2015). Methods for multilevel ordinal data in prevention science. Prevention Science, 16(7), 997–1006. https://doi.org/10.1007/s11121-014-0495-x

33.

Herodotou

Rienties

Boroowa

Zdrahal

Hlosta

(2019). A large-scale implementation of predictive learning analytics in higher education: The teachers’ role and perspective. Education Technology Research and Development, 67(5), 1273–1306. https://doi.org/10.1007/s11423-019-09685-0

34.

Hosmer

D. W.

Lemeshow

Sturdivant

R. X.

(2013). Applied logistic regression (3rd ed.). John Wiley & Sons.

35.

Hox

J. J.

(2002). Multilevel analysis. Lawrence Erlbaum Associates.

36.

Hwang

J.,

Taylor

J. C.

(2016). Stemming on STEM: A STEM education framework for students with disabilities. Journal of Science Education, 19(1), 39–49. https://doi.org/10.14448/jsesd.09.0003

37.

Individuals with Disabilities Education Act (2004). IDEA 2004, P.L. 108-446, codified as amended at title U.S.C. §20 U.S.C. 1400[d].

38.

Kline

R. B.

(2005). Principles and practices of structural equation modeling (2nd ed.). The Guilford Press.

39.

Kruschke

J. K.

(2011). Doing Bayesian data analysis: A tutorial with R and BUGS. Elsevier, Inc.

40.

Larkan-Skinner

K.,

Shedd

J. M.

(2021). Real-time data predictive analytics: Where does IR fit? New Directions for Institutional Research, 2020(185–186), 11–24. https://doi.org/10.1002/ir.20326

41.

Lipscomb

Haimson

Liu

A. Y.

Burghardt

Johnson

D. R.

Thurlow

M. L.

(2017). Preparing for life after high school: The characteristics and experiences of youth in special education. Findings from the national longitudinal transition study 2012. Volume 2: Comparisons across disability groups: Full report (NCEE 2017–4018). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

42.

Liu

A. Y.

Lacoe

Lipscomb

Haimson

Johnson

D. R.

Thurlow

M. L.

(2018). Preparing for life after high school: The characteristics and experiences of youth in special education. Findings from the national longitudinal transition study 2012. Volume 3: Comparisons over time (Full report) (NCEE 2018-4007). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

43.

Maenner

M. J.

Yeargin-Allsopp

Braun

K. V. N.

Christensen

D. L.

Schieve

L. A.

(2016). Development of a machine learning algorithm for the surveillance of autism spectrum disorder. PLoS One, 11(12), e0168224. https://doi.org/10.1371/journal.pone.0168224

44.

Mandinach

E. B.

(2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71–85. https://doi.org/10.1080/00461520.2012.667064

45.

Mandrekar

J. N.

(2010). Receiver operator characteristic curves in diagnostic test assessment. Journal of Thoracic Oncology, 5(9), 1315–1316. https://doi.org/10.1097/jto.0b013e3181ec173d

46.

Mason

S. J.,

Graham

N. E.

(2002). Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society, 128(584), 2145–2166. https://doi.org/10.1256/003590002320603584

47.

McConnell

A. E.

Martin

J. E.

Hennessey

M. N.

(2015). Indicators of postsecondary employment and education for youth with disabilities in relation to GPA and general education. Remedial and Special Education, 36(6), 327–336. https://doi.org/10.1177/0741932515583497

48.

Meserole

(2018). What is machine learning? Brookings Institution. https://www.brookings.edu/research/what-is-machine-learning/

49.

Metz

C. E.

(1978). Basic principles of ROC analysis. Seminars in Nuclear Medicine, 8(4), 283–298. https://doi.org/10.1016/S0001-2998(78)80014-2

50.

Molfenter

N. F.

Hartman

Neugart

Webb

(2017). Let’s get to work Wisconsin: Launching youth with intellectual and developmental disabilities into the workforce. Journal of Vocational Rehabilitation, 47, 379–390. https://doi.org/10.3233/JVR-170910

51.

Nassif

A. B.

Azzeh

Banitaan

Neagu

(2016). Guest editorial: Special issue on predictive analytics using machine learning. Neural Computing & Applications, 27(1), 2153–2155. https://doi.org/10.1007/s00521-016-2327-3

52.

National Center for Education Statistics (2020). Students with disabilities – indicator. https://nces.ed.gov/programs/coe/indicator_cgg.asp

53.

National Science Foundation . (2021). Women, minorities, and persons with Disabilities in science and engineering. Special Report NSF 21-321. National Center for Science and Engineering Statistics.

54.

Newman

Wagner

Knokey

A.-M.

Marder

Nagle

Shaver

Wei

Cameto

Contreras

Ferguson

Greene

Schwarting

(2011). The post-high school outcomes of young adults with disabilities up to 8-years after high school. A report from the national longitudinal transition study-2 (NLTS2) (NCSER 2011-3005). SRI International.

55.

Palmer

I.,

Carpenter-Hubin

(2020). Using predictive analytics in the real world: Practice scenarios for ethical decision-making. New Directions for Institutional Research, 2019(183), 47–50. https://doi.org/10.1002/ir.20312

56.

Panesar

S. S.

D’Souza

R. N.

Yeh

F.-C.

Fernandez-Miranda

J. C.

(2019). Machine learning versus logistic regression in a small, heterogeneous glioma database. World Neurosurgery X, 2, 100012. https://doi.org/10.1016/j.wnsx.2019.100012

57.

Plasman

J. S.,

Gottfried

M. A.

(2018). Applied STEM coursework, high school dropout rates, and students with learning disabilities. Educational Policy, 32(5), 664–696. https://doi.org/10.1177/0895904816673738

58.

Porter

K. E.,

Balu

(2016). Predictive modeling of K-12 academic outcomes: A primer for researchers working with education data. MDRC.

59.

Rello

Baeza-Yates

Ali

Bigham

J. P.

Serra

(2020). Predicting risk of dyslexia with an online gamified test. PLoS One, 15(12), e0241687. https://doi.org/10.1371/journal.pone.0241687

60.

Schildkamp

(2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61(3), 257–273. https://doi.org/10.1080/00131881.2019.1625716

61.

Sprunger

N. S.

Harvey

M. W.

Quick

M. M.

(2018). Special education transition predictors for post-school success: Findings from the field. Preventing School Failure, 62(2), 116–128. https://doi.org/10.1080/1045988X.2017.1393789

62.

StataCorp (2021). Stata statistical software version 17. [Computer Software].

63.

STEM Education Act (2015). Public law (P.L.) 114–59, 42 U.S.C. § 1861.

64.

Stone

J. V.

(2013). Bayes’ rule: A tutorial introduction to Bayesian analysis. Sebtel Press.

65.

Test

D. W.

Mazzotti

V. L.

Mustian

A. L.

Fowler

C. H.

Kortering

Kohler

(2009). Evidence-based secondary transition predictors for improving postschool outcomes for students with disabilities. Career Development for Exceptional Individuals, 32(3), 160–181. https://doi.org/10.1177/0885728809346960

66.

U.S. Department of Labor – Office of Disability Employment Policy (2022). Disability statistics April 2022: Disability employment statistics ages 16 years and over. https://www.dol.gov/agencies/odep

67.

U.S. Department of Labor (2019). Disability statistics – November 2019. Office of Disability Employment Policy. https://www.dol.gov/agencies/odep

68.

Van de Schoot

Depaoli

King

Kramer

Martens

Tadesse

M. G.

Vannucci

Gelman

Veen

Willemsen

Yau

(2021). Bayesian statistics and modeling. Nature Reviews, 1(1), 1–26. https://doi.org/10.1038/s43586-020-00001-2

69.

Van Ravenzwaaij

Cassey

Brown

(2018). A simple introduction to Markov chain Monte-Carlo sampling. Psychonomic Bulletin & Review, 25(1), 143–154. https://doi.org/10.3758/s13423-016-1015-8

70.

Wagner

E.,

Longanacker

(2016). Scaling student success with predictive analytics: Reflections after four years in the data trenches. Change: The Magazine of Higher Learning, 48(1), 52–59. https://doi.org/10.1080/00091383.2016.1121087

71.

Wilcox

Fernandez-Conde

Kowbel

(2021). Using evidence-based practice and data-based decision making in inclusive education. Education Sciences, 11(3), 1–11. https://doi.org/10.3390/educsci11030129

72.

Yin

Shaewitz

Megra

(2014). An uneven playing field: The lack of equal pay for people with disabilities. American Institutes for Research.

73.

Yperman

Becker

Valkenborg

Popescu

Hellings

Van Wijmeersch

Peeters

L.M.

(2020). Machine learning analysis of motor evoked potential time series to predict disability progression in multiple sclerosis. BMC Neurology, 20(105), 1–15. https://doi.org/10.1186/s12883-020-01672-w

74.

Zhang

(2020). Predictive analytics in the era of big data: Opportunities and challenges. Annals of Translational Medicine, 8(4), 68–68. https://doi.org/10.21037/atm.2019.10.97