Identifying Critical Factors When Predicting Remedial Mathematics Completion Rates

Abstract

Completion of remedial mathematics has been identified as one of the keys to college success. However, completion rates in remedial mathematics have been low and are of much debate across America. This study leverages machine learning techniques in trying to predict and understand completion rates in remedial mathematics. The purpose of this study is to build machine learning models that can predict students that are least likely to complete remedial mathematics and identify which factors are most influential when computing those predictions. The study discovers random forest as the highest performing model. Furthermore, the study reveals that the remedial course a student begins with, credit completion rate, math placement score, and high school G.P.A as the most influential predictors of completion rates. The study also offers future research directions, especially in how to improve the performance of the machine learning models.

Keywords

mathematics education mathematics remediation machine learning

Get full access to this article

View all access options for this article.

References

Meng

(2010, August). Voting-averaged combination method for regressor ensemble. In International Conference on Intelligent Computing (pp. 540–546). Springer, Berlin, Heidelberg.

Bahr

P. R.

(2008). Does mathematics remediation work?: A comparative analysis of academic attainment among community college students. Research in Higher Education, 49(5), 420–450. https://doi.org/10.1007/s11162-008-9089-4

Bahr

P. R

. (2010). Making sense of disparities in mathematics remediation: What is the role of student retention?. Journal of College Student Retention: Research, Theory & Practice, 12(1), 25–49.

Bahr

P. R.

(2011). A typology of students’ use of the community college. New Directions for Institutional Research, 2011(S1), 33–48. https://doi.org/10.1002/ir.415

Bahr

P. R.

(2012). Deconstructing remediation in community colleges: Exploring associations between course-taking patterns, course outcomes, and attrition from the remedial math and remedial writing sequences. Research in Higher Education, 53(6), 661–693. https://doi.org/10.1007/s11162-011-9243-2

Bahr

P. R.

(2013). The aftermath of remedial math: Investigating the low rate of certificate completion among remedial math students. Research in Higher Education, 54(2), 171–200. https://doi.org/10.1007/s11162-012-9281-4

Bailey

(2009). Challenge and opportunity: Rethinking the role and function of developmental education in community college. New Directions for Community Colleges, 2009(145), 11–30. https://doi.org/10.1002/cc.352

Bailey

Jeong

D. W.

Cho

S. W.

(2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29(2), 255–270. https://doi.org/10.1016/j.econedurev.2009.09.002

Bettinger

E. P.

Long

B. T.

(2009). Addressing the needs of underprepared students in higher education does college remediation work? Journal of Human Resources, 44(3), 736–771. https://doi.org/10.1353/jhr.2009.0033

10.

Bonaccorso

(2018). Machine learning algorithms: Popular algorithms for data science and machine learning. Packt Publishing Ltd.

11.

Breiman

(2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324

12.

Breneman

D. W.

Haarlow

W. N.

(1998). Remedial education: Costs and consequences. Fordham Report, 2(9), 1–22.

13.

Bryk

A. S.

Treisman

(2010). Make math a gateway, not a gatekeeper. Chronicle of Higher Education, 56(32), B19–B20.

14.

Burges

C. J.

(1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167. https://doi.org/10.1023/A:1009715923555

15.

Cabrera

A. F.

Nora

Castaneda

M. B

. (1993). College persistence: Structural equations modeling test of an integrated model of student retention. The Journal of Higher Education, 64(2), 123–139.

16.

Caison

A. L.

(2007). Analysis of institutionally specific retention research: A comparison between survey and institutional database methods. Research in Higher Education, 48(4), 435–451. https://doi.org/10.1007/s11162-006-9032-5

17.

Calcagno

J. C.

Crosta

Bailey

Jenkins

(2007). Stepping stones to a degree: The impact of enrollment pathways and milestones on community college student outcomes. Research in Higher Education, 48(7), 775–801. https://doi.org/10.1007/s11162-007-9053-8

18.

Calcagno

J. C.

Long

B. T.

(2008). The impact of postsecondary remediation using a regression discontinuity approach: Addressing endogenous sorting and noncompliance (No. w14194). National Bureau of Economic Research.

19.

Cawley

G. C.

Talbot

N. L

. (2010). On over-fitting in model selection and subsequent selection bias in performance evaluation. The Journal of Machine Learning Research, 11, 2079–2107.

20.

Çelik

Osmanoglu

U. O.

(2019). Comparing to techniques used in customer churn analysis. Journal of Multidisciplinary Developments, 4(1), 30–38.

21.

Chai

K. E.

Gibson

(2015). Predicting the risk of attrition for undergraduate students with time based modelling. International Association for Development of the Information Society.

22.

Chawla

N. V.

Bowyer

K. W.

Hall

L. O.

Kegelmeyer

W. P.

(2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16(3), 321–357. https://doi.org/10.1613/jair.953

23.

Chen

(2016). Remedial coursetaking at US public 2-and 4-year institutions: Scope, experiences, and outcomes. Statistical Analysis Report. NCES 2016–405. National Center for Education Statistics.

24.

Clotfelter

C. T.

Ladd

H. F.

Vigdor

J. L.

(2010). Teacher credentials and student achievement in high school a cross-subject analysis with student fixed effects. Journal of Human Resources, 45(3), 655–681. https://doi.org/10.1353/jhr.2010.0023

25.

Collins

F. S.

Varmus

. (2015). A new initiative on precision medicine. New England Journal of Medicine, 372(9), 793–795.

26.

Cortez

Cerdeira

Almeida

Matos

Reis

(2009). Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47(4), 547–553. https://doi.org/10.1016/j.dss.2009.05.016

27.

Cortez

Embrechts

M. J.

(2011, April). Opening black box data mining models using sensitivity analysis. In 2011 IEEE Symposium on Computational Intelligence and Data Mining (CIDM) (pp. 341–348). IEEE.

28.

Đambić

Krajcar

Bele

(2016). Machine learning model for early detection of higher education students that need additional attention in introductory programming courses. International Journal of Digital Technology & Economy, 1(1), 1–11. https://doi.org/10.31785/ijdte.4.1.1

29.

DeAngelo

Franke

Hurtado

Pryor

J. H.

Tran

(2011). Completing college: Assessing graduation rates at four-year institutions.

30.

(2010). A comparative analysis of machine learning techniques for student retention management. Decision Support Systems, 49(4), 498–506. https://doi.org/10.1016/j.dss.2010.06.003

31.

. (2011). Predicting student attrition with data mining methods. Journal of College Student Retention: Research, Theory & Practice, 13(1), 17–35.

32.

Embrechts

M. J.

Bennett

K. P.

Breneman

(2002). www.drugmining.com

33.

Essa

Ayad

(2012, April). Student success system: Risk analytics and data visualization using ensembles of predictive models. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 158–161).

34.

Fay

M. P.

(2020). Learning to fail? Student experiences in remedial mathematics in community colleges (Doctoral dissertation). City University of New York.

35.

Friedman

J. H.

(2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189–1232.

36.

Gansemer-Topf

A. M.

Schuh

J. H.

(2006). Institutional selectivity and institutional expenditures: Examining organizational factors that contribute to retention and graduation. Research in Higher Education, 47(6), 613–642. https://doi.org/10.1007/s11162-006-9009-4

37.

Greene

R. A.

(2018). The effect of modality on student achievement and course completion in a developmental mathematics course.

38.

Hern

Snell

(2010). Exponential attrition and the promise of acceleration in developmental English and math. Research and Planning Group of the California Community Colleges. www.rpgroup.org/rpsearch/results/taxonomy% 3A17

39.

Hosch

B. J.

(2008, November). Institutional and student characteristics that predict graduation and retention rates. Paper presented at the Annual Meeting of the North East Association for Institutional Research. Retrieved from http://www.ccsu.edu/uploaded/departments/AdministrativeDepartments/Institutional_Research_and_Assessment/Research/20081104a.pdf

40.

Howell

Walkington

(2022). Factors associated with completion: Pathways through developmental mathematics. Journal of College Student Retention: Research, Theory & Practice, 24(1), 43–78. https://doi.org/10.1177/1521025119900985

41.

Hursh

(2007). Assessing No Child Left Behind and the rise of neoliberal education policies. American Educational Research Journal, 44(3), 493–516. https://doi.org/10.3102/0002831207306764

42.

Jaggars

S. S.

Hodara

Cho

S. W.

. (2015). Three accelerated developmental education programs: Features, student outcomes, and implications. Community College Review, 43(1), 3–26.

43.

Jia

J. W.

Mareboyana

(2014). Predictive models for undergraduate student retention using machine learning algorithms. In Transactions on engineering technologies (pp. 315–329). Springer.

44.

Jimenez

Sargrad

Morales

Thompson

(2016). Remedial education: The cost of catching up. Center for American Progress.

45.

Joseph

H. R.

(2014, September). Promoting education: A state of the art machine learning framework for feedback and monitoring E-Learning impact. In 2014 IEEE Global Humanitarian Technology Conference-South Asia Satellite (GHTC-SAS) (pp. 251–254). IEEE.

46.

Khazanov

(2011). Mentoring at-risk students in a remedial mathematics course. Mathematics & Computer Education, 45(2).

47.

King

J. B.

McIntosh

Bell-Ellwanger

Schak

Metzger

Bass

English

(2017). Developmental education: Challenges and strategies for reform. U.S. Department of Education, Office of Planning, Evaluation and Policy Development. https://www2.ed.gov/ about/offices/list/opepd/education-strategies.pdf

48.

Kohavi

(1995, August). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai (Vol. 14, No. 2, pp. 1137–1145).

49.

Kotsiantis

S. B.

Pierrakeas

C. J.

Pintelas

P. E

(2003, September). Preventing student dropout in distance learning using machine learning techniques. In International Conference on Knowledge-based and Intelligent Information and Engineering Systems (pp. 267–274). Berlin, Heidelberg: Springer.

50.

Kučak

Juričić

Đambić

(2018). Machine learning in education-a survey of current research trends. Annals of DAAAM & Proceedings, 29.

51.

Kumar

Singh

A. J.

Handa

(2017). Literature survey on educational dropout prediction. International Journal of Education and Management Engineering, 7(2), 8. https://doi.org/10.5815/ijeme.2017.02.02

52.

Sun

(2012). Forecasting business failure: The use of nearest-neighbour support vectors and correcting imbalanced samples – evidence from the Chinese hotel industry. Tourism Management, 33(3), 622–634. https://doi.org/10.1016/j.tourman.2011.07.004

53.

Long

R. G.

(2008). The crux of the method: Assumptions in ordinary least squares and logistic regression. Psychological Reports, 103(2), 431–434. https://doi.org/10.2466/pr0.103.2.431-434

54.

O. H.

Huang

A. Y.

Huang

J. C.

Lin

A. J.

Ogata

Yang

S. J.

(2018). Applying learning analytics for the early prediction of Students’ academic performance in blended learning. Journal of Educational Technology & Society, 21(2), 220–232.

55.

Luan

Tsai

C. C

. (2021). A review of using machine learning approaches for precision education. Educational Technology & Society, 24(1), 250–266.

56.

Lykourentzou

Giannoukos

Nikolopoulos

Mpardis

Loumos

(2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53(3), 950–965. https://doi.org/10.1016/j.compedu.2009.05.010

57.

Merseth

K. K.

(2011). Update: Report on innovations in developmental mathematics--moving mathematical graveyards. Journal of Developmental Education, 34(3), 32.

58.

Mitchell

T. M.

(1997). Artificial neural networks. Machine Learning, 45, 81–127.

59.

Mokhtar

M. R.

Shamsudin

M. F.

(2020). How to keep up with customer? Journal of Postgraduate Current Business Research, 5(1).

60.

Molinaro

A. M.

Simon

Pfeiffer

R. M.

(2005). Prediction error estimation: A comparison of resampling methods. Bioinformatics (Oxford, England), 21(15), 3301–3307. https://doi.org/10.1093/bioinformatics/bti499

61.

Nouri

Larsson

Saqr

(2019, September). Identifying factors for master thesis completion and non-completion through learning analytics and machine learning. In European Conference on Technology Enhanced Learning (pp. 28–39). Springer, Cham.

62.

Pandya

(2015). C5. 0 algorithm to improved decision tree with feature selection and reduced error pruning. International Journal of Computer Applications, 117(16), 18–21. https://doi.org/10.5120/20639-3318

63.

Patro

Sahu

K. K.

(2015). Normalization: A preprocessing stage. arXiv preprint arXiv:1503.06462.

64.

Porter

K. B

. (2008). Current trends in student retention: A literature review. Teaching and Learning in Nursing, 3(1), 3–5.

65.

Potdar

Pardawala

T. S.

Pai

C. D.

(2017). A comparative study of categorical variable encoding techniques for neural network classifiers. International Journal of Computer Applications, 175(4), 7–9. https://doi.org/10.5120/ijca2017915495

66.

Reinheimer

McKenzie

(2011). The impact of tutoring on the academic success of undeclared students. Journal of College Reading and Learning, 41(2), 22–36. https://doi.org/10.1080/10790195.2011.10850340

67.

Ruck

D. W.

Rogers

S. K.

Kabrisky

(1990). Feature selection using a multilayer perceptron. Journal of Neural Network Computing, 2(2), 40–48.

68.

Saltelli

Tarantola

Campolongo

Ratto

(2004). Sensitivity analysis in practice: A guide to assessing scientific models (Vol. 1). Wiley.

69.

Samuel

A. L

. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3, 210–229.

70.

Sanford

Hunter

J. M.

(2011). Impact of performance funding on retention and graduation rates. Education Policy Analysis Archives, 19, 33. https://doi.org/10.14507/epaa.v19n33.2011

71.

Scott-Clayton

Rodriguez

(2015). Development, discouragement, or diversion? New evidence on the effects of college remediation policy. Education Finance and Policy, 10(1), 4–45. https://doi.org/10.1162/EDFP_a_00150

72.

Shahiri

A. M.

Husain

(2015). A review on predicting student's performance using data mining techniques. Procedia Computer Science, 72, 414–422. https://doi.org/10.1016/j.procs.2015.12.157

73.

Siwatu

K. O.

(2011). Preservice teachers’ sense of preparedness and self-efficacy to teach in America’s urban and suburban schools: Does context matter? Teaching and Teacher Education, 27(2), 357–365. https://doi.org/10.1016/j.tate.2010.09.004

74.

Sparks

Malkus

(2013). First-Year Undergraduate Remedial Coursetaking: 1999-2000, 2003-04, 2007-08. Statistics in Brief. NCES 2013-013. National Center for Education Statistics.

75.

The White House

Office of the Press Secretary. FACT SHEET: President Obama’s Precision Medicine Initiative

. (2015). Retrieved December 23, 2021, from https://www.whitehouse.gov/the-press-office/2015/01/30/fact-sheet-president-obama-s-precisionmedicine- initiative.

76.

Tinto

(1993). Leaving college: Rethinking the cause and cures of student attrition (2nd ed.). Chicago: University of Chicago.

77.

Tsai

S. C.

Chen

C. H.

Shiao

Y. T.

Ciou

J. S.

T. N.

(2020). Precision education with statistical learning and deep learning: A case study in Taiwan. International Journal of Educational Technology in Higher Education, 17(1), 1–13. https://doi.org/10.1186/s41239-019-0174-x

78.

Weiss

S. M.

Kulikowski

C. A.

(1991). Computer systems that learn: Classification and prediction methods from statistics, neural nets, machine learning, and expert systems. Morgan Kaufmann Publishers Inc.

79.

Wickersham

Wang

. (2016). What’s life got to do with it? The role of life experiences in shaping female community college students’ transfer intent in STEM fields of study. Community College Journal of Research and Practice, 40(12), 1001–1012.

80.

Williams

(2011). A call for change: Narrowing the achievement gap between White and minority students. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 84(2), 65–71. https://doi.org/10.1080/00098655.2010.511308

81.

Wotaifi

T. A.

(2019). Mining of completion rate of higher education based on fuzzy feature selection model and machine learning techniques.

82.

Zhao

Cumming

M. T.

Leshchinskaya

Johnson

(2018). The effects of remedial math on retention and graduation rates. In 13th INTERNATIONAL SCIENTIFIC FORUM, ISF 2018 (p. 524).

83.

Zientek

L. R.

Lane

F. C.

Sechelski

Shupp

(2022). Effects of delaying college-level mathematics course enrollment for remedial students. Journal of College Student Retention: Research, Theory & Practice, 24(2), 474–498. https://doi.org/10.1177/1521025120923113