Mobilizing Developmental Education: The Causal Effect of Mobile App Courseware on the College Outcomes of Developmental Education Students

Abstract

Developmental education, in which college students deemed unprepared for college-level coursework enroll in non-credit-bearing courses, is widespread in American higher education. This study evaluates the effect of mobile app courseware on the college outcomes of developmental education students. We used a research design that randomly assigned course sections to receive access to the apps or not. The results show that access to the apps significantly improved student performance in developmental education outcomes and marginally improved medium-term college persistence and performance but did not affect credential attainment in the study timeframe. Despite a number of barriers to implementation, the results suggest the intervention has the potential to improve the short-term outcomes of developmental education students in addition to being low-cost and scalable.

Keywords

community colleges evaluation postsecondary education higher education experimental research focus group interviews

Get full access to this article

View all access options for this article.

References

Alrasheedi

Capretz

L. F.

Raza

(2015). A systematic review of the critical factors for success of mobile learning in higher education (university students’ perspective). Journal of Educational Computing Research, 52(2), 257–276.

Angrist

J. D.

Imbens

G. W.

Rubin

D. B.

(1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.

Angrist

J. D.

Pischke

J. S.

(2009). Mostly harmless econometrics: An empiricist’s companion. Princeton University Press.

Bahr

P. R.

(2008). Does mathematics remediation work? A comparative analysis of academic attainment among community college students. Research in Higher Education, 49(5), 420–450.

Bailey

Bashford

Boatman

Squires

Weiss

Doyle

Valentine

J. C.

LaSota

Polanin

J. R.

Spinney

Wilson

Yeide

Young

S. H.

(2017). Strategies for postsecondary students in developmental education—A practice guide for college and university administrators, advisors, and faculty. Institute of Education Sciences, What Works Clearinghouse.

Bailey

Jaggars

S. S.

(2016). When college students start behind. The Century Foundation. https://academiccommons.columbia.edu/doi/10.7916/D82B8ZHF

Bailey

Jeong

D. W.

Choo

(2008). Referral, enrollment, and completion in developmental education sequences in community colleges. Community College Research Center, Teachers College, Columbia University.

Barnett

E. A.

Bergman

Kopko

Reddy

Belfield

C. R.

Roy

(2018). Multiple measures placement using data analytics: An implementation and early impacts report. Center for the Analysis of Postsecondary Readiness, Community College Research Center, Teachers College, Columbia University. https://ccrc.tc.columbia.edu/media/k2/attachments/multiple-measures-placement-using-data-analytics.pdf

Belfied

Jenkins

Lahr

(2016). Is corequisite remediation cost-effective? Early findings from Tennessee (CCRC Research Brief No. 62). Community College Research Center, Teachers College, Columbia University. https://academiccommons.columbia.edu/doi/10.7916/D8HX1CNF

10.

Bersin

(2004). The blended learning book: Best practices, proven methodologies, and lessons learned. John Wiley.

11.

Bettinger

E. P.

Boatman

Long

B. T.

(2013). Student supports: Developmental education and other academic programs. The Future of Children, 23(1), 93–115.

12.

Bloom

H. S.

Bos

J. M.

Lee

(1999). Using cluster random assignment to measure program impacts: Statistical implications for the evaluation of education programs. Evaluation Review, 23(4), 445–469.

13.

Boulay

Goodson

Olsen

McCormick

Darrow

Frye

Gan

Harvill

Sarna

(2018). The investing in innovation fund: Summary of 67 evaluations (Final report. NCEE 2018-4013). National Center for Education Evaluation and Regional Assistance.

14.

Boykin

Prince

(2015). Inside ASAP: A resource guide on program structure, components, and management. City University of New York.

15.

Center for Community College Student Engagement. (2013). A matter of degrees: Engaging practices, engaging students (high-impact practices for community college student engagement). Community College Leadership Program, The University of Texas at Austin.

16.

Chen

deNoyelles

(2013, October). Exploring students’ mobile learning practices in higher education. Educause Review. https://er.educause.edu/articles/2013/10/exploring-students-mobile-learning-practices-in-higher-education

17.

Chen

Caves

L. R.

Pretlow

Caperton

S. A.

Bryan

Cooney

(2020). Courses taken, credits earned, and time to degree: A first look at the postsecondary transcripts of 2011–12 beginning postsecondary students (NCES 2020-501). National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2020501

18.

Chen

Simone

(2016). Remedial coursetaking at U.S. public 2- and 4-year institutions: Scope, experiences, and outcomes. National Center for Education Statistics, U.S. Department of Education. https://nces.ed.gov/pubs2016/2016405.pdf

19.

Crompton

Burke

(2018). The use of mobile learning in higher education: A systematic review. Computers & Education, 123, 53–64.

20.

Edgecombe

N. D.

Cormier

M. S.

Bickerstaff

S. E.

Barragan

(2013). Strengthening developmental education reforms: Evidence on implementation efforts from the scaling innovation project (CCRC Working Paper No. 61). Columbia University. https://doi.org/10.7916/D8QV3JJ3

21.

Education Commission of the States. (2021). 50-state comparison: Developmental education policies. https://www.ecs.org/50-state-comparison-developmental-education-policies/

22.

Fay

M. P.

Barnett

E. A.

Chavarín

(2017). How states are implementing transition curricula: Results from a national scan. Community College Research Center, Teachers College, Columbia University. https://ccrc.tc.columbia.edu/media/k2/attachments/ccrc-research-brief-how-states-implementing-transition-curricula-results-national-scan.pdf

23.

Ganga

Mazzariello

Edgecombe

(2018). Developmental education: An introduction for policymakers. Center for the Analysis of Postsecondary Readiness, Community College Research Center, Teachers College, Columbia University. https://ccrc.tc.columbia.edu/media/k2/attachments/developmental-education-introduction-policymakers.pdf

24.

Garrison

D. R.

Kanuka

(2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7, 95–105.

25.

Giani

M. S.

(2016). Bossier Parrish Community College First in the World evaluation: Baseline data analysis. Giani Consulting & Evaluation.

26.

Graham

C. R.

Woodfield

Harrison

J. B.

(2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4–14.

27.

Gupta

S. K.

(2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2(3), 109–112. https://dx.doi.org/10.4103%2F2229-3485.83221

28.

Hedges

L. V.

Hedberg

E. C.

(2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87.

29.

Hellevik

(2009). Linear versus logistic regression when the dependent variable is a dichotomy. Quality & Quantity, 43(1), 59–74.

30.

Hodara

Jaggars

S. S.

(2014). An examination of the impact of accelerating community college students’ progression through developmental education. The Journal of Higher Education, 85(2), 246–276. https://doi.org/10.1080/00221546.2014.11777326

31.

Hodara

Jaggars

S. S.

Karp

M. J. M.

(2012). Improving developmental education assessment and placement: Lessons from community colleges across the country (CCRC Working Paper No. 51). Community College Research Center, Teachers College, Columbia University. https://academiccommons.columbia.edu/doi/10.7916/D8SB4F49

32.

Hughes

K. L.

Scott-Clayton

J. E.

(2011). Assessing developmental assessment in community colleges (CCRC Working Paper No. 19). Community College Research Center, Teachers College, Columbia University. https://files.eric.ed.gov/fulltext/ED516079.pdf

33.

Hurwitz

Smith

Niu

Howell

(2015). The Maine question: How is 4-year college enrollment affected by mandatory college entrance exams? Educational Evaluation and Policy Analysis, 37(1), 138–159. https://doi.org/10.3102/0162373714521866

34.

International Telecommunications Union. (2016). ICT facts and figures 2016. http://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2016.pdf

35.

Jackson

Kurlaender

(2016). K-12 postsecondary alignment and school accountability: Investigating high school responses to California’s Early Assessment Program. American Journal of Education, 122(4), 477–503. https://doi.org/10.1086/687273

36.

Jaggars

S. S.

Hodara

Cho

S-W.

(2015). Three accelerated developmental education programs: Features, student outcomes, and implications. Community College Review, 43(1), 3–26. https://doi.org/10.1177/0091552114551752

37.

Kaliisa

Picard

(2017). A systematic review on mobile-learning in higher education: The African perspective. The Turkish Online Journal of Educational Technology, 16(1), 1–13.

38.

Kane

T. J.

Boatman

Kozakowski

Bennett

Hitch

Weisenfeld

(2021). Is college remediation a barrier or a boost? Evidence from the Tennessee SAILS program. Journal of Policy Analysis and Management. Advance online publication. https://doi.org/10.1002/pam.22306

39.

Kapp

K. M.

(2012). The gamification of learning and instruction: Game-based methods and strategies for training and education. John Wiley.

40.

Kraft

M. A.

(2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253.

41.

Kulik

J. A.

Cohen

P. A.

Ebeling

B. J.

(1980). Effectiveness of programmed instruction in higher education: A meta-analysis of findings. Educational Evaluation and Policy Analysis, 2(6), 51–64.

42.

Kurlaender

Lusher

Case

(2020). Is early start a better start? Evaluation California state university’s early start remediation policy. Journal of Policy Analysis and Management, 39(2), 348–375. https://doi.org/10.1002/pam.22198

43.

Logue

A. W.

Douglas

Watanabe-Rose

(2019). Corequisite mathematics remediation: Results over time and in different contexts. Educational Evaluation and Policy Analysis, 41(3), 294–315.

44.

Lortie-Forgues

Inglis

(2019). Rigorous large-scale RCTs are often uninformative: Should we be concerned? Educational Researcher, 48(3), 158–166.

45.

Miller

Headlam

Manno

Cullinan

(2020). Increasing community college graduation rates with a proven model. https://www.mdrc.org/sites/default/files/ASAP_OH_3yr_Impact_Report_1.pdf

46.

Mokher

C. G.

Leeds

D. M.

Harris

J. C.

(2018). Adding it up: How the Florida College and Career Readiness Initiative impacted developmental education. Educational Evaluation and Policy Analysis, 40(2), 219–242. https://doi.org/10.3102/0162373717744891

47.

Murray

D. M.

(1998). Design and analysis of group-randomized trials. Oxford University Press Inc.

48.

Murray

D. M.

Varnell

S. P.

Blitstein

J. L.

(2004). Design and analysis of group-randomized trials: Review of recent methodological developments. American Journal of Public Health, 94(3), 423–432.

49.

Nodine

Dadgar

Venezia

Bracco

K. R.

(2013). Acceleration in developmental education: Game changers series. WestEd. https://www.wested.org/resources/acceleration-in-developmental-education-game-changers-series/

50.

Pew Research Center. (2019). Mobile fact sheet. http://www.pewinternet.org/fact-sheet/mobile/

51.

Pheatt

L. E.

Trimble

M. J.

Barnett

(2016). Improving the transition to college: Estimating the impact of high school transition courses on short-term college outcomes (CCRC Working Papers No. 86). Community College Research Center, Teachers College, Columbia University. https://academiccommons.columbia.edu/doi/10.7916/D8SB45QM

52.

Pimmer

Mateescu

Grohbiel

(2016). Mobile and ubiquitous learning in higher education settings: A systematic review of empirical studies. Computers in Human Behavior, 63, 490–501.

53.

Puma

M. J.

Olsen

R. B.

Bell

S. H.

Price

(2009). What to do when data are missing in group randomized controlled trials. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences.

54.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). SAGE.

55.

Roediger

H. L.

III Butler

A. C.

(2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. https://doi.org/10.1016/j.tics.2010.09.003

56.

St. Amour

(2020, June 30). Looming budget cuts threaten proven program. Inside Higher Ed. https://www.insidehighered.com/news/2020/06/30/experts-worry-proposed-cuts-cuny-asap-foreshadow-trend-higher-ed

57.

Scott-Clayton

J. E.

(2012). Do high-stakes placement exams predict college success? (CCRC Working Paper No. 41). Community College Research Center, Teachers College, Columbia University. https://files.eric.ed.gov/fulltext/ED529866.pdf

58.

Scrivener

Weiss

M. J.

Ratledge

Rudd

Sommo

Fresques

(2015). Doubling graduation rates: Three-year effects of CUNY’s Accelerated Study in Associate Programs (ASAP) for developmental education students. MDRC. https://www.mdrc.org/sites/default/files/doubling_graduation_rates_fr.pdf

59.

Sedlacek

W. E.

(2004). Beyond the big test: Noncognitive assessment in higher education. Jossey-Bass.

60.

Skinner

B. F.

(1954). The science of learning and the art of teaching. Harvard Educational Review, 24, 86–97.

61.

Sommo

Cullinan

Manno

Blake

Alonzo

(2018, December). Doubling graduation rates in a new state: Two-year findings from the ASAP Ohio demonstration [MDRC, Policy Brief]. https://ssrn.com/abstract=3353494

62.

Sommo

Mayer

Rudd

Cullinan

(2012). Commencement day: Six-year effects of a freshman learning community program at Kingsborough Community College. MDRC. https://www.mdrc.org/sites/default/files/Commencement%20Day%20FR.pdf

63.

Thorndike

E. L.

(1912). Education: A first book. Macmillan.

64.

Thorne

(2003). Blended learning: How to integrate online and traditional learning. Kogan Page.

65.

Valentine

J. C.

Konstantopoulos

Goldrick-Rab

(2017). What happens to students placed into developmental education? A meta-analysis of regression discontinuity studies. Review of Educational Research, 87(4), 806–833.

66.

Venezia

Hughes

K. L.

(2013). Acceleration strategies in the new developmental education landscape. New Directions for Community Colleges, 164, 37–45.

67.

Visher

M. G.

Weiss

M. J.

Weissman

Rudd

Wathington

H. D.

(2012). The effects of learning communities for students in developmental education: A synthesis of findings from six community colleges. National Center for Postsecondary Research. https://files.eric.ed.gov/fulltext/ED533825.pdf

68.

Wachen

Jenkins

Belfield

Van Noy

(2012). Contextualized college transition strategies for adult basic skills students: Learning from Washington State’s I-BEST program model. Community College Research Center, Teachers College, Columbia University.

69.

What Works Clearinghouse. (2020). Procedures handbook, Version 4.1. Institute of Education Sciences. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Procedures-Handbook-v4-1-508.pdf

70.

Jaggars

S. S.

(2011). The effectiveness of distance education across Virginia’s community colleges: Evidence from introductory college-level math and English courses. Educational Evaluation and Policy Analysis, 33(3), 360–377. https://doi.org/10.3102/0162373711413814

71.

Young

D. B.

Ley

(2003). Self-regulation support offered by developmental educators. Journal of Developmental Education, 27(2), 2–4, 6, 8, 10.

72.

Zachry

Schneider

(2011). Unlocking the gate: What we know about improving developmental education. MDRC. https://www.mdrc.org/sites/default/files/full_595.pdf

73.

Zeidenberg

Cho

Jenkins

(2010). Washington State’s Integrated Basic Education and Skills Training program (I-BEST): New evidence of effectiveness. Community College Research Center, Teachers College, Columbia University. https://doi.org/10.7916/D8KD262K

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.27 MB