Developmental education, in which college students deemed unprepared for college-level coursework enroll in non-credit-bearing courses, is widespread in American higher education. This study evaluates the effect of mobile app courseware on the college outcomes of developmental education students. We used a research design that randomly assigned course sections to receive access to the apps or not. The results show that access to the apps significantly improved student performance in developmental education outcomes and marginally improved medium-term college persistence and performance but did not affect credential attainment in the study timeframe. Despite a number of barriers to implementation, the results suggest the intervention has the potential to improve the short-term outcomes of developmental education students in addition to being low-cost and scalable.
AlrasheediM.CapretzL. F.RazaA. (2015). A systematic review of the critical factors for success of mobile learning in higher education (university students’ perspective). Journal of Educational Computing Research, 52(2), 257–276.
2.
AngristJ. D.ImbensG. W.RubinD. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.
3.
AngristJ. D.PischkeJ. S. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton University Press.
4.
BahrP. R. (2008). Does mathematics remediation work? A comparative analysis of academic attainment among community college students. Research in Higher Education, 49(5), 420–450.
5.
BaileyT.BashfordJ.BoatmanA.SquiresJ.WeissM.DoyleW.ValentineJ. C.LaSotaR.PolaninJ. R.SpinneyE.WilsonW.YeideM.YoungS. H. (2017). Strategies for postsecondary students in developmental education—A practice guide for college and university administrators, advisors, and faculty. Institute of Education Sciences, What Works Clearinghouse.
BaileyT.JeongD. W.ChooS. (2008). Referral, enrollment, and completion in developmental education sequences in community colleges. Community College Research Center, Teachers College, Columbia University.
BelfiedC.JenkinsD.LahrH. (2016). Is corequisite remediation cost-effective? Early findings from Tennessee (CCRC Research Brief No. 62). Community College Research Center, Teachers College, Columbia University. https://academiccommons.columbia.edu/doi/10.7916/D8HX1CNF
10.
BersinJ. (2004). The blended learning book: Best practices, proven methodologies, and lessons learned. John Wiley.
11.
BettingerE. P.BoatmanA.LongB. T. (2013). Student supports: Developmental education and other academic programs. The Future of Children, 23(1), 93–115.
12.
BloomH. S.BosJ. M.LeeS. (1999). Using cluster random assignment to measure program impacts: Statistical implications for the evaluation of education programs. Evaluation Review, 23(4), 445–469.
13.
BoulayB.GoodsonB.OlsenR.McCormickR.DarrowC.FryeM.GanK.HarvillE.SarnaM. (2018). The investing in innovation fund: Summary of 67 evaluations (Final report. NCEE 2018-4013). National Center for Education Evaluation and Regional Assistance.
14.
BoykinD.PrinceA. (2015). Inside ASAP: A resource guide on program structure, components, and management. City University of New York.
15.
Center for Community College Student Engagement. (2013). A matter of degrees: Engaging practices, engaging students (high-impact practices for community college student engagement). Community College Leadership Program, The University of Texas at Austin.
ChenX.CavesL. R.PretlowJ.CapertonS. A.BryanM.CooneyD. (2020). Courses taken, credits earned, and time to degree: A first look at the postsecondary transcripts of 2011–12 beginning postsecondary students (NCES 2020-501). National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2020501
18.
ChenX.SimoneS. (2016). Remedial coursetaking at U.S. public 2- and 4-year institutions: Scope, experiences, and outcomes. National Center for Education Statistics, U.S. Department of Education. https://nces.ed.gov/pubs2016/2016405.pdf
19.
CromptonH.BurkeD. (2018). The use of mobile learning in higher education: A systematic review. Computers & Education, 123, 53–64.
20.
EdgecombeN. D.CormierM. S.BickerstaffS. E.BarraganM. (2013). Strengthening developmental education reforms: Evidence on implementation efforts from the scaling innovation project (CCRC Working Paper No. 61). Columbia University. https://doi.org/10.7916/D8QV3JJ3
GarrisonD. R.KanukaH. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7, 95–105.
25.
GianiM. S. (2016). Bossier Parrish Community College First in the World evaluation: Baseline data analysis. Giani Consulting & Evaluation.
26.
GrahamC. R.WoodfieldW.HarrisonJ. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4–14.
HedgesL. V.HedbergE. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87.
29.
HellevikO. (2009). Linear versus logistic regression when the dependent variable is a dichotomy. Quality & Quantity, 43(1), 59–74.
30.
HodaraM.JaggarsS. S. (2014). An examination of the impact of accelerating community college students’ progression through developmental education. The Journal of Higher Education, 85(2), 246–276. https://doi.org/10.1080/00221546.2014.11777326
31.
HodaraM.JaggarsS. S.KarpM. J. M. (2012). Improving developmental education assessment and placement: Lessons from community colleges across the country (CCRC Working Paper No. 51). Community College Research Center, Teachers College, Columbia University. https://academiccommons.columbia.edu/doi/10.7916/D8SB4F49
32.
HughesK. L.Scott-ClaytonJ. E. (2011). Assessing developmental assessment in community colleges (CCRC Working Paper No. 19). Community College Research Center, Teachers College, Columbia University. https://files.eric.ed.gov/fulltext/ED516079.pdf
33.
HurwitzM.SmithJ.NiuS.HowellJ. (2015). The Maine question: How is 4-year college enrollment affected by mandatory college entrance exams?Educational Evaluation and Policy Analysis, 37(1), 138–159. https://doi.org/10.3102/0162373714521866
JacksonJ.KurlaenderM. (2016). K-12 postsecondary alignment and school accountability: Investigating high school responses to California’s Early Assessment Program. American Journal of Education, 122(4), 477–503. https://doi.org/10.1086/687273
36.
JaggarsS. S.HodaraM.ChoS-W.XuD. (2015). Three accelerated developmental education programs: Features, student outcomes, and implications. Community College Review, 43(1), 3–26. https://doi.org/10.1177/0091552114551752
37.
KaliisaR.PicardM. (2017). A systematic review on mobile-learning in higher education: The African perspective. The Turkish Online Journal of Educational Technology, 16(1), 1–13.
38.
KaneT. J.BoatmanA.KozakowskiW.BennettC.HitchR.WeisenfeldD. (2021). Is college remediation a barrier or a boost? Evidence from the Tennessee SAILS program. Journal of Policy Analysis and Management. Advance online publication. https://doi.org/10.1002/pam.22306
39.
KappK. M. (2012). The gamification of learning and instruction: Game-based methods and strategies for training and education. John Wiley.
40.
KraftM. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253.
41.
KulikJ. A.CohenP. A.EbelingB. J. (1980). Effectiveness of programmed instruction in higher education: A meta-analysis of findings. Educational Evaluation and Policy Analysis, 2(6), 51–64.
42.
KurlaenderM.LusherL.CaseM. (2020). Is early start a better start? Evaluation California state university’s early start remediation policy. Journal of Policy Analysis and Management, 39(2), 348–375. https://doi.org/10.1002/pam.22198
43.
LogueA. W.DouglasD.Watanabe-RoseM. (2019). Corequisite mathematics remediation: Results over time and in different contexts. Educational Evaluation and Policy Analysis, 41(3), 294–315.
44.
Lortie-ForguesH.InglisM. (2019). Rigorous large-scale RCTs are often uninformative: Should we be concerned?Educational Researcher, 48(3), 158–166.
MokherC. G.LeedsD. M.HarrisJ. C. (2018). Adding it up: How the Florida College and Career Readiness Initiative impacted developmental education. Educational Evaluation and Policy Analysis, 40(2), 219–242. https://doi.org/10.3102/0162373717744891
47.
MurrayD. M. (1998). Design and analysis of group-randomized trials. Oxford University Press Inc.
48.
MurrayD. M.VarnellS. P.BlitsteinJ. L. (2004). Design and analysis of group-randomized trials: Review of recent methodological developments. American Journal of Public Health, 94(3), 423–432.
PheattL. E.TrimbleM. J.BarnettE. (2016). Improving the transition to college: Estimating the impact of high school transition courses on short-term college outcomes (CCRC Working Papers No. 86). Community College Research Center, Teachers College, Columbia University. https://academiccommons.columbia.edu/doi/10.7916/D8SB45QM
52.
PimmerC.MateescuM.GrohbielU. (2016). Mobile and ubiquitous learning in higher education settings: A systematic review of empirical studies. Computers in Human Behavior, 63, 490–501.
53.
PumaM. J.OlsenR. B.BellS. H.PriceC. (2009). What to do when data are missing in group randomized controlled trials. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences.
54.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). SAGE.
55.
RoedigerH. L.IIIButlerA. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. https://doi.org/10.1016/j.tics.2010.09.003
Scott-ClaytonJ. E. (2012). Do high-stakes placement exams predict college success? (CCRC Working Paper No. 41). Community College Research Center, Teachers College, Columbia University. https://files.eric.ed.gov/fulltext/ED529866.pdf
SedlacekW. E. (2004). Beyond the big test: Noncognitive assessment in higher education. Jossey-Bass.
60.
SkinnerB. F. (1954). The science of learning and the art of teaching. Harvard Educational Review, 24, 86–97.
61.
SommoC.CullinanD.MannoM.BlakeS.AlonzoE. (2018, December). Doubling graduation rates in a new state: Two-year findings from the ASAP Ohio demonstration [MDRC, Policy Brief]. https://ssrn.com/abstract=3353494
ThorndikeE. L. (1912). Education: A first book. Macmillan.
64.
ThorneK. (2003). Blended learning: How to integrate online and traditional learning. Kogan Page.
65.
ValentineJ. C.KonstantopoulosS.Goldrick-RabS. (2017). What happens to students placed into developmental education? A meta-analysis of regression discontinuity studies. Review of Educational Research, 87(4), 806–833.
66.
VeneziaA.HughesK. L. (2013). Acceleration strategies in the new developmental education landscape. New Directions for Community Colleges, 164, 37–45.
67.
VisherM. G.WeissM. J.WeissmanE.RuddR.WathingtonH. D. (2012). The effects of learning communities for students in developmental education: A synthesis of findings from six community colleges. National Center for Postsecondary Research. https://files.eric.ed.gov/fulltext/ED533825.pdf
68.
WachenJ.JenkinsD.BelfieldC.Van NoyM. (2012). Contextualized college transition strategies for adult basic skills students: Learning from Washington State’s I-BEST program model. Community College Research Center, Teachers College, Columbia University.
XuD.JaggarsS. S. (2011). The effectiveness of distance education across Virginia’s community colleges: Evidence from introductory college-level math and English courses. Educational Evaluation and Policy Analysis, 33(3), 360–377. https://doi.org/10.3102/0162373711413814
71.
YoungD. B.LeyK. (2003). Self-regulation support offered by developmental educators. Journal of Developmental Education, 27(2), 2–4, 6, 8, 10.
ZeidenbergM.ChoS.JenkinsD. (2010). Washington State’s Integrated Basic Education and Skills Training program (I-BEST): New evidence of effectiveness. Community College Research Center, Teachers College, Columbia University. https://doi.org/10.7916/D8KD262K
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
