Using Learning Analytics to Predict At-Risk Students in Online Graduate Public Affairs and Administration Education

Abstract

In this global information age, schools that teach public affairs and administration must meet the needs of students. Increasingly, this means providing students information in online classrooms to help them reach their highest potential. The acts of teaching and learning online generate data, but to date, that information has remained largely untapped for assessing student performance.

Using data generated by students in an online Master of Public Administration program, drawn from the Marist College Open Academic Analytics Initiative,1 we identify and analyze characteristics and behaviors that best provide early indication of a student being academically at risk, paying particular attention to the usage of online tools. We find that fairly simple learning analytics models achieve high levels of sensitivity (over 80% of at-risk students identified) with relatively low false positive rates (13.5%). Results will be used to test interventions for improving student performance in real time.

Get full access to this article

View all access options for this article.

References

Allen

I. E.

Seaman

(2006). Making the grade. Online education in the United States. Needham, MA: Sloan Consortium.

Allen

I. E.

Seaman

(2011). Going the distance: Online education in the United States, 2011. Needham, MA: Sloan Consortium.

Arnold

(2010). Signals: Applying academic analytics. EDUCAUSE Review, http://www.educause.edu/ero/article/signals-applying-academic-analytics.

Austin

E. K.

(2009). Limits to technology-based distance education in MPA curricula. Journal of Public Affairs Education, 15(2), 161–176.

Barth

T. J.

(2004). Teaching PA online: Reflections of a skeptic. International Journal of Public Administration, 27(6), 439–455.

Bocchi

Eastman

J. K.

Swift

C. O.

(2004). Retaining the online learner: Profile of students in an online MBA program and implications for teaching them. Journal of Education for Business, 79(4), 245–253.

Bonk

C. J.

Reynolds

T. H.

(1997). Learner-centered Web instruction for higher-order thinking, teamwork, and apprenticeship. In Khan

B. H.

, Web-based instruction (167–178). Englewood Cliffs, NJ: Educational Technology Publications.

Brower

R. S.

Klay

W. E.

(2000). Distance learning: Some fundamental questions for public affairs education. Journal of Public Affairs Education, 6(4), 215–231.

Campbell

H. E.

(2006). Cheating, public administration education, and online courses: An essay and call to arms. Journal of Public Affairs Education, 12(1), 33–47.

10.

Campbell

J. P.

(2007). Utilizing student data within the course management system to determine undergraduate student academic success: An exploratory study (Doctoral dissertation). Retrieved from ProQuest (ISBN 054930293X).

11.

Conrad

Donaldson

J. A.

(2004). Engaging the online learner: Activities and resources for creative instruction. San Francisco: Jossey-Bass.

12.

Davenport

Harris

(2007). Competing on analytics. Boston, MA: Harvard Business School Press.

13.

Dringus

L. P.

Ellis

(2005). Using data mining as a strategy for assessing asynchronous discussion forums. Computers & Education, 45(1), 141–160.

14.

Ebdon

(1999). Teaching public finance administration online: A case study. Journal of Public Affairs Education, 5(3), 237–246.

15.

Elwood

Klenowski

(2002). Creating communities of shared practice: The challenges of assessment use in learning and teaching. Assessment & Evaluation in Higher Education, 27(3), 243–256.

16.

Feenberg

(1999). Distance learning: Promise or threat. Crosstalk, 7(1), 12–14.

17.

Gibson

P. A.

Dunning

P. T.

(2012). Creating quality online course design through a peerreviewed assessment. Journal of Public Affairs Education, 18(1), 209–228.

18.

Gikandi

J. W.

Morrow

Davis

N. E.

(2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351.

19.

Ginn

Hammond

(2012). Online education in public affairs: Current and emerging issues. Journal of Public Affairs Education, 18(2), 247–270.

20.

Grandzol

J. R.

(2004). Teaching MBA statistics online: A pedagogically sound process approach. Journal of Education for Business, 79(4), 237–244.

21.

Jacobi

Astin

Ayala

(1987). College student outcomes assessment. Washington, DC: Clearinghouse on Higher Education.

22.

Jayaprakash S., Moody E., Lauría E., Regan J., & Baron J. (2014). Early alert of academically atrisk students: An open source analytics initiative. Journal of Learning Analytics, 1(1), 6–47.

23.

Karaksha

Grant

Anoopkumar-Dukie

Nirthanan

S. N.

Davey

A. K.

(2013). Student engagement in pharmacology courses using online learning tools. American Journal of Pharmaceutical Education, 77(6), 125.

24.

Kolowich

(2013). The professors who make the MOOCs. The Chronicle of Higher Education, Retrieved from http://chronicle.com/article/The-Professors-Behind-theMOOC/13790/#id=overview

25.

Kuh

G. D.

(2003). What we’re learning about student engagement from NSSE. Change, 35, 24–31.

26.

Lauría E., Moody E., Jayaprakash S., Jonnalagadda N., & Baron J. (2013). Open academic analytics initiative: Initial research findings. In Suthers Verbert Duval Ochoa , Proceedings of the 2013 Learning Analytics and Knowledge Conference (pp. 150–154), Leuven, Belgium.

27.

Lauría

Baron

Devireddy

Sundararaju

Jayaprakash

(2012). Mining academic data to improve college student retention: An open source perspective. In Shum, Gasevic, and Ferguson , Proceedings of the 2012 Learning Analytics and Knowledge Conference (pp. 139–142), Vancouver, BC, Canada.

28.

Leavitt

W. M.

Richman

R. S.

(1997). The high tech MPA: Distance learning technology and graduate public administration education. Journal of Public Administration Education, 3(1), 13–27.

29.

Klinger

D. A.

(2000). Hierarchical linear modelling of student and school effects on academic achievement. Canadian Journal of Education, 25(1), 41–55.

30.

Mandinach

(2005). The development of effective evaluation methods for e-learning: A concept paper and action plan. The Teachers College Record, 107(8), 1814–1836.

31.

Mingus

M. S.

(1999). Toward understanding the culture of Internet-mediated learning. Journal of Public Affairs Education, 5(3), 225–235.

32.

Morris

L. V.

S.-S.

Finnegan

C. L.

(2005). Predicting retention in online general education courses. The American Journal of Distance Education, 19(1), 23–36.

33.

Nelson-Le Gall

(1981). Help-seeking: An understudied problem-solving skill in children. Developmental Review, 1, 224–226.

34.

O’Leary

Stowers

G. N. L.

(1999). Computer conferencing in the public affairs classroom. Journal of Public Affairs Education, 5(1), 57–66.

35.

Palloff

R. M.

Pratt

(2007). Building online learning communities: Effective strategies for the virtual classroom. San Francisco, CA: John Wiley & Sons.

36.

Perez-Pena

(2012, July 17). Top universities test the online appeal of free. The New York Times, p. A15.

37.

Rahm

Reed

B. J.

(1997). Going remote: The use of distance learning, the World Wide Web, and the Internet in graduate programs of public affairs and administration. Public Productivity & Management Review, 20(4), 459–474.

38.

Rahm

Reed

B. J.

Rydl

T. L.

(1999). Internetmediated learning in public affairs programs: Issues and implications. Journal of Public Affairs Education, 5(3), 213–223.

39.

Robinson

C. C.

Hullinger

(2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84(2), 101–109.

40.

Rogers

E. M.

(2010). Diffusion of innovations. New York, NY: Simon and Schuster.

41.

Stowers

G. N.

(1995). Getting left behind? Gender differences in computer conferencing. Public Productivity & Management Review, 19(2), 143–159.

42.

Swan

Shen

Hiltz

S. R.

(2006). Assessment and collaboration in online learning. Journal of Asynchronous Learning Networks, 10(1), 45–62.

43.

Van der Merwe

(2011). Can online learning boost academic performance? A microeconomics study. International Business & Economics Research Journal (IBER), 10(8), 45–56.

44.

Vonderwell

S. K.

Boboc

(2013). Promoting formative assessment in online teaching and learning. TechTrends, 57(4), 22–27.

45.

Vonderwell

Liang

Alderman

(2007). Asynchronous discussions and assessment in online learning. Journal of Research on Technology in Education, 39(3), 309.

46.

Waldrop

(2012, March 13). Massive open online courses, aka MOOCs, transform higher education and science. Scientific American. Retrieved from http://www.scientificamerican.com/article/massive-open-online-courses-transform-higher-education-and-science/

47.

Yang

Cornelious

L. F.

(2005). Preparing instructors for quality online instruction. Online Journal of Distance Learning Administration, 8(1). Retrieved from http://www.westga.edu/∼distance/ojdla/spring81/yang81.htm.

48.

Zaiane

O. R.

Luo

(2001). Web usage mining for a better web-based learning environment. In Proceedings of conference on advanced technology for education, Banff, Alberta (pp. 60–64).