PACARD: A New Interface to Increase Mobile Learning App Engagement,Distributed Through App Stores

Abstract

This study proposed to enhance mobile learning engagement with PACARD (Personalized Adaptive CARD-based interface) that combines several technologies including card-based interface, personalized adaptation, push notifications, and badges. To evaluate our proposal, we distributed a mobile learning application (app) called English Practice on the Google Play store, and 63,824 online users were recruited to join the experiment. Six metrics of app engagement (app retention, session length, session count, total time consumption, average duration per day, and uninstall rate) were logged and analyzed. Results are presented in three parts. First, PACARD increased the number of user sessions, duration of use, and retention of app. Second, investigating PACARD also provided detailed reports about users, including their habits and behaviors, thus providing greater understanding of PACARD use. Third, PACARD improved learning achievement. Because of the large number of participants, this study's findings are reliable and widely applicable. PACARD is easy to implement and tailor to most mobile devices and mobile learning apps on the market. Indeed, it benefits educators and mobile app developers as well as learners themselves.

Keywords

mobile learning app engagement card-based design app store personalized adaptive

Get full access to this article

View all access options for this article.

References

Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., & Ananthanarayanan, V. (2017). NMC horizon report: 2017 higher education edition. Austin, Texas: The New Media Consortium. Retrieved from https://www.nmc.org/publication/nmc-horizon-report-2017-higher-education-edition/.

Adams Becker

Freeman

Giesinger Hall

Cummins

Yuhnke

(2016) The NMC/CoSN horizon report: 2016 K-12 edition, Austin, TX: The New Media Consortium.

Adams, P. (2015). Why cards are the future of the web. Retrieved from https://blog.intercom.io/why-cards-are-the-future-of-the-web/.

Adler, B. (2014). The 8 mobile app metrics that matter. Retrieved from http://info.localytics.com/blog/the-8-mobile-app-metrics-that-matter.

Albert, D. (2015). Pushing push—Have notifications come of age? Retrieved from https://www.sitepoint.com/pushing-push-notifications-come-age/.

Ally

Tsinakos

(2014) Increasing access through mobile learning, Vancouver, Canada: Commonwealth of Learning.

Anderson, A., Huttenlocher, D., Kleinberg, J., & Leskovec, J. (2014). Engaging with massive online courses. Paper presented at the 23rd international conference on World wide web, Seoul, Korea.

Apple. (2015). iPad in education. Retrieved from http://www.apple.com/education/ipad/apps-books-and-more.

Baddeley

A. D.

(1997) Human memory: Theory and practice, Exeter, England: Psychology Press.

10.

Böhmer

Krüger

(2014) A case study of research through the app store: Leveraging the system UI as a playing field for improving the design of smartphone launchers. International Journal of Mobile Human Computer Interaction (IJMHCI) 6(2): 32–45.

11.

Bosomworth, D. (2015). Mobile marketing statistics 2015. Smart Insights site. Retrieved from http://www.plenuminvest.dk/docs/mobile-marketing-statistics-2015.docx.

12.

Buzzsprout. (2015). Mobile learning: Why tech savvy educators are turning to podcasts. Retrieved from https://www.buzzsprout.com/blog/2015/01/15/mobile-learning.

13.

Cao, J. (2015). How cards are taking over Web design. Retrieved from http://thenextweb.com/dd/2015/06/16/how-cards-are-taking-over-web-design/.

14.

Caple, C. (1996). The effects of spaced practice and spaced review on recall and retention using computer assisted instruction (PhD dissertation). Raleigh, NC: North Carolina State University.

15.

Charleer, S., Klerkx, J., Odriozola, S., Luis, J., & Duval, E. (2013). Improving awareness and reflection through collaborative, interactive visualizations of badges. Paper presented at the 3rd Workshop on Awareness and Reflection in Technology Enhanced Learning (ARTEL13), Paphos, Cyprus.

16.

Chen, A. (2015). New data shows losing 80% of mobile users is normal, and why the best apps do better. Retrieved from http://andrewchen.co/new-data-shows-why-losing-80-of-your-mobile-users-is-normal-and-that-the-best-apps-do-much-better/.

17.

Cherner

Dix

Lee

(2014) Cleaning up that mess: A framework for classifying educational apps. Contemporary Issues in Technology and Teacher Education 14(2): 158–193.

18.

Chiong

Shuler

(2010) Learning: Is there an app for that? Investigations of young children's usage and learning with mobile devices and apps, New York, NY: The Joan Ganz Cooney Center at Sesame Workshop.

19.

Chittaro

Vianello

(2016) Evaluation of a mobile mindfulness app distributed through on-line stores: A 4-week study. International Journal of Human Computer Studies 86: 63–80.

20.

Clarke

Miles

(2003) Changing systems to personalize learning: Introduction to the personalization workshops, Providence, RI: Education Alliance at Brown University.

21.

Cohen

Hadley

Frank

(2011) Young children, apps & iPad. U.S. Department of Education Ready to Learn Program 200: 5–10.

22.

Cutrell, E., Czerwinski, M., & Horvitz, E. (2001). Notification, disruption, and memory: Effects of messaging interruptions on memory and performance. Proceedings of the IFIP Conference on Human-Computer Interaction (INTERACT 2001) (pp 263–269). Tokyo, Japan: IOS Press.

23.

Cutter, J. D. (2015). Card based design: How mobile is shaping the future of online content. Retrieved from http://graphicadesign.com/news/card-based-design-how-mobile-is-shaping-the-future-of-online-content.

24.

Czerwinski, M., Horvitz, E., & Wilhite, S. (2004). A diary study of task switching and interruptions. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Vienna, Austria.

25.

Dreyer

Nel

(2003) Teaching reading strategies and reading comprehension within a technology-enhanced learning environment. System 31(3): 349–365.

26.

Edge, D., Searle, E., Chiu, K., Zhao, J., & Landay, J. A. (2011). MicroMandarin: Mobile language learning in context. Paper presented at the proceedings of the SIGCHI conference on human factors in computing systems, Vancouver, BC, Canada.

27.

Ellis

N. C.

(1995) The psychology of foreign language vocabulary acquisition: Implications for CALL. Computer Assisted Language Learning 8(2–3): 103–128.

28.

Farago, P. (2012). App engagement: The matrix reloaded. Flurry Insights. Retrieved from http://www.flurry.com/bid/90743/App-Engagement-The-Matrix-Reloaded.

29.

Fenech, T. (2015). The 9 Keys to taming promiscuous app users and increasing user retention. Retrieved from https://www.helpshift.com/blog/the-9-keys-to-taming-promiscuous-app-users-and-increasing-user-retention/.

30.

Ferreira

Kostakos

Dey

A. K.

(2012) Lessons learned from large-scale user studies: Using android market as a source of data. International Journal of Mobile Human Computer Interaction (IJMHCI) 4(3): 28–43.

31.

Flashcardlearner.com. What is spaced repetition? Retrieved from http://www.flashcardlearner.com/articles/what-is-spaced-repetition/.

32.

Gassler, G., Hug, T., & Glahn, C. (2004). Integrated micro learning–An outline of the basic method and first results. Paper presented at the International Conference on Interactive Computer Aided Learning (ICL 2004), Villach, Austria.

33.

Godwin-Jones

(2011) Emerging technologies: Mobile apps for language learning. Language Learning & Technology 15(2): 2–11.

34.

Goodwin, K., & Highfield, K. (2012). iTouch and iLearn: An examination of “educational” apps. Paper presented at the early education and technology for children conference, March 14–16, Salt Lake City, UT.

35.

Hamari

(2017) Do badges increase user activity? A field experiment on the effects of gamification. Computers in Human Behavior 71: 469–478.

36.

Hoch, D. (2014). App retention improves—Apps used only once declines to 20%. Retrieved from http://info.localytics.com/blog/app-retention-improves.

37.

Hong

J.-C.

Hwang

M.-Y.

Tai

K.-H.

Chen

Y.-L.

(2014) Using calibration to enhance students' self-confidence in English vocabulary learning relevant to their judgment of over-confidence and predicted by smartphone self-efficacy and English learning anxiety. Computers & Education 72: 313–322.

38.

Hulstijn

J. H.

Hulstijn

J. H.

(2001) Intentional and incidental second language vocabulary learning: A reappraisal of elaboration, rehearsal and automaticity: Cognition and second language instruction, Cambridge, England: Cambridge University Press.

39.

Iqbal, S. T., & Horvitz, E. (2007). Disruption and recovery of computing tasks: Field study, analysis, and directions. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, San Jose, California, USA.

40.

Iqbal, S. T., & Horvitz, E. (2010). Notifications and awareness: A field study of alert usage and preferences. Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, CSCW 2010 (pp. 27–30). Savannah, GA: ACM.

41.

Jackson

(2015) Pro android wearables: Building apps for smartwatches, Berkely, CA: Apress.

42.

Klementi, K. (2015). Cards as the new creative canvas in web design. Retrieved from http://blog.templatemonster.com/2015/01/14/cards-as-the-new-creative-canvas-in-web-design/.

43.

Kosir, D. (2015). 5 methods for increasing app engagement & user retention. Retrieved from http://clearbridgemobile.com/5-methods-for-increasing-app-engagement-user-retention/.

44.

Kothari, N. (2016). The 9 key metrics for measuring user engagement for ecommerce apps. Retrieved from https://mofluid.com/blog/the-9-key-metrics-for-measuring-user-engagement-for-ecommerce-apps/.

45.

Kumulos. (2015). Push notifications—5 best ways to maximize engagement. Retrieved from https://www.kumulos.com/2015/11/12/push-notifications-5-best-ways-to-maximize-engagement/.

46.

Lake, C. (2014). 15 delicious examples of card-based web design. Retrieved from https://econsultancy.com/blog/64646-15-delicious-examples-of-card-based-web-design/.

47.

Law, P. (2015). Digital badging at The Open University: Recognition for informal learning. Paper presented at the The Open and Flexible Higher Education Conference 2014: New Technologies and the Future of Teaching and Learning, Krakow, Poland.

48.

Leene, A. (2006). Microcontent is everywhere. Proceedings of the microlearning conference 2006 (pp. 20–40). Innsbruck, Austria: Innsbruck University.

49.

Leitner

(1972) So lernt man lernen: Der Weg zum Erfolg [How to learn to learn: The Way to Success], Freiburg, Germany: Herder.

50.

Lele, S. (2015). 8 Tips for boosting app retention: Create an app for the long run. Retrieved from http://blog.personagraph.com/8-tips-for-boosting-app-retention-create-an-app-for-the-long-run.

51.

Levy

Kennedy

(2005) Learning Italian via mobile SMS. In: Agnes

K.-H.

John

(eds) Mobile learning: A handbook for educators and trainers, Oxon, England: Routledge, pp. 76–83.

52.

Localytics. (2014). Push messaging drives 88% more app launches. Retrieved from http://info.localytics.com/blog/push-messaging-drives-88-more-app-launches-for-users-who-opt-in.

53.

Lucey

T. A.

Laney

J. D.

(2012) Reframing financial literacy: Exploring the value of social currency, Charlotte, NC: IAP.

54.

Marcellino, C., & Santamaria, J. (2012). Managing notification service connections and displaying icon badges: Google Patents No. US 8135392 B2 United States.

55.

Miluzzo, E., Lane, N. D., Lu, H., & Campbell, A. T. (2010). Research in the app store era: Experiences from the cenceme app deployment on the iphone. Paper presented at the First Workshop on Research in the Large at UbiComp, Copenhagen, Denmark.

56.

Munson, S. A., & Consolvo, S. (2012). Exploring goal-setting, rewards, self-monitoring, and sharing to motivate physical activity. Paper presented at the 6th International Conference on Pervasive Computing Technologies for Healthcare, San Diego, CA, USA.

57.

Newmann

F. M.

(1989) Student engagement and high school reform. Educational Leadership 46(5): 34–36.

58.

O'Hare, E., & Cinekid, C. (2014). Mobile apps for children. Retrieved from http://www.cinekid.nl/projects/research.

59.

Pachler

Bachmair

Cook

(2009) Mobile learning: Structures, agency, practices, New York, NY: Springer Science & Business Media.

60.

Panel, M. (2005). Trends report: The new standards for mobile app retention. Retrieved from https://mixpanel.com/blog/2013/11/04/trends-report-the-new-standards-for-mobile-app-retention/.

61.

Pelton, T., & Francis Pelton, L. (2011) Design principles for making meaningful mathematics apps. Proceedings of Society for Information Technology & Teacher Education International Conference (pp 2199–2204). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).

62.

Pham, X. L., & Chen, G. D. (2015). PACARD: A personalized adaptive card-based interface that enable learners to access and recall learning element efficiently. Paper presented at the 23rd international conference on computers in education (ICCE 2015) (pp. 245–255), Hangzhou, China.

63.

Pham

X. L.

Chen

G. D.

Nguyen

T. H.

Hwang

W. Y.

(2016a) Card-based design combined with spaced repetition: A new interface for displaying learning elements and improving active recall. Computers & Education 98: 142–156.

64.

Pham, X. L., Chen, G. D., Nguyen, T. H., & Hwang, W. Y. (2016b). Effects of push notifications on learner engagement in a mobile learning app. Paper presented at the IEEE International Conference on Advanced Learning Technologies (ICALT 2016), Austin, TX, USA.

65.

Pilgrim

Bledsoe

Reily

(2012) New technologies in the classroom. Delta Kappa Gamma Bulletin 78(4): 16–20.

66.

Rakestraw

T. L.

Eunni

R. V.

Kasuganti

R. R.

(2013) The mobile apps industry: A case study. Journal of Business Cases and Applications 9: 1.

67.

Rhodes, W. (2016). 28 metrics that matter for your app. Retrieved from https://savvyapps.com/blog/mobile-app-analytics.

68.

Riconscente

(2011) Mobile learning game improves 5th graders' fractions knowledge and attitudes, Los Angeles, CA: GameDesk Institute.

69.

Rossignol, J. (2016). iOS and android capture combined 98.4% share of smartphone market. Retieved from http://www.macrumors.com/2016/02/18/ios-android-market-share-q4-15-gartner/.

70.

Rossing

J. P.

Miller

W. M.

Cecil

A. K.

Stamper

S. E.

(2012) iLearning: The future of higher education? Student perceptions on learning with mobile tablets. Journal of the Scholarship of Teaching and Learning 12(2): 1–26.

71.

Sanchez

C. A.

Branaghan

R. J.

(2011) Turning to learn: Screen orientation and reasoning with small devices. Computers in human behavior 27(2): 793–797.

72.

Sharples, M. (2006). Big issues in mobile learning. Paper presented at the Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative, University of Nottingham, England.

73.

Shuler

(2009) iLearn: A content analysis of the iTunes app store's education section, New York, NY: The Joan Ganz Cooney Center at Sesame Workshop.

74.

Shuler, C., Levine, Z., & Ree, J. (2012). iLearn II: An analysis of the education category of Apple's app store. Paper presented at the The Joan Ganz Cooney Center at Sesame Workshop, New York, USA.

75.

Snapwiz. (2013). Improving student engagement by integrating adaptive and collaborative learning technologies. Retrieved from http://ww1.prweb.com/prfiles/2013/10/22/11256816/Improving-Student-Engagement white paper.pdf.

76.

TalentLMS. (2014). e-learning concepts, trends, applications. Retrieved from http://www.talentlms.com/elearning/elearning-101-jan2014-v1.1.pdf.

77.

Technavio. (2015). Global education apps market—Market study 2015–2019. Retrieved from http://www.technavio.com/report/global-education-apps-market-market-study-2015-2019.

78.

Tolub, Y. (2016). Mobile app analytics: The 12 most important metrics to measure. Retrieved from http://www.uxmatters.com/mt/archives/2016/08/mobile-app-analytics-the-12-most-important-metrics-to-measure.php.

79.

Tsai

C.-W.

Shen

P.-D.

Tsai

M.-C.

Chen

W.-Y.

(2017) Exploring the effects of web-mediated computational thinking on developing students' computing skills in a ubiquitous learning environment. Interactive Learning Environments 25: 762–777.

80.

Tvarozek, J., & Brza, T. (2014) Engaging students in online courses through interactive badges. Paper presented at the international conference on e-learning (p. 89). Spain: University of La Laguna.

81.

Walker

(2011) Evaluating the effectiveness of apps for mobile devices. Journal of Special Education Technology 26(4): 59–63.

82.

Wang

Fesenmaier

D. R.

(2003) Assessing motivation of contribution in online communities: An empirical investigation of an online travel community. Electronic Markets 13(1): 33–45.

83.

Warren

Meads

Srirama

Weerasinghe

Paniagua

(2014) Push notification mechanisms for pervasive smartphone applications. IEEE Pervasive Computing 13(2): 61–71.

84.

Wozniak

Gorzelanczyk

E. J.

(1994) Optimization of repetition spacing in the practice of learning. Acta Neurobiologiae Experimentalis 54: 59.

85.

Zare, S. (2011). Personalization in mobile learning for people with special needs. International conference on universal access in human-computer interaction. Applications and Services (pp. 662–669). Berlin, Germany: Springer.

86.

Zinevych, S. (2014). How a content-rich mobile app can boost your customer retention. Retrieved from http://publ.com/blog/2014/09/24/how-a-content-rich-mobile-app-can-boost-your-customer-retention/.

87.

Zydney

J. M.

Warner

(2015) Mobile apps for science learning: Review of research. Computers & Education 94: 1–17.