AwareNotifications: Multi-device semantic notification handling with user-defined preferences

Abstract

With the increase of connected devices and online services, the number of notifications received by each person is growing. Although notifications are useful to inform users about important information such as new messages and events, the continuous interruptions, the notification duplication, and the rigid delivery can be sources of discomfort. To overcome these issues, we present AwareNotifications, an intelligent system based on user-defined preferences to manage multi-device notifications. AwareNotifications is powered by Semantic Web technologies. By directly exploiting user preferences in the semantic reasoning process, the system is able to identify suitable device(s), modality, and moment(s) to deliver the incoming user notifications. We evaluated AwareNotifications in a user study with 15 participants, in which we compared our system with the “traditional” notification delivery system. The study confirms the perceived effectiveness of AwareNotifications, and provides insights to further improve the system.

Keywords

Internet of Things notifications notification delivery Semantic Web reasoning

Get full access to this article

View all access options for this article.

References

P.D.

Adamczyk and

B.P.

Bailey, If not now, when?: The effects of interruption at different moments within task execution, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’04, ACM, New York, NY, USA, 2004, pp. 271–278. ISBN 1-58113-702-8. doi:10.1145/985692.985727.

Avrahami and

S.E.

Hudson, Responsiveness in instant messaging: Predictive models supporting inter-personal communication, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’06, ACM, New York, NY, USA, 2006, pp. 731–740. ISBN 1-59593-372-7. doi:10.1145/1124772.1124881.

B.P.

Bailey,

P.D.

Adamczyk,

T.Y.

Chang and

N.A.

Chilson, A framework for specifying and monitoring user tasks, Computers in Human Behavior 22(4) (2006), 709–732, ISSN 0747-5632. doi:10.1016/j.chb.2005.12.011.

B.P.

Bailey and

S.T.

Iqbal, Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management, ACM Transactions on Computer–Human Interaction 14(4) (2008), 21:1–21:28, ISSN 1073-0516. doi:10.1145/1314683.1314689.

B.P.

Bailey and

J.A.

Konstan, On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state, Computers in Human Behavior 22(4) (2006), 685–708, ISSN 0747-5632. doi:10.1016/j.chb.2005.12.009.

B.R.

Barricelli and

Valtolina, Designing for end-user development in the Internet of things, in:

Díaz,

Pipek,

Ardito,

Jensen,

Aedo and

Boden, eds, End-User Development: 5th International Symposium, IS-EUD 2015, Madrid, Spain, May 26–29, 2015. Proceedings, Springer International Publishing, Cham, Germany, 2015, pp. 9–24. ISBN 978-3-319-18425-8. doi:10.1007/978-3-319-18425-8_2.

Berners-Lee,

Hendler and

Lassila, The semantic web, Scientific American 284(5) (2001), 28–37. doi:10.1038/scientificamerican0501-34.

Böhmer,

Lander,

Gehring,

D.P.

Brumby and

Krüger, Interrupted by a phone call: Exploring designs for lowering the impact of call notifications for smartphone users, in: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, CHI ’14, ACM, New York, NY, USA, 2014, pp. 3045–3054. ISBN 978-1-4503-2473-1. doi:10.1145/2556288.2557066.

Chen and

Kotz, A survey of context-aware mobile computing research, Technical report, Dartmouth College, Hanover, NH, USA, 2000.

10.

Czerwinski,

Cutrell and

Horvitz, Instant messaging and interruption: Influence of task type on performance, in: OZCHI 2000 Conference Proceedings, ACM, 2000, pp. 356–361.

11.

Czerwinski,

Horvitz and

Wilhite, A diary study of task switching and interruptions, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’04, ACM, New York, NY, USA, 2004, pp. 175–182. ISBN 1-58113-702-8. doi:10.1145/985692.985715.

12.

J.E.

Fischer,

Greenhalgh and

Benford, Investigating episodes of mobile phone activity as indicators of opportune moments to deliver notifications, in: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI ’11, ACM, New York, NY, USA, 2011, pp. 181–190. ISBN 978-1-4503-0541-9. doi:10.1145/2037373.2037402.

13.

J.E.

Fischer,

Yee,

Bellotti,

Good,

Benford and

Greenhalgh, Effects of content and time of delivery on receptivity to mobile interruptions, in: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI ’10, ACM, New York, NY, USA, 2010, pp. 103–112. ISBN 978-1-60558-835-3. doi:10.1145/1851600.1851620.

14.

Fogarty,

S.E.

Hudson,

C.G.

Atkeson,

Avrahami,

Forlizzi,

Kiesler,

J.C.

Lee and

Yang, Predicting human interruptibility with sensors, ACM Transactions on Computer–Human Interaction 12(1) (2005), 119–146, ISSN 1073-0516. doi:10.1145/1057237.1057243.

15.

Fogarty,

A.J.

Ko,

H.H.

Aung,

Golden,

K.P.

Tang and

S.E.

Hudson, Examining task engagement in sensor-based statistical models of human interruptibility, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’05, ACM, New York, NY, USA, 2005, pp. 331–340. ISBN 1-58113-998-5. doi:10.1145/1054972.1055018.

16.

Ho and

S.S.

Intille, Using context-aware computing to reduce the perceived burden of interruptions from mobile devices, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’05, ACM, New York, NY, USA, 2005, pp. 909–918. ISBN 1-58113-998-5. doi:10.1145/1054972.1055100.

17.

Hong,

Suh and

S.-J.

Kim, Context-aware systems, Expert Systems with Applications 36(4) (2009), 8509–8522, ISSN 0957-4174. doi:10.1016/j.eswa.2008.10.071.

18.

Horridge and

Bechhofer, The OWL API: A Java API for OWL ontologies, Semantic Web 2(1) (2011), 11–21, ISSN 1570-0844. doi:10.3233/SW-2011-0025.

19.

Horrocks,

P.F.

Patel-Schneider,

Boley,

Tabet,

Grosof and

Dean, SWRL: A semantic web rule language combining OWL and RuleML, W3C Member Submission, World Wide Web Consortium, 2004.

20.

Horvitz, Principles of mixed-initiative user interfaces, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’99, ACM, New York, NY, USA, 1999, pp. 159–166. ISBN 0-201-48559-1. doi:10.1145/302979.303030.

21.

Horvitz and

Apacible, Learning and reasoning about interruption, in: Proceedings of the 5th International Conference on Multimodal Interfaces, ICMI ’03, ACM, New York, NY, USA, 2003, pp. 20–27. ISBN 1-58113-621-8. doi:10.1145/958432.958440.

22.

S.T.

Iqbal and

Horvitz, Notifications and awareness: A field study of alert usage and preferences, in: Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, CSCW ’10, ACM, New York, NY, USA, 2010, pp. 27–30. ISBN 978-1-60558-795-0. doi:10.1145/1718918.1718926.

23.

Kern and

Schiele, Context-aware notification for wearable computing, in: Proceedings of the 7th IEEE International Symposium on Wearable Computers, ISWC ’03, IEEE Computer Society, Washington, DC, USA, 2003, pp. 223–230. ISBN 0-7695-2034-0. doi:10.1109/ISWC.2003.1241415.

24.

Laput,

Zhang and

Harrison, Synthetic sensors: Towards general-purpose sensing, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, ACM, New York, NY, USA, 2017, pp. 3986–3999. ISBN 978-1-4503-4655-9. doi:10.1145/3025453.3025773.

25.

Mark,

Gudith and

Klocke, The cost of interrupted work: More speed and stress, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, ACM, New York, NY, USA, 2008, pp. 107–110. ISBN 978-1-60558-011-1. doi:10.1145/1357054.1357072.

26.

Mashhadi,

Mathur and

Kawsar, The myth of subtle notifications, in: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp ’14 Adjunct, ACM, New York, NY, USA, 2014, pp. 111–114. ISBN 978-1-4503-3047-3. doi:10.1145/2638728.2638759.

27.

Mehrotra,

Hendley and

Musolesi, PrefMiner: Mining user’s preferences for intelligent mobile notification management, in: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’16, ACM, 2016. doi:10.1145/2971648.2971747.

28.

Mehrotra,

Pejovic,

Vermeulen,

Hendley and

Musolesi, My phone and me: Understanding people’s receptivity to mobile notifications, in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, ACM, New York, NY, USA, 2016, pp. 1021–1032. ISBN 978-1-4503-3362-7. doi:10.1145/2858036.2858566.

29.

Newtson and

Engquist, The perceptual organization of ongoing behavior, Journal of Experimental Social Psychology 12(5) (1976), 436–450, ISSN 0022-1031. doi:10.1016/0022-1031(76)90076-7.

30.

Okoshi,

Nozaki,

Nakazawa,

Tokuda,

Ramos and

A.K.

Dey, Towards attention-aware adaptive notification on smart phones, Pervasive and Mobile Computing 26 (2016), 17–34, ISSN 1574-1192. doi:10.1016/j.pmcj.2015.10.004.

31.

Okoshi,

Ramos,

Nozaki,

Nakazawa,

A.K.

Dey and

Tokuda, Reducing users’ perceived mental effort due to interruptive notifications in multi-device mobile environments, in: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’15, ACM, New York, NY, USA, 2015, pp. 475–486. ISBN 978-1-4503-3574-4. doi:10.1145/2750858.2807517.

32.

Pejovic and

Musolesi, InterruptMe: Designing intelligent prompting mechanisms for pervasive applications, in: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’14, ACM, New York, NY, USA, 2014, pp. 897–908. ISBN 978-1-4503-2968-2. doi:10.1145/2632048.2632062.

33.

Pejovic and

Musolesi, Anticipatory mobile computing: A survey of the state of the art and research challenges, ACM Computing Surveys 47(3) (2015), 47:1–47:29, ISSN 0360-0300. doi:10.1145/2693843.

34.

Perera,

A.B.

Zaslavsky,

Christen and

Georgakopoulos, Context aware computing for the Internet of things: A survey, IEEE Communications Surveys Tutorials 16(1) (2014), 414–454. doi:10.1109/SURV.2013.042313.00197.

35.

Pielot,

Church and

de Oliveira, An in-situ study of mobile phone notifications, in: Proceedings of the 16th International Conference on Human–Computer Interaction with Mobile Devices and Services, MobileHCI ’14, ACM, New York, NY, USA, 2014, pp. 233–242. ISBN 978-1-4503-3004-6. doi:10.1145/2628363.2628364.

36.

Pielot,

Dingler,

J.S.

Pedro and

Oliver, When attention is not scarce – Detecting boredom from mobile phone usage, in: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’15, ACM, New York, NY, USA, 2015, pp. 825–836. ISBN 978-1-4503-3574-4. doi:10.1145/2750858.2804252.

37.

Poppinga,

Heuten and

Boll, Sensor-based identification of opportune moments for triggering notifications, IEEE Pervasive Computing 13(1) (2014), 22–29, ISSN 1536-1268. doi:10.1109/MPRV.2014.15.

38.

Rosenthal,

A.K.

Dey and

Veloso, Using decision-theoretic experience sampling to build personalized mobile phone interruption models, Springer, Berlin, Heidelberg, 2011, pp. 170–187. ISBN 978-3-642-21726-5. doi:10.1007/978-3-642-21726-5_11.

39.

Sahami Shirazi,

Henze,

Dingler,

Pielot,

Weber and

Schmidt, Large-scale assessment of mobile notifications, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, ACM, New York, NY, USA, 2014, pp. 3055–3064. ISBN 978-1-4503-2473-1. doi:10.1145/2556288.2557189.

40.

Stumpf,

Rajaram,

Li,

W.-K.

Wong,

Burnett,

Dietterich,

Sullivan and

Herlocker, Interacting meaningfully with machine learning systems: Three experiments, International Journal of Human–Computer Studies 67(8) (2009), 639–662, ISSN 1071-5819. doi:10.1016/j.ijhcs.2009.03.004.

41.

University of Oxford, HermiT OWL Reasoner, 2016, http://www.hermit-reasoner.com/.

42.

University of Stanford, A free, open-source ontology editor and framework for building intelligent systems, 2016, http://protege.stanford.edu/.

43.

Ur,

McManus,

Pak Yong Ho and

M.L.

Littman, Practical trigger-action programming in the smart home, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, ACM, New York, NY, USA, 2014, pp. 803–812. doi:10.1145/2556288.2557420.

44.

Ur,

Pak Yong Ho,

Brawner,

Lee,

Mennicken,

Picard,

Schulze and

M.L.

Littman, Trigger-action programming in the wild: An analysis of 200,000 IFTTT recipes, in: Proceedings of the 34rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’16, ACM, New York, NY, USA, 2016, pp. 3227–3231. doi:10.1145/2858036.2858556.

45.

M.H.

Vastenburg,

D.V.

Keyson and

de Ridder, Considerate home notification systems: A user study of acceptability of notifications in a living-room laboratory, International Journal of Human–Computer Studies 67(9) (2009), 814–826, ISSN 1071-5819. doi:10.1016/j.ijhcs.2009.06.002.

46.

Weber,

Mayer,

Voit,

Ventura Fierro and

Henze, Design guidelines for notifications on smart TVs, in: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, TVX ’16, ACM, New York, NY, USA, 2016, pp. 13–24. ISBN 978-1-4503-4067-0. doi:10.1145/2932206.2932212.

47.

Weber,

Voit,

Kratzer and

Henze, In-situ investigation of notifications in multi-device environments, in: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’16, ACM, New York, NY, USA, 2016, pp. 1259–1264, http://doi.acm.org/10.1145/2971648.2971732 . ISBN 978-1-4503-4461-6. doi:10.1145/2971648.2971732.