Understanding smartphone notifications’ user interactions and content importance

https://doi.org/10.1016/j.ijhcs.2019.03.001Get rights and content

Highlights

  • Smartphone users exhibit different styles to interact with notifications.

  • This work uncovers four distinct methods for interaction.

  • Users frequently neglect to filter away unneeded smartphone notifications.

  • The importance of notification contents can not be derived from interaction.

  • Combining user-reported importance with context to machine learning training data improves notification filtering accuracy drastically.

Abstract

We present the results of our experiment aimed to comprehensively understand the combination of 1) how smartphone users interact with their notifications, 2) what notification content is considered important, 3) the complex relationship between the interaction choices and content importance, and lastly 4) establish an intelligent method to predict user's preference to seeing an incoming notification. We use a dataset of notifications received by 40 anonymous users in-the-wild, which consists of 1) qualitative user-labelled information about their preferences on notification's contents, 2) notification source, and 3) the context in which the notification was received. We assess the effectiveness of personalised prediction models generated using a combination of self-reported content importance and contextual information. We uncover four distinct user types, based on the number of daily notifications and interaction choices. We showcase how usage traits of these groups highlight the requirement for notification filtering approaches, e.g., when specific users habitually neglect to manually filter out unimportant notifications. Our machine learning-based predictor, based on both contextual sensing and notification contents can predict the user's preference for successfully acknowledging an incoming notification with 91.1% mean accuracy, crucial for time-critical user engagement and interventions.

Introduction

Mobile notifications allow applications to inform users of incoming messages, new system events, and reminders, without requiring explicit interaction. Users receive upwards from 60 daily notifications (Pielot et al., 2014; Shirazi et al., 2014), of which many are considered unimportant by the recipient. In response, researchers aim to reduce the interruptive nature of unwanted notifications (Mehrotra et al., 2016; Okoshi et al., 2015; Oliveira et al., 2014; Poppinga et al., 2014) via sensing technologies or by understanding the qualitative nature of notifications. While a large body of work exists on predicting notification-driven interruptibility through situational context, these methods fail to capture the other side of the challenge - is a notification important to the user. Thus, there is a need for better understanding of the relationship between interacting with notifications – how users choose to interact – and the perceived importance of the notification contents.

Here, we aim to understand the underlying importance of individual notifications, how users interact with them, and which factors influence their interaction choices. To investigate the motivation for interacting with notifications, we use self-reported information about the importance of notification contents, notification source, as well as the context of presentation. We capture the motivation in terms of notifications the user would prefer to see regardless of the interaction, e.g., notifications that should be presented even if habitually ignored or dismissed, and the notifications that the user might consider irrelevant or disrupting. Our findings highlight the diverse nature of users’ strategy for manually filtering out notifications in terms of how often users opt to interact with notifications and the interaction choices, and the ever-present need for a notification management system, aiming to prevent information overload - especially considering how frequently users neglect to manually filter out excess notifications.

We also evaluate a notification management system based on these principles. Our system predicts notification importance based on semantic analysis of the similarity of arriving notification and previous notifications. The system also passively collects information about the user's context and combines the aforementioned importance with user context to create a detailed prediction model used to assess whether the user wishes to see the new notification or not. This combined approach shows vast improvements over previous similar systems, highlighting how understanding notification contents can further increase prediction accuracy in filtering out unwanted notifications.

Section snippets

Related work

The role of smartphones has moved away from simple messaging and news-reading to an extended tool aiming to help the user in other aspects of life, e.g., personal health, work, or keeping up with larger social circles, noteworthy when presenting notifications from different, but equally important sources (Gouveia et al., 2015). The notification content (Fischer et al., 2010) and the identification of opportune moments for presenting notifications (Fischer et al., 2011; Iqbal and Bailey, 2008;

Notification diary

We developed an application called Notification Diary to collect contextual and user-originated qualitative information about notifications - the user-perceived importance of the notifications - and how users interacted with those notifications. We deployed Notification Diary on Google's Play Store and made intermittent advertisement campaigns using social media, and on our university campus. The data collection occurred during the first quarter of 2017 (i.e., January - March). The application

Furthering the knowledge in notification interactions

Total of 40 individuals contributed during our data collection period. 113,197 notifications were generated in the dataset. Summary of the logged notifications and their interactions are displayed in Table 2. On average, users interacted with 12.3% of all notifications (results of rows E and F in Table 2), and the majority of the interactions (78.9%) are swipes. Daily notifications ranged from one notification to 2073 (M = 313.4, SD = 803.2, median = 185, IQR = 64 - 393), and number of daily

Distinct notification filtering styles and habits

As the interaction choices differ for each individual user, with each user having personal preferences and configurations – such as installed applications and generic smartphone usage traits, we next aim to differentiate between different types of users. Identifying groups of users as opposed to generalising, or treating each user individually, can be an effective mean of identifying similarities in users (Meyer et al., 2017), and in developing accurate and autonomous intelligent systems (

Combined (Content-Contextual) intelligent notification filtering

In our application, the user can enable two options within the prediction mode – an intelligent method for hiding unwanted notifications – which can be enabled once a required amount (50 labelled samples) of training data is collected by the application. The first option (prediction mode) enables the application to generate machine learning models, and the second option (notification hiding) hides incoming notifications according to the insight provided by the generated models. We use both

Discussion

Intelligent notification management systems traditionally assess the user's situation via usage context for delivering notifications. The importance of individual notifications (and their delivery context) is measured via click-ratios under the binary assumption that clicked notifications are desired and important, while dismissed notifications are not seen as important. This notion has been the basis of multiple works (Mehrotra et al., 2016; Okoshi et al., 2017; Pielot et al., 2017), and while

Conclusion

We collected smartphone notification data in combination with user-labelled information on the importance and timing of notifications. Our results highlight that previous work, which assumed that user's perceived importance of a notification correlates with the notification's interaction, is unfounded in generating knowledge for automatically filtering out unwanted notifications. Many users frequently and habitually dismiss or ignore the majority of their notifications – regardless of their

Declaration of interests

none

Acknowledgements

This work is partially funded by the Academy of Finland (Grants 286386-CPDSS, 285459-iSCIENCE, 304925-CARE, 313224-STOP), and Marie Skłodowska-Curie Actions (645706-GRAGE).

Aku Visuri is a human-computer interaction PhD student at the Center for Ubiquitous Computing located at the University of Oulu, Finland. His background is in designing and implementing solutions for companies in the healthcare business. He received his MSc in Computer Science and Information Networks from University of Oulu in 2016. In his PhD work, the focus is on ubiquitous computing and quantified-self (QS). Specifically understanding users of QS applications and technologies, such as

References (40)

  • Nv. Berkel et al.

    The experience sampling method on mobile devices

    ACM Comput. Surv.

    (2017)
  • H.H. Clark

    Using Language

    (1996)
  • T. Dingler et al.

    I'll be there for you: quantifying attentiveness towards mobile messaging

  • D. Ferreira et al.

    Contextual experience sampling of mobile application micro-usage

  • J.E. Fischer et al.

    Investigating episodes of mobile phone activity as indicators of opportune moments to deliver notifications

  • J.E. Fischer et al.

    Investigating episodes of mobile phone activity as indicators of opportune moments to deliver notifications

  • J.E. Fischer et al.

    Effects of content and time of delivery on receptivity to mobile interruptions

  • R. Gouveia et al.

    How do we engage with activity trackers?: a Longitudinal study of habito

  • M. Hall et al.

    The WEKA data mining software: an update

    SIGKDD Explor. Newsl.

    (2009)
  • S.R. Hiltz et al.

    Structuring computer-mediated communication systems to avoid information overload

    Commun. ACM

    (1985)
  • J. Ho et al.

    Using context-aware computing to reduce the perceived burden of interruptions from mobile devices

  • S.T. Iqbal et al.

    Effects of intelligent notification management on users and their tasks

  • L.A. Leiva et al.

    Back to the app: the costs of mobile application interruptions

  • H. Lopez-Tovar et al.

    Managing smartphone interruptions through adaptive modes and modulation of notifications

  • A. Mathur et al.

    Engagement-aware computing: modelling user engagement from mobile contexts

  • A. Matic et al.

    Boredom-computer interaction: boredom proneness and the use of smartphone

  • A. Mehrotra et al.

    PrefMiner: mining User's Preferences for Intelligent Mobile Notification Management

  • A. Mehrotra et al.

    Designing content-driven intelligent notification mechanisms for mobile applications

  • A. Mehrotra et al.

    Designing content-driven intelligent notification mechanisms for mobile applications

  • A. Mehrotra et al.

    My Phone and Me: understanding people's receptivity to mobile notifications

  • Cited by (35)

    • Sound of silence: Does Muting Notifications Reduce Phone Use?

      2022, Computers in Human Behavior
      Citation Excerpt :

      The Do not Disturb feature offered by Apple, for instance, shuts down all the notifications, but our study suggests that it is important and would lead to a more positive user experience if notifications could be classified based on content genres as well as individual differences of users. In addition to users’ customization of notification type, studies have shown that machine learning algorithms can also help filter out unimportant notifications, and thereby personalize the notifications received by mobile phone users (Visuri et al., 2019). In general, there appears to be a need for a middle ground between completely shutting off notifications to actively alerting users.

    • Hush now! Context factors behind smartphone ringer mode changes

      2021, Pervasive and Mobile Computing
      Citation Excerpt :

      There is a rapidly growing body of publications related to individual notification management that take into account the contexts of user activity, location, time and social circumstances amongst others. So far, the literature has focused on determining user receptivity and response time (e.g. depending on content, social relationship with sender, generating application etc. [7,15–18]), predicting opportune moments to disrupt the user and deliver a notification (e.g. [19–23], or to assess the perceptibility of notifications (e.g.[3,15,24,25]). For the interested reader, [14] provides excellent coverage of further important work in this domain.

    • Vividness of news push notifications and users’ response

      2020, Technological Forecasting and Social Change
      Citation Excerpt :

      The constant reception of push notifications means there is steep competition for the attention of the user (Knobloch et al., 2003; Aranda et al., 2016), who is obliged to select which notifications to interact with – by clicking through – and which to discard – by swiping them away. This choice of whether to open or ignore a push notification depends on a wide variety of factors (Visuri et al., 2019). The identification of opportune timing in their delivery has attracted considerable research interest; many of these studies have sought to examine disruption caused by push notifications, how to avoid this, and how to develop comprehensive systems for managing them in a non-disruptive way (Mehrotra et al., 2016).

    View all citing articles on Scopus

    Aku Visuri is a human-computer interaction PhD student at the Center for Ubiquitous Computing located at the University of Oulu, Finland. His background is in designing and implementing solutions for companies in the healthcare business. He received his MSc in Computer Science and Information Networks from University of Oulu in 2016. In his PhD work, the focus is on ubiquitous computing and quantified-self (QS). Specifically understanding users of QS applications and technologies, such as wearable devices, and designing new methods to enable users with more efficiency.

    Niels van Berkel is a PhD student at the University of Melbourne, where he is part of the Interaction Design Lab. In his PhD research, the focus is on active human sensing through ubiquitous devices, most prominently smartphones. A large portion of his work focuses on the methodological aspects of active data collection (e.g., Experience Sampling Method. He has a background in interaction design and computer science and has been involved in the design and development of a wide variety of projects.

    Tadashi Okoshi is a Project Assistant Professor of Graduate School of Media and Governance, Keio University. He is a computer scientist focusing on distributed systems, mobile and ubiquitous computing, context-aware computing, and “attention-aware” computing. His current research topic is human-attention-awareness and its management in ubiquitous computing and cyber physical systems. He holds B.A. in Environmental Information (1998), Master of Media and Governance (2000) from Keio University, M.S. in Computer Science (2006) from Carnegie Mellon University, and Ph.D. in Media and Governance (2015) from Keio University, respectively. He has 7 years of experience of entrepreneurship, software architecting, product management, and project management in IT industries.

    Jorge Goncalves is a Lecturer in Human-Computer Interaction at the University of Melbourne where he is part of the Interaction Design Lab. Previously, he worked as a postdoctoral researcher at the University of Oulu in the Center for Ubiquitous Computing. He received a PhD with distinction (2015) in Computer Science and Engineering from the University of Oulu, and a BSc (2009) / MSc (2011) in Computer Science and Engineering from the Madeira Interactive Technologies Institute, University of Madeira (Portugal) under the Carnegie Mellon University | Portugal partnership. His research interests include ubiquitous computing, HCI, crowdsourcing, civic engagement, social computing, and mobile sensing.

    Vassilis Kostakos is a Professor in Human-Computer Interaction at the University of Melbourne School of Computing and Information Systems. He is a Marie Curie Fellow, a Fellow in the Academy of Finland Distinguished Professor Program, and a Founding Editor of the PACM IMWUT journal. He holds a PhD in Computer Science from the University of Bath. His research interests include ubiquitous computing (Ubicomp), human-computer interaction (HCI), social computing, and Internet of Things.

    We hereby declare that all aforementioned authors have all made substantial contributions to our work titled: Understanding Smartphone Notifications’ User Interactions and Content Importance

    This work has not been published previously nor is it currently under review in any other medium.

    Our submission consists of total of 11.5K words and is included in this document starting from page four. All figures should be colorized, when applicable, and are included within the submission.

    We hereby declare that we, the authors, nor any corresponding financial or governing body affiliated with this submission have no competing or other interests that might influence our work.

    View full text