Tawfiq Ammari
Jofish Kaye
Janice Y. Tsai
Frank Bentley
Skip Abstract Section
Abstract
Voice has become a widespread and commercially viable interaction mechanism with the introduction of voice assistants (VAs), such as Amazon’s Alexa, Apple’s Siri, Google Assistant, and Microsoft’s Cortana. Despite their prevalence, we do not have a detailed understanding of how these technologies are used in domestic spaces. To understand how people use VAs, we conducted interviews with 19 users, and analyzed the log files of 82 Amazon Alexa devices, totaling 193,665 commands, and 88 Google Home Devices, totaling 65,499 commands. In our analysis, we identified music, search, and IoT usage as the command categories most used by VA users. We explored how VAs are used in the home, investigated the role of VAs as scaffolding for Internet of Things device control, and characterized emergent issues of privacy for VA users. We conclude with implications for the design of VAs and for future research studies of VAs.
- Tawfiq Ammari, Priya Kumar, Cliff Lampe, and Sarita Schoenebeck. 2015. Managing children’s online identities: How parents decide what to disclose about their children online. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 1895--1904. Google Scholar
Digital Library
- Ken Anderson, Dawn Nafus, Tye Rattenbury, and Ryan Aipperspach. 2009. Numbers have qualities too: Experiences with ethno-mining. In Ethnographic Praxis in Industry Conference Proceedings, Vol. 2009. Wiley Online Library, 123--140.Google ScholarCross Ref
- Agathe Battestini, Vidya Setlur, and Timothy Sohn. 2010. A large scale study of text-messaging use. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI’10). ACM, New York, NY, 229--238. Google ScholarDigital Library
- Genevieve Bell, Mark Blythe, and Phoebe Sengers. 2005. Making by making strange: Defamiliarization and the design of domestic technologies. ACM Trans. Comput.-Hum. Interact. 12, 2 (June 2005), 149--173. Google ScholarDigital Library
- Genevieve Bell and Paul Dourish. 2007. Yesterday’s tomorrows: Notes on ubiquitous computing’s dominant vision. Personal Ubiquitous Comput. 11, 2 (Jan. 2007), 133--143. Google ScholarDigital Library
- Genevieve Bell and Joseph Kaye. 2002. Designing technology for domestic spaces: A kitchen manifesto. Gastronomica 2, 2 (2002), 46--62.Google ScholarCross Ref
- Frank R. Bentley and Ying-Yu Chen. 2015. The composition and use of modern mobile phonebooks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 2749--2758. Google ScholarDigital Library
- Frank R. Bentley, Nediyana Daskalova, and Brooke White. 2017. Comparing the reliability of amazon mechanical turk and survey monkey to traditional market research surveys. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’17). ACM, New York, NY, 1092--1099. Google ScholarDigital Library
- Anne-jorunn Berg. 1995. A gendered socio-technical construction: The smart house. In Information Technology and Society: A Reader, Nick Heap (Ed.). Sage, London.Google Scholar
- Lindsay Blackwell, Emma Gardiner, and Sarita Schoenebeck. 2016. Managing expectations: Technology tensions among parents and teens. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work 8 Social Computing (CSCW’16). ACM, New York, NY, 1390--1401. Google ScholarDigital Library
- Rodney A. Brooks. 1997. The intelligent room project. In Proceedings of the 2nd International Conference on Cognitive Technology (CT’97). IEEE, 271--278. Google ScholarDigital Library
- A. J. Bernheim Brush, Bongshin Lee, Ratul Mahajan, Sharad Agarwal, Stefan Saroiu, and Colin Dixon. 2011. Home automation in the wild: Challenges and opportunities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, 2115--2124. Google ScholarDigital Library
- Yi Cheng, Kate Yen, Yeqi Chen, Sijin Chen, and Alexis Hiniker. 2018. Why doesn’t it work?: Voice-driven interfaces and young children’s communication repair strategies. In Proceedings of the 17th ACM Conference on Interaction Design and Children (IDC’18). ACM, New York, NY, 337--348. Google ScholarDigital Library
- Andy Crabtree and Tom Rodden. 2004. Domestic routines and design for the home. Comput. Supported Coop. Work 13, 2 (April 2004), 191--220. Google ScholarDigital Library
- Andy Crabtree and Peter Tolmie. 2016. A day in the life of things in the home. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work 8 Social Computing (CSCW’16). ACM, New York, NY, 1738--1750. Google ScholarDigital Library
- Wenrui Diao, Xiangyu Liu, Zhe Zhou, and Kehuan Zhang. 2014. Your voice assistant is mine: How to abuse speakers to steal information and control your phone. In Proceedings of the 4th ACM Workshop on Security and Privacy in Smartphones 8 Mobile Devices (SPSM’14). ACM, New York, NY, 63--74. Google ScholarDigital Library
- Colin Dixon, Ratul Mahajan, Sharad Agarwal, A. J. Brush, Bongshin Lee, Stefan Saroiu, and Victor Bahl. 2010. The home needs an operating system (and an app store). In Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks (Hotnets-IX). ACM, New York, NY, 18:1--18:6. Google ScholarDigital Library
- Marcia Ford and William Palmer. 2018. Alexa, are you listening to me? An analysis of Alexa voice service network traffic. Pers. Ubiquitous Comput. 2 (Jun. 2018), 1--13. Google ScholarDigital Library
- Ramanathan Guha, Vineet Gupta, Vivek Raghunathan, and Ramakrishnan Srikant. 2015. User modeling for a personal assistant. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining (WSDM’15). ACM, New York, NY, 275--284. Google ScholarDigital Library
- Jason I. Hong and James A. Landay. 2004. An architecture for privacy-sensitive ubiquitous computing. In Proceedings of the 2nd International Conference on Mobile Systems, Applications, and Services (MobiSys’04). ACM, New York, NY, 177--189. Google ScholarDigital Library
- S. S. Intille. 2002. Designing a home of the future. IEEE Perv. Comput. 1, 2 (April 2002), 76--82. Google ScholarDigital Library
- Joseph ’Jofish’ Kaye, Joel Fischer, Jason Hong, Frank R. Bentley, Cosmin Munteanu, Alexis Hiniker, Janice Y. Tsai, and Tawfiq Ammari. 2018. Panel: voice assistants, UX design and research. In Proceeding of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA’18). ACM, New York, NY, panel01:1--panel01:5. Google ScholarDigital Library
- Julie A. Kientz, Shwetak N. Patel, Brian Jones, Ed Price, Elizabeth D. Mynatt, and Gregory D. Abowd. 2008. The Georgia tech aware home. In Proceeding of the Extended Abstracts on Human Factors in Computing Systems (CHI EA’08). ACM, New York, NY, 3675--3680. Google ScholarDigital Library
- Julia Kiseleva, Kyle Williams, Jiepu Jiang, Ahmed Hassan Awadallah, Aidan C. Crook, Imed Zitouni, and Tasos Anastasakos. 2016. Understanding user satisfaction with intelligent assistants. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR’16). ACM, New York, NY, 121--130. Google ScholarDigital Library
- Predrag Klasnja, Sunny Consolvo, Tanzeem Choudhury, Richard Beckwith, and Jeffrey Hightower. 2009. Exploring privacy concerns about personal sensing. In Pervasive Computing (Lecture Notes in Computer Science), Hideyuki Tokuda, Michael Beigl, Adrian Friday, A. J. Bernheim Brush, and Yoshito Tobe (Eds.). Springer, Berlin Heidelberg, 176--183. Google ScholarDigital Library
- Josephine Lau, Benjamin Zimmerman, and Florian Schaub. 2018. Alexa, are you listening?: Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. Proc. ACM Hum.-Comput. Interact. 2, CSCW (2018), 102. Google ScholarDigital Library
- Bin Liu, Mads Schaarup Andersen, Florian Schaub, Hazim Almuhimedi, S. A. Zhang, Norman Sadeh, Alessandro Acquisti, and Yuvraj Agarwal. 2016. Follow my recommendations: A personalized privacy assistant for mobile app permissions. In Proceeding of the Symposium on Usable Privacy and Security. https://www.usenix.org/system/files/conference/soups2016/soups2016-paper-liu.pdf. Google ScholarDigital Library
- Silvia Lovato and Anne Marie Piper. 2015. “Siri, is this you?”: Understanding young children’s interactions with voice input systems. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC’15). ACM, New York, NY, 335--338. Google ScholarDigital Library
- Ewa Luger and Tom Rodden. 2013. An informed view on consent for UbiComp. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’13). ACM, New York, NY, 529--538. Google ScholarDigital Library
- Ewa Luger and Abigail Sellen. 2016. “Like having a really bad PA”: The gulf between user expectation and experience of conversational agents. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 5286--5297. Google ScholarDigital Library
- Melissa Mazmanian and Simone Lanette. 2017. “Okay, one more episode”: An ethnography of parenting in the digital age. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW’17). ACM, New York, NY, 2273--2286. Google ScholarDigital Library
- Donald McMillan, Moira McGregor, and Barry Brown. 2015. From in the wild to in vivo: Video analysis of mobile device use. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 494--503. Google ScholarDigital Library
- Emily McReynolds, Sarah Hubbard, Timothy Lau, Aditya Saraf, Maya Cakmak, and Franziska Roesner. 2017. Toys that listen: A study of parents, children, and internet-connected toys. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 5197--5207. Google ScholarDigital Library
- Michael F. McTear. 2002. Spoken dialogue technology: Enabling the conversational user interface. ACM Comput. Surv. 34, 1 (Mar. 2002), 90--169. Google ScholarDigital Library
- Sarah Mennicken and Elaine M. Huang. 2012. Hacking the natural habitat: An in-the-wild study of smart homes, their development, and the people who live in them. In Pervasive Computing (Lecture Notes in Computer Science). Springer, Berlin, Heidelberg, 143--160. Google ScholarDigital Library
- Mohammad-Mahdi Moazzami, Daisuke Mashima, Ulrich Herberg, Wei-Pen Chen, and Guoliang Xing. 2016. SPOT: A smartphone-based control app with a device-agnostic and adaptive user-interface for IoT devices. In Proceedings of the International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp’16). ACM, New York, NY, 670--673. Google ScholarDigital Library
- John Moore, Gerd Kortuem, Andrew Smith, Niaz Chowdhury, Jose Cavero, and Daniel Gooch. 2016. DevOps for the urban IoT. In Proceedings of the 2nd International Conference on IoT in Urban Space (Urb-IoT’16). ACM, New York, NY, 78--81. Google ScholarDigital Library
- Aarthi Easwara Moorthy and Kim-Phuong L. Vu. 2014. Voice activated personal assistant: Acceptability of use in the public space. In Human Interface and the Management of Information. Information and Knowledge in Applications and Services (Lecture Notes in Computer Science). Springer, Cham, 324--334. Google ScholarDigital Library
- Alistair Morrison, Donald McMillan, and Matthew Chalmers. 2014. Improving consent in large scale mobile HCI through personalised representations of data. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI’14). ACM, New York, NY, 471--480. Google ScholarDigital Library
- Pardis Naeini, Sruti Bhagavatula, Hana Habib, Martin Degeling, Lujo Bauer, Lorrie Cranor, and Norman Sadeh. 2017. Privacy Expectations and Preferences in an IoT World. In Proceedings of the 13th Symposium on Usable Privacy and Security (SOUPS’17). Santa Clara, CA, 399–412. https://www.usenix.org/conference/soups2017/technical-sessions/presentation/naeini. Google ScholarDigital Library
- Clifford Nass, Youngme Moon, B. J. Fogg, Byron Reeves, and D. Christopher Dryer. 1995. Can computer personalities be human personalities? Int. J. Hum.-Comput. Studies 43, 2 (Aug. 1995), 223--239. Google ScholarDigital Library
- Helen Nissenbaum. 2009. Privacy in Context: Technology, Policy, and the Integrity of Social Life. Stanford University Press. Google-Books-ID: _NN1uGn1Jd8C. Google ScholarDigital Library
- Kenneth Olmstead. 2017. Nearly half of Americans use digital voice assistants, mostly on their smartphones. http://www.pewresearch.org/fact-tank/2017/12/12/nearly-half-of-americans-use-digital-voice-assistants-mostly-on-their-smartphones/.Google Scholar
- Antti Oulasvirta, Aurora Pihlajamaa, Jukka Perkiö, Debarshi Ray, Taneli Vähäkangas, Tero Hasu, Niklas Vainio, and Petri Myllymäki. 2012. Long-term effects of ubiquitous surveillance in the home. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 41--50. http://dl.acm.org/citation.cfm?id=2370224. Google ScholarDigital Library
- Leysia Palen and Paul Dourish. 2003. Unpacking privacy for a networked world. ACM, 129--136. Google ScholarDigital Library
- Erika Shehan Poole, Marshini Chetty, Rebecca E. Grinter, and W. Keith Edwards. 2008. More than meets the eye: Transforming the user experience of home network management. In Proceedings of the 7th ACM Conference on Designing Interactive Systems (DIS’08). ACM, New York, NY, 455--464. Google ScholarDigital Library
- Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice interfaces in everyday life. In Proceedings of the 36th Annual ACM Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY. Google ScholarDigital Library
- Martin Porcheron, Joel E. Fischer, and Sarah Sharples. 2017. “Do animals have accents?”: Talking with agents in multi-party conversation. In Proceedings of the Conference on Computer Supported Cooperative Work and Social Computing (CSCW’17). ACM, New York, NY, 207--219. Google ScholarDigital Library
- Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. Accessibility came by accident: Use of voice-controlled intelligent personal assistants by people with disabilities. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 459. Google ScholarDigital Library
- Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. “Alexa is my new BFF”: Social roles, user satisfaction, and personification of the Amazon echo. In Proceedings of the Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’17). ACM, New York, NY, 2853--2859. Google ScholarDigital Library
- Juan Ramos. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the 1st Instructional Conference on Machine Learning, Vol. 242. 133--142.Google Scholar
- Stuart Reeves and Barry Brown. 2016. Embeddedness and sequentiality in social media. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work 8 Social Computing (CSCW’16). ACM, New York, NY, 1052--1064. Google ScholarDigital Library
- Jennifer A. Rode, Eleanor F. Toye, and Alan F. Blackwell. 2005. The domestic economy: A broader unit of analysis for end user programming. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems (CHI EA’05). ACM, New York, NY, 1757--1760. Google ScholarDigital Library
- Benjamin Romano. 2017. Managing the internet of things. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE’17). ACM, New York, NY, 777--778. Google ScholarDigital Library
- Xin Rong, Adam Fourney, Robin N. Brewer, Meredith Ringel Morris, and Paul N. Bennett. 2017. Managing uncertainty in time expressions for virtual assistants. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 568--579. Google ScholarDigital Library
- Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Inf. Process. Manag. 24, 5 (1988), 513--523. Google ScholarDigital Library
- Nicole Shechtman and Leonard M. Horowitz. 2003. Media inequality in conversation: How people behave differently when interacting with computers and people. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’03). ACM, New York, NY, 281--288. Google ScholarDigital Library
- Peter Tolmie, Andy Crabtree, Stefan Egglestone, Jan Humble, Chris Greenhalgh, and Tom Rodden. 2010. Digital plumbing: The mundane work of deploying UbiComp in the home. Personal and Ubiquitous Computing 14, 3 (Apr. 2010), 181--196. Google ScholarDigital Library
- Peter Tolmie, Andy Crabtree, Tom Rodden, Chris Greenhalgh, and Steve Benford. 2007. Making the home network at home: Digital housekeeping. In ECSCW 2007. Springer, London, 331--350.Google ScholarCross Ref
- Janice Y. Tsai, Patrick Kelley, Paul Drielsma, Lorrie Faith Cranor, Jason Hong, and Norman Sadeh. 2009. Who’s viewed you?: The impact of feedback in a mobile location-sharing application. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, 2003--2012. Google ScholarDigital Library
- Giorgio Vassallo, Giovanni Pilato, Agnese Augello, and Salvatore Gaglio. 2010. Phase coherence in conceptual spaces for conversational agents. In Semantic Computing. Wiley-Blackwell, 357--371.Google Scholar
- Sergey Volokhin and Eugene Agichtein. 2018. Understanding music listening intents during daily activities with implications for contextual music recommendation. In Proceedings of the on Human Information Interaction 8 Retrieval (CHIIR’18). ACM, New York, NY, 313--316. Google ScholarDigital Library
- Rick Wash, Emilee Rader, and Chris Fennell. 2017. Can people self-report security accurately?: Agreement between self-report and behavioral measures. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 2228--2232. Google ScholarDigital Library
- Thomas Zachariah, Noah Klugman, Bradford Campbell, Joshua Adkins, Neal Jackson, and Prabal Dutta. 2015. The internet of things has a gateway problem. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications (HotMobile’15). ACM, New York, NY, 27--32. Google ScholarDigital Library
Index Terms
- Music, Search, and IoT: How People (Really) Use Voice Assistants
Recommendations
Alexa, Are You Listening?: Privacy Perceptions, Concerns and Privacy-seeking Behaviors with Smart Speakers
Smart speakers with voice assistants, like Amazon Echo and Google Home, provide benefits and convenience but also raise privacy concerns due to their continuously listening microphones. We studied people's reasons for and against adopting smart speakers, ...
Understanding the Long-Term Use of Smart Speaker Assistants
Over the past two years the Ubicomp vision of ambient voice assistants, in the form of smart speakers such as the Amazon Echo and Google Home, has been integrated into tens of millions of homes. However, the use of these systems over time in the home has ...
Use of Intelligent Voice Assistants by Older Adults with Low Technology Use
Special Issue on HCI and the Body:?Reimagining Women's Health and Regular PapersVoice assistants embodied in smart speakers (e.g., Amazon Echo, Google Home) enable voice-based interaction that does not necessarily rely on expertise with mobile or desktop computing. Hence, these voice assistants offer new opportunities to different ...
Comments