Yan Wang
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Recent advances in interactive technology are being used to enrich our interactions around food and drinks. Making use of sound to enrich dining - providing "gustosonic" experiences - has recently garnered interest as an exciting area of ubiquitous computing. However, these experiences have tended to focus on eating. In response to the lack of drinking-focused experiences, we explore the combining playful design and drinking activities so as to allow users to experience playful personalized sounds via drinking through "Sonic Straws". We present the findings of an in-the-wild study that highlights how our system supported self-expression via playful drinking actions, facilitated pleasurable social drinking moments, and promoted reflection on drinking practices. We discuss the implications of this work for designers of future gustosonic experiences, including how to amplify entertainment experiences around drinking/eating, how to highlight the joy coming from multisensory experiences, and how to facilitate mindful engagement with what one drinks. Ultimately, we aim to contribute to the enrichment of dining experiences through playful design.
Supplemental Material
Available for Download
zip
wang.zip (86.8 MB)
Supplemental movie, appendix, image and software files for, Toward Understanding Playful Beverage-based Gustosonic Experiences
- June Ahn, Elizabeth Bonsignore, Derek L Hansen, Kari Kraus, and Carman Neustaedter. 2016. Pervasive Play. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 3317--3324.Google Scholar
- Ferran Altarriba Bertran, Jared Duval, Katherine Isbister, Danielle Wilde, Elena Márquez Segura, Oscar Garcia Pañella, and Laia Badal León. 2019. Chasing play potentials in food culture to inspire technology design. In Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. 829--834.Google ScholarDigital Library
- Ferran Altarriba Bertran, Danielle Wilde, Ernő Berezvay, and Katherine Isbister. 2019. Playful human-food interaction research: State of the art and future directions. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 225--237.Google ScholarDigital Library
- Laurensia Anjani, Terrance Mok, Anthony Tang, Lora Oehlberg, and Wooi Boon Goh. 2020. Why do people watch others eat food? An Empirical Study on the Motivations and Practices of Mukbang Viewers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--13.Google ScholarDigital Library
- Peter Arnold, Rohit Ashok Khot, and Florian 'Floyd' Mueller. 2018. "You Better Eat to Survive" Exploring Cooperative Eating in Virtual Reality Games. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. 398--408.Google ScholarDigital Library
- Audacity. 2021. Free, open source, cross-platform audio software. https://www.audacityteam.org/.Google Scholar
- Jean-François Augoyard. 2014. Sonic experience: a guide to everyday sounds. McGill-Queen's Press-MQUP.Google Scholar
- Maribeth Back and D Des. 1996. Micro-narratives in sound design: Context, character, and caricature in waveform manipulation. Georgia Institute of Technology.Google Scholar
- Monique Faye Baier and Michael Burmester. 2019. Not Just About the User: Acceptance of Speech Interaction in Public Spaces. In Proceedings of Mensch und Computer 2019. 349--359.Google Scholar
- Emilie Baltz. 2021. LICKESTRA - Emilie Baltz. http://emiliebaltz.com/experiments/lickestra/.Google Scholar
- Ferran Altarriba Bertran and Danielle Wilde. 2018. Playing with food: reconfiguring the gastronomic experience through play. In Proceedings of the 1st International Conference on Food Design and Food Studies (EFOOD 2017), October.Google ScholarCross Ref
- Virginia Braun, Victoria Clarke, Virginia Braun, and Victoria Clarke. 2008. Using thematic analysis in psychology. Qualitative Research in Psychology 0887, January (2008), 77--101. https://doi.org/10.1191/1478088706qp063oaGoogle Scholar
- Marion Buchenau and Jane Fulton Suri. 2000. Experience prototyping. In Proceedings of the 3rd conference on designing interactive systems: Processes, practices, methods, and techniques. 424--433.Google ScholarDigital Library
- Sheila Campbell. 2004. Dietary Reference Intakes: Water, potassium, sodium, chloride, and sulfate. Clinical Nutrition Insight 30, 6 (2004), 1--4.Google Scholar
- Yoram Chisik, Patricia Pons, and Javier Jaen. 2018. Gastronomy meets ludology: towards a definition of what it means to play with your (digital) food. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. 155--168.Google ScholarDigital Library
- Hanwool Choe. 2019. Eating together multimodally: Collaborative eating in mukbang, a Korean livestream of eating. Language in Society 48, 2 (2019), 171--208.Google ScholarCross Ref
- Anne-Sylvie Crisinel, Stefan Cosser, Scott King, Russ Jones, James Petrie, and Charles Spence. 2012. A bittersweet symphony: systematically modulating the taste of food by changing the sonic properties of the soundtrack playing in the background. Food Quality and Preference 24, 1 (2012), 201--204.Google ScholarCross Ref
- Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies: Why and how. In Proceedings of the 1st international conference on Intelligent user interfaces. 193--200.Google ScholarDigital Library
- Danièle Dubois, Catherine Guastavino, and Manon Raimbault. 2006. A cognitive approach to urban soundscapes: Using verbal data to access everyday life auditory categories. Acta Acustica United with Acustica 92, 6 (2006), 865--874.Google Scholar
- Frank Feltham, Frank Vetere, and Stephan Wensveen. 2007. Designing tangible artefacts for playful interactions and dialogues. In Proceedings of the 2007 conference on Designing pleasurable products and interfaces. 61--75.Google ScholarDigital Library
- John Ferrara. 2012. Playful design: Creating game experiences in everyday interfaces. Rosenfeld Media.Google Scholar
- Rowanne Fleck and Geraldine Fitzpatrick. 2010. Reflecting on reflection: Framing a design landscape. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction. 216--223.Google ScholarDigital Library
- William Gaver, John Bowers, Andy Boucher, Andy Law, Sarah Pennington, and Nicholas Villar. 2006. The history tablecloth: Illuminating domestic activity. In Proceedings of the 6th conference on Designing Interactive systems. 199--208.Google ScholarDigital Library
- Andrea Grimes and Richard Harper. 2008. Celebratory technology: new directions for food research in HCI. In Proceedings of the SIGCHI conference on human factors in computing systems. 467--476.Google ScholarDigital Library
- Yuki Hashimoto, Naohisa Nagaya, Minoru Kojima, Satoru Miyajima, Junichiro Ohtaki, Akio Yamamoto, Tomoyasu Mitani, and Masahiko Inami. 2006. Straw-like user interface: Virtual experience of the sensation of drinking using a straw. In Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology. 50-es.Google ScholarDigital Library
- Dwight B Heath. 1999. Drinking and pleasure across cultures. Alcohol and pleasure: A health perspective (1999), 61--72.Google Scholar
- Yasamin Heshmat, Carman Neustaedter, Kyle McCaffrey, William Odom, Ron Wakkary, and Zikun Yang. 2020. FamilyStories: Asynchronous Audio Storytelling for Family Members Across Time Zones. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--14.Google ScholarDigital Library
- Kristina Höök and Jonas Löwgren. 2012. Strong concepts: Intermediate-level knowledge in interaction design research. ACM Transactions on Computer-Human Interaction (TOCHI) 19, 3 (2012), 1--18.Google ScholarDigital Library
- Ruth Sancho Huerga, Jennifer Lade, and Florian Mueller. 2016. Designing play to support hospitalized children. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play. 401--412.Google ScholarDigital Library
- Hiroshi Ishii. 2004. Bottles: A transparent interface as a tribute to mark weiser. IEICE Transactions on information and systems 87, 6 (2004), 1299--1311.Google Scholar
- Kalle Jegers. 2007. Pervasive game flow: understanding player enjoyment in pervasive gaming. Computers in Entertainment (CIE) 5, 1 (2007), 9-es.Google ScholarDigital Library
- Eric Jéquier and Florence Constant. 2010. Water as an essential nutrient: the physiological basis of hydration. European journal of clinical nutrition 64, 2 (2010), 115--123.Google Scholar
- Martin Kaltenbrunner, Sergi Jorda, Gunter Geiger, and Marcos Alonso. 2006. The reactable: A collaborative musical instrument. In 15th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE'06). IEEE, 406--411.Google ScholarDigital Library
- Gül Kaner, Hüseyin Uğur Genç, Salih Berk Dinçer, Deniz Erdoğan, and Aykut Coşkun. 2018. GROW: a smart bottle that uses its surface as an ambient display to motivate daily water intake. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1--6.Google ScholarDigital Library
- Rohit Ashok Khot, Jeewon Lee, Deepti Aggarwal, Larissa Hjorth, and Florian 'Floyd' Mueller. 2015. Tastybeats: Designing palatable representations of physical activity. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2933--2942.Google ScholarDigital Library
- Rohit Ashok Khot, Florian Mueller, et al. 2019. Human-food interaction. Foundations and Trends® in Human-Computer Interaction 12, 4 (2019), 238--415.Google Scholar
- Da-jung Kim, Yeoreum Lee, Saeyoung Rho, and Youn-kyung Lim. 2016. Design opportunities in three stages of relationship development between users and self-tracking devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 699--703.Google Scholar
- Naoya Koizumi, Hidekazu Tanaka, Yuji Uema, and Masahiko Inami. 2011. Chewing Jockey. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology - ACE '11. 1. https://doi.org/10.1145/2071423.2071449Google ScholarDigital Library
- David Lee. 2014. The Origins of an Everyday Behavior: Why do People Share Meals? Ph.D. Dissertation. The University of Mississippi.Google Scholar
- Zach Lieberman. 2021. Drink Up Fountain --- YesYesNo Interactive Projects. http://www.yesyesno.com/drink-up-fountain/.Google Scholar
- Peter Linley, Helen Dovey, Eveline de Bruin, Catherine Transler, Joy Wilkinson, John Maltby, and Robert Hurling. 2013. Two simple, brief, naturalistic activities and their impact on positive affect: Feeling grateful and eating ice cream. Psychology of Well-Being: Theory, Research and Practice 3, 1 (2013), 6. https://doi.org/10.1186/2211-1522-3-6Google ScholarCross Ref
- Robyn Longhurst. 2003. Semi-structured interviews and focus groups. Key methods in geography 3, 2 (2003), 143--156.Google Scholar
- Andrés Lucero, Jussi Holopainen, Elina Ollila, Riku Suomela, and Evangelos Karapanos. 2013. The playful experiences framework as a guide for expert evaluation. In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces. 221--230.Google ScholarDigital Library
- Andrés Lucero, Evangelos Karapanos, Juha Arrasvuori, and Hannu Korhonen. 2014. Playful or gameful? Creating delightful user experiences. interactions 21, 3 (2014), 34--39.Google ScholarDigital Library
- Jane McGonigal. 2003. A real little game: The performance of belief in pervasive play. Proceedings of DiGRA 2003 (2003).Google Scholar
- Edoardo Micheloni, Marco Tramarin, Antonio Rodà, and Federico Chiaravalli. 2019. Playing to play: A piano-based user interface for music education video-games. Multimedia Tools and Applications 78, 10 (2019), 13713--13730.Google ScholarDigital Library
- Ine Mols, Elise Van den Hoven, and Berry Eggen. 2016. Informing design for reflection: An overview of current everyday practices. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. 1--10.Google ScholarDigital Library
- Ine Mols, Elise Van Den Hoven, and Berry Eggen. 2016. Technologies for everyday life reflection: Illustrating a design space. In Proceedings of the TEI'16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. 53--61.Google ScholarDigital Library
- Markus Montola, Jaakko Stenros, and Annika Waern. 2009. Pervasive games: theory and design. CRC Press.Google ScholarDigital Library
- Florian 'Floyd' Mueller, Richard Byrne, Josh Andres, and Rakesh Patibanda. 2018. Experiencing the body as play. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.Google ScholarDigital Library
- Florian 'Floyd' Mueller, Tuomas Kari, Rohit Khot, Zhuying Li, Yan Wang, Yash Mehta, and Peter Arnold. 2018. Towards experiencing eating as a form of play. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. 559--567.Google Scholar
- Florian 'Floyd' Mueller, Yan Wang, Zhuying Li, Tuomas Kari, Peter Arnold, Yash Dhanpal Mehta, Jonathan Marquez, and Rohit Ashok Khot. 2020. Towards experiencing eating as play. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction. 239--253.Google ScholarDigital Library
- NVivo. 2021. Qualitative Data Analysis Software | NVivo - QSR International. https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home.Google Scholar
- Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, and Ellen Yi-Luen Do. 2017. Virtual lemonade: Let's teleport your lemonade!. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 183--190.Google ScholarDigital Library
- Nimesha Ranasinghe, Kuan-Yi Lee, and Ellen Yi-Luen Do. 2014. FunRasa: an interactive drinking platform. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. 133--136.Google ScholarDigital Library
- Nimesha Ranasinghe, Thi Ngoc Tram Nguyen, Yan Liangkun, Lien-Ya Lin, David Tolley, and Ellen Yi-Luen Do. 2017. Vocktail: A virtual cocktail for pairing digital taste, smell, and color sensations. In Proceedings of the 25th ACM international conference on Multimedia. 1139--1147.Google ScholarDigital Library
- Yvonne Rogers. 2011. Interaction design gone wild: Striving for wild theory. interactions 18, 4 (2011), 58--62.Google ScholarDigital Library
- Virpi Ed Roto. 2011. User experience white paper. UXwhitepaper (2011).Google Scholar
- Katie Salen, Katie Salen Tekinbaş, and Eric Zimmerman. 2004. Rules of play: Game design fundamentals. MIT press.Google ScholarDigital Library
- Charles Spence. 2010. The multisensory perception of flavour. Psychologist 23, 9 (2010).Google Scholar
- Charles Spence. 2015. Eating with our ears: assessing the importance of the sounds of consumption on our perception and enjoyment of multisensory flavour experiences. Flavour 4, 1 (2015), 3.Google ScholarCross Ref
- Charles Spence. 2016. Sound: The forgotten flavor sense. In Multisensory Flavor Perception. Elsevier, 81--105.Google Scholar
- Charles Spence. 2017. Gastrophysics: The new science of eating. Penguin UK.Google Scholar
- Charles Spence and Betina Piqueras-Fiszman. 2013. Technology at the dining table. Flavour 2, 1 (2013), 1--13.Google ScholarCross Ref
- Charles Spence and Qian Wang. 2015. Wine and music (II): Can you taste the music? Modulating the experience of wine through music and sound. Flavour 4, 1 (2015), 33. https://doi.org/10.1186/s13411-015-0043-zGoogle ScholarCross Ref
- Bernard Suits. 1984. Games and utopia: Posthumous reflections. Simulation & Games 15, 1 (1984), 5--24.Google ScholarCross Ref
- JaYoung Sung, Rebecca E Grinter, and Henrik I Christensen. 2009. "Pimp my roomba": Designing for personalization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 193--196.Google ScholarDigital Library
- Chie Suzuki, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2014. Affecting Tumbler: Affecting our flavor perception with thermaldback. In Proceedings of the 11th Conference on Advances in Computer Entertainment Technology. 1--10.Google ScholarDigital Library
- Kirsten A van den Bosch, Tjeerd C Andringa, Wolter Peterson, Wied AJJM Ruijssenaars, and Carla Vlaskamp. 2017. A comparison of natural and non-natural soundscapes on people with severe or profound intellectual and multiple disabilities. Journal of Intellectual & Developmental Disability 42, 3 (2017), 301--307.Google ScholarCross Ref
- Carlos Velasco and Marianna Obrist. 2021. Multisensory experiences: A primer. Frontiers in Computer Science 3 (2021), 12.Google ScholarCross Ref
- Ron Wakkary, Audrey Desjardins, and Sabrina Hauser. 2016. Unselfconscious interaction: A conceptual construct. Interacting with Computers 28, 4 (2016), 501--520.Google ScholarDigital Library
- Peter Walla, Maria Richter, Stella Färber, Ulrich Leodolter, and Herbert Bauer. 2010. Food-evoked changes in humans. Journal of Psychophysiology 24, 1 (2010), 25--32. https://doi.org/10.1027/0269-8803/a000003Google ScholarCross Ref
- Qian Janice Wang, Charles Spence, and Klemens Knoeferle. 2020. Timing is everything: Onset timing moderates the crossmodal influence of background sound on taste perception. Journal of Experimental Psychology: Human Perception and Performance (2020).Google Scholar
- Yan Wang, Zhuying Li, Robert S Jarvis, Joseph La Delfa, Rohit Ashok Khot, and Florian 'Floyd' Mueller. 2020. WeScream! Toward Understanding the Design of Playful Social Gustosonic Experiences with Ice Cream. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. 951--963.Google ScholarDigital Library
- Yan Wang, Zhuying Li, Robert S Jarvis, Angelina Russo, Rohit Ashok Khot, and Florian 'Floyd' Mueller. 2019. Towards Understanding the Design of Playful Gustosonic Experiences with Ice Cream. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 239--251.Google ScholarDigital Library
- Danielle Wilde. 2011. Extending body and imagination: moving to move. International Journal on Disability and Human Development 10, 1 (2011), 31--36.Google ScholarCross Ref
- Danielle Wilde and Ferran Altarriba Bertran. 2019. Participatory Research through Gastronomy Design: a designerly move towards more playful gastronomy. International Journal of Food Design 4, 1 (2019), 3--37.Google ScholarCross Ref
- Mert Yildiz and Aykut Coşkun. 2019. Wwall: A Public Water Dispenser System to Motivate Regular Water Intake in the Office Environment. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion. 347--352.Google ScholarDigital Library
Index Terms
- Toward Understanding Playful Beverage-based Gustosonic Experiences
Recommendations
WeScream!: Toward Understanding the Design of Playful Social Gustosonic Experiences with Ice Cream
DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems ConferencePrior psychology studies have shown that eating ice cream increases happiness, while human-computer interaction work has shown that interactive technology can enrich the eating experience. We explore the opportunity to combine these two through WeScream!...
Towards Understanding the Design of Playful Gustosonic Experiences with Ice Cream
CHI PLAY '19: Proceedings of the Annual Symposium on Computer-Human Interaction in PlayThere is an increasing trend that explores the convergence of digital play and eating to support a playful relationship with food. We note that interactive sound, although prevalent in digital game design, has only received limited attention in this ...
Sonic Straws: A beverage-based playful gustosonic system
UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable ComputersWe present Sonic Straws, a beverage-based gustosonic system that allows users to experience playful personalized sounds via drinking through straws. The system consists of two straws, a customized lid, and a holder containing a microcontroller. It ...
Comments