skip to main content
10.1145/3472306.3478344acmconferencesArticle/Chapter ViewAbstractPublication PagesivaConference Proceedingsconference-collections
research-article

Enhancing Conversational Agents with Empathic Abilities

Authors Info & Claims
Published:14 September 2021Publication History

ABSTRACT

Conversational agents are getting increasingly popular and find applications in health and customer services. Conversations in these fields are often emotionally charged. It is, therefore, necessary to handle the conversation with some degree of empathy to be effective. In this work, we leverage advances in the field of natural language processing to create a dialogue system that can convincingly generate empathic responses to text-based messages. To improve the system's ability to converse with empathy, we train the language model on empathic conversations and inject additional emotional information in the response generation. We propose two chatbots: a benchmark bot and an empathic bot. Additionally, we implement an emotion classifier that allows us to predict the emotional state of text-based messages. We evaluate both chatbots in quantitative studies and compare them with human responses in qualitative studies involving human judges. Our evaluation shows that our empathic chatbot outperforms the benchmark bot and even the human-generated responses in terms of perceived empathy. Additionally, we achieve state-of-the-art results in terms of response quality using transformer-based language models. Finally we report that we can double the initial performance of the emotion classifier using undersampling techniques, yielding a final F1-score of 0.81 in six basic emotions.

References

  1. Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a Human-like Open-Domain Chatbot. arXiv:2001.09977 [cs.CL]Google ScholarGoogle Scholar
  2. Rafael E. Banchs. 2017. On the construction of more human-like chatbots: Affect and emotion analysis of movie dialogue data. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 1364--1367. https://doi.org/10.1109/apsipa.2017.8282245Google ScholarGoogle ScholarCross RefCross Ref
  3. Aaron Ben-Zeev. 2000. The Complexity of Emotions. In The Subtlety of Emotions. The MIT Press. https://doi.org/10.7551/mitpress/6548.003.0004Google ScholarGoogle ScholarCross RefCross Ref
  4. C. Daryl Cameron. 2018. Motivating empathy: Three methodological recommendations for mapping empathy. Social and Personality Psychology Compass 12, 11 (oct 2018), e12418. https://doi.org/10.1111/spc3.12418Google ScholarGoogle ScholarCross RefCross Ref
  5. Jacky Casas, Marc-Olivier Tricot, Omar Abou Khaled, Elena Mugellini, and Philippe Cudré-Mauroux. 2020. Trends & Methods in Chatbot Evaluation. In Companion Publication of the 2020 International Conference on Multimodal Interaction (Virtual Event, Netherlands). Association for Computing Machinery, New York, NY, USA, 280--286. https://doi.org/10.1145/3395035.3425319 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yin Hei Chan and Andrew Kwok Fai Lui. 2018. Encoding emotional information for sequence-to-sequence response generation. In 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD). IEEE, 113--116. https://doi.org/10.1109/icaibd.2018.8396177Google ScholarGoogle ScholarCross RefCross Ref
  7. Benjamin M.P. Cuff, Sarah J. Brown, Laura Taylor, and Douglas J. Howat. 2016. Empathy: A Review of the Concept. Emotion Review 8, 2 (2016), 144--153. https://doi.org/10.1177/1754073914558466Google ScholarGoogle ScholarCross RefCross Ref
  8. Paul Ekman. 1992. An argument for basic emotions. Cognition and Emotion 6, 3--4 (may 1992), 169--200. https://doi.org/10.1080/02699939208411068Google ScholarGoogle ScholarCross RefCross Ref
  9. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. ACL, 1615--1625. https://doi.org/10.18653/v1/d17-1169Google ScholarGoogle ScholarCross RefCross Ref
  10. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The Curious Case of Neural Text Degeneration. CoRR abs/1904.09751 (2019). arXiv:1904.09751 http://arxiv.org/abs/1904.09751Google ScholarGoogle Scholar
  11. Qintong Li, Hongshen Chen, Zhaochun Ren, Pengjie Ren, Zhaopeng Tu, and Zhumin Chen. 2020. EmpDG: Multi-resolution Interactive Empathetic Dialogue Generation. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), 4454--4466. https://doi.org/10.18653/v1/2020.coling-main.394Google ScholarGoogle Scholar
  12. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Asian Federation of Natural Language Processing, Taipei, Taiwan, 986--995. https://aclanthology.org/I17-1099Google ScholarGoogle Scholar
  13. Zhaojiang Lin, Peng Xu, Genta Indra Winata, Farhad Bin Siddique, Zihan Liu, Jamin Shin, and Pascale Fung. 2020. CAiRE: An End-to-End Empathetic Chatbot. Proceedings of the AAAI Conference on Artificial Intelligence 34, 09 (Apr. 2020), 13622--13623. https://doi.org/10.1609/aaai.v34i09.7098Google ScholarGoogle ScholarCross RefCross Ref
  14. Bingjie Liu and S. Shyam Sundar. 2018. Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot. Cyberpsychology, Behavior, and Social Networking 21, 10 (oct 2018), 625--636. https://doi.org/10.1089/cyber.2018.0110Google ScholarGoogle ScholarCross RefCross Ref
  15. Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: MIMicking Emotions for Empathetic Response Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 8968--8979. https://doi.org/10.18653/v1/2020.emnlp-main.721Google ScholarGoogle ScholarCross RefCross Ref
  16. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 527--536. https://doi.org/10.18653/v1/P19-1050Google ScholarGoogle ScholarCross RefCross Ref
  17. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training.Google ScholarGoogle Scholar
  18. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 5370--5381. https://doi.org/10.18653/v1/P19-1534Google ScholarGoogle ScholarCross RefCross Ref
  19. Armin Seyeditabari, Narges Tabari, and Wlodek Zadrozny. 2018. Emotion Detection in Text: a Review. CoRR abs/1806.00674 (2018). arXiv:1806.00674 http://arxiv.org/abs/1806.00674Google ScholarGoogle Scholar
  20. Néna Roa Seïler and Paul Craig. 2016. Empathetic Technology. In Emotions, Technology, and Design. Elsevier, 55--81. https://doi.org/10.1016/b978-0-12-801872-9.00004-1Google ScholarGoogle Scholar
  21. Weiyan Shi and Zhou Yu. 2018. Sentiment Adaptive End-to-End Dialog Systems. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 1509--1519. https://doi.org/10.18653/v1/P18-1140Google ScholarGoogle ScholarCross RefCross Ref
  22. Timo Spring, Jacky Casas, Karl Daher, Elena Mugellini, and Omar Abou Khaled. 2019. Empathic response generation in chatbots. In Proceedings of 4th Swiss Text Analytics Conference (SwissText 2019), 18--19 June 2019, Winterthur, Switzerland. 18-19 June 2019, CEUR-WS. http://ceur-ws.org/Vol-2458/paper1.pdfGoogle ScholarGoogle Scholar
  23. Xiao Sun, Xiaoqi Peng, and Shuai Ding. 2017. Emotional Human-Machine Conversation Generation Based on Long Short-Term Memory. Cognitive Computation 10, 3 (dec 2017), 389--397. https://doi.org/10.1007/s12559-017-9539-4Google ScholarGoogle Scholar
  24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. CoRR (2017). arXiv:1706.03762 http://arxiv.org/abs/1706.03762 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Dennis L Wilson. 1972. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics 2, 3 (1972), 408--421.Google ScholarGoogle ScholarCross RefCross Ref
  26. Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. CoRR abs/1901.08149 (2019). arXiv:1901.08149 http://arxiv.org/abs/1901.08149Google ScholarGoogle Scholar
  27. Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press. https://doi.org/10.1145/3025453.3025496 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ozge Nilay Yalcin. 2019. Evaluating Empathy in Artificial Agents. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE. https://doi.org/10.1109/acii.2019.8925498Google ScholarGoogle Scholar
  29. Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2017. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. CoRR abs/1704.01074 (2017). arXiv:1704.01074 http://arxiv.org/abs/1704.01074Google ScholarGoogle Scholar
  30. Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The Design and Implementation of XiaoIce, an Empathetic Social Chatbot. Computational Linguistics (jan 2020), 1--62. https://doi.org/10.1162/coli_a_00368Google ScholarGoogle Scholar
  31. Zhiheng Zhou, Man Lan, and Yuanbin Wu. 2018. A Neural Generation-based Conversation Model Using Fine-grained Emotion-guide Attention. In 2018 International Joint Conference on Neural Networks (IJCNN). IEEE. https://doi.org/10.1109/ijcnn.2018.8488988Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Enhancing Conversational Agents with Empathic Abilities

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in
            • Published in

              cover image ACM Conferences
              IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents
              September 2021
              238 pages
              ISBN:9781450386197
              DOI:10.1145/3472306

              Copyright © 2021 ACM

              Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 14 September 2021

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • research-article
              • Research
              • Refereed limited

              Acceptance Rates

              Overall Acceptance Rate53of196submissions,27%

              Upcoming Conference

              IVA '24
              ACM International Conference on Intelligent Virtual Agents
              September 16 - 19, 2024
              GLASGOW , United Kingdom

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader