ABSTRACT
Conversational agents are getting increasingly popular and find applications in health and customer services. Conversations in these fields are often emotionally charged. It is, therefore, necessary to handle the conversation with some degree of empathy to be effective. In this work, we leverage advances in the field of natural language processing to create a dialogue system that can convincingly generate empathic responses to text-based messages. To improve the system's ability to converse with empathy, we train the language model on empathic conversations and inject additional emotional information in the response generation. We propose two chatbots: a benchmark bot and an empathic bot. Additionally, we implement an emotion classifier that allows us to predict the emotional state of text-based messages. We evaluate both chatbots in quantitative studies and compare them with human responses in qualitative studies involving human judges. Our evaluation shows that our empathic chatbot outperforms the benchmark bot and even the human-generated responses in terms of perceived empathy. Additionally, we achieve state-of-the-art results in terms of response quality using transformer-based language models. Finally we report that we can double the initial performance of the emotion classifier using undersampling techniques, yielding a final F1-score of 0.81 in six basic emotions.
- Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a Human-like Open-Domain Chatbot. arXiv:2001.09977 [cs.CL]Google Scholar
- Rafael E. Banchs. 2017. On the construction of more human-like chatbots: Affect and emotion analysis of movie dialogue data. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 1364--1367. https://doi.org/10.1109/apsipa.2017.8282245Google ScholarCross Ref
- Aaron Ben-Zeev. 2000. The Complexity of Emotions. In The Subtlety of Emotions. The MIT Press. https://doi.org/10.7551/mitpress/6548.003.0004Google ScholarCross Ref
- C. Daryl Cameron. 2018. Motivating empathy: Three methodological recommendations for mapping empathy. Social and Personality Psychology Compass 12, 11 (oct 2018), e12418. https://doi.org/10.1111/spc3.12418Google ScholarCross Ref
- Jacky Casas, Marc-Olivier Tricot, Omar Abou Khaled, Elena Mugellini, and Philippe Cudré-Mauroux. 2020. Trends & Methods in Chatbot Evaluation. In Companion Publication of the 2020 International Conference on Multimodal Interaction (Virtual Event, Netherlands). Association for Computing Machinery, New York, NY, USA, 280--286. https://doi.org/10.1145/3395035.3425319 Google ScholarDigital Library
- Yin Hei Chan and Andrew Kwok Fai Lui. 2018. Encoding emotional information for sequence-to-sequence response generation. In 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD). IEEE, 113--116. https://doi.org/10.1109/icaibd.2018.8396177Google ScholarCross Ref
- Benjamin M.P. Cuff, Sarah J. Brown, Laura Taylor, and Douglas J. Howat. 2016. Empathy: A Review of the Concept. Emotion Review 8, 2 (2016), 144--153. https://doi.org/10.1177/1754073914558466Google ScholarCross Ref
- Paul Ekman. 1992. An argument for basic emotions. Cognition and Emotion 6, 3--4 (may 1992), 169--200. https://doi.org/10.1080/02699939208411068Google ScholarCross Ref
- Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. ACL, 1615--1625. https://doi.org/10.18653/v1/d17-1169Google ScholarCross Ref
- Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The Curious Case of Neural Text Degeneration. CoRR abs/1904.09751 (2019). arXiv:1904.09751 http://arxiv.org/abs/1904.09751Google Scholar
- Qintong Li, Hongshen Chen, Zhaochun Ren, Pengjie Ren, Zhaopeng Tu, and Zhumin Chen. 2020. EmpDG: Multi-resolution Interactive Empathetic Dialogue Generation. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), 4454--4466. https://doi.org/10.18653/v1/2020.coling-main.394Google Scholar
- Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Asian Federation of Natural Language Processing, Taipei, Taiwan, 986--995. https://aclanthology.org/I17-1099Google Scholar
- Zhaojiang Lin, Peng Xu, Genta Indra Winata, Farhad Bin Siddique, Zihan Liu, Jamin Shin, and Pascale Fung. 2020. CAiRE: An End-to-End Empathetic Chatbot. Proceedings of the AAAI Conference on Artificial Intelligence 34, 09 (Apr. 2020), 13622--13623. https://doi.org/10.1609/aaai.v34i09.7098Google ScholarCross Ref
- Bingjie Liu and S. Shyam Sundar. 2018. Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot. Cyberpsychology, Behavior, and Social Networking 21, 10 (oct 2018), 625--636. https://doi.org/10.1089/cyber.2018.0110Google ScholarCross Ref
- Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: MIMicking Emotions for Empathetic Response Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 8968--8979. https://doi.org/10.18653/v1/2020.emnlp-main.721Google ScholarCross Ref
- Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 527--536. https://doi.org/10.18653/v1/P19-1050Google ScholarCross Ref
- Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training.Google Scholar
- Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 5370--5381. https://doi.org/10.18653/v1/P19-1534Google ScholarCross Ref
- Armin Seyeditabari, Narges Tabari, and Wlodek Zadrozny. 2018. Emotion Detection in Text: a Review. CoRR abs/1806.00674 (2018). arXiv:1806.00674 http://arxiv.org/abs/1806.00674Google Scholar
- Néna Roa Seïler and Paul Craig. 2016. Empathetic Technology. In Emotions, Technology, and Design. Elsevier, 55--81. https://doi.org/10.1016/b978-0-12-801872-9.00004-1Google Scholar
- Weiyan Shi and Zhou Yu. 2018. Sentiment Adaptive End-to-End Dialog Systems. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 1509--1519. https://doi.org/10.18653/v1/P18-1140Google ScholarCross Ref
- Timo Spring, Jacky Casas, Karl Daher, Elena Mugellini, and Omar Abou Khaled. 2019. Empathic response generation in chatbots. In Proceedings of 4th Swiss Text Analytics Conference (SwissText 2019), 18--19 June 2019, Winterthur, Switzerland. 18-19 June 2019, CEUR-WS. http://ceur-ws.org/Vol-2458/paper1.pdfGoogle Scholar
- Xiao Sun, Xiaoqi Peng, and Shuai Ding. 2017. Emotional Human-Machine Conversation Generation Based on Long Short-Term Memory. Cognitive Computation 10, 3 (dec 2017), 389--397. https://doi.org/10.1007/s12559-017-9539-4Google Scholar
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. CoRR (2017). arXiv:1706.03762 http://arxiv.org/abs/1706.03762 Google ScholarDigital Library
- Dennis L Wilson. 1972. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics 2, 3 (1972), 408--421.Google ScholarCross Ref
- Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. CoRR abs/1901.08149 (2019). arXiv:1901.08149 http://arxiv.org/abs/1901.08149Google Scholar
- Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press. https://doi.org/10.1145/3025453.3025496 Google ScholarDigital Library
- Ozge Nilay Yalcin. 2019. Evaluating Empathy in Artificial Agents. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE. https://doi.org/10.1109/acii.2019.8925498Google Scholar
- Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2017. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. CoRR abs/1704.01074 (2017). arXiv:1704.01074 http://arxiv.org/abs/1704.01074Google Scholar
- Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The Design and Implementation of XiaoIce, an Empathetic Social Chatbot. Computational Linguistics (jan 2020), 1--62. https://doi.org/10.1162/coli_a_00368Google Scholar
- Zhiheng Zhou, Man Lan, and Yuanbin Wu. 2018. A Neural Generation-based Conversation Model Using Fine-grained Emotion-guide Attention. In 2018 International Joint Conference on Neural Networks (IJCNN). IEEE. https://doi.org/10.1109/ijcnn.2018.8488988Google ScholarCross Ref
Index Terms
- Enhancing Conversational Agents with Empathic Abilities
Recommendations
Empathic Chatbot Response for Medical Assistance
IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual AgentsIs it helpful for a medical physical health chatbot to show empathy? How can a chatbot show empathy only based on short-term text conversations? We have investigated these questions by building two different medical assistant chatbots with the goal of ...
Empathy framework for embodied conversational agents
AbstractEmpathy is a complex socio-emotional behavior that results from the interaction between affective and cognitive mechanisms. Equipping embodied conversational agents (ECAs) with empathic capacity can benefit from the integration and ...
A formal model of emotions for an empathic rational dialog agent
Recent research has shown that virtual agents expressing empathic emotions toward users have the potential to enhance human---machine interaction. To provide empathic capabilities to a rational dialog agent, we propose a formal model of emotions based ...
Comments