Abstract
When people are in an interview, with the interview questions, people’s emotions will change differently. Therefore, it is very helpful to detect people’s emotions in real-time. To do so, comprehensive data collection was performed through the voice recording platform and the Empatica E4 wristband (biofeedback). Also, through using both existing feed-forward deep neural network technology and machine learning, we implemented an artificial deep neural network that aims to detect real emotions using multiple sensors: voice and biometrics. The artificial deep neural network we implemented consistently achieved an accuracy of 85% in our testing set and 79% in validation sets to determine the emotional scale. The research also assists with understanding how to detect emotional ranges and the important role that it plays in interviews and conversations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Keras api reference (2015). https://keras.io/api/
Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
Albraikan, A., Hafidh, B., El Saddik, A.: iAware: a real-time emotional biofeedback system based on physiological signals. IEEE Access 6, 78780–78789 (2018)
Bear, M., Connors, B., Paradiso, M.A.: Neuroscience: Exploring the Brain. Jones & Bartlett Learning, LLC, Burlington (2020)
Domínguez-Jiménez, J.A., Campo-Landines, K.C., Martínez-Santos, J., Delahoz, E.J., Contreras-Ortiz, S.: A machine learning model for emotion recognition from physiological signals. Biomed. Signal Process. Control 55, 101646 (2020)
Kwon, S.J.: Artificial Neural Networks. Nova Science Publishers, New York (2011)
Marechal, C., et al.: Survey on AI-based multimodal methods for emotion detection (2019)
Ng, H.W., Nguyen, V.D., Vonikakis, V., Winkler, S.: Deep learning for emotion recognition on small datasets using transfer learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 443–449 (2015)
Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)
Sharifirad, S., Jafarpour, B., Matwin, S.: How is your mood when writing sexist tweets? detecting the emotion type and intensity of emotion using natural language processing techniques. arXiv preprint arXiv:1902.03089 (2019)
Spoletini, P., Brock, C., Shahwar, R., Ferrari, A.: Empowering requirements elicitation interviews with vocal and biofeedback analysis. In: 2016 IEEE 24th International Requirements Engineering Conference (RE), pp. 371–376. IEEE (2016)
Steppan, M., Fürer, L., Schenk, N., Schmeck, K., et al.: Machine learning facial emotion recognition in psychotherapy research: a useful approach? (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Aledhari, M., Razzak, R., Parizi, R.M., Srivastava, G. (2021). Deep Neural Networks for Detecting Real Emotions Using Biofeedback and Voice. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12664. Springer, Cham. https://doi.org/10.1007/978-3-030-68799-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-68799-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68798-4
Online ISBN: 978-3-030-68799-1
eBook Packages: Computer ScienceComputer Science (R0)
