Abstract
During gameplay, a player experiences emotional turmoil. In most of the cases, these emotions directly reflect the outcome of the game. Adapting game features based on players’ emotions necessitates a way to detect the current emotional state. Researchers in the area of “video game user research” has studied biometric data as a way to address the diverse characteristics of players, their individual preferences, gameplay expertise, and experiences. Identification of the player’s current state is fundamental for designing a game, which interacts with the player adaptively. In this paper, we present an artificially intelligent game framework with smart features based on automatic facial expression recognition and adaptive game features based on the gamer’s emotion. The gamer’s emotions are recognized at run-time during gameplay using Deep Convolutional Neural Networks (CNN), and the game is adapted accordingly to the emotional condition. Once identified, these features directly modify critical parameters of the underlying game engine to make the game more exciting and challenging.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Blom, P.M., et al.: Towards personalised gaming via facial expression recognition. In: AAAI, pp. 30–36 (2014)
Jakowski, L.: umario. https://github.com/jakowskidev/uMario_Jakowski
Jostine, H.: Mememoji. https://github.com/JostineHo/mememoji
Leahy, A., Clayman, C., Mason, I., Lloyd, G., Epstein, O.: Computerised biofeedback games: a new method for teaching stress management and its use in irritable bowel syndrome. J. R. Coll. Physicians Lond. 32(6), 552–556 (1998)
Lelis, L.H.S., Reis, W.M.P., Gal, Y.: Procedural generation of game maps with human-in-the-loop algorithms. IEEE T-G 10(3), 271–280 (2018)
Li, W., Abtahi, F., Tsangouri, C., Zhu, Z.: Towards an “In-the-Wild" emotion dataset using a game-based framework. arXiv e-prints (2016)
Lobel, A., Gotsis, M., Reynolds, E., Annetta, M., Engels, R.C., Granic, I.: Designing and utilizing biofeedback games for emotion regulation: the case of nevermind. In: CHI EA, pp. 1945–1951 (2016)
Lopes, A.T., de Aguiar, E., Souza, A.F.D., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. PR 61, 610–628 (2017)
Mirza-Babaei, P., Nacke, L., Fitzpatrick, G., White, G., McAllister, G., Collins, N.: Biometric storyboards: visualising game user research data, May 2012
Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: WACV, pp. 1–10 (2016)
babaei Pejman, M., Sebastian, L., Emma, F.: Understanding the contribution of biometrics to games user research. In: DiGRA International Conference: Think Design Play, January 2011
Viola, P., Jones, M.J.: Robust real-time face detection. IJCV 57(2), 137–154 (2004)
Weerdmeester, J., van Rooij, M., Harris, O., Smit, N., Engels, R.C., Granic, I.: Exploring the role of self-efficacy in biofeedback video games. In: CHI PLAY, pp. 453–461 (2017)
Bai, Y., Guo, L., Jin, L., Huang, Q.: A novel feature extraction method using pyramid histogram of orientation gradients for smile recognition. In: ICIP (2009)
Yao, A., Shao, J., Ma, N., Chen, Y.: Capturing au-aware facial features and their latent relations for emotion recognition in the wild. In: ICMI (2015)
Zafar, M.A., Ahmed, B., Al-Rihawi, R., Gutierrez-Osuna, R.: Gaming away stress: using biofeedback games to learn paced breathing. IEEE T-AC 11, 519–531 (2018)
Zhang, J., Huang, K., Yu, Y., Tan, T.: Boosted local structured hog-lbp for object localization. In: CVPR, pp. 1393–1400 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Patidar, I., Modh, K.S., Chattopadhyay, C. (2020). Artificially Intelligent Game Framework Based on Facial Expression Recognition. In: Babu, R.V., Prasanna, M., Namboodiri, V.P. (eds) Computer Vision, Pattern Recognition, Image Processing, and Graphics. NCVPRIPG 2019. Communications in Computer and Information Science, vol 1249. Springer, Singapore. https://doi.org/10.1007/978-981-15-8697-2_29
Download citation
DOI: https://doi.org/10.1007/978-981-15-8697-2_29
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-8696-5
Online ISBN: 978-981-15-8697-2
eBook Packages: Computer ScienceComputer Science (R0)