Abstract
In order to design affective interactive systems, experimental grounding is required for studying expressions of emotion during interaction. In this paper, we present the EmoTaboo protocol for the collection of multimodal emotional behaviours occurring during human-human interactions in a game context. First annotations revealed that the collected data contains various multimodal expressions of emotions and other mental states. In order to reduce the influence of language via a predetermined set of labels and to take into account differences between coders in their capacity to verbalize their perception, we introduce a new annotation methodology based on 1) a hierarchical taxonomy of emotion-related words, and 2) the design of the annotation interface. Future directions include the implementation of such an annotation tool and its evaluation for the annotation of multimodal interactive and emotional behaviours. We will also extend our first annotation scheme to several other characteristics interdependent of emotions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kaiser, S., Wehrle, T., Edwards, P.: Multi-modal emotion measurement in an interactive computer game. In: A pilot study. 8th Conference of the International Society for Research on Emotions (1993)
Wang, N., Marsella, S.: An Emotion Evoking Game. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, Springer, Heidelberg (2006)
Picard, R.: Affective Computing. MIT Press, Cambridge (1997)
Poggi, I., Pelachaud, C., de Rosis, F., Carofiglio, V., De Carolis, B.: GRETA. A Believable Embodied Conversational Agent. In: Stock, O., Zancarano, M. (eds.) Multimodal Intelligent Information Presentation, Kluwer, Dordrecht (2005)
Prendinger, H., Descamps, S., Ishizuka, M.: Scripting Affective Communication with Life-like Characters. In Web-based Interaction Systems. Applied Artificial, Intelligence Journal 16(7-8), 519–553 (2002)
Ekman, P.: Emotions revealed. Understanding faces and feelings. In: Weidenfeld & Nicolson (2003)
Douglas-Cowie, E., Campbell, N., Cowie, R., Roach, P.: Emotional speech - Towards a new generation of databases. In Speech Communication 40, 33–60 (2003)
Schröder, M.: Experimental study of affect burst. In: Speech Communication. Special Issue following the ISCA Workshop on Speech and Emotion, vol. 40, pp. 99–116 (2003)
Wallbott, H.G.: Bodily expression of emotion. In European Journal of Social Psychology 28, 879–896 (1998)
Martin, J.C., Abrilian, S., Devillers, L.: Annotating Multimodal Behaviors Occurring during Non Basic Emotions. 1st Int. Conf. Affective Computing and Intelligent Interaction (ACII 2005), Beijing, China (October 2005)
Le Chenadec, G., Maffiolo, V., Chateau, N., Colletta, J.: Creation of a corpus of multimodal spontaneous expressions of emotions in Human-Machine Interaction. In: 5th Int. Conference on Language Resources and Evaluation (LREC 2006), Genova, Italy (May 2006)
Bänziger, T., Pirker, H., Scherer, K.: GEMEP – Geneva Multimodal Emotion Portrayals: A corpus for the study of multimodal emotional expressions. In: Workshop "Corpora for research on emotion and affect (LREC 2006), Genova, Italy (May 2006)
Reidsma, D., Heylen, D., Ordelman, R.: Annotating emotion in meetings. 5th Int. Conf. on Language Resources and Evaluation (LREC 2006), Genova, Italy (May 2006)
Campbell, N.: A Language-Resources Approach to Emotion: Corpora for the analysis of Expressive Speech. In: Workshop Corpora for research on Emotion and Affect (LREC 2006), Genova, Italy (May 2006)
Poggi, I.: Mind Markers. In: 5th International Pragmatics Conference, Mexico (July 1996)
Forgas, J.: Affective influences on attitudes and judgements. In: Davidson, R., Scherr, K., Goldsmith, H. (eds.) Handbook of Affective Studies, Oxford University Press, Oxford (2003)
Krone, H.W.: Individual differences in emotional reactions and coping. In: Davidson, R.J., Scherer, K.R., Goldsmith, H.H (eds.) Handbook of Affective Sciences, pp. 698–725. Oxford University Press, New York (2003)
Knutson, B.: Facial expressions of emotion influence interpersonal trait inferences. In Journal of Nonverbal Behavior 20, 165–182 (1996)
Baron-Cohen, S., Riviere, A., Fukushima, M., French, D., Hadwin, J., Cross, P., Bryant, C., Sotillo, M.: Reading the Mind in the Face. A Cross-cultural and Developmental Study, Visual Cognition 3, 39–59 (1996)
Baron-Cohen, S., Wheelwright, S., Jolliffe, T.: Is There a Language of the Eyes? Evidence from Normal Adults, and Adults with Autism or Asperger Syndrome. Visual Cognition 4(3), 311–331 (1997)
Hall, J.A., Coats, E.J., Smith LeBeau, L.: Nonverbal behavior and the vertical dimension of social relations: A meta-analysis
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N.: Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine 18(1), 32–80 (2001)
Argyle, M.: Bodily communication. 2 edn., London and New York, Routledge. Taylor & Francis. 0-415-05114-2 (2004)
Thomas, F., Johnson, O.: Disney Animation.The illusion of life. Abbeville Press (1981)
Eysenck H. J. and Eysenck S. B. G.: Manuel de l’Inventaire de Personnalité d’Eysenck. Paris. Les éditions du Centre de Psychologie Appliquée (1971)
Craggs, R., Wood, M.M.: A categorical annotation scheme for emotion in the linguistic content. In: Presented at Affective Dialogue Systems (ADS 2004), Kloster Irsee, Germany (2004)
Scherer, K.R.: Emotion. In: Stroebe, M.H.W. (ed.) In Introduction to Social Psychology: A European perspective, pp. 151–191. Blackwell, Oxford (2000)
Aubergé, V., Audibert, N., Rilliard, A.: Auto-annotation: an alternative method to label expressive corpora. In: LREC 2006 Workshop on Emotional Corpora, Genova, Italy, pp. 45–46 (2006)
Valitutti, A., Strapparava, C., Stock, O.: Developing affective lexical resources. Psychology 2(1), 61–83 (2004)
Mathieu, Y.Y.: Annotations of Emotions and Feelings in Texts. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, Springer, Heidelberg (2005)
Barrett, L.F.: Solving the emotion paradox: categorization and the experience of emotion. In Personality and Social Psychology Review 10(1), 20–46 (2006)
Devillers, L., Abrilian, S., Martin, J.C.: Representing real life emotions in audiovisual data with non basic emotional patterns and context features. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, Springer, Heidelberg (2005)
Core, M.G., Allen, J.F.: Coding Dialogues with the DAMSL Annotation Scheme. In: presented at AAAI Fall Symposium on Communicative Action in Humans and Machines, Menlo Park, California (1997)
Kipp, M.: Gesture Generation by Imitation - From Human Behavior to Computer Character Animation. Boca Raton, Florida: Dissertation.com (2004)
Zara, A., Martin, J.C., Devillers, L., Maffiolo, V., Le Chenadec, G.: Gestural expressions of emotion in human interactions. International Society for Gesture Studies Conference, Evanston United States (accepted, 2007)
Lindquist, K.A., Barrett, L.F., Bliss-Moreau, E., Russel, J.A.: Language and the perception of emotion. Emotion 6(1), 125–138 (2006)
Innes-Ker, A., Niedenthal, P.M.: Emotion concepts and emotional states in social judgment and categorization. Journal of Personality and Social Psychology 83, 804–816 (2002)
Russel, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 45, 1281–1288 (1980)
Plutchik, R.: The emotion. University Press of America, Boston (1991)
Galati, D., Innes-Ker, A., Niedenthal, P.M.: Emotion concepts and emotional states in social judgment and categorization. Journal of Personality and Social Psychology 83, 804–816 (2002)
Gebhard, P., ALMA: - A Layered Model of Affect Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005), Utrecht (2005)
Wiggins, J.S.: The Five-Factor Model of Personality. Theoretical perspectives, The Guildford Press (1996)
Martin, J. C.: Multimodal Human-Computer Interfaces and Individual Differences. Habilitation à diriger des recherches en Informatique. Université Paris XI (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zara, A., Maffiolo, V., Martin, J.C., Devillers, L. (2007). Collection and Annotation of a Corpus of Human-Human Multimodal Interactions: Emotion and Others Anthropomorphic Characteristics. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2007. Lecture Notes in Computer Science, vol 4738. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74889-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-540-74889-2_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74888-5
Online ISBN: 978-3-540-74889-2
eBook Packages: Computer ScienceComputer Science (R0)