Abstract
In this article we present a parameterized model for generating multimodal behavior based on cultural heuristics. To this end, a multimodal corpus analysis of human interactions in two cultures serves as the empirical basis for the modeling endeavor. Integrating the results from this empirical study with a well-established theory of cultural dimensions, it becomes feasible to generate culture-specific multimodal behavior in embodied agents by giving evidence for the cultural background of the agent. Two sample applications are presented that make use of the model and are designed to be applied in the area of coaching intercultural communication.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
CUlture-adaptive BEhavior Generation.
http://corpus.amiproject.org/ (last visited: 02 April 2009).
http://www.bas.uni-muenchen.de/Bas/BasMultiModaleng.html (last visited: 02 April 2009).
Since various kinds of measures were used in the empirical data, they are normalized into 1 to 7.
References
Aylett R, Paiva A, Vannini N, Enz S, André E, Hall L (2009) But that was in another country: agents and intercultural empathy. In: Proceedings of AAMAS
Bennett MJ (1986) A developmental approach to training for intercultural sensitivity. Int J Intercult Relat 10(2):179–195
Bevacqua E, Raouzaiou A, Peters C, Caridakis G, Karpouzis K, Pelachaud C, Mancini M (2006) Multimodal sensing, interpretation and copying of movements by a virtual agent. In: PIT. pp 164–174
Bull PE (1987) Posture and gesture. Pergamon Press, Oxford
Caridakis G, Raouzaiou A, Bevacqua E, Mancini M, Karpouzis K, Malatesta L, Pelachaud C (2007) Virtual agent multimodal mimicry of humans. Lang Resour Eval 41:367–388
Cassell J, Sullivan J, Prevost S, Churchill E (eds) (2000) Embodied conversational agents. MIT Press, Cambridge
De Rosis F, Pelachaud C, Poggi I (2004) Transcultural believability in embodied agents: a matter of consistent adaptation. In: Payr S, Trappl R (eds) Agent culture: human-agent interaction in a multicultural world. Lawrence Erlbaum Associates, London, pp 75–106
Douglas-Cowie E, Cowie R, Cox C, Amir N, Heylen D (2008) The Sensitive artificial listener: an induction technique for generating emotionally coloured conversation. In: Proceedings of LREC workshop on corpora for research on emotion and affect
Duncan S (1974) On the structure of speaker-auditor interaction during speaking turns. Lang Soc 3:161–180
Ekman P (1992) Telling lies—clues to deceit in the marketplace, politics, and marriage, 3rd edn. Norton and Co. Ltd, New York
Hofstede G (1986) Cultural differences in teaching in learning. Int J Intercult Relat 10:301–320
Hofstede G (1991) Cultures and organisations—intercultural cooperation and its importance for survival, software of the mind. Profile Books
Hofstede G (2001) Cultures consequences: comparing values, behaviors, institutions, and organizations across. Nations Sage Publications, Thousand Oaks, London
Hofstede GJ, Pedersen PB, Hofstede G (2002) Exploring culture: exercises, stories and synthetic cultures. Intercultural Press, Yarmouth
Jan D, Herrera D, Martinovski B, Novick D, Traum D (2007) A computational model of culture-specific conversational behavior. In: Pelachaud C, Martin J-C, André E, Chollet G, Karpouzis K, Pelé D (eds) Intelligent virtual agents (IVA’07). Springer, Berlin, pp 45–56. http://dx.doi.org/10.1007/978-3-540-74997-4_5
Jensen FV (2001) Bayesian networks and decicion graphs. Springer, New York
Johnson C (1994) Gender, legitimate authority, and leader-subordinate conversations. Am Sociol Rev 59:122–135
Johnson WL, Choi S, Marsella S, Mote N, Narayanan S, Vilhjálmsson H (2004) Tactical language training system: supporting the rapid acquisition of foreign language and cultural skills. In: Proceedings of InSTIL/ICALL—NLP and speech technologies in advanced language learning systems
Jovanovic N, Akker ROD, Nijholt A (2006) A corpus for studying addressing behavior in multi-party dialogues. Lang Resour Eval 40(1):5–23
Khaled R, Biddle R, Noble J, Barr P, Fischer R (2006) Persuasive interaction for collectivist cultures. In: Piekarski W (ed) The seventh Australasian user interface conference (AUIC 2006). pp 73–80
Kipp M, Neff M, Kipp KH, Albrecht I (2007) Towards natural gesture synthesis: evaluating gesture units in a data-driven approach to gesture synthesis. In: Pelachaud C et al (eds) Intelligent virtual agents (IVA), Berlin, Springer, pp 15–28
Lee J, Marsella S (2006) Nonverbal behavior generator for embodied conversational agents. In: Gratch J et al (eds) Intelligent virtual agents, Berlin, Springer, pp 243–255
Leffler A, Gillespie DL, Conaty JC (1982) The effects of status differentiation on nonverbal behavior. Soc Psychol Q 45(3):153–161
Marcus A (2000) Cultural dimensions and global web user-interface design: what? So What? Now What? In: HFWEB2000 conference proceedings
McCrae RR, John OP (1992) An introduction to the five factor model and its applications. J Personality 60:175–215
McNeill D (1992) Hand and mind—what gestures reveal about thought. The University of Chicago Press, Chicago, London
Nakano YI, Reinstein G, Stocky T, Cassell J (2003) Towards a model of face-to-face grounding. In: Proceedings of the association for computational linguistics. pp 553–561
Payr S, Trappl R (eds) (2004) Agent culture: human-agent interaction in a multicultural world. Lawrence Erlbaum Associates, London
Pelachaud C (2005) Multimodal expressive embodied conversational agents. In: Proceedings of ACM Multimedia. pp 683–689
Reeves B, Nass C (1996) The media equation—how people treat computers, television and new media like real people and places. Cambridge University Press, Cambridge
Rehm M, André E (2007) More than just a friendly phrase—multimodal aspects of polite behaviors in agents. In: Nishida T (ed) Conversational informatics. Wiley, Chichester, pp 69–84
Rehm M, André E, Nakano Y, Nishida T, Bee N, Endrass B, Huang H-H, Wissner M (2007) The CUBE-G approach—coaching culture-specific nonverbal behavior by virtual agents. In: Mayer I, Mastik H (eds) Proceedings of ISAGA 2007
Rehm M, Bee N, André E (2008) Wave like an Egyptian—acceleration based gesture recognition for culture-specific interactions. In: Proceedings of HCI 2008 culture, creativity, interaction. pp 13–22
Rehm M, André E, Bee N, Endrass B, Wissner M, Nakano Y, Lipi AA, Nishida T, Huang H-H (2009) Creating standardized video recordings of multimodal interactions across cultures. In: Kipp M et al (eds) Multimodal copora. Berlin, Springer
Schmidt SM, Yeh RS (1992) The structure of leader influence: a cross-national comparison. J Cross-Cult Psychol 23:251–264
Teng JTC, Calhoun KJ, Cheon MJ, Raeburn S, Wong W (1999) Is the east really different from the west: a cross-cultural study on information technology and decision making. In: Proceedings of the 20th international conference on Information Systems. pp 40–46
Ting-Toomey S (1999) Communicating across cultures. The Guilford Press, New York
Wahlster W (ed) (2006) SmartKom: foundations of multimodal dialogue systems. Springer, Berlin
Warren R, Diller DE, Leung A, Ferguson W, Sutton JL (2005) Simulating scenarios for research on culture and cognition using a commercial role-play game. In: Kuhl ME, Steiger NM, Armstrong FB, Joines JA (eds) Proceedings of the 2005 winter simulation conference
Acknowledgments
The work described in this article is funded by the German Research Foundation (DFG) under research Grant RE 2619/2-1 (CUBE-G) and the Japan Society for the Promotion of Science (JSPS) under a Grant-in-Aid for Scientific Research (C) (19500104).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rehm, M., Nakano, Y., André, E. et al. From observation to simulation: generating culture-specific behavior for interactive systems. AI & Soc 24, 267–280 (2009). https://doi.org/10.1007/s00146-009-0216-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-009-0216-3