Abstract
In this work we have enhanced the perception of a humanoid robot by integrating it with a social state estimation system. We present a user study of the humanoid Nao robot as a social mediator, comprising two sets of experiments. In the first sets of experiments, the participants rate their understanding of feedback messages delivered via the Nao robot. They also assess two modalities to deliver the feedback: audio only and audio combined with gestures. In almost all cases there is an improvement of 10% or more when audio and gesture modalities are combined to deliver feedback messages. For the second sets of experiments the sociofeedback system was integrated with the Nao robot. The participants engage in two-person scenario-based conversations while the Nao robot acts as a mediator. The sociofeedback system analyzes the conversations and provides feedback via Nao. Subsequently, the participants assess the received sociofeedback with respect to various aspects, including its content, appropriateness, and timing. Participants also evaluate their overall perception of Nao as social mediator via the Godspeed questionnaire. The results indicate that the social feedback system is able to detect the social scenario with 93.8% accuracy and that Nao can be effectively used to provide sociofeedback in discussions. The results of this paper pave the way to natural human–robot interactions for social mediators in multi-party dialog systems.
Similar content being viewed by others
References
Fong T, Nourbakhsh I, Dautenhahn K (2003) A survey of socially interactive robots. Robot Auton Syst 42(3):143–166
Li H, Cabibihan J-J, Tan YK (2011) Towards an effective design of social robots. Int J Soc Robot 3(4):333–335
Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2010) A communication robot in a shopping mall. IEEE Trans Robot 26(5):897–913
Thrun S, Bennewitz M, Burgard W, Cremers AB, Dellaert F, Fox D, Hahnel D, Rosenberg C, Roy N, Schulte J, Schulz D (1999) Minerva: a second-generation museum tour-guide robot. In: Proceedings of IEEE international conference on robotics and automation (ICRA 99)
Tanaka F, Cicourel A, Movellan JR (2007) Socialization between toddlers and robots at an early childhood education center. Proc Ntl Acad Sci 104:17954–17958
Graf B, Parlitz C, Hägele M (2009) Robotic home assistant care-o-bot\(^{\textregistered }\) 3 product vision and innovation platform. HCI 2:312–320
Williams K, Breazeal C (2012) A reasoning architecture for human-robot joint tasks using physics-, social-, and capability-based logic. In: IROS, pp 664–671
François D, Polani D, Dautenhahn K (2008) Towards socially adaptive robots: a novel method for real time recognition of human-robot interaction styles. In Humanoids, pp 353–359
Papadopoulos F, Dautenhahn K, Ho WC (2012) Exploring the use of robots as social mediators in a remote human–human collaborative communication experiment. Paladyn 3(1):1–10
Dautenhahn K (2007) Socially intelligent robots: dimensions of human–robot interaction. Phil Trans R Soc B Biol Sci 362(1480):679–704
Bohus D, Horvitz E (2009) Dialog in the open world: platform and applications. In: Proceedings of the 2009 international conference on multimodal interfaces, ACM, pp 31–38
Salah AA, Gevers T, Sebe N, Vinciarelli A (2010) Human behavior understanding. In: First international workshop, HBU 2010, Istanbul, Turkey, August 22, 2010, Proceedings, vol 6219. Springer
Salah AA, Ruiz-del Solar J, Meriçli Ç, Oudeyer P-Y (2012) Human behavior understanding for robotics. In: Human behavior understanding. Springer, pp 1–16
Rasheed U, Tahir Y, Dauwels S, Dauwels J (2013) Real-time comprehensive sociometrics for two-person dialogs. In: Human behavior understanding. Springer, pp 196–208
Pentland AS (2010) Honest signals. MIT press, Cambridge
Tahir Y, Rasheed U, Dauwels S, Dauwels J, Thalmann N, Thalmann D (2013) Nao as social mediator: a user study. In: Robots in public spaces:towards multi-party, short-term, dynamic human–robot interaction, Springer
Bartneck C, Croft E, Kulic D (2008) Measuring the anthropomorphism, animacy, likeability, perceived intelligence and perceived safety of robots. In: Metrics for HRI workshop, technical report, vol 471, pp 37–44, Citeseer
Breazeal C, Scassellati B (1999) A context-dependent attention system for a social robot. rn, vol 255, p 3
Sidner CL, Dzikovska M (2005) A first experiment in engagement for human–robot interaction in hosting activities. In: Advances in natural multimodal dialogue systems. Springer, pp 55–76
Pollack ME, Brown L, Colbry D, Orosz C, Peintner B, Ramakrishnan S, Engberg S, Matthews JT, Dunbar-Jacob J, McCarthy CE et al (2002) Pearl: a mobile robotic assistant for the elderly. In: AAAI workshop on automation as eldercare, pp 85–91
Ishiguro H, Ono T, Imai M, Maeda T, Kanda T, Nakatsu R (2001) Robovie: an interactive humanoid robot. Ind Robot Int J 28(6):498–504
Billard A (2003) Robota: clever toy and educational tool. Robot Autonom Syst 42(3):259–269
Kidd CD, Taggart W, Turkle S (2006) A sociable robot to encourage social interaction among the elderly. In: Robotics and automation, 2006. ICRA 2006. IEEE international conference on proceedings 2006, IEEE, pp 3972–3976
Welch KC, Lahiri U, Warren Z, Sarkar N (2010) An approach to the design of socially acceptable robots for children with autism spectrum disorders. Int J Soc Robot 2(4):391–403
Fujimoto I, Matsumoto T, De Silva PRS, Kobayashi M, Higashi M (2011) Mimicking and evaluating human motion to improve the imitation skill of children with autism through a robot. Int J Soc Robot 3(4):349–357
Schiavone G, Formica D, Taffoni F, Campolo D, Guglielmelli E, Keller F (2011) Multimodal ecological technology: from childs social behavior assessment to child-robot interaction improvement. Int JSoc Robot 3(1):69–81
Cabibihan J-J, Javed H, Ang M Jr, Aljunied SM (2013) Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. Int J Soc Robot 5:1–26
Powers A, Kiesler S (2006) The advisor robot: tracing people’s mental model from a robot’s physical attributes. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction. ACM, pp 218–225
Foster ME, Gaschler A, Giuliani M, Isard A, Pateraki M, Petrick R, (2012) Two people walk into a bar: Dynamic multi-party social interaction with a robot agent. In: Proceedings of the 14th ACM international conference on multimodal interaction. ACM, pp 3–10
Darpa cognitive agent that learns and organizes (calo) project
Tur G, Stolcke A, Voss L, Peters S, Hakkani-Tur D, Dowding J, Favre B, Fernández R, Frampton M, Frandsen M et al (2010) The calo meeting assistant system. IEEE Trans Audio Speech Lang Process 18(6):1601–1611
Al Moubayed S, Beskow J, Skantze G, Granström B (2012) Furhat: a back-projected human-like robot head for multiparty human–machine interaction. In Cognitive behavioural systems. Springer, pp 114–130
Harris J, Sharlin E (2011) Exploring the affect of abstract motion in social human–robot interaction. In: Robot–Man IEEE, pp 441–448
Saerbeck M, Bartneck C (2010) Perception of affect elicited by robot motion. In: Proceedings of the 5th ACM/IEEE international conference on Human–robot interaction. IEEE Press, pp 53–60
Beck A, Hiolle A, Mazel A, Cañamero L (2010) Interpretation of emotional body language displayed by robots. In: Proceedings of the 3rd international workshop on affective interaction in natural environments. ACM, pp 37–42
Ham J, Bokhorst R, Cabibihan J (2011) The influence of gazing and gestures of a storytelling robot on its persuasive power. In: International conference on social robotics
Delaborde A, Devillers L (2010) Use of nonverbal speech cues in social interaction between human and robot: emotional and interactional markers. In: Proceedings of the 3rd international workshop on affective interaction in natural environments. ACM, pp 75–80
Yu L, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 856–863
Hall MA (1999) Correlation-based feature selection for machine learning. University of Waikato, Hamilton
Werner K, Oberzaucher J, Werner F (2012) Evaluation of human robot interaction factors of a socially assistive robot together with older people. In: Complex, intelligent and software intensive systems (CISIS), conference on 2012 sixth international. IEEE, pp 455–460
Nadine robot (2015)
Tahir Y, Chakraborty D, Maszczyk T, Dauwels S, Dauwels J, Thalmann N, and Thalmann D (2015) Real-time sociometrics from audio-visual features for two-person dialogs. In: 2015 IEEE international conference on digital signal processing (DSP). IEEE, pp 823–827
Acknowledgements
This research is supported by the BeingTogether Centre, a collaboration between Nanyang Technological University (NTU) Singapore and University of North Carolina (UNC) at Chapel Hill. The BeingTogether Centre is supported by the National Research Foundation, Prime Ministers Office, Singapore under its International Research Centres in Singapore Funding Initiative.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tahir, Y., Dauwels, J., Thalmann, D. et al. A User Study of a Humanoid Robot as a Social Mediator for Two-Person Conversations. Int J of Soc Robotics 12, 1031–1044 (2020). https://doi.org/10.1007/s12369-018-0478-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12369-018-0478-3