Skip to main content
Log in

A User Study of a Humanoid Robot as a Social Mediator for Two-Person Conversations

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

In this work we have enhanced the perception of a humanoid robot by integrating it with a social state estimation system. We present a user study of the humanoid Nao robot as a social mediator, comprising two sets of experiments. In the first sets of experiments, the participants rate their understanding of feedback messages delivered via the Nao robot. They also assess two modalities to deliver the feedback: audio only and audio combined with gestures. In almost all cases there is an improvement of 10% or more when audio and gesture modalities are combined to deliver feedback messages. For the second sets of experiments the sociofeedback system was integrated with the Nao robot. The participants engage in two-person scenario-based conversations while the Nao robot acts as a mediator. The sociofeedback system analyzes the conversations and provides feedback via Nao. Subsequently, the participants assess the received sociofeedback with respect to various aspects, including its content, appropriateness, and timing. Participants also evaluate their overall perception of Nao as social mediator via the Godspeed questionnaire. The results indicate that the social feedback system is able to detect the social scenario with 93.8% accuracy and that Nao can be effectively used to provide sociofeedback in discussions. The results of this paper pave the way to natural human–robot interactions for social mediators in multi-party dialog systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Fong T, Nourbakhsh I, Dautenhahn K (2003) A survey of socially interactive robots. Robot Auton Syst 42(3):143–166

    Article  Google Scholar 

  2. Li H, Cabibihan J-J, Tan YK (2011) Towards an effective design of social robots. Int J Soc Robot 3(4):333–335

    Article  Google Scholar 

  3. Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2010) A communication robot in a shopping mall. IEEE Trans Robot 26(5):897–913

    Article  Google Scholar 

  4. Thrun S, Bennewitz M, Burgard W, Cremers AB, Dellaert F, Fox D, Hahnel D, Rosenberg C, Roy N, Schulte J, Schulz D (1999) Minerva: a second-generation museum tour-guide robot. In: Proceedings of IEEE international conference on robotics and automation (ICRA 99)

  5. Tanaka F, Cicourel A, Movellan JR (2007) Socialization between toddlers and robots at an early childhood education center. Proc Ntl Acad Sci 104:17954–17958

    Article  Google Scholar 

  6. Graf B, Parlitz C, Hägele M (2009) Robotic home assistant care-o-bot\(^{\textregistered }\) 3 product vision and innovation platform. HCI 2:312–320

    Google Scholar 

  7. Williams K, Breazeal C (2012) A reasoning architecture for human-robot joint tasks using physics-, social-, and capability-based logic. In: IROS, pp 664–671

  8. François D, Polani D, Dautenhahn K (2008) Towards socially adaptive robots: a novel method for real time recognition of human-robot interaction styles. In Humanoids, pp 353–359

  9. Papadopoulos F, Dautenhahn K, Ho WC (2012) Exploring the use of robots as social mediators in a remote human–human collaborative communication experiment. Paladyn 3(1):1–10

    Google Scholar 

  10. Dautenhahn K (2007) Socially intelligent robots: dimensions of human–robot interaction. Phil Trans R Soc B Biol Sci 362(1480):679–704

    Article  Google Scholar 

  11. Bohus D, Horvitz E (2009) Dialog in the open world: platform and applications. In: Proceedings of the 2009 international conference on multimodal interfaces, ACM, pp 31–38

  12. Salah AA, Gevers T, Sebe N, Vinciarelli A (2010) Human behavior understanding. In: First international workshop, HBU 2010, Istanbul, Turkey, August 22, 2010, Proceedings, vol 6219. Springer

  13. Salah AA, Ruiz-del Solar J, Meriçli Ç, Oudeyer P-Y (2012) Human behavior understanding for robotics. In: Human behavior understanding. Springer, pp 1–16

  14. Rasheed U, Tahir Y, Dauwels S, Dauwels J (2013) Real-time comprehensive sociometrics for two-person dialogs. In: Human behavior understanding. Springer, pp 196–208

  15. Pentland AS (2010) Honest signals. MIT press, Cambridge

    Google Scholar 

  16. Tahir Y, Rasheed U, Dauwels S, Dauwels J, Thalmann N, Thalmann D (2013) Nao as social mediator: a user study. In: Robots in public spaces:towards multi-party, short-term, dynamic human–robot interaction, Springer

  17. Bartneck C, Croft E, Kulic D (2008) Measuring the anthropomorphism, animacy, likeability, perceived intelligence and perceived safety of robots. In: Metrics for HRI workshop, technical report, vol 471, pp 37–44, Citeseer

  18. Breazeal C, Scassellati B (1999) A context-dependent attention system for a social robot. rn, vol 255, p 3

  19. Sidner CL, Dzikovska M (2005) A first experiment in engagement for human–robot interaction in hosting activities. In: Advances in natural multimodal dialogue systems. Springer, pp 55–76

  20. Pollack ME, Brown L, Colbry D, Orosz C, Peintner B, Ramakrishnan S, Engberg S, Matthews JT, Dunbar-Jacob J, McCarthy CE et al (2002) Pearl: a mobile robotic assistant for the elderly. In: AAAI workshop on automation as eldercare, pp 85–91

  21. Ishiguro H, Ono T, Imai M, Maeda T, Kanda T, Nakatsu R (2001) Robovie: an interactive humanoid robot. Ind Robot Int J 28(6):498–504

    Article  Google Scholar 

  22. Billard A (2003) Robota: clever toy and educational tool. Robot Autonom Syst 42(3):259–269

    Article  Google Scholar 

  23. Kidd CD, Taggart W, Turkle S (2006) A sociable robot to encourage social interaction among the elderly. In: Robotics and automation, 2006. ICRA 2006. IEEE international conference on proceedings 2006, IEEE, pp 3972–3976

  24. Welch KC, Lahiri U, Warren Z, Sarkar N (2010) An approach to the design of socially acceptable robots for children with autism spectrum disorders. Int J Soc Robot 2(4):391–403

    Article  Google Scholar 

  25. Fujimoto I, Matsumoto T, De Silva PRS, Kobayashi M, Higashi M (2011) Mimicking and evaluating human motion to improve the imitation skill of children with autism through a robot. Int J Soc Robot 3(4):349–357

    Article  Google Scholar 

  26. Schiavone G, Formica D, Taffoni F, Campolo D, Guglielmelli E, Keller F (2011) Multimodal ecological technology: from childs social behavior assessment to child-robot interaction improvement. Int JSoc Robot 3(1):69–81

    Article  Google Scholar 

  27. Cabibihan J-J, Javed H, Ang M Jr, Aljunied SM (2013) Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. Int J Soc Robot 5:1–26

    Article  Google Scholar 

  28. Powers A, Kiesler S (2006) The advisor robot: tracing people’s mental model from a robot’s physical attributes. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction. ACM, pp 218–225

  29. Foster ME, Gaschler A, Giuliani M, Isard A, Pateraki M, Petrick R, (2012) Two people walk into a bar: Dynamic multi-party social interaction with a robot agent. In: Proceedings of the 14th ACM international conference on multimodal interaction. ACM, pp 3–10

  30. Darpa cognitive agent that learns and organizes (calo) project

  31. Tur G, Stolcke A, Voss L, Peters S, Hakkani-Tur D, Dowding J, Favre B, Fernández R, Frampton M, Frandsen M et al (2010) The calo meeting assistant system. IEEE Trans Audio Speech Lang Process 18(6):1601–1611

    Article  Google Scholar 

  32. Al Moubayed S, Beskow J, Skantze G, Granström B (2012) Furhat: a back-projected human-like robot head for multiparty human–machine interaction. In Cognitive behavioural systems. Springer, pp 114–130

  33. Harris J, Sharlin E (2011) Exploring the affect of abstract motion in social human–robot interaction. In: Robot–Man IEEE, pp 441–448

  34. Saerbeck M, Bartneck C (2010) Perception of affect elicited by robot motion. In: Proceedings of the 5th ACM/IEEE international conference on Human–robot interaction. IEEE Press, pp 53–60

  35. Beck A, Hiolle A, Mazel A, Cañamero L (2010) Interpretation of emotional body language displayed by robots. In: Proceedings of the 3rd international workshop on affective interaction in natural environments. ACM, pp 37–42

  36. Ham J, Bokhorst R, Cabibihan J (2011) The influence of gazing and gestures of a storytelling robot on its persuasive power. In: International conference on social robotics

  37. Delaborde A, Devillers L (2010) Use of nonverbal speech cues in social interaction between human and robot: emotional and interactional markers. In: Proceedings of the 3rd international workshop on affective interaction in natural environments. ACM, pp 75–80

  38. Yu L, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 856–863

  39. Hall MA (1999) Correlation-based feature selection for machine learning. University of Waikato, Hamilton

    Google Scholar 

  40. Werner K, Oberzaucher J, Werner F (2012) Evaluation of human robot interaction factors of a socially assistive robot together with older people. In: Complex, intelligent and software intensive systems (CISIS), conference on 2012 sixth international. IEEE, pp 455–460

  41. Nadine robot (2015)

  42. Tahir Y, Chakraborty D, Maszczyk T, Dauwels S, Dauwels J, Thalmann N, and Thalmann D (2015) Real-time sociometrics from audio-visual features for two-person dialogs. In: 2015 IEEE international conference on digital signal processing (DSP). IEEE, pp 823–827

Download references

Acknowledgements

This research is supported by the BeingTogether Centre, a collaboration between Nanyang Technological University (NTU) Singapore and University of North Carolina (UNC) at Chapel Hill. The BeingTogether Centre is supported by the National Research Foundation, Prime Ministers Office, Singapore under its International Research Centres in Singapore Funding Initiative.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Justin Dauwels.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tahir, Y., Dauwels, J., Thalmann, D. et al. A User Study of a Humanoid Robot as a Social Mediator for Two-Person Conversations. Int J of Soc Robotics 12, 1031–1044 (2020). https://doi.org/10.1007/s12369-018-0478-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-018-0478-3

Keywords

Navigation