Skip to main content

Advertisement

SpringerLink
Log in

Evaluating the Engagement with Social Robots

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

To interact and cooperate with humans in their daily-life activities, robots should exhibit human-like “intelligence”. This skill will substantially emerge from the interconnection of all the algorithms used to ensure cognitive and interaction capabilities. While new robotics technologies allow us to extend such abilities, their evaluation for social interaction is still challenging. The quality of a human–robot interaction can not be reduced to the evaluation of the employed algorithms: we should integrate the engagement information that naturally arises during interaction in response to the robot’s behaviors. In this paper we want to show a practical approach to evaluate the engagement aroused during interactions between humans and social robots. We will introduce a set of metrics useful in direct, face to face scenarios, based on the behaviors analysis of the human partners. We will show how such metrics are useful to assess how the robot is perceived by humans and how this perception changes according to the behaviors shown by the social robot. We discuss experimental results obtained in two human-interaction studies, with the robots Nao and iCub respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Admoni H, Dragan A, Srinivasa SS, Scassellati B (2014) Deliberate delays during robot-to-human handovers improve compliance with gaze communication. In: Proceedings of the 2014 ACM/IEEE international conference on human–robot interaction, HRI ’14, pp 49–56

  2. Andry P, Blanchard A, Gaussier P (2011) Using the rhythm of nonverbal human-robot interaction as a signal for learning. IEEE Trans Auton Ment Dev 3(1):30–42

    Article  Google Scholar 

  3. Anzalone SM, Chetouani M (2013) Tracking posture and head movements of impaired people during interactions with robots. In: New trends in image analysis and processing-ICIAP 2013. Springer, Berlin, pp 41–49

  4. Anzalone SM, Ghidoni S, Menegatti E, Pagello E (2013) A multimodal distributed intelligent environment for a safer home. In: Intelligent autonomous systems 12. Springer, Berlin, pp 775–785

  5. Anzalone SM, Ivaldi S, Sigaud O, Chetouani M (2013) Multimodal people engagement with icub. In: Biologically inspired cognitive architectures 2012. Springer, Berlin, pp 59–64

  6. Anzalone SM, Tilmont E, Boucenna S, Xavier J, Jouen AL, Bodeau N, Maharatna K, Chetouani M, Cohen D (2014) How children with autism spectrum disorder behave and explore the 4-dimensional (spatial 3d+ time) environment during a joint attention induction task with a robot. Res Autism Spectr Disord 8(7):814–826

    Article  Google Scholar 

  7. Argall BD, Browning B, Veloso M (2011) Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot. Robot Auton Syst 59(3–4):243–255

    Article  Google Scholar 

  8. Baron-Cohen S (1997) Mindblindness: an essay on autism and theory of mind. MIT press, Cambridge

    Google Scholar 

  9. Bertenthal BI, Boyer TW, Han JM (2012) Social attention is not restricted to the eyes: pointing also automatically orients direction of attention. The Annual Meeting of the Psychonomic Society, Minneapolis, MN

  10. Boucenna S, Anzalone S, Tilmont E, Cohen D, Chetouani M (2014) Learning of social signatures through imitation game between a robot and a human partner. Auton Ment Dev IEEE Trans 6(3):213–225

  11. Boucenna S, Gaussier P, Andry P, Hafemeister L (2014) A robot learns the facial expressions recognition and face/non-face discrimination through an imitation game. Int J Soc Robot 6(4):633–652

    Article  Google Scholar 

  12. Boucenna S, Narzisi A, Tilmont E, Muratori F, Pioggia G, Cohen D, Chetouani M (2014) Interactive technologies for autistic children: a review. Cogn Comput 6(4):1–19

    Article  Google Scholar 

  13. Breazeal C (2003) Toward social robots. Robot Auton Syst 42:167–175

    Article  Google Scholar 

  14. Breazeal C, Kidd CD, Thomaz AL, Hoffman G, Berlin M (2005) Effects of nonverbal communication on efficiency and robustness in human–robot teamwork. In: IEEE/RSJ international conference on intelligent robots and systems, pp 383–388

  15. Breazeal CL (2000) Sociable machines: expressive social exchange between humans and robots. Ph.D. thesis, Massachusetts Institute of Technology

  16. Breazeal CL (2004) Designing sociable robots. MIT press, Cambridge

    Google Scholar 

  17. Brethes L, Menezes P, Lerasle F, Hayet J (2004) Face tracking and hand gesture recognition for human–robot interaction. In: IEEE international conference on robotics and automation, vol 2. IEEE, pp 1901–1906

  18. Brick T, Scheutz M (2007) Incremental natural language processing for hri. In: ACM/IEEE international conference on human–robot interaction, HRI ’07. ACM, New York, pp 263–270

  19. Bruner J, Feldman C (1993) Theories of mind and the problems of autism. In: Baron-Cohen SE, Tager-Flusberg HE, Cohen DJ (eds) Understanding other minds: perspectives from autism. Oxford University Press

  20. Cantor N, Kihlstrom JF (1987) Personality and social intelligence. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  21. Cantrell R, Scheutz M, Schermerhorn P, Wu X (2010) Robust spoken instruction understanding for hri. In: 5th ACM/IEEE international conference on human–robot interaction, pp 275–282

  22. Choi BC, Pak AW (2005) A catalog of biases in questionnaires. Prev Chronic Dis 2(1):A13

    Google Scholar 

  23. Crespi N, Molina B, Palau C et al (2011) Qoe aware service delivery in distributed environment. In: Advanced information networking and applications (WAINA), 2011 IEEE Workshops of International Conference on, pp 837–842. IEEE

  24. Cristinacce D, Cootes T (2006) Feature detection and tracking with constrained local models. In: Proceedings of British machine vision conference, vol 3. pp 929–938

  25. Dautenhahn K (1995) Getting to know each otherartificial social intelligence for autonomous robots. Robot Auton Syst 16(2):333–356

    Article  Google Scholar 

  26. Dautenhahn K (2007) Socially intelligent robots: dimensions of human–robot interaction. Philos Trans R Soc B 362(1480):679–704

    Article  MATH  Google Scholar 

  27. Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C, Viaux S, Cohen D (2012) Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans Affect Comput 3(3):349–365

    Article  Google Scholar 

  28. Delaherche E, Dumas G, Nadel J, Chetouani M (2014) Automatic measure of imitation during social interaction: a behavioral and hyperscanning-eeg benchmark. Pattern Recognit Lett. doi:10.1016/j.patrec.2014.09.002

  29. Ekman P, Friesen WV (1981) The repertoire of nonverbal behavior: categories, origins, usage, and coding. In: Kendon A, Sebeok TA, Umiker-Sebeok J (eds) Nonverbal communication, interaction, and gesture: selections from Semiotica. Walter de Gruyter, pp 57–106

  30. Fischer K, Lohan K, Saunders J, Nehaniv C, Wrede B, Rohlfing K (2013) The impact of the contingency of robot feedback on hri. In: Collaboration Technologies and Systems (CTS), 2013 international conference on. IEEE, pp 210–217

  31. Fong T, Nourbakhsh I, Dautenhahn K (2003) A survey of socially interactive robots. Robot Auton Syst 42:143–166

    Article  Google Scholar 

  32. Furnham A (1986) Response bias, social desirability and dissimulation. Personality Individ Differ 7(3):385–400

    Article  Google Scholar 

  33. Ghidoni S, Anzalone SM, Munaro M, Michieletto S, Menegatti E (2014) A distributed perception infrastructure for robot assisted living. Robot Auton Syst 62(9):1316–1328

    Article  Google Scholar 

  34. Hall J, Tritton T, Rowe A, Pipe A, Melhuish C, Leonards U (2014) Perception of own and robot engagement in human–robot interactions and their dependence on robotics knowledge. Robot Auton Syst 62(3):392–399

    Article  Google Scholar 

  35. Harris TK, Banerjee S, Rudnicky AI (2005) Heterogeneous multi-robot dialogues for search tasks. In: Proceedings of the AAAI spring symposium intelligence, Citeseer

  36. Ishiguro H (2006) Interactive humanoids and androids as ideal interfaces for humans. In: Proceedings of the 11th international conference on intelligent user interfaces. ACM, New York, pp. 2–9

  37. Ishiguro H (2007) Android science. In: Robotics research. Springer, Berlin, pp 118–127

  38. Ishii R, Shinohara Y, Nakano T, Nishida T (2011) Combining multiple types of eye-gaze information to predict users conversational engagement. 2nd workshop on eye gaze on intelligent human machine interaction

  39. Ivaldi S, Anzalone SM, Rousseau W, Sigaud O, Chetouani M (2014) Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement. Front Neurorobotics 8(5):1–23

    Google Scholar 

  40. Ivaldi S, Nguyen S, Lyubova N, Droniou A, Padois V, Filliat D, Oudeyer PY, Sigaud O (2014) Object learning through active exploration. IEEE Trans Auton Ment Dev 6(1):56–72

    Article  Google Scholar 

  41. Kamide H, Mae Y, Kawabe K, Shigemi S, Hirose M, Arai T (2012) New measurement of psychological safety for humanoid. In: Proceedings of the seventh annual ACM/IEEE international conference on human–robot interaction. ACM, New York, pp. 49–56

  42. Kamide H, Mae Y, Takubo T, Ohara K, Arai T (2010) Development of a scale of perception to humanoid robots: Pernod. In: Intelligent robots and systems (IROS), 2010 IEEE/RSJ International Conference on. IEEE, pp 5830–5835

  43. Kaplan F, Hafner V (2004) The challenges of joint attention. Lund University Cognitive Studies, Lund

  44. Kihlstrom JF, Cantor N (2000) Social intelligence. Handb Intell 2:359–379

  45. Kulic D, Croft EA (2007) Affective state estimation for human–robot interaction. Robot IEEE Trans 23(5):991–1000

    Article  Google Scholar 

  46. Laghari KUR, Connelly K (2012) Toward total quality of experience: a qoe model in a communication ecosystem. Commun Mag IEEE 50(4):58–65

    Article  Google Scholar 

  47. Lee C, Lesh N, Sidner CL, Morency LP, Kapoor A, Darrell T (2004) Nodding in conversations with a robot. In: CHI’04 extended abstracts on human factors in computing systems. ACM, New York, pp 785–786

  48. Lee J, Chao C, Bobick AF, Thomaz AL (2012) Multi-cue contingency detection. Int J Soc Robot 4(2):147–161

    Article  Google Scholar 

  49. Lemaignan S, Fink J, Dillenbourg P (2014) The dynamics of anthropomorphism in robotics. In: Proceedings of the 2014 ACM/IEEE international conference on human–robot interaction. ACM, New york, pp 226–227

  50. Miller PH (2010) Theories of developmental psychology. Macmillan, London

    Google Scholar 

  51. Mower E, Feil-Seifer DJ, Mataric MJ, Narayanan S (2007) Investigating implicit cues for user state estimation in human–robot interaction using physiological measurements. In: The 16th IEEE international symposium on robot and human interactive communication, 2007 (RO-MAN 2007). IEEE, pp 1125–1130

  52. Natale L, Nori F, Metta G, Fumagalli M, Ivaldi S, Pattacini U, Randazzo M, Schmitz A, Sandini G (2012) Intrinsically motivated learning in natural and artificial systems, chap. The iCub platform: a tool for studying intrinsically motivated learning. Springer, Berlin

  53. Nickerson RS (1998) Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol 2(2):175

    Article  Google Scholar 

  54. Obhi SS, Sebanz N (2011) Moving together: toward understanding the mechanisms of joint action. Exp Brain Res 211(3):329–336

    Article  Google Scholar 

  55. O’Brien HL, Toms EG (2008) What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci Technol 59(6):938–955

    Article  Google Scholar 

  56. Payne SL (1951) The art of asking questions. Princeton University Press, Princeton

  57. Raake A, Egger S (2014) Quality and quality of experience. In: Quality of experience. Springer, Berlin, pp 11–33

  58. Rich C, Ponsler B, Holroyd A, Sidner CL (2010) Recognizing engagement in human–robot interaction. In: Proceedings of ACM/IEEE international conference on human–robot interaction (HRI). ACM Press, New York, pp 375–382

  59. Rousseau W, Anzalone SM, Chetouani M, Sigaud O, Ivaldi S (2013) Learning object names through shared attention. In: IROS-Int. workshop on developmental social robotics. pp 1–6

  60. Sanghvi J, Castellano G, Leite I, Pereira A, McOwan PW, Paiva A (2011) Automatic analysis of affective postures and body motion to detect engagement with a game companion. In: 6th ACM/IEEE international conference on human–robot interaction. ACM, New York, pp 305–311

  61. Scassellati B (2005) Quantitative metrics of social response for autism diagnosis. In: IEEE international workshop on robot and human interactive communication, 2005 (ROMAN 2005). IEEE, pp 585–590

  62. Scassellati B (2007) How social robots will help us to diagnose, treat, and understand autism. In: Robotics research. Springer, Berlin, pp 552–563

  63. Short E, Hart J, Vu M, Scassellati B (2010) No fair!! an interaction with a cheating robot. In: 5th ACM/IEEE international conference on human–robot interaction. ACM, New York, pp 219–226

  64. Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124

    Article  Google Scholar 

  65. Sidner C, Lee C, Kidds C, Lesh N, Rich C (2005) Explorations in engagement for humans and robots. Artif Intell 166(1):140–164

    Article  Google Scholar 

  66. Sidner CL, Kidd CD, Lee C, Lesh N (2004) Where to look: a study of human–robot engagement. In: Proceedings of the 9th international conference on intelligent user interfaces. ACM, New York, pp 78–84

  67. Tapus A, Mataric M, Scasselati B (2007) Socially assistive robotics [grand challenges of robotics]. IEEE Robot Autom Mag 14(1):35–42

    Article  Google Scholar 

  68. Thorndike EL (1920) Intelligence and its uses. Harper’s magazine, New York

    Google Scholar 

  69. Tomasello M (1995) Joint attention as social cognition. In: Moore C, Dunham PJ (eds) Joint attention: its origins and role in development. Lawrence Erlbaum Associates, Inc. pp 103–130

  70. Tomasello M, Farrar MJ (1986) Joint attention and early language. Child Dev 57:1454–1463

    Article  Google Scholar 

  71. Vaussard F, Fink J, Bauwens V, Retornaz P, Hamel D, Dillenbourg P, Mondada F (2014) Lessons learned from robotic vacuum cleaners entering the home ecosystem. Robot Auton Syst 62(3):376–391

    Article  Google Scholar 

  72. Vázquez M, May A, Steinfeld A, Chen WH (2011) A deceptive robot referee in a multiplayer gaming environment. In: International conference on Collaboration Technologies and Systems (CTS), 2011. IEEE, pp 204–211

  73. Vernon PE (1933) Some characteristics of the good judge of personality. J Soc Psychol 4(1):42–57

    Article  Google Scholar 

  74. Vinciarelli A, Pantic M, Bourlard H (2009) Social signal processing: survey of an emerging domain. Image Vis Comput 27(12):1743–1759

    Article  Google Scholar 

  75. Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D’Errico F, Schröder M (2012) Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affect Comput 3(1):69–87

    Article  Google Scholar 

  76. Weisman O, Delaherche E, Rondeau M, Chetouani M, Cohen D, Feldman R (2013) Oxytocin shapes parental motion during father-infant interaction. Biol Lett. doi:10.1098/rsbl.2013.0828

  77. Yannakakis GN, Hallam J, Lund HH (2008) Entertainment capture through heart rate activity in physical interactive playgrounds. User Model User-Adapt Inter 18(1–2):207–243

    Article  Google Scholar 

  78. Zhao S (2003) Toward a taxonomy of copresence. Presence 12(5):445–455

    Article  Google Scholar 

  79. Zhao W, Chellappa R, Phillips PJ, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Investissiments d’Avenir program (SMART ANR-11-IDEX-0004-02) through Project EDHHI/SMART, the ANR Project Pramad, and by the European Commission, within the projects CoDyCo (FP7-ICT-2011-9, No. 600716) and and Michelangelo Project (FP7-ICT No.288241).

Author information

Authors and Affiliations

  1. Sorbonne Universités, UPMC Université Paris 06, ISIR UMR7222, Paris, France

    Salvatore M. Anzalone & Mohamed Chetouani

  2. CNRS, Institut des Systèmes Intelligents et de Robotique UMR7222, Paris, France

    Salvatore M. Anzalone & Mohamed Chetouani

  3. ETIS Laboratory, UMR CNRS 8051, The University of Cergy-Pontoise, ENSEA, Cergy-Pontoise, France

    Sofiane Boucenna

  4. Inria, 54600, Villers-lès-Nancy, France

    Serena Ivaldi

  5. Intelligent Autonomous Systems Lab, TU Darmstadt, Darmstadt, Germany

    Serena Ivaldi

  6. Université de Lorraine, Loria, UMR n 7503, 54500, Vanduvre-lès-Nancy, France

    Serena Ivaldi

  7. CNRS, Loria, UMR n 7503, 54500, Vandoeuvre-lès-Nancy, France

    Serena Ivaldi

Authors
  1. Salvatore M. Anzalone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sofiane Boucenna
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Serena Ivaldi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Chetouani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salvatore M. Anzalone.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anzalone, S.M., Boucenna, S., Ivaldi, S. et al. Evaluating the Engagement with Social Robots. Int J of Soc Robotics 7, 465–478 (2015). https://doi.org/10.1007/s12369-015-0298-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-015-0298-7

Keywords

Search

Navigation