Tom Giraud
ABSTRACT
New technologies for autism focus on the training of either social skills or motor skills, but not both. Such a dichotomy omits a wide range of joint action tasks that require the coordination of two persons (e.g. moving a heavy furniture). The training of these physical tasks performed in dyad has great potential to foster inclusiveness while having an impact on both social and motor skills. In this paper, we present the design of a tangible and virtual interactive system for the training of children with Autism Spectrum Disorder (ASD) in performing joint actions. The proposed system is composed of a virtual character projected onto a surface on which a tangible object is magnetized: both the user and the virtual character hold the object, thus simulating a joint action. We report and discuss preliminary results of a field training study, which shows the potential of the interactive system.
- Michael J. Richardson, Kerry L. Marsh, and Reuben M. Baron. Judging and actualizing intrapersonal and interpersonal affordances. J. Exp. Psychol. Hum. Percept. Perform., vol. 33, no 4, p. 845-859, 2007. https://doi.org/10.1037/0096-1523.33.4.845Google Scholar
Cross Ref
- Richard C. Schmidt and Michael J. Richardson. Dynamics of Interpersonal Coordination. In Coordination: Neural, Behavioral and Social Dynamics, A. Fuchs and V. K. Jirsa, Ed. Springer Berlin Heidelberg, 2008, p. 281-308. https://doi.org/10.1007/978-3-540-74479-5_14Google ScholarCross Ref
- Jacqueline Nadel. Perception–action coupling and imitation in autism spectrum disorder. Dev. Med. Child Neurol., vol. 57, no s2, p. 55-58, 2015. https://doi.org/10.1111/dmcn.12689Google ScholarCross Ref
- Ouriel Grynszpan, Patrice L. (Tamar) Weiss, Fernando Perez-Diaz, and Eynat Gal. Innovative technology-based interventions for autism spectrum disorders: A meta-analysis. Autism, vol. 18, no 4, p. 346-361, 2014. https://doi.org/10.1177/1362361313476767Google ScholarCross Ref
- Bertram O. Ploog, Alexa Scharf, DeShawn Nelson, and Patricia J. Brooks. Use of Computer-Assisted Technologies (CAT) to enhance social, communicative, and language development in children with autism spectrum disorders. J. Autism Dev. Disord., vol. 43, no 2, p. 301-322, 2013. https://doi.org/10.1007/s10803-012-1571-3Google ScholarCross Ref
- Simon Provoost, Ho Ming Lau, Jeroen Ruwaard, and Heleen Riper. Embodied Conversational Agents in Clinical Psychology: A Scoping Review. J. Med. Internet Res., vol. 19, no 5, p. e151, 2017. https://doi.org/10.2196/jmir.6553Google ScholarCross Ref
- Gill A. Francis, William Farr, Silvana Mareva, and Jenny L. Gibson. Do Tangible User Interfaces promote social behaviour during free play? A comparison of autistic and typically-developing children playing with passive and digital construction toys. Res. Autism Spectr. Disord., vol. 58, p. 68-82, 2019. https://doi.org/10.1016/j.rasd.2018.08.005Google ScholarCross Ref
- Eva Hornecker and Jacob Buur. Getting a grip on tangible interaction: a framework on physical space and social interaction. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montreal, Quebec, Canada, 2006, p. 437–446. https://doi.org/10.1145/1124772.1124838Google ScholarDigital Library
- Ylva Fernaeus, Jakob Tholander, and Martin Jonsson. Towards a New Set of Ideals: Consequences of the Practice Turn in Tangible Interaction. Proceedings of the 2Nd International Conference on Tangible and Embedded Interaction, New York, NY, USA, 2008, p. 223–230. https://doi.org/10.1145/1347390.1347441Google ScholarDigital Library
- Victoria Tam, Mirko Gelsomini, and Franca Garzotto. Polipo: a Tangible Toy for Children with Neurodevelopmental Disorders. Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction, Yokohama, Japan, 2017, p. 11–20. https://doi.org/10.1145/3024969.3025006Google ScholarDigital Library
- Amani I. Soysa and Abdullah Al Mahmud. Tangible Play and Children with ASD in Low-Resource Countries: A Case Study. Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, Sydney NSW, Australia, 2020, p. 219–225. https://doi.org/10.1145/3374920.3374951Google ScholarDigital Library
- Daniel B. Legoff and Michael Sherman. Long-term outcome of social skills intervention based on interactive LEGO© play. Autism, vol. 10, no 4, p. 317-329, 2006. https://doi.org/10.1177/1362361306064403Google ScholarCross Ref
- William Farr, Nicola Yuill, and Steve Hinske. An augmented toy and social interaction in children with autism. IJART, vol. 5, p. 104-125, 2012. https://doi.org/10.1504/ijart.2012.046270Google ScholarCross Ref
- Sergi Jordà, Gunter Geiger, Marcos Alonso, and Martin Kaltenbrunner. The reacTable: Exploring the Synergy Between Live Music Performance and Tabletop Tangible Interfaces. Proceedings of the 1st International Conference on Tangible and Embedded Interaction, New York, NY, USA, 2007, p. 139–146. https://doi.org/10.1145/1226969.1226998Google ScholarDigital Library
- Lilia Villafuerte, Milena Markova, and Sergi Jorda. Acquisition of Social Abilities Through Musical Tangible User Interface: Children with Autism Spectrum Condition and the Reactable. CHI ’12 Extended Abstracts on Human Factors in Computing Systems, New York, NY, USA, 2012, p. 745–760. https://doi.org/10.1145/2212776.2212847Google ScholarDigital Library
- Anne M. Piper, Eileen O'Brien, Meredith R. Morris, and Terry Winograd. SIDES: A Cooperative Tabletop Computer Game for Social Skills Development. Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, New York, NY, USA, 2006, p. 1–10. https://doi.org/10.1145/1180875.1180877Google ScholarDigital Library
- Paul Dietz and Darren Leigh. DiamondTouch: A Multi-user Touch Technology. Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA, 2001, p. 219–226. https://doi.org/10.1007/s00146-009-0199-0Google ScholarDigital Library
- Eynat Gal, Nirit Bauminger, Dina Goren-Bar, Fabio Pianesi, Oliviero Stock, Massimo Zancanaro and Patrice L. (Tamar) Weiss. Enhancing social communication of children with high-functioning autism through a co-located interface. AI Soc., vol. 24, no 1, p. 75,2009. https://doi.org/10.1007/s00146-009-e0199-0Google ScholarDigital Library
- Alberto Battocchi, Ayelet Ben‐Sasson, Gianluca Esposito, Eynat Gal, Fabio Pianesi, Daniel Tomasini, Paola Venuti, Patrice Weiss and Massimo Zancanaro. Collaborative puzzle game: a tabletop interface for fostering collaborative skills in children with autism spectrum disorders. J. Assist. Technol., vol. 4, no 1, p. 4-13, 2010. https://doi.org/10.5042/jat.2010.0040Google ScholarCross Ref
- Leonardo Giusti, Massimo Zancanaro, Eynat Gal, and Patrice L. (Tamar) Weiss. Dimensions of Collaboration on a Tabletop Interface for Children with Autism Spectrum Disorder. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2011, p. 3295–3304. https://doi.org/10.1145/1978942.1979431Google ScholarDigital Library
- Louanne E. Boyd, Kathryn E. Ringland, Oliver L. Haimson, Helen Fernandez, Maria Bistarkey, and Gillian R. Hayes. Evaluating a Collaborative iPad Game's Impact on Social Relationships for Children with Autism Spectrum Disorder. ACM Trans Access Comput, vol. 7, no 1, p. 3:1–3:18, 2015. https://doi.org/10.1145/2751564Google ScholarDigital Library
- J. Wade, A. Sarkar, A. Swanson, A. Weitlauf, Z. Warren, and N. Sarkar. Process Measures of Dyadic Collaborative Interaction for Social Skills Intervention in Individuals with Autism Spectrum Disorders. ACM Trans. Access. Comput., vol. 10, p. 1-19, 2017. https://doi.org/10.1145/3107925Google ScholarDigital Library
- Lian Zhang, Qiang Fu, Amy Swanson, Amy S. Weitlauf, Zachary Warren, and Nilanjan Sarkar. Design and Evaluation of a Collaborative Virtual Environment (CoMove) for Autism Spectrum Disorder Intervention. TACCESS, vol. 11, p. 11-11, 2018, https://doi.org/10.1145/3209687Google ScholarDigital Library
- Melanie Jouaiti and Patrick Henaff. Robot-Based Motor Rehabilitation in Autism: A Systematic Review. Int. J. Soc. Robot., vol. 11, no 5, p. 753-764, 2019. https://doi.org/10.1007/s12369-019-00598-9Google ScholarCross Ref
- Kerstin Dautenhahn and Aude Billard. Games Children with Autism Can Play with Robota, a Humanoid Robotic Doll. Universal Access and Assistive Technology, London, 2002, p. 179-190. https://doi.org/10.1007/978-1-4471-3719-1_18Google ScholarCross Ref
- Zhi Zheng, Eric M. Young, Amy R. Swanson, Amy S. Weitlauf, Zachary E. Warren, and Nilanjan Sarkar. Robot-Mediated Imitation Skill Training for Children With Autism. IEEE Trans. Neural Syst. Rehabil. Eng., vol. 24, no 6, p. 682-691, 2016. https://doi.org/10.1109/TNSRE.2015.2475724Google ScholarCross Ref
- Aude Billard, Ben Robins, Jacqueline Nadel, and Kerstin Dautenhahn. Building Robota, a Mini-Humanoid Robot for the Rehabilitation of Children with Autism. Assist. Technol., vol. 19, no 1, p. 37-49, 2007. https://doi.org/10.1080/10400435.2007.10131864Google ScholarCross Ref
- Audrey Duquette, François Michaud, and Henri Mercier. Exploring the use of a mobile robot as an imitation agent with children with low-functioning autism. Auton. Robots, vol. 24, no 2, p. 147-157, 2008. https://doi.org/10.1007/s10514-007-9056-5Google ScholarDigital Library
- Sara Ali, Faisal Mehmood, Darren Dancey, Yasar Ayaz, Muhammad Jawad Khan, Noman Naseer, Rita De Cassia Amadeu, Haleema Sadia, and Raheel Nawaz. An Adaptive Multi-Robot Therapy for Improving Joint Attention and Imitation of ASD Children. IEEE Access, vol. 7, p. 81808-81825, 2019. https://doi.org/10.1109/ACCESS.2019.2923678Google ScholarCross Ref
- Sylvie Serret, Stephanie Hun, Galina Iakimova, Jose Lozada, Margarita Anastassova, Andreia Santos, Stephanie Vesperini and Florence Askenazy. Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new Serious game: a pilot study. Mol. Autism, vol. 5, no 1, p. 37, 2014. https://doi.org/10.1186/2040-2392-5-37Google ScholarCross Ref
- Charline Grossard, Stéphanie Hun, Arnaud Dapogny, Estelle Juille, Fanny Hamel2, Heidy Jean-Marie, Jérémy Bourgeois, Hugues Pellerin, Pierre Foulon, Sylvie Serret, Ouriel Grynszpan, Kevin Bailly and David Cohen. Teaching Facial Expression Production in Autism: The Serious Game JEMImE. Creat. Educ., vol. 10, no 11, p. 2347, 2019. https://doi.org/10.4236/ce.2019.1011167Google ScholarCross Ref
- Matthieu Courgeon, Gilles Rautureau, Jean-Claude Martin, and Ouriel Grynszpan. Joint Attention Simulation Using Eye-Tracking and Virtual Humans. IEEE Trans. Affect. Comput., vol. 5, no 3, p. 238-250, 2014. https://doi.org/10.1109/TAFFC.2014.2335740Google ScholarCross Ref
- Sara Bernardini, Kaska Porayska-Pomsta, and Tim J. Smith. ECHOES: An intelligent serious game for fostering social communication in children with autism. Inf. Sci., vol. 264, p. 41-60, 2014. https://doi.org/10.1016/j.ins.2013.10.027Google ScholarDigital Library
- Erik Marchi, Bjorn Schuller, Alice Baird, Simon Baron-Cohen, Amandine Lassalle, Helen O'Rielly, Delia Pigat, Peter Robinson, Ian Davies, Tadas Baltrusaitis, Ofer Golan, Shimrit Fridenson-Hayo, Shahar Tal, Shai Newman, Noga Meir-Goren, Antonio Camurri, Stefano Piana, Sven Bolte, Metin Sezgin, Nese Alyuz, Agnieszka Rynkiewicz, Aurelie Baranger. The ASC-Inclusion Perceptual Serious Gaming Platform for Autistic Children. IEEE Trans. Games, vol. 11, no 4, p. 328-339, 2019. https://doi.org/10.1109/TG.2018.2864640Google ScholarCross Ref
- Charline Grossard, Ouriel Grynspan, Sylvie Serret, Anne-Lise Jouen, Kevin Bailly, and David Cohen. Serious games to teach social interactions and emotions to individuals with autism spectrum disorders (ASD). Comput. Educ., vol. 113, p. 195-211, 2017. https://doi.org/10.1016/j.compedu.2017.05.002Google ScholarCross Ref
- Jacqueline Nadel, How Imitation Boosts Development: In Infancy and Autism Spectrum Disorder. Oxford University Press, 2014.Google Scholar
- Gerardo Herrera, Xavier Casas, Javier Sevilla, Luis Rosa, Carlos Pardo, Javier Plaza, Rita Jordan and Sylvain Le Groux. Pictogram Room: Natural Interaction Technologies to Aid in the Development of Children with Autism. Annu. Clin. Health Psychol., vol. 08, p. 39-44, 2012.Google Scholar
- Gerardo Herrera, Patricia Perez-Fuster, and Gael Poli, . Pictogram Room: son efficacite dans le trouble du spectre de l'autisme (TSA). Enfance, vol. N° 1, no 1, p. 31-50, 2018. https://doi.org/10.3917/enf2.181.0031Google ScholarCross Ref
- Elinor Ochs and Olga Solomon. Autistic Sociality. Ethos, vol. 38, no 1, p. 69‑92, 2010. https://doi.org/10.1111/j.1548-1352.2009.01082.xGoogle ScholarCross Ref
- Michael J. Muller. Participatory design: the third space in HCI. The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, USA: L. Erlbaum Associates Inc., 2007, p. 1051–1068.Google Scholar
- Laura Benton and Hilary Johnson. Widening participation in technology design: A review of the involvement of children with special educational needs and disabilities. Int. J. Child-Comput. Interact., vol. 3-4, p. 23-40, 2015. https://doi.org/10.1016/j.ijcci.2015.07.001Google ScholarCross Ref
- Lynn Westbrook. Qualitative research methods: A review of major stages, data analysis techniques, and quality controls. Library & Information Science Research, vol. 16, no 3, p. 241‑254, 1994. https://doi.org/10.1016/0740-8188(94)90026-4Google ScholarCross Ref
- Thomas Groenewald. A Phenomenological Research Design Illustrated. International Journal of Qualitative Methods, vol. 3, no 1, p. 1‑26, 2004. https://doi.org/10.1177/160940690400300104Google ScholarCross Ref
- Kangsoo Kim, Ryan Schubert, and Greg Welch. Exploring the Impact of Environmental Effects on Social Presence with a Virtual Human. Intelligent Virtual Agents, Cham, 2016, p. 470-474. https://doi.org/10.1007/978-3-319-47665-0_57Google ScholarCross Ref
- Alyssa Alcorn, Helen Pain, Gnanathusharan Rajendran, Tim Smith, Oliver Lemon, Kaska Porayska-Pomsta, Mary Ellen Foster, Katerina Avramides, Christopher Frauenberger, and Sara Bernardini. Social Communication between Virtual Characters and Children with Autism. Artificial Intelligence in Education, Berlin, Heidelberg, 2011, p. 7‑14. https://doi.org/10.1007/978-3-642-21869-9_4Google ScholarCross Ref
- Alyssa M. Alcorn, Helen Pain, and Judith Good. Motivating children's initiations with novelty and surprise: initial design recommendations for autism. Proceedings of the 2014 conference on Interaction design and children, New York, NY, USA, 2014, p. 225–228. https://doi.org/10.1145/2593968.2610458Google ScholarDigital Library
- Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Trans. Pattern Anal. Mach. Intell., 2019. https://doi.org/10.1109/tpami.2019.2929257Google ScholarDigital Library
- Tadas Baltrusaitis, Amir Zadeh, Yao C. Lim, and Louis-Philippe Morency. OpenFace 2.0: Facial Behavior Analysis Toolkit. 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), 2018, p. 59-66. https://doi.org/10.1109/FG.2018.00019Google ScholarDigital Library
Index Terms
- “Can you help me move this over there?”: training children with ASD to joint action through tangible interaction and virtual agent
Index terms have been assigned to the content through auto-classification.
Recommendations
Tangible Play and Children with ASD in Low-Resource Countries: A Case Study
TEI '20: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied InteractionTangible User Interfaces (TUIs) can bridge real-world physical objects with the digital world, which is beneficial for children with ASD. However, at present, most TUIs have been developed for children in affluent countries. Hence, this paper presents ...
How Do Typically Developing Children and Children with ASD Play a Tangible Game?
Human-Computer Interaction – INTERACT 2019AbstractA tangible user interface (TUI) brings a strong educational potential for both typically developing children and children with ASD, as it fosters tactile stimulation, development of cognitive skills and perceptual skills. However, no previous ...
Interactive Pretend Play (iPPy) Toys for Children with ASD
OzCHI '19: Proceedings of the 31st Australian Conference on Human-Computer-InteractionIn the recent past, Tangible User Interface (TUI) has shown its positive impact on children with ASD in affluent countries. However, limited research is conducted on designing low-cost TUI for children in low-resource countries. Hence, this paper ...
Comments