A Case Study of a Robot-Assisted Speech Therapy for Children with Language Disorders

Abstract

The aim of this study was to explore the potential of using a social robot in speech therapy interventions in children. A descriptive and explorative case study design was implemented involving the intervention for language disorder in five children with different needs with an age ranging from 9 to 12 years. Children participated in sessions with a NAO-type robot in individual sessions. Qualitative methods were used to collect data on aspects of viability, usefulness, barriers and facilitators for the child as well as for the therapist in order to obtain an indication of the effects on learning and the achievement of goals. The main results pointed out the affordances and possibilities of the use of a NAO robot in achieving speech therapy and educational goals. A NAO can contribute towards eliciting motivation, readiness towards learning and improving attention span of the children. The results of the study showed the potential that NAO has in therapy and education for children with different disabilities. More research is needed to gain insight into how a NAO can be applied best in speech therapy to make a more inclusive education conclusions.

1. Introduction

Inclusive education, and its inclusion in a sustainable society, needs to be based on respect for and values for diversity and individual differences [1]. UNESCO considers that having an inclusive education means strengthening the process by which we reach out to all learners [2], with and without disabilities, as it is supported by the guiding principles of the forth Sustainable Development Goal (SDG 4). Inclusive education allows children to learn without any discrimination. As students have different learning styles and educational needs, we must find methods to decrease barriers. Sustainable education’s main purpose is to promote student autonomy in their future [3]. Inclusive education refers to education in which children are supported to give them equal opportunities [4]. Inclusive education has increasingly taken on a broader focus. Inclusion has to do with the participation of the entire educational community and reducing barriers to learning. Therefore, to achieve this, education must be linked to local realities [5,6]. Inclusive education works with a framework of values that includes participation and respect for diversity. Thus, one of the elements to be considered is to increase the possibilities of children’s participation by offering them tools to increase their self-confidence and social skills [7,8]. Participation consists of recognizing and valuing the identity of each student and their concern for their personal and social wellbeing. In this sense, all the resources, services and aids that support this wellbeing must be considered as facilitators of inclusive education [9]. Involving technology and robots in the inclusive education may help to work towards these equal opportunities [10].

So, educational robots may help as an assistive tool for children with problems in specific fields [10]. Therefore, there have been a growing interest in exploring the benefit of children–robot interaction for educational purposes using social robotics. Social robots support each particular educational need for every child [11,12]. Hence, teachers and education therapists may use social robots as assistants to make their practice easier with children with different needs [13,14,15].

In this context, the research problem derives from the necessity to explore the potential of logopaedic and pedagogical NAO-robot-based interventions in children with different diagnoses of speech impairment and learning related to oral language and literacy. In this context, this paper will be guided by the following question: would integrating a social robot in the logopaedic interventions result in an improvement of the child’s learning process or therapeutic evolution?

As a result, the aim of the paper is:

• To study if the results of the intervention with a social robot fits the expectations of the therapist.
• To analyse the response of the children while having a robot in their sessions.

We begin the article with a review of the main concepts embraced by this study, together with a review of the literature. Then, we present the methodology followed, the activity developed in the therapy centre as well as a detailed description of the data collection instruments to conclude with an explanation of the data analysis. Finally, we present the results and discussion.

2. Background and Related Works

A requisite for sustainable education is to ensure that all the children have the appropriate conditions to integrate themselves and their needs in the educational systems [16]. Assistive technologies and robots have a transformative potential to achieve integration and inclusion of children with specific education needs in formal and informal educational settings, developing new opportunities for inclusion. In particular, social robots are widely used, as they provide children support and opportunities to increase their performance in educational activities. Therefore, the interplay between children and robots in real time can be quite positive for their learning [17] and their social development [18]. Social robots have a design that allow them to communicate and socially interact with children in different roles [19]. Due to their simplicity in the communication interaction, they can be a powerful tool in inclusive education eliciting cooperation in different ways—following instructions [20], increasing their social skills [21] or provoking unexpected situations because robots are unable to identify misunderstandings [22]. As robots have predictable responses and provide a simple and reliable environment, they may encourage children with language disorders to improve their communication skills [23]. The social robot can be used not only as a communicator or a teacher but also as a mediator to interact with others [24] stimulating the child capabilities and skills.

In particular, we can find several examples of using NAO-type robots with children with attention deficit disorder (ADD) or with speech disorders [12,14,23,25] due to the fact that their communication interaction tools are programmed with predictable and simple functions for the children.

Language acquisition is essential for children’s physical, social and cognitive development [26]. Language allows communicating with others. Through language children give meaning to the world around them, they can exchange information, express feelings, emotions and thoughts and acquire new knowledge [27]. Children acquire during the early years complex components of language in areas such as phonetic, homological, semantic, morphosyntactic and pragmatic language. The hemispheric specialization for language is a natural process that emerges as a combination of genetic inheritance and context. There is enough evidence of the importance of environmental stimuli in the development of language components. Stimuli should be adapted to the child’s state of development and give children opportunities for communicative experience [28]. Children with oral language deficits, regardless of the cause, may experience limited communication and social interactions as well as delays in reading and writing learning [29,30]. Different authors have explored the importance of reinforce the social networks of children in inclusive settings [31]. In this context, a humanoid robot can be an additional therapeutic medium for the education and improvement of relationships for children with disabilities [32,33]. In particular, it could be a useful tool in the speech–language therapy field. Using a robot as an assistive education mechanism could relieve the time and effort of therapists and child’s family [23]. However, there is hardly any use of robots to support the speech treatment, and most research has used other support such as a computer, communication board or talker [13,34,35]. Without detracting from their use, their efficiency is still poor due to the impossibility of taking turns, precisely one of the most important objectives when it comes to improving communication [36]. In recent years, there has been an increased number of studies about how using robots can support teachers during their lessons [22,33,37,38,39]. Social robots have been used for students with autism spectrum disorders (ASD), Down’s syndrome, cerebral palsy and motor and intellectual disabilities. Most of the experiences showed improvements in the performance of abilities, interaction with peers and engagement during the robotics sessions. Both teachers and parents considered this support as positive [40]. They encourage researchers to make more studies testing different robots, activities and also collecting parents’ and teachers’ voices [14,15]. Lately, the fact that children can face a robot with less fear than in a human interaction due to the robot’s predictability seems to be another reason to promote the use of robots in therapeutic and educational contexts [12,23].

3. Materials and Methodology

The presented work is based on a qualitative exploratory case study approach. When an explorative case study approach is adopted, it is because the relevant behaviours of the study cannot be manipulated and to understand a phenomenon in context is sought [41]. This methodology allows us to understand the meaning of the experience. This framework provides the opportunity to disclose behaviours and perceptions of the participants in the study about the issues studied [42]. This happens when the object of study is the behaviour of children. The purpose of the study is not to make a comparison with other cases, just to explore the phenomenon in context. Given the complexity of the research, the research team is made up of different professional profiles who have collaborated from their areas of knowledge. The people responsible for the technical part (engineers/programmers) have overseen programming the robot according to the needs identified for each case. Three researchers, specialists in qualitative analysis and education, have carried out the definition of the research and the analysis of the data. In addition, one of those has also a degree on psychology master on children with special needs. Finally, the speech therapist was also involved. The speech therapist also held regular meetings with the pedagogical team of the different children’s schools both to coordinate and monitor the objectives of the intervention, as well as to open up possibilities for future inclusion of the robot in the classrooms.

As explained above, to carry out this intervention we decided to use the Aldebaran Robotics NAO, now SoftBank Robotics [43], due to its friendly and non-threatening appearance well suited for child–robot interaction [44,45,46]. This robotic platform widely used with children in therapy and in education [15,38,46,47,48,49,50], health and social care [51,52,53,54]. So, the NAO robot is ready to be incorporated to assist the speech therapist through educational exercises and recreational activities in the intervention. Our NAO robot was named “EBA”, from educational behaviour aid (Figure 1).

3.1. Design

Before designing the intervention, the team of researchers met the speech therapist in the centre where the sessions were going to be held. This meeting will serve to share the possibilities that working with EBA offers to the therapies and to know the pedagogical needs during the sessions.

Once the different possibilities of using the robot were analysed, the speech therapist, together with the psychologist of the therapy centre, defined the inclusion criteria in the experiment. These criteria were mainly the type of speech and oral language impairment and, consequently, their reading and writing deficiencies and if those fitted adequately to work with the NAO robot. As a result, considering all the agents involved (families and children), they selected the cases where the use of EBA could be helpful.

In order to ensure ethics and protection of children, the necessary precautions were taken prior to the study. The study protocol was approved by the institutional ethical commission of the university (CIPI 18,069). After the selection of children, their parents were informed about the purpose of the study, and we gave them a consent form to be signed. Informed consent involved explanation of the objectives of the study to be carried out in an understandable manner and that the sessions would be video recorded. Additionally, they were informed that their consent was requested to be able to publish the video recordings, photographs or voice recordings that were made during the speech therapy activities and the possibility of using them in academic publications. All the consent forms were reviewed with each family involved in the study, and all questions posed were answered before consent was obtained. So, they were able to decide whether they agreed to their child’s participation and to the videotaping of the sessions before the experiments took place and data were gathered. They were notified that EBA will support the sessions for a limited time. Then, the “Information sheet and informed consent” signed by the parents was collected as well as the children’s verbal consent prior to their participation.

3.2. Cases Description

Five Spanish-language children, aged 9–12, were recruited by the speech therapist to participate in our explorative case study. The diagnosis about the children’s cognitive and language level of development was made by the psychologist and speech therapist from the centre where the study took place. The participants’ description is listed below.

Child 1 is 10 years old, born with a cleft palate and cleft lip. At three and a half years of age, the child began speech therapy intervention for oral language and internal language structuring. In the first diagnosis, the child did not present developmental delay in the three areas of internal structuring of oral language (pragmatic, semantic and morphosyntactic). The child underwent two surgeries to correct the cleft palate and cleft lip after which the bucolinguofacial hypotonic alteration and the fear of speaking in public has overcome. The child has problems in oral language, nasality, vocalization and psychology in the use of language. Work is done on the phonemes/n/by nasality and/r/, techniques of vocalization and to increase voice volume, continuously monitoring the child orthodontic process.

Child 2 is 9 years old who has spent 3 years in speech therapy intervention and also in psychological treatment. The child has a specific language impairment (SLI), attention-deficit hyperactivity disorder (ADHD) with comorbidity, dyslexia and disruptive mood dysregulation disorder (DMDD) with oppositional defiant disorder (ODD). The child has a developmental language disorder, and it is very difficult to catch his attention and empathize with him. The child has phonological short-term memory deficits. Most of the time child 2 is unmotivated to interact with other persons and has a low rate of initiations in interaction.

Child 3 is 9 years old with SLI without attention-deficit disorder (ADD) who has spent 3 years in speech therapy intervention. Child 3 has affectation at emotional level with misuse of oral language with symptoms of illegibility, alteration of the internal structuring of language in the pragmatic, semantic and morphosyntactic areas and strong agrammatism. In turn, she presented a severe phonological disorder in the form of multidisciplinary in the phonemes/s/,/r/(simple and vibrant) and/l/in the structures of vc, vcv, vcvc, cv, cvc, cvcc, cvcv and cvccv and strong bucolinguofacial hypotonia. Child 3 has got mixed dyslexia with evolving dysgraphic and dysortographic symptoms, much involvement in semantic code and difficulties in the integration and execution of their executive functions. Direct intervention is focused on dyslexic symptoms, their discrimination process and oral language, as well as on the process of reading-writing maturational reinforcement. Cognitive and neuropsychological rehabilitation tasks try to stimulate and improve the child performance in any domain of cognition: language, attention, motivation, memory, executive functioning, spatio-temporal orientation, calculation, visual perception, etc.

Child 4 is 9 years old and has spent 3 years in speech therapy intervention and in psychological treatment. Child 4 has a language developmental delay and several types of dyslalia. The child is terrified of being inferior to others and has great self-demands when interacting with other persons.

Child 5 is 12 years old who has spent 3 years in speech therapy intervention. Child 5 is dyslexic with evolving dysgraphic and dysortographic symptoms with an ADD. Child 5 has got learning problems, both in math and language. There is no presence of dysexecutive syndrome as such, nor of an alteration in the integration, placement and use of executive functions. What must be achieved is a greater balance of these functions so that they are given equally and in parallel. The stimulation of the cognitive functions is achieved through restoration and rehabilitation strategies oriented to improve the performance in different domains of cognition such as language, attention, motivation, calculation and visual perception.

3.3. Enviromental Setup

The intervention was conducted during ordinary therapy sessions at the speech therapist centre in a 5 × 4 m room. There was a desk, a computer screen atop, the NAO robot (EBA) and two chairs in front (Figure 1 and Figure 2).

EBA was positioned either on the floor or on the table looking to the child. The therapist is always present and lead the sessions. The child interacts directly with EBA. Therapist and child can move around the room.

It is important to remark that the use of this social robot implies that an engineer- programmer must participate actively in the design and programming of the robot. In our case, a member of the research team carried out this work. He oversaw programming the robot with the actions determined by the therapist and to make the necessary changes to adapt these actions to the different behaviours of the children. For this reason, it was necessary for the engineer to attend the therapy sessions. He was always sitting on one side of the table with a camera placed behind him in a fronto-lateral position to videotape the sessions so as to capture the facial expressions of the children as they interacted with EBA. His presence made it possible for him to act as an observer at the session.

The programmer is inside the room with EBA when the session starts, in order to make sure that everything is working properly and do not disrupt the session. The speech therapist and the child enter the room together. The intervention starts by introducing EBA to the child, to become familiar with the robot. The child interacts during the sessions, both with the robot and the therapist.

3.4. Data Gathering and Analysis

Thirty-minute sessions with children were conducted once a week for 30 weeks. For a later analysis, these sessions with EBA were video recorded. Due to personal reasons, some children could not attend to all the sessions.

Multiple data sources were gathered and triangulated to give validity to the study:

• Two semi-structured interviews conducted to the therapist;
• Therapist and programmer personal diaries of each session;
• Video recording of the sessions.

The semi-structured interviews conducted to the therapist were done to understand in depth the therapy sessions. Semi-structured interviews were used, because they enable narrowing around the needed topics while allowing the interviewed to feel free to talk about whatever she considered important [55]. The interviews were held in person and were recorded under informed consent. We interviewed her once at the beginning of the experience and once after the video observations, when the study had finished. The initial interview, which gathered background information about the children, the centre, the speech therapist and the robot technology use, lasted about 45 min. The second interview served to review the evolution of each child during all the sessions and to better understand some of the findings of the video observations and the reflective diaries. The guiding questions sought to complete information on the pre-identified constructs of interest and to induce any other emerging categories or find unexpected information. The questions were face validated by the research team, according to the research objectives and their wording. These questions helped us to go deeper into the results obtained and the evolution of each case.

Two researchers jointly conducted the first interview, the video observation and post-observation and post-experiment interview to establish consistent interview and observation procedures. Researchers also made detailed field notes and kept research diaries summarizing each observation and interview and identifying potential themes that could be used when developing the coding system for data analysis. The therapist and the programmer made a detailed reflective diary after each session with each child. The therapist wrote it and the programmer recorded it. In this study, it is intended to examine the response of children to therapy, so it was planned to use the video observations to separate the interviews and diaries of the programmer and the speech therapist, giving us the opportunity to select the video material that will fit our research questions. In this way, the videos will capture details of each situation and will be a powerful addition to interviews and diaries [56,57].

Analysis Procedure

The qualitative data (semi-structured interviews, reflective diaries and videos) were thematically analysed through coding and interpretative techniques using NVivo 13 identifying patterns or themes in them [58,59]. Thematic analysis is defined as a “data reduction and analysis strategy by which qualitative data are segmented, categorized, summarized, and reconstructed in a way that captures the important concepts within the data set” [60]. While focusing on the interpretive analysis of the interview transcripts and diaries, video recordings and notes taken during the observations were also reviewed to clarify and triangulate data. The process of analysing the data was:

• A professional service verbatim transcribed the content of the audio-recorded interviews and reflective diaries of the sessions.
• First round of data analysis consisted in reading the transcribed material and viewing videos of the sessions to triangulate the findings of the transcribed material. An inductive thematic analytic approach was used in order to identify content units and emerging themes and patterns in the data. This served to establish the initial categories, which were related in meaning. Each coded unit could have more than one category. This analysis was done independently.
• Then, we had successive meetings to establish the dynamics of coding, discuss the differences in the excerpts selected and the categories chosen.
• The material was reviewed, and the discrepancies were resolved through analytic conversations until reaching a consensus regarding the excerpts, the observational categories and the emerging themes.
• Based on a data-reduction process, we started with a “book of categories” previously agreed, with a brief description of each one that was expanded during the process. From these initial dimensions, an inductive process was carried out and some other categories emerged (

  Table 1. Dimensions and categories.

  Dimensions (Definitions)	Categories	Sub-Categories
  Intervention (observations related to the therapy itself and the objectives worked)	Speech therapist	Formulation of questions and answers
  		Comprehension and construction of sentences
  		Articulation and pronunciation
  		Voice volume
  		Dictations
  		Games
  		Literacy
  		Reading comprehension
  	Memorization	Through the multiplication tables
  Readiness towards learning (children’s attitudes towards learning)	Motivation	Positive reinforcement
  	Attention
  Emotional aspects (affective behaviour)	Empathy	Creating a link with EBA
  	Other affective components	Absence of value judgments
  		Absence of non-verbal language
  		Frustration
  Relationship with EBA (established relationship between child and robot)	Behavioural changes
  	Interaction styles	Teacher
  		Resource
  		Friend
  	Personalization
  	Ambivalence	Changing roles
  Relationship with the programmer (relationship of the children with the programmer)

  ).
• In the second round of analysis, we used a deductive thematic approach. The process was completed by saturating the information and with new interpretations of meanings being developed.
• The most important quotes to describe the results were extracted and grouped to organize all the information found, looking for correlations between them.
• Finally, a narrative connecting the findings was written from the triangulation of all the data analysed.
• Videos were identified using the following codes C1, C2, C3, C4 and C5 for each child, followed by the date of the recording.

4. Sessions with EBA

The study was focused on analysing how the NAO robot use would effectively affect the children’s response in the intervention according to the therapist. To do this, we focused on the identification of the factors that promoted, due to the integration of the NAO-robot in the sessions, an improvement in the child’s learning process.

Consequently, after the first meetings with the speech therapist, the sessions with EBA were designed so that the following aspects were worked on.

For the children with dyslalia and dyslexia:

• Reading comprehension: short/medium- and long-term memory.
• Literacy.
• Storytelling.
• Tales.
• Vocabulary.
• Phonological awareness.
• Articulation and phonetical–phonological pronunciation.
• Phonetic segmentation.

For the children with ADD:

• Attention.
• Writing.

For the children with SLI:

• Oral and written comprehension.
• Reading and writing.

The objective of the sessions was to identify the factors that promote a substantial or a partial improvement of the child’s learning process or evolution after the integration of the NAO robot inside the sessions.

We started with a common software for all the cases [61]. Those were based on the previous experience of the speech therapist with the children. After each session, adaptations to this software were made such as to make EBA functioning related to each participant.

While entering in the session, the first script was always to say “Hello” to the child and a short conversation about the children mood. This greeting script is developed to provide emotional interaction. “EBA” asks the child to tell them how they feel and asks for the homework they have from the school.

Depending on the answers, the session starts with the exercises pre-programmed inside a module. Whether the child completes or not the tasks correctly, EBA’s mission will be to provide positive reinforcement encouraging the child. These reinforcements are different for each child, as they are connected to the responses given, to their behaviour and to the child’s difficulties during the therapy session. The answers of “EBA” sometimes include sounds and movements such as raising the arms or conveying different forms of feedback to reflect behaviour approval and positive reinforcement.

The modules were reviewed by the speech therapist before being used.

For the module of reading comprehension, as the objectives were to strengthen reading and to work memory and sustained attention, “EBA” mission was to ask questions about the text that has been read, giving positive feedback to the child.

For the module of dictations, stories and vocabulary and improvement of oral comprehension, the objective was to strengthen vocabulary, as well as written and oral comprehension. To achieve it, “EBA” will ask the child to tell a story and ask questions about it and make dictations checking the spelling.

For the module of articulation and phonetic–phonological pronunciation, the aim is to develop the grapheme–phoneme correspondence, to recognize and to identify the auditory discrimination of the different phonemes and to develop pronunciation skills. In that case, “EBA” assumes the role of a student or of a teacher, and the child the opposite one, and indicates the child’s understanding or not of what he says.

During the module of phonological awareness and phonetic segmentation, the objectives are to strength the lexical and phonological routes of reading and writing, to enhance the grapheme–phoneme conversion and to identify syllabic structures cv, vc, cvvc, cvcv, ccv, cvv, ccvc. So, “EBA” will ask the child for words starting with a letter or will ask the child to identify how many syllables are contained in a word told.

Finally, for the literacy skills module, the objectives are to increase the concentration of the child and strengthen auditory processing, to be able to respond to a verbal question and to develop active listening skills. To achieve them, “EBA” will ask the child to repeat more clearly everything he does not say properly, will ask questions about a listened story and will give instructions to the child for all the activities defined.

The aim of the activities made with “EBA” was to work on the therapist’s objectives for each child. As the programmer is there, the robot can change its behaviour during the intervention when it is required.

5. Results and Discussion

The results presented arise from the qualitative analysis of the information retrieved, based on the categories emerged that respond to the general objectives of the study. They come from the triangulation of the data from the diaries of the programmer and the speech therapist, the video observations and speech therapist interview. Therefore, they are shown based on the main dimensions (Table 1), showing some of the excerpts retrieved (in italic). The categories and dimensions are bold highlighted.

Technical results about the robot architecture can be found in Egido et al. [61].

5.1. About the Intervention

In relation to the results of the intervention in each case, we have identified two large categories according to the objectives worked on.

On the one hand, there are those relating to speech therapy. In this case, the sessions’ work was mainly on oral and written language. According to the vision of the therapist, integrating a robot in the therapy sessions can be fruitful regarding the formulation of questions and answers. As Cangelosi et al. remarked [62], the linguistic input that children receive is less motley in comparison with speech directed at adults. The robot uses the characteristics of child-directed speech, such as less words and simple grammatical constructions [23]. When EBA poses any question, the child must endeavour to answer by vocalizing correctly and with good pronunciation. Otherwise, the robot will not understand the answer, as it can be seen in the videos. Furthermore, if the child wants EBA to understand the answer given, the sentence structure must be such that the information is coherent and clear. Likewise, when the therapist asks the child to ask questions to EBA, the child should do so in a clear and orderly manner. As a result, the support that EBA offers implies a development of children’s ability to comprehend and construct sentences of any length. For example, child 3 feels securer and, consequently, talks to EBA non-stop with better arguments.

“So, if you notice in C3, the child begins to better understand longer sentences. Yes, and applies it. I think it has helped in their learning, in their reading.”

[Second interview with the speech therapist]

As the children wanted to be understood by EBA, in some cases, this turns into increasing their voice volume. Analysis of the data revealed that increasing the volume of the voice was a challenge for the children, and when the video clips were viewed in relation to oral language practice, the discussions that were shared with the entire research team revealed how they do it. The speech therapist commented that “When he sees me waiting for the answers, because I give them enough time to wait, it feels like an eternity, but suddenly it seems that C1 has broken her shyness to speak. She strives to be heard by the robot. When EBA says ‘I’m a little deaf’, C1 smiles and thus increases the volume of her voice”.

It is observed in the videos how, for instance, child 1 has broken her shyness to speak. She strives to be heard by the robot and, thus, naturally increased the volume of her voice.

“Well, C1 is always glad a lot to see EBA, I think she is the one who is most happy and we will work with mostly vocal mobility as she is the one who has had a jaw surgery and phonetize and raise her voice for EBA being able of listening to her. It is the way to encourage her, tell her that EBA does not listen well and that she needs her to raise her voice, something that she is not normally doing. Because in this way we begin to help her open more phonetically the words you need to hear, I mean, say, so that EBA will listen to her whenever she tells it a story or something.”

[Programmer’s diary (13 December 2017)]

“… It was a moment that ‘EBA’ said ‘Speaks me higher’ and the child saw that EBA didn’t respond because he could not hear. She started up inadvertently, i.e., not gave the command “turn up the volume of your voice” because she has fully retracted that order. But she needed EBA listening to her, then there appeared the volume. So, there has been a significant change and from there it is true that parents have spoken with me and they say they have noticed that she speaks a little higher….”

[Second interview with the speech therapist]

In learning written language, the robot participated in different word construction games as well as dictations. These exercises contributed positively, according to the opinion of the therapist in the development of these exercises.

On the other hand, exercises were carried out to improve the children’s memorization. To this end, work was done on learning the multiplication tables.

“… in learning the multiplication tables, for example, I think he’s learned thanks to EBA, as there was no way until it appeared. I told David: ‘Please, put the tables in the software, and let’s see’. Hey, it was wonderful!”

[Second interview with the speech therapist]

Therefore, it can be observed that the turn-taking situations between the robot and the children, since the use of appropriate linguistic expressions are important for the development of children’s language, it produces an increase in the duration of speech and interaction [22,23,63].

5.2. Willingness to Learn

An important issue to consider in interventions related to learning in children is to identify how the use of the robot influences these learnings. In this sense, we have identified some categories related to motivation, attention during exercises (both sustained and targeted attention) and the use of positive reinforcement.

The speech therapist contemplated the support given by EBA in the sessions as a motivating element for the children. The speech therapist infers causal relationships between aspects of EBA and results in the progress of children. She perceived that working with EBA is a source of pride for one case, leading to increased motivation and increased willingness to learn, having a better performance on the sessions as it can be found in other studies [23,24].

“She tells it to her friends in class and had to send a photo of the robot…. She told me that she is one of the few children in the school who has a robot. She says so and therefore she does her best to work or play with EBA.”

[Second interview with the speech therapist]

The robot’s company can encourage the child to participate more in the conversation regardless of the hits [39].

In some cases, it is observed in the videos how the same verbal reward (for example saying “very well”) is more reinforcing when it comes from the robot than when it is issued by the speech therapist. This is attributed to the child perception of support and unconditional approval by the speech therapist, while the robot is seen as an objective evaluator. This positive reinforcement, therefore, motivates the children to learn.

“EBA helped her a lot with the questions/answers, because it understands her and immediately says ‘very good, very good’ and for her, having a robot saying ‘very good’ is much better. I used to tell her, why do you get so happy? And she says: ‘Because it is a robot.’ Sure, I can say ‘good, you have done it very well’ and I give her a big kiss, but I… she knows that I love her and that there is an affective part and not in the robot. She knows whether she is doing well or not, I will always love her. But if the robot tells her that she has done it well, she knows it is because she really has done it well, and she is aware that she has done it well. And that is what is helping her in that aspect: she is talking to EBA more, constructs the sentences better and understands at a comprehensive level the way EBA constructs sentences.”

[Second interview with the speech therapist]

Another consequence of the cooperation of EBA is that the speech therapist noticed that the children focused their attention more both sustained (that is maintained for a certain time period), as targeted.

“Yes, they keep sustained attention more because it is indeed curious, sustained attention both at the level of learning and focusing of attention, such as movement. This child, when EBA is not going from room to room, he stays here, he rises, stands up, moves around the chair, but not out of here, and that’s mean body control because what interests him is here.”

[Second interview with the speech therapist]

“Then, in all the cases a greater sustained attention has been achieved, well, there are children who doesn’t have an attention and we don’t work it, but with all there is a greater sustained attention”

[Second interview with the speech therapist]

“In the case of EBA, it is a robot, at first, of course they pay more attention, as if…, you know, the novelty, but not sustained attention, is the surface or physical sustained attention that we all have to something new, but they focus sustained attention when it intervenes in the integration of learning. From this novelty, the sustained attention appeared, the focused one has been appearing throughout the sessions. And this is very important. Look, when I talk about it, I also realize that things come out.”

[Second Interview with the speech therapist]

As Encarnação et al. remarked, sustained attention and self-regulation could be promoted in a virtual environment where there are fewer distracting factors [64].

5.3. Emotional Aspects

As the children gradually learn to work with the robot, different interactions not directly related with the therapy emerged, such as empathic interactions or dyadic and even triadic interactions. These aspects are included in the dimension called emotional aspects. In this group, the emotional response of the children creating a link with EBA and showing some empathy should be highlighted.

“I remember one day in a session were EBA had a broken finger,… Well, the parents had to phone me! Their child was scared because we had put an electrical tape on EBA finger, and he thought it was a cast. He had normalized EBA as if it were human. When the parents phone me they said that their child had slept badly, that ‘poor EBA’… that ‘Daddy please, phone EBA to see how she is, because she is in the hospital for sure….’ But of course, you have to get to know this child. That this child says that and sees a fellow man as his fellow man…, because his equals… well, he has many behaviour problems because he has an oppositional defiant disorder. So, normally, he attacks his peers. His greatest communication is to attack…. And suddenly, being so worried about EBA? It was a great leap that has also taken him out. That is very nice, it moves me.”

[Second interview with the speech therapist]

Other affective elements appeared. For example, in some cases, the unique power of working with a robot seems to have a beneficial impact on children, who feel special in their environment in a positive way.

“They brag in the school that they have a robot. We are talking about children with difficulties, with disorders, who have always felt different in the group…, who have always been treated as different, they are taken out of the classroom because they go to support classes because they are different. And it turns out that they now have a robot that no one else has. And brag about it… And they feel important for once in their life. I think it is very important.”

[Second interview with the speech therapist]

In other cases, interaction with EBA has been key when intervening when the child was very upset.

In another session with this child, EBA calmed his hyper-activity in an effective way. […] He came angry about something with his father, a coin or something he lost in the car, he brought something to show me. Another child, many children, they get angry and you, in two minutes, have calmed them down, ‘come on, your brain, so and so’, we change, and we start working, and that’s it. But not like this child, no, because he has an oppositional disorder. Then he came angry, stood there, he tried to attack me, he began to bite himself. This child, I’ve been working with him for three years: the first year there were attacks, he practically escaped me, tried to attack me, I had to slow him down a lot, I had to do physical restraint, and that’s over. But last year, that day he attacked himself, he begins to bite, he begins to pinch, he begins to hit his head on the table and I, because I was next to him, sitting on the floor, ready to intervene contention because the next step is to attack me or the one next to him. So, David went super-fast and programmed EBA, and EBA started talking to C2, C2 didn’t even look at it. He entered and did not look at them, there was his problem and his aggression and his coin…Suddenly, EBA began to speak to him with what David was introducing: “Hello C2! How are you? What’s wrong? Don’t be like that, you’ll find the coin when you go out”, or something like that. And in less than 3 min, he approached EBA, started talking to her, told her about the problem. That this child tells you?… May be, he would tell the psychologist about it later. If I had spent the whole session with him lying on the ground, I wouldn’t have succeeded, because it has already happened to me, because he also gives himself feedback. This kid what he didn’t do with the robot is feedback on the problem.”

[Second interview with the speech therapist]

According to the speech therapist, thanks to EBA’s absence of value judgments, the objective responses that EBA gives to children increase their self-esteem, and therefore, they have a better development of their learning.

No prejudice, no value judgments, and that makes children raise their self-esteem.

[Second interview with the speech therapist]

This can be clearly seen in some sessions such as the one where the child has to play to the balloon introducing the letters that EBA says inside it and forming a word: “We can see the speech therapist trying to positively reinforce the child’s response, but the child ignored her. However, when EBA confirms to the child how well he has done it and encourages him to continue playing, it is seen how the child changes his attitude and strives to continue doing well”.

The NAO robot has movement but does not express feelings. Initially, to work on EBA’s non-verbal language, the research team considered the option of using different colours in the robot’s eyes to simulate mood swings. However, throughout the sessions, the therapist realized that this lack turned out to be positive for our case studies. The complexity of non-verbal messages may decrease the desire to interact in some children with communication deficits; however, the fact that EBA was predictable and had simple conversational functions could help interaction [23].

“That’s one of the things I have mentioned it to David. The non-verbal language does not come out and it is essential for communication and nonverbal communication, and in EBA it does not appear, but children don’t miss it.”

[Second interview with the speech therapist]

“Using this form that comes spontaneously, because I could not work otherwise, then comes spontaneously, children adopt this affective form and transmitted, and have taken from EBA, have made a transmission, they give it and take it to EBA as if EBA really gives that affection. So, what happens when that emotional part? Integration of learning occurs better, it stays, it is not only learnt, but it integrates because there is a part that they have to show or not or learn just because EBA says so.”

[Second Interview with the speech therapist]

It is important to note that, in some cases in which the programmer was unable to make EBA respond quickly to the needs of the moment, a certain frustration is detected in the children, which leads to them disconnecting partly from the session.

“So, sometimes it’s very complicated to program the computer… that is, the robot, while we are speaking as the software needs time to load and today this has happened and C4 wanted immediate results, and I couldn’t give them. So, I’ve seen the child, mmm, ‘frustrated?’… Until everything was working again, but then it was difficult for me to catch his attention.”

[Programmer’s diary (28 May 2018)]

This is seen in this video session: It is observed that the engineer is nervous, looking at the child and the speech therapist, while the program compiles the new changes that he has just introduced. It looks like the child has suddenly lost all attention in the session and wants to get up and leave.

5.4. Relationship with EBA

An important dimension that appears in this study is the relationship that each child establishes with the robot. This relationship changed over time and did not develop in the same way in all cases, and differences were observed between them. Both from the observation of the videos of the sessions and from the diaries of the therapist and the programmer, behavioural changes were observed. Children that initially showed fear or suspicion towards EBA, after several sessions changed their behaviour during the sessions.

“He, at first, was afraid of EBA and did not approach it. If it moved, he got scared. But he went getting used to it, he normalized it.”

[Second interview with the speech therapist]

Even child 2, for example, managed to calm his aggressiveness on some occasions thanks to the affective relationship with EBA.

“‘If I kick EBA and throw it out the window, will I break it?’ And I say ‘yes, you break EBA, and if you do the same to me maybe you will hurt me too. And we will stop functioning, me and EBA’. And he said, ‘I will not do it because she’s my friend’. But he says so loudly, and I thought that it could have happened. Do you remember? At the beginning I told you, this child, you must be careful with EBA because he is able to catch it and release it from the balcony.”

[Second interview with the speech therapist]

These behavioural changes observed in the children can open the door to the incorporation of the robot in the classroom with the aim of inclusion. The speech therapist speaks about the meetings hold in the children’s schools, to see how EBA is affecting their behaviour.

“EBA facilitates inclusion inside and outside the classroom because its support and reinforce the learning of speech and consequently of oral language and communication. It seems that it has also stimulated the emotionality of individual and group social language process. It helps children to recognize the differences through EBA, which is different, and learns and teaches at the same time. So, I would say that EBA facilitates the inclusion objective of achieving equality and equity. Finally, the originality of EBA invites curiosity and reinforces the students’ strengths and capabilities. I see a lot of possibilities incorporating EBA in the classrooms.”

[Second interview with the speech therapist]

Interaction styles established between the child and the robot take on different nuances in different aspects or differ from one case to another. Although, initially, “EBA” was programmed to act as a teacher, in some cases, children showed a great challenge in becoming EBA’s teacher rather than being their students.

“One day I said, ‘Hey, you know, EBA is going to teach reading?’ And he said ‘Well, I already know. I’m going to teach her because EBA is a robot and robots do not know, you have to shove things in those circuits.’ So, I said, let’s flip it…. This is the goal now, he is the teacher of EBA.”

[Second interview with the speech therapist]

“Then, he has been teaching EBA, the NAO, how that game works so that the next day we meet, EBA has learned and we can work with it.”

[Programmer’s diary (12 November 2017)]

While observing the video this was commented on by the researchers as follows:

“C5 looks all the time trying to be the protagonist in this session, nor the speech therapist, nor EBA, showing the robot what it should be learnt.”

“Well then, this exchange of roles ‘student–teacher’, ‘teacher–student; children like C4 that… he has managed to connect to the robot because he is its teacher. He is not taught; he teaches the robot. What has happened with this? With this we can get to alleviate a little feedback that he has self-imposed himself. Because in order to teach, you must have certain trait of humility, even if the child does not know, takes it out, otherwise, he can’t teach. So, he can overcome this self-imposed rigorousness.”

[Second interview with the speech therapist]

Child 5 just sees the robot as a resource.

“For C5 EBA looks like a robot. C5 is older than the others. It is the only case that sees EBA as a robot. As a resource to learn. That is, C5 is not going to become a friend of EBA.”

[Second interview with the speech therapist]

Additionally, most of them see EBA as a friend, personalizing it.

“In this case, with this girl, what has been achieved and I have seen a change with the activities that EBA has done with the girl, because for us it is EBA, the children do not see it as a robot,… At first yes, but later it is as… They ask when does EBA come? As if EBA were a friend of her.”

[Second interview with the speech therapist]

The category of personalization refers to the way in which the children, in their relationship with the robot, have personalized EBA, which is closely linked to the displays of empathy that have been shown. The children start asking questions about the robot, such as: “Did it sleep well?”, “Does it have siblings?” Or “Why don’t you build its parents?” However, this relationship with the robot is not shown in the same way in every case.

“For C3, he spoke very softly out of absolute shyness, he respects her, it is difficult for him to approach see it there as… She sees it as a robot and I think she is beginning to open up and she is the one who asks the most about EBA, curiously, and asked ‘can you create a dog for her?’, because she has dogs.”

[Second interview with the speech therapist]

In child 2, we have observed a certain ambivalence, that is, one day empathizing intensely, and another day ignoring it. An example of this is described in the following excerpt and verified by watching the videos of this case (C2): “it is observed how the child sometimes directly addresses the robot and sometimes approaches the programmer to ask questions about its use.”

“He has empathized with EBA, to such an extent that we have seen his maturation period with respect to the robot. He is the only child we are working with who differs, that is, he distinguishes. Suddenly, he gets to work with EBA and normalizes it as an equal, mind you that it is difficult for this child to see someone as an equal, and suddenly he needs to ask David something about EBA, about how EBA works, he gets up and stands next to David at the computer to see EBA networks. And in a matter of seconds he sits back down and has EBA as an equal again. For this child it is very difficult to do that, very complicated, it is something that the school is trying to do by all means and here he does it and nobody has said ‘do it’, it has come out because of the intervention that is being done.”

[Second interview with the speech therapist]

The appearance of the fact that children see “EBA” as an equal or as their student, caused that some changes could be made in the way of working on the objectives of the sessions in a more efficient way. As a result, an improvement is observed in children’s vocalization and in the construction and structure of sentences. In this way, children’s self-confidence increases as the scenario in which the child adopts the role of robot teacher generates a high level of commitment, motivation and self-confidence [65].

5.5. Relationship with the Programmer

Given that in the sessions, in addition to the robot, a new person was introduced, we established as a dimension the relationship of the child with the programmer. Although during the sessions, the person of reference for the child was the speech therapist, given that over time they have acquired a significant degree of confidence, in some cases, the child interacted with the programmer. However, in most cases, children ignored the programmer (David) during the sessions, detail that was observed in the video observations. Children always get into the room with the speech therapist and usually do not even look at the camera or to the programmer:

“Well, David is part of the furniture in the sessions. In mean, the children just greeted me and EBA. At the beginning I had to tell them, ‘Hey this is David, greets him’ … but then I gave up, because I realized that, for the children, he didn’t exist directly. Look! Not only did they not be aware that David was there, but they didn’t even realize there was a camera recording them”

[Second interview with the speech therapist]

On the other hand, the relationship established between the programmer and the child has helped in the integration of EBA in the sessions and the achievement of the objectives pursued by the therapist, such as when they ask questions to him directly because they want a new software in EBA.

“The thing about C2 is that, if EBA doesn’t understand him, he stands up and goes to David, ‘Why doesn’t he understand me?’, ‘Let’s see, where is it? Where is his brain?’ That is, he tries to understand the head of EBA. He interacts less than the others because he is more aware of the process of what is happening, so I would like him to interact. He is the only one who has not asked about EBA, the only one, and when he comes, he is happy, but he has not asked, the others have. When does EBA come? When does EBA come? He does not, but the way he interacts with EBA is different, there is no hook, but there is a hook with the whole process and there is a hitch with the Dragon Ball…. Yes, he has not humanized EBA, for him it is the robot that comes and sometimes he doesn’t pay any attention, sometimes you’re here and I look at David and say ‘I’m going to start working with other things because no way.’”

[Second interview with the speech therapist]

6. Conclusions, Limitations and Future Research

The study presented explored if the results of the intervention with a social robot fits the expectations of the therapist regarding the initial speech therapy objectives. Additionally, we wanted to examine the children interaction with the robot and their progress while using it as a support for the speech therapist.

Based on the qualitative analysis of our results, after conducting our study, we can conclude that incorporating a NAO-type robot, “EBA”, as an assistant to the speech therapist has some advantages but also certain limitations. We learned from each case and from each session.

According to the speech therapist, some factors have been identified in improving the child’s learning process. Using a social robot as a speech therapist assistant triggered an increase in the children’s motivation during the sessions and a greater commitment. In fact, one of the conclusions is that the robot can also play another supporting role in the sessions having different effects to those of the therapist and this change in children’s behaviour seeing “EBA” as an equal or as their student causes a different way of working on the objectives of each session, and as a result, there is an improvement in the children’s vocalization and in the construction and structure of sentences increasing the children’s confidence in themselves.

We found that children generally welcomed the novelty of interacting with a robot, as if it were a game rather than a therapy session. At the same time, delays or errors in the robot’s responses generated feelings of frustration and disconnection with the robot among the children, making it difficult to interact in the rest of the session.

The specific characteristics of each child made the experience with the robot something unique. Therefore, the design of activities and answers of the robot had to be personalized for each case. This customization requires the presence of the programmer in the sessions to adapt the robot software to the different activities and response in real time. Consequently, at least during the first sessions, the collaboration of experts from both seems to be required. This continuous technical assistance allows the therapist to concentrate on his/her work and not be distracted by technical aspects. However, advances in social robot programming may, in the future, allow anyone with little training to program such a robot.

A common element that contributed to the acceptance of the robot was the assimilation of non-judgment. This is an essential aspect in educational and therapeutic contexts. Since the robot uses the reinforces without making value judgments, its value becomes higher. Likewise, as the “EBA” language is better structured, children understand it better. Perhaps a greater consideration of non-verbal communication aspects is lacking, in those children with oral language difficulties. However, this lack, at times, results in useful outcomes for children during the intervention process.

Among the limitations of the study, one could say the sample was small because five cases have been analysed, but each case had its own peculiarities, which makes each case valuable in itself. However, an analysis with cases that have different logopaedic needs may give greater depth to this study. So, it would be convenient to expand the sample for future researches on the topic. The therapist has been in permanent coordination with each child’s school throughout the process to establish the objectives and carry out the appropriate follow-up. However, the study could have been completed by interviewing the children’s parents and teachers, so that new themes for analysis would have emerged. For future actions, we should deepen the subject using different approaches and methodological designs to provide a deeper vison of the phenomenon.

Another limitation of the study can be having used just one camera to analyse non-verbal information. The analysis of the children’s facial and body micro-movements can provide relevant information about their emotional states, the level of satisfaction in the interaction and comfort during the work sessions. To perform this analysis properly, it is required to use several video cameras located in different positions or a 360º camera.

This study highlights the importance that the use of social robots can have in support of speech therapies. It begins to improve the children’s development, promoting a better integration in their schools for an inclusive education. As the therapist mentioned, some of the objectives for inclusive education could be promoted by incorporating EBA in the classrooms in future projects. Understanding the perceived challenges after the study and the benefits in the integration of children can serve to inform other speech therapy centres about how to carry out similar experiences. More research is needed to gain insight into how EBA can be applied best in speech therapy to make more inclusive education.