Turns Out Kids Really Engage with Robots—Especially When They’re Chatty and Proactive

Table of Links

• Abstract and I. Introduction
• II. The Casa Paradigm and Gamification
• III. Land of Hands: The Game
• IV. Methods
• V. User Study
• VI. Results
• VII. Discussion

  • VIII. Conclusion

  • IX. Acknowledgments and References

VI. RESULTS

The purpose of our study was to explore the extent of learning about good hand hygiene practises from the gameplay and the influence of a pro-social robot’s nudges on the learning and interactions between the child and the robot in a collaborative game-play setting. In this section, we present our findings for our research questions in separate subsections.

A. Learning

Our first research question was ”To what extent does learning about hand hygiene take place in collaborative play between a child and a social robot”? To assess how well children reflected upon what they learnt about hand hygiene through the proposed platform, we devised a rubrics grid [32].

The rubric has three categories: “Awareness of good and bad hand hygiene habits”, “Knowledge regarding the importance of handwashing”, and “Demonstration/Explanation of the six steps of Handwashing”. Evaluation of each category was done on a 3-point scale:

• Grade 3 representing “Independently Answered” - If the child explained what they learnt through the game without any cues from the researcher,

• Grade 2 representing “Verbal Cues were needed” - If the child required verbal prompts from the researcher to reflect on their learning and

• Grade 1 representing “Physical Prompts were needed” - If the researcher had to use physical cues to help the child remember what was learnt through the game.

We conducted a two-sample t-test (α = 0.05) to compare the mean rubrics points obtained by the children for the With-Nudges and Without-Nudges conditions. The data was normally distributed and there was homogeneity of variance as assessed by Levene’s Test for Equality of Variances. The results showed that there was no statistically significant difference in the points for the With-Nudges (M = 6.4, SD = 0.986) and Without-Nudges (M = 6, SD = 0.756) conditions; t(38) = 1.247, p = 0.223.

To further understand the extent of learning that took place, we asked the children two questions before and after the study: Question 1 - “Can you show the princess the handwashing steps you know?” and Question 2 - “What is the minimum time we should wash our hands for?”.

For Question 1, we performed a one-way repeated measures ANOVA (α = 0.05) to compare the mean number of handwashing steps demonstrated by the children before and after the study. The results showed that the number of handwashing steps demonstrated by the children differed statistically significantly before and post the study (F(1, 31) = 407.485, p<0.0005) The box plot in Figure 8 shows the comparison of mean scores pre and post-test.

For Question 2, according to the Centers for Disease Control and Prevention [33], the minimum duration of handwashing is 20 seconds. Hence, if children answered Question 2 correctly, we coded the data as “1”, and if they were wrong, we coded it as “0”. Since data obtained from Question 2 has a dependent variable that is dichotomous in nature with two mutually exclusive categories (“1” and “0”), we performed a McNemar’s test with correction on the data. The results determined that there was a statistically significant difference in the proportion of correct answers pre-and poststudy, p<0.0005.

We also tested for differences in the learning outcomes between genders but found that there was no statistical significance in any of the above conditions.

B. Interaction

Our second research question was ”To what extent do a pro-social robot’s nudges influence the learning, interaction, and engagement of a child with a robot in a collaborative gameplay setting”? To study the extent of children’s social interaction and engagement with HakshE, we measured two factors, namely:

1. Interaction Level (IL)

   
   

2. Frequency and duration of verbal responses and facial expressions

Interaction Level (IL): Fridin [34] successfully measured the level of interaction (IL) between children and robots using a unique index based upon the the KindSAR interaction measurement index [35]. From their study, we adopted the following index to measure the quality of child-robot interaction at a stage “S” in the interactive session:

Here, the variable EC=3 if the child looks at the platform during the interaction, EC = 1 if the child looks at the researcher for help/explanation during the interaction, and EC=0 if the child does not interact with the platform at all. We consider two affective factors in our study - facial expressions and verbal responses. The variable F=1 if the child expresses facial emotions (such as smiles and surprised reactions) and F=3 if the child expresses emotions via verbal responses.

We acknowledge that our study has limitations due to its online nature. We could not control the positioning of the children in front of their laptop cameras. This led to the video only capturing children’s faces up-till their neck most of the time and not their entire bodies. Hence other social engagement cues such as gestures could not be captured and measured.

We performed detailed video analysis of the data and manually calculated the values of IL. In total there were 797 interactions from the study population (n=32). We further categorized the IL data according to the Nudges conditions (With-Nudges Vs Without-Nudges) and the gender conditions (Boys Vs Girls). Inspection of the Shapiro-Wilk test revealed that the IL data was non-normally distributed for both the nudges conditions and the gender conditions. Also, there was no homogeneity of variance as assessed by Levene’s Test for Equality of Variances. Hence, we performed a robust Welch ANOVA followed by a GamesHowell post-hoc test on our data.

Welch’s ANOVA determined that the mean IL scores differed statistically significantly between the With-Nudges and Without-Nudges conditions (F (1,797) = 12.721; p = 0.00038). The Games-Howell post-hoc test also revealed statistical significance between the independent groups (p = 0.00038; the mean of the With-Nudges condition was greater than the mean of the Without-Nudges condition). Similarly, IL was also significantly affected by gender (F (1,797) = 14.163; p = 0.00018), with the means significantly higher for girls than boys.

Frequency and duration of verbal responses and facial expressions: Next, we measured both the frequency and duration (seconds) for both the nudges conditions. We used ELAN software [37] to transcribe and categorize each utterance and expression made by the children into two factors - “Verbal Responses” and “Facial Expressions”.

• Verbal Responses: Children’s responses to the robot such as words, sentences, and vocalisations were considered to be verbal responses [11]. We observed that the most common verbal responses made by the children during the gameplay were: “Help me”, “Okay”, “Thank you”, “Yes” and “No”. Utterances such as “What” and “Huh” were not considered to be a sign of social engagement with the robot as it generally depicted situations where children could not understand or hear what the robot was saying. Within a 5-second time frame, if there was a change in the context of what the child was saying, we counted it as two different responses.

• Facial Expressions: Children’s facial expressions that signify positive social engagement with a robot include “Smiling” and “Surprised” expressions [10], [11]. All types of smiles and surprised expressions of the children in response to the robot were considered to be positive facial expressions. Expressions such as “Frustration” and “Grinding teeth” were not considered to be signs of social engagement. The duration of children’s smiles and surprised reactions were calculated until we observed a change in their facial expressions.

Inspection of Q-Q Plots revealed that the frequency and duration data for both conditions were not normally distributed. Therefore, we conducted a non-parametric Mann–Whitney U test (α = 0.05) for these two factors in the With-Nudges and Without-Nudges conditions. The results (refer to Figure 9) are summarized below.

We once again tested for differences in interaction and engagement between genders but found that there was no statistical significance in any of the above conditions.