1. Nichola Lubold
  2. http://www.public.asu.edu/~nlubold/homepage.html
  4. Arizona State University ASU
  1. Amy Ogan
  3. Carnegie Mellon University
  1. Heather Pon-Barry
  3. Mount Holyoke College
  1. Erin Walker
  3. University of Pittsburgh

Understanding the Influence of a Social, Adaptive Robotic Learning Companion

NSF Awards: 1637809

2019 (see original presentation & discussion)

Grades 6-8

Learning companions have the potential to provide not only cognitive support to learners but socio-emotional support through social behavior. Socio-emotional support can be a critical element to a learner’s success, influencing their self-efficacy and motivation. As a learning companion’s ability to be socially responsive increases, so do vital learning outcomes. In this video, we introduce Nico, a Nao teachable robotic learning companion that has potential to enhance motivation, self-efficacy, and learning. Middle school students who are novices in a domain learn by tutoring Nico in ratios and proportions. The Nao robotic learning companion is capable of different social, adaptive responses that have potential to engage learners, enhance social responses, and increase learning. With this companion, this project explores how different social behaviors can create positive STEM outcomes. A series of studies have been conducted in several middle schools in the southwestern United States, and positive effects on learning and social responses were found. This video describes Nico and the potential of social, teachable robotic companions for influencing motivation, enhancing learning, and increasing participation in STEM.

This video has had approximately 287 visits by 253 visitors from 161 unique locations. It has been played 200 times.
activity map thumbnail Click to See Activity Worldwide
Map reflects activity with this presentation from the 2019 STEM for All Video Showcase: Innovations in STEM Education website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Original Discussion from the 2019 STEM for All Video Showcase
  • Icon for: Susan Jo Russell

    Susan Jo Russell

    Principal scientist
    May 13, 2019 | 12:51 p.m.

    Hi Nichola,

    Thanks for your intriguing video.  I've never seen this kind of teachable robot used in a classroom before.  I can see why students would be engaged by interacting with a robot that responds to them. In a short video, it's difficult to get a sense of how students develop math ideas through this interaction. Is it possible for you to give an example of how interaction with the robot leads to a student's development of understanding?

    I'm also curious about your slide about learning gains.  Can you say more about how you determined these learning gains (what kinds of problems you used to determine these gains, how often students had worked with the robot, etc.)? And what do the numbers on the slide mean in relation to your approach to determining these gains?

    Thanks so much.

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 14, 2019 | 06:00 p.m.

    Hi Susan - thank you so much for watching our video! To answer your first question regarding how students develop math ideas, one of the goal's of the teachable robot is to help cement understanding through teaching - by teaching, learners may attend more to the problems, consider their own misconceptions, and elaborate on their own knowledge to construct explanations. In order to guide the learner's teaching, we provided the worked-out solutions to the problems the learners taught Nico and gave them time to review the solutions before they began teaching. Then, while they were teaching Nico, Nico would prompt for explanations from the learner and ask questions about the learner's approach to solving a particular problem. Nico would also elaborate, so for example, the learner might say "Now multiply by two" and Nico would respond with, "Oh okay. We multiply by two. Is that because we have twice as many bags?" Engaging in the activity of explaining and responding to Nico facilitated learning by encouraging learners to reflect on what they know. The participants interacted with Nico for 30 minutes in a single interaction. To measure learning, we utilized a pretest-posttest design with an A and B form of the test. The two forms were isomorphic and counterbalanced within condition (half of the participants in each condition received test A as the pretest and test B as the posttest, and vice versa). The tests consisted of 10 procedural and conceptual questions around ratios. We then calculated the normalized learning gain. The numbers reflected in the slide are the average normalized learning gains per condition (a social condition, where the robot spoke socially, an adaptive social condition where the robot adapted and spoke socially, and a control condition where the robot was neither social nor adaptive).

  • May 14, 2019 | 02:51 a.m.

    Very interesting video. I was wondering how the robot responses back to the student. Is it adaptive? If so how is  the robot programmed to respond. What is the role of the teacher? Does every child have an individual robot?

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 14, 2019 | 06:12 p.m.

    Hi Teruni - we appreciate your questions!  The robot responds via spoken dialogue that is generated via a dialogue system consisting of an automatic speech recognition engine (via the Web Speech API), a dialogue manager, a text-to-speech engine (built into the Nao robot), and feature extraction/transformation module. The dialogue manager utilizes a chat-bot based design to generate responses using artificial intelligent markup language. With this, we introduce some social dialogue responses such as praise and using the student's name. We also have the ability to adapt to paraverbal features of speech such as tone of voice, loudness and speaking rate. In the learning results presented, the robot adapted to the student's tone of voice, or pitch, modeling a human-dialogue phenomenon known as entrainment. With entrainment, people adapt to one another to become more similar over the course of a conversation. Nico modeled this behavior and we found that when Nico adapted on pitch and spoke socially, the participants had higher learning gains. You can find out more about the dialogue system and these results in these two papers...

    Lubold, Nichola, et al. "Automated pitch convergence improves learning in a social, teachable robot for middle school mathematics." International Conference on Artificial Intelligence in Education. Springer, Cham, 2018.

    Lubold, Nichola. Producing Acoustic-Prosodic Entrainment in a Robotic Learning Companion to Build Learner Rapport. Diss. Arizona State University, 2018.

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 14, 2019 | 06:14 p.m.

    Ooops missed your other questions! The role of the teacher involves helping Nico solve Nico's math problems - we provide a narrative to explain why Nico wants to answer the ratio-based problems that are posed (Nico is going camping with its friends). There is only one robot however so we had the robot in a separate room and each student came in, individually, to interact with it. 

  • Icon for: Beth Sappe

    Beth Sappe

    Director - STEM Mathematics
    May 14, 2019 | 02:54 p.m.

    Thanks for sharing this interesting video. What a unique way to incorporate SEL and math instruction. How are teachers able to incorporate into the class (is there more than 1 Nico?). Does this study include any student math growth data?

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 14, 2019 | 06:29 p.m.

    Hi Beth - thank you for watching it! For now, we are purely exploring basic research questions around the design of teachable robots, how they should behave, and how we can use robots like Nico to explore open questions learning, such as the role of rapport in influencing learning between peer tutors and building self-efficacy. This last aspect of Nico can provide insight into immediate strategies for teachers in the classroom. In terms of Nico, there is only one robot so it is currently not deployable to all students in a classroom. Erin Walker, one of the presenters on this video, has explored using teachable robots with groups of students versus the one-on-one interaction we designed with Nico. This may have some promise for teachers being able to incorporate something like Nico in the near term. There are also explorations of whether virtual agents can have similar effects. Let me know if you have any additional questions about this.    

  • Icon for: Denise Schultz

    Denise Schultz

    Instructional Math Coach
    May 14, 2019 | 09:40 p.m.

    Hi Nichola! This is all very interesting.  I can envision so many implications to this technology in the classroom.  We have a 6th grade student in our building who is a select mute, choosing to speak only at home to his parents.  He has also been diagnosed with Austim so he is currently in a specialized 12:1:1 program at our school.  At school he only communicates with us on a whiteboard but we have found him to be quiet witty and displays evidence of thinking deeply about the mathematics at times.  I couldn't help to think about him as I watched your video.  It makes me wonder if your research with Nico has included students with special needs particulary in the area of social interactions and/or speech challenges.

    I'm also wondering if you collected any other data beside the data you mention on measuring student learning.  I know collecting data on the affective side to learning is tricky but did your research include measuring students' feelings of responsibility to Nico's learning?  

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 16, 2019 | 01:17 p.m.

    Hi Denise - thank you for your questions and I definitely see potential for Nico with students with special needs. We did not explore this with any of the students who interacted with Nico, but I know that there are groups who interested in this and have shown positive results with other kind of interactive robots. Definitely something to think about for future work.

    We did not measure feelings of responsibility but we did collect measures of self-efficacy and self-reported rapport as well as asking students about their goals. We also coded for rapport-building dialogue on the part of the student. We are in the process of exploring the self-efficacy and goals. For self-reported rapport, we did not find that the learners differed significantly across conditions - in general, they all really liked the robot and reported fairly high self-reported rapport. However, when we looked at rapport-building dialogue (i.e. did they praise the robot, use Nico's name, etc.), we found that learners who taught a robot that used social dialogue and adapted to them used more rapport-building dialogue and that this was positively correlated with ratings of self-reported rapport.  This seems to indicate that the robot's behavior did influence how they felt (or at least behaved) towards to the robot. 

    Discussion is closed. Upvoting is no longer available

    Denise Schultz
  • Icon for: Brian Drayton

    Brian Drayton

    May 15, 2019 | 08:45 a.m.

    Thanks for this video, which has engendered some interesting discussion.  I have an obvious question which I don't think other visitors to your presentation have raised yet: 

    How does this compare to students' experiences from teaching each other (e.g. a la Palincsar's "reciprocal teaching")?  

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 16, 2019 | 01:08 p.m.

    That's a great question!  We did not compare directly (i.e. we did not have a condition) where students taught one another versus teaching the robotic companion so we can only comment on what we observed from students teaching the robot versus other prior studies of human peer-to-peer tutoring, and it is particularly challenging as the robot behaves more consistently than human partners in human peer-to-peer tutoring. We did design the dialogue responses of the robot based on observations of students teaching one another from prior studies.

    While not necessarily an answer to your question, one observation I can make is that different students had different responses to teaching the robot; for example students who were more comfortable with robots were more likely to accept social behavior from Nico. These findings are less likely to transfer to human-human peer tutoring. On the other hand, finding that higher rapport and higher learning were associated with social behaviors that were based on what we observe people do provides some insight into what might make peer-to-peer tutoring successful.    

  • Icon for: William Swift

    William Swift

    Coordinating Producer. PBS NewsHour STEM and Health Student Reporting Labs
    May 16, 2019 | 10:47 a.m.

    Very cool, I am going to be working with a Pepper robot at the Smithsonian, telling a story about their program.  Using robots to engage students and the general public is a new trend/field and it will be interesting to see where it leads.  great work.

  • Icon for: Nichola Lubold

    Nichola Lubold

    Lead Presenter
    May 16, 2019 | 12:46 p.m.

    Thank you!  That sounds like an awesome project; looking forward to hearing more about it. 

  • Icon for: Tom Yeh

    Tom Yeh

    May 20, 2019 | 05:52 p.m.

    Great work! I really like your project flipping the relationship between robots and students. There have been much emphasis of building smart AI or robots to teach students. Your work is refreshing because it is empowering students to be "teachers, that students are smarter than robots, not the other way around. In our own robotic research involving preschool children (https://www.mindscribe.org/, not related to STEM), we found children are more open to share their stories to a robot compared to a person (friend, parent, or teacher) because a robot is perceived by children as less judgmental. If you ever get to compared student-robot and student-student teaching, I would speculate that "social pressure" or "social judgement" could be lower in the student-robot condition. At any rate, I hope your project will inspire others to start thinking more about students teaching to rather than being taught by robots.

  • Further posting is closed as the event has ended.

Multiplex Discussion
  • Members may log in to post to this discussion.