Hello Dorothy! Thank you so much for your questions! We are just starting to take our next steps towards working with students to get their feedback on the implementation of the AI agent in their group. The exciting part about this work is that we are still deciding the form the agent will take (e.g., a robot embodiment, a disembodied voice, an avatar on a computer screen, a computer display, or other option). It will definitely be fun to see students' reactions to the robot embodiments and how the robots impact the students' experiences! 

The AI agent will augment teacher instruction, but teachers will continue doing the amazing work that they are already doing in classrooms! Depending on the role of the agent, it may be able to help students with collaboration and understanding of content as well as display data to teachers. We aim for teachers to continue in their roles while the agent best supports teachers' particular needs. Our upcoming participatory design work with teachers will be critical for helping us to further define the exact role of the agent. 

Thank you so much Dorothy, yes we completely agree! We look forward to sharing our results!

Chris, I am one of the co-presenters of this video--one of the Georgia Tech collaborators. I attended a talk you gave at GT a few months back. Your concept of teaching 'judgment' skills vs. 'reckoning' abilities has stuck with me and I think it is very applicable to the work we are doing at iSAT to teach and support collaborative problem solving. Glad to see another NSF AI institute represented--looking forward to seeing how your institute evolves. Thanks for stopping by our page.

Happy to stay in touch about this issue. Perhaps we can have a cross-institute discussion about it.

Thank you so much Dalila! That's an excellent question! We started off the design sprint with several different ideas for how an AI agent could support teachers and students, and then narrowed our focus to three specific metaphors:

• Community Builder: The Community Builder would be an AI Agent to support students in developing trusting collaborative relationships with each other. The Community Builder would uplift non-normative forms of collaboration to create more equitable collaborative spaces. 
• Co-Pilot: The Co-Pilot would be an AI Agent to support collaborative problem solving through promoting discussions within and across small groups. The Co-Pilot would primarily focus on supporting domain knowledge (e.g., helping students understand how to block program).
• Augmenter: The Augmentor would be an AI Agent to support teachers and students by providing data through a visual display. The Augmentor could help to relay information between teachers and students as well as guide collaboration (i.e., augmenting teacher instruction). 

We had groups of undergraduate students, graduate students, post-doctoral researchers, and early career scholars who participated in the design sprint event to create these metaphors. The designs of the metaphors were heavily influenced by feedback we had received from teachers and students including through the iSAT Learning Futures Workshops. We are looking forward to bringing these ideas back to teachers and students for feedback in the upcoming months! It will definitely be interesting to see which components are most helpful to different teachers and students depending on their specific needs.

This is great.  I look forward to talking more and we are  happy to share anything you want on Educator CIRCLS. (We'll tweet about your video, too!) We think a lot about how AI systems and teachers might partner.  And of course, relevant for the AI Engage Institute, too. 

I hope we can build collaboration among the three Institutes focused on education. Some of the AI-agents we are building relate to community building.

Thank you so much Judi and Chris! Yes that would be great!

Hi Nathan! Thank you so much, we are enjoying deeply exploring the ways in which AI can support collaboration between humans (particularly between middle and high school students)! 

One critical insight that emerged from the design sprint was the idea of using AI to support “non-normative” forms of collaboration (as part of the Community Builder metaphor for an AI Agent). Non-normative forms of collaboration include behaviors that are commonly discouraged in classrooms or are perceived as unproductive (e.g., “off-task” discussions). Often, however, these non-normative behaviors make for more equitable collaborative dynamics as well as present an opportunity for building relationships between students. This brings to the forefront exciting directions for AI in terms of being able to capture and support the variety of normative and non-normative ways in which students collaborate. This will likely require AI models that triangulate speech with gesture, posture, eye-movements, and other forms of multi-modal data to understand the different types of collaboration that are occurring across classroom settings (which can be particularly challenging to capture given the amount of movement and noise during group work). We are looking forward to pushing the boundaries in this area to see how we could best support our teachers and students!

Ooh, this is a really nice angle. I wonder if you can say more about how you'll identify when normative or non-normative behaviors are "good"?  Is that something you imagine the teacher might help with? 

From a measurement perspective, we are are working to model CPS dynamics in real time. One example we discuss is with off-task/topic conversation. Sometimes this off-topic conversation is a tool students use to become integrated with group members. It first appears unproductive to CPS, but it is actually contributing to group member inclusion and group cohesion. We are working to model this communication behavior over time in order to assess what types of non-normative CPS communication tend to benefit the group during different phases of collaboration. 

Rachel or Indrani can speak to ways the teacher might help with this with regard to the teacher dashboards. 

Yes, teachers will be critical to this component! We have conducted participatory design sessions with teachers to better understand their dashboard needs in the classroom, which we will use to inform our designs. We are also in the process of considering how we can integrate teacher feedback within a dashboard to inform our AI models, particularly for complex areas that vary or may look different across classrooms.

As the institute grows, we will be and currently are developing the role of the agent alongside young people/students and teachers. Currently, the students in the Learning Futures Workshops are generating their own creative ideas for the parter (e.g., a simple light up display that monitors collaboration norms--somewhat of an artifact of how collaboration is going to ground the whole group). The role of community builder has also resonated with them. When interacting with teachers, we are currently focused on what they would like to see on both after-action review dashboards and in-real-time dashboards. We are working with teachers to ensure the dashboards are customizable and allow teachers to engage fully with the class during class time (i.e., design in ways that prevent them from having to look only at their iPads during class time).

Arthur Graesser's work with CPS measurement has been influential to the skills we are looking to measure with the iSAT CPS measurement framework. The work presented in your video is also relevant to the iSAT framework. One of our goals is to build out more affective skills/behaviors that contribute to successful CPS -- I will be adding confidence to that body of skills after seeing your video!

Hi Joshua, thanks for your comment. We are certainly considering diversity -- along social and economic dimensions, as well as learning and interaction preferences for students. There are a couple hurdles we are working to address. 1) Much of the collaboration measurement literature is focused on drawing insights from students' communications (e.g., transcripts). We recognize communication is not every student's contribution to a group, so we are working on multimodal measurement, and will now also look into ideas we've seen in other presenters' STEM for All videos for this, as well. 2) The data we are collecting from schools tends to have a bias regarding those students (parents) who opt into our data collection efforts. Generally, those who skew higher on levels of socioeconomic status are more likely to be willing to participate in data collection. In forthcoming data collection efforts, this is a priority that we will address.

Hi Josh, thanks for your great questions! 

• There are often multiple activities going on in the classroom at any point in time, with the teacher only able to support one individual or group at a time. One of our goals is for the AI agent to be able to offload some of the support so that the teachers can more easily facilitate or help students who need their help the most. This may include (but not be limited to) recognizing inequitable participation (which the teacher may not have noticed in one group while they are helping another) and encouraging more collaborative behaviors.

• Our intention is not to have the AI agent replace a human in the classroom, but rather, be a complementary tool to the teacher / staff support. One teacher is not always able to reach multiple students or groups simultaneously, which is where the AI would have an advantage if a teacher is already busy helping a particular group. However, the teacher will also have access to group collaboration metrics and using human-in-the-loop features, be able to better train the AI when to intervene.

• We are working very closely with teachers and students (currently across the US) through events such as the Learning Futures workshop that are grounded in co-design. This helps us to iteratively build and redesign based on what we learn from our students, from their past and current perspectives (e.g., collaboration experiences in the classroom), and their visions of ideal or desirable environment(s). A key component of our design process is considering customizable features that allow for addressing the wide range of different specific needs across our users.

Hi Indrani.

Thanks for your response.

I understand that the agent is not to replace a human. So, assuming the lead teacher in the classroom stays the same, my curiosity is around the value of an AI agent vs. a second trained human, either a less-experienced co-teacher or a paraprofessional. And of course, I was mainly curious about the costs & benefits around educational and social outcomes, recognizing that there are huge practical implications, financial and otherwise, but choosing to ignore those for now.