Thanks for your question Dionne. One of the unanticipated challenges on this project was the extent of training and human resources needed to generate specific and targeted formative feedback to share with each preservice teacher (based on his or her video recorded discussion performance). Building on the current research and literature, we developed a scoring rubric and rater training materials focused on five dimensions of high-quality argumentation-focused discussions (e.g., eliciting student ideas; promoting peer interactions; etc.). Raters not only scored each video based on these five dimensions, but also wrote formative feedback linked to each dimension for each video to identify strengths and areas of growth for each preservice teacher. These formative feedback reports were used by both the preservice teachers and their teacher educators during the formative semester, and they are one of the reasons we hypothesize we saw the growth that we did in the formative semester. The formative feedback is also something that we think is important as part of a larger cycle of enactment to support teacher learning and are interested in examining how the nature and type of the formative feedback, including self-reflection, influences teacher learning from the simulated teaching experiences.

Hi Sabrina -- Thanks for taking a look and for the questions. Teachers definitely influence the content of the simulated teaching experiences in a few ways. First, we had teachers (both current and retired) who were a part of our mathematics and science task development teams from the very start of this project. As part of the task development teams, they helped to decide the content focus of each of the eight performance tasks, including the student learning goals for each discussion and the student work that we embedded in each task. Second, when the preservice elementary teachers engage in the discussions in the simulated classroom, they influence the direction of the discussion in the simulated classroom and the content that is (or is not) addressed throughout the 20 minute discussion. The interactor (that is the human-in-the-loop who is trained to act and respond as the five student avatars during the discussion) is trained to follow each teacher's lead during the discussion and to engage with the content, as prompted by the teacher during the simulated teaching experience. As for next steps, we are hoping to build out a similar set of performance tasks at the lower elementary level (K-2) and study the use of those tasks within educator preparation programs. We are particularly interested in examining under what conditions and for what purposes (e.g., use of different simulation use models and/or types of feedback) using these types of experiences supports teacher learning. We are also interested in expanding into the in-service teacher space to see how such experiences could be embedded into professional development and teacher communities of practice.

Thank you for sharing. I am now exploring the data in the repository. This is a fantastic resource. I will look into adding some of our data here as well. 

One more question - are the interactions using a machine learning approach to determine the responses of the avatars (based on the PST remarks), or is a human playing the part of each of the student avatars in real time? If it is the latter, what do you find are the advantages of simulating the interactions in the computer, versus doing it "live" in the classroom?

I'm so glad you were able to get to the data. This is our first time using the resource, and its such a fantastic tool for getting the work out there into the world.

In this work, a human is playing the part of the student avatars - in fact one human is playing all five avatars. Although its computer aided in the sense that there are some automated movement sequences to manage things like the whole group raising their hands, voice modulation to make sure they all sound different from one another and consistent across various actors, etc. Remy commented about this as well and I'll talk a bit more about machine learning in that response.

Carrie Straub, our co-pi at Mursion, wrote a nice white paper that outlines the pros of working with simulated students in general as an approach.

From my point of view, probably the most important advantage is simply your control, pedagogically, over what the teacher in the simulator is faced with or asked to do. There are a number of scholars who work on what I would describe as rigorous rehearsals, in which significant time goes into helping preservice teachers act as children would for the benefit of their peers who practice teaching them, but I'd say that is not the norm - in most cases the teachers do their best but, given that they are not trained actors and are just starting to learn how children think, representing children authentically is quite difficult, as is not giving into the inevitable urge to help your peer by learning. Because the actor on the back end is trained extensively they respond much more authentically to how children might, and more consistently, meaning that whatever dilemma of practice I've decided as a teacher educator that I want my preservice teachers to face, I can feel sure that each and every one of them will encounter that dilemma.That has mattered immensely in our work, because preservice teachers opportunity to even observe argumentation-focused discussions, much less really dig into leading them, is so limited in field placements that you could wait a whole semester hoping it would happen once for each of your preservice teachers and find that it didn't for most of them. The simulator means you can guarantee it happened for every single one of them the same week, at the moment that makes sense within your course planning. I would also argue (but just my opinion) that given the cost of the technology this is why it makes sense to use it to simulate the types of ambitious instruction that we want to see more of but don't yet, making it hard to provide preservice teachers the type of support they need to enact it. Thats probably where the payoff is highest.

In terms of authenticity, the students definitely look animated - no one would mistake them for real people - but in truth its not more of a stretch to imagine them as real than it is one to imagine your classmates as children.

The AI question is such a timely one. The technology we work with (Mursion) is what I describe as non-artificial intelligence, as in, human beings driving the simulation behind the scenes. A bunch of folks are working toward AI interactions in this space, and it seems like the natural next step but it also doesn't seem like we are quite there yet. AI does pretty well on interactions that are tightly scripted but a dialogue between 6 players, even if five of them are artificial, and even constrained to one scenario, can play out really differently across a 20 minute time span. We've been exploring what it would take to produce automated feedback, building on the work we did in this project to score and provide feedback to the pre-service teachers based on their video-recorded teaching sessions. Even that is complicated, and it doesn't have to happen in the moment with instant speech recognition the way it would need to to drive the conversation itself. My hunch, although I could be wrong, is that what will happen first is that AI will make progress in analyzing common patterns of recognition in fits and starts, unevenly, with more success in recognizing some patterns than others, but that this might provide really useful feedback loops to the interactors (the actors who control the avatars) by prompting them with various responses to consider based on what the teacher has just said, still allowing them to make final judgments about what makes sense in the moment but providing a support to mitigate some of the cognitive load this work places on them.

In terms of challenges, I don't have a background in machine learning, but having had a lot of conversations I'd say almost certainly one answer is that we can train machines to do a lot of things, possibly anything, if we work at it and have enough data, but that is a significant challenge when your data point is a 20 minute long video that is fairly expensive to come by. Over the five years of this project and a couple of related ones, we have in hand a couple hundred videos that are usable for this type of analysis, which is a huge data set from the standpoint of educational research and small potatoes in the machine learning world.