Hi Nancy. Thank you for checking out our video. 

We haven't dug into the data on what pre-service teachers actually learned, but we did strategically design the math and science tasks to try to invite argumentation. The student solutions that we created to guide the actors in the simulated environments are nuanced in ways that call pre-service teachers' attention to supporting the student avatars to provide evidence and reasoning in explaining their work.

With the four teacher educators that we have observed this spring implementing our Online Practice Suite in their elementary math and science methods courses, we have found that opportunities for pre-service teachers to self-reflect on their questioning and the ways in which it elicits students' thinking have been helpful. The teacher educators are also providing models of productive practices (either directly from our simulations or in other forms) that the pre-service teachers have mentioned are helpful to their learning. 

This is interesting.  I will look forward to hearing more about how teachers are benefiting from the use of your tools.

Hi Michael,

Great questions, I would love to hear more about your work. The majority of our simulations are driven by non-artificial intelligence at the moment (ie, human beings). That said, the training for the interactors who drive the simulations behind the scenes is extensive, and the human training does set parameters that could be thought of as similar to cues. The way we embed this in teacher training is within methods courses, so there is also a lot of non-artificial intelligence being used by the teacher educators as they help their students (the preservice teachers) make sense of an reflect on their experiences. However, I think there's the potential for diverse approaches, including PD and school-based lesson study types of situations.

We've thought a bit about AI and NLP interactions and there's probably a couple of productive directions. On related projects we've been exploring how to analyze performances and produce meaningful feedback automatically, but this work is emergent. Eventually one could imagine automating the actors, but my hunch is that before the tech gets there its likely it might get to an intermediate step in which automatic analysis provides prompts or information to the human actors allowing them to activate various paths without having to do all the work themselves, so maybe semi-automated.

We've selected a classroom practice that is pretty difficult - facilitating argumentation - so I guess I would say we are ambitious with respect to what can be emulated. That said, I think there are some things that are really hard to imagine doing well, such as practices that unfold over time (setting norms, building trust with students). I'd be curious from your perspective where you think the limits are.

Thanks for this question Michael & the response Heather.

As a follow-up, is the immersive classroom something that can be engaged with a laptop or with a VR headset (if the latter, what headsets are supported)?

Hi Karl -- Thanks for the question and for taking a look at our video. We have two immersive classrooms that we are using on this project. The avatar-based simulations use the Mursion classroom environment with a small group of student avatars; this classroom can appear on either a television or a laptop screen. The virtual teaching simulator (the immersive classroom with a whole class of students) can be engaged with using a laptop or a VR headset. However, due to the pandemic, we are currently only able to use the virtual teaching simulator on a laptop since all of our sessions have been conducted virtually at this time.

Hi and thanks for the questions about VTS. Right now the application is only computer-based. Now that much of the initial development is out of the way we are planning to move to develop for headsets, probably for Oculus and Vive. Please let us know if you have further questions. 

That sounds like a great idea for a grant proposal Jonee- we should talk!

Agreed! Our group has an email address (vearmiproject@ncsu.edu) but let me know if you have a better idea for connecting

Hi Jeremy,

Great question. In addition to serving as a Co-PI, I am also a teacher educator on the project who has used this directly with my own pre-service teachers (PSTs). Each of the three components has its own strengths. The chatbox had less of a steep learning curve. In my experience, it was a very helpful tool for my PSTs to start considering how they can ask questions to try to elicit one student's thinking at a time; this was the Eliciting Learner Knowledge tool by my colleagues at MIT who lead the Focused Practice Spaces (FPS) work on the project. Using this tool, they were able to practice asking open-ended questions, asking for claims and evidence-based reasoning, and trying to do these things without telling, evaluating, or funneling the discussion towards a right answer. While this was a terrific tool, I was eager to move to the next one, the Avatar-Based Simulations (ABS), which utilized the Mursion classroom of five students that my colleagues at ETS, I, and my graduate students at Towson University support. For ABS, my PSTs needed to not only elicit student knowledge, but do so across a range of students/groups - and then move onto engaging those students in critique and eventually consensus. This tool enabled my PSTs to practice in real time how to engage multiple students in an argumentation discussion. Finally, the third activity in our Online Practice Suite, the Virtual Teaching Simulator (VTS) supported by my colleagues at Indiana University and UNC Chapel Hill, enabled my PSTs to learn to transition from small group discussion to whole class discussion. This enabled them to practice multiple skills, including monitoring, before shifting into the whole class argumentation discussion. While I am a big fan of the chatbox (FPS) technology and feel that it has much to offer, I greatly appreciate the additional affordances that the other tools provide.

You asked about tradeoffs and there are some, of course. There is more of a learning curve for the PSTs to learn how to use and be comfortable within the real-time avatar simulations - ABS and VTS. They require our PSTs to connect to or logon to different systems in order to participate remotely, as they have been all semester. Also, on the back end, ABS and VTS both require one or more actors who are trained to "be" the avatars; this takes a significant amount of training and time (and thus money) to train these actors and then coordinate and deliver the sessions that the PSTs facilitate.

Thanks for your excellent question. Please let us know if you have others!

Pam Lottero-Perdue

Its a bit different across the three activities. FPS naturally affords some feedback, offering the participants in the eliciting student thinking activity, for example, information at the end asking the "student" and "teacher" the same questions and letting them know whether or not they match, and it provides a transcript of the interaction for reflection. The teacher educators used this one in class, and this allowed them to immediately engage their PSTs in reflection, in some cases then having them go right back in and repeat the activity to see if they could make changes. Each of the two avatar simulations finishes with an opportunity for the PST to reflect on the discussion - in ABS for example, after leading the discussion among the student avatars, the PST is dropped back into a room with an adult avatar who was earlier introduced as the guide to check that everything was working and get them started. The adult asks them a few questions about how things went to capture their reactions in the moment, and this is part of the video-record the PST receives later. We haven't systematically analyzed these self-reflections, but we do have some evidence from past project work that the in the moment reflection is a bit more raw and honest compared to after the fact reflections, which provides some interesting instructional material the teacher educator can choose to make use of. We aren't providing automatic feedback on the avatar simulations (or non-automatic feedback for that matter), although in related work we did do provide human-generated feedback and found, as you suggest, that it was a powerful driver of PST learning. So for this project, its really up to the teacher educator whether or how they make that happen.

We are hoping in future work to press hard on the problem of how one might generate automatic feedback that is sufficiently accurate and actionable to support learning, although at the moment, its still a use of machine learning that is exploratory, not fully developed. Important to develop though, because feedback is so important, and the cost of producing it is a significant challenge to scale!