Thanks, Miriam.  We hope to provide opportunities for practice that might not be readily available.  The ability to facilitate classroom discussion is something that even veteran teachers may need support with, so we find that this technology gives a safe place to practice and make mistakes.

Hi, Courtney! :) We are eager to see how the teacher educators provide formative feedback as well.  When we began talking about the project, I had it in my mind that we would be structuring the feedback, but actually that was not the case.  One of the interesting facets of this project is that we are not prescriptive in terms of feedback,  but looking to see how teacher educators use this technology to inform their teaching, and what method of feedback they find to be most helpful.  As we start in on the formative tasks in the coming year and teacher educators are integrating the simulations into their coursework, we will surely find out a lot. 

We are working out the feedback question now. Our plan is that there are (at least) two layers to this. First, our raters will provide some feedback as they score performances. We won't be giving the preservice teachers scores (and they've made in quite clear in focus groups that they don't want them!) but we will provide feedback, which will be aligned to the scoring rubric but written in narrative form and citing specific examples. Second, as Carrie mentioned, the teacher educators may choose to give additional feedback. Beyond this, its possible that the teacher educators will choose to structure course assignments in ways that provide other types of feedback - for example asking the preservice teachers to reflect on their own or others' performances.

Hi, Brian!  I can help to respond to question 2, and I am sure Heather has great knowledge in this area too.  The technology allows for both type of interactions, whether the teacher is mediating the conversation, or has built the skills with the group so that they can discuss with minimal teacher intervention. One of the challenges of the work is knowing when to shift the simulation in response to the teacher's behaviors.  For example, should students "unlock" and begin discussing after 1, 2, 3, ... prompts, and if some prompts to discuss are more effective than others, how should students respond differently?  These questions often arise when creating simulations and student profiles to support class discussion. 

Great questions! So far we are only in tryouts and early data collection, so haven't had the opportunity to observe change. But some of the things that stand out in the performances so far are (not surprisingly) a tendency for teacher-talk to dominate, and for the participants to struggle with the idea of what argumentation is. I think we already know from prior studies that these things are hard to change, but our hope is that doing it "with training wheels on" might make it a little easier in a number of ways. First, having that common experience with peers and the chance to receive feedback should help with noticing. Second, having a chance to engage multiple times should create a space of relative freedom in which to try out some more ambitious and perhaps intimidating teaching moves.

We don't plan in this study to study the bridge between this approximation and working with real students, but its a fantastic follow up idea.

So far we haven't seen that behavior. Our tryout participants have predominantly been novice teachers, ranging from very early in coursework before fieldwork to a few who have been in the classroom for less than a year. For the most part they take it seriously, and if anything find it a bit nerve racking because they are teaching in the lab in front of research staff. And to the extent that they occasionally don’t give it a full effort I’d say it falls much more in the category of not preparing sufficiently and not deliberately trying to undermine the effort. There also isn't technically avatar "coding" in terms of their responses because, although the images are of course coded and the interaction mediated by software, they are controlled on the back end by a trained human being, a "simulation specialist". That said, I think your point still holds. Whether computer coded or dictated by the training and experience of the simulation specialist, participants can still push the envelope by attempting things that are truly unexpected, perhaps because the want to break the system as you suggest, but also perhaps because we just haven’t anticipated some of the things they could do or say adequately to prepare the simulation specialist. The training is extensive, but it doesn’t mean something really unexpected could not happen. I’d say we have strong training to cover most of the likely approaches participants will take, some general sets of parameters to help in the moment if something that makes sense but is less likely occurs, but in truth if a participant were to go 100% off the intended path (say, they decide not to teach math today but instead to discuss world literature), well, in those cases we really rely on the simulation specialist’s background training in improvisation to figure out what to do in the moment that makes sense as a response. Again, not something we’ve seen, but you never know:)

Heather, now I understand the situation better - thanks. I thought that this was a sim in the usual sense of an independently coded set of responses that might be adaptive, but were hard-wired. I wasn't aware that there was a real-time behind the scenes individual monitoring and changing the avatar responses. That is a rational way to do this, though of course less scalable. Have you considered crowd-sourcing responses of avatars though? Though difficult, that would be cool in that it could bring to bear the experiences of many teachers.

In the present study this is not a question we have yet taken on, although its clearly one of the more pressing ones for future research to address. One of the things I find exciting about the technology is that I think it pushes us to think systematically about what we really think the theory of action is behind how novice teachers learn to teach by practicing teaching. Pushes us because to set up a simulation you have to make choices, and making those choices adopts a theory of action, whether explicit or implicit. We’ve taken some stands in this work. For example, we hypothesize that practice leading a discussion in a smaller group of students helps prepare the novice to lead discussions in larger groups; likewise, we hypothesize that taking on ambitious teaching first absent the challenges of classroom behavior will make it more likely a novice can handle ambitious teaching in the face of those challenges, and we choose to use student avatars who behave. These are big claims about how novice teachers learn. That said, they aren’t bigger than the ones we routinely make in teacher preparation; ones like the assumption that peer teaching or microteaching in the context of a methods class where the “students” are adults and peers will help you learn to teach children who are not your peers, or even the assumption that student teaching under the watchful eye of a cooperating teacher prepares you do teach without that support. Note that my point is not that I don’t believe these claims (I do), but that we don’t often articulate them or support them with rigor.

So far, the data we’ve collected has been from small scale tryouts largely aimed at refining the tasks. Our participants in tryouts have engaged in the simulation just a single time, and without feedback or support. And so far, while we do see some variation in approach, the majority demonstrate more an IRE interaction pattern than true discussion. That said, the simulation can certainly handle it, and in our training we have worked towards this and put it in the test by going in and acting the part of the teacher ourselves, sometimes over exaggerating the degree to which we step back just to make sure the simulation specialists are prepared to deal with that if and when it happens. So I guess the conclusion from this is that, with notable similarity to what happens with novice teachers in real life outside the simulator, the students are available and able to drive the discussion but for the most part the teachers don’t know to or don’t know how to make that happen.

Starting next year all of this changes from our point of view as we collect data in methods courses where each preservice teacher uses the simulator multiple times and has the benefit of being able to reflect on their video record, receive feedback, and have the support of the course instructor. And we certainly hope that ultimately they are able to facilitate the discussion in a way that lets students drive. But we will have to keep you posted on that because we just don’t have the data yet!

Great question, Eli!  Heather addressed the main issue in her response below, so that should help to clarify how the simulation can respond in real time to the learners.  We use the virtual environment to create what I like to think of as a digital veil, that allows the simulation specialist to inhabit multiple characters (avatars) at the same time - up to 5 avatars at once!  These avatars can be operated by the simulation specialist or run by the computer through a series of algorithms.  The "human in the loop" adds the spontaneity and moves the simulation past being a basic logic-branching type of simulation, and into something much more interactive and realistic.