468 Views
Icon for: Kurt VanLehn

KURT VANLEHN

Arizona State University

FW-HTF: The future of classroom work: Automated Teaching Assistants

NSF Awards: 1840051

2021 (see original presentation & discussion)

Grades 6-8

When students work collaboratively in groups on challenging problems, they can achieve great learning.  Unfortunately, groups often do not work collaboratively.  Although the teacher can visit and help, the teacher can only visit one group at a time.  Our system, named FACT (Formative Assessment with Computational Technology), assists teachers in several ways.  If a group stops collaborating and lets one individual take over, FACT reminds them to collaborate.  If that fails, it shows an alert on the teacher's dashboard.  If a group gets done early and stops working, FACT assigns them new work (prepared by the teacher, of course).  FACT continuously monitors the students' work, with progress bars for both correct and incorrect work.  When the teacher looks at the student's work remotely, FACT highlights regions where errors exist, and tapping on such a region displays the misconception behind the error.  If the teacher needs to make a brief announcement to all the groups, pressing the Pause button causes their screens to lock and display "Eyes on teacher" until the teacher unpauses the class.  FACT continuously monitors the students work, with progress bars for both correct and incorrect work.  FACT has been deployed in both face-to-face and Zoom classrooms in an extensive series of formative evaluations. 

This video has had approximately 81 visits by 67 visitors from 46 unique locations. It has been played 33 times.
activity map thumbnail Click to See Activity Worldwide
Map reflects activity with this presentation from the 2021 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2021 STEM For All Video Showcase (17 posts)
  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Facilitator
    Senior Project Director
    May 11, 2021 | 06:27 a.m.

    Kurt and team, I enjoyed learning about your formative assessment program. I have a sense of how it works in the classroom. Would you say a bit more about some of the advantages or disadvantages of using it for remote teaching and learning contexts?

  • Icon for: Jaime Gutierrez

    Jaime Gutierrez

    Facilitator
    Research Associate II
    May 11, 2021 | 04:55 p.m.

    Hi Kurt and team, thanks for sharing. I'm curious to hear more about what it means to collaborate effectively and what that looks like in a classroom while also using technology. The video highlights that children talk as they work. What are metrics are used measure collaboration?

    Also would be interested in learning more about what teachers and students think about this.

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 11, 2021 | 06:16 p.m.

    Thanks for your question, Jaime.  Teachers varied in the priority they assign to collaboration.  Some valued collaboration and spent several weeks at the beginning of the school year teaching students how and why to do it.  Others don't care much about collaboration and rarely used group work.  But FACT's lessons require group work (FACT doesn't but the lessons do).  FACT currently classifies the group's dynamics using their user interface actions.  For instance, if both students move the same card within a few seconds, then they are probably collaborating.  If they are simultaneously moving different cards, then they are probably working independently.  However, if only one student is moving a card, it is ambigous.  We can tell from their speech whether they are discussing the move before one student executes it (collaboration) or there is no discussion (no collaboration).  We have developed a speech-based addition to FACT that converts students' speech to binary (silence vs. speaking) audio, and will be testing it as soon as schools reopen in order to see if it improves the detection of collaboration.

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 11, 2021 | 06:22 p.m.

    Thanks for your question, Catherine.  We used FACT in classes where the students had been using Zoom for months.  In classes where the students were used to collaborative work in breakout rooms, one room per group, things went smoothly when FACT was introduced.  In other classes, the whole collaboration-in-breakout-room routine was unfamiliar and proceeded rockily.  In both classes, students preferred to stay muted despite reminders from the teaching staff.  That as the biggest disadvantage.  The biggest advantage was when the students did talk, then their audio was high quality, which we believe will allow us to detect collaboration and perhaps even mathematical thinking more accurately.  

  • Icon for: Marcia Linn

    Marcia Linn

    Higher Ed Faculty
    May 12, 2021 | 02:44 p.m.

    Hi Kurt! It's great to learn about the exciting work you are doing. This is especially valuable during COVID remote learning where teachers are struggling to set tasks that engage students in collaboration during breakouts. I wonder if you have some guidance on how to design activities that lead to effective collaboration?  See our work exploring the ways teachers use data while teaching for social justice in science--including helping students interpret scientific data. Enjoy, Marcia

  • Icon for: Channa Comer

    Channa Comer

    Facilitator
    STEM Educator
    May 12, 2021 | 07:56 p.m.

    Hello Kurt,

    Thank you for your presentation. What an innovative idea! Do you foresee this technology being adapted for other subjects? What is the time commitment for teachers to learn the technology, prepare and upload the activities for FACT? How do you measure successful collaboration and any subsequent impact on student learning?

  • Icon for: Brian Gane

    Brian Gane

    Higher Ed Faculty
    May 12, 2021 | 09:49 p.m.

    Hi Kurt and team, impressive system and demonstration! Have teachers using the system taken advantage of the misconception identification feature as shown in the video? If so, how useful have they found it and what methods do they use with their students to respond? 

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 13, 2021 | 10:52 p.m.

    Hi Marcia, Thanks for stopping bye.  No, I don't have advice on designing activities to enhance collaboration.  Our forte is providing software the helps designers effectively implement their designs.  However, we work with some great designers (the MAP folks), and their lessons have lots of clever collaboration-enhancing idea.s

     

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 13, 2021 | 11:01 p.m.

    Hi Channa.  Thanks for the 3 questions.  (1) The FACT software only works with a small set of math lessons.  That was intentional, since once we find a set of pedagogically effective interactions, it should be possible for others to generalize the software so that teachers can author their own lessons and enjoy the benefits of such interactions.  (2)  All our teachers chose lessons that they were familiar with in their paper form, so all they had to learn was how to use FACT.  This took about an hour.  (3)  FACT currently measures collaboration by the chronological patterns of group's editing actions.  At our next trial, FACT will include the group members' speech as well. 

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 13, 2021 | 11:07 p.m.

    Hi Brian.  We originally thought that teachers would really appreciate knowing the misconceptions, so FACT displays them for both individuals and displays the misconceptions that are most prevalent in the class.  However, teachers were not able to use such detailed information during class as they addressed either individual students or the whole class.  They merely noted where the error occurred and what the corresponding correct concept was.  They said that they wanted to be able to study the misconceptions after class was over, but during class, they could only give prompts and hints that encouraged learning the correct concept rather unlearning the misconception.   

  • Icon for: Brian Gane

    Brian Gane

    Higher Ed Faculty
    May 15, 2021 | 02:08 p.m.

    Hi Kurt, thanks. Definitely makes sense, I imagine there is so much for teachers to monitor in the moment and especially as they're still learning the system and how to best facilitate/monitor their students' collaboration. But using that information after class definitely sounds promising! 

  • Icon for: margaret smith

    margaret smith

    Higher Ed Faculty
    May 14, 2021 | 04:56 p.m.

    This would be useful for my college classes too!

  • Icon for: Raffaella Borasi

    Raffaella Borasi

    Higher Ed Faculty
    May 16, 2021 | 06:19 p.m.

    This is a fascinating application of AI to the education field - and the only one I know of that was funded by the Future of Work program!  I am part of a group that was recently awarded a Future of Work planning grant, so I would love to know more about who you integrated the Future Work, Future Workers and Future Technology dimensions required by Future of Work grants. 

  • Icon for: Jeremy Roschelle

    Jeremy Roschelle

    Researcher
    May 17, 2021 | 05:15 p.m.

    I enjoyed catching up on our work with this, Kurt! Those math problems are SimCalc-type questions and I used to have a long discussion back and forth with Ken K about how AI (or tutoring) and SimCalc could fit together. We never got anywhere. This seems like a really smart fit to me. Thanks for sharing.

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 17, 2021 | 07:40 p.m.

    Hi Raffaella,

    Our Future Work is middle school teaching; the Future Workers are teachers; and the Future Technology are automated teaching assistants.  Hopes this helps with your planning.

    --Kurt

  • Icon for: Kurt VanLehn

    Kurt VanLehn

    Lead Presenter
    Prof
    May 17, 2021 | 07:49 p.m.

    Hi Jeremy,

    Yeah, all us Pittsburghers once thought that machines should teach and teachers should do everything else.  Tutoring SimCalc-type questions would be "half challenging" in that AI could do the analysis of student performance, but AI would find it very challenging to interact with the student so as to encourage learning. 

    I now live way outside Pittsburgh, and I believe that teachers should teach the hard, confusing and/or interesting stuff, and machines should do everything else (except handle non-social behavior).   If teachers need an analysis of a student's performance to do their job, then the AI can supply it, but not all teachers want those analyses all the time.

    -- Kurt

  • Icon for: Susan Warshaw

    Susan Warshaw

    External Evaluator
    May 18, 2021 | 03:00 p.m.

    I have taught courses comprised of adult learners where collaborative problem solving assignments were implemented.  It was challenging for me to jump from one group to another to keep them focused. This tool would have been most helpful. Adult learners are tempted to socialize rather than dig into the assignment.     

  • Further posting is closed as the event has ended.

Multiplex Discussion
  • Members may log in to post to this discussion.