2180 Views (as of 05/2023)
  1. Marcia linn
  2. http://wise-research.berkeley.edu/mclinn
  3. Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. UC Berkeley, Educational Testing Service
  1. Sarah Bichler
  2. Post-Doctoral Researcher
  3. Presenter’s NSFRESOURCECENTERS
  4. UC Berkeley
  1. Kelly Billings
  2. Graduate Student Researcher
  3. Presenter’s NSFRESOURCECENTERS
  4. UC Berkeley
  1. Allison Bradford
  2. Graduate Student Researcher
  3. Presenter’s NSFRESOURCECENTERS
  4. UC Berkeley
  1. Libby Gerard
  2. https://gse.berkeley.edu/libby-gerard
  3. Researcher
  4. Presenter’s NSFRESOURCECENTERS
  5. UC Berkeley
  1. Weiying Li
  2. Graduate Student Researcher
  3. Presenter’s NSFRESOURCECENTERS
  4. UC Berkeley
  1. Jonathan Lim-Breitbart
  2. Designer and Developer
  3. Presenter’s NSFRESOURCECENTERS
  4. UC Berkeley
  1. Ken Steimel
  2. https://steimel.info
  3. Assistant Research Engineer
  4. Presenter’s NSFRESOURCECENTERS
  5. Educational Testing Service
Facilitators’
Choice

Natural Language Processing Technologies to Inform Practices in Science (TIPS)

NSF Awards: 2101669

2022 (see original presentation & discussion)

Grades 6-8, Grades 9-12

Middle school science teachers want technology to amplify their impact on the hundreds of students with multiple, diverse ideas in their classes. By exploiting advances in natural language processing (NLP) this project detects the rich ideas that students develop during everyday experience. Aligned with the Knowledge Integration pedagogy, this project takes advantage of NLP methods for detecting student ideas and creates adaptive guidance that gets each student started on reconsidering their own ideas, considering their peers’ ideas, and developing more coherent explanations for complex scientific phenomena. This work continues a successful partnership between University of California, Berkeley, Educational Testing Service (ETS), and science teachers and paraprofessionals from six middle schools enrolling students from diverse racial, ethnic, and linguistic groups whose cultural experiences may be neglected in science instruction. The idea detection technology adapts to each student's cultural and linguistic entry points into the topic. Our video illustrates how TIPS empowers students to use their ideas as a starting point for deepening science understanding.

This video has had approximately 275 visits by 219 visitors from 131 unique locations. It has been played 154 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2022 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2022 STEM For All Video Showcase (12 posts)
  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Facilitator
    Senior Project Director
    May 10, 2022 | 12:12 p.m.

    Hello Marcia and team! This is fascinating work. I would love to know more about so many aspects of it. For instance, would you say more about how this web-based system helps students consider their peers' ideas?

  • Icon for: Allison Bradford

    Allison Bradford

    Co-Presenter
    Graduate Student Researcher
    May 10, 2022 | 03:30 p.m.

    Hi Catherine! Thank you for your question. As we further develop the NLP techniques that detect the ideas expressed in each student response, we hope to use the technology to dynamically partner students. For example, this would enable us to automatically pair students who express different ideas about a phenomenon and use the Thought Buddy to scaffold a discussion where the students consider each other's ideas. While we our still in early stages of this project, we are leveraging our partner teachers' insights about how to engage students with their peers' ideas. For example, they have suggested having students compare the prompts they received from the Thought Buddy or displaying interesting chat examples to prompt whole class discussion.

  • Icon for: Allison Bradford

    Allison Bradford

    Co-Presenter
    Graduate Student Researcher
    May 10, 2022 | 02:48 p.m.
    Welcome to our video! As we embark on this project, we are grappling with many open questions. For example, multiple ideas (both normative and alternative) are often detected in a student explanation. These alternative ideas typically represent thoughtful effort from the student. With this in mind, we wonder:
    1. Which idea should the thought buddy respond to?
    2. Which serves as a more productive starting point?
    3. How do we design guidance that confers respect and encourages reflection?

     

    We welcome your insight into these questions and any other comments or questions you have for our team!  

     

  • Icon for: Mike Szydlowski

    Mike Szydlowski

    Facilitator
    K-12 Science Coordinator
    May 11, 2022 | 09:29 p.m.

    I love this idea and want to try it!  Like any good program or idea, this takes time. What would you say that the teachers gave up to spend more time on this process?  Do they feel it was a good trade off?  

  • Icon for: David Campbell

    David Campbell

    Facilitator
    Program Officer, retired
    May 12, 2022 | 10:27 a.m.

    Hi Marcia and team!  I loved hearing the students say they didn't have to worry about "being wrong."  It seems the thought buddy removes some of the peer pressure students face in the classroom.  Following up on Mike's question, many teachers feel they face conflicting challenges of "covering the material" and taking the time of take advantage of students' individual ideas.   I wonder if your teacher participants expressed that concern.

  • Icon for: Sarah Bichler

    Sarah Bichler

    Co-Presenter
    Post-Doctoral Researcher
    May 12, 2022 | 10:57 a.m.

    Hi Mike and David, 

    thanks for your interest and this important question. Teachers I worked with definitely expressed the concern not being able to cover the material - when trying to let students explore more, bringing in students' ideas, letting students revise, etc. With respect to the dialog, teachers really appreciated that each student was able to discuss with the avatar and that each student got unique feedback, something they could not have done in the time the chat took. I have seen teachers weave in students' ideas to jump-start a class discussion, it did not take long at all :). We hope that the dialog makes many ideas visible, placing them for being taken up beyond the chat and working with teachers to find ways in which students' ideas can become central to instruction and ways in which these ideas can help to "cover the material". 

  • Icon for: Amy Robertson

    Amy Robertson

    Researcher
    May 16, 2022 | 01:52 a.m.

    This is so cool, y'all!  I have so many thoughts bubbling up.  As someone who deeply values responsive teaching -- particularly responsive teaching that attends closely to the *substance* of students' thinking -- I've been somewhat skeptical of NLP because it feels like it can miss some of the depth and connections within student ideas.  BUT some folks on our team just did an analysis of student responses using SLP and found that with a little bit of additional work on the part of analysts (looking at some of the top example responses), they could identify many of the same patterns that another team of researchers did without NLP.  So in this case, NLP was a really helpful tool. 

    I'm processing out loud a bit, but I'm curious about the trade-offs you see along the lines of being able to discern and respond to substance...  And we'd love to chat with you offline sometime.  We share a deep value in framing students' ideas as generative and finding ways to build from there.

  • Icon for: Sarah Bichler

    Sarah Bichler

    Co-Presenter
    Post-Doctoral Researcher
    May 16, 2022 | 04:32 a.m.

    Hi Amy, thanks so much for stopping by!

    I love these thoughts bubbling up! I feel if the human coding pays attention to the depth and connections within student ideas, the machine is likely to pick it up. We refine our coding multiple times and go through a rigorous training procedure before coding or annotating our training data. Our results show that if the model can detect an idea, it actually does it very well (accurate and is sensitive to the idea). We were more worried about the infrequent ideas, that we don't find ways to make these visible, an area we are still working on. 

    With respect to your question, I feel there are trade-offs responding to different ideas students express. If a student has multiple ideas, which one do we respond to and why? We don't know yet what's most effective. For example, a student thinks that a metal bowl conducts cold faster than a wood bowl. I see two productive ideas. The student could explore the idea of "cold flow" more, investigate, and develop it. Or, the student could expand and build on the idea of the different rates of thermal conductivity...

    I'd love to talk more!

    Sarah

     
    1
    Discussion is closed. Upvoting is no longer available

    Amy Robertson
  • Icon for: Ann Podleski

    Ann Podleski

    Higher Ed Faculty
    May 16, 2022 | 11:20 a.m.

    I think this type of tool has a lot of potential for some students.  And every different "tool" for engaging and promoting student reflection and thinking is helpful for teachers.  The fact that students can "ask anything" and not worry about what the teacher might think is one of the benefits I can see for this.  Nothing should totally replace the "human" interaction, but I can see this enhancing learning and teaching and allow for teacher's time to be used in the ways that are most effective for each different student.  And I think there are clear trade-offs, but each additional venue we provide is a plus.  Thanks for this work!

  • Icon for: Sarah Bichler

    Sarah Bichler

    Co-Presenter
    Post-Doctoral Researcher
    May 16, 2022 | 11:26 a.m.

    Hi Ann, 

    thanks so much for sharing your thoughts with us! I speak for our entire team when I say I absolutely agree with you that no technology should replace the teacher, nor would it ever be able to do that. We are working closely with teachers to learn how these tools best support them in their practice and to refine the tools we design such that we can maximize their value for practice. The potential of such tools might be that they help us scale what effective teachers do anyways. When teachers are facing many demands on different levels of teaching, such tools might help them provide the support to their students they would like to. 

    Best, 

    Sarah

     
    1
    Discussion is closed. Upvoting is no longer available

    Ann Podleski
  • Icon for: Ann Podleski

    Ann Podleski

    Higher Ed Faculty
    May 16, 2022 | 11:30 a.m.

    Totally agree - anything that can help teachers provide support to students is a plus!!!  (And I don't know if this has the option for the teacher to look at what students are saying - again that would take time, but asynchronous, which is helpful.  Also, the more I can see what students are thinking, the better I can help all students.

  • Icon for: Sarah Bichler

    Sarah Bichler

    Co-Presenter
    Post-Doctoral Researcher
    May 16, 2022 | 04:19 p.m.

    Hi Ann, thanks for keeping the conversation going! Yes, teachers can see the entire dialog students had with the avatar in their WISE teacher tools view. They can see the initial explanation, the adaptive feedback from the avatar and the student's response + how they revised their explanation. It does take time to go through the chats for all your students. I have seen a teacher look at some of them to get a better sense of how students are thinking about the science between class periods or at the end of the school day. It's kinda fun reading through. Other teachers looked at what specific students had said to see if these students needed guidance or not. Yet another teacher printed the chats and used them for discussion in the class! We're also thinking about using these chats during PD when teachers have time to really dive in and explore their students' ideas. But, as you're saying, it does take time!

  • Further posting is closed as the event has ended.