Hello Berkeley team! Thank you for your video, this is such a great way to customize a WISE unit. I was wondering if as part of this work you have tried to measure students' shifts related to their knowledge about (or positions on) racial equity, and if you have found any relationships between this and their science learning. One might hypothesize these learning outcomes might be synergistic but I imagine it would not be straightforward to measure this. 

What incredibly important and thoughtful work! I really appreciate the explicit integration of systemic racism into an existing science learning platform in such an authentic way. And in a way that stems from what the educators were interested in bringing into their conversations in the classroom. Such a powerful example of making data human-centered and personal, understanding the historical implications of the world we see today -- emphasizing the need for change in the future! Did you get feedback from teachers about how things went? Including what conversations came up?

Dear Caitlin, thank you so much for the feedback! Teachers who are running the units participated in a workshop in which we discussed ways to support students during conversations about racial equity in science class. Looking at student data, we get the impression that students really appreciate learning about issues like urban heat islands. Many of our teachers are still teaching distance. We are currently interviewing teachers - my colleagues will share more details about what we learn from teachers!

Best, 

Sarah

Thank you for your comments and interest in our efforts to welcome each student to science. We are investigating ways to support teaches to explore social justice issues, consistent with many students' concerns about the environment, BLM, and personal health. We seek to transform the goals of science courses from primarily preparing students for careers in science, to ensuring that each student, independent of race, culture, sexual orientation, or economic status is welcomed in science. This means developing the identity of each student as a science learner and respecting the science accomplishments of each person. Please tell us how your work can help us advance this goal.

Hi Marcia, Thanks for sharing such a great piece of work! I'd love to have a follow up discussion on applying NLP-supported innovations in classrooms to support teacher practices. Also quite interested in integrating social justice issues into student science learning. - Lei

I'd love to talk with you all more about how you are using NLP. It looks really powerful and I'd love to see more. Thanks for this work as a model for how to bring in new, important topics in science to make them more relevant for students.

Dear Lei and Judi, thanks so much for showing interest in our work! We'd love to share more about how we use NLP and implement NLP innovations in classrooms. Here is some more info:

1. We designed "milestone items" for several inquiry units. These items are complex and students have to link multiple ideas to explain a phenomenon. For example, in our Genetics of Extinction unit, students are asked "Why do siblings look similar but not the same?". Students can use ideas about genetic inheritance and genetic variation (gathered in the unit) to explain.

2. We develop a rubric to assess Knowledge Integration and also code for NGSS 3-dimensional understanding (DCI, CCC, or SEP dependent on the item). For example, another milestone in the Genetics of Extinction unit ask students to use evidence from a graph (SEP) to explain how species adapt to their environment.

3. Brian Riordan at ETS trains an automated scoring model and then we implement it in the units. Students' responses are scored in real time and teachers can view a report that is generated from the automated scores.

4. The report shows score distributions for KI and NGSS subscores and features example student responses. The report also includes recommended instructional actions that promote KI and/or are tailored to the ideas present in the respective classroom. For example, if students score really high on explaining natural selection but do not yet include evidence from the graph, recommended actions suggest ways to support students to use evidence.

Our reports now also show how students progress when they revised their responses after a teacher's intervention. In addition, teachers can view initial and revised responses (and KI scores) for each individual student. This way, teachers can more easily help those students who need the most help.

Looking forward to talking more and learning about your ideas how we can improve this work or what other work we can build on!

Best, 
Sarah

Hi Marcia and team,

The use of data in studying social injustices is really important, and you clearly have some rigorous research going on!  It may be beyond the scope of your project (or too controversial) but how you look at changes in students' attitudes about issues like anti-racism or climate change?  It might be interesting to use some common instruments across projects, and our project is in the early stages of research about youth's understanding of social issues related to COVID.  

Hi Jan!

Thanks so much for your feedback. It would be great to share resources and learning from other groups how we can best approach measuring change in attitudes towards and knowledge of anti-racism and/or racial equity in relation to climate change!

We are curious to learn more about your project and what items to measure understanding of social issues you are developing. We're also curious about your COVID instructional unit - we have designed a WISE unit about COVID and social equity too!

Dear Berkeley team! Wow, what a great presentation. It is so interesting to hear about two really important topics: helping teachers to see components of student responses related to the three dimensions and the integration of issues of social justice in the classroom. I am curious to know more about the teachers and their views on the place of social justice in science. While it seems like the teachers in the video were very keen to integrate the two, many science teachers shy away from these topics because they are unsure how to handle the discussion or questions that students may bring up. Did you all experience any of this resistance? Or how might you all help a teacher see the importance of bringing anti-racism, social justice into science? This is something I am very interested in and would love the thoughts of others working in the area. 

Marcia, what a rich set of tools you have created! I'll want to look at them more. You showed a child at home working; are these materials primarily for remote or online learning, or are they accompanied by hands-on investigations? What is the balance? And I see a lot of text; how dependent are the materials on children's reading levels?

Nice work....I am wondering if there were issues that were studied that were not only "urban" issues but also dealt directly with race? The two seem to get conflated sometimes....any ideas? thanks for the great work!

Great work, team! This environment is a rich resource for students' and teachers' learning. Building on Pendred's question above, I wondered about affordances and constraints of exploring issues that relate to anti-racism in a virtual environment. Are there opportunities built in to engage social learning among the youth participants? 

On a different note, I wondered if you're finding certain topics in the science curriculum that lend themselves better to integration with justice-oriented issues than others. 

Thanks for sharing! I'd love to explore this platform in more depth and plan to be delving into the website!

Dear Heidi, thanks so much for your comment and questions! I personally think that yes, there are certain topics in the science curriculum that lend themselves better to integration with justice-oriented issues than others. Climate change is one of them. I think this is because climate change impacts the world and us living in it "visibly" and students care about the environment. It is, in my opinion, easy to expand this and include social or political issues when teaching the science. I personally don't see how I can link social issues as easily with other science units I work on, for example Musical Instruments and the Physics of Sound Waves. However, now I am thinking about this and already have ideas about noise pollution... So I guess the more we think about how any science topic could be related, the more we learn about how it is related to social issues or how it can be taught and learned in relation to social issues. Thanks so much for sparking new ideas with this question!

Sarah

What a great project! I agree it's wonderful to have an authentic connection between racial justice and climate change. I wonder if the students might consciously avoiding using racism in their explanations - it's such a brave topic to raise for them as well as the teachers... and probably particularly challenging in a virtual space where communication can feel so tenuous. Virtual bravery is the highest form!  Do you know how the parents are perceiving this focus on racial justice in science? They may be listening/participating more than if their children were in an in-person class.

There seem to be three key elements:

1. The curricular materials and how they frame the phenomenon students are exploring.
2. The teacher facilitation.
3. The NLP scoring of student writing.

Are there other components that you feel are critical in the success of the project?

It seems that helping teachers facilitate and navigate potentially difficult conversations must be challenging. It would be great to learn more about the professional learning program and how it supports teachers beyond the STEM aspects of the curricular units.

I'm also curious about the decision to pull out specific scores for each dimension of the NGSS, separately scoring DCI, SEP, and CCC. In some of the NGSS-aligned assessment projects I've been involved in we had some deep discussions about how to score for progress toward NGSS performance expectations. The standards are written to integrate these dimensions, so how do you think about teasing them apart for separate scores?

Dear Daniel, 

thanks for raising such important questions! Separately scoring DCI, CCC, and SEP is something we also discuss a lot, within and beyond our research group. Here are some of my thoughts:

1. The teachers I have worked with highly appreciated seeing NGSS dimensions scored. They wanted ways to see if their students are meeting PEs. Scoring dimensions separately also has the advantage that teachers learn in which area students need more guidance. They can then target their instruction and supports they offer. I think it has great practical value to tease them apart. 
2. The KI scoring we do/show in addition to the subscores is (this can vary from item to item or researcher to researcher) often designed to evaluate how well students integrate ideas from both subscores. For example, if all my students are scoring KI 5 (or if one is), it means they explain natural selection and use evidence from the graph. This is one way of approaching it. However, I think we do not yet have a good approach to assess integrated 3-dimensional learning, one that could be standardized and/or is used by multiple groups and across science domains. I think we don't yet have agreement (or know enough about it) how items measuring integrated 3-dimensional understanding look like. Is that one item or is it a set of items? Is there one score that indicates degree of "integration" or are there subscores for each dimension that are aggregated? Can we empirically separate the 3 dimensions? These questions are intriguing and I would love to hear your thoughts!
3. Because I have seen the practical value of scoring dimensions separately, I started thinking about the underlying ability we are interested in. Would we say a student who explains a DCI, transfers knowledge across domains, and e.g., uses experimentation and or evidence, i.e., whose work would receive a high score on each dimension, integrated the three dimensions? In other words, is a student who is proficient in all of the three dimensions a student who we say has integrated 3-dimensional understanding? Or is integrated 3-dimensional understanding something more than the sum of its parts? I am so curious to hear your thoughts here as well!!

Best, 
Sarah

Addressing anti-racism is a thoughtful idea. It is not clear to me how the teachers used the web-based units to customize instruction for the students. Were the teachers and the students from diverse populations? How are the goals/objectives of the project evaluated? How is the Black Lives Matters concept incorporated into the project? It is stated that the project wants the students to connect their experiences in project activities to systemic reform. More clarity is needed to explain how this goal is accomplished. The concept of the project is commendable but not certain if the results intended will be accomplished   

Hi Marcia, Libby, and the Berkeley team! Thank you for the excellent video. I love the way of applying NLP to support responsive teaching. What an excellent idea! The Climate Change unit is a great example of infusing social justice into science learning. I’m wondering if you see any difference in terms of student learning (conceptual and affective) using the social justice based approach? Do students incorporate more social justice related issues in their explanations? I’m also curious how teachers and students respond to the NLP supported teaching? Do teachers feel more confident in responsive teaching? 

Hi Helen!

Thanks so much for all these wonderful questions! The teachers I have worked with expressed that the report helped them understand a) the ideas students have or struggle with and b) the ways in which students think about taught concepts. One teacher I worked with taught the Musical Instruments unit. From the report generated from responses in their classroom, the teacher learned that students confused pitch and volume. The teacher expressed that they wouldn't have known if it is amplitude, pitch, etc. that trips kids up and because they saw the report, they designed a hands-on activity (this was right before shelter-in-place) supporting students to connect different properties of a sound wave and the wave's visual representation. In sum, this teacher felt the report helped to respond more to students' ideas. 

With respect to students, in an ongoing study I used the NLP scoring to automatically branch students to targeted guidance. To explain to students why they might see different activities on their screen (this may not matter during online instruction but some teachers facilitate WISE during synchronous Zoom sessions), we added an explanation about the computer scoring their work and detecting ideas they might have missed. We are curious to learn what they think! A previous study from the group found that students trust computer feedback more when we transparently communicate what the computer does (and tell them that the computer tries to respond to them individually).

I'd love to hear your thoughts on what kinds of questions we could ask students to find out how they perceive NLP supported learning and what they would need to interact with automated guidance!

Best, 

Sarah

Inspiring work building on what I know to be decades of effort in science learning. So appreciate the attention to natural language processing as (I think?) a way to help teachers assess students' learning.

I'd love to know more about how you created the backend of the parts that allowed students to look for connections between climate change issues and systemic racism. 

Really timely and necessary work! I am curious about possible student outcomes from this unit, especially in terms of action and engagement. One part that was very clear in the video is related to "being part" of the context of the unit and understanding how students are personally affected by climate change and systemic racism. What about in terms of being part of a community? or a global citizen?

Hi Marcia and team - it's so fascinating to see how WISE continues to support teachers and students in important ways!  Kelly, I can see why you asked the questions you did of our project about caring for students' and teachers' social and emotional wellbeing as they are invited to grapple with racism.  I see the connections between our two projects, and I'm eager to hear more about how you and your team prepare and support teachers and students for this challenging work that might feel, to some at first peripheral to science learning and doing.  Thanks for thinking with me!

Hi Marcia and team!  What an interesting approach!  In our video, "Learning with Purpose as a Cultural Strength,"  we make the case that having a purpose that benefits a larger group is a powerful motivator for learning, especially for children from underserved backgrounds.  It seems that your project is harnessing that in a really productive way!

Best regards,  Barbara

Hi Marcia and team!

This research is fascinating and imperative. I often conceptualize white supremacy as a virus, education being one of its victors. It's inspiring to see real solutions to real problems. How would you hope to measure students' improved practice of anti-racism?

Best,

Roberta (she/her) 

This is fascinating material and so central to helping children learn. Cross-connecting deep issues in STEM with the climate change, anti-racism, and other urgent contemporary issues is such a wise approach for engaging students. Congratulations on developing such a thought provoking and helpful program! Clever, practical, and uplifting!