It is encouraging to see that WISE has a goal to so quickly adapt to the COVID-19 circumstances. Do you have ways to tracking any increases in uses of WISE in the last months? Plans for the future? How do you measure quality of use for teachers and/or students? 

Thanks for providing this video and best wishes with this project as it goes forward.

WISE looks like a great resource for teachers who are trying to engage students in online science instruction during COVID-19. I'm curious about how WISE supports students in developing explanations. How do you support students in becoming ever more adept at this critical science and engineering practice?

This is a very insightful video, and of course WISE continues to evolve, improve and impress.

The partnership with ETC for natural language processing is impressive when considering automation in online learning going to scale, extending the reach of the "instructor". Kudos!

I'm curious, how if at all (or if even appropriate) might it leverage the prior NSF work of Jim Minstrell (who looked at student responses, and facets or "snippets" in his online system based for particular understanding of science concepts) to help diagnose student  understanding and preconceptions, providing targeted feedback?

Easily said, I'm curious what the algorithm is to garner automated feedback (word count, key phrases, etc.). Love to hear more there if not too complex!

Dear Albert, 

thanks for the kind words and the inquiry into the details of our automated scoring. We are coding students' explanations with respect to the three-dimensional understanding called for by NGSS, in addition to Knowledge Integration. Depending on what the specific milestone item in a particular unit lends itself to, we have automated scoring of DCI, CCC, or SEP and an overall Knowledge Integration score. In Genetics of Extinctions for example, we score understanding of natural selection (DCI), using evidence from a graph (SEP), and Knowledge Integration. The teacher report gives an overview of how students are doing with respect to DCI and SEP, as well as how well they are integrating their ideas. With this detailed report, teachers are supported in diagnosing student understanding and noticing how students express certain ideas, and ultimately responding to these ideas in a targeted fashion. 

Particularly during distance learning, we designed the teacher report to encourage investigation of student ideas and building on them by adding "Support Activities" in WISE units that target specific ideas (for example, using evidence from a graph). Teachers can use the analytics as well as student responses to determine which idea a student might need help with and then provide targeted support by suggesting the respective activity.

Another example of targeted feedback from earlier this year (during regular classroom instruction) that was facilitated by the teacher report: The overview of the aggregated student scores and investigation of individual students' responses in the "Musical Instruments and the Physics of Sound Waves" unit revealed that students confuse ideas about the pitch and the amplitude of a sound wave, especially how these two concepts are depicted in a drawing. The teacher then added classroom activities that helped kids distinguish between pitch and amplitude.

I hope this addressed your questions and others will add to address the parts I did not!

Sarah

Very timely as teachers are trying to find effective resources for students in remote learning environments..  Do you have any student engagement data? and data to show student growth in understanding these topics. Also interested in knowing how much teacher time is needed to give feedback.  Thanks!  Great work!

Hi Mary, 

thanks for asking these questions! We're so curious to find out the answers and are currently working on answering these questions :).

This does not stem from a systematic analysis yet, but a first impression is a) there are kids who cannot log in to online curricula because they don't have access (no internet, no computers), b) not all students who log in for a teacher's run actually do work, c) some kids actually provide very detailed answers to open response items :)! A more systematic analysis is on its way.

Also with respect to student understanding, we will look at their learning gains but have not yet done so. From interviews with two teachers I know that some students do really great work and others do the bare minimum, it will be interesting to see to what extent the engagement predicts how much kids learn. 

Thanks especially for your last question! I am adding it to the interview questions, I haven't asked how long it took teachers to send feedback comments. The report provided definitely saves teachers time as they get an aggregated overview and they have expressed that it takes load off of them (finding an appropriate item, looking at all the responses, identifying themes, coming up with ways to respond). The report for distance learning includes pre-made comments that teachers can send to students via copy and paste. There are multiple comments suggested and teachers can choose which one is best suited for individual students. Using the pre-made comments is probably quicker than writing your own. Teachers also opted to combine their own comments with the pre-made ones, and some teachers use this function more than others.

I hope to have answered your questions!

Take care,

Sarah

Hi Margo

Thank you for your interest in our work. I add to the comments from Sarah, one interesting finding so far is that one teacher reported that some students who normally are distracted in class are actually more engaged when working remotely. We wonder if these students are better able to concentrate at home. Another possibility is that they are discussing their work with a sibling or friend. The greatest challenge teachers report is that some students have not been able to show up at all. Although districts are providing computers and internet providers are offering free accounts, some students are not able to access these services or do not have stable places to work.

Cordially,

Marcia

This looks like a very well-developed and much needed program, so congratulations on your work. It must be very satisfying to see it meeting the needs of teachers and students right now! Can you explain how teachers receive training or orientation in order to effectively use these tools?

Hi Lynda,

Thank you for your interest and question. Teachers using STRIDES for remote instruction include spontaneous users who found our open source materials on the web (www.wise.berkeley.edu), teachers who have previously participated in face-to-face workshops, and teachers who are at schools where a teacher participated in a workshop. Spread across school sites has been rapid: when one teacher uses WISE they often introduce their peers. New teachers are mentored by veteran users. In addition, research staff often coach new users, especially in schools that are participating in a research investigation.

Take care,

Marcia

This is such a timely initiative to learn about given our new normal. I love the natural language processing feature in the instructor dashboard!

Thank you for sharing Marcia and team,

The outbreak of COVID-19 certainly brings resources such as WISE into greater focus.  I love your use of office hours for instructors to have their questions addressed.

The discussion tool layout looks compelling. Something I find challenging with using something like Google Documents is supporting interaction across students.  What type of strategies do you use to guide student discussions in WISE units?

Hi all,

Thank you for sharing such interesting work. I really appreciated discovering the tool - much needed in the current circumstances - and reading this discussion thread.

I had a couple of questions:

- This looks like an interesting resource to share with our partner elementary science TOSA. What are the grade levels of students who engage with WISE and do you observe that some age groups benefit more than others from it?

- Could you please say more about how you communicate or collaborate with partners, and overall how you make the partnership successful?

I look forward to reading more about WISE.

Best,

Coralie

I really appreciate this tool especially during these uncertain times. I was wondering how you suggest teachers could use this tool with the current curricula they may be using.

Thank you for asking. At wise.berkeley.edu teachers can locate units for major middle school science topics. Each unit features an interactive model, guidance for students to learn the PEs associated with the topic, and activities that engage students in integrating their ideas. In addition, there are tools for teachers to monitor progress of each student, comment on student work, and assign scores. It is possible to customize the units to specific curricular needs and to interleave the activities with elements of the curriculum.

Try one out and let me know what you think

Marcia

This looks like a very great program! I love the variety of visualizations and models that students can explore. 

Is the program open access? How can schools request access to the program?

Thank you for your interest. WISE is free and open source. Check it out at WISE.Berkeley.edu

You can preview projects and explore how they work. Or register for free and set up a trial with students.

Let us know how you would like to collaborate.

Marcia

Great. We also have powerful authoring tools that can be used to create new units for any age group. Your materials on aquatic life forms would be a great focus for a WISE project. I love the images!!

Marcia

Sounds great. Students in my design class often use WISE authoring.

Marcia

STRIDES looks great and it seems like it could be a very timely tool in the sense that it can help us during COVID-19. Congrats on a constructing something that works when the world needs it most. 

Hi Andy,

Thank you for your interest in our work. Apropos to your work, students do seek out peers to collaborate on WISE units, definitely increasing the time they spend on science.

Take care,

Marcia

What a great tool and so apropos to the surge in distance learning taking place!  Always been a big fan of Dr. Linn's lab since grad school with Dr. Jon Vitale and enjoyed discussing the evolution of their work with Dr. Jim Slotta last summer at our EMIC conference here at UW-Madison.  As a learning scientist, its great to see such a comprehensive customizable system that provides curricula accompanied by timely feedback, instructions, and scaffolds.  Moreover, using NLP to score students explanations provides good data for generating the Teacher Action Reports, and their recommendations and comments to students.  And is it that current investigations with WISE are focused on teacher integration and learning and curricular delivery?  

Sarah mentioned earlier in the thread some automated scoring of students outputs; Can can y'all expound on how various metrics coordinate over time to assess learning gains? 

Hi Michael, 

thanks for watching our video! let me try to answer your question - and feel free to follow up, if I am not addressing it!

The automated scoring of student explanations results in histograms that give teachers an overview with respect to their students' scores on DCI, CCC, SEP, and KI. For GenEx for example, we scored DCI, SEP, and KI. To generate the report, we define "combinations" of these scores to come up with responsive instructional interventions. Your class might get a low score for SEP, but a high score for DCI and and a low score for KI (on average), which results in a recommended action that helps kids focus on using evidence from the graph (to up their SEP). When kids revise their answers, the score overviews and report are updated. Ideally, the intervention supported kids to include evidence in their explanations. Kids now have high scores on DCI and SEP, maybe KI is still low. This means, they have the ideas and they use evidence, but they don't yet link their ideas. Now, this combination of average scores triggers a new and different recommended action. 

I hope this example makes it more clear how the various metrics coordinate over time to give insight into students' learning progression. 

We showed the TAP for the original submission and then again for the revision so teachers could compare. For distance learning, we have one report that gets updated with every new answer (or revision) that is submitted (the logistics of having 2 reports and for kids having to submit answers on different steps for the same thing works during regular classroom instruction but not for the distance situation).

Best,

Sarah

Being an online instructor this project caught my interest.  I am wondering if anyone is working on creating a library of science/engineering simulations that can be used for designing  online and blended learning environments?     

Thank you for your interest in our project. We are working on adding a repository of open source models and simulations to our authoring tool. This is currently in preliminary testing.

Let us know if you would like to collaborate on it.

Take Care,

Marcia

What an impressive project! Science educators of all levels can really benefit from considering this model. The self-grading tool must be such a lifeline for teachers; both in assisting with guidance on the feedback provided, as well as continuing to reinforce the educator’s own understanding of the material. Adding a repository and continuing to support the development of additional materials would be excellent for maintaining and growing a supportive community. What a great resource for our current education challenges. Thank you for supporting teachers so meaningfully. 

This is a really neat project! It has some similarities with ours, with respect to our Teacher Dashboard, where teachers can see students respond in real time and provide individual feedback through reports.

I am impressed with your automatic scoring program. While I am not working on assessment for ConnectedBio, improving assessment and feedback is one of my personal research interests. Perhaps we should have further connections offline about that -- and maybe a joint collaboration in the future would bridge or embed our ConnectedBio simulations into your scoring system and larger database!

Hi Rebecca, 

I'm excited to learn more about your project and the ConnectedBio simulations! Feel free to reach out to any of us to continue the conversation!

Best,

Sarah