Thank you for your comment! We have immediate plans to implement the interventions in all sections of the Introduction to Environmental Science course and have been assisting faculty in the School of Religion and others to develop argumentation lessons for their courses using argument driven inquiry with scaffolding. We will be working this summer to develop an online friendly version of all three phases since covid has us all planning for the online learning space!

In terms of the dramatic changes, I think this intervention with scaffolding reinforces best teaching practices and moving away from the monotony of ppt lectures.

Students need time to develop critical thinking skills and the three phase approach provides that time under a common topic. The engaging lessons, especially the taste test, kept their attention. Students really felt supported and gained confidence in their ability to complete the argumentation tasks. They recognized this was a different learning environment, liked it, committed to it and thrived.

Sarah, thank you for your comments and questions.

The OLI argumentation course does come with a module about weighing evidence. Due to time constraints right up against our Thanksgiving break I decided to skip this module. I provided the students with the articles and videos that they used for their sources of evidence in phase two, which were of high quality. If I were to send them off to find their own evidence, I think that keeping this module would be useful. We are also working on an intervention for this course on identifying pseudoscience, anecdotal evidence and misleading visualizations. I think the new intervention will enhance this one as well and lead to adding the weighing evidence component back in.

Phase 1, OLI modules, has built in scaffolding in the design of the lessons. They start with vocabulary and then work up to interactive components where they identify and build their argument diagrams. In Phase 2, articles and videos, they were given one week to complete the readings and videos as homework with embedded questions to help guide them in on what information was key to the upcoming in class discussions. In class, I divided the students into small groups and provided each with a handout guiding them through the argument diagramming as well as printout of the articles and transcripts of the videos for reference. Each group was assigned one of the three topics (climate change, social issues, or human health) to explore and gather evidence that supported and contradicted their argument. We then came together as a class to diagram all the components and determine if the evidence overall supported or rejected our argument about the benefits of GMO vs organic foods. For Phase 3, as we completed the taste test they worked on a similar handout guiding them through the argument diagramming comparing cost and preference. 

Thanks for the detailed answer!

You're welcome! 

Thank you!

Yes! We love Ms. McCall and work with her often!! 

There definitely are standards at the middle school and high school level that should be supporting argumentation skills. I did see some publications of similar frameworks for middle and high school.

Going into this intervention, I wasn't sure what to expect in students performance on the pretest. I knew that in K-12 they should have been introduced to these topics, but anecdotally myself and others weren't seeing evidence on assignments that they were using justification. I designed the pre/post test to assess abilities from beginner to advanced. Happily, those pretests showed that around 65-70% of the students could identify basic claims and evidence in a multiple choice setting, but struggled more in the free response. On the pretest we saw the students default to prior knowledge and personal experiences when developing their written justifications and not using the evidence in front of them. There was definitely a shift on the post test where they moved away from their feelings, to weighing the evidence they were presented with to make their decisions.

We also used the pretest in other non intervention sections to get a larger baseline, and the results were similar in all sections. That shows us argumentation lessons from k-12 are carrying over into higher ed. I do plan on developing a more robust "persuasion" component where students can engage in debate on the topic. I think the assessment scores show we can challenge them more on argumentation in this course!

Sarah and team: What a great success story!  Thanks to Raphael for emailing me about it.

Leigh: One recent effort here at CMU toward helping younger students (high school) with persuasive communication is here this educational game,  Persuasion Invasion (http://nickdiana.com/persuasioninvasion/), by a PhD student of ours, Nick Diana.  I'm sure Nick would be happy to help you get into use if you have a relevant student group you want to try it with!

Thank you for coming to check out the video and for sharing your resources!

Thank you for your comments and questions. 

Yes we did discuss which live of evidence do you prioritize: preference and costs. There were obviously differing opinions among the students! Generally costs were a huge driver to the students, which makes sense for college students with limited funds.

We worked on a module initially with native and nonnative plants and are trying to work on another with our general education religion course. We are hoping to expand to other topics too! It's a very adaptable intervention.

Thank you! Absolutely, the intervention strategy is highly adaptable. The OLI course content is not science based content so it is a great place to start. The lessons in phase two and three can be adapted to which ever topic you want! Vaccines would be a great topic to cover.

Thank you for sharing your work as well! We also saw a very interesting transition in thinking with the students. On the pretest they were asked to read two short paragraphs (one pro GMO and the other anti GMO). They were asked to make an argument for or against using the evidence provided. On the pretest, despite the evidence there many of them talked about their feelings and showed strong preconceived notions on the topic and used anecdotal evidence in their claims. On the pretest, that changed completely to them using stronger scientific evidence for their justifications. This transition is something we are hoping to quantify in future iterations of the intervention   

Thank you! We are working on our publications surrounding this intervention right now. Dr. Torres has publications in this area: https://www.researchgate.net/profile/Hector_Torres11

The students as part of phase two were given readings about what GMOs and organic foods with some vocabulary based homework questions to help prepare them for the readings and videos they would be using as evidence for their justifications in class. There is a module in OLI that provides a framework for evaluating evidence which can be assigned and utilized in phase 2 and 3 of the intervention. I opted to skip that module mainly for time issues. We were right up against Thanksgiving break, and the module added another layer of instruction we didn't have time for. We also did not have a baseline for how well students understood argumentation either, which we now do. 

On the pretest for this section and the other sections of Environmental Science that did not participate in the intervention, their scores showed 68-71% of the students were able to recognize the premise, evidence and conclusions of arguments already. This is encouraging us to include the module on weighing evidence, and also further developing the course discussions into a debate in order to strengthen their persuasion skills.

The OLI module was already developed by Carnegie Mellon, but you do have the option to customize the course to fit your needs. The course modules take the students through argument and argument diagramming vocabulary (such as the different types of arguments), then transition to identifying argument structures. 

Phase two consisted of some online reading and questions about GMOs and non GMO foods. Then students were assigned readings and videos with additional questions which they used as their evidence for the in class diagramming activity. The students were broken up into small groups and given printouts of the articles and video transcripts to discuss. They had a handout similar to the OLI course which had them looking for evidence and creating a diagram for their group's topic. 

In Phase 3 they had the same handout to help them develop the argument for the taste test component.

We do have comparison data available that we will be processing this summer. There are around 10 sections of Environmental Science taught each semester by five faculty. Our colleagues have been very supportive of trying out new interventions in this course. We have developed, tested, and have had two course adaptations adopted by the other faculty after we have tested and presented the improvements in academic success (open educational resources and visual literacy). They were eager to test out this intervention in Spring 2020, but unfortunately covid-19 has delayed that. We're hoping next year to get data on how well the intervention transfers.

Thanks for the collaboration. 

Thank you very much! That is a very kind comment and appreciated :)

We have not adapted the intervention to our majors courses, but that it certainly possible and something we plan on doing in the future. I think you can agree that our STEM majors need to possess argumentation skills to combat misinformation out there! 

I think if you tie in the hard science of radiation and radioactivity, with the socioscientific issue of green energy and the use of nuclear power plants you would have transformative learning experience. Through the exploration of Fukishima and Cheronybl disasters in juxtaposition with the need for sustainable energies to combat climate change, they would learn about the principles of radiation and radioactivity in powerful way.

Thank you for your comments and for watching!