5236 Views (as of 05/2023)
  1. Sarah Krejci
  2. https://aquaticresearchlab.org/
  3. Assistant Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. Bethune-Cookman University
  1. Raphael Isokpehi
  2. https://scholar.google.com/citations?user=kVCRbxIAAAAJ&hl
  3. Professor of Biology
  4. Presenter’s NSFRESOURCECENTERS
  5. Bethune-Cookman University
  1. Hector Torres
  2. Associate Professor
  3. Presenter’s NSFRESOURCECENTERS
  4. Bethune-Cookman University

Investigating the Effects of Socioscientific Argumentation Development on Stu...

NSF Awards: 1623371

2020 (see original presentation & discussion)

Undergraduate

The challenges of modern society including climate change, pollution, and educational disparities require the collaboration of scientists, educators, politicians, economists, and the public. Argumentation forms the basis of collaborative discussion which two parties can utilize to resolve an issue. Argumentation can be used to promote critical thinking in students, improve the uptake of information and develop rational decision making used to make more informed decisions. An argumentation intervention with a three-stage Argument-Driven Inquiry (ADI) model with scaffolding was tested in a nonmajor Environmental Science course. The first stage of the ADI model initiates the lesson using an online cyber-learning tutoring system on Argument Diagraming, available through Carnegie Melon University’s Online Learning Initiative (OLI). Two in-class lessons were designed to develop and reinforce argumentation skills from OLI. In these lessons, students explored the impacts of Genetically-Modified and Organic foods on society, climate change, and human health. Student academic success was measured with a pre/post test of argumentation skills resulting in a significant increase of 7.5% overall and a 18.7% increase in written argumentation skills. Quantitative and qualitative student satisfaction surveys demonstrated the transition of difficulty through the three ADI lessons, with most students finding the lesson easy by the third stage. End of course scores increased compared to other semesters, from an average score of 68% to 81%, while the course withdraws decreased from 11 students to zero. This successful argumentation intervention is modular, allowing it to be applied to a variety of topics and courses beyond environmental science.

This video has had approximately 516 visits by 348 visitors from 178 unique locations. It has been played 277 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2020 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2020 STEM For All Video Showcase (36 posts)
  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 5, 2020 | 01:04 p.m.

    Thank you for viewing our research synopsis! We are very excited to share our intervention development and successes with everyone and encourage others to consider our socioscientific argumentation and argument driven inquiry with scaffolding can be utilized in their courses. Has the video given you any ideas of how to incorporate these concepts in your course?

  • Icon for: Overtoun Jenda

    Overtoun Jenda

    Facilitator
    Assistant Provost and Professor of Mathematics
    May 5, 2020 | 01:56 p.m.

    Thanks. Very interesting. What do you attribute the dramatic change in academic performance to? Can this be easily scaled up to other courses? Has it been tried on your campus yet?

     
    1
    Discussion is closed. Upvoting is no longer available

    Lorna Quandt
  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 5, 2020 | 02:06 p.m.

    Thank you for your comment! We have immediate plans to implement the interventions in all sections of the Introduction to Environmental Science course and have been assisting faculty in the School of Religion and others to develop argumentation lessons for their courses using argument driven inquiry with scaffolding. We will be working this summer to develop an online friendly version of all three phases since covid has us all planning for the online learning space!

    In terms of the dramatic changes, I think this intervention with scaffolding reinforces best teaching practices and moving away from the monotony of ppt lectures.

    Students need time to develop critical thinking skills and the three phase approach provides that time under a common topic. The engaging lessons, especially the taste test, kept their attention. Students really felt supported and gained confidence in their ability to complete the argumentation tasks. They recognized this was a different learning environment, liked it, committed to it and thrived.

     
    2
    Discussion is closed. Upvoting is no longer available

    Holly Morin
    Sasha Palmquist
  • Icon for: Sarah Bichler

    Sarah Bichler

    Researcher
    May 6, 2020 | 12:46 a.m.

    Thanks for sharing your video! Students' increased use of justifications is a wonderful outcome of your intervention!

    I'd be curious to learn more about the use of evidence when arguing about socio-scientific issues. Are your students guided in discussing "quality" of evidence? Is arguing about SSI different than arguing about scientific issues?

    What are examples of scaffolds you implemented in each of the three phases?

    Thanks for sharing more!

    Sarah

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 6, 2020 | 09:13 a.m.

    Sarah, thank you for your comments and questions.

    The OLI argumentation course does come with a module about weighing evidence. Due to time constraints right up against our Thanksgiving break I decided to skip this module. I provided the students with the articles and videos that they used for their sources of evidence in phase two, which were of high quality. If I were to send them off to find their own evidence, I think that keeping this module would be useful. We are also working on an intervention for this course on identifying pseudoscience, anecdotal evidence and misleading visualizations. I think the new intervention will enhance this one as well and lead to adding the weighing evidence component back in.

    Phase 1, OLI modules, has built in scaffolding in the design of the lessons. They start with vocabulary and then work up to interactive components where they identify and build their argument diagrams. In Phase 2, articles and videos, they were given one week to complete the readings and videos as homework with embedded questions to help guide them in on what information was key to the upcoming in class discussions. In class, I divided the students into small groups and provided each with a handout guiding them through the argument diagramming as well as printout of the articles and transcripts of the videos for reference. Each group was assigned one of the three topics (climate change, social issues, or human health) to explore and gather evidence that supported and contradicted their argument. We then came together as a class to diagram all the components and determine if the evidence overall supported or rejected our argument about the benefits of GMO vs organic foods. For Phase 3, as we completed the taste test they worked on a similar handout guiding them through the argument diagramming comparing cost and preference. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Sasha Palmquist
  • Icon for: Sarah Bichler

    Sarah Bichler

    Researcher
    May 6, 2020 | 12:24 p.m.

    Thanks for the detailed answer!

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 6, 2020 | 01:30 p.m.

    You're welcome! 

  • Icon for: Lorna Quandt

    Lorna Quandt

    Researcher
    May 6, 2020 | 08:18 a.m.

    Lovely work and really nice video! These kinds of scientific skills are so critical. Thank you for doing this work. 

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 6, 2020 | 09:14 a.m.

    Thank you for watching and your kind comments!

  • Icon for: Kate Meredith

    Kate Meredith

    President - GLAS Education
    May 6, 2020 | 01:03 p.m.

    Congratulations on a very clearly delivered video.  I was able to get a good overview of the project in 3 min!  

     
    1
    Discussion is closed. Upvoting is no longer available

    Holly Morin
  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 6, 2020 | 01:30 p.m.

    Thank you!

  • Icon for: Hector Torres

    Hector Torres

    Co-Presenter
    Associate Professor
    May 6, 2020 | 04:31 p.m.

    We at B-CU have created a coordinated effort to introduce the importance of developing the argumentation skills of our student through the use of Socioscientific issues. How is it that student graduating with a STEM degree can contribute to society in solving real and relevant socioscientific issues? How can we develop faculty members to look at teaching and learning from the societal and better yet from that which is relevant to our students? These questions and others can be explore through the lenses of our work at B-CU on Socioscientific Argumentation...enjoy the ride and let us know your thoughts..

  • Icon for: Sasha Palmquist

    Sasha Palmquist

    Facilitator
    Senior Manager of Community
    May 6, 2020 | 10:20 p.m.

    Congrats on producing a compelling video and on your significant findings! It seems to me that this intervention has great potential to increase the science literacy of students. I would like to hear more about how you measured student learning in this course and whether you have any plans to administer a delayed assessment with this sample to see whether the gains that you detected persist over time. Thanks!

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 7, 2020 | 10:34 a.m.

    Thank you for your questions and for watching!

    Our assessment for the intervention was a pre and post test followed by a survey. I'll explain the general course design below.

    The pre/post test provided the students with two short passages that had contradictory arguments for GMOs and Organic foods. For the multiple choice components the students had to identify the premise, evidence, and appropriate argument diagram for each passage. For the written component they had to develop their own argument using the information in the two passages. Those were scored from 0-5 using Sadler and Fowler's (2006) framework. I also included a similar written component on the exam for that module. The overall course scores at the end of the presentation were from the same professor (me) teaching the course. In 2018 we had switched to an open educational resources course design which I had been piloting for a few semesters online. In the next semester I made my first version of the SSI intervention which showed an improvement in academic success and then this more robust version which improved the course significantly. The three semesters presented had the same course structure. Student scores were a mixture of online assignments, in class activities, and exams. The only difference throughout the three semesters were the students in the course and the SSI intervention. 

    We have discussed tracking the students who had the intervention over time and comparing their academic success compared to their peers in other sections which didn't have the SSI. 

  • Icon for: Hector Torres

    Hector Torres

    Co-Presenter
    Associate Professor
    May 7, 2020 | 11:32 a.m.

    We are just at the beginning or as they say at the "tip of the iceberg" in understanding the level and connection of argument-driven inquiry, socioscientific issue, and intervention strategies. We are currently making progress and believe that as this work continues to disseminate, we will have the appropriate set of intervention strategies to juxtaposition socioscientific issues and argumentation as a venue for increasing STEM education.

  • Icon for: Leigh Peake

    Leigh Peake

    Informal Educator
    May 7, 2020 | 03:31 p.m.

    I wasn't aware of the Carnegie-Mellon work in this area so I appreciate learning about it and about how you've implemented it at B-CU (where my friend Junell McCall calls home!). Have this framework ever been tried with younger students? Or, alternatively, what do you wish those working at younger grades would have done better to prepare your students in this kind of persuasive communication?

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 7, 2020 | 04:30 p.m.

    Yes! We love Ms. McCall and work with her often!! 

    There definitely are standards at the middle school and high school level that should be supporting argumentation skills. I did see some publications of similar frameworks for middle and high school.

    Going into this intervention, I wasn't sure what to expect in students performance on the pretest. I knew that in K-12 they should have been introduced to these topics, but anecdotally myself and others weren't seeing evidence on assignments that they were using justification. I designed the pre/post test to assess abilities from beginner to advanced. Happily, those pretests showed that around 65-70% of the students could identify basic claims and evidence in a multiple choice setting, but struggled more in the free response. On the pretest we saw the students default to prior knowledge and personal experiences when developing their written justifications and not using the evidence in front of them. There was definitely a shift on the post test where they moved away from their feelings, to weighing the evidence they were presented with to make their decisions.

    We also used the pretest in other non intervention sections to get a larger baseline, and the results were similar in all sections. That shows us argumentation lessons from k-12 are carrying over into higher ed. I do plan on developing a more robust "persuasion" component where students can engage in debate on the topic. I think the assessment scores show we can challenge them more on argumentation in this course!

     

  • Ken Koedinger

    Researcher
    May 12, 2020 | 04:02 p.m.

    Sarah and team: What a great success story!  Thanks to Raphael for emailing me about it.

    Leigh: One recent effort here at CMU toward helping younger students (high school) with persuasive communication is here this educational game,  Persuasion Invasion (http://nickdiana.com/persuasioninvasion/), by a PhD student of ours, Nick Diana.  I'm sure Nick would be happy to help you get into use if you have a relevant student group you want to try it with!

     
    1
    Discussion is closed. Upvoting is no longer available

    Raphael Isokpehi
  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 06:50 p.m.

    Thank you for coming to check out the video and for sharing your resources!

  • Icon for: Jeanne Reis

    Jeanne Reis

    Facilitator
    Director
    May 8, 2020 | 01:09 p.m.

    Great work, and fabulous results related to student success, decreased level of perceived difficulty, and reduction in attrition!

    Expanding student thinking and argumentation skills is so important, especially in a world where we have more information at our fingertips than ever. I could see student engagement increasing as they deepened and broadened their exploration of the topic.

    It's notable that two of the three GMO products won out in taste testing and cost. Did you discuss that result with the students? How did those findings impact their argumentation for or against each category, and how were individual preferences and fiscal realities weighed alongside climactic and societal benefits? 

    Have you developed units on topics other than GMO / organic foods? 

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 8, 2020 | 01:15 p.m.

    Thank you for your comments and questions. 

    Yes we did discuss which live of evidence do you prioritize: preference and costs. There were obviously differing opinions among the students! Generally costs were a huge driver to the students, which makes sense for college students with limited funds.

    We worked on a module initially with native and nonnative plants and are trying to work on another with our general education religion course. We are hoping to expand to other topics too! It's a very adaptable intervention.

     
    1
    Discussion is closed. Upvoting is no longer available

    Jeanne Reis
  • Icon for: Lily Raines

    Lily Raines

    Manager of Science Outreach
    May 9, 2020 | 12:15 p.m.

    What an interesting and clearly successful approach! Do you believe other courses could benefit from this? I'm thinking a discussion on vaccines (dosing, fears, availability around the world, etc.) would be particularly interesting for students in health courses.

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 10:31 a.m.

    Thank you! Absolutely, the intervention strategy is highly adaptable. The OLI course content is not science based content so it is a great place to start. The lessons in phase two and three can be adapted to which ever topic you want! Vaccines would be a great topic to cover.

  • Icon for: David Sittenfeld

    David Sittenfeld

    Informal Educator
    May 9, 2020 | 05:56 p.m.

    This is fascinating! I love the way you have the students diagram and identify the argument components.  

    We have an exhibit at our Museum that aims to get people to consider and explore their socio-scientific arguments around a number of socio-scientific questions: http://exhibits.mos.org/explore-the-exhibit/provocative-questions.  We often do facilitated forum programs to help inform these unfacilitated exhibit components.  We actually have found that visitors are less sure of their own opinions about the questions after going through this process, which we think may mean that they are thinking more carefully about their own evidence and values as they form opinions. But it's exciting and affirming to see that your student participants are getting so much self-efficacy and co-benefits with respect to their STEM academic outcomes more broadly.  Congrats!  I'm definitely going to dig more into your work.

     
    1
    Discussion is closed. Upvoting is no longer available

    Raphael Isokpehi
  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 10:36 a.m.

    Thank you for sharing your work as well! We also saw a very interesting transition in thinking with the students. On the pretest they were asked to read two short paragraphs (one pro GMO and the other anti GMO). They were asked to make an argument for or against using the evidence provided. On the pretest, despite the evidence there many of them talked about their feelings and showed strong preconceived notions on the topic and used anecdotal evidence in their claims. On the pretest, that changed completely to them using stronger scientific evidence for their justifications. This transition is something we are hoping to quantify in future iterations of the intervention   

  • Icon for: Brian Drayton

    Brian Drayton

    Co-Director
    May 11, 2020 | 10:17 a.m.

    This sounds really promising (and can you recommend anything you've written? Sorry if I missed a reference in the rich discussion above).

        One thing I've missed  in a lot of argumentation work is the role of helping the "arguers" clarify their explanatory/theoretical understanding of the actual knowledge that might be relevent - both as a source of evidence (this is often included) and as a framework for evaluating evidence and building justifications or deriving predictions.  How does theory fit in to the argumentation that your students are learning?

       For example, what models of organic vs conventional agriculture are used or developed as the basis for the students' work?  How much of this is left to them to construct (a laborious task!)?  Or are they presented with some definitions, as in case-based learning?

     

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 10:45 a.m.

    Thank you! We are working on our publications surrounding this intervention right now. Dr. Torres has publications in this area: https://www.researchgate.net/profile/Hector_Torres11

    The students as part of phase two were given readings about what GMOs and organic foods with some vocabulary based homework questions to help prepare them for the readings and videos they would be using as evidence for their justifications in class. There is a module in OLI that provides a framework for evaluating evidence which can be assigned and utilized in phase 2 and 3 of the intervention. I opted to skip that module mainly for time issues. We were right up against Thanksgiving break, and the module added another layer of instruction we didn't have time for. We also did not have a baseline for how well students understood argumentation either, which we now do. 

    On the pretest for this section and the other sections of Environmental Science that did not participate in the intervention, their scores showed 68-71% of the students were able to recognize the premise, evidence and conclusions of arguments already. This is encouraging us to include the module on weighing evidence, and also further developing the course discussions into a debate in order to strengthen their persuasion skills.

     
    1
    Discussion is closed. Upvoting is no longer available

    Raphael Isokpehi
  • Icon for: Michael I. Swart

    Michael I. Swart

    Researcher
    May 11, 2020 | 12:04 p.m.

    Great contextualized work on argumentation, and a very clear conceptual framework operationalizing sense-making, persuasion, articulation.  Can you expound on the scaffolds?  How are they designed?  Are they operationalizing any constructs that you are able to attribute better learning to other than generalized participation?  And great data as a within-groups control?  Are you planning any control comparisons?  Thanks for sharing!

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 11:16 a.m.

    The OLI module was already developed by Carnegie Mellon, but you do have the option to customize the course to fit your needs. The course modules take the students through argument and argument diagramming vocabulary (such as the different types of arguments), then transition to identifying argument structures. 

    Phase two consisted of some online reading and questions about GMOs and non GMO foods. Then students were assigned readings and videos with additional questions which they used as their evidence for the in class diagramming activity. The students were broken up into small groups and given printouts of the articles and video transcripts to discuss. They had a handout similar to the OLI course which had them looking for evidence and creating a diagram for their group's topic. 

    In Phase 3 they had the same handout to help them develop the argument for the taste test component.

    We do have comparison data available that we will be processing this summer. There are around 10 sections of Environmental Science taught each semester by five faculty. Our colleagues have been very supportive of trying out new interventions in this course. We have developed, tested, and have had two course adaptations adopted by the other faculty after we have tested and presented the improvements in academic success (open educational resources and visual literacy). They were eager to test out this intervention in Spring 2020, but unfortunately covid-19 has delayed that. We're hoping next year to get data on how well the intervention transfers.

  • Icon for: Raphael Isokpehi

    Raphael Isokpehi

    Co-Presenter
    Professor of Biology
    May 12, 2020 | 07:58 p.m.

    Thanks for the collaboration. 

  • Jorge Martinez-Soto

    Informal Educator
    May 11, 2020 | 01:59 p.m.

    Just an amazing and interesting video to watch!!! 

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 11:17 a.m.

    Thank you very much! That is a very kind comment and appreciated :)

  • Icon for: Paul Bergeron

    Paul Bergeron

    Researcher
    May 11, 2020 | 05:41 p.m.

    Thanks for sharing this work! It's been great seeing the different projects focusing on specific scientific practices, as our group has been working with instructors on transformations involving all three dimensions of scientific learning. I especially like the focus here as, I think, there's a tendency to focus on constructing explanations instead of generating arguments. Do you have any tie ins with having students engage in the practice of generating hypothesis and asking questions? We've not seen that happen in the data from our PD project.

    This has given me the idea of how to go about teaching a unit on radiation and radioactivity. I've mostly been thinking about the Under Rep curriculum for physics, which is great but treats things (from my limited knowledge) as separate from the instruction of physics content. Focusing on instruction through the practice of argumentation from evidence, I think, would be one way to meld the Under Rep curriculum with the traditional instruction.

     
    1
    Discussion is closed. Upvoting is no longer available

    Raphael Isokpehi
  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 11:28 a.m.

    We have not adapted the intervention to our majors courses, but that it certainly possible and something we plan on doing in the future. I think you can agree that our STEM majors need to possess argumentation skills to combat misinformation out there! 

    I think if you tie in the hard science of radiation and radioactivity, with the socioscientific issue of green energy and the use of nuclear power plants you would have transformative learning experience. Through the exploration of Fukishima and Cheronybl disasters in juxtaposition with the need for sustainable energies to combat climate change, they would learn about the principles of radiation and radioactivity in powerful way.

  • VISHWA TRIVEDI

    Higher Ed Faculty
    May 12, 2020 | 01:58 p.m.

    Fascinating! Cool stimulating argument components

  • Icon for: Sarah Krejci

    Sarah Krejci

    Lead Presenter
    Assistant Professor
    May 12, 2020 | 06:50 p.m.

    Thank you for your comments and for watching!

  • Further posting is closed as the event has ended.