4218 Views (as of 05/2023)
  1. Brian Gane
  2. Research Assistant Professor
  3. Presenter’s NSFRESOURCECENTERS
  4. University of Illinois at Chicago
  1. Samuel Arnold
  2. Graduate Research Assistant
  3. Presenter’s NSFRESOURCECENTERS
  4. University of Illinois at Chicago
  1. Liz Lehman
  2. https://stemed.uchicago.edu/staff?data-target-rollout-thumb-id=liz-lehman
  3. School Development Manager
  4. Presenter’s NSFRESOURCECENTERS
  5. University of Chicago
  1. JIm Pellegrino
  2. Professor
  3. Presenter’s NSFRESOURCECENTERS
  4. University of Illinois at Chicago

Collaborative research: Improving multi-dimensional assessment and instructio...

NSF Awards: 1813737, 1813938

2021 (see original presentation & discussion)

Grades K-6

The main goal of this project is to better understand how to build and sustain the capacity of science teachers in Grades 3-5 to instruct and formatively assess students in ways that are aligned with contemporary science education frameworks and the Next Generation Science Standards. We are using a professional learning model that places instructionally-supportive assessments at the forefront, and centers discourse around students using their disciplinary knowledge and practices to engage in sensemaking and reasoning. In order to foster such discourse, and for teachers to then put those ideas and strategies into practice, teachers need good multi-dimensional assessments that are aligned to NGSS performance expectations and afford students the opportunities to use all three dimensions of the NGSS in an integrated manner. Unfortunately, few assessments like this exist, especially ones that are open and free for all teachers and students to use. Therefore, in the first stage of the project, we have been collaborating with a cohort of elementary teachers from two Illinois school districts to co-develop instructionally-supportive assessment tasks and to use those tasks formatively with their students. We are implementing and disseminating these technology-enhanced assessment tasks through an online portal that teachers and students can use to access the assessments and build them into classroom instruction and student learning.

This video has had approximately 477 visits by 347 visitors from 162 unique locations. It has been played 170 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2021 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2021 STEM For All Video Showcase (18 posts)
  • Icon for: Brian Gane

    Brian Gane

    Lead Presenter
    Research Assistant Professor
    May 10, 2021 | 06:11 p.m.

    Welcome and thank you for joining in discussion around our elementary science instruction and assessment project! To date our project has made substantial progress co-developing quality assessments to support elementary science teachers’ instruction, but we have much more to do over the next two years to build and grow our community. 

    We welcome discussion on all aspects of our project. We are especially interested in hearing from other teachers and educators that have also been developing science assessments for use in elementary classrooms. And for those educators who are already using multi-dimensional assessments with your students, what challenges have you encountered?

    Likewise, we are interested in hearing from educators and researchers about how they have continued to support communities of teacher learners over time as they engage with and teach to the NGSS. What has worked well and what challenges are you still encountering?

     
    1
    Discussion is closed. Upvoting is no longer available

    Remy Dou
  • Icon for: Daniel Damelin

    Daniel Damelin

    Researcher
    May 11, 2021 | 11:53 a.m.

    You can also check out a recent article in The Concord Consortium's @Concord newsletter, which describes some of the work we have collaborated on with this excellent team who produced the video above. 

     
    2
    Discussion is closed. Upvoting is no longer available

    Mary Nyaema
    Remy Dou
  • Icon for: Brian Gane

    Brian Gane

    Lead Presenter
    Research Assistant Professor
    May 11, 2021 | 10:56 p.m.

    Nice article that covers both the elementary and middle school tasks, thanks for sharing Dan! The middle school physical science task in Figure 1 really showcases the opportunities of technology-enhanced assessments, integrating a video of the experimental materials and procedure with an interactive graphing tool for students to visualize the provided data.

    Why did you choose to include one trial with unknown mass and how does the graphing tool handle that trial?

  • Icon for: Daniel Damelin

    Daniel Damelin

    Researcher
    May 11, 2021 | 11:23 p.m.

    The goal is to provide an opportunity for students to generate a data visualization that would help them answer a question. Where there is an unknown mass, CODAP (the graphing tool) will just not plot a point (as it is not possible without both coordinate values), so students see the rest of the data on a scatterplot and must interpolate to provide evidentiary support for their understanding of the relationship between mass, velocity, and kinetic energy.

     
    1
    Discussion is closed. Upvoting is no longer available

    Brian Gane
  • Icon for: Dionne Champion

    Dionne Champion

    Facilitator
    Research Assistant Professor
    May 11, 2021 | 01:38 p.m.

    Thanks for sharing your work. I have been working on a project that is trying to develop culturally sustaining multimodal assessments of computational thinking.  What are the biggest lessons you have learned in working to develop these assessments? What are some of the challenges?

     
    3
    Discussion is closed. Upvoting is no longer available

    Carla Strickland
    Sabrina De Los Santos
    Remy Dou
  • Icon for: Brian Gane

    Brian Gane

    Lead Presenter
    Research Assistant Professor
    May 11, 2021 | 11:22 p.m.

    Wow, sounds interesting! What approaches have you taken to weave in multiple modalities? 

    One of the biggest challenges we face over and over again is time! Developing quality assessments requires a principled approach with consistent documentation and methods for evaluating their validity and utility. We use a multi-step, iterative and collaborative process that involves individuals with diverse perspectives and experiences. We have a great team of collaborators that we have been building this community and developing assessments for two years. I think this model of collaborative and distributed development is one that has potential to be adapted for many communities of teachers and educators. 

    Another challenge, especially with NGSS, is developing multi-dimensional rubrics that can help teachers evaluate student responses and provide feedback to students along those multiple dimensions.

     
    1
    Discussion is closed. Upvoting is no longer available

    Sabrina De Los Santos
  • Icon for: Carla Strickland

    Carla Strickland

    Digital Development Manager
    May 18, 2021 | 02:20 p.m.

     

    I agree that your project sounds super interesting, Dionne. We'd love to hear more about it.


    I've been spending some time thinking about how turned off our Black and Brown students and teachers have been by traditional standardized assessments -- even as newer fields like CS/CT Ed seem to be hurtling towards that very thing.


    Maybe with the disruption of COVID-19, we can act to further alleviate the disproportionate harm cause by traditional assessments and develop scalable versions of assessments that actually help our teachers and students.


     

  • Icon for: Nonye Alozie

    Nonye Alozie

    Researcher
    May 11, 2021 | 04:31 p.m.

    Really nice video, Brian! Glad to see this work continue into new directions!

     
    2
    Discussion is closed. Upvoting is no longer available

    Carla Strickland
    Remy Dou
  • Icon for: Brian Gane

    Brian Gane

    Lead Presenter
    Research Assistant Professor
    May 11, 2021 | 11:24 p.m.

    Thanks Nonye, it has been nice to continue and build on the lessons learned from when we were working together to develop assessments for the middle school life science standards!

  • Icon for: Kevin Garner

    Kevin Garner

    K-12 Administrator
    May 11, 2021 | 10:14 p.m.

    Thank you for sharing your important work!  We having start working on building teacher capacity as well as multi-dimensional assessments.  What I saw looks amazing.  I would love to talk to you more about your project and see some of the products that you said were available for viewing.  Great work!

     
    1
    Discussion is closed. Upvoting is no longer available

    Remy Dou
  • Icon for: Liz Lehman

    Liz Lehman

    Co-Presenter
    School Development Manager
    May 12, 2021 | 06:24 p.m.

    Kevin, we'd love to chat with you more about building teacher capacity in this area! One aspect of the project that is still under development is creating a virtual learning community to connect teachers around this topic. You can also see the first set of tasks on the Next Generation Science Assessments portal's elementary tasks page.

  • Icon for: Remy Dou

    Remy Dou

    Facilitator
    Assistant Professor
    May 11, 2021 | 10:36 p.m.

    Comments: I truly appreciate this work and the attention given by the project team as a whole to teacher-led research and development. Specifically with elementary teachers, I'm inspired by the comments made regarding teachers' increased sense of confidence in relation to both teaching and assessing science content, as well as by existing opportunities to broadly disseminate this work. 

    Questions: Having just recently seen a video on the tendency of elementary STEM teachers to create their own curriculum rather than adopt existing curriculum during the COVID-19 transition to virtual learning, I'm curious about the pandemics' effects on your work. Specifically, did you have an opportunity to see how your teachers implemented these assessments during virtual teaching? Did they adopt or adapt the assessments in unexpected and/or interesting ways?

     
    1
    Discussion is closed. Upvoting is no longer available

    Sabrina De Los Santos
  • Icon for: Liz Lehman

    Liz Lehman

    Co-Presenter
    School Development Manager
    May 12, 2021 | 06:30 p.m.

    Remy, we have been very inspired by our teacher collaborators as well! We love working with them!

    As you noted, elementary teachers are often on their own for science curriculum planning. That was part of our reasoning for designing the tasks to be curriculum-neutral. The teacher collaborators have been a great source of first-hand experience in terms of where they are looking for lesson and unit resources, how assessment tasks like these fit in, and what supports are most useful to teachers. Due to COVID-19, we were unfortunately not able to observe any of the assessments in action this year. Some teachers were able to implement some tasks in their virtual and/or hybrid teaching, though, and we have gotten great feedback from them that we are using during the revision process. We are hopefully that we'll be able to do a lot of observations next year!

     
    3
    Discussion is closed. Upvoting is no longer available

    Brian Gane
    Sabrina De Los Santos
    Remy Dou
  • Icon for: Sabrina De Los Santos

    Sabrina De Los Santos

    Facilitator
    Research and Development Associate
    May 12, 2021 | 12:29 a.m.

    Thank you for sharing this innovative and important work. I appreciate the ways you collaborate with teachers and I am curious to learn more about the evaluation in the next couple of years. The video mentions the online tools can be accessed by teachers across the world. I am wondering if you have a sense of whether teachers outside the USA are using your assessments and whether they are adapting them in any ways?

  • Icon for: Liz Lehman

    Liz Lehman

    Co-Presenter
    School Development Manager
    May 13, 2021 | 11:02 a.m.

    Hi Sabrina, thanks for asking this! The first set of tasks were only recently released for public use on the Next Generation Science Assessments portal's elementary tasks page, so we don't yet have any information about international use. It is definitely something we should keep track of, though, thank you for the idea!

     
    1
    Discussion is closed. Upvoting is no longer available

    Sabrina De Los Santos
  • Icon for: Brian Gane

    Brian Gane

    Lead Presenter
    Research Assistant Professor
    May 15, 2021 | 01:23 p.m.

    Hi Sabrina, adding to Liz's response:

    While the elementary tasks are brand new, the middle schools tasks have been available for several years and they do have some data on international use or access (I'll see if I can dig it up). We definitely think that science educators across the world will find these assessment tasks valuable because they let students use science practices (e.g., developing a quantitative model, analyzing and interpreting data, asking useful questions, etc.) to work with their science ideas and concepts (e.g., life cycles of plants and animals, forces and motion, etc.). 

    Your other question about adapting the assessments is an interesting one as well! One of the ways we are seeing teachers adapting the tasks is through the way they use the tasks with their students. Because these tasks are designed to support teachers' instruction in their day-to-day teaching, teachers can build them more authentically into their classroom learning environment. Many are not using them as a "quiz" that students do silently at their desks. Instead, teachers may have students act out a portion of the scenario, they may follow up the task the next day with a classroom demonstration to extend students' thinking, they might assign pairs/groups to work on one task, or complete the task as a group. We're looking forward to working with teachers over the next two years to better support all these use cases and more!

  • Icon for: Samuel Severance

    Samuel Severance

    Researcher
    May 17, 2021 | 06:15 p.m.

    Hi Brian and team! So great to see this assessment work at the elementary level.

    It sounds like the teachers who participated in the co-design of the tasks gained a great deal from the experience. I'm wondering if you could speak about (1) where you saw the most/least growth in terms of teachers' expertise and capacity for design, and also (2) what sort of supports you found most successful in the design space for supporting teachers and researchers in collaborating effectively.

  • Icon for: Brian Gane

    Brian Gane

    Lead Presenter
    Research Assistant Professor
    May 18, 2021 | 10:52 a.m.

    Hi Sam, thanks and I agree, there is a dearth of resources, especially assessment resources, at the elementary level! 

    Regarding (#1), we saw teachers really dig in to the specific performance expectation and all three dimensions of the NGSS that compose that performance expectation/standard. This level of detail allowed us to develop a shared understanding of what students need to know and be able to do to meet that standard. These specific and detailed conversations about the goal for students' learning then allowed us to then design tasks aligned to the specific dimensions of the standard. 

    Regarding (#2), we made heavy use of scaffolds, templates, and heuristics for the different stages of the design process that all focused on making our underlying approach to assessment development -- Evidence Centered Design -- more manageable. Over time we have experimented with involving teachers differentially in the design phases, to better emphasize their work on phases where their perspective and expertise is critical. 

    As we move into the next phase, working with a wider group of teachers as they use tasks in their classrooms, we aim to answer both of your questions but from the perspective of use of assessments in the classroom, rather than the design of such assessments.

  • Further posting is closed as the event has ended.