4777 Views (as of 05/2023)
  1. Jamie Mikeska
  2. Research Scientist
  3. Presenter’s NSFRESOURCECENTERS
  4. ETS Educational Testing Service
  1. Emily Borda
  2. Professor, Director
  3. Presenter’s NSFRESOURCECENTERS
  4. Western Washington University
  1. Katherine Castellano
  2. Senior Research Scientist
  3. Presenter’s NSFRESOURCECENTERS
  4. ETS Educational Testing Service
  1. Dante Cisterna
  2. Assoc. Research Developer
  3. Presenter’s NSFRESOURCECENTERS
  4. ETS Educational Testing Service
  1. Dan Hanley
  2. Director of STEM Education Research and Evaluation
  3. Presenter’s NSFRESOURCECENTERS
  4. Western Washington University
  1. Debi Hanuscin
  2. Profeesor
  3. Presenter’s NSFRESOURCECENTERS
  4. Western Washington University
  1. Josie Melton
  2. Post Doctoral Research Associate and Senior Instructor
  3. Presenter’s NSFRESOURCECENTERS
  4. Western Washington University

Developing and Validating Assessments to Measure and Build Elementary Teacher...

NSF Awards: 1813254, 1814275

2020 (see original presentation & discussion)

Adult learners

Effective teaching requires understanding that goes beyond knowing the subject matter and includes professional knowledge that teachers draw upon as they engage in the work of teaching within a specific discipline, which we call content knowledge for teaching (CKT). Supporting the development of teachers’ CKT requires valid and reliable assessments to measure teachers’ CKT and instructional materials to build teachers’ CKT. This video will highlight the ongoing work from a collaborative research project across two institutions – Educational Testing Service and Western Washington University – to develop and examine how CKT science assessment items and instructional tasks targeting matter and its interactions at the elementary (K-5) level can be utilized in elementary teacher education settings to evaluate and support elementary science teachers’ ability to apply their content knowledge to the work of teaching science. We will describe the summative and formative CKT assessment tools that we have developed for use within elementary science method and content courses and explain how we envision their productive use within these settings. Overall this video will offer an insider’s look at a set of tools that can be used to positively impact preservice elementary teachers’ CKT in one critically important science content area and detail the learning opportunities these tools afford for both preservice elementary teachers and the teacher educators that work with them.

This video has had approximately 545 visits by 424 visitors from 225 unique locations. It has been played 264 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2020 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2020 STEM For All Video Showcase (26 posts)
  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Lead Presenter
    Research Scientist
    May 4, 2020 | 01:57 p.m.

    Welcome to our video! Thank you for taking the time to watch our video and comment on the research and development work we’ve been doing to support elementary science teacher educators and the preservice teachers they work with. We are excited to share with you the work that we’ve been doing to help build and assess elementary preservice teachers’ content knowledge for teaching (CKT) in one science content area: matter and its interactions. During the first two years of the project, we’ve been busy working to develop a set of instructional materials – what we call CKT Packets – for elementary teacher educators to use as part of the elementary science content and methods courses and a CKT matter summative assessment instrument to measure the preservice teachers’ CKT over time. We look forward to hearing from you with any questions, comments, or ideas about how the field might use these types of CKT tools in teacher education and professional development settings.

     
    2
    Discussion is closed. Upvoting is no longer available

    Stefan Irby
    Amy Wagler
  • Icon for: Hannah Sevian

    Hannah Sevian

    Higher Ed Faculty
    May 4, 2020 | 09:27 p.m.

    I'm curious how you are building in ways to support preservice elementary teachers' ability to apply ideas about matter and its interactions in the context of different crosscutting concepts, such as cause and effect, mechanism and explanation, or patterns, or systems and system models. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Lead Presenter
    Research Scientist
    May 5, 2020 | 11:17 a.m.

    Good question! For each of the CKT matter items we developed, we identified the crosscutting concept (in addition to the disciplinary core idea and scientific practice) that is most closely aligned to what is being assessed within that item. Each CKT matter item starts with a description of an instructional scenario that situates the pre-service elementary teacher into a specific aspect of the work of teaching science. Sometimes the items require the teachers to more explicitly apply matter ideas in the context of the crosscutting concepts, and other times the connection is more implicit. At a recent conference, I talked with another researcher about this and we agreed that strengthening these connections and making them more explicit would be another way to broaden the impact of this research and development work. Our partners at Western Washington University have also developed CKT matter packets and some of the reading pages and portions of the lesson plans they designed for teacher educators also supports these connections.

  • Icon for: Debi Hanuscin

    Debi Hanuscin

    Co-Presenter
    Profeesor
    May 5, 2020 | 12:53 p.m.

    Hi, Hannah-

    The CKT Packets include explicit connections to the CCCs--for example, the Packet focused on the content idea of conservation of matter is linked to Energy and Matter. Each Packet has a section that outlines connections to the standards and the expanded lesson plans help teacher educators elaborate on those connections.

  • Icon for: Susan Kowalski

    Susan Kowalski

    Researcher
    May 5, 2020 | 01:48 p.m.

    I enjoyed your video. It sounds like the packets are a great resource for pre-service teachers to deepen their understanding of important science concepts and how to teach them. Have you thought about making these packets available to in-service teachers? I'm wondering if teachers might be able to use the packets in a professional learning community.

     
    2
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
    Judi Fusco
  • Icon for: Josie Melton

    Josie Melton

    Co-Presenter
    Post Doctoral Research Associate and Senior Instructor
    May 5, 2020 | 02:35 p.m.

    Hi Susan,

    Yes, packets can also be used for in-service teachers to develop their CKT.  Packets can be accessed at  https://cktscience.org/ .  Note that you must register in order to access materials on the website.  Some packets are still in development, but will be made available once they are ready.  Thanks for your interest!

     
    1
    Discussion is closed. Upvoting is no longer available

    Judi Fusco
  • Billy Jackson

    Higher Ed Faculty
    May 5, 2020 | 08:08 p.m.

    I am curious as to how you see the materials being used.  Unlike in math, in many places, preservice elementary teachers often do not take content courses specifically designed at understanding elementary science ideas as a much deeper and fundamental level required to teach it.  This is probably why starting elementary report that they feel the least prepared to teach science among all of the subjects they are responsible for.

    I like having the assessment particularly in that it seems to a goal of the project to validate the instrument specifically for preservice teachers, as other instruments have focused on inservice teachers mostly in their validation efforts.  I wander if the assessment might be used as a basis for future efforts among policymakers to try and advocate for change in how the future teachers are prepared in science to mirror what is happening in math.

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Josie Melton

    Josie Melton

    Co-Presenter
    Post Doctoral Research Associate and Senior Instructor
    May 6, 2020 | 12:41 a.m.

    At Western Washington University, where we are developing the instructional materials, the packets have been piloted in elementary science methods courses and science content courses offered specifically for future elementary teachers.  We have found that the packets deepen the content focus in methods courses and provide more opportunities to highlight teaching practices in the science content courses.  Another potential use is for in-service professional development.  Thanks for the question!

  • Billy Jackson

    Higher Ed Faculty
    May 5, 2020 | 08:08 p.m.

    I am curious as to how you see the materials being used.  Unlike in math, in many places, preservice elementary teachers often do not take content courses specifically designed at understanding elementary science ideas as a much deeper and fundamental level required to teach it.  This is probably why starting elementary report that they feel the least prepared to teach science among all of the subjects they are responsible for.

    I like having the assessment particularly in that it seems to a goal of the project to validate the instrument specifically for preservice teachers, as other instruments have focused on inservice teachers mostly in their validation efforts.  I wander if the assessment might be used as a basis for future efforts among policymakers to try and advocate for change in how the future teachers are prepared in science to mirror what is happening in math.

  • Icon for: Alison Heimowitz

    Alison Heimowitz

    Facilitator
    School and Teacher Liaison
    May 5, 2020 | 10:02 p.m.

    Like Billy Jackson, I am interested in learning more about how you see these materials being used given that pre-service elementary teachers generally lack a deep understanding of science concepts and principles. Because not all students learn science the same way, I'm also wondering in what ways you were able to incorporate culturally responsive teaching into the packets?

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Lead Presenter
    Research Scientist
    May 6, 2020 | 11:42 a.m.

    In the CKT packets, there is definitely a focus on attending to the variation and complexity of scientific ideas that students bring with them and understanding the various sources for those ideas. This focus provides a way for teacher educators to help their pre-service teachers learn about science concepts and principles but in the context of considering student ideas. We also think it will support the pre-service teachers in bringing an equity-focused lens to their future science instruction -- as learning how to elicit and attend carefully to students and what they bring with them is vitally important!

  • Icon for: Debi Hanuscin

    Debi Hanuscin

    Co-Presenter
    Profeesor
    May 7, 2020 | 12:04 p.m.

    I'll add that we designed the CKT packets to be flexibly used either in content or pedagogy courses-- there are several teacher/preservice teacher content-focused curricula (e.g., PET, Physics by Inquiry, Making Sense of Science) that these would work in concert with. 

  • Icon for: Albert Byers, Ph.D.

    Albert Byers, Ph.D.

    Researcher
    May 5, 2020 | 11:15 p.m.

    What a wonderful project, and as I see colleagues such as Deborah H and Sean S at Horizon Research, with ETS, you all have assembled a wonderful team of deep thinkers. I look forward to looking at these assessment tools for elementary pre-service teachers in more detail!

    As an aside, and excuse the simplicity of this question before I am able to look at the tasks in more depth for pre-service teachers: How are these tools similar or in part draw upon the notion of Krajcik's "educative instructional materials" for teachers (providing teacher background support in subject matter and pedagogical implementation) or the work of Shulman (year's ago) looking at pedagogical content knowledge within specific disciplines. I see some synergy, and it may be as simple as it is an "assessment task" for pre-service teachers!

    Very exciting and needed project! Thank you for sharing.

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Lead Presenter
    Research Scientist
    May 6, 2020 | 11:48 a.m.

    Thank you for your kind words Albert. If you want to take a look at the current CKT packets and example CKT matter assessment items, you can do so at our project's website at cktscience.org. You'll need to register for a free account to view the packets and sample items.

    In response to your questions, in the design of the packets we did take up the idea of educative instructional materials -- but for teacher educators! That is what we hope is part of the contribution of this work to the larger field. On the cktscience.org website, you can read more about the various design heuristics and educative features we've incorporated into the CKT packets. In addition, many of the CKT matter items require teachers to leverage their pedagogical content knowledge -- particularly their knowledge of students' ideas and knowledge of instructional strategies -- in this content area. Those are great connections to the work that we are doing on this project. 

  • Icon for: Albert Byers, Ph.D.

    Albert Byers, Ph.D.

    Researcher
    May 6, 2020 | 12:31 p.m.

    Thank you Jamie. I really appreciate the work and what this contribution will make. I definitely will dig deeper and seek to learn more! Years ago, I led an effort developing little online nuggets of just-enough, just-in-time, just-for-me science content and PCK modules called Science Objects/SciPacks at NSTA (with a GREAT team of internal folks and external experts) and we spent significant time thinking along these lines (drawing from the latest research at the time). This is impressive work, building on the prior body of knowledge and will have meaningful impact. Kudos! You've got my vote.

  • Icon for: Debi Hanuscin

    Debi Hanuscin

    Co-Presenter
    Profeesor
    May 7, 2020 | 12:06 p.m.

    Thank YOU, Al-- One of our faculty workshop participants told us that the packets were like "Page Keeley's books but for faculty!"-- we tried to base them on tasks that could be implemented easily, while also supplying the supplemental materials and information to help develop preservice teachers' ideas more deeply. 

  • Icon for: Judi Fusco

    Judi Fusco

    Facilitator
    Senior Researcher Stem Teaching and Learning
    May 6, 2020 | 12:59 p.m.

    I enjoyed your video and think the work you are doing with pre-service teachers is so needed.  So the focus is on content, but it seems like, since these are situated in practice, there would be some pedagogical knowledge benefits, too? What do you see?  (I know that's not the focus, but I got curious as I watched, especially as your advisor Sean Smith brought it up, too.)  

    Great job!

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Josie Melton

    Josie Melton

    Co-Presenter
    Post Doctoral Research Associate and Senior Instructor
    May 6, 2020 | 06:04 p.m.

    Just to add to Jamie's response, the packets are structured so that they target one content sub-topic about matter and its interactions, and one teaching practice.  So, as preservice teachers engage in the lesson provided in the packet, they are deepening their content knowledge while learning about instructional decisions that teachers make when teaching about that content.  If you are interested, feel free to check out some packets on our website by signing up to access the tools:  https://cktscience.org/

     

     
    1
    Discussion is closed. Upvoting is no longer available

    Judi Fusco
  • Icon for: Judi Fusco

    Judi Fusco

    Facilitator
    Senior Researcher Stem Teaching and Learning
    May 10, 2020 | 03:22 p.m.

    Thanks, Josie and Jamie for your replies.  I will check out the packets.  C and P are certainly interrelated.  Very interesting work you have!

  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Lead Presenter
    Research Scientist
    May 6, 2020 | 05:26 p.m.

    Our work is focused at the intersection of content and practice. To build and measure the preservice teachers' CKT in this area, we focus on how they leverage their content knowledge to engage in science teaching practices (e.g., eliciting student ideas; selecting instructional resources to address specific learning goals; etc.)-- or what we refer to as the 'Work of Teaching Science.' As part of the CKT matter assessment development process, we conducted cognitive interviews with almost 80 preservice teachers to examine how these items functioned and to determine the knowledge that teachers used to respond to each item. Overwhelmingly we found that the teachers had to both attend to and draw upon aspects of the content and aspects of content teaching in order to successfully respond to the CKT matter items, which suggests these item types measure more than only content knowledge alone and having teachers interact with these types of items within the CKT Packets would also have some pedagogical benefits as well.

     
    1
    Discussion is closed. Upvoting is no longer available

    Judi Fusco
  • Icon for: Kenne Dibner

    Kenne Dibner

    Facilitator
    Senior Program Officer
    May 6, 2020 | 08:09 p.m.

    Hi Jamie! Nice to see you in this virtual context - hope you and your family are doing well!

    Great video, as I would expect. This is super interesting work. I'm wondering about the evaluation component - did you happen to notice any differences in teacher performance? We know so little about how in service PD matters for student performance down the road - I'd be interested in knowing if you could account for any differences in teachers' uptake of the materials.

    Great seeing your face here!

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Lead Presenter
    Research Scientist
    May 7, 2020 | 08:25 a.m.

    Hi Kenne! It is great to see you too. Thanks so much for taking a look at our video. Right now we are in the second year of this four year research project. The first two years have been focused on developing these tools so we do not have much evaluation data to share at this point (although we do have a lot of data from our field test, pilots, and cognitive interviews for the CKT matter summative instrument). However, we are gearing up to integrate the tools within elementary science content and method courses starting this fall and plan to examine the extent to which using the CKT Packets helps to improve the preservice teachers' CKT about matter. As part of that work, we will be able to learn more about how both teacher educators and their preservice teachers use, understand, and interact with these materials. The connection to student performance down the road is a good one -- we are not able to examine that connection yet, but hope to in future work!

     
    1
    Discussion is closed. Upvoting is no longer available

    Judi Fusco
  • Icon for: Kenne Dibner

    Kenne Dibner

    Facilitator
    Senior Program Officer
    May 7, 2020 | 01:30 p.m.

    It's so hard to connect student performance to teacher PD, but having teacher use data would be helpful. Exciting!

     

  • Icon for: Kenne Dibner

    Kenne Dibner

    Facilitator
    Senior Program Officer
    May 7, 2020 | 01:32 p.m.

    Also, I meant to tell you to look out for the new NASEM study on Science in PK-5! First committee meeting was this week!

     
    1
    Discussion is closed. Upvoting is no longer available

    Dante Cisterna
  • Icon for: Michael I. Swart

    Michael I. Swart

    Researcher
    May 11, 2020 | 05:16 p.m.

    Thank you for sharing this work.  For the assessment, do you have a resource site? Curious about the types of items, scoring, validity, reliability? 

    In the video, was mentioned that CKT has a metacognitive component to connect pedagogical knowledge with content knowledge.  Can you expound on the approach of the CKT framework and any theoretical constructs that undergird its operationalization? 

  • Icon for: Katherine Castellano

    Katherine Castellano

    Co-Presenter
    Senior Research Scientist
    May 12, 2020 | 07:53 a.m.

    Hi Michael, 


    Thanks for your interest in our work. We have our CKT packets and sample items available at our website: https://cktscience.org/. You just need to register to access them. We are still in the midst of finalizing the scaling of our assessment so those technical details are not posted yet. 


    We relied on the "Work of Teaching Science" framework in our theorization of the CKT construct. We provide more information about this on our "CKT" tab of our website: https://cktscience.org/ckt/. 

  • Further posting is closed as the event has ended.