Thank you for visiting the CADRE video. With this video we are hoping to bring a new kind of discussion to the video showcase.  The video is not about one project’s work. It is about all of our work to improve education through research. The video is meant to foster discussion about different approaches to having a broad impact. What are these approaches? How are we making impacts? With whom? And how do we know we are making an impact? What new approaches are needed? We invite you to respond to these questions.

Catherine, you certainly have a big impact in terms of numbers. I have used some of these resources myself and found them to be "one of a kind" that help me support learning. What have you learned about scaling up in terms of how you market these resources, make them available, innovate on them based on user input? 

Yes, very inspirational stories of the successful dissemination of NSF funded research to achieve broader impacts! I know that there is often a tension between achieving wide dissemination and maintaining sufficient fidelity of implementation to ensure that the intervention continues to function as originally designed. I’m wondering how the projects highlighted have grappled with this dilemma as they scale up their efforts and if they’ve been able to study the conditions under which interventions can successfully scale, such as the comment in the video about the advantages of interventions delivered online? Also welcome would be any advice or lessons learned on how these projects have been able to reach millions of students!

Having seen some of these projects at their very beginnings, it is incredible to hear of their very significant impacts. For materials development and dissemination, quantitative results are impressive. BUT for the DRK-12 projects focused on professional development for strategies, can I assume it is still more difficult to identify impacts? What are some successful measures?

Barbara, you raise a great question about the impact of projects focused on professional development. In a previous DRK-12 collaborative project between Jonathan Osborne and myself at Stanford, and Craig Strang and Emily Weiss at Lawrence Hall of Science, the Stanford team studied the impact of PD facilitated by the LHS team on science discourse practices and students’ science achievement.  Using a measure of science discourse that we developed, we did find an impact on teachers’ and students’ discourse practices. We also found some evidence for an impact on student achievement; however, this finding is based on very limited data due to difficulties in attrition and collecting data. One article we published describes the instrument and presents initial findings:

Fishman, E.J., Borko, H., Osborne, J., Gomez, F., Rafanelli, S., Reigh, E., Tseng, A., Million, S., & Berson, E. (2017). A practice-based professional development program to support scientific argumentation from evidence in the elementary classroom. Journal of Science Teacher Education. http://dx.doi.org/10.1080/1046560X.2017.1302727

We are currently revising an article that includes analyses of the entire data set.

I’d also like to add another layer of complexity: projects focused on scaling up by preparing PD leaders. In another DRK-12 project, Karen Koellner, Jennifer Jacobs and I worked with a local school district to prepare site-based teacher leaders to facilitate the Problem-Solving Cycle model of mathematics professional development at their schools. We were able to identify changes in teachers’ knowledge and some features of instructional practice, although our analyses were descriptive due in large part to attrition in the samples of teacher leaders and teachers.  The nesting of students within classrooms (teachers), and teachers within schools (teacher leaders) also added to the challenge of attributing changes to the preparation of the PD leaders. The project is described in our book, which also lists additional publications:

Borko, H., Jacobs, J., Koellner, K., & Swackhamer, L. (2015). Mathematics professional development: Improving teaching using the Problem-Solving Cycle and Leadership Preparation models.  New York: Teachers College Press.

My current project adds yet another layer of complexity in that it is a Research-Practice Partnership to adapt and enact the Problem-Solving Cycle and Leadership Preparation models. We don’t have findings to share yet. You can learn about the project in this video:  http://stemforall2018.videohall.com/p/1299

I agree that it’s more difficult to identify the impacts of projects focused on professional development. As Amy pointed out, we are making progress in developing measurement tools and identifying outcomes, in large part because of DRK-12 funding and CADRE efforts. I also look forward to reading about other projects’ experiences and insights!

As a prior state education department member, I am particularly interested in how we facilitate the dissemination of effective strategies and programs to educators and leaders across the K-12 system. The work CADRE is doing to coordinate across federally funded projects and support dissemination is an excellent step in making such work from the research community more systemic and readily available to the practitioner community. As a project advisor for CADRE I am grateful for the work Catherine, Amy, and all the staff are undertaking to disseminate to a wide audience.

With that said, I am interested in hearing from educators, leaders, and policy makers about what how findings and products from the research community would be most useful for your work. Is it the actual products that are developed that have the potential to be taken up in your school, district, or state? Is it the broader strategies or findings that you could apply to your own program development? Or might it be a process that you can employ as you review or build STEM programs?

Hello, Katherine and colleagues. It’s incredible that you’ve been able to reach so many students and teachers. What do you envision as next steps for your work? Have you investigated the longitudinal impact of Engineering is Elementary on students’ interest in engineering specifically and STEM fields in general?

Jake's comments brought back an idea I've been thinking about for a while. It is not a long term or systemic solution, but it could lead to some short and mid-term improvements in STEM education.  So much of all of our dissemination efforts focus on awareness building - building familiarity and knowledge of practitioners, policy makers, education leaders, researchers and others. While there have been resources for this type of  dissemination centers/networks/etc., in the recent past, there have not been resources set aside for states, districts or schools, who would like to move beyond awareness to initiate planning or early implementation efforts. Is this something worth pursuing with foundations, corporations, or government agencies? 

Hi, 

I appreciate both the video and the questions posed regarding the impact of education research and the question of "who" is actually benefiting from it. I think these are critical questions that all researchers should consider when designing and implementing their studies, particularly when it comes to developing and testing new technologies, practices, and programs. From personal experience, I've seen some great ideas that bring about a promise for a substantial impact that meets a lot of structural and systematic barriers that often reduce the level of impact or the potential of the project. I wonder how people are accounting for those structural and systematic barriers that occur. One prime example for such would be implementing new technologies or practices without formal professional development for the teachers or considering the actual infrastructure of the school to support and sustain those technologies and programs. 

More often than not, when we run into these types of barriers, we find great ideas that are either implemented or scaled to districts and sites that have the resources to support them leaving those underresourced without the opportunity. Or, those same ideas only last for the intermittent period of the research grant, which again leaves those underresourced without the opportunity. In our efforts of promoting "STEM for All," I cannot help but wonder if we are truly fulfilling this mantra, or if we are promoting a "STEM for All, but with specific conditions."

How did you (are you) accounting for institutional and systematic barriers? Were these a part of the conversation during the planning phase or did they come about after initiating the project? In facing those barriers, who (which demographic of students, teachers, etc.) were most detrimentally affected and how are you accounting for it? 

A few delayed responses. To Amy and Hilda, sounds like there has been a lot of progress and I look forward to reading the articles and reports you cited. To Judi and Jake, it seems like there may be some opportunities on the horizon. 

Hello Catherine, Amy, Barbara and all,

As a former science teacher and now an instructional coach I enjoyed both the video and the questions posed about the impact of education research, who is actually benefiting from it, and which professional development strategies are most impactful.

I would like to comment on  Barbara’s question, “So much of all of our dissemination efforts focus on awareness building - building familiarity and knowledge of practitioners, policymakers, education leaders, researchers and others. While there have been resources for this type of dissemination centers/networks/etc., in the recent past, there have not been resources set aside for states, districts or schools, who would like to move beyond awareness to initiate planning or early implementation efforts. Is this something worth pursuing with foundations, corporations, or government agencies?”

As an instructional coach, I work with teachers sharing education research, modeling practice, and debriefing instructional strategies. There is an awareness among teachers about STEM teaching, as well as some familiarity and knowledge about products that have the potential to impact student learning. What continues to be a challenge is helping teachers with the day-to-day “how to implement” STEM learning.  Teachers are trying to use education research to make changes in their practice but they are doing so while they are “flying the plane.”

Resources set aside for states, districts or schools, who would like to move beyond awareness to initiate planning or early implementation efforts might be a way to provide the support needed for schools/teachers in a manner that would free teachers from “building the plane” while they are flying it.  I wonder if this would help teachers develop a deeper, thoughtful understanding of how to implement effective STEM learning.

Terrell,

A delayed response to your comment: I agree with Catherine that you are raising very important points. Research-Practice Partnerships are one approach that I and others have taken to address structural and systemic conditions. Typically, the partners – often university researchers and district personnel – work collaboratively to identify a problem of practice and plan possible solutions, and they continue to collaborate as they work iteratively to design, enact, and study the solutions. I mentioned our research team’s partnership with a local school district focused on mathematics professional development in an earlier post. You can learn about the project on this video:  http://stemforall2018.videohall.com/p/1299

Thank you, Catherine, for noting our project’s interest in systemic approaches for introducing STEM to schools!

And thank you, Terrell, for your thoughtful query:  How did you (are you) accounting for institutional and systematic barriers? Were these a part of the conversation during the planning phase or did they come about after initiating the project? In facing those barriers, who (which demographic of students, teachers, etc.) were most detrimentally affected and how are you accounting for it? 

In our Center’s experience, "institutional and systematic barriers" are present in every school. Our NSF project, "Systemic Transformation of Inquiry Learning Environments 2.0," (http://videohall.com/p/1182), identifies key elements for developing the capacity of teachers to design STEM projects in their individual school settings. In this way we are able to co-create and tailor the project to the needs of the teachers and administrators in each school. "Situating" the process (Putnam & Borko, 2000) in this fashion is fundamental to our approach.  Because we are focused on implementing key elements rather than a lockstep program, we have some flexibility in addressing barriers, even as they emerge.

However, what is considered a "barrier" varies according to one's perspective. Teachers may see the lack of collaborative planning time as a barrier, even though the administrators may not yet identify this as a problem. The partnership between the university researchers, teachers, and administrators is critical, as Hilda Borko notes (above).  Strong partnerships create "shared meaning" (Fullan, 2015) rooted in the needs and interests of the school.  Shared meaning, in turn, makes it possible for those involved to identify creative ways to remove barriers inhibiting the effective implementation of the project.