Hi Stephen - During the pandemic, our work focused on development. The museum was closed and local schools were remote or not allowing additional people into their buildings. Our findings (from teacher interviews) from the first two years of testing indicated that teachers found the program valuable and appreciated the connections and supports for instruction connecting the field trip to their classrooms. Our observations of classrooms helped us to identify ways of adapting the initial lessons to better support students and teachers (e.g., changing materials to reduce prep time and ensure they were easily attainable). We will continue to test materials once the schools open back up and can bring students on field trips and will focus research and evaluation on student outcomes, specifically in underserved schools. 

I'd love to hear more (maybe papers?) when you're ready. Really interesting work! Also, very interested in how you conducted your DBIR. Its always interesting to see how design changes across iterations

Hi Stephen - We do have have some early publications from this work. I put them up on a website just for this event. You can find them here: https://moxi.education.ucsb.edu/publications

Hi Billy, I'm glad you enjoyed the student talk. I personally always want to hear what the students are saying! The evaluation of classroom activities was done as part of our Design-based Implementation Research (DBIR) methods to inform the development and revision of curriculum. Data collection included field notes, video recordings, student work, and teacher surveys and interviews. Feedback on draft curricula was also collected from teachers. Revisions to the curriculum resulting from evaluation findings included simplification of activities, changes in choices of materials, reduction of teacher-centered instruction, differentiation for a variety of abilities, changes in language and graphical representation of data to target specific grade levels and be accessible to a wide range of abilities, and the addition of additional educative resources to support teachers in learning how to implement engineering design lessons in their classrooms. These changes have made the curriculum easier for teachers to implement and accessible to a broad audience of students.

Thanks Ron! I really like the DBIR approach, which in many ways parallels the engineering design process that the project is focused on. It's great that you are using multiple sources of evidence from students and teachers to create a robust picture of what is happening with students and teachers.

Kemi - Thank you. Our lessons are targeted to grade level bands. That is, some are appropriate for grades 2-3 and others for grades 4-6. Because MOXI serves students in all grades, sometimes in back-to-back implementations, we design the museum experiences to be able to serve multiple age levels with few changes. The associated classroom activities are more targeted for specific grade levels and meet grade level standards. We have found that the engineering and science tasks can serve multiple grade levels if we adjust the math and reading/writing expectations. This is especially applicable to the types of data they collect and how we guide their analysis. The curricula for our first few modules are in the process of being edited and we anticipate that they will be available soon. 

Hi Diana, Thanks for the question. Analysis of teacher interviews has provided insight into teachers' perceptions of student learning outcomes from the programs. Teachers discuss opportunities for students to make real-world connections and develop 21st century skills through the engineering activities. Many mention communication and collaboration as key skills these programs have allowed their students to develop. Multiple teachers also mentioned students developing the ability to learn from "failure" as they optimize their designs. Teachers describe students as "engaged", "excited", and "involved" throughout the program. The engineering activities have also provided students a space to take initiative in their learning as teachers realize they can step back and allow the students to guide the process. Providing the classroom connection in addition to the museum field trip has allowed students to gain increased comfort with the engineering design process.

What a great quote! Thanks for sharing. Children are born to be explorers and innovators, and they tend to jump fearlessly into these open-ended challenges. I share Abigail's sentiment that her generation will be able to tackle the problems of the future!

I feel the same way. - thanks for sharing.

I second your point and question, Lauren! We are so conditioned against failure or perceived negative outcomes in our work that we often miss important relationships and opportunities for understanding. The same can be said of the students and professional development for teachers here. Was there anything that did not work and gave you additional ideas for what to change or how to improve your understanding?

Likewise, the frames where students and teachers are not just smiling (simply to communicate happiness) seem much more meaningful for the engagement in the content and interactions. And a quick recommendation about the visual style of the presentation - the split-frame segments seem to take away from an otherwise carefully crafted peek into the project. Consider revising in future iterations of the video. So well done otherwise! Exciting project and very clever theme selections and project design. I would have wanted to be part of such a program!

Hi Lauren,

We are currently analyzing how children persist through and learn from failure during the Engineering Exploration design challenges. One case study of a group of fourth grade boys in particular demonstrates the varying degrees to which failure can be observed in student behavior. Interestingly, in this one specific instance, there was a direct relationship between the severity of the student's response to failure and their creativity and innovation as they subsequently iterated their designs.

After taking a risk and failing, the students with minimal response (a shoulder shrug or a comment about wasted time) returned to their initial design ideas in subsequent iterations and did not make significant improvements. In contrast, one student who fell apart when his design failed (tears, pouting, isolation from the group, hesitation to re-engage, pessimism towards the group's second design) eventually re-engaged and made a final iteration of his initial design which incorporated learning from the group's failed design (that he had only observed and not participated in). His final design was one of the most innovative of the whole class.

His behavior was interpreted by many as being disruptive and the rest of his group was praised for their behavior and how well they completed the activity. When asked about the group's second design, he stated "we're failing." His comment was dismissed by both the group and the person asking the question. However, in analyzing the subsequent dialog, he appeared to have a clear idea of the criteria of the problem, a model for predicting the motion of the object being designed, and a sense that the constraint of time was critical. At one point he says "Do it! Test it! Right now!" This language is consistent with the engineering practice of testing prototypes early and often. His awareness of criteria and constraints and his use of a conceptual model to predict an outcome are also consistent with the practices of engineering. Yet his everyday language and behavior in dealing with failure were easily misinterpreted and his contributions to the group and final design were ignored.

We are working on understanding how children articulate emerging engineering practices through their everyday language, actions and gestures. This understanding will help teachers identify the alignment between students' everyday language and problem-solving skills and the practices of engineering, the most important of which is persisting through and learning from failure.