Thank you for the feedback and questions. This study is underway in the classrooms of ten teachers, and will scale up next fall to 40 teachers. Our research questions focus on the validity of the assessments, their use in classrooms, and the policy implications of our findings for developing systems of assessments from the classroom to state levels. Student engagement is included among the research questions and is being studied through classroom observations and think-aloud studies with individuals and pairs of students. The assessments are built upon system models that can be used to explain system-level phenomena based on an understanding of the interactions among the components of the system. The system models are used in the development process for the assessments, in the professional development for educators, and emphasized in the coaching system, reports to students and teachers, and highlighted in follow-on classroom activities designed for formative use in tandem with the curriculum-embedded assessments. Although the study is underway, past research and data collected to date show that the assessments are feasible for teachers to use in their science classes, engaging to students, and useful for monitoring and adjusting instruction. Preliminary data also point to the benefits for student learning, particularly for English learners and students with disabilities. The primary challenges facing the program are adjusting to changes in the education landscape and competing demands on teacher time associated with adoption and implementation of new standards and multiple models for distributing Earth, life, and physical science topics across the middle grades.

Thank you for the feedback and follow-up to the questions above. Teachers’ and students’ responses to the program have been overwhelmingly positive. In interviews and surveys, teachers highlight the benefits of immediate feedback, interactivity, and visuals. In classroom observations and think-alouds, students are focused, on task and, when provided with opportunities to work in pairs, discuss ways to use the simulations to investigate and engage in scientific discourse to make sense of the phenomena.

Thank you for the feedback and questions. The SimScientists program is built upon a base of research in learning and cognition. The key principles guiding development are Evidence-Centered Assessment Design (Mislevy, Almond, & Lukas, 2003), Model-Based Learning (Gobert & Buckley, 2000), Universal Design for Learning (Rose & Meyer, 2000), Universal Design for Computer-Based Testing (Harns, Burling, Hanna & Dolan, 2006), and Formative Assessment for Students and Teachers (CCSSO, 2005). More information about our research base is available on our website at simscientists.org.
Bayes’ Nets are used as part of the evidence model to synthesize evidence across a variety of tasks, student response patterns, and process data from student investigations. The Bayes’ Nets specify how this evidence is used to estimate students’ ability on a variety of science and engineering practices and core ideas in science.

Very interesting! Getting good estimates of students’ ability is very challenging – I’ll take a look at your website to read more about how you’re tackling this problem. Thanks.

Thank you for the feedback and questions. Please see my response above about our design principles. The current project is part of an ongoing line of work in the SimScientists program at WestEd. The SimScientists program has developed suites of instructional modules, curriculum-embedded assessments for formative use, end-of-unit benchmark assessments, and end-of-year assessments for 11 topics in middle school science and one topic in high school biology. The program has also developed an online professional development program for educators and a Learning Management System to deliver the modules to students and reports to teachers and students. You can find descriptions of our current and prior research and development projects on our website at simscientists.org.