Thanks, Sharon!

Hi Tami,

Our Engineering Internships have been designed for middle school students. We have conducted field tests with over a thousand students, and are midway through research trials with hundreds more.

The "fictional" project director, who greets the students in the initial video, is who the daily email is from, and also s/he also provides personalized feedback to students on their designs. (the teacher actually is the one to decide what feedback to send students, and when—but that is behind the scenes) The Teacher plays the role of internship coordinator. She coaches students from the side, as help is requested, and advise students on what the expectations of an intern are, treating this as students' first foray into the "work world". We set things up in this way for several reasons: 1) to create a more authentic feel—where students are "on first." Even though these internships are used in a school context, we want students to feel like that are in an internship, and that this is their opportunity to shine. Teachers have commented that students really step up to the opportunity; 2) so teachers, new to engineering, wouldn't feel like they need have engineering expertise in order to use these with students—many teachers have told us they have really enjoyed being in a sidelines role, both as a change of pace, but also seeing their students really tackle challenges handed to them "by someone else"; and 3) to provide a more student-centered experience.

Yes—Cary, correctly anticipated my answer to your last comment. Students engage in a full inquiry cycle, but our assessment focuses in on one of the three core ideas in engineering from the NGSS—Optimization (and we break that down further).

Thanks so much for taking the time to look at the video and ask questions!

Jacquey

HI Cary!

Our digital internships do include hands-on experiences. Each internship begins with a Research Phase, where students come to understand the problem they are trying to solve, and become familiar with the variables involved. It is in this Research Phase that students engage in hands-on experiences. For instance, in the Force and Motion Digital Internship, students are challenged to design and test emergency supply drop pods, attempting to minimize cargo damage to ensure the safety of the cargo, maximize the shell condition so the pod can be used for shelter, and keep pod costs low so that more pods can be build and dropped to an many people in need as possible. In the Research Phase, students conduct the classic egg drop experience. This serves as a physical model for the design challenge, and helps them gain experience in manipulating the variables of mass, velocity, and duration of experienced collision as ways to reduce impact force. They then move into the second phase of their internship—the Design Phase. We created digital design tools where students conduct their ultimate design of solutions. While students learn a lot from the hands-on egg drop experience that couldn't be learned otherwise, they aren't able to make and test countless iterations, as they are with the digital design tool. In this case, using a digital model is more fruitful in learning about optimization than the physical model of the egg drop, and it bears authenticity to how many engineers work. The third phase of the experience is the Proposal Phase, where students write a proposal (responding to the RFP they got at the beginning of the experience). The proposals they write are actually design arguments, where students muster evidence to support their proposed design. 

Hope this makes sense!

Jacquey

Thanks for your questions, Vivian!  We designed the digital engineering internships to follow science units on specific topics. The idea is that students will engage in learning relevant science, and then apply that science as they take on the design challenge in the Digital Engineering Internship. So for instance, the internship described that involves students in creating emergency supply drop pods would be used as a culminating experience following a unit on Force and Motion. The whole experience serves in some ways as an assessment of their ability to apply their newfound science knowledge to a novel problem. Given this, we haven't offered students choice of which internship to take on. Because students turn in their proposals (basically an argument supported by evidence to support their proposed design), we have seen that students are successful in coming up with a range of defensible designs, and can support why they think so. We don't yet have results regarding the different pathways students take to come up with a solution that optimizes for X or Y.  Stay tuned!

That's a great question. In the units, students work early on to specify the problem (e.g., articulating/operationalizing design criteria and constraints), develop initial design solutions, and through iterative testing of their designs, get great practice analyzing data.

We are initially focusing on optimization because it pulls in some of those practices (e.g., attention to design criteria, revising designs based on empirical evidence) and because we expect it to be reasonably tractable by analyzing student interactions with the digital design tool. That is to say, we expect the moves students make in the digital environment (that we are applying our learning analytics model to assess) to reveal a lot about how well they are optimizing solutions. Our long term hope is that we can parlay what we learn about using analytics to assess optimization to assess other science and engineering practices--stay tuned! 

I really like the rich problem scenarios that the team has created and offered to students in the online internships. The problems are also inviting to students. I want to do this!  

I am curious about the learning analytics model you are using and if you can tell me more about this. My colleagues at the Concord Consortium are using several analytic methods, one of which uses Monte-Carlo Bayesian Knowledge Tracing to assess knowledge growth using physics game log files in the context of a high school inquiry activity, also funded by the National Science Foundation. The possibility of being able to scale to many many students dynamic online assessments and formative feedback on more complex learning tasks and activities is exciting. 

Hi Sherry,
Thanks for your enthusiasm and your question about our analytics model! We, too, are currently exploring the use of several analytics models. Our main analytics model is a Dynamic Belief Network - similar to the BKT method you mention, but positing (and fitting for) additional connections between skills. We are currently developing two layers of interpretation of the log-file data to feed into our Dynamic Belief Network: 1) a top-down approach consisting of evaluations of student behavior along expert-specified patterns, and 2) a bottom-up approach that uses those pattern evaluations (along with the rest of the log-files) as features in a machine learning analysis to discover additional "optimization-aligned" patterns of behavior.
And yes, the potential applications and scalability of these kinds of automated analyses are super exciting! Such a wonderful time to be in the field!

This is also an area of interest for me in terms of scaling this sort of activity.  There seem to be some clear trade-offs in terms of the use of models and controlled environments vs. real world applications from the availability of meaningful learning analytics, to the ability to ensure that all learners regardless of their geographic location or other demographic factor have the same opportunity to explore and play. 

I’m wondering if there is any difference in the level of engagement/motivation/learning outcome achievement of students who work in the in these type of controlled environments vs. those who work in a truly authentic environment on a hands on project.

Thanks in advance for any thoughts you may have.

Thanks, Dale and Michael, those are great questions--and you've nicely captured some of the key trade-offs. In piloting and field trials, we've found that kids are really invested in the problems that are the focus of the existing internships (we pilot and iterate on the ideas to make sure that they are solving problems that are meaningful to them). But, the idea of kids parlaying these experiences, and their newfound engineering chops, to tackle local problems is a really cool idea that i hope we can explore in the not-too-distant future! And, wow, doing so would enable some great research to examine those differences.

Agreed!  It's really exciting to see learning occur in more deeply situated and even authentic environments, with assessments that have real-world application.  I hope to hear how this project evolves and appreciate the great work!

thanks!

Thanks, Leah! The Engineering Internship units themselves are commercially available as part of the Amplify Science middle school curriculum. The analytics/digital assessment elements are under development as part of this ongoing NSF study, but we'll be taking what we learn and incorporating it into the units for a later version of the curriculum. 

That is a cool project, indeed! Thanks for the tip!

Thanks! I'd say that the transition is a little fuzzy (we don't have a fixed cut off for one or the other), and we've seen some evidence that kids go back into an 'explore' mode at different moments throughout their iterative design process. What we're trying to work out is a sense of when it is particularly important to be exploring, and when it's particularly important to approach the design task systematically--and we expect that this is in part a function of the structure of the Internships, and in part a function of how the student is approaching the problem space.  

thanks!