Great questions, and thanks so much for the interest!

Your questions about the hands-on experiences are a great place to start because that is how our curricula start. We have hands-on activities that help make the science and engineering concepts explicit before the students see them applied in the video game. For example, we have students consider how to work on the bottom of a large ship before they see the Dry Dock interface in the game. And, we have a lesson in stability before students' ships are put through the Stability ship. We found (qualitatively) that students were playing the game instead of applying STEM concepts through the game, so we solved this initial problem by frontloading the curriculum with STEM activities that relate to their game play.

Our two off-line competitions eventually have an online component so that students can compare their work nationwide. Our first offline competition involves designing boats using recyclables for different purposes (e.g., strongest ship, tallest ship). So far, these events have been live competitions. This summer, we will ask students to upload pictures of their ship and documentation of their tests to our password-protected FLEET forum. Our second contest brings students inside the video-game design process by asking them to select or create 3D objects. Students submit a full explanation with each object.

Your research question is a top-of-mind issue for us; one that we could use some community support. Our program has an end-of-year survey that gauges STEM interest and game feedback. The program is primed for an intervention study like you hint at, but it’s not something that we have assembled with our community yet. Given that everything in our program is free, we would love to partner with a university or district to create a pre-post study for academic growth.

Hi Michael,

My colleagues and I at the University of Akron are interested in discussing collaborating or partnering on a pre-post study. We are in the final steps of establishing an Urban STEM teaching, research, and evaluation center. This research fits well with our mission. Our research team consists of experts in evaluation/research design, educational psychology, science curriculum and instruction, and others! We are interested in collaborating on the research or in serving in the evaluation capacity.

Please, contact me if you would like to discuss a collaboration!

Kristin Koskey - koskey@uakron.edu

We will definitely email you directly later today. Thank you!!

We are excited to partner with researchers to investigate different aspects of the program.

(This is an open invitation to other researchers.)

Great questions, and thanks so much for the interest!

I think your first question speaks to an issue that we have seen too. We have found it takes conscious effort to emphasize the science in the presence of an engaging video game. The lessons are set up with verbal cues to introduce concepts in hands-on learning and then use the same language in team-based game play. Since students play the game in teams, we find that they discuss using this language throughout the class day. Unfortunately, we do not have research that captures whether these theoretical academic gains are truly gained and maintained.

Our outcome measure asks students about their attitudes toward STEM and possible STEM future selves. It also asks historical information about their STEM activities and their experience with FLEET. Our intended outcome is addressing ONR’s need for more naval engineers, which our measure does address fairly fully. Now, that our program has matured, I think it is becoming relevant to measure academic gains. This research is not planned or funded currently, but we would love to work with others to do this study.

First, thanks so much for the insightful comments.

Our lessons are “setting up” the game play, but I would love to work with you and a group of students to try the materials in the reverse order. Our lessons are highly editable, so it would be easy for us to reorder the activities as you describe.

Our initial issue was noticing that students were making incremental improvement in the game, but the students were not using science knowledge that they learned in a traditional classroom environment. Our solution was to structure the lessons as you described; however, this idea may be a better solution. Please consider downloading the game and/or applying for the implementation grant. We would love to explore this concept more completely.

Thanks so much for sharing!

We agree with your summary of the research, and saw some validation in our student interactions. When students played the game first, the language is video-game-centric (e.g., "points", "it won't let me", "my ship is good"). When students interacted with science activities first, their language involved more science concepts (e.g., "speed", "design improved", "we tested"). That said, our sample sizes are still relatively small and do not encompass all possible learning environments. We would be excited to work with anyone that has a learning environment tailored to a video-game-first implementation.  

Very thoughtful questions!

From a STEM perspective, Missions initially challenge young engineers to work on problems that take 10-20 minutes to solve. As they iterate on their solutions, those times drop down to 6-7 minutes for elite teams. As you rightly guessed, these solutions relate to STEM careers and occupational duties. Students simulate search and rescue missions, or design ships for working with autonomous underwater vehicles, and get to try on these duties as possible future selves.

From a gameplay perspective, missions are capstone projects that provide additional limitations (including budgets) to their ships. Then, they are able to participate in open-water exercises for long periods of time. Students use shorter tests to see how design choices affect their ship, and then fully test their designs in missions.

We have integrated scaffolding into are program through two design choices. First, we encourage participating to use engineering teams rather than individual players.  By working together, students support each other as they work with these science and engineering concepts. Second, the aforementioned science activities give students hands-on experience with the concepts. This continually growing set of experiences is a resource for them to creatively use as they consider and test different solutions.

Thanks again for the thoughtful questions, and please feel free to follow up with more!

Our grant focused on high school participation, but we have a mix of middle and high school partners. The heart of the STEM content is the engineering design process and topics related to forces and motion. In the NGSS, we cite alignment to MS-ETS1, MS-PS2, MS-PS3, HS-PS2, HS-PS3, and HS-ETS1.

The related curriculum is standards-aligned so we go into greater depth with forces and motion in the high school curriculum. We also structure the lessons to use an existing six-step engineering process with middle school students. However, the high school curriculum starts with two different engineering design processes (one from Massachusetts DOE and the other from NASA). Then students work together to design the engineering process they will use this school year.

I also wanted to mention that the activities require students to present findings and ideas, use video resources and informational texts. We align these lessons to the appropriate Informational Text and Speaking & Listening standards. These multiple alignments help FLEET fit within different classes in a standard school day.

And, we are pursuing partnerships to build aligned math resources for FLEET.

Thanks so much for the excellent questions!

Pleased to virtually meet you! ASNE has hundreds of members in your area as you can imagine. I am going to be in your area Friday meeting with a couple schools and community organizations. I would be happy to stop by, just email me at mbriscoe@navalengineers.org.

As for your questions, we fund schools but work directly with teachers. Each district is different, so we have created district-wide partnerships in some areas, and worked with individual educators in other. Right now, about 25% of our schools are integrating FLEET, while 75% are introducing FLEET as an OST activity. Since our program is completely free, we often are the “off-season” programming for Robotics Clubs, and we also work with STEM and Video Game clubs.

I always emphasize that our education resources are highly editable. We want educators to pick and choose what will work best for their situation. Looking ahead to next school year, we will be sharing implementation lessons learned from schools integrating FLEET into the school day since that pathway has more challenges.

Please reach out if you want to participate more, or keep an eye on our website (fleetengineering.org) as we share new resources and ideas.

Thanks so much for this feedback! 

Your question about the best way to facilitate learning making experiences between learning environments is excellent. We have not fully addressed it yet, but we have some insights from our iterative curriculum design effort. Qualitatively, we are finding better understanding by using similar vocabulary between the hands-on and virtual experiences. And, then repeat this language over multiple days so students have time to practice with the terms in context. For example, in the first lesson students learn about an engineering process and then work together to design their own engineering process (or use the one presented). Students make artifacts documenting this process, which are hung in the space. Then, we come back to this artifact and use the same terminology as students use the iterative process to think, prototype, test, analyze, and reconsider their designs. 

Those initial findings could certainly be bolstered by more research. It's been a theme in these comments and one takeaway that we are going to pursue in the months and years ahead.

Thanks so much for your valuable input!