Thanks Cyrus! Good questions. Before we started building the simulations we did a bunch of interviews with students to get a sense of the types of gestures that they perform naturally when reasoning quantitatively. For example, when students were describing multiplicative growth, like successive doubling, they often used a “stacking” gesture where they were essentially taking a quantity and folding it onto itself. Based on those observations, we frequently encourage students to scale up the quantity they are working with (e.g., the magnitude of an Earthquake) by using a folding gesture. But the cool thing about the gesture recognition system our team has built is that it’s highly customizable, so really any gesture that can be recognized by the Kinect could be used for multiplication. The biggest constraint is just making sure the gestures we use for different operations are distinct, which is one of the reasons that we’ve prompted students to use leg gestures like kicking to represent a decrease in quantity. Kicking wasn’t a gesture that we saw many kids do spontaneously in our initial interviews (they were seated, after all), but the metaphor of kicking away a block off of a stack of blocks seemed to resonate with the students.

We have not targeted symmetry/chirality in this project, but it’s a good idea. Obviously this is relevant for learning in chemistry, but do you have ideas for other STEM topics where these concepts are at play? I know that Mitch Nathan at UW Madison is doing some cool embodied tech for doing geometric proofs, but if these specific concepts are hit upon in their work. So far we have mostly focused on scale, and right now we are building sims that examine rates of change in a few different science content areas.

Thanks for your reply. It definitely helped me understand your research.

As for other areas, there are some aspects of physics and electromagnetism that involve chirality, such as the direction that a current will flow (see: this page) that I feel have enormous potential as they are not obvious to many new students of physics.

Thanks so much for sharing your work.

-Cyrus

Hey Jeremy, thanks for the question. Yes indeed, gesture is ubiquitous and serves many purposes. On another project, GRASP, my team focuses specifically on the role that gesture plays in constructing explanations of science phenomena. On that project we try to provide visualizations that augment the hand gestures students make as a way of getting them to hone in on causal mechanisms.

On the ELASTICS project I think the way we are using gesture is a little different—we’re trying to provide a platform for students to get a physical sense of STEM ideas that don’t have readily available physical corollaries. So, for example, when dealing with really big or really small quantities, our bodies are pretty limited in directly representing those quantities—our arms only go out so far and our fingers can only be perceived to be so close. But we can embody the *operations* that can get us to those quantities fairly quickly. In ELASTICS a student can compare 1 to 1 million with just six gestures representing “times 10.” The important thing, we think, is that whatever that gesture is has a “times 10” meaning to the student, which we think we are accomplishing by building a metaphor around the gesture, and by contrasting it with gestures that do other things, like adding 1 unit at a time.

This answer doesn’t come close to addressing all the interactional uses of gesture, but we think that creating a set of “embodied primitives” that can be used for engaging across science topics is fairly novel, and hopefully strengthens the transfer potential of integrating gesture in STEM education.

Hey Martin, this is exactly what we're setting out to find--thanks for articulating it better than I have been. ;) I think we have good evidence so far for part of this argument. We're seeing improvement on students' general reasoning about scale, particularly differences between linear and non-linear scales after using a gesture augmented ELASTICS sim that focuses on on one topic (e.g., Earthquakes). What we are collecting data on right now is whether or not prior use--about 2 weeks previously--of the Earthquake sim improves performance and overall quant reasoning for people using a second sim (acids and bases, pH scale). In other words, is this general quantitative reasoning capacity that was facilitated with gestures initially transferable to a new domain. Analysis of pilot data looked promising--we'll keep you posted on findings. Would be fun to talk more with you about the kinds of environments you think gestural interactions for seeding transfer could be tried and tested.