Understanding allied spatial transformations and mathematical concepts significantly contribute to STEM learning in fields of computer graphics, computer-aided design, computer vision, robotics, quantum mechanics, and more. However, the difficulties students face when learning transformations and their mathematical representations are well documented. While there are mixed views on the impact of geometric visualization in learning linear algebra, and less research with respect to learning and assessment of haptic and proprioceptive sensation-based math learning, based on literature and the team’s preliminary work, our objectives of this research are set to (1) systematically investigate Augmented Reality and AI-powered integrated embodiment and visualization in learning mathematical representations of spatial transformations and (2) create an innovative learning environment for STEM and broader STEAM learning. We will create an AR/AI Classroom and test its effects on learning major rotation/orientation representations in the Euclidean space as test cases: Axis-Angle, Euler Angles, Matrices, and Quaternions, which are representative for different levels of abstraction in allied spatial and math concepts. Workshops, in which physical manipulation takes the forms of playing and transforming LEGO or 3D-printed geometry models, are designed to evaluate the effectiveness of the AR/AI Classroom. The project will employ combined research methods including experiments, quasi-experiments, and statistical analysis. University students’ learning outcomes will be quantitatively analyzed by comparing math skills in pre- and post-workshop tests and qualitatively assessed by the multimodal learning analytics.
Discussion from the 2022 STEM For All Video Showcase (18 posts)
Wei Yan
Lead Presenter Professor of Architecture, Texas A&M University
May 9, 2022 | 11:26 p.m.
Thank you for visiting our “AR-Classroom” in the 2022 STEM for All Video Showcase!
The mathematics describing spatial transformations can be very difficult for college students. While moving something in the physical world may be easy to understand, describing the same operation with math in the digital world can be daunting. Our project AR-Classroom is developing new technology using Augmented Reality (AR) and Artificial Intelligence (AI) to improve the teaching and learning of these difficult concepts in STEM, e.g., 3D rotation sequences: Yaw, Roll, and Pitch, and their math representations: Axis-Angle, Euler Angles, Matrices, and Quaternions. This NSF- and Texas A&M University-funded project, which is in its early stage with developed AR Apps and promising pilot study results, aims to create novel learning tools and advance knowledge across disciplines of spatial and mathematical pedagogies, by exploring: 1. the role of AR to allow the interplay between physical and virtual manipulatives (using LEGO® as examples!) to engage students in embodied learning 2. the capabilities of AR to make difficult invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation 3. the features of AR that help students see relationships between spatial manipulations and mathematical operations
Your comments, questions, and advice will be greatly appreciated!
What a great way to make linear transformations more dynamic and tangible! I was curious, in what ways do you envision your AR materials could be incorporated into a math classroom? Thank you in advance for any insights your team is willing to share!
Wei Yan
Lead Presenter Professor of Architecture, Texas A&M University
May 10, 2022 | 09:31 p.m.
Thanks a lot for your comments and question. The AR materials can become AR Apps that run on teachers’ and students’ mobile phones/tablets or AR glasses in a math, STEM, or STEAM classroom. For an example classroom scenario, see the following lecture video by Dr. Christopher Lum at Boeing:
At time 1 minute 29 seconds, to explain Euler Angles, on top of the lecture video, Dr. Lum tried to draw the reference coordinate-frame and the aircraft body-frame over the physical aircraft. Using the physical aircraft and the virtual frames together was very effective in teaching the complex spatial transformations and math. Isn’t it the very idea of Augmented Reality? Imagine Dr. Lum had AR in his lecture! Imagine each student in the lecture holding an aircraft model, rotating it, and seeing Euler Angles in hand! We hope Dr. Lum, other teachers, and their students could benefit from Augmented Reality and this project AR-Classroom for teaching and learning Euler Angles and other math representations of spatial transformations.
In biochemistry and molecular biology, there are some concepts that benefit from understanding spatial arrangements and relationships. Have you considered expanding these ideas to something like this? Would love to talk about this idea more if it makes sense!
Wei Yan
Lead Presenter Professor of Architecture, Texas A&M University
May 10, 2022 | 11:02 p.m.
Thank you, Daniel!
Spatial transformations and relationships have broad applications in STEM, including the fields you suggested: biochemistry and molecular biology, and also other fields, such as computer graphics, computer-aided design, computer vision, robotics, quantum mechanics, etc. Our project is developing and testing the AR-Classroom technology that takes the advantage of the unique power of integrated embodied learning and registered (aligned model/content) visualization, for spatial transformations and their mathematical representations. In different fields, the learning topics and the physical manipulatives are different: aircraft models for aerospace engineering, scale models for architectural design options, molecular models for understanding molecular structure and bonds, etc. In addition to LEGO models, we also plan to test 3D-printing models in AR-Classroom, to enable various types of geometry and topology of the models. We did think of future applications in teaching and learning physics, and will be excited to consider biochemistry and molecular biology!
Coby Scrudder
Researcher
May 11, 2022 | 05:09 p.m.
Thank you for the comment! I’m one of the student researchers working on this project. I hadn’t considered this for biochemistry and molecular biology applications, but it seens that it would be a great match. Especially for stereochemistry and R/S and D/L notation, it would be useful to be able to see a virtual model of a molecule and see how the molecule changes when you switch between the different stereoisomers and see how diastereomers and enantiomers are formed.
I don’t have much experience with biochemistry beyond what we touched on in my organic chemistry course, so if you have any more ideas, I’d love to hear them!
Lorna Quandt
Facilitator Asst. Professor, Educational Neuroscience
May 10, 2022 | 09:40 p.m.
Hi team--thanks for sharing this video! The use of AR to support this kind of STEM learning is a nice application to see. I was curious, do you have particular hypotheses about which kinds of STEM/engineering concepts will be most amenable to augmented reality transformations? In the example you showed, do the students create their own Lego pieces to use in the activity? It's a great way to add on the layer of AR onto Legos, which are already a good manipulable for learning spatial skills!
Wei Yan
Lead Presenter Professor of Architecture, Texas A&M University
May 11, 2022 | 11:57 a.m.
Thanks Lorna for your great questions!
One of our hypotheses is that AR’s unique power of integrating embodied learning (through students playing with physical objects) and registered visualization (that can align abstract information and virtual models with the physical objects) may enhance understanding of complex spatial transformations, especially rotations, and their math representations, which are important to learn for many STEM concepts: computer graphics (animation to avoid gimbal lock), computer vision (object pose estimation), robotics (navigation), physics (spins of elementary particles), etc.
In our project experiments, students used existing LEGO designs and assembled the physical models. They also built some of the virtual LEGO models using CAD software tools. In an AR class this semester at Texas A&M University, a team of our students designed and built both the virtual and physical LEGO models for AR experiments. We expect that the teachers can guide students to design and build new virtual and physical LEGO models or 3D print models of various geometry and topology, to best represent example models of the STEM concepts. In the future, AR-Classroom can allow new virtual models to be imported for the users to play them with their physical models in the AR learning environment.
Thanks again for your inspiring questions!
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Lorna Quandt
Marcelo Worsley
Facilitator Assistant Professor
May 10, 2022 | 10:58 p.m.
Loved hearing about this work. It's also great to see the ways that this work is improving student learning. I'm wondering if you have had any chance to disaggregate the data and see if the improvements are across the board, or if there are certain students based on race, major, gender identity, etc., that experience particularly large improvements in performance.
Zohreh Shaghaghian
Co-Presenter PhD candidate
May 11, 2022 | 03:53 p.m.
Thanks Marcelo for your interest in this project. That’s a great question indeed. Our user study (after the pilot study mentioned in the video), conducted on 59 undergraduate students, revealed that students in STEM fields had better improvement in their math test compared to their peers in other fields. The difference between gender was negligible. However, our observations and students’ answers of subjective questionnaires showed that even students of non-STEM were very interested in playing with the app and thought that the visual graphics helped them understand the concepts.
The project’s broader impact aims to support many students who struggle with spatial and math reasoning, especially those from underrepresented groups in STEM. Gaining a fundamental understanding of spatial transformations can uniquely contribute to students’ learning and development of spatial reasoning and allied mathematical skills, leading to improved STEM studies.
It is interesting that their math improved but not as much their spatial skills. How many of your students came in with poor spatial skills (i.e., <21 on the PSVT:R)? Research shows that low spatial skills result in lower retention and GPAs in STEM majors. Do you think if you worked to improve student's spatial skills (for the low performing students) that their math skills would improve? Did you see a correlation between their math skills and spatial skills? You should check out my video about the work we are doing in teaching spatial and sketching skills. Love your research.
Zohreh Shaghaghian
Co-Presenter PhD candidate
May 12, 2022 | 10:44 a.m.
Thanks Lelli for your comment and for sharing the video on your research! That's right! Based on studies in literature, students with better spatial skills usually perform better in STEM majors. In our pilot study, we used the short version of PVRT (with 20 questions). Based on the pre-test, only two students had low spatial skills (i.e., <13). But, it is worth mentioning that studies also acknowledge that spatial skill may only improve in multiple sessions and over a long period which was not our case. Hence, we expected that their PVRT score might not improve significantly. However, in this workshop, we were specifically interested if visualizing mathematical graphics in a spatial environment can help students intuitively understand the logic behind the numbers, matrices, and equations. Also, part of our app (which has not been demonstrated here and was not part of this study) is LEGO assembly through AR that may help students' spatial skills (LEGO Assembly through AR).
Dan Roy
Facilitator Research Scientist, Interest-based Learning Mentor, Learning Game Designer
May 12, 2022 | 10:21 a.m.
Great video and tool. I'm curious about embodied learning. Do you think the size of the models impacts that? Students are making small movements with their fingers, hands, and arms. Maybe that's enough, but I wonder if they were moving their whole bodies through larger spaces if that would have any impact on their learning.
As a learning game designer, I see great potential in your tool to be playful, solving spatial puzzles in a game world using the transformations you're trying to teach. Have you considered that direction? I'm actually working on an AR learning game now for learning geometric transformations at the middle/high school level. Maybe we can compare notes sometime.
How big were the learning gains you saw? Did that match your expectations? Any surprises?
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Lorna Quandt
Zohreh Shaghaghian
Co-Presenter PhD candidate
May 13, 2022 | 01:06 p.m.
Thanks Dan for your comment! You alluded to a very valid point. Embodied learning may better help students with whole-body movement. Please also see our second prototype (watch) where we have intended to integrate more embodied learning while playing. Let us know what you think! Solving spatial puzzles in a gaming environment is a very interesting idea that our app could help! Thanks for sharing! I would love to discuss with you more about our research and findings! Later in our user study (after the pilot study mentioned in the video), conducted on 59 students, we found significant learning gain (p<0.05) on students' math skills, specifically on transformation matrices! Still, their PVRT (Purdue Visualization of Rotation Test) score did not improve significantly after the workshop (20-40 minutes of playing with the app). However, we still are researching what specific features of AR helped students in the significant learning gain or whether graphical representations were the major contributor rather than AR environment.
1
Discussion is closed. Upvoting is no longer available
Very interesting research. Thank you for sharing this information. I can see many useful applications of this program for learning mathematics. I look forward to learning more about your project.
Zohreh Shaghaghian
Co-Presenter PhD candidate
May 15, 2022 | 02:01 a.m.
Thanks Leanne for your interest! You can follow some of our projects here: BRICKxAR
This is such a cool project. I love that you used LEGOs. I feel like this could also be applied to geology. It would be awesome to have a structural geology class that incorporated this AR and VR program in relation to how rocks fold and break, especially since a lot of what you have to go off of is not visible on the surface. I'm looking forward to hearing more about your project!
Wei Yan
Lead Presenter Professor of Architecture, Texas A&M University
May 16, 2022 | 10:12 p.m.
Thank you Leah for your comments!
A geology class using AR/VR would be wonderful! Visualizing the invisible rock layers may be related to what we created using AR at the building scale: visualizing the cross section views of building components such as walls, ceilings, and floors, with surrounding mixed physical and virtual environments. Please see: https://www.youtube.com/watch?v=z0aeYb6fG58&t=148s
VR visualization would be great for a classroom and a lab, but AR visualization may assist the fieldwork in geology? We agree that there are a lot of opportunities for research and education.
Wei Yan
Professor of Architecture, Texas A&M University
Thank you for visiting our “AR-Classroom” in the 2022 STEM for All Video Showcase!
The mathematics describing spatial transformations can be very difficult for college students. While moving something in the physical world may be easy to understand, describing the same operation with math in the digital world can be daunting. Our project AR-Classroom is developing new technology using Augmented Reality (AR) and Artificial Intelligence (AI) to improve the teaching and learning of these difficult concepts in STEM, e.g., 3D rotation sequences: Yaw, Roll, and Pitch, and their math representations: Axis-Angle, Euler Angles, Matrices, and Quaternions.
This NSF- and Texas A&M University-funded project, which is in its early stage with developed AR Apps and promising pilot study results, aims to create novel learning tools and advance knowledge across disciplines of spatial and mathematical pedagogies, by exploring:
1. the role of AR to allow the interplay between physical and virtual manipulatives (using LEGO® as examples!) to engage students in embodied learning
2. the capabilities of AR to make difficult invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation
3. the features of AR that help students see relationships between spatial manipulations and mathematical operations
Your comments, questions, and advice will be greatly appreciated!
Project AR-Classroom Team
Texas A&M University
https://sites.google.com/tamu.edu/ar-classroom
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2119549
Zachary Bettersworth
What a great way to make linear transformations more dynamic and tangible! I was curious, in what ways do you envision your AR materials could be incorporated into a math classroom? Thank you in advance for any insights your team is willing to share!
Wei Yan
Professor of Architecture, Texas A&M University
Thanks a lot for your comments and question. The AR materials can become AR Apps that run on teachers’ and students’ mobile phones/tablets or AR glasses in a math, STEM, or STEAM classroom. For an example classroom scenario, see the following lecture video by Dr. Christopher Lum at Boeing:
https://www.youtube.com/embed/GJBc6z6p0KQ?start=0&end=2036
At time 1 minute 29 seconds, to explain Euler Angles, on top of the lecture video, Dr. Lum tried to draw the reference coordinate-frame and the aircraft body-frame over the physical aircraft. Using the physical aircraft and the virtual frames together was very effective in teaching the complex spatial transformations and math. Isn’t it the very idea of Augmented Reality? Imagine Dr. Lum had AR in his lecture! Imagine each student in the lecture holding an aircraft model, rotating it, and seeing Euler Angles in hand! We hope Dr. Lum, other teachers, and their students could benefit from Augmented Reality and this project AR-Classroom for teaching and learning Euler Angles and other math representations of spatial transformations.
Daniel Serrano
This project is so cool.
In biochemistry and molecular biology, there are some concepts that benefit from understanding spatial arrangements and relationships. Have you considered expanding these ideas to something like this? Would love to talk about this idea more if it makes sense!
Wei Yan
Professor of Architecture, Texas A&M University
Thank you, Daniel!
Spatial transformations and relationships have broad applications in STEM, including the fields you suggested: biochemistry and molecular biology, and also other fields, such as computer graphics, computer-aided design, computer vision, robotics, quantum mechanics, etc. Our project is developing and testing the AR-Classroom technology that takes the advantage of the unique power of integrated embodied learning and registered (aligned model/content) visualization, for spatial transformations and their mathematical representations. In different fields, the learning topics and the physical manipulatives are different: aircraft models for aerospace engineering, scale models for architectural design options, molecular models for understanding molecular structure and bonds, etc. In addition to LEGO models, we also plan to test 3D-printing models in AR-Classroom, to enable various types of geometry and topology of the models. We did think of future applications in teaching and learning physics, and will be excited to consider biochemistry and molecular biology!
Coby Scrudder
Thank you for the comment! I’m one of the student researchers working on this project. I hadn’t considered this for biochemistry and molecular biology applications, but it seens that it would be a great match. Especially for stereochemistry and R/S and D/L notation, it would be useful to be able to see a virtual model of a molecule and see how the molecule changes when you switch between the different stereoisomers and see how diastereomers and enantiomers are formed.
I don’t have much experience with biochemistry beyond what we touched on in my organic chemistry course, so if you have any more ideas, I’d love to hear them!
Lorna Quandt
Asst. Professor, Educational Neuroscience
Hi team--thanks for sharing this video! The use of AR to support this kind of STEM learning is a nice application to see. I was curious, do you have particular hypotheses about which kinds of STEM/engineering concepts will be most amenable to augmented reality transformations? In the example you showed, do the students create their own Lego pieces to use in the activity? It's a great way to add on the layer of AR onto Legos, which are already a good manipulable for learning spatial skills!
Wei Yan
Professor of Architecture, Texas A&M University
Thanks Lorna for your great questions!
One of our hypotheses is that AR’s unique power of integrating embodied learning (through students playing with physical objects) and registered visualization (that can align abstract information and virtual models with the physical objects) may enhance understanding of complex spatial transformations, especially rotations, and their math representations, which are important to learn for many STEM concepts: computer graphics (animation to avoid gimbal lock), computer vision (object pose estimation), robotics (navigation), physics (spins of elementary particles), etc.
In our project experiments, students used existing LEGO designs and assembled the physical models. They also built some of the virtual LEGO models using CAD software tools. In an AR class this semester at Texas A&M University, a team of our students designed and built both the virtual and physical LEGO models for AR experiments. We expect that the teachers can guide students to design and build new virtual and physical LEGO models or 3D print models of various geometry and topology, to best represent example models of the STEM concepts. In the future, AR-Classroom can allow new virtual models to be imported for the users to play them with their physical models in the AR learning environment.
Thanks again for your inspiring questions!
Lorna Quandt
Marcelo Worsley
Assistant Professor
Loved hearing about this work. It's also great to see the ways that this work is improving student learning. I'm wondering if you have had any chance to disaggregate the data and see if the improvements are across the board, or if there are certain students based on race, major, gender identity, etc., that experience particularly large improvements in performance.
Zohreh Shaghaghian
PhD candidate
Thanks Marcelo for your interest in this project. That’s a great question indeed. Our user study (after the pilot study mentioned in the video), conducted on 59 undergraduate students, revealed that students in STEM fields had better improvement in their math test compared to their peers in other fields. The difference between gender was negligible. However, our observations and students’ answers of subjective questionnaires showed that even students of non-STEM were very interested in playing with the app and thought that the visual graphics helped them understand the concepts.
The project’s broader impact aims to support many students who struggle with spatial and math reasoning, especially those from underrepresented groups in STEM. Gaining a fundamental understanding of spatial transformations can uniquely contribute to students’ learning and development of spatial reasoning and allied mathematical skills, leading to improved STEM studies.
Lelli Van Den Einde
It is interesting that their math improved but not as much their spatial skills. How many of your students came in with poor spatial skills (i.e., <21 on the PSVT:R)? Research shows that low spatial skills result in lower retention and GPAs in STEM majors. Do you think if you worked to improve student's spatial skills (for the low performing students) that their math skills would improve? Did you see a correlation between their math skills and spatial skills? You should check out my video about the work we are doing in teaching spatial and sketching skills. Love your research.
Zohreh Shaghaghian
PhD candidate
Thanks Lelli for your comment and for sharing the video on your research!
That's right! Based on studies in literature, students with better spatial skills usually perform better in STEM majors. In our pilot study, we used the short version of PVRT (with 20 questions). Based on the pre-test, only two students had low spatial skills (i.e., <13). But, it is worth mentioning that studies also acknowledge that spatial skill may only improve in multiple sessions and over a long period which was not our case. Hence, we expected that their PVRT score might not improve significantly. However, in this workshop, we were specifically interested if visualizing mathematical graphics in a spatial environment can help students intuitively understand the logic behind the numbers, matrices, and equations.
Also, part of our app (which has not been demonstrated here and was not part of this study) is LEGO assembly through AR that may help students' spatial skills (LEGO Assembly through AR).
Dan Roy
Research Scientist, Interest-based Learning Mentor, Learning Game Designer
Great video and tool. I'm curious about embodied learning. Do you think the size of the models impacts that? Students are making small movements with their fingers, hands, and arms. Maybe that's enough, but I wonder if they were moving their whole bodies through larger spaces if that would have any impact on their learning.
As a learning game designer, I see great potential in your tool to be playful, solving spatial puzzles in a game world using the transformations you're trying to teach. Have you considered that direction? I'm actually working on an AR learning game now for learning geometric transformations at the middle/high school level. Maybe we can compare notes sometime.
How big were the learning gains you saw? Did that match your expectations? Any surprises?
Lorna Quandt
Zohreh Shaghaghian
PhD candidate
Thanks Dan for your comment! You alluded to a very valid point. Embodied learning may better help students with whole-body movement. Please also see our second prototype (watch) where we have intended to integrate more embodied learning while playing. Let us know what you think!
Solving spatial puzzles in a gaming environment is a very interesting idea that our app could help! Thanks for sharing! I would love to discuss with you more about our research and findings!
Later in our user study (after the pilot study mentioned in the video), conducted on 59 students, we found significant learning gain (p<0.05) on students' math skills, specifically on transformation matrices! Still, their PVRT (Purdue Visualization of Rotation Test) score did not improve significantly after the workshop (20-40 minutes of playing with the app).
However, we still are researching what specific features of AR helped students in the significant learning gain or whether graphical representations were the major contributor rather than AR environment.
Dan Roy
Leanne Ketterlin Geller
Very interesting research. Thank you for sharing this information. I can see many useful applications of this program for learning mathematics. I look forward to learning more about your project.
Zohreh Shaghaghian
PhD candidate
Thanks Leanne for your interest! You can follow some of our projects here: BRICKxAR
Leah Wiitablake
This is such a cool project. I love that you used LEGOs. I feel like this could also be applied to geology. It would be awesome to have a structural geology class that incorporated this AR and VR program in relation to how rocks fold and break, especially since a lot of what you have to go off of is not visible on the surface. I'm looking forward to hearing more about your project!
Wei Yan
Professor of Architecture, Texas A&M University
Thank you Leah for your comments!
A geology class using AR/VR would be wonderful! Visualizing the invisible rock layers may be related to what we created using AR at the building scale: visualizing the cross section views of building components such as walls, ceilings, and floors, with surrounding mixed physical and virtual environments. Please see: https://www.youtube.com/watch?v=z0aeYb6fG58&t=148s
and Figure 14 and 16 of https://arxiv.org/pdf/2204.03207.pdf
VR visualization would be great for a classroom and a lab, but AR visualization may assist the fieldwork in geology? We agree that there are a lot of opportunities for research and education.
Further posting is closed as the event has ended.