2530 Views (as of 05/2023)
  1. Wei Yan
  2. http://bim-sim.org/wyan/
  3. Mattia Flabiano III AIA/ Page Southerland Page Design Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. Texas A&M University
  1. Ziad Ashour
  2. Ph.D. Candidate
  3. Presenter’s NSFRESOURCECENTERS
  4. Texas A&M University
  1. Zohreh Shaghaghian
  2. https://www.linkedin.com/in/zohreh-shahrzad-shaghaghian-5994783a/
  3. PhD candidate
  4. Presenter’s NSFRESOURCECENTERS
  5. Texas A&M University
  1. Preston Tranbarger
  2. Undergraduate Student
  3. Presenter’s NSFRESOURCECENTERS
  4. Texas A&M University
  1. Philip Yasskin
  2. https://www.math.tamu.edu/~yasskin/
  3. Associate Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. Texas A&M University
  1. Shu-Hao Yeh
  2. Ph.D candidate
  3. Presenter’s NSFRESOURCECENTERS
  4. Texas A&M University

Using Augmented Reality and Artificial Intelligence to Improve Teaching and L...

NSF Awards: 2119549

2022 (see original presentation & discussion)

Undergraduate

Understanding allied spatial transformations and mathematical concepts significantly contribute to STEM learning in fields of computer graphics, computer-aided design, computer vision, robotics, quantum mechanics, and more. However, the difficulties students face when learning transformations and their mathematical representations are well documented. While there are mixed views on the impact of geometric visualization in learning linear algebra, and less research with respect to learning and assessment of haptic and proprioceptive sensation-based math learning, based on literature and the team’s preliminary work, our objectives of this research are set to (1) systematically investigate Augmented Reality and AI-powered integrated embodiment and visualization in learning mathematical representations of spatial transformations and (2) create an innovative learning environment for STEM and broader STEAM learning. We will create an AR/AI Classroom and test its effects on learning major rotation/orientation representations in the Euclidean space as test cases: Axis-Angle, Euler Angles, Matrices, and Quaternions, which are representative for different levels of abstraction in allied spatial and math concepts. Workshops, in which physical manipulation takes the forms of playing and transforming LEGO or 3D-printed geometry models, are designed to evaluate the effectiveness of the AR/AI Classroom. The project will employ combined research methods including experiments, quasi-experiments, and statistical analysis. University students’ learning outcomes will be quantitatively analyzed by comparing math skills in pre- and post-workshop tests and qualitatively assessed by the multimodal learning analytics.

This video has had approximately 511 visits by 355 visitors from 160 unique locations. It has been played 229 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2022 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2022 STEM For All Video Showcase (18 posts)
  • Icon for: Wei Yan

    Wei Yan

    Lead Presenter
    Professor of Architecture, Texas A&M University
    May 9, 2022 | 11:26 p.m.

    Thank you for visiting our “AR-Classroom” in the 2022 STEM for All Video Showcase!

    The mathematics describing spatial transformations can be very difficult for college students. While moving something in the physical world may be easy to understand, describing the same operation with math in the digital world can be daunting. Our project AR-Classroom is developing new technology using Augmented Reality (AR) and Artificial Intelligence (AI) to improve the teaching and learning of these difficult concepts in STEM, e.g., 3D rotation sequences: Yaw, Roll, and Pitch, and their math representations: Axis-Angle, Euler Angles, Matrices, and Quaternions.
    This NSF- and Texas A&M University-funded project, which is in its early stage with developed AR Apps and promising pilot study results, aims to create novel learning tools and advance knowledge across disciplines of spatial and mathematical pedagogies, by exploring:
          1. the role of AR to allow the interplay between physical and virtual manipulatives (using LEGO® as examples!) to engage students in embodied learning
          2. the capabilities of AR to make difficult invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation
          3. the features of AR that help students see relationships between spatial manipulations and mathematical operations


    Your comments, questions, and advice will be greatly appreciated!

    Project AR-Classroom Team
    Texas A&M University
    https://sites.google.com/tamu.edu/ar-classroom
    https://www.nsf.gov/awardsearch/showAward?AWD_ID=2119549

  • Icon for: Zachary Bettersworth

    Zachary Bettersworth

    Graduate Student
    May 10, 2022 | 12:49 p.m.

    What a great way to make linear transformations more dynamic and tangible! I was curious, in what ways do you envision your AR materials could be incorporated into a math classroom? Thank you in advance for any insights your team is willing to share!

  • Icon for: Wei Yan

    Wei Yan

    Lead Presenter
    Professor of Architecture, Texas A&M University
    May 10, 2022 | 09:31 p.m.

    Thanks a lot for your comments and question. The AR materials can become AR Apps that run on teachers’ and students’ mobile phones/tablets or AR glasses in a math, STEM, or STEAM classroom. For an example classroom scenario, see the following lecture video by Dr. Christopher Lum at Boeing:


    https://www.youtube.com/embed/GJBc6z6p0KQ?start=0&end=2036


    At time 1 minute 29 seconds, to explain Euler Angles, on top of the lecture video, Dr. Lum tried to draw the reference coordinate-frame and the aircraft body-frame over the physical aircraft. Using the physical aircraft and the virtual frames together was very effective in teaching the complex spatial transformations and math. Isn’t it the very idea of Augmented Reality? Imagine Dr. Lum had AR in his lecture! Imagine each student in the lecture holding an aircraft model, rotating it, and seeing Euler Angles in hand! We hope Dr. Lum, other teachers, and their students could benefit from Augmented Reality and this project AR-Classroom for teaching and learning Euler Angles and other math representations of spatial transformations.

  • Icon for: Daniel Serrano

    Daniel Serrano

    K-12 Administrator
    May 10, 2022 | 03:24 p.m.

    This project is so cool.

    In biochemistry and molecular biology, there are some concepts that benefit from understanding spatial arrangements and relationships. Have you considered expanding these ideas to something like this? Would love to talk about this idea more if it makes sense!

  • Icon for: Wei Yan

    Wei Yan

    Lead Presenter
    Professor of Architecture, Texas A&M University
    May 10, 2022 | 11:02 p.m.

    Thank you, Daniel!


    Spatial transformations and relationships have broad applications in STEM, including the fields you suggested: biochemistry and molecular biology, and also other fields, such as computer graphics, computer-aided design, computer vision, robotics, quantum mechanics, etc. Our project is developing and testing the AR-Classroom technology that takes the advantage of the unique power of integrated embodied learning and registered (aligned model/content) visualization, for spatial transformations and their mathematical representations. In different fields, the learning topics and the physical manipulatives are different: aircraft models for aerospace engineering, scale models for architectural design options, molecular models for understanding molecular structure and bonds, etc. In addition to LEGO models, we also plan to test 3D-printing models in AR-Classroom, to enable various types of geometry and topology of the models. We did think of future applications in teaching and learning physics, and will be excited to consider biochemistry and molecular biology!

  • Icon for: Coby Scrudder

    Coby Scrudder

    Researcher
    May 11, 2022 | 05:09 p.m.

    Thank you for the comment! I’m one of the student researchers working on this project. I hadn’t considered this for biochemistry and molecular biology applications, but it seens that it would be a great match. Especially for stereochemistry and R/S and D/L notation, it would be useful to be able to see a virtual model of a molecule and see how the molecule changes when you switch between the different stereoisomers and see how diastereomers and enantiomers are formed.

    I don’t have much experience with biochemistry beyond what we touched on in my organic chemistry course, so if you have any more ideas, I’d love to hear them!

  • Icon for: Lorna Quandt

    Lorna Quandt

    Facilitator
    Asst. Professor, Educational Neuroscience
    May 10, 2022 | 09:40 p.m.

    Hi team--thanks for sharing this video! The use of AR to support this kind of STEM learning is a nice application to see. I was curious, do you have particular hypotheses about which kinds of STEM/engineering concepts will be most amenable to augmented reality transformations? In the example you showed, do the students create their own Lego pieces to use in the activity? It's a great way to add on the layer of AR onto Legos, which are already a good manipulable for learning spatial skills! 

  • Icon for: Wei Yan

    Wei Yan

    Lead Presenter
    Professor of Architecture, Texas A&M University
    May 11, 2022 | 11:57 a.m.

    Thanks Lorna for your great questions!


    One of our hypotheses is that AR’s unique power of integrating embodied learning (through students playing with physical objects) and registered visualization (that can align abstract information and virtual models with the physical objects) may enhance understanding of complex spatial transformations, especially rotations, and their math representations, which are important to learn for many STEM concepts: computer graphics (animation to avoid gimbal lock), computer vision (object pose estimation), robotics (navigation), physics (spins of elementary particles), etc.


    In our project experiments, students used existing LEGO designs and assembled the physical models. They also built some of the virtual LEGO models using CAD software tools. In an AR class this semester at Texas A&M University, a team of our students designed and built both the virtual and physical LEGO models for AR experiments. We expect that the teachers can guide students to design and build new virtual and physical LEGO models or 3D print models of various geometry and topology, to best represent example models of the STEM concepts. In the future, AR-Classroom can allow new virtual models to be imported for the users to play them with their physical models in the AR learning environment.


    Thanks again for your inspiring questions!

     
    1
    Discussion is closed. Upvoting is no longer available

    Lorna Quandt
  • Icon for: Marcelo Worsley

    Marcelo Worsley

    Facilitator
    Assistant Professor
    May 10, 2022 | 10:58 p.m.

    Loved hearing about this work. It's also great to see the ways that this work is improving student learning. I'm wondering if you have had any chance to disaggregate the data and see if the improvements are across the board, or if there are certain students based on race, major, gender identity, etc., that experience particularly large improvements in performance.

  • Icon for: Zohreh Shaghaghian

    Zohreh Shaghaghian

    Co-Presenter
    PhD candidate
    May 11, 2022 | 03:53 p.m.

    Thanks Marcelo for your interest in this project. That’s a great question indeed. Our user study (after the pilot study mentioned in the video), conducted on 59 undergraduate students, revealed that students in STEM fields had better improvement in their math test compared to their peers in other fields. The difference between gender was negligible. However, our observations and students’ answers of subjective questionnaires showed that even students of non-STEM were very interested in playing with the app and thought that the visual graphics helped them understand the concepts.

    The project’s broader impact aims to support many students who struggle with spatial and math reasoning, especially those from underrepresented groups in STEM. Gaining a fundamental understanding of spatial transformations can uniquely contribute to students’ learning and development of spatial reasoning and allied mathematical skills, leading to improved STEM studies.

  • Icon for: Lelli Van Den Einde

    Lelli Van Den Einde

    Higher Ed Faculty
    May 11, 2022 | 04:29 p.m.

    It is interesting that their math improved but not as much their spatial skills. How many of your students came in with poor spatial skills (i.e., <21 on the PSVT:R)? Research shows that low spatial skills result in lower retention and GPAs in STEM majors. Do you think if you worked to improve student's spatial skills (for the low performing students) that their math skills would improve? Did you see a correlation between their math skills and spatial skills? You should check out my video about the work we are doing in teaching spatial and sketching skills. Love your research.

  • Icon for: Zohreh Shaghaghian

    Zohreh Shaghaghian

    Co-Presenter
    PhD candidate
    May 12, 2022 | 10:44 a.m.

    Thanks Lelli for your comment and for sharing the video on your research!
    That's right! Based on studies in literature, students with better spatial skills usually perform better in STEM majors. In our pilot study, we used the short version of PVRT (with 20 questions). Based on the pre-test, only two students had low spatial skills (i.e., <13). But, it is worth mentioning that studies also acknowledge that spatial skill may only improve in multiple sessions and over a long period which was not our case. Hence, we expected that their PVRT score might not improve significantly. However, in this workshop, we were specifically interested if visualizing mathematical graphics in a spatial environment can help students intuitively understand the logic behind the numbers, matrices, and equations.
    Also, part of our app (which has not been demonstrated here and was not part of this study) is LEGO assembly through AR that may help students' spatial skills (LEGO Assembly through AR).

  • Icon for: Dan Roy

    Dan Roy

    Facilitator
    Research Scientist, Interest-based Learning Mentor, Learning Game Designer
    May 12, 2022 | 10:21 a.m.

    Great video and tool. I'm curious about embodied learning. Do you think the size of the models impacts that? Students are making small movements with their fingers, hands, and arms. Maybe that's enough, but I wonder if they were moving their whole bodies through larger spaces if that would have any impact on their learning.

    As a learning game designer, I see great potential in your tool to be playful, solving spatial puzzles in a game world using the transformations you're trying to teach. Have you considered that direction? I'm actually working on an AR learning game now for learning geometric transformations at the middle/high school level. Maybe we can compare notes sometime.

    How big were the learning gains you saw? Did that match your expectations? Any surprises?

     
    1
    Discussion is closed. Upvoting is no longer available

    Lorna Quandt
  • Icon for: Zohreh Shaghaghian

    Zohreh Shaghaghian

    Co-Presenter
    PhD candidate
    May 13, 2022 | 01:06 p.m.

    Thanks Dan for your comment! You alluded to a very valid point. Embodied learning may better help students with whole-body movement. Please also see our second prototype (watch) where we have intended to integrate more embodied learning while playing. Let us know what you think! 
    Solving spatial puzzles in a gaming environment is a very interesting idea that our app could help! Thanks for sharing! I would love to discuss with you more about our research and findings! 
    Later in our user study (after the pilot study mentioned in the video), conducted on 59 students, we found significant learning gain (p<0.05) on students' math skills, specifically on transformation matrices! Still, their PVRT (Purdue Visualization of Rotation Test) score did not improve significantly after the workshop (20-40 minutes of playing with the app).
    However, we still are researching what specific features of AR helped students in the significant learning gain or whether graphical representations were the major contributor rather than AR environment. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Dan Roy
  • Icon for: Leanne Ketterlin Geller

    Leanne Ketterlin Geller

    Higher Ed Faculty
    May 14, 2022 | 09:39 p.m.

    Very interesting research. Thank you for sharing this information. I can see many useful applications of this program for learning mathematics. I look forward to learning more about your project. 

  • Icon for: Zohreh Shaghaghian

    Zohreh Shaghaghian

    Co-Presenter
    PhD candidate
    May 15, 2022 | 02:01 a.m.

    Thanks Leanne for your interest! You can follow some of our projects here: BRICKxAR

  • Icon for: Leah Wiitablake

    Leah Wiitablake

    Graduate Student
    May 16, 2022 | 06:03 p.m.

    This is such a cool project. I love that you used LEGOs. I feel like this could also be applied to geology. It would be awesome to have a structural geology class that incorporated this AR and VR program in relation to how rocks fold and break, especially since a lot of what you have to go off of is not visible on the surface. I'm looking forward to hearing more about your project!

  • Icon for: Wei Yan

    Wei Yan

    Lead Presenter
    Professor of Architecture, Texas A&M University
    May 16, 2022 | 10:12 p.m.

    Thank you Leah for your comments!


    A geology class using AR/VR would be wonderful! Visualizing the invisible rock layers may be related to what we created using AR at the building scale: visualizing the cross section views of building components such as walls, ceilings, and floors, with surrounding mixed physical and virtual environments. Please see: https://www.youtube.com/watch?v=z0aeYb6fG58&t=148s


    and Figure 14 and 16 of https://arxiv.org/pdf/2204.03207.pdf


    VR visualization would be great for a classroom and a lab, but AR visualization may assist the fieldwork in geology? We agree that there are a lot of opportunities for research and education.

  • Further posting is closed as the event has ended.