Welcome, visitors! Thank you for watching our video about the SAIL project at Gallaudet University. In the past year we have made major strides toward our goal, which is to create a proof-of-concept ASL learning experience in virtual reality.

We have completed the motion capture recordings of ASL content, created 3D signing avatars from the recordings and we've built interactive lessons of introductory ASL content. Next up, we will conduct an EEG cognitive neuroscience experiment that will help us see how the "embodied learning" aspect of SAIL influences ASL learning. Importantly, this work is conducted by a deaf-led team, and we are committed to holding deaf talent at the center of everything we are doing.

Through this work, we see great potential for an entirely new way option for learning ASL in the future--from a native ASL user in the comfort of your own home! We welcome questions and comments on any aspect of this project. And again, thank you for your interest. 

Very interesting work!
I can see this being particularly useful now that hand tracking technologies are becoming more affordable and prevalent (i.e., Oculus Quest). Do you have any thoughts / plans on using this technology in a non-headset context? I can imagine that the VR headset is more immersive and effective, but wonder about programs that may not have access to such devices.

Thank you for the response!

I'm excited to share this with some of my colleagues in our ASL program.

Hi! This is super interesting and amazing especially right now with issues of social distancing. The recent switch to all online classes has made my ASL professors and other interpreting majors realize how difficult it is to learn this language 2 dimentionally and in a non-interactive setting. My question is, is the signer's signs being perceived by the headset itself? And if so how close to the body can it perceive signs. For example if you are signing PLEASE or CURIOUS would it be able to see those signs?

Your team has done a really amazing job combining VR and embodiment for teaching and learning. I can see from the above that you are considering many future pathways. I wonder most about the technology demands for full motion capture as you consider how to scale this up for a broad user base. As a second question, why go with full VR versus AR that can allow the world around people to be in the image? I am truly asking since I don't yet have a clear answer and see many trade offs. This is something my colleagues and I are also pondering as we consider how to support embodied collaboration among learners and teachers.

This is absolutely amazing! Firstly, that this is a deaf-led team is fantastic, and I'm glad to see these innovations driven by the people who understand these kinds of needs the most. Secondly, this idea is something I really connect with! My partner and I were talking yesterday about how they want to learn ASL but our University doesn't have classes. While I suggested some places for online learning, we did talk about the difficulty of learning such a three-dimensional language in an online setting, especially with covid19. Using motion capture technology toward this vision is fantastic and I hope to see continued application of this in the future! 

Great video and project, and so important at this time especially when in-person learning isn't an option. I'm curious if you have any thoughts about how motion capture/VR/avatars like yours could extend beyond teaching ASL. I could see this technology being harnessed to teach all sorts of things virtually now. Maybe a virtual science lab, where students could manipulate virtual chemicals or other virtual materials safely from their own homes?

Great project. Thanks for sharing.

This is an awesome project. Thanks for working on this. How are students working on your team recruited? Is it a summer project or do you work on this throughout the year? Do you have industry partners yet? 

This was really fascinating to watch for someone who knows neither ASL nor VR technologies ... Now I want to learn both! I wondered if you've interacted at all with Nick Giudice or Rich Corey at the VEMI lab at UMaine? Might be interesting to compare notes. Meanwhile, I'm interested in the questions above about the trade-offs around the investment needed for VR. That is an earnest project design struggle we all struggle with no matter what level of technology. Thanks for the interesting work & video.

Greetings! This looks like a very intense project, and you've made great strides! I really like how you modified the helmet to make it more usable and accessible for ASL.

I'm wondering, do you think you will be able to use this technology to assess how well the students are learning the language? Will students be able to film themselves signing and put it side-by-side with the avatars, or even be able to use tech like the new gaming consoles use (or the updated gloves you use to make the avatars) to judge their movements for accuracy? I'd love to be able to take a class this way, and then get scores based on the accuracy of my signing!

This is a truly impressive project! Your video was a fantastic snapshot of the exceptional team and the quality of your work to date. As a DC local, I can't wait until I can stop by your lab for an in-person tour :-) Since that is likely to be a while, I am very interested in the range of age groups with which you have tested this interface. What might the challenges and opportunities be for elementary youth learning as compared to HS or undergraduate populations? 

Such very important work, Lorna and team.  Well done all round!  What a shame that sign-language is not the same all over the English-speaking world.  Will you be able to use that technology with BSL?

Diana May, London

Love this! What a creative project all around! 

What a captivating project with a truly impressive collaboration between different disciplines. 

Agreed. Nice video presentation.

Hi Lorna! Good to see you in this forum again. Last year, I shared the ASL Clear project in the showcase, and we chatted about neuroscience terminology.

Great video, excellent work, by a fabulous group! 

It looks like your team is tackling two major computer science challenges related to signed languages: making avatars that express ASL in a natural way from head to torso, and computer recognition / translation of the signer's utterances by an avatar in a VR environment. Pretty amazing! 

Which of those elements has been the most challenging to develop?  

Have you considered applications such as creating signing characters in video games? 

You mentioned that you'd be seeking industry partners at some point. Does your lab work in partnership with any other universities or institutions? 

Hi Lorna, all of us at CIRCL remain big fans of your work! Really enjoyed the video.

I have been fascinated with the people who are signing on television news reports for various medical authorities or national figures. Their role in providing information for the hearing impaired is so important. I enjoyed Signing Avatars. . 

This is truly fascinating and an amazing project- thank you for sharing this innovative project!

I had to stop by and say hi and see the new video.  Really clean explanations.  Thanks and hope you all are well!

Great work. Can't wait to see this become a tool that every child uses in elementary so that more of our communities can speak ASL. This technology looks very cool, very detailed and very labor intensive.  We also work using motion capture, and are moving towards AR as well, and it was great to find this project amongst this years S4A showcase: https://stemforall2020.videohall.com/presentations/1705.  They seem to be developing AR that can leverage phones.  This would be huge in our work in high schools where all the students have phones, and maybe there is some work they are doing that could complement this great work too.