Thanks for your response and questions, Karen. This past year, we had several team members who didn't want to use VR. One stuck with this position, reporting that they didn't like electronics around their head, which was more than fine. Indeed, one of the things we've been clear on from the beginning is that we're not trying to push anyone toward the use of VR; instead, we're trying to explore the affordances of the technology and inform the design of VR experiences to help make them worthwhile and inclusive for those who do use them. Another team member found a way to try VR on their own, instead of in the team context; this, too, was fine. Comfortable entry into an experience is important.

More generally, we're designing around several factors like perceived movement while the player is stationary, which is an issue in VR for many participants, causing motion sickness. Previous research has suggested this might be especially common for neurodivergent participants. And our research is looking at various other sensory input factors, including levels of brightness, clutter, and noise, as well as color preferences. This research is ongoing.

I hope this answers your questions, and thanks, again.

Thanks Mia, we agree. I have worked on several other co-design projects with neurodiverse learners in the past but many other members of the team have not and for most of the students this was a new process for them as well. I believe this was a valuable process both for us as design and research professionals to get outside our standards modes of operating and to look at different and sometimes overall better ways of helping the team as a whole engage. I believe this was also a great experience for the interns who not only got to contribute to shaping this VR experience but also gained valuable soft skills such as working within a team, sharing ideas, advocacy, collaboration, and other skills that hopefully will serve them will in whatever they choose to pursue next.

Tara, thanks for your positive thoughts. As per your questions, the pandemic made it quite challenging to meet as a team. We didn't actually get all of us together at Landmark until this year! That was an exciting and long anticipated meeting. The energy in the room was invigorating! You are right that technology was an important part of our staying connected. Zoom, like many other projects I'm sure, was our prime virtual f2f tool. We also depended heavily on Miro--a very versatile visual collaboration tool. It was essential for brainstorming, documenting, and providing feedback on various ideas moving between team members in different parts of the country. 

I think what really helped me feel comfortable and trust everyone was having a space where we did sometimes get distracted and talk about other interests, and Teon or whoever was leading the meeting would also prompt or reference specific people during discussions. What also helped that they did was having break out groups that were smaller and changed each time so that we would all get a chance to work together in smaller groups, which certainly helped a lot, and to build off of that, us Landmark students would meet with 1-2 of the Terc team every week, separate from our main group meeting, and they help form connections that helped with comfort and trust significantly as well.

Also, since the TERC team and other landmark students already knew each other when I joined, when they would tease or praise each other on things they already knew about each other, it did make it a lot more comfortable and feel a lot less like a rigid work environment. That feeling that everyone is friends definitely helped me trust everyone and feel a lot more comfortable.

another thing that I think is really important and is often overlooked, is recognizing that not everyone is comfortable talking at all points. So while we would be pushed to share everything when needed and prompted to talk, if we were just having a day where we didn't want to talk or feel comfortable talking on a subject, it was never forced, and that definitely helped me trust the group more and feel comfortable.

Hi Lorna, thanks for visiting our project page, it is nice to run into you here as well. This game is intended to be a mystery STEM science game aimed at deployment on museum floors. Our plans for working with museum partners have been impacted by the pandemic but we are finally able to start working with one of our partners the Boston Museum of Science on gathering some data on the impact of VR on the performance of museum visitors on VR tasks. The game contains a number of STEM puzzles aimed at increasing interest in STEM. Game puzzles include elements such as circuits, astrogeology, and astrobiology as well as elements informed by the experiences of neurodivergent learners including communication challenges and sensory difficulties.

Hi, Dan. Thanks for your comment and questions.

The setting and broad nature of the game were predetermined — the game would take place on a science station on the moon Europa; the player would be alone there; the game would address STEM skills and concepts, with the purposes of building awareness of and interest in STEM; and it would specifically try to be inclusive of players with sensory, attention, and social differences. A mystery storyline for the game existed... but was thrown out almost immediately because of the earliest co-design sessions. Letting go of that story wasn't difficult, because the reasons and ideas coming from the co-design process were so exciting; however, getting to a new storyline was challenging.

We didn't limit who could join the co-design team based on type of neurodiversity. And instead of focusing on a particular neurodiversity for the design of the game, we focused on sensory, attention, and social differences. This resulted in a range of in-game design choices, affecting elements ranging from user interfaces to major narrative points. For some sensory examples, (1) preliminary research suggested that both neurodivergent and (so called) neurotypical participants had very similar preferences in terms of brightness, but the neurodivergent were more consistent and stronger in their preferences. In other words, it's possible to get this right for a broad array of players, but getting it wrong will more negatively impact neurogivergent ones. The game is using this info to set default brightness levels, and in-game controls for brightness are easily and readily accessible — think dimmer switches. (2) The main fictional lifeform on Europa, a cephalopod-like creature called the Minos, are used to the darkness of the sub-ice oceans, where bioluminescence is one of the main sources of light. Thus, the Minos have sensory issues with light (both visual and UV), and exploring this and their tolerance levels is central to the story.

The game is for a broad audience that includes, but is certainly not limited, to neurodivergent players. Indeed, the intent is to make a game that is as broadly inclusive as possible.

Hope this answers your questions. I'd be happy to communicate with you more, if you'd like. Thanks, again.

Hi Brianna. This was part of a paid research opportunity for all our co-designers. Not only did the co-designer work as part of the design and research teams, but we also provided opportunities for them to directly interact with our development partners and dedicated some sessions for them to meet with the development team to discuss career pathways into various aspects of XR development. Several of our interns have expressed interest in pursuing careers in design and development. This opportunity presented participants with a view of the breadth of skills that are needed in those areas including UX design, storyline development, coding, graphic design, animation, etc. which many hand not seriously considered before.

What an awesome opportunity for them, Ibrahim.  I hope you're doing well.  If any of the students are majoring in computing fields, we'd love to have them participate in AccessComputing's mentoring and career development activities.

Thanks Brianna, I will pass the information onto them.

Hi, Marcelo.

At this time, we don't have any plans for long-term studies, though I think this sounds like a wonderful idea. The closest idea we've considered is possibly adding some case studies to our research efforts. These might begin to get at some of the impacts of longer engagement, as well as more individual and deep perspectives.

Thanks for your questions.

Hi! I was one of the codesign Landmark Students. I actually had never used VR before, and now I have only used it once- roughly last December my friend offered to show me with his VR headset. I found that most of the knowledge needed was about general video games rather than specialized VR knowledge. I personally treated a lot of it as if it was a minecraft style video game, where you do walk around and it has a similar feel to what I had assumed VR games are like. I did learn some thing and have to use that information that was specific to VR, but it was things like "writing and typing is really hard in VR" "dizziness is a factor" "lots of text/reading is not ideal for VR", which are things that I just had to be told and then could factor in.

I would say that the minimum technology knowledge someone would need is the understanding of what VR is, having seen or played a variety of different video games or be willing to try some. :)

Raffaelia, that's a wonderful pair of related questions.

First, COVID had a major impact on this for us. The original plan involved a lot of in-person meetings, including sessions in VR and even the coding of some prototype elements. Instead, everything for the first year and more was over Zoom, and we were largely unable to get early participants into headsets. As a result, those early participants were largely "hypothetical" VR fans — interested enough in VR, game design, technology, and (often) computer science to want to work on the project, but without (much) actual exposure to the tech.

More recently, we did start to get people into headsets IF they wanted (see response to Karen Royer's comment above), to have more direct interaction between the co-design team and our development partner, and to have team members choose to focus on VR-specific details, such as exactly how the grasping of objects will work in-game and what the strengths and limitations of this should mean for task and puzzle design. In an ideal world, I would have had this — and more — all along, but in thinking about your open question and considering your video, it's possible the non-ideal of the last two years led me to realize the minimum might be lower than I would have guessed.

Once this is said, it'll sound obvious, but interests and skills in storytelling, drawing, STEM, psychology, etc. became things we valued more highly in later recruitment efforts, not downplaying the tech-aspects but further highlighting the design-aspects.

Happy to communicate with you more about this, and thanks again.

Hi! I cant talk about creating the structure or any of it, but I can talk about what Teon used with us from my perspective, and one of the TERC people can probably elaborate on some parts too :)
The way our process was set up was that we would have a group meeting once a week, a mentor meeting once a week that was more one-on-one with 1-2 of the TERC group and 1-2 Landmark student/s to talk about what they are working on, and then we would be given a task or task options for the following meetings (often we would have projects we were already working on individually, so we would often do that OR a specific task)

The meetings were usually on zoom but the TERC group did come to Landmark College a handful of times. We didn't really need ice breakers, though my first week or two we had some. Most meetings started with us just chatting about life in general for the first 5 minutes, and we would get distracted or off topic all the time. I always found it fun to talk about randomly specific biology topics with Jamie since it's a topic we both know extensively, but the rest of the group didnt know as much- like just mentioning "siphonophores". To me, it felt as if we were a group of friends who decided to do a project together, rather than people who started working together for the purpose of the project. I think that atmosphere made it so we did not need ice breakers.

For the material outside of the meetings- at first we were given specific tasks, things like "write a scenario if the player came across a computer that said 'incoming signal'", or a handful of tasks and we had to pick one of them. After a few weeks, we were given the option to focus on some tasks a bit more long term or keep doing the weekly ones, and most of us found things we wanted to focus on, and we would work on them between meetings and use the meetings as more of a share out discussion and feedback, and the weekly tasks changed to reflect that more.

In terms of face-to-face; due to covid restrictions at Landmark, they were only able to visit once in the Fall semester, and then scheduling meant that the TERC group visited once in the Spring as well, with part of the TERC group visiting a second time. Occasionally us Landmark students would chat about the tasks or ideas when we'd bump into each other at school, but most of the co-design process ended up being on Zoom. 

Shari, Becky has provided a wonderful response from the Landmark perspective, as well as a wonderful overview of the process we eventually arrived at, so let me simply fill in with some of how we got there and a few "behind the scene" elements.

This project began in August 2020, so it was entirely proposed pre-COVID and entirely conducted post-COVID. We were figuring out our structures and processes as we went along. Moreover, the strengths and challenges of individual team members, at both Landmark and EdGE, varied greatly, shaping how we worked individually and as a team... and meaning extensive flexibility had to be built into our processes.

I used icebreakers whenever new-members joined the team, but the EdGE team members didn't change and we kept Landmark team members for long enough stretches (one semester, two semesters, and even four semesters), so icebreakers were not a major element.

Our agendas, tools, approaches, and schedules evolved over time, matching the needs and personalities of participants, as well as the stage of the work. Not everyone is comfortable talking in group settings; not everyone is as confident in or willing to express or stand behind their ideas, especially if someone else has already spoken up about something; not everyone can capture or share their ideas in the same ways; etc. This is always true, but given that many of our team members are neurodivergent, I couldn't "conveniently" ignore this. We adopted tools like Miro boards, and processes like simultaneous brainstorming (with specific questions, specific lengths of quiet time for everyone to generate and capture unfiltered ideas on digitial sticky-notes, review times where stickies were read and ideas that resonated marked, discussion time, and possibly another round). I encouraged the use of chat for those who didn't feel as comfortable speaking up, noting that even contributions like a simple "I agree" are invaluable. The proposal talks a lot about Design Documents, but a "document" could be something written, drawn, assembled, or audio/video recorded. Some members worked best talking through their ideas and having a partner help "capture" them, while others wanted their partners to be "hands off" their documentation, and still others had truly joint materials. During meetings, I often called on individuals by name. For some members, this involved additional steps before and during the meeting so no one was "put on the spot" (e.g., a simple, "Are you willing to go next?" in private chat was vitally important for one team member, with "no" being an acceptable answer, but usually serving as an opportunity for them to gather themselves). I credited ideas to the people who generated them, getting myself and everyone in the habit of acknowledging different people's contributions and building ownership in them. Even after team members left the project, their ideas would be part of the discussion and noted as such. I learned the importance of having very concrete agendas, and the pre-sharing of these, especially if they involved any shareouts. And as Becky notes, with our latest co-design team configuration, we spent time "off topic," joking around, checking in, and otherwise treating each other as whole individuals. Of course meeting time is precious, and not everyone likes the same types of group interactions, but designing is hard, so taking the time and figuring out the ways to let everyone feel as comfortable as possible was vital.

Basically, I don't think any of what we employed in terms of strategies, structures, and processes were new or even innovative, but I think we were very deliberate and mindful about what we did and how we did it in ways that aren't typical. I spent way more time and effort preparing for meetings, creating tasks and related scaffolds, going through and processing what was generated, etc. than I expected, but I think it was worth it.

Hi, Jamie. Yes, the Museum of Science Boston and the Pacific Science Center have been been part of the project since the proposal stage. Unfortunately, COVID seriously messed with our museum-related plans.

I'm pleased to share that work with and at the MOS is scheduled to start the end of this month! We're using a specialized VR build, set in one room of our Europan station, to research the impact of different brightnesses, noise levels, and amounts of clutter on participant performance in VR. We'll be gathering data from participants who do or do not, by self report, have various neurodiversities, including autism and ADHD. We'll also be conducting this research on the Landmark College campus, which wasn't in the original plan. We're very appreciative of this, and we're excited to have one of our co-design interns, Katie, working with us on this over the summer.

PacSci is only now beginning to reopen for the public. We've stayed in touch over the last few years, but otherwise let them do what they needed to be doing. Going forward, we'll certainly see if/when our partnership on this project might fit.

Thanks for the question. It's important to acknowledge these partners, even if they haven't yet been a big part of the process.