NSF Awards: 1620904
2019 (see original presentation & discussion)
Adult learners
CAST, in partnership with the University of Massachusetts-Amherst, and YouthBuild USA, are building and now studying our web-based digital STEMfolio for opportunity youth ages 16-24 who are neither in school nor working. STEMfolio is a portable technology-based learning environment that promotes young people’s agency and accompanies them through different learning and employment contexts. Our theory of action is based on the understanding that opportunity youth may benefit from flexible educational models that provide scaffolds and supports so they can build STEM competencies and explore STEM careers. STEMfolio also supports young people in connecting classroom learning to relevant real-world skills.
STEMfolio was developed by our team using design-based research and design thinking practices. We engaged with young people and staff in rapid prototyping to develop tool components based on the framework of Universal Design for Learning (UDL) which include; an e-portfolio to collect evidence of STEM learning and competency building, video case stories of near-peer role models engaged in STEM careers, rubrics to help evaluate understanding of STEM and STEM competencies, and built-in scaffolds and supports based on the UDL framework
The project is currently investigating the impact of STEMfolio in a randomized control trial in coordination with YouthBuild sites across the United States.
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Welcome to the CEE-STEM project, please take a look at our video and think about: How can we better recognize and measure young people's knowledge and understanding of STEM?
John Fraser
Thanks for the great intervention concept. I'm glad to see this transformative model for learning that doesn't assume academic models that have failed to be inclusive in the past look to move a few deck chairs.
Of course, my main question relates to the evidence your collecting. Have you a model for assessment that can ensure that learners can be credentialed in ways that will help potential employers to consider their accomplishments as equivalent to traditional school? Are you working with employers to think about how these types of programs might also create implicit bias against their potential for a job or career advancement, and strategies to overcome that systemic discrimination?
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Thank you John, we appreciate your comments and questions. In developing content and assessments for this tool, we took into account the competencies from K-12 education in the Next Generation Science Standards as well as labor relevant competencies so that the tool allows learners to identify those with evidence that can be shown to prospective employers. In addition, we have included rubrics for help guide users in responding and identifying competencies. These are also available for teachers to use in providing feedback for young people. Finally, the tool also contains a formative evaluation system in which all users take common challenges throughout the use of the tool and monitor progress over time.
Another aspect of the tool is the ability to create what we call collections of evidence gathered in STEMfolio. Users can send organized evidence focused in a specific domain or certain competencies to engage an employer in a hiring conversation, or in application to further education. It may be that this discussion around evidence helps with some of the stereotypes that employers could have about these alternative pathways out of high school.
With regard to prospective employers, the issue may be more that employers have implicit bias around young people from marginalized communities and that is exasperated by the lack of resources they have been given. This means for showcasing skills despite long standing systemic discrimination is a first step, but more needs to be done to help educate employers and young people about how to make sense of skills and their relationship to job opportunities given that employers have implicit bias and young people face systemic discrimination.
Mac Cannady
Research Group Director
Great video and a really cool resource!
I am wondering how broadly STEMfolio might be used. For example, we have several STEM projects with programs working with opportunity youth and could benefit from more ways for the students to demonstrate their proficiency. Is STEMfolio designed in a way that allows users from outside of youth builds to use the tool?
thanks!
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Currently the STEMfolio is being studied in an RCT in several YouthBuild sites around the country to determine it's impact and efficacy. We are optimistic about positive outcomes. STEMFolio is a web-based tool, thus making availability quite broad to those with devices and an internet connection. We too believe that participants in other STEM-oriented programs for young people could also benefit from use. The team here at CAST is reaching out to potential partners, user groups, and funders to broaden application, use and continue development of STEMfolio. In presentations and talks about this project, we have elicited interest from secondary and post secondary educators as well.
Catherine McCulloch
It is great to see this work with youth ages 16-24 who are neither in school nor working. I have several questions:
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Thank you for your support in our efforts in working with opportunity youth. You are asking good questions Catherine. We are partnering directly with YouthBuild USA and they have put us into contact with YouthBuild sites across the country. This partnership and collaboration is priceless and invigorating.
Regarding potential partners for job experiences, we are always interested in developing new connections. Additionally, many YouthBuild sites offer training in health care in addition to fields such as culinary arts and construction. Each YouthBuild site is unique in offerings which are frequently tied to the specific community.
The duration of YouthBuild efforts ranges from site to site as well, I recommend you visit the YouthBuild USA website for more information. The duration of our STEMfolio research study is 15-20 weeks. Our desire is for the STEMfolio to be a supportive and functional tool for use during and beyond a young person's formal education and training.
Catherine McCulloch
Thank you, Tracy!!
Susan McKenney
Professor
This is really amazing - thank you for sharing your work! I'm interested to learn more about your theory of action in terms of how this will work. Namely, I learned from the video is that the app supports users in collecting data (images, video, etc) that supports articulation of competencies (and provision of evidence for claiming competence/experience). I am guessing that this was informed by an understanding of what kinds of data/competencies the recognisers (i.e. potential employers) would value. Can you share any insights on that theme?
Thanks!
Susan
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Susan, thank you for your positive comments and questions. The competencies we include in STEMfolio were defined by a series of sources; the Next Generation Science Standards were our starting point. Additionally, we looked to industry and identified expectations and regulations in the workforce, as well as the mission goals, values, and expectations of YouthBuild USA. We worked with young people, staff and advisors to create competency statements in user friendly and actionable language.
Susan McKenney
Betsy Stefany
This is a truly neat tool that keeps kids' progress literally in their own hands! The use of the data fulfills a system that I'm working with to increase engagement with portfolio development for interns and other Extended Learning Opportunities (ELO)?
Definitely fits with the Bluetooth era, to form an improved system that supports student and industry needs.
Have you merged with the unions' materials and competency expectations...and how can we see more of STEMfolio involved with field studies?
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Thank you for your wonderful feedback Betsy, we're pleased you like STEMfolio. Yes, users literally have their own work and data at their fingertips, thanks to mobile technology. We have been in contact with unions and union training institutes, to learn about their training processes, expectations and evaluations, and found them to be interested and highly informative for our development and future thinking for applications of STEMfolio. We are also reaching out to CTE institutions and industry employers.
Sam Catherine Johnston
Hi Betsy,
We have started to look at this tool in the context of trade organizations, in particular apprenticeship pathways as a means of recognizing prior learning, especially for individuals and groups that are less likely to effectively demonstrate what they know when given few options or high stakes tests. If you have further thoughts about trade organizations and other workforce applications, we would love to hear about it. Sam
K. Renae Pullen
Elementary Science Specialist
I absolutely love giving young people autonomy as the learn, grow, and engage in work. I'm very interested in better understanding how the trainers and teachers are involved in supporting the young people through STEMfolio. Also, what resources (e.g. mobile device, data, career counseling) are or will be provided for the young people participating?
Patricia Ganley
K. Renae Pullen
Jonathan Lewis
This is a really innovative tool aimed at a demographic that seems susceptible to through the cracks. Do you imagine the use scaling up to the point where you might have regional or trade-specific hubs that do the mentoring and management (e.g., vetting of submitted "collections")?
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Hello Jonathan and thank you for your comments and question. Yes, we do imagine scaling up in several ways. We are looking at the potential of the tool in CTE settings as well as training centers such as Apprenticeship programs in the trades as an example. Thank you for your ideas.
Jonathan Lewis
Jennifer Stevens
Wow! STEMfolio is such a powerful tool! I love how it allows students to take charge of their own learning as well as capture the activities and experiences that are building their STEM skill repertoire! Your question about measuring students' knowledge and understanding of STEM relates to a question we have about how we measure a student's readiness for work-based learning activities. Employers say that employability skills (often termed "soft" skills) are even more important in the work environment than the knowledge and technical skills needed in the occupation. In Virginia, our new high school graduation requirements deem that a student must demonstrate that he/she has the 5cs - critical thinking, creativity, communication, citizenship, and collaboration but I have seen little work being done to help schools assess this. Do you think STEMfolio is a tool that provides a way to assess these skills? Would the application need to be tweaked for use with students who are still in school? If so, how?
Jennifer
Patricia Ganley
Some of the competencies built into the tool now are those so-called “soft” skills, like communication and collaboration, we are calling them career skills. I think using multiple media in addition to text provides richer ways for young people to demonstrate those skills. Rubrics are built into the tool for both teachers and students to help guide the quality of responses.
There may be barriers to use in schools, for example, schools may require higher security, although each user has a password protected account. The tool is still a prototype that we are testing at YouthBuild sites right now, but it’s great to start thinking of broader uses.
Thanks for your thoughts about this.
Jennifer Stevens
Betsy Stefany
Jennifer,
Also want to know more about extensions to STEMfolio and interested that you are from Virginia as I’m using Hollins University’s guidance on evaluating college level internships located developed by NACE, National Association of Colleges and Employers to structure the high school level intern involved with our project as part of the ELO (Extended Learning Opportunities) option in NH!
I am in the process of involving ongoing technology to formative access progress and find the balance of Bluetooth with broad structure enables better support for the innovation. Most school systems that truly need to offer ELO’s currently lack the administrative structure. Eager to hear more on how Virginia will address their expectations for their students possibly with documentation that is national in scope.
Jennifer Stevens
Carolyn Staudt
Tracey and team, It is so important to focus on STEM careers with young people that may not be in school or bound for college. Is there a way to see STEMfolio on my phone? I would like to see some of the UDL features that you are using.
Tracey Hall
Senior Reserch Scientist/Instructional Designer
Hello Carolyn, we're pleased to hear all this great recognition of our focus with STEMfolio. To answer your question about STEMfolio, it is a responsive design and does work on smart phones, tablets, laptops and desktops. At this point, STEMfolio is a prototype and we are studying it in a pilot which means it is not yet available outside our research. We do strive to have this become a product and available to the public for opportunity youth and others.
Melissa Demetrikopoulos, Ph.D.
This is a very exciting project! How do these young adults get involved in the project if they are not in school or working? What methods do you and your partners use to recruit them?
Sam Catherine Johnston
Hi Melissa,
Thanks for your question. We are partnered with YouthBuild USA which is a network of approximately 300 programs across the USA and beyond that provide pathways into education, careers and foster community leadership. YouthBuild serves young people living in poverty that have not had adequate support or resources in education and come to YouthBuild to complete their high school equivalency and build skills through work and community leadership opportunities. Through YouthBuild we have recruited from this network of programs for both our co-design phase and now for our RTC. In terms of criteria look for programs with stable funding and leadership, interest in supporting young people to access STEM careers, capacity to use this tool for 15-20 weeks, and then we try to have a mix of programs including urban/rural, serving large/small groups of young people. Thanks, Sam
Marc Lesser
Tracey / Sam / All - terrific work. Thanks for sharing it.
I'm curious what you've learned specifically about motivations as they relate to youth actually using the app. Like us, I'm guessing that you've done a lot to infuse the tech into the programming and instruction itself, but we're eager to get with like-minded projects to discuss what things you're finding that improve the users' experience, whether technical design or human interaction.
Here's a link to our project in case it helps to get a sense of what we're up to. Thanks for your thoughts in advance:-)
Sam Catherine Johnston
Hi Marc,
Great project and great video. We are just at the start of our experimental study so much of our learning around Youth actually using the app will come in our post surveys and interviews. That said, we learned a lot by codesigning the tools with young people and staff about what they needed to make this useful in their context. For example, youth and staff were the ones who made it clear that they needed to be able to author their own challenges rather than just have a set to choose from. They also wanted a separate domain for documenting career skills so those could be pulled out separately from STEM skills. If you are at EDC locally, perhaps we can find a time in the Fall to meet and share our respective learning. Sam
Marc Lesser
Hey, Sam - I am in NYC but my co-pi is at EDC. I'm actually Chief Learning Officer at Mouse.
That said, I'd love to get together along the way and share approach and findings. I think there's a lot of parallel to the two efforts. We've been working for years to develop platforms that help students to document work, so hopefully we could share learning that's been useful. Thanks so much for the reply.
Further posting is closed as the event has ended.