This project follows the trajectories of youth who are STEM-interested, academically successful people who participated in an out-of-school, in-depth science research and mentoring program through the New York City Science Research Mentoring Consortium (NYCSRMC) while they were in high school and are now in college. NYCSRMC is a partnership among 24 institutions across NYC who share the goal of engaging youth in STEM research experiences working alongside scientists. As the project is entering its sixth year, we review the research questions, share findings and plans for the next phase of the study. Undergirded by a learning ecosystems approach, the study focuses on constructs that are critical for shaping one’s awareness, commitment and persistence to STEM, including: mentor/peer relationships in high school, college, and beyond, the role of family/friends in supporting studies, college/career goals, impact of pandemic on course taking, sense of belonging and othering throughout the trajectory, and the awareness and use of various types of resources. Findings will highlight the importance of participating in scientific communities of practice at an early age and how that participation mediates a sense of belonging, self-efficacy in engaging in practices of science, and strong interest in declaring STEM majors. We also describe pandemic-related impacts on these participants and end by posing questions to the audience on specific problems of practice as we continue this study and focus on documenting the ways participants encounter successes and obstacles, especially as related to implicit and explicit bias.
Research / Evaluation, Broadening Participation, Workforce Development
Am Museum of Nat History
EHR Core Research (ECR)
Related Content for Longitudinal Study of STEM Pathways
What a great project - I hope you are able to continue this for many more years. Look forward to learning more about this and looking at some of the disaggregated components that you are able to link to STEM identity and belonging in STEM.
1
Discussion is closed. Upvoting is no longer available
I love the multiple factors and multiple frameworks used to look at this program. I can see this providing important information for moving this type of research forward. Very interesting work.
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Karen Hammerness
Rhonda Christensen
Preeti Gupta
Lead Presenter Director of Youth Learning and Research
May 10, 2022 | 09:07 a.m.
Thank you John and Katherine! Yes, we are excited that we can continually to study the pathways of such a larger number of youth are interested in STEM in their formative years. We have learned so much about what it means to set up longitudinal study for this audience and in particular because the youth are from so many different programs in the city. We have also learned a lot about instrument design, and approaches to data collection that are better fits for this audience. We look forward to sharing our model about alumni co-researchers and how that has played out, what has worked, not worked etc. And of course, we have tons of findings to share especially because we pivoted during the start of the pandemic and added questions specific to the impact of the pandemic on these same students! Looking forward to a great discussion this week!
Has any of this work been published as yet (do not think I saw a link or reference in the video)? If not, do you have a timeline or specific journal you are targeting for this work? Definitely excited to see these results.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Karen Hammerness
Karen Hammerness
May 10, 2022 | 08:45 p.m.
John, We have some book chapters out about this work, and also we published a brief piece about the key principles of the mentored research programs--in Phi Delta Kappan. It shares a bit about the research as well, but focuses more on how the program itself works and the guiding principles of the design. Here's a link if that is helpful: https://kappanonline.org/partnerships-advanced-...
I echo the comments from John and Catherine. What great work. I love the way you are exploring factors across so many different engagements and activities. I'm curious about how you go about recruiting participation from different internship/activity providers? Do you share "best practice" findings and do you find that providers are receptive to this. Does the information you can provide ever lead to changes or redesigns in programs?
1
Discussion is closed. Upvoting is no longer available
I second (or in this case, fourth!) what's been said above. This is an amazing project and is clearly so multi-faceted. How easy - or difficult - do you think it would be to replicate this project in another location? Have others tried?
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Karen Hammerness
Andresse St Rose
Preeti Gupta
Lead Presenter Director of Youth Learning and Research
May 10, 2022 | 07:02 p.m.
Hi colleagues,
Great questions. I will use a bit of time to get through some of it.
Laura - replicating a mentored science research program is not difficult if you have several contextual factors lined up. Do you have a target audience that can easily get to your site, do you have a pool of scientists interested in this work and willing to both work with youth but also be humble enough to learn in the process? Do you have funds to pay the students and scientists a stipend and some money for supplies? Do you have a person who can pull this all together and coordinate the details for you becasue there ARE a lot of details. Feel free to reach out to me and we can talk about what it means to develop a program based on this model.
Barry - In our case, we don't recruit internship providers. We had a donor, the Pinkerton Foundation, that was interested in seeding some internships in diverse institutions based on our model around the city. In 2013, we started with five sites and over the years have expanded to 24 sites. Each of the sites gets funding from different sources but that seed money was critical. So call our group of 24 sites a consortium partnership. We have regular partner meetings throughout the year. In that time we talk about issues of recruitment, mentor preparation, college and career support work, and many other such things. In the last several years, our conversations have been undergirded with construct of equity and inclusion and so when we talk about recruitment, about mentor preparation..it is all to improve our work with those lenses. So then yes, the study findings are critical and totally inform program refinement. We have lots of examples of findings informing small tweaks to big initiatives. For example...the patterns in our findings that alumni need some very particular supports has led us to develop an alumni initiative with some very specific activities. Hope this all helps!
Karen Hammerness
May 10, 2022 | 08:42 p.m.
Hi colleagues, just adding on to what Preeti shared, and Laura--wanted to offer that we recently published a short piece in Phi Delta Kappan about some of the key principles that we think help support the partnership and the consortium of mentoring programs (things like shared funding; stipends for students; key principles and vision, etc.). Let us know if it would be helpful to share that with you, and Preeti would be a great resource of course for all the details on the program.
Preeti and Karen, thank you both! Karen, I just read the article you posted from Phi Delta Kappan. I really appreciated the opportunity to read that, as well as all of your questions, Preeti, that people interested in replicating a mentored science research program should ask themselves. What a great model you've created. Again, many thanks.
1
Discussion is closed. Upvoting is no longer available
It is so great to be able to study participants longitudinally to really find out what happens to those who experience these research projects. I look forward to reading research findings.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Karen Hammerness
Andresse St Rose
Facilitator Director of Educational Research and Evaluation
May 11, 2022 | 05:53 p.m.
This is exciting and important work! I am happy to learn about your study and look forward to seeing more as you publish. With any longitudinal research especially there will be attrition, are you able to keep track or follow up with (to whatever extent) students who stop participating in the study? I would be curious if they continue to pursue STEM or STEM-related endeavors or topics with overlapping skillsets and competencies.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Rhonda Christensen
Karen Hammerness
May 11, 2022 | 09:38 p.m.
Thanks Andresse, yes, we are definitely aware of and working on how to take attrition--especially for the quantitative data we are collecting through surveys-- into account. We have assumed that we will probably have some attrition, especially from losing touch with people or changing emails and ways to reach them ....and as you point out, all the prior work on these kinds of studies we know we have to anticipate some attrition. One thing is that we have deliberately tried to start with an even larger sample so that we can still, with some attrition by the last years, have a big enough group to have some explanatory power/generalizability. And, fortunately, we also have worked with these students and know many of them, so over the last five or six years we have had a pretty good response rate and interest in the work from them, which really helps. Further, we do try to track down students who haven't responded right away and have had some success with that; but with the students who leave the study and we can't reach or locate any longer--that will probably be attrition we have to accept as part of the work. Fortunately, with the interviews and the co-researchers helping us, we will have deep qualitative data that we will benefit from learning from, and can also turn to if we have any challenges with our survey and more quantitative data where we might have more attrition. Thanks so much for your interest, and let us know if you have any suggestions about addressing attrition!
Ruby Mendenhall
Higher Ed Administrator
May 11, 2022 | 09:01 p.m.
Hi,
My name is Ruby, and I am an advisor on the project. This is a beautiful video because it captures the challenged that marginalized youth experience and some of the solutions to those challenges. It is important to give our youth unprecedented access to these fields and to eliminate barriers such as racial microggressions. Great job everyone!!
This is truly an inspiring project. The collaborations as well as the individualized nature of the different sites allow for personalization! The findings are encouraging, especially as related to identity and belonging. Thank you for sharing.
Janet Coffey
Facilitator Program Director, Science Learning
May 13, 2022 | 01:24 p.m.
This is such an important study - thank you for taking it on (and thank you to NSF for funding). I suspect they'll be much to learn from those who drop out for whatever reason, so I hope you can continue to find ways to stay in touch with those students. Given the consortium model, are you learning from the variation that likely will exist across the sites - either in terms of the nature of the research program and/or the mentoring? Also, do some programs/sites offer the opportunity to learn from the benefit of a cohort model for these programs (as opposed to individual placements)? I look forward to tracking this project - for findings and methodological insights into doing this longitudinal work. Thanks!
Karen Hammerness
May 13, 2022 | 01:31 p.m.
Hi Janet, Thank you so much for your comments! And gosh, yes, thank you NSF :). Totally agree with trying to find ways to stay in touch with students who drop out of the study, if we can....and fortunately we have a large sample of people who are staying or leaving STEM, so that also helps us look at persistence at least from that aerial view. In the prior 5 years, we did a lot of analysis across program sites, and we did not see a great deal of variation in the data by program site. Some of the sites tend to have more demographic differences in terms of the backgrounds of students who participate, but we didn't see much variation in terms of opportunities to learn certain science practices. Still...definitely something we intend to keep an eye on. Preeti can answer more about the cohort model, but I believe a number of the programs have more than one student placed with a mentor.
Preeti Gupta
Lead Presenter Director of Youth Learning and Research
May 13, 2022 | 03:43 p.m.
Hi Janet, At just about every site, there are 2-4 students with a mentor. For safety reasons, we don't do 1:1. In terms of cohorts, all of the programs recruit a cohort a students annually and or that reason, the whole consortium is based on a cohort design.
Wanted to add one note about retention and attrition, we were very intentional is trying to get a hold of alumni who had participated in the study in 2017 and 2018 but had not completed our 2022 survey yet. We sent many personalized emails, and used the function in qualtrics that tells you whether the recipient has opened the email, started the survey or even clicked on the link. We then personalized our messages based on that data. We had roughly 40 bounce back emails. That is actually great considering we sent to over 500 names. We then went on an email hunt a few different ways. We did recover roughly 12 people that way. So i would say we as a team spent about 50 hours just chasing down students to take the survey but felt it was worth the investment in time.
Janet Coffey
Facilitator Program Director, Science Learning
May 15, 2022 | 02:10 p.m.
Thank you both for the responses. I'm excited to keep on eye on what you are learning. It's really an important study.
Hi Karen, This is an amazing program... sense of belonging, contributions to the science community and someone of their background can be involved in science. So incredible! Could we setup a time to chat about working together to help some of your students participate in the Journal of Emerging Investigators (JEI)? I think our programs could support one another.
Karen Hammerness
May 14, 2022 | 08:32 p.m.
Hi Scott, Oh thank you and yes, I had the same thought about the complementarity of our programs when I looked at your video! Let's definitely chat. I think Preeti and my colleagues would be really intrigued by the possibilities! Please feel free to reach out via my email at the museum; let me know if you need the contact information.
I love the work you are doing in this study, and how it has the potential to inform both formal and informal education practices. I'm wondering if you have any plans to study international contexts with some of the same instruments and protocols with an eye on comparative results that could inform all? I work a great deal internationally and often I think we need to do a better job learning from each other!
Karen Hammerness
May 16, 2022 | 02:21 p.m.
Thanks Karen! I agree, it has a lot of implications for out of school and in-school learning environments and practices. No, we haven't had a chance to reach out much yet to international networks, but we do hope to in this next stage of the study and are looking forward to presenting some of this work at some international conferences next year. I also have a lot of international research partners, in my research on teacher education (another line of work I engage in) and have learned so much from that comparative work. We welcome any connections or suggestions you have for supporting those opportunities for our community.
Hi I'm really interested in this work and I think it bridges lots of priorities in STEM Ed. I'm curious to know more about the authentic science experiences and if all youth engage in the same experiences or is there variety?
Preeti Gupta
Lead Presenter Director of Youth Learning and Research
May 16, 2022 | 04:52 p.m.
Hi Auganee,
The authentic science experiences are quite varied because each site has lots of different scientists each engaged in their own pursuits. Some may be studying lizards or turtles, others might be in a geological study, others might be look at flora at a beach shoreline and others might be working with data coming in from a telescope. The actual research can take place at a field site in NYC or in a lab. What is most important is that students are doing actual work related to the research question, not just cleaning pipettes, etc.
John Kaup
What a great project - I hope you are able to continue this for many more years. Look forward to learning more about this and looking at some of the disaggregated components that you are able to link to STEM identity and belonging in STEM.
Karen Hammerness
Catherine Quinlan
I love the multiple factors and multiple frameworks used to look at this program. I can see this providing important information for moving this type of research forward. Very interesting work.
Karen Hammerness
Rhonda Christensen
Preeti Gupta
Director of Youth Learning and Research
Thank you John and Katherine! Yes, we are excited that we can continually to study the pathways of such a larger number of youth are interested in STEM in their formative years. We have learned so much about what it means to set up longitudinal study for this audience and in particular because the youth are from so many different programs in the city. We have also learned a lot about instrument design, and approaches to data collection that are better fits for this audience. We look forward to sharing our model about alumni co-researchers and how that has played out, what has worked, not worked etc. And of course, we have tons of findings to share especially because we pivoted during the start of the pandemic and added questions specific to the impact of the pandemic on these same students! Looking forward to a great discussion this week!
John Kaup
Has any of this work been published as yet (do not think I saw a link or reference in the video)? If not, do you have a timeline or specific journal you are targeting for this work? Definitely excited to see these results.
Karen Hammerness
Karen Hammerness
John, We have some book chapters out about this work, and also we published a brief piece about the key principles of the mentored research programs--in Phi Delta Kappan. It shares a bit about the research as well, but focuses more on how the program itself works and the guiding principles of the design. Here's a link if that is helpful: https://kappanonline.org/partnerships-advanced-...
Danielle Dani
Thanks for sharing this article!
Barry Fishman
Professor
I echo the comments from John and Catherine. What great work. I love the way you are exploring factors across so many different engagements and activities. I'm curious about how you go about recruiting participation from different internship/activity providers? Do you share "best practice" findings and do you find that providers are receptive to this. Does the information you can provide ever lead to changes or redesigns in programs?
Karen Hammerness
Laura Ettinger
I second (or in this case, fourth!) what's been said above. This is an amazing project and is clearly so multi-faceted. How easy - or difficult - do you think it would be to replicate this project in another location? Have others tried?
Karen Hammerness
Andresse St Rose
Preeti Gupta
Director of Youth Learning and Research
Hi colleagues,
Great questions. I will use a bit of time to get through some of it.
Laura - replicating a mentored science research program is not difficult if you have several contextual factors lined up. Do you have a target audience that can easily get to your site, do you have a pool of scientists interested in this work and willing to both work with youth but also be humble enough to learn in the process? Do you have funds to pay the students and scientists a stipend and some money for supplies? Do you have a person who can pull this all together and coordinate the details for you becasue there ARE a lot of details. Feel free to reach out to me and we can talk about what it means to develop a program based on this model.
Barry - In our case, we don't recruit internship providers. We had a donor, the Pinkerton Foundation, that was interested in seeding some internships in diverse institutions based on our model around the city. In 2013, we started with five sites and over the years have expanded to 24 sites. Each of the sites gets funding from different sources but that seed money was critical. So call our group of 24 sites a consortium partnership. We have regular partner meetings throughout the year. In that time we talk about issues of recruitment, mentor preparation, college and career support work, and many other such things. In the last several years, our conversations have been undergirded with construct of equity and inclusion and so when we talk about recruitment, about mentor preparation..it is all to improve our work with those lenses. So then yes, the study findings are critical and totally inform program refinement. We have lots of examples of findings informing small tweaks to big initiatives. For example...the patterns in our findings that alumni need some very particular supports has led us to develop an alumni initiative with some very specific activities. Hope this all helps!
Karen Hammerness
Hi colleagues, just adding on to what Preeti shared, and Laura--wanted to offer that we recently published a short piece in Phi Delta Kappan about some of the key principles that we think help support the partnership and the consortium of mentoring programs (things like shared funding; stipends for students; key principles and vision, etc.). Let us know if it would be helpful to share that with you, and Preeti would be a great resource of course for all the details on the program.
Laura Ettinger
Preeti and Karen, thank you both! Karen, I just read the article you posted from Phi Delta Kappan. I really appreciated the opportunity to read that, as well as all of your questions, Preeti, that people interested in replicating a mentored science research program should ask themselves. What a great model you've created. Again, many thanks.
Karen Hammerness
Rhonda Christensen
It is so great to be able to study participants longitudinally to really find out what happens to those who experience these research projects. I look forward to reading research findings.
Karen Hammerness
Andresse St Rose
Director of Educational Research and Evaluation
This is exciting and important work! I am happy to learn about your study and look forward to seeing more as you publish. With any longitudinal research especially there will be attrition, are you able to keep track or follow up with (to whatever extent) students who stop participating in the study? I would be curious if they continue to pursue STEM or STEM-related endeavors or topics with overlapping skillsets and competencies.
Rhonda Christensen
Karen Hammerness
Thanks Andresse, yes, we are definitely aware of and working on how to take attrition--especially for the quantitative data we are collecting through surveys-- into account. We have assumed that we will probably have some attrition, especially from losing touch with people or changing emails and ways to reach them ....and as you point out, all the prior work on these kinds of studies we know we have to anticipate some attrition. One thing is that we have deliberately tried to start with an even larger sample so that we can still, with some attrition by the last years, have a big enough group to have some explanatory power/generalizability. And, fortunately, we also have worked with these students and know many of them, so over the last five or six years we have had a pretty good response rate and interest in the work from them, which really helps. Further, we do try to track down students who haven't responded right away and have had some success with that; but with the students who leave the study and we can't reach or locate any longer--that will probably be attrition we have to accept as part of the work. Fortunately, with the interviews and the co-researchers helping us, we will have deep qualitative data that we will benefit from learning from, and can also turn to if we have any challenges with our survey and more quantitative data where we might have more attrition. Thanks so much for your interest, and let us know if you have any suggestions about addressing attrition!
Ruby Mendenhall
Hi,
My name is Ruby, and I am an advisor on the project. This is a beautiful video because it captures the challenged that marginalized youth experience and some of the solutions to those challenges. It is important to give our youth unprecedented access to these fields and to eliminate barriers such as racial microggressions. Great job everyone!!
Danielle Dani
This is truly an inspiring project. The collaborations as well as the individualized nature of the different sites allow for personalization! The findings are encouraging, especially as related to identity and belonging. Thank you for sharing.
Janet Coffey
Program Director, Science Learning
This is such an important study - thank you for taking it on (and thank you to NSF for funding). I suspect they'll be much to learn from those who drop out for whatever reason, so I hope you can continue to find ways to stay in touch with those students. Given the consortium model, are you learning from the variation that likely will exist across the sites - either in terms of the nature of the research program and/or the mentoring? Also, do some programs/sites offer the opportunity to learn from the benefit of a cohort model for these programs (as opposed to individual placements)? I look forward to tracking this project - for findings and methodological insights into doing this longitudinal work. Thanks!
Karen Hammerness
Hi Janet, Thank you so much for your comments! And gosh, yes, thank you NSF :). Totally agree with trying to find ways to stay in touch with students who drop out of the study, if we can....and fortunately we have a large sample of people who are staying or leaving STEM, so that also helps us look at persistence at least from that aerial view. In the prior 5 years, we did a lot of analysis across program sites, and we did not see a great deal of variation in the data by program site. Some of the sites tend to have more demographic differences in terms of the backgrounds of students who participate, but we didn't see much variation in terms of opportunities to learn certain science practices. Still...definitely something we intend to keep an eye on. Preeti can answer more about the cohort model, but I believe a number of the programs have more than one student placed with a mentor.
Preeti Gupta
Director of Youth Learning and Research
Hi Janet, At just about every site, there are 2-4 students with a mentor. For safety reasons, we don't do 1:1. In terms of cohorts, all of the programs recruit a cohort a students annually and or that reason, the whole consortium is based on a cohort design.
Wanted to add one note about retention and attrition, we were very intentional is trying to get a hold of alumni who had participated in the study in 2017 and 2018 but had not completed our 2022 survey yet. We sent many personalized emails, and used the function in qualtrics that tells you whether the recipient has opened the email, started the survey or even clicked on the link. We then personalized our messages based on that data. We had roughly 40 bounce back emails. That is actually great considering we sent to over 500 names. We then went on an email hunt a few different ways. We did recover roughly 12 people that way. So i would say we as a team spent about 50 hours just chasing down students to take the survey but felt it was worth the investment in time.
Janet Coffey
Program Director, Science Learning
Thank you both for the responses. I'm excited to keep on eye on what you are learning. It's really an important study.
Scott Soldat-Valenzuela
Hi Karen,
This is an amazing program... sense of belonging, contributions to the science community and someone of their background can be involved in science. So incredible! Could we setup a time to chat about working together to help some of your students participate in the Journal of Emerging Investigators (JEI)? I think our programs could support one another.
Karen Hammerness
Hi Scott, Oh thank you and yes, I had the same thought about the complementarity of our programs when I looked at your video! Let's definitely chat. I think Preeti and my colleagues would be really intrigued by the possibilities! Please feel free to reach out via my email at the museum; let me know if you need the contact information.
Karen Page
I love the work you are doing in this study, and how it has the potential to inform both formal and informal education practices. I'm wondering if you have any plans to study international contexts with some of the same instruments and protocols with an eye on comparative results that could inform all? I work a great deal internationally and often I think we need to do a better job learning from each other!
Karen Hammerness
Thanks Karen! I agree, it has a lot of implications for out of school and in-school learning environments and practices. No, we haven't had a chance to reach out much yet to international networks, but we do hope to in this next stage of the study and are looking forward to presenting some of this work at some international conferences next year. I also have a lot of international research partners, in my research on teacher education (another line of work I engage in) and have learned so much from that comparative work. We welcome any connections or suggestions you have for supporting those opportunities for our community.
Joelle Molloy
I appreciate the connection you've unearthed between making valuable contributions and a decreased sense of imposter syndrome. Great work, well done.
aujanee young
Hi I'm really interested in this work and I think it bridges lots of priorities in STEM Ed. I'm curious to know more about the authentic science experiences and if all youth engage in the same experiences or is there variety?
Preeti Gupta
Director of Youth Learning and Research
Hi Auganee,
The authentic science experiences are quite varied because each site has lots of different scientists each engaged in their own pursuits. Some may be studying lizards or turtles, others might be in a geological study, others might be look at flora at a beach shoreline and others might be working with data coming in from a telescope. The actual research can take place at a field site in NYC or in a lab. What is most important is that students are doing actual work related to the research question, not just cleaning pipettes, etc.
Further posting is closed as the event has ended.