YouthAstroNet, an NSF-supported research project led by the Center for Astrophysics | Harvard & Smithsonian, seeks to inspire youth traditionally underrepresented in the sciences to pursue STEM career pathways and cultivate STEM identity. YouthAstroNet engages middle school-aged youth from across the country in authentic astronomical science experiences. The research study investigates which key features of the program correlate with five outcomes: Astronomy Content Knowledge, STEM Career Interest, Science Affinity, STEM Identity, and Computer Affinity.
Project research data sources include pre- and post-surveys administered to youth and educators, and embedded web analytics of participant activity. When controlling for the different program treatment features along with students’ prior attitudes, interests, and demographics, two program features were found to be predictive of small but significant student gains in the above 5 outcomes. These were the degree to which participants reported using the 'Core' program tools and activities (e.g. using telescopes, image processing software, asking questions); and the degree to which they reported making authentic connections between science and other topics.
YouthAstroNet gives participants an early entry into authentic scientific inquiry. The program provides tools for capturing and processing real-time astronomical data, with opportunities for open-ended and personalized investigations. Participants experience STEM investigations that align with professional scientific pursuits, where the projects do not have a right answer, but multiple possible outcomes. The program seeks to develop a sense of scientific success based on the execution of a real scientific skill. This concept of success promotes the development of science identity and career interest for diverse learners.
Science, Research / Evaluation, Informal Learning
CFA Harvard & Smith
Innovative Technology Experiences for Students and Teachers (ITEST)
Related Content for YouthAstroNet: STEM identity through authentic inquiry
Discussion from the 2019 STEM for All Video Showcase (15 posts)
Brian Drayton
Facilitator Co-Director
May 13, 2019 | 06:54 a.m.
I really enjoyed this video, I love the mutlpie ways the kids can interact both with images, and the astronomy in the images. I was wondering what measures you used for science affinity and science identity — I could imagine these might overlap?
I gather that this is an out-of-school program? If so, is there any thought of bringing it into classrooms — especially in schools that have a NASA connection already?
Mary Dussault
Lead Presenter Principal Investigator
May 13, 2019 | 11:07 a.m.
Hi Brian,
For our "science affinity" measure, we used the 10-item core "Common Instrument" self-report survey developed by Gil Noam et al at The PEAR Institute - it includes items such as "Science is something I get excited about," "I like to participate in science projects," and "I get excited about learning about new discoveries or inventions."
For our "science identity" measure, we used items that our group has used in a number of studies that ask the degree to which our participants see themselves, and believe themselves to be recognized by others (peers, teachers, family), as a "science person." (See, for example, Hazari, Zahra, et al. “The Science Identity of College Students: Exploring the Intersection of Gender, Race, and Ethnicity.” Journal of College Science Teaching, vol. 42, no. 5, 2013, pp. 82–91. JSTOR, www.jstor.org/stable/43631586.)
While these 2 outcomes measures do tend to be correlated, they also differentiate as separate factors in a factor analysis of participant responses.
The ITEAMS/YouthAstroNet program was optimized as an out-of-school program, but a significant fraction of our educator participants were classroom teachers who implemented the program in extended day learning time situations, and some during their actual normal classroom hours.
Classroom teachers are welcome to sign up to join the YouthAstroNet Program -- now that we are at the tail end our our NSF-funded research study, we are opening up the online platform for self-signup, and indeed, we are hoping to make those NASA connections as well.
Thanks so much for your interest!
Mary Dussault
Lead Presenter Principal Investigator
May 13, 2019 | 12:09 p.m.
Thank you for visiting our YouthAstroNet Program on the STEM for All Video Showcase. We are completing a final no-cost extension year of this NSF ITEST project, originally titled "Innovative Technology-Enabled Astronomy for Middle Schools,” and are working on our final analysis and writing an article for a peer-reviewed journal publications. The program revolves around participation in an inclusive online learning community of students, educators, and adult astronomy mentors centered on the use of our MicroObservatory Robotic Telescope Network and associated image analysis software.
To date, nearly 3000 youth participants have experienced the program, mostly in OST settings but also in middle school classrooms. Educators are introduced to the program through an online learning module. The program has several core elements, and then a variety of curricular resources for educators to adapt the program to their local context. (See our 2017 Showcase Video)
In this video, we highlight the results of our research study examining which (if any) program factors contribute to positive gains in 5 outcome measures: science affinity, STEM career interest, computer affinity, science identity, and science (astronomy) content knowledge. Our analysis controlled for the wide variety of program implementation features by asking students the degree to which they experienced 34 different instructional strategies. A factor analysis of student responses reduced these 34 to a set of 5 correlated “treatment variables” — 1. Core YouthAstronet Implementation (e.g., use of telescopes, image software, asking questions, posting images for others); 2. Discussions of STEM Careers (e.g. talking about STEM career options, benefits of becoming a scientist); 3. Discussions of Underrepresentation in Science (e.g., explicit discussion of work of female or minority scientists); 4. Hands-On Activities; and 5. Making STEM Connections (e.g., Connecting science to everyday life, art, or history, having virtual or face-to-face interactions with guest speakers).
When controlling for these 5 different program treatment factors along with students’ prior attitudes, interests, and demographics, only the first and last were found to be predictive of small but significant student gains: 1. the degree to which students reported experiencing the primary learning activities of the YouthAstroNet program, and 5. Making STEM Connections.
Educators: What support would you need to implement YouthAstroNet?
Researchers: All of our “treatment variables” are typically hypothesized to contribute to positive outcomes, but we were only able to demonstrate that relationship with 2 of the factors. What do you think about this?
Great project and interesting research. I wonder, was there an impact of these "treatment variables" on all of your outcome variables, or just some of them? And did you test for interaction effects, e.g., that Discussions of under-representation in science might have a differential impact on female or minority students, or that there might be an age correlation in the impact of hands on activities? Also, I wonder how much within class agreement there was on the instructional strategies experienced, as this kind of post hoc reporting by learners is an interesting way to tap key features of "instructional practice".
Mary Dussault
Lead Presenter Principal Investigator
May 15, 2019 | 12:50 p.m.
Hi Jim,
Thanks for the great questions.
The "Core YouthAstroNet Implementation" variable did have a small but significant positive effect on all 5 outcomes, while the experience of "Making STEM Connections" had positive effect on the 4 attitudinal variables, but no effect on astronomy content knowledge (I should note here that our quantitative measure for astronomy content knowledge was relatively thin - just 3 multiple choice items. We do have lots of qualitative data showing astronomy content knowledge gains that has not been correlated with the pre/post survey data).
Here is a link to our regression model, with the impact Effect Sizes reported as units of Standard Deviation. You'll see here the other interesting significant effect is that students who reported explicit discussions of underrepresentation in STEM seem to have a negative astronomy content outcome -- but to answer your question, we have not yet tested for interaction effects, so that may indeed be something going on here. We are doing a fuller analysis right now to include interactions, as well as investigating issues like the within class agreement you ask about, since we actually have educator survey data asking about the same implementation features.
Annette Eshelman
Informal Educator
May 14, 2019 | 12:26 p.m.
Exciting to see the final analysis of the program and how we are changing outcomes for our youth.
Mary Dussault
Lead Presenter Principal Investigator
May 14, 2019 | 01:02 p.m.
Hi Annette,
So nice to hear from one of our YouthAstroNet educators! - I thought you might be particularly interested in the finding that making relevant connections between science and art, or science and everyday life contributes to those positive outcomes. While we haven't broken down our data by individual program, I know your innovations as an arts educator help students be especially attentive to the visual aspects of their astrophotography creations!
Ginger Fitzhugh
Facilitator Senior Research Associate
May 14, 2019 | 05:56 p.m.
I appreciate your question, Mary, re: why only two of the factors were related to outcomes. I am curious about your response to Jim's question re: possible interaction effects. I also wonder if you collected any qualitative data that might help explain your findings (from the students themselves, their teachers, observation rubrics, etc.).
Mary Dussault
Lead Presenter Principal Investigator
May 15, 2019 | 01:08 p.m.
Hi Ginger,
As I mentioned to Jim, we're working on testing for those possible interaction effects now. I suspect that there's a bit of a "fidelity of implementation" effect here - the program has a few key core elements, and then lots of varied supplementary activities that educators can pick and choose from to adapt the program to their own local context and to emphasize different aspects of connections to robotic telescope observing. For example, many educators borrowed our classroom set of electronic circuit-making kits to pursue engineering design challenges related to the sub-systems of a robotic telescope - but these hands-on activities may have resulted on losing the focus on asking questions about the science of the objects being imaged.
About half of the 54 educators of the students in this analysis were brand new adopters of the YouthAstroNet Program Model and first-time users of the MicroObservatory telescopes and image-analysis procedures. So these results may suggest that the degree to which educators can integrate the primary innovations of our project into their instruction can vary, and that we need to emphasize the importance of this in our online professional development materials. Our deeper dive into the data will also have to look at any interaction effects with the level of experience of the educator with the program (but we may not have enough statistical power to be able to say anything here)
Victor van den Bergh
Facilitator User Researcher & Evaluator
May 14, 2019 | 07:39 p.m.
What a nice program. The images look wonderful, and it's so powerful to hear about children posing very mature questions, such as "How do we fit into the universe?" after participating in YouthAstroNet. Your instructors are clearly excited about the program, too.
I'm curious to know more about the aspect of YouthAstroNet that encourages children to make connections between science, art, history, and everyday life. Could you provide some more details about what the component looks like? Your post mentions guest speakers and I would love to know which fields and disciplines within the arts you have invited to speak with your participants (for example, visual, music, dance, poetry?), and how those guest speakers have gone about helping students make those connections.
Thank you for sharing your work!
Erika Wright
Co-Presenter Education Specialist
May 15, 2019 | 03:29 p.m.
Hi Victor,
Thanks so much for the positive feedback. We really enjoy working with the educators who use our programs. Their enthusiasm inspires us daily!
Aspects of YouthAstroNet that fall under the Making STEM Connections “treatment variable” varies from program to program, and it is often driven by the educator. A favorite activity of ours is called Astropoetry. In this activity participants are guided to look closely at their images then encouraged to create poems inspired by features they observe in the object. Annette, the YouthAstroNet educator who commented above, is actually an art educator who infuses her program with artistic skills and techniques, and uses the opportunity to talk about communicating science. Others have done projects around the historical figures that our telescopes are named after (Annie Jump Cannon, Benjamin Banneker, Cecilia Payne Gaposchkin, Donald Menzel, and Edward Pickering).
The guests speakers that we coordinate are actually scientists and staff here at the Center for Astrophysics (although many educators bring in local guest speakers of their own). They speak with programs around the country, not only about their science, but also about who they are as people, and what sorts of things they do outside of science. We encourage them to speak about their educational background, and share what everyday life is like for a scientist. Often they will share the skills they use in their work, and how those skills might be used in other fields.
We, too, are an out-of-school program and can see the great benefits from your work. I am curious how YouthAstroNet could accompany an already existing program.
We have a summer camp soon with Aerospace as the focus. We didn't venture out to space.
Can activities from YouthAstroNet be used individually, or should only be used as the entire program? Is it available outside of your area?
How many educators per student did you have trained in the PEAR instrument? Is it a difficult instrument to use for disaggregating data? Is there a fee?
Erika Wright
Co-Presenter Education Specialist
May 15, 2019 | 04:17 p.m.
Hi Jeannie,
The great thing about YouthAstroNet is that it is designed to be flexible, and used in a number of different settings. In fact other educators have implemented YouthAstroNet as part of an already existing summer camp. It sounds like you have a great summer camp lined up yourself. If you’re interested in signing up to join the YouthAstroNet Program and infusing our activities into program, you can self-signup on our online platform.
Educators are given the choice to pick and choose the activities that work best for them and their site. However, as Mary says above, use of the “Core YouthAstroNet Implementation” is where we found significant positive effect on our outcome variables – these activities include requesting and processing images with our telescopes, asking questions, and posting their images for others to see. Because this can all be done through our online portal, we have educators across the country participating in the program.
Educators implementing YouthAstroNet were not actually trained on the PEAR Instrument Suite. We used the 10-item “Common Instrument” from PEAR in conjunction with science identity, STEM career interest, and other items developed by our group. Students completed the survey online during their program, then all research and analysis was performed here at the CfA. If you’re interested in the PEAR Instrument Suite for use at your own site, it is freely available.
Margaret Glass
Informal Educator
May 15, 2019 | 10:17 p.m.
It’s great to hear and see the progress of the YouthAstroNet Program, and to hear how you are framing some of the research outcomes. Will this program continue to be available to educators and youth after the grant ends? Which contexts and settings provide the likeliest environments for continued use?
Margaret
Mary Dussault
Lead Presenter Principal Investigator
May 16, 2019 | 06:18 p.m.
Hi Margaret,
Yes, the program WILL continue to be available, thanks to ongoing funding from NASA's Universe of Learning. I like your question about "likeliest learning environments" - here are several that have seemed to thrive: We've had a number of museums implement school vacation or summer camp programs that result in student-produced exhibits that work well (like the Capture the Colorful Cosmos model we first piloted for the International Year of Astronomy). Some museums have also partnered with local schools to help facilitate a "boundary-crossing" formal/informal education program. We also have a cluster of 21st Century Community Learning Centers where the program works well as part of their overall ongoing after school environment. The educators featured in the video are from the Timothy Smith Network a local network of community technology centers with a mission of equity and access. All the centers have computer labs, making the technology very easy to implement. This year, they are also trying to integrate some of the resources of their maker spaces ("Fab Labs") with 3D printers to have students go from 2d images to 3d representations.
1
Discussion is closed. Upvoting is no longer available
Brian Drayton
Co-Director
I really enjoyed this video, I love the mutlpie ways the kids can interact both with images, and the astronomy in the images. I was wondering what measures you used for science affinity and science identity — I could imagine these might overlap?
I gather that this is an out-of-school program? If so, is there any thought of bringing it into classrooms — especially in schools that have a NASA connection already?
Mary Dussault
Principal Investigator
Hi Brian,
For our "science affinity" measure, we used the 10-item core "Common Instrument" self-report survey developed by Gil Noam et al at The PEAR Institute - it includes items such as "Science is something I get excited about," "I like to participate in science projects," and "I get excited about learning about new discoveries or inventions."
For our "science identity" measure, we used items that our group has used in a number of studies that ask the degree to which our participants see themselves, and believe themselves to be recognized by others (peers, teachers, family), as a "science person." (See, for example, Hazari, Zahra, et al. “The Science Identity of College Students: Exploring the Intersection of Gender, Race, and Ethnicity.” Journal of College Science Teaching, vol. 42, no. 5, 2013, pp. 82–91. JSTOR, www.jstor.org/stable/43631586.)
While these 2 outcomes measures do tend to be correlated, they also differentiate as separate factors in a factor analysis of participant responses.
The ITEAMS/YouthAstroNet program was optimized as an out-of-school program, but a significant fraction of our educator participants were classroom teachers who implemented the program in extended day learning time situations, and some during their actual normal classroom hours.
Classroom teachers are welcome to sign up to join the YouthAstroNet Program -- now that we are at the tail end our our NSF-funded research study, we are opening up the online platform for self-signup, and indeed, we are hoping to make those NASA connections as well.
Thanks so much for your interest!
Mary Dussault
Principal Investigator
Thank you for visiting our YouthAstroNet Program on the STEM for All Video Showcase. We are completing a final no-cost extension year of this NSF ITEST project, originally titled "Innovative Technology-Enabled Astronomy for Middle Schools,” and are working on our final analysis and writing an article for a peer-reviewed journal publications. The program revolves around participation in an inclusive online learning community of students, educators, and adult astronomy mentors centered on the use of our MicroObservatory Robotic Telescope Network and associated image analysis software.
To date, nearly 3000 youth participants have experienced the program, mostly in OST settings but also in middle school classrooms. Educators are introduced to the program through an online learning module. The program has several core elements, and then a variety of curricular resources for educators to adapt the program to their local context. (See our 2017 Showcase Video)
In this video, we highlight the results of our research study examining which (if any) program factors contribute to positive gains in 5 outcome measures: science affinity, STEM career interest, computer affinity, science identity, and science (astronomy) content knowledge. Our analysis controlled for the wide variety of program implementation features by asking students the degree to which they experienced 34 different instructional strategies. A factor analysis of student responses reduced these 34 to a set of 5 correlated “treatment variables” — 1. Core YouthAstronet Implementation (e.g., use of telescopes, image software, asking questions, posting images for others); 2. Discussions of STEM Careers (e.g. talking about STEM career options, benefits of becoming a scientist); 3. Discussions of Underrepresentation in Science (e.g., explicit discussion of work of female or minority scientists); 4. Hands-On Activities; and 5. Making STEM Connections (e.g., Connecting science to everyday life, art, or history, having virtual or face-to-face interactions with guest speakers).
When controlling for these 5 different program treatment factors along with students’ prior attitudes, interests, and demographics, only the first and last were found to be predictive of small but significant student gains: 1. the degree to which students reported experiencing the primary learning activities of the YouthAstroNet program, and 5. Making STEM Connections.
Educators: What support would you need to implement YouthAstroNet?
Researchers: All of our “treatment variables” are typically hypothesized to contribute to positive outcomes, but we were only able to demonstrate that relationship with 2 of the factors. What do you think about this?
Jim Hammerman
Great project and interesting research. I wonder, was there an impact of these "treatment variables" on all of your outcome variables, or just some of them? And did you test for interaction effects, e.g., that Discussions of under-representation in science might have a differential impact on female or minority students, or that there might be an age correlation in the impact of hands on activities? Also, I wonder how much within class agreement there was on the instructional strategies experienced, as this kind of post hoc reporting by learners is an interesting way to tap key features of "instructional practice".
Mary Dussault
Principal Investigator
Hi Jim,
Thanks for the great questions.
The "Core YouthAstroNet Implementation" variable did have a small but significant positive effect on all 5 outcomes, while the experience of "Making STEM Connections" had positive effect on the 4 attitudinal variables, but no effect on astronomy content knowledge (I should note here that our quantitative measure for astronomy content knowledge was relatively thin - just 3 multiple choice items. We do have lots of qualitative data showing astronomy content knowledge gains that has not been correlated with the pre/post survey data).
Here is a link to our regression model, with the impact Effect Sizes reported as units of Standard Deviation. You'll see here the other interesting significant effect is that students who reported explicit discussions of underrepresentation in STEM seem to have a negative astronomy content outcome -- but to answer your question, we have not yet tested for interaction effects, so that may indeed be something going on here. We are doing a fuller analysis right now to include interactions, as well as investigating issues like the within class agreement you ask about, since we actually have educator survey data asking about the same implementation features.
Annette Eshelman
Exciting to see the final analysis of the program and how we are changing outcomes for our youth.
Mary Dussault
Principal Investigator
Hi Annette,
So nice to hear from one of our YouthAstroNet educators! - I thought you might be particularly interested in the finding that making relevant connections between science and art, or science and everyday life contributes to those positive outcomes. While we haven't broken down our data by individual program, I know your innovations as an arts educator help students be especially attentive to the visual aspects of their astrophotography creations!
Ginger Fitzhugh
Senior Research Associate
I appreciate your question, Mary, re: why only two of the factors were related to outcomes. I am curious about your response to Jim's question re: possible interaction effects. I also wonder if you collected any qualitative data that might help explain your findings (from the students themselves, their teachers, observation rubrics, etc.).
Mary Dussault
Principal Investigator
Hi Ginger,
As I mentioned to Jim, we're working on testing for those possible interaction effects now. I suspect that there's a bit of a "fidelity of implementation" effect here - the program has a few key core elements, and then lots of varied supplementary activities that educators can pick and choose from to adapt the program to their own local context and to emphasize different aspects of connections to robotic telescope observing. For example, many educators borrowed our classroom set of electronic circuit-making kits to pursue engineering design challenges related to the sub-systems of a robotic telescope - but these hands-on activities may have resulted on losing the focus on asking questions about the science of the objects being imaged.
About half of the 54 educators of the students in this analysis were brand new adopters of the YouthAstroNet Program Model and first-time users of the MicroObservatory telescopes and image-analysis procedures. So these results may suggest that the degree to which educators can integrate the primary innovations of our project into their instruction can vary, and that we need to emphasize the importance of this in our online professional development materials. Our deeper dive into the data will also have to look at any interaction effects with the level of experience of the educator with the program (but we may not have enough statistical power to be able to say anything here)
Victor van den Bergh
User Researcher & Evaluator
What a nice program. The images look wonderful, and it's so powerful to hear about children posing very mature questions, such as "How do we fit into the universe?" after participating in YouthAstroNet. Your instructors are clearly excited about the program, too.
I'm curious to know more about the aspect of YouthAstroNet that encourages children to make connections between science, art, history, and everyday life. Could you provide some more details about what the component looks like? Your post mentions guest speakers and I would love to know which fields and disciplines within the arts you have invited to speak with your participants (for example, visual, music, dance, poetry?), and how those guest speakers have gone about helping students make those connections.
Thank you for sharing your work!
Erika Wright
Education Specialist
Hi Victor,
Thanks so much for the positive feedback. We really enjoy working with the educators who use our programs. Their enthusiasm inspires us daily!
Aspects of YouthAstroNet that fall under the Making STEM Connections “treatment variable” varies from program to program, and it is often driven by the educator. A favorite activity of ours is called Astropoetry. In this activity participants are guided to look closely at their images then encouraged to create poems inspired by features they observe in the object. Annette, the YouthAstroNet educator who commented above, is actually an art educator who infuses her program with artistic skills and techniques, and uses the opportunity to talk about communicating science. Others have done projects around the historical figures that our telescopes are named after (Annie Jump Cannon, Benjamin Banneker, Cecilia Payne Gaposchkin, Donald Menzel, and Edward Pickering).
The guests speakers that we coordinate are actually scientists and staff here at the Center for Astrophysics (although many educators bring in local guest speakers of their own). They speak with programs around the country, not only about their science, but also about who they are as people, and what sorts of things they do outside of science. We encourage them to speak about their educational background, and share what everyday life is like for a scientist. Often they will share the skills they use in their work, and how those skills might be used in other fields.
Jeannie Whitlock
We, too, are an out-of-school program and can see the great benefits from your work. I am curious how YouthAstroNet could accompany an already existing program.
We have a summer camp soon with Aerospace as the focus. We didn't venture out to space.
Can activities from YouthAstroNet be used individually, or should only be used as the entire program? Is it available outside of your area?
How many educators per student did you have trained in the PEAR instrument? Is it a difficult instrument to use for disaggregating data? Is there a fee?
Erika Wright
Education Specialist
Hi Jeannie,
The great thing about YouthAstroNet is that it is designed to be flexible, and used in a number of different settings. In fact other educators have implemented YouthAstroNet as part of an already existing summer camp. It sounds like you have a great summer camp lined up yourself. If you’re interested in signing up to join the YouthAstroNet Program and infusing our activities into program, you can self-signup on our online platform.
Educators are given the choice to pick and choose the activities that work best for them and their site. However, as Mary says above, use of the “Core YouthAstroNet Implementation” is where we found significant positive effect on our outcome variables – these activities include requesting and processing images with our telescopes, asking questions, and posting their images for others to see. Because this can all be done through our online portal, we have educators across the country participating in the program.
Educators implementing YouthAstroNet were not actually trained on the PEAR Instrument Suite. We used the 10-item “Common Instrument” from PEAR in conjunction with science identity, STEM career interest, and other items developed by our group. Students completed the survey online during their program, then all research and analysis was performed here at the CfA. If you’re interested in the PEAR Instrument Suite for use at your own site, it is freely available.
Margaret Glass
It’s great to hear and see the progress of the YouthAstroNet Program, and to hear how you are framing some of the research outcomes. Will this program continue to be available to educators and youth after the grant ends? Which contexts and settings provide the likeliest environments for continued use?
Margaret
Mary Dussault
Principal Investigator
Hi Margaret,
Yes, the program WILL continue to be available, thanks to ongoing funding from NASA's Universe of Learning. I like your question about "likeliest learning environments" - here are several that have seemed to thrive: We've had a number of museums implement school vacation or summer camp programs that result in student-produced exhibits that work well (like the Capture the Colorful Cosmos model we first piloted for the International Year of Astronomy). Some museums have also partnered with local schools to help facilitate a "boundary-crossing" formal/informal education program. We also have a cluster of 21st Century Community Learning Centers where the program works well as part of their overall ongoing after school environment. The educators featured in the video are from the Timothy Smith Network a local network of community technology centers with a mission of equity and access. All the centers have computer labs, making the technology very easy to implement. This year, they are also trying to integrate some of the resources of their maker spaces ("Fab Labs") with 3D printers to have students go from 2d images to 3d representations.
Margaret Glass
Further posting is closed as the event has ended.