The CitizenSciGirls project includes videos and online games that engage children in citizen science. A third-party summative evaluation led by Multimedia Research’s Dr. Barbara Flagg examined how SciGirls’ multimedia content contributed to girls’ experience of citizen science.
Fifth grade girls at five geographically diverse sites were randomly assigned to one of two groups. The treatment group experienced citizen science-themed SciGirls episodes and games, followed by a live citizen science session with the FrogWatchUSA project. The control group only experienced the live citizen science session without prior SciGirls exposure. Dr. Flagg’s evaluation specifically examined how SciGirls multimedia contributed to girls’ (1) interest in the FrogWatch session and citizen science generally, (2) self-efficacy in the FrogWatch session and citizen science generally, and (3) learning about the practice of citizen science.
The treatment group demonstrated significantly higher levels of interest than the control group in their live session and significantly better understanding of citizen science. Within this treatment group, minority girls were even more strongly impacted by pre-exposure to SciGirls compared to their non-minority peers. For example, these diverse girls displayed higher interest in their FrogWatch session, in finding out more about other citizen science projects, a greater likelihood to look for a future citizen science activities, greater perceived efficacy in doing other citizen science projects, more similarity to the video girls, and stronger interest in their SciGirls experience.
Citizen Science, Research / Evaluation, Informal Learning
Twin Cities PBS, Cornell Lab Orn, NGCP
Advancing Informal STEM Learning (AISL)
Related Content for Citizen SciGirls: Summative Eval. of SciGirls Season Three
Discussion from the 2017 STEM for All Video Showcase (20 posts)
Sarah Carter
Lead Presenter Manager, STEM Media & Education
May 14, 2017 | 08:22 p.m.
Welcome! Thanks for viewing the video about Citizen SciGirls. We welcome all comments, questions, and feedback about our project. As this project is complete we're particularly interested in hearing about audiences that would be interested in this research and other projects that are looking at the role of video in engaging youth in informal education experiences.
Richard Hudson
Co-Presenter Director of Science Production, Retired
May 14, 2017 | 10:03 p.m.
The series of SciGirls episodes on Citizen Science was a rich collaboration with the Citizen Science group at the Cornell Lab of Ornithology and with six different CS programs across the nation. It was a tremendously exciting learning experience for all of us on the SciGirls production team. It also afforded us the opportunity to collaborate on a unique, controlled experimental evaluation of how media can stimulate and enhance a live hands-on citizen science experience. (See http://citizenscience.org/2016/12/06/research-s... ) With the myriad ways that young people can access video today -- on their tablets and smart phones, as well as traditional television -- video can be an even more important component of STEM learning. While we have always known that video can produce both interest and learning, it has not been clear how video can contribute to a richer and deeper hands-on inquiry experience. This research demonstrates the power of this synergy. We hope STEM educators may be inspired to find other ways to integrate video into their hands-on STEM activities, and we look forward to the discussions this week. - Richard Hudson, Executive Producer
Sarah Garlick
Facilitator Director of Science Policy and Outreach
May 15, 2017 | 12:42 p.m.
This is great! The study design and results of the control and treatment groups are really interesting. I'm curious to know how closely tied the FrogWatch sessions were with the SciGirls media, in terms of content and/or other programming attributes. In other words, do you think other existing citizen science programs could have similar success by leveraging the same SciGirls media (assuming permissions)? Is this a possible avenue for scaling up? Or do you envision replication involving citizen science programs developing their own customized video content?
Sarah Carter
Lead Presenter Manager, STEM Media & Education
May 15, 2017 | 12:55 p.m.
These are great questions. The treatment girls watched three episodes of SciGirls prior to participating in FrogWatch including one episode that showed a group of four friends learning about and participating in FrogWatch specifically. Because the episodes show some good examples of the process that you can go through when participating in citizen science we think they are a great example no matter what type of citizen science project you may be participating in. The recommendations for the citizen science field from our evaluator, Dr. Barbara Flagg are below:
Leaders of citizen science partners should utilize SciGirls multimedia prior to their citizen science sessions with preteen girls to increase interest and learning, particularly those projects featured in the videos.
Informal and formal educators should use SciGirls’ resources - video in particular - to introduce the practice of citizen science generally and to generate interest and participation more broadly in other citizen science projects.
Researchers should explore the differential impact of SciGirls on minority girls’ interest and efficacy with larger samples to shed light on how peer-oriented multimedia influences youth outcomes, how minorities might respond differently to contributory model citizen science, and how groups differ in their pathways of science interest and efficacy.
We haven't discussed scale up yet but these are some interesting ideas. Thanks!
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Heidi Carlone
Sarah Garlick
Richard Hudson
Co-Presenter Director of Science Production, Retired
May 15, 2017 | 06:27 p.m.
As Sarah mentions, the treatment group in the evaluation watched two other episodes besides the one showcasing Frog Watch, which they then did. The control group only did the live project activity.
Each episode of the series features a different group of girls and a different Citizen Science project. And in each episode, we intentionally introduced the basic idea of participatory, Citizen Science. That is perhaps why the treatment group demonstrated a deeper understanding of what citizen science is than the control group, which only took part in the hands-on training and data-gathering. (The control group subjects had the opportunity to watch the videos after their experience, and were evaluated a second time as part of the study.)
So we would imagine that the episodes would serve any educator who wanted to introduce youth to citizen science, regardless of what their project might be.
The other useful part of the episodes is the extension activity. In some of the episodes, the girls created a presentation about their research and shared it with the public (Frogwatch: https://goo.gl/f7Yyh6 ). In other episodes, the girls conducted a related inquiry that they designed themselves (Monarch Watch project: https://goo.gl/acKffg ). We believe such extension activities can give added meaning and value to the citizen science data gathering and reporting.
We very much look forward to hearing from people guiding youth in citizen science experiences about who may use these videos.
They are universally accessible: on PBS Learning Media, on the SciGirls website, and even on a YouTube channel. Girls (and boys!) can watch the episodes on their tablets or smartphones as part of a live CS experience. We'd love reports from the field on how this works! - Richard
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Sarah Garlick
Heidi Carlone
Facilitator Professor
May 15, 2017 | 09:37 p.m.
I admit that I've been a fan of SciGirls for a while now. :) The SciGirls show is so compelling. I love the idea of diverse girls, working together, to solve a problem that demands scientific thinking, and the emphasis on citizen science is empowering. I'm wondering: How long did each project last? Has your team been able to track the girls over time? What happens to their long-term interests, if anything? I study science identity, and "recognition by others" is a big part of developing and sustaining science identity. The video component of this really showcases girls' scientific thinking in the best possible way, and is a stunning example of garnering "recognition by others". I'm interested in the girl participants as well as those who watch(ed) the program regularly. What a wonderful example of the power of video!
Sarah Carter
Lead Presenter Manager, STEM Media & Education
May 16, 2017 | 10:57 a.m.
When shooting an episode we work with the girls for about six days so it's not unlike an intensive summer camp experience. The difference being they have great opportunities to work closely with a STEM mentor and get some unique behind the scenes experiences. Unfortunately we haven't had the funds to do any official follow up research or evaluation on girls that have been in SciGirls episodes but we've talked about it.
Our current season and outreach project Latina SciGirls is looking at middle school-age Hispanic girls' positive STEM identity development. The research study being done by the University of Colorado Boulder will test the hypothesis: The SciGirls model, when augmented to address specific barriers to STEM engagement of Hispanic girls ages 8 to 13 and their parents, will promote the development of positive STEM-related identities in Hispanic girls. The study will investigate Hispanic girls' personal experiences engaging with new episodes featuring bilingual girls and mentors and bilingual activities and how those experiences contribute to their STEM-related identity development against cultural and gender-based stereotypes.
Discussion is closed. Upvoting is no longer available
Found helpful:
Julia Griffin
Heidi Carlone
Heidi Carlone
Facilitator Professor
May 16, 2017 | 07:48 p.m.
Sarah,
Oh, that's great! I'm so glad to hear that identity-related outcomes are of interest in this next iteration of SciGirls. I haven't seen the LatinaSciGirls program-- I'll have to look for it now.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Sarah Carter
Richard Hudson
Co-Presenter Director of Science Production, Retired
May 17, 2017 | 12:37 p.m.
The Latina SciGirls episodes won't be released until next year, but all of our episodes are available with lip-sync Spanish voices on PBS Learning Media. What will distinguish our new episodes is that they are all filmed with native Spanish speakers!
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Heidi Carlone
Richard Hudson
Co-Presenter Director of Science Production, Retired
May 16, 2017 | 11:45 a.m.
The published research on SciGirls Season 3 includes a lengthy discussion of "self-efficacy," which is key to the development of a science identity. You can find the report here:
The research was a controlled study of two groups of viewers: one group watched SciGirls media prior to doing the Frogwatch investigation; the control only took part in the Frogwatch investigation, with introduction by the instructor.
In addition to the findings mentioned in the video, Barbara Flagg wrote;
Within the treatment group that was exposed to SciGirls, minority girls demonstrated significantly higher interest than non-minorities in finding out more about other citizen science projects; greater likelihood to look for a future citizen science project to do; and stronger belief in their efficacy to be good at doing other citizen science projects.
She also notes that this is a finding worthy of further study!
2
Discussion is closed. Upvoting is no longer available
Found helpful:
Heidi Carlone
Sarah Garlick
Heidi Carlone
Facilitator Professor
May 16, 2017 | 07:49 p.m.
Richard-- I will definitely look at the report-- thanks! Great work, all around.
Claire Quimby
Facilitator Research Associate
May 16, 2017 | 08:08 p.m.
This sounds like a great project, and your video is really compelling too. Did you talk with the girls at all about their reactions to the videos? Curious to know if (aside from the information conveyed about citsci) any particular elements really resonated with them and maybe contributed to their increased engagement with the Frogwatch activity.
Richard Hudson
Co-Presenter Director of Science Production, Retired
May 17, 2017 | 11:21 a.m.
The Season 3 research (see the link earlier in this thread) focused very specifically on engagement and understanding of citizen science. We attributed the engagement to the research-based strategies that we use in all of our SciGirls work, which we call "The SciGirls Seven." (See http://www.scigirlsconnect.org/scigirls/ )
An earlier evaluation analyzed how viewer comments about the episodes aligned with some these strategies:
• One strategy suggested that girls are motivated by projects they find personally relevant and meaningful. More than half (55%) of the SciGirls viewers felt the engineering design projects were important or relevant to their personal interests.
• Another strategy proposed that girls are motivated when they can approach projects in their own way, applying their creativity, unique talents and preferred learning styles. Half (50%) of viewers liked how the real girls used their own creativity and talent to solve problems and complete the engineering projects.
• A third strategy noted that girls benefit from collaboration, especially when they can participate and communicate fairly. Almost half (45%) of the SciGirls viewers were energized by the teamwork and constructive interactions of the real girls.
• A final strategy indicated that girls benefit from relationships with role models and mentors. One-third (31%) of viewers identified the onscreen girls as role models for themselves and/or noted the help of the onscreen mentors in the real girls’ projects.
(The remaining 3 strategies are techniques that educators should use when working with girls, and as such are less visible in the shows themselves.)
Hope that answers your question, and that you'll enjoy episodes of SciGirls.
Search for SciGirls on YouTube to watch anywhere, any time!
Claire Quimby
Facilitator Research Associate
May 18, 2017 | 06:48 a.m.
Thanks, Richard - that does answer my question. I wanted to add that one thing I really like about this study is that the control group also took part in Frog Watch and had that hands-on experience. Sometimes the control group in educational studies is just a group of students listening to a traditional lecture or lesson from their teacher, which naturally sets up the treatment group for higher levels of engagement and success.
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Sarah Carter
Richard Hudson
Co-Presenter Director of Science Production, Retired
May 18, 2017 | 08:41 a.m.
Claire - I agree that controlled studies often suffer this flaw. We went one step further: the girls in the control group, after their hands-on Frogwatch experience, watched the SciGirls media, with followup questions. So this gives even more insights into the positive effect of media engagement.
Very interesting project. It's great to see how video products can engage young children in a positive manner. Do you have any idea if watching SciGirls makes girls (or boys) more likely to participate in citizen science projects that are not profiled in the videos?
Within the treatment group that was exposed to SciGirls, minority girls demonstrated significantly higher interest than non-minorities in finding out more about other citizen science projects; greater likelihood to look for a future citizen science project to do; and stronger belief in their efficacy to be good at doing other citizen science projects.
Deborah Hanuscin
Professor
May 18, 2017 | 09:36 a.m.
What a great partnership and fantastic resource!
1
Discussion is closed. Upvoting is no longer available
Found helpful:
Sarah Carter
Maureen Holden
K-12 Teacher
May 18, 2017 | 10:16 a.m.
Fantastic Project! The evidence shows that with great role models students will be more engaged in their learning and they themselves will become great role models
1
Discussion is closed. Upvoting is no longer available
Sarah Carter
Manager, STEM Media & Education
Welcome! Thanks for viewing the video about Citizen SciGirls. We welcome all comments, questions, and feedback about our project. As this project is complete we're particularly interested in hearing about audiences that would be interested in this research and other projects that are looking at the role of video in engaging youth in informal education experiences.
Richard Hudson
Director of Science Production, Retired
The series of SciGirls episodes on Citizen Science was a rich collaboration with the Citizen Science group at the Cornell Lab of Ornithology and with six different CS programs across the nation. It was a tremendously exciting learning experience for all of us on the SciGirls production team. It also afforded us the opportunity to collaborate on a unique, controlled experimental evaluation of how media can stimulate and enhance a live hands-on citizen science experience. (See http://citizenscience.org/2016/12/06/research-s... ) With the myriad ways that young people can access video today -- on their tablets and smart phones, as well as traditional television -- video can be an even more important component of STEM learning. While we have always known that video can produce both interest and learning, it has not been clear how video can contribute to a richer and deeper hands-on inquiry experience. This research demonstrates the power of this synergy. We hope STEM educators may be inspired to find other ways to integrate video into their hands-on STEM activities, and we look forward to the discussions this week. - Richard Hudson, Executive Producer
Sarah Garlick
Director of Science Policy and Outreach
This is great! The study design and results of the control and treatment groups are really interesting. I'm curious to know how closely tied the FrogWatch sessions were with the SciGirls media, in terms of content and/or other programming attributes. In other words, do you think other existing citizen science programs could have similar success by leveraging the same SciGirls media (assuming permissions)? Is this a possible avenue for scaling up? Or do you envision replication involving citizen science programs developing their own customized video content?
Sarah Carter
Manager, STEM Media & Education
These are great questions. The treatment girls watched three episodes of SciGirls prior to participating in FrogWatch including one episode that showed a group of four friends learning about and participating in FrogWatch specifically. Because the episodes show some good examples of the process that you can go through when participating in citizen science we think they are a great example no matter what type of citizen science project you may be participating in. The recommendations for the citizen science field from our evaluator, Dr. Barbara Flagg are below:
We haven't discussed scale up yet but these are some interesting ideas. Thanks!
Heidi Carlone
Sarah Garlick
Richard Hudson
Director of Science Production, Retired
As Sarah mentions, the treatment group in the evaluation watched two other episodes besides the one showcasing Frog Watch, which they then did. The control group only did the live project activity.
Each episode of the series features a different group of girls and a different Citizen Science project. And in each episode, we intentionally introduced the basic idea of participatory, Citizen Science. That is perhaps why the treatment group demonstrated a deeper understanding of what citizen science is than the control group, which only took part in the hands-on training and data-gathering. (The control group subjects had the opportunity to watch the videos after their experience, and were evaluated a second time as part of the study.)
So we would imagine that the episodes would serve any educator who wanted to introduce youth to citizen science, regardless of what their project might be.
The other useful part of the episodes is the extension activity. In some of the episodes, the girls created a presentation about their research and shared it with the public (Frogwatch: https://goo.gl/f7Yyh6 ). In other episodes, the girls conducted a related inquiry that they designed themselves (Monarch Watch project: https://goo.gl/acKffg ). We believe such extension activities can give added meaning and value to the citizen science data gathering and reporting.
We very much look forward to hearing from people guiding youth in citizen science experiences about who may use these videos.
They are universally accessible: on PBS Learning Media, on the SciGirls website, and even on a YouTube channel. Girls (and boys!) can watch the episodes on their tablets or smartphones as part of a live CS experience. We'd love reports from the field on how this works! - Richard
Sarah Garlick
Heidi Carlone
Professor
I admit that I've been a fan of SciGirls for a while now. :) The SciGirls show is so compelling. I love the idea of diverse girls, working together, to solve a problem that demands scientific thinking, and the emphasis on citizen science is empowering. I'm wondering: How long did each project last? Has your team been able to track the girls over time? What happens to their long-term interests, if anything? I study science identity, and "recognition by others" is a big part of developing and sustaining science identity. The video component of this really showcases girls' scientific thinking in the best possible way, and is a stunning example of garnering "recognition by others". I'm interested in the girl participants as well as those who watch(ed) the program regularly. What a wonderful example of the power of video!
Sarah Carter
Manager, STEM Media & Education
When shooting an episode we work with the girls for about six days so it's not unlike an intensive summer camp experience. The difference being they have great opportunities to work closely with a STEM mentor and get some unique behind the scenes experiences. Unfortunately we haven't had the funds to do any official follow up research or evaluation on girls that have been in SciGirls episodes but we've talked about it.
Our current season and outreach project Latina SciGirls is looking at middle school-age Hispanic girls' positive STEM identity development. The research study being done by the University of Colorado Boulder will test the hypothesis: The SciGirls model, when augmented to address specific barriers to STEM engagement of Hispanic girls ages 8 to 13 and their parents, will promote the development of positive STEM-related identities in Hispanic girls. The study will investigate Hispanic girls' personal experiences engaging with new episodes featuring bilingual girls and mentors and bilingual activities and how those experiences contribute to their STEM-related identity development against cultural and gender-based stereotypes.
You can find more information on Latina SciGirls at: informalscience.org
Julia Griffin
Heidi Carlone
Heidi Carlone
Professor
Sarah,
Oh, that's great! I'm so glad to hear that identity-related outcomes are of interest in this next iteration of SciGirls. I haven't seen the LatinaSciGirls program-- I'll have to look for it now.
Sarah Carter
Richard Hudson
Director of Science Production, Retired
The Latina SciGirls episodes won't be released until next year, but all of our episodes are available with lip-sync Spanish voices on PBS Learning Media. What will distinguish our new episodes is that they are all filmed with native Spanish speakers!
Heidi Carlone
Richard Hudson
Director of Science Production, Retired
The published research on SciGirls Season 3 includes a lengthy discussion of "self-efficacy," which is key to the development of a science identity. You can find the report here:
http://theoryandpractice.citizenscienceassociat...
The research was a controlled study of two groups of viewers: one group watched SciGirls media prior to doing the Frogwatch investigation; the control only took part in the Frogwatch investigation, with introduction by the instructor.
In addition to the findings mentioned in the video, Barbara Flagg wrote;
Within the treatment group that was exposed to SciGirls, minority girls demonstrated significantly higher interest than non-minorities in finding out more about other citizen science projects; greater likelihood to look for a future citizen science project to do; and stronger belief in their efficacy to be good at doing other citizen science projects.
She also notes that this is a finding worthy of further study!
Heidi Carlone
Sarah Garlick
Heidi Carlone
Professor
Richard-- I will definitely look at the report-- thanks! Great work, all around.
Claire Quimby
Research Associate
This sounds like a great project, and your video is really compelling too. Did you talk with the girls at all about their reactions to the videos? Curious to know if (aside from the information conveyed about citsci) any particular elements really resonated with them and maybe contributed to their increased engagement with the Frogwatch activity.
Richard Hudson
Director of Science Production, Retired
The Season 3 research (see the link earlier in this thread) focused very specifically on engagement and understanding of citizen science. We attributed the engagement to the research-based strategies that we use in all of our SciGirls work, which we call "The SciGirls Seven." (See http://www.scigirlsconnect.org/scigirls/ )
An earlier evaluation analyzed how viewer comments about the episodes aligned with some these strategies:
• One strategy suggested that girls are motivated by projects they find personally relevant and meaningful. More than half (55%) of the SciGirls viewers felt the engineering design projects were important or relevant to their personal interests.
• Another strategy proposed that girls are motivated when they can approach projects in their own way, applying their creativity, unique talents and preferred learning styles. Half (50%) of viewers liked how the real girls used their own creativity and talent to solve problems and complete the engineering projects.
• A third strategy noted that girls benefit from collaboration, especially when they can participate and communicate fairly. Almost half (45%) of the SciGirls viewers were energized by the teamwork and constructive interactions of the real girls.
• A final strategy indicated that girls benefit from relationships with role models and mentors. One-third (31%) of viewers identified the onscreen girls as role models for themselves and/or noted the help of the onscreen mentors in the real girls’ projects.
(The remaining 3 strategies are techniques that educators should use when working with girls, and as such are less visible in the shows themselves.)
Hope that answers your question, and that you'll enjoy episodes of SciGirls.
Search for SciGirls on YouTube to watch anywhere, any time!
Claire Quimby
Research Associate
Thanks, Richard - that does answer my question. I wanted to add that one thing I really like about this study is that the control group also took part in Frog Watch and had that hands-on experience. Sometimes the control group in educational studies is just a group of students listening to a traditional lecture or lesson from their teacher, which naturally sets up the treatment group for higher levels of engagement and success.
Sarah Carter
Richard Hudson
Director of Science Production, Retired
Claire - I agree that controlled studies often suffer this flaw. We went one step further: the girls in the control group, after their hands-on Frogwatch experience, watched the SciGirls media, with followup questions. So this gives even more insights into the positive effect of media engagement.
Julia Griffin
Very interesting project. It's great to see how video products can engage young children in a positive manner. Do you have any idea if watching SciGirls makes girls (or boys) more likely to participate in citizen science projects that are not profiled in the videos?
Sarah Carter
Manager, STEM Media & Education
Good question Julia. You can read an article written by Barbara Flagg, the evaluator on this project here: Contribution of Multimedia to Girls' Experience of Citizen Science
Excerpt from the article:
Within the treatment group that was exposed to SciGirls, minority girls demonstrated significantly higher interest than non-minorities in finding out more about other citizen science projects; greater likelihood to look for a future citizen science project to do; and stronger belief in their efficacy to be good at doing other citizen science projects.
Deborah Hanuscin
What a great partnership and fantastic resource!
Sarah Carter
Maureen Holden
Fantastic Project! The evidence shows that with great role models students will be more engaged in their learning and they themselves will become great role models
Sarah Carter
Kathryn Guimond
Great work Sarah, so proud to be your colleague and excited to present at ASTC together! Kathryn
Sarah Carter
Further posting is closed as the event has ended.