Thanks for sharing your project! :)
I’d love to hear about how you think about understanding the cumulative impact of the various SciGirls initiatives. Some kids watch the show, some kids participate in the out-of-school programs, some kids do both. – A similar question – How do you think about refining the various interventions to be more effective? What data do you collect to drive those decisions?
Thanks!
- Colleen

Great questions!

Cumulative Impacts – SciGirls is featured on 91% of U.S. TV households, with a 100% market penetration in the top 25 markets. SciGirls episodes have been broadcast over 44,500 times on 481 public television channels. Based on available Nielsen audience measurements, SciGirls has achieved an estimated 25 million gross viewer impressions. Online at PBSKids.org, we average 300,000 visitors a month to play games and watch videos. For outreach, in 2014-15, our partners reported 8,797 girls (66%) and 2,999 boys (34%) participated in a SciGirls program, for a total of 11,796 students in 345 SciGirls programs. Overall, since 2011, we have had nearly 40,000 youth participate in one of 800 reported SciGirls programs.

All SciGirls activities are based on an established body of educational research that has become the foundation of– the television series and our training for STEM educators and role models. A coherent set of educational strategies, drawn from two decades of research which apply to girls from grade school to college have been collected into a set of guiding principles in the publication, SciGirls Seven: How to Engage Girls in STEM (http://scigirlsconnect.org/page/scigirls-seven) that describes the strategies and provides tips on using them in educational settings.

SciGirls is committed to ongoing, rigorous evaluation to inform our work! SciGirls programs combine gender-equitable, inquiry-based teaching strategies and outreach activities that have proven to increase middle school girls’ interest in STEM and improve their attitudes toward these fields. Dr. Barbara Flagg of Multimedia Research conducted summative evaluations of SciGirls Seasons One and Two, and found that the videos increase interest in and understanding of scientific inquiry and engineering design. Viewers found the shows fun and educational, and felt that seeing successful female onscreen characters amplified their confidence in their own abilities. Knight-Williams Inc. conducted summative evaluation of SciGirls outreach efforts. Her evaluations showed that informal educators believe SciGirls resources are reliable tools that build confidence, deepen STEM skills and spark a passion for science. Our evaluations can be found at http://www.tpt.org/science/evaluations/

A recent summative evaluation of Season 3 that focused on episodes of citizen science showed that girls who watched episodes and played online games prior to an outreach experience helped girls understand the features of the practice of citizen science, e.g. that citizen science brings together a wide community of scientists and volunteers to work together and share data to which the public, as well as scientists, have access. In addition, minority girls (in this group) had higher interest in finding out more about other citizen science projects, greater likelihood to look for a future citizen science project to do, greater perceived efficacy in doing other citizen science projects, more similarity to the video girls, and stronger interest in their SciGirls experience. This is interesting in that it connect our media work and our online activities directly to the outreach space.

How do we refine our SciGirls interventions to be more effective? Recently, our ‘SciGirls Strategies’ project allowed us to create a new web series of role model profiles that are more explicit about challenges/barriers/strategies in response to research that supports the efficacy of that tactic.

As noted, our fourth season will be filmed in Spanish as a response to evaluation data from two prior projects NSF funded, SciGirls en Español and SciGirls en la Familia, in which the evaluation showed the programs positively influenced participants and raised families’ awareness of opportunities available to their daughters but also showed a strong desire for episodes featuring Spanish-speaking girls from Hispanic communities.

Our newest award, SciGirls Code is in response to the dropping numbers of young women and women in technology and computing!

Two questions about this great program/project:
(1) How do you identify girls to be featured in the program? Is it a steady set of girls, or do they vary across episodes?
(2) How do you measure impact of a program like this? Is “impact” primarily about impressions, or do you also look for learning in some more concrete sense?

We measure the impact of the show in a few ways through both research and evaluation. Dr. Barbara Flagg of Multimedia Research conducted summative evaluations of SciGirls Seasons One and Two, and found that the videos increase interest in and understanding of scientific inquiry and engineering design. Viewers found the shows fun and educational, and felt that seeing successful female onscreen characters amplified their confidence in their own abilities.

Our evaluations can be found at http://www.tpt.org/science/evaluations/

Here’s how we think about the “net” impact: If you look at the percent of viewers who show strong positive learning outcomes, ranging from roughly 40% to 80% depending on the metric, then you can extrapolate from the number of views. That gives us confidence that we have produced positive outcomes for over 10 million girls over the three seasons of SciGirls, plus secondary impacts on educators and their students through outreach.

This is a fascinating project, with multiple components and the potential for such wide reach! I am very excited to look more thoroughly at the research findings you have linked to in this thread.

Have you done anything with the data collected from the online component? Do you have a model or an idea of how the components fit together for participants (i.e. there is a TV media component, but then actual activities online, as well as resources for further exploration on the web; I am so curious as to how girls navigate these, and if different communities with different types of access navigate them differently. Exciting work!

This is such an interesting program with broad reach and impact. I’m really interested in Caitlin’s question about the on-line component. I’m also interested to know how the questions or challenges for each episode are identified. Do youth provide input on this? I’m looking forward to going on-line to watch the new role model profiles Rita mentioned above!

We definitely work with girls to determine our web-based questions and challenges. Our primary online experience has youth choosing STEM activities to do in the real world and telling others about it, these quests and our monthly citizen science challenges came from focus sessions with youth and the input of STEM educators in the field.

For our current season we reached out explicitly to a large group of girls and parents about themes and topics of interest to them (aka student/family advisory committee). Topics are selected based on a variety of inputs including diversity of professions and fields of STEM study and input from our on the ground focus groups, our advisory board and our educators around the US.