Great work! I support your efforts to learn about birds in hopes that it will make folks more invested in conservation.

Thanks for your question, Erica! As part of the evaluation for this project, we did ask participants to share some demographic information about themselves, their experiences with citizen science and watching wildlife cams, and how interested they'd be in participating in different stages of the scientific process.  

Thanks for watching and thanks for your question Erica.  Like Ben mentioned, we have collected demographic info on participants (at least those who've been willing to engage as part of the research/evaluation process.

We've learned that those participating (and sharing info about themselves) tend to be slightly older--with more than 60% of Hawk Talk survey respondents saying that they are age 55 and 51% of Battling Birds survey respondents). Data pre-dating this project seems to suggest that many people in the Bird Cam community are coming to be involved somewhat later in their lives-- and find that it is a satisfying activity during retirement or in instances where they might be homebound for various reasons.

Participants/survey respondents have also tended to be female (83% female in the Hawk Talk project/study and 74% female in Battling Birds). It also tends to be a fairly homogenous group race/ethnicity wise, with as many as 89% describing themselves as "Caucasian" on the Hawk Talk survey.  But, a good number of participants do seem to be from countries other than the US (which means some participants see things like a study of a bird feeder in Ithaca, New York as a chance to see new and exotic birds.) 

In addition to other demographic characteristics, we are also asking participants to share info on the extent of their past bird cam watching and birding experience, and about their science knowledge/experience (including professional experiences). We have also been exploring ways to measure bird/bird biology knowledge as another variable that may have an impact on people's experiences or lead to different outcomes as a result of participation in this project.

Happy viewing!! 

Hi Dave, thanks so much for your question about K-12 integration. I've reached out to our K-12 team to ask about the latest numbers for their downloadable lessons for "Life in a Nest." We're also excited about the future potential for engaging classrooms in activities in which students can become contributors shaping the investigation. One can imagine live data annotation tools also being made available some day so classrooms could conduct their own collaborative investigations using wildlife cams.

Hi Overtoun, your question gets at the heart of something that we think has a lot of potential if such projects can be scaled up to achieve comparative studies rather than studies focusing on just a single cam. With our project in its infancy, we haven't moved beyond collecting data from individual cam locations, but we imagine that some day there could be networks of communities and cam operators that would enable data collection from, say, 60 different Osprey nests around the world. With a meaningful sample size like that, participants could help make discoveries to illuminate so many interesting factors related to behavior, ecology, geographic variation, and as you suggest, habitats such as urban vs. more natural settings.

Hi Bradley—thanks for your note and questions! The research team does work with the participants to help refine the investigation questions before the community takes a vote. One key is to steer participants towards questions that we might actually be able to answer based on what we can observe on cam. For this project, we devised a sorting activity, where participants could sort different questions based on whether they thought they were cam-testable or not. After we figured out which questions could be answered with the cam, we worked with participants to refine questions further so they they were specific, interesting, and measurable enough to move forward with an investigation. From there, the community voted! 

In this particular project, there was a heavy interest in investigating the types of vocalizations that hawks make in the nest, and if these vocalizations were related to certain behaviors. Overall, we didn't get much feedback from folks who were disappointed that the questions they were most interested in didn't get chosen. If we had the time and resources, we wish we could investigate all of the questions that were proposed, because they were really good! 

Hi Bradley, thank you for your questions! Each time we start an investigation, the community posts thoughts and potential investigation questions on the Wonder Board. From there we give them the opportunity to learn more about which questions are answerable on the cam and how to refine them further with a sorting activity, live question and answer, and Question Design Board. From there, we summarize the discussions and have them vote on a narrowed list of questions. The question that rises to the top is the one that we investigate. 

We did advise participants along the way, trying to provide answers while also guiding them to think about things on their end. Interestingly, we continue to find that participants will default to the scientists. While that makes sense, we found ourselves surprised that just how often that comes up.

We haven't had any sense if participants feel negatively if a question is not chosen, but should consider that as a possibility when designing our evaluation. Thank you for bringing that up! 

Great question, Michael! We know that a subset of participants are "super users" that are involved with the project every step of the way. Others may only engage with one step of the process, such as collecting data. We track participant activity when we are able to, but we also rely on survey data where participants self-report their involvement in each step of the process.

Thank you Saira or your comment. Leveraging the existing Zooniverse platform has enabled us to use their strong platform without re-inventing the wheel and reach a much wider audience, while also enabling us to innovate new technologies related to live video annotation. The partnership with Zooniverse has been a win-win for all!

Hi Melanie,

Thank you for your comment. We are learning a lot about the co-created process and where people are most comfortable in participating in the science process. The Nest Cams have been used in formal and informal settings with youth - they are hugely popular with educators and students alike. Here is a free set of downloads for using the cams with youth in grades 3-5: https://www.birds.cornell.edu/k12/life-in-a-nest/. We love to hear from educators, so please let us know how you use these materials! 

Hi Catherine, thanks for watching the video and for your question. We are approaching learning from a couple different angles. 

The self-rated approach (i.e., asking participants if they learned/what they know): In surveys with Hawk Talk project participants we asked respondents to indicate how much they'd learned about conducting research using bird cam footage -- this self-rated assessment of learning indicated that all but 8% of respondents felt they had leaned new things about doing this type of research. For this year's Hawk Cam Study, we've also added questions that ask participants to rate how familiar they are with the lives of Red-tailed Hawks, how comfortable they are in sharing their knowledge of Red-tailed Hawks with others, and the extent to which they feel they can make valuable contributions to the scientific study of birds. 

The reflective approach (i.e., asking participants to share info about what they'd learned): Interviews and open ended comment boxes allowed participants to share examples of the types of things they had learned. For the Hawk Talk project, participants noted that they learned a great deal from the Cornell scientists. Some participants said that while they value the citizen science model, citizens need guidance from researchers who can underline the importance of systematic data collection and a rigorous investigation process.

The knowledge-based approach (i.e., asking questions to assess what participants learn): more recently, we've been piloting a series of knowledge questions as a more objective way to assess learning.  In our first attempt we found that the questions generated were too easy and therefore didn't provide enough room for participants to exhibit knowledge acquisition as a result of program participation.  In our most recent attempt (i.e., for this year's Hawk Cam study), project team members succeeded in creating a series of questions that did result in some variation across survey respondents and we look forward to seeing on post-participation survey if participation in the program lead's to more correct responses on the post-participation survey.

The transfer-based and related activity-based approach (i.e., what does newly acquired knowledge or other aspects of the participation experience prompt participants to do?):  All of our participant surveys have asked a series of questions about participants' bird-related behaviors, e.g., feeding wild birds, making donations to organizations that helps birds or other wildlife, volunteering to help birds, watching birds outside or at a feeder, watching live cams featuring birds, taking action to help protect birds, reading about birds, reading about science content online or in print, and participating in citizen science projects. This is another area where it can be challenging to see changes that result from participation because the desire to participate in projects like "Hawk Talk" seems to correlate with many of these behaviors. We're keeping our eyes peeled nonetheless!

Thank so much for sharing our project, Krista! We think the Bird Cams Lab is a great platform for K-12 students to join in on the co-creation process. PBS Learning Media also created some great resources that are in line with the project as part of PBS NATURE's American Spring Live event last year. You can check them out here: https://www.pbslearningmedia.org/resource/asl19-sci-birdcams/citizen-science-for-your-classroom-cornell-bird-cams-lab/support-materials/.

Some of the content is geared towards a previous investigation that we were running titled "Battling Birds", but there is a lot of good evergreen content available here. And of course, new investigations are currently underway! 

Hi Leigh, thank you for your comment! We too were surprised by some of our early findings that suggested people were not confident in their abilities to ask questions suitable for an investigation. In the natural cam environment, people ask questions about the birds all the time. But when we ask them to formulae questions to be considered for a scientific investigation, there was hesitancy. This highlighted a few things for us. First, an acknowledgement that asking scientifically valid questions is hard, even for scientists! Second, some individuals felt that they did not have the right "credentials" for posing such questions. Third, engaging audiences that do not think of themselves as scientists to formulate scientific questions, often requires facilitation and support as well as enthusiasm and encouragement. It also reinforced the notion that even in a co-created project, not everyone has to engage in every aspect of the science process. Co-created projects allow for the opportunity to engage in all aspects of science, but in most cases, people will pick and choose the activities where they feel they can best contribute. I found similar results in my own dissertation examining learning and engagement in six different citizen science projects. 

Hi Michael--thank you for asking about the learning outcomes. We're interested in the "reader to leader" framework and the question of whether and how participants may progress through activities with differing levels of involvement, and whether some ultimately emerge as super-user contributors or leaders within the community. Are participants more motivated to engage when they can influence the process and outcomes of research, and do they learn more along the way? We don't know the answer to this yet, but so far we are finding that different participants seem to have different comfort zones--some may really enjoy engaging with scientists and other viewers to shape the outcomes; others may be happy to contribute by collecting data; and still others might be happy to vote on which direction to take, or to explore findings, but aren't interested in engaging in conversation. At the beginning, I think we had an idealized notion of progression through these activities but now that we are seeing patterns of participation, it seems more obvious in retrospect that not everyone needs to participate in every phase to contribute or learn in valuable ways. Just as happens with professional scientific teams, different people may provide different specialized skills, knowledge, or efforts to create a collaborative whole. One of our biggest questions is whether participants are perceiving the entire effort, or just their own part. Our evaluations in the coming year will focus on understanding what different kinds of contributors are taking away from the experience, in terms of learning, skills, and attitudes, and to understand what effect if any their participation had on those outcomes. We are assessing this using several approaches including pre-post surveys that include self-reported data and knowledge-based questions; Zoom interviews; and a look at pathways/journeys for a subsample of participants.

Thanks for the question Michael! In addition to the Reader to Leader framework that Miyoko mentions, we are also using design-based research principles for creating and supporting socio-technical environments for mass collaboration (Fischer 2016). For example, the design process allows for the inclusion of at least some low-threshold activities while creating a high ceiling for progression from simple tasks requiring little time to more complex tasks. Participants should be able to participate in one or more activities that fit in with the others, enabling flexibility with timing and level of effort, and the possibility of progression to more involved roles. 

With respect to your question about students, while we are aware that teachers use the cams in their classrooms, our research study does not include students, thus we are not currently assessing student learning outcomes. Given the interest in using the cams in the classroom however, future work may focus on student learning! 

Thanks for commenting, Emily! Please do share with your team. We've known for a long time that observing birds on the cams sparks people to ask all types of questions about the things that they see, and we are excited to get to dive deeper into investigating those questions with the community in this project. If you are thinking about integrating the cams into your curriculum, you should also know that we have a lot of resources available for teachers:

The Cornell Lab's K-12 program has created some free lesson plans that utilize the cams. 

PBS Learning Media also created some great resources that are in line with the project (including some "I Wonder" activities) as part of PBS NATURE's American Spring Live event last year. 

Thank you!

Hi Jeanne, thanks for your question about demographics and expansion to new audiences. We would like to see more diversity in participation, but we haven't explored this since we are first testing how well we do with our existing core audience, and learning what we need to do to create a coherent and rewarding experience. So far our challenges have ranged from learning how to enable participants to follow a project through time, since each investigation has several phases and requires months to complete; how to support learning along the way; and how to accelerate research design to enable timely collection, analysis, and presentation of data. By the end of the project next year, we hope to understand what works for this group, and gauge whether to seek funding to tailor these experiences to reach and engage additional participants. It's been heartening to see the interest and enthusiasm in this forum for adaptation of these opportunities for K-12 classrooms!

Hi Holly--thank you for asking about the participation rate. We have found that the number of participants varies quite a lot depending on the activity. For example, with our pilot project, Hawk Talk, about 220 people signed up for the project, enabling us to stay in touch with them by email as the project evolved. Of those, 37 contributed to generating and debating 50 questions for possible investigation; 17 joined a live Q&A, plus 40 additional views of the recorded session; 60 participated in a question-sorting exercise to identify which questions were feasible to test using the cams ; and more than 100 people cast votes to select the question. A much larger number--1,440--collected data using the Zooniverse platform. We haven't yet analyzed how many people participated through more passive activities such as reading the email updates and reports. With the pilot project complete, we're now scaling up the next projects with larger recruitment efforts to see if we can attract and retain larger number of participants. After seeing the different rates of participation for different components of the project, we're very curious to learn more about how learning, skills, and attitudes may or may not differ depending on these different types of participation.

Excellent!  Many thanks for providing these details! And I agree it will be interesting to see how (or how not) different skills/attitudes differ depending on rates of participation. Perhaps the collection of the data is an "easy" step, but the other parts require more of a commitment, which might be something folks super interested in the project will follow through with, but others will not.  But then again, what pushes those other people to take that "extra step".  Will you be conducting participant surveys to gauge these different participation variables?  All very interesting indeed! (what drives participation, etc.). Good luck as you scale up!

Hi Holly! In our most recent investigation we had participants take a pre-survey and will be rolling out a post-survey at the end. We'll also be conducting interviews to get at some of of the more nuanced reasons why people may be taking that "extra step" you mentioned. We're excited to see what we can uncover!

David, that is a great question. We go back and forth with participants on multiple discussion boards as we narrow in on the questions and summarize discussions in blog posts. While we can't highlight every question brought up, we do try to touch on themes and the pros/cons of pursuing them. As we narrow in and then vote, we always try to communicate that all the ideas and questions are great, but as in any scientific investigation we need to pick and choose what we'll do. 

In choosing the final wording of the question and how we will collect data, we've leaned more on the researchers so that we can make sure it is understandable across participation levels. Hopefully that answers your question.

Hi Meena, that is a great question about what we'd choose to modify about our approach. One thing I'd hypothetically love to see is the ability to include live data collection and live Q&As directly on the live-streaming pages where people are watching the cams. Currently, data collection happens in a separate Bird Cams Lab space and our Q&As have been on Zoom. Our investigation space is separate from the normal cam viewing pages so that we will have the ability to ask people to sign up so we can stay in touch with them about activities and progress; and to provide an environment specifically focused on the investigation, since not all viewers may be interested in that aspect, and we didn't want to disrupt their normal viewing experience. If we had more time and capacity, it would be interesting to design an environment that could expose the data collection to this broader audience and test if that can be done successfully. The other thing would be to create a more seamless online experience and greater agility throughout the process. Currently, we offer two forms of data collection--live and archived--on two different platforms designed for that, but that requires asking participants to log in to different environments and user experiences. Additionally, we'd love to see a more seamless way to collect live data as a form of rapid "pre-processing" of information that enables us to surface clips of interest for more detailed data collection during archived "post-processing," a method that remains time-consuming and challenging at this time. It is heartening that despite these obstacles, we are able to bring participants along through the entire process, but evaluations will help us assess the degree of effectiveness in participants' understanding and learning along the way.

Hi Betsy, thanks so much for signing up! It's great to hear how your program is using a multifaceted approach to connect different branches of science and promote learning.