3829 Views (as of 05/2023)
  1. Brian Kruse
  2. Director, Teacher Learning Center
  3. Presenter’s NSFRESOURCECENTERS
  4. Astronomical Society of the Pacific
  1. Elysa Corin
  2. http://www.instituteforlearninginnovation.org/Staff/elysa-corin/
  3. Senior Researcher
  4. Presenter’s NSFRESOURCECENTERS
  5. Institute for Learning Innovation
  1. Suzy Gurton
  2. Assistant Director
  3. Presenter’s NSFRESOURCECENTERS
  4. National Radio Astronomy Observatory
  1. Anna Hurst
  2. Director of Museum, Park, & Library Programs
  3. Presenter’s NSFRESOURCECENTERS
  4. Astronomical Society of the Pacific
  1. Kari O'Connell
  2. Senior Researcher
  3. Presenter’s NSFRESOURCECENTERS
  4. Oregon State University
  1. Dennis Schatz
  2. http://www.dennisschatz.org
  3. Senior Fellow
  4. Presenter’s NSFRESOURCECENTERS
  5. Institute for Learning Innovation
  1. Linda Shore
  2. CEO
  3. Presenter’s NSFRESOURCECENTERS
  4. Astronomical Society of the Pacific

On-The-Spot Assessment to Improve Scientist Engagement with the Public

NSF Awards: 1811022

2021 (see original presentation & discussion)

Graduate, Adult learners, Informal / multi-age, All Age Groups

When approaching working with public audiences, it is important to remember that they are more than just passive recipients of information. Choices the presenters make, and how they design and facilitate an activity, can significantly impact the learning experience and satisfaction of the individual.  Our On The Spot Feedback (OTSF) project was originally designed to enhance the ability of scientists to better interact with their audience during face to face presentations.  The basic idea was to train them to use a variety of tactics to gauge audience response, make adjustments in real time to their delivery, and to signal to an audience that their questions, perspectives, and experiences are valued in the interaction.  A side effect of the COVID-19 pandemic was to eliminate face to face interactions between scientists and the public.  While not designed to address virtual presentations, the OTSF team quickly made adjustments to take the project entirely online.  Drawing on our partners’ extensive experience delivering online professional development, the team began engaging OTSF participants in learning how to actively interact with a virtual audience.  Participating scientists have a keen interest in outreach, and are playing an important role in the development, and testing of audience engagement and assessment tactics.  Implementation of these tested tactics will hopefully result in better presentations, and greater audience understanding of both the findings and process of science. 

This video has had approximately 355 visits by 253 visitors from 128 unique locations. It has been played 153 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2021 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2021 STEM For All Video Showcase (14 posts)
  • Icon for: Brian Kruse

    Brian Kruse

    Lead Presenter
    Director, Teacher Learning Center
    May 10, 2021 | 04:03 p.m.

    Hello Everyone and thank you for viewing our video!

    Originally designed to research and develop tactics presenters can use during in-person outreach, our On-the-Spot Feedback project pivoted to helping scientists actively garner feedback during virtual presentations.  We would love to hear from you about tactics you use to actively engage participants, both virtually and in-person, particularly how you use feedback from your audience to make "on-the-spot" assessments and adjustments to your presentations. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Suzy Gurton
  • Icon for: Chip Bruce

    Chip Bruce

    Facilitator
    Professor Emeritus
    May 11, 2021 | 07:38 a.m.

    It's notable how your project and several others have had to adapt to covid times. That's in turn led to new techniques and reflection on communication per se.

    Your comments about teaching scientists how to garner feedback sound excellent. It reminds me of an old paper by Easley and Zwoyer on "teaching by listening." Their point, as I recall, was not only that teachers can learn better what students know and what their difficulties are, but also that the student learns by articulating their understanding.

    Do you have ways to measure the growth in the scientists' understanding? Are they comfortable making the adjustments needed?

     
    1
    Discussion is closed. Upvoting is no longer available

    Shihadah Saleem
  • Icon for: Kari O'Connell

    Kari O'Connell

    Co-Presenter
    Senior Researcher
    May 11, 2021 | 05:55 p.m.

    Hi Chip,

    For the research portion of the project, we are investigating change in scientists' motivation, perceptions, and self-efficacy. Our research instruments are based on on Ajzen’s theory of planned behavior (Ajzen, 1991, 2005), assuming that increased self-efficacy in engaging with audiences may result from improved communication skills, ultimately changing scientists' outreach practice towards stronger interactive audience engagement strategies.

    We are just launching the field tester part of the project after the initial design testing phase, which we used to develop both OTS tactics and the research instruments.

    Let me know if you have other questions,

    Kari

     
    1
    Discussion is closed. Upvoting is no longer available

    Suzy Gurton
  • Icon for: Nancy Staus

    Nancy Staus

    Facilitator
    Senior Researcher, STEM Education
    May 11, 2021 | 11:13 a.m.

    Improving communication of science to the public is so important! Can you share some of the on-the-spot tactics you recommend to participants and the kinds of adjustments that are recommended during a presentation based on audience responses?

     
    1
    Discussion is closed. Upvoting is no longer available

    Shihadah Saleem
  • Icon for: Suzy Gurton

    Suzy Gurton

    Co-Presenter
    Assistant Director
    May 11, 2021 | 02:30 p.m.

    Hi Nancy, the tactics mentioned in the video include polling, drawing, modeling, kinesthetic activities, accomplish a task, think-pair-share and questioning. These are common engagement tactics, so the big challenge is to structure them to get useful feedback that will inform their next step, help to adjust vocabulary and complexity, or even adjust the goal of their presentation (simplify or go for it).
    A good example in the video is the drawing tactic. The scenario that we stated for the participants, was imagine a researcher has been asked to give a presentation on some aspect of diet. They could start off by giving their audience the outline of a human form and asking the participants to draw the digestive system. If all they get back is a tube from the mouth to a sack (the stomach) and some squiggly bits to where the waste goes out the other end, then they know folks aren't sure about the complexity of it all and where other organs (liver, gall bladder) and glands come into play. So they've got to spend a little more time on giving context, but if people are including 6-10 components, then they've got an audience that is ready to dig into details and complexity.
    Sometimes the tactic could be labeled multiple ways, for instance polling and building a model may be the same activity. If the goal is to see what your audience knows about the distances between objects, and you're in a virtual environment, you could put up a picture of the earth and ask everyone to use the stamp/annotate feature to put the moon at the same scale on the slide. Each participant is building a model of the Earth/moon system and you're taking a poll of the group based on their answers. Depending on the goal of the question, you could follow up with a request for what informed their choice or you might simply get a confirmation of a common preconceived notion that needs to be cleared up before you move on.
    Both of these examples are of getting some feedback of current understanding before moving on.
    Our OTSF guide has a deeper dive into each tactic with examples of how it might be used in a variety of sciences and examples of the goal of the tactic and what to do with the feedback you get.
    Hope that helps,
    Suzy

     
    2
    Discussion is closed. Upvoting is no longer available

    Sasha Palmquist
    Shihadah Saleem
  • Icon for: Shihadah Saleem

    Shihadah Saleem

    Facilitator
    Sr. Manager of Youth Leadership and Alumni Programs
    May 11, 2021 | 04:36 p.m.

    Really great project, it's so important for scientists and researchers to improve their science engagement with the public. I'm curious to know which fields of STEM needed more public engagement/understanding per complex topics, and which seemed  "easier" to digest (understand in laymen terms) and provide feedback? Can it be inferred that the topics discussed/learned in school (middle and high school) are better received from the audience?

  • Icon for: Suzy Gurton

    Suzy Gurton

    Co-Presenter
    Assistant Director
    May 11, 2021 | 05:09 p.m.

    Hi Shihadah,
    This is a great question, but we haven't done this specific analysis. I will say that we tried to make the training and materials not specific to any particular area of science, but applicable across fields. We did find that examples that were outside the participant's field of expertise were particularly great learning experiences, because it put the experts in the position of being the learner in a new content area. While training a bunch of astronomers, the examples of digestion, bird ID and immune systems had them all squirming and realizing what it is like to be an interested learner and not an expert. So, in the spirit of "know your audience" we tried to mix up our examples and use ones that were outside the content expertise of the participating scientists.
    Best wishes,
    Suzanne Gurton

     
    2
    Discussion is closed. Upvoting is no longer available

    Sasha Palmquist
    Shihadah Saleem
  • Icon for: Shihadah Saleem

    Shihadah Saleem

    Facilitator
    Sr. Manager of Youth Leadership and Alumni Programs
    May 11, 2021 | 09:34 p.m.

    Thank you for your response Suzy! Do the researchers/scientists in the project feel that they are better communicators overall? Have their immediate friends and/or family observed a change in the way they speak about their work? I know in my graduate work (years ago), as I spoke about my thesis to friends and family their eyes glazed over, I had to quickly learn how to pivot and re-engage on another level lol.

  • Icon for: Suzy Gurton

    Suzy Gurton

    Co-Presenter
    Assistant Director
    May 11, 2021 | 09:51 p.m.

    We're early enough in the project, that I'll reach out to one of our research partners to supplement this answer. Part of their protocol is to observe an outreach event where they are applying our strategy and tactics. I can tell you that this is building on a workshop that we have done for the last 7 years at the American Astronomical Society and those participants have told us exactly that. They are misunderstood in their families and they struggle to explain in simple terms what they are doing in their research. We have always encouraged them to take a step back and provide context for what they are studying and what motivates them and they have great stories about how it has helped them relate what they are doing. It certainly isn't universal. Some simply talk about the value of astronomy and not their specific area of research since it is so narrow and specific.

     
    1
    Discussion is closed. Upvoting is no longer available

    Shihadah Saleem
  • Icon for: Kari O'Connell

    Kari O'Connell

    Co-Presenter
    Senior Researcher
    May 12, 2021 | 04:25 p.m.

    We are investigating change in scientists' motivation, perceptions, and self-efficacy. Our research instruments are based on on Ajzen’s theory of planned behavior (Ajzen, 1991, 2005), assuming that increased self-efficacy in engaging with audiences may result in improved communication skills and ultimately changing scientists' outreach practice towards stronger interactive audience engagement strategies .

     
    2
    Discussion is closed. Upvoting is no longer available

    Shihadah Saleem
    Suzy Gurton
  • Icon for: Shihadah Saleem

    Shihadah Saleem

    Facilitator
    Sr. Manager of Youth Leadership and Alumni Programs
    May 12, 2021 | 07:58 p.m.

    Have you thought about ways to partner with museum educators/informal educators from science-based cultural institutions? More often than not, these on the spot communication techniques are employed and evaluated almost daily as they (we) engage with various audiences and learning levels. It could be a cool opportunity for all to have a share-out about best practices.

  • Icon for: Elysa Corin

    Elysa Corin

    Co-Presenter
    Senior Researcher
    May 12, 2021 | 08:16 p.m.

    Hi Shihadah, 

    Absolutely! Many on our project team have led former lives as museum/informal educators, and quite a few are still active in these spaces. Our own experiences using On-the-Spot communication in museum/informal environments have definitely influenced our model and our thinking. We're also working with museum educators at several museums across the country to offer our workshop training to the scientists from their communities that they partner with for outreach. In the future, we're hope our finalized model and training materials will not only be of use to scientists interested in improving their public engagement, but also to museum/informal educators and others who are heavily engaged in sharing STEM with the public. 

    Best,

    Elysa

  • Icon for: Vivian White

    Vivian White

    Informal Educator
    May 17, 2021 | 01:26 p.m.

    Great question! I'm excited to transfer these ideas into informal learning spaces when you all finish this phase. Thanks for the great work. It has been a joy to see it evolve into something maybe even more impactful with the pandemic changes. 

     

     
    1
    Discussion is closed. Upvoting is no longer available

    Elysa Corin
  • Icon for: James Callahan

    James Callahan

    Informal Educator
    May 17, 2021 | 06:39 p.m.

    Absolutely, we will indeed be following the OTSF program.

    As far as Twitter goes... literally:  Twitter:  https://twitter.com/astrosocietypac

    Astronomical Society of the Pacific has so long been the gold standard of engaging, involving science with the public.  It makes total sense that you continue to lead with the On the Spot Feedback program.

    It is always a pleasure to be with AstroSocietyPac at major STEM events, such as the Bay Area Science Festival in San Francisco.

    Like you, we have made successful adaptations to operations virtually.  In our case, with LIVE remote Climate Science Action Labs.  Clearly, these will continue, as with hybrid events long after the pandemic is over.

    It makes total sense: On the spot feedback is so essential during mass scale in-person events.  It has been essential to continue to get live feedback with remote and virtual events as well.

    Thank you, are very dear Astronomy for everyone friends!

    -- Mobile Climate Science Labs, proudly utilizing the resources of CLEAN.  Cleanet.org

  • Further posting is closed as the event has ended.