Thank you for coming to look at our video.  In this NSF-funded ChemAttitudes project, museum educators and researchers worked together using a process called “design-based research” to simultaneously revise a theoretical framework and a set of hands-on activities that embody strategies for increasing public interest, sense of relevance, and feelings of self-efficacy toward learning about chemistry. You can find links to all of the outputs of the project at https://www.nisenet.org/chemattitudes, including a digital version of the kit of materials distributed in 2018 and a Let’s Do Chemistry framework and strategies guide. Beyond all the material posted online, the project team has presented sessions and workshops at the Association of Science-Technology Centers and American Chemical Society professional meetings. Now we are working on a Let’s Do Chemistry Train-the-Trainer Workshop in order to spread the findings of the project throughout the chemistry community so that they can be put to use by chemists who engage with public audiences at National Chemistry Week and all kinds of other events. We were originally planning to run one-day hands-on workshops at a variety of professional conferences, but we are now developing a virtual workshop with four one-hour sessions and prep work to do before each.  We’d love your thoughts related to this current work:

• What are the best ways of reaching practitioners of informal education and public outreach broadly throughout a field like chemistry?
• Has anyone revised hands-on in-person workshops for the online virtual environment? 
• What has worked well for you, either as a presenter or a participant in a virtual workshop?
• What strategies have helped the workshop facilitate learning by doing, rather than just listening to presentations?

We, of course, have our own ideas about this but would love to learn from you.

Your video was very informative and your approach to educating students via hands-on experiences is certainly an excellent way to pique the interest of students.  A program at the University of Cincinnati targeted this approach by educating teachers on how to develop "hands-on" lessons for their classrooms.  As an extension of this work, STEMucation Academy was formed.  You can access more hands-on units of instruction at our website and more specifically at the following link (http://stemucationacademy.com/units/).  We have modified our training to included both in-person and online courses.  What has worked well for us?  We assign a personal coach to each teacher who takes our course to help guide them through our training program.  This is an added expense for our program but have found it to be effective way to deliver on-line training while still providing access to an expert via email, phone and texting. 

Hi Larry and David! Loved hearing about this project and am interested in hearing more about the development of the professional learning experience for teachers you are working on. The ACCT project that I am working with in the Boston Public Schools has recently shifted to virtual online PD due to the Covid crisis. You can find more information about the ACCT project  on the American Chemical Society’s ChemEd Xchange, www.chemedx.org/acct, and our team would be happy to speak more with you about the research and activities with teachers. The first few years we developed and offered face to face year long PD and next year we were planning to move to a hybrid model of online and face to face. However we decided to move to the online model for SY 2020-2021. We have a video featured  here as well--check out --Formative Assessment with a Bang!  http://stemforall2020.videohall.com/p/1809. Again great to see the work you are doing!

Can you tell me more about the kits and instructional materials you sent out to Museums?  Marianne

Hi Marianne!  I was just telling Sharon the other day about your project and wondering whether you were going to have a video.  Now I know!  I will check out your video later today.  It would be really interesting to think with your team about the interactions between what you have learned about formal ed interactions and what we've done with informal chemistry learning.

You can view all of the digital kit materials here, including the research-to-practice guide.  https://www.nisenet.org/chemistry-kit. Each activity contains training videos including example interactions/facilitations, along with activity and facilitation guides.  There are also a number of PD materials mentioned above.

Or here on the ACS site: https://www.acs.org/content/acs/en/education/outreach/explore-science-lets-do-chemistry.html

Thanks and hope you're all doing well during this crazy time!

Larry et al.

Nice to see your project.  As a former chemist, I appreciate this.  I think Joe Griffith did a similar chemistry project (for older students and adults?) at one of the Smithsonian Museums quite a while ago in the 1980s

George Hein

 I love how you have taken on some of the more challenging aspects of chemistry experiences (materials, mess, and safety) to help informal education leaders across the country help students get excited about chemistry! I'm interested in your ideas for virtual presentation. Have you considered using short video clips of what students would do hands-on, but stopping the videos at critical moments to ask students in an interactive online setting, "What happens next? What should we do next? What are your predictions?" If you've tried this or something like it, I'd love to hear how it went.

Hi Susan, I'm so happy to hear you appreciate our process!

While we as a project team are moving forward with our upcoming Train-the-Trainer workshops to help chemistry professionals learn how to implement the LDC framework and train others to facilitate hands-on activities, the Museum of Science has been moving full speed ahead with "Science in Action" livestreams where we do just what you are asking! Check out https://www.mos.org/mos-at-home to find some of our videos and livestreams using zoom webinar. David and I actually hosted a live chemistry demo where I asked viewers about ocean acidification and it was so wonderful to ask questions and have viewers respond in the chat. It seems to be working really well for us live. I have not been a part of programs that try this pre-recorded yet. Although I am sure someone will be trying that soon in this Covid-19 world!

Best,
Emily

Thank you for this video on building positive attitudes towards chemistry. Do you have any research or thoughts upon how to change negative attitudes that may have developed for students exposed to a poor chemistry course in high school? At the college level, it frequently seems impossible to change these impressions once they are formed.

This is definitely an issue that was referred to in the National Academies 2016 report on Effective Chemistry Communication in Informal Environments. I will toss the question to my co-presenters who I think will have some thoughts about it.

Such terrific work! I am wondering how you assess "what works" at different points in your DBR process. I'm specifically thinking about measuring your progress toward the goals of stimulating interest, a sense of relevance, and feelings of self-efficacy.

I'm so excited to see how far you have taken this amazing project! Seven years ago, I was trying to find a keynote speaker, when I chaired a Gordon conference on chemistry education, who could talk about chemistry activities and exhibits in museums, and that was when I learned what the NAS report you mention in your video also said -- that chemistry is extremely underrepresented in science museums. It is SO important that you are tackling this problem! The only active chemistry exhibits I could find were one at the Museum of Science and Technology in Chicago, and one at the Museum of Science in Boston. So I invited the person who developed the MoS exhibit, MJ Morse, who has since retired from MoS (it turns out that my now-collaborator, Marianne Dunne in our project, who also commented above, had worked with MJ when she worked at MoS -- small world we have in Boston!). At the Gordon conference, MJ presented findings about her "Chemistry of Smells" exhibit and associated PD for teachers at MoS, and the design of how she created it, which involved a lot of learning about and interviewing experts on the chemistry of smell. It was so impressive to learn about all the behind-the-scenes design that went into how she developed that excellent program! Which brings me to a wondering question about your project -- What are the ways that you are engaging with chemistry experts as you develop the outstanding informal science chemistry activities that you are sharing with museums and other informal science groups?

Hannah,

Glad to hear about your past interaction with MJ. As for the current project, there are several ways in which we interacted with chemistry experts. From the beginning the project was conceived in collaboration with staff of the American Chemical Society. Mary Kirchhoff (who appears in the video) and I served on the National Academies committee that worked on the Effective Chemistry Communication report noted in the project description above on this page. After that report was completed we started to talk about this potential project. Eventually four or five members of the ACS staff, mostly those involved with education and public outreach, participated in the project in a variety of ways, serving on the leadership team and on several of the working groups. Some of them participated in weekly calls and all in periodic project team meetings. The project's official advisory committee, members of which participated in annual project meetings, involved several working chemists who were recommended by our ACS staff colleagues. The advisors also included a number of folks who were trained in chemistry but now worked outside the chemistry field, in fields like informal science education or social sciences. They participated in selected and analyzing the initial candidate activities for our design-based research testing, reviewing project progress from time to time, and helping us as we moved on to next stages of the project. And then we involved chemists as activities were tested on the floors of the Museum. One of the ways we did that was in connection with annual National Chemistry Week and Chemists Celebrate Earth Week events, during which a large number of chemists served as facilitators for hands-on activities in the Museum. And now we are designing an online train-the-trainer workshop and have 15 chemists who will be the participants in our pilot workshop. They will go through the experience and give us feedback to help us improve the workshop for later presentation to the broader chemistry community.  There's more but I have to run. One of my colleagues or I will finish the story when we can get back online.

Hi Hannah, in addition to all of the great stuff that Larry has shared - ACS also brought in a group of chemists who have particular focus and expertise on outreach using hands-on activities who serve on their Committee on Community Activities as external reviewers before the final step of our development process.  These people gave us very helpful feedback and advice on the activities and training materials in a way similar to what happens with a peer-reviewed paper.  We also got external review on our safety training guide and certain other materials.  

I always love seeing the great work going on at MOS -- thank you for sharing!

Great to see such outreach and interests spurring interests in Chemistry. Can you expound on the constructs and mechanisms of the theoretical framework and how it is connected by the interventions/programs and its relation to the interest, relevance, self-efficacy constructs?

I would like to find out how Sci-Port Discovery Center in Shreveport, LA can be included in these discussions.  Louisiana did not have much involement in this study and we need to be involved!     Dianne Clark, Executuive Director of Sci-Port

Diane, thanks for your interest. I see that two locations in Louisiana got physical Let's Do Chemistry kits, but that Sci-Port was not one of them. For the immediate future, the digital kit is available online and has lots of resources developed in the project including a publication laying out the work of the project and both the framework and ideas about how to use it. You can find links to all of the components of the digital kit here: https://www.nisenet.org/chemistry-kit. We have done sessions and a workshop at ASTC conference and sent 250 kits out mostly to science and children's museums, so our current PD is focused on chemists and especially chemists that reach audiences under-represented in STEM.  We are just at the point of developing a workshop and then sharing the tools with others for conducting similar workshops. We'll keep your interest in mind as we move forward. In the mean time perhaps the guide that is part of the kit could be helpful   https://www.nisenet.org/sites/default/files/chem_ldc_framework_and_strategies_guide.pdf. There are lots of other training resources included in the digital kit as well. I hope they can be helpful as we work through the current phase of the project.

This is such a wonderful project, and I love the fact that you frame this as a follow-up to the NRC study!  Congrats to what seems like a successful projects across all sorts of criteria.  Two quick questions:

How do professional chemists react to these opportunities?  Did you get a sense that they feel empowered to do outreach with these resources?  Have you been able to convince some to try who would otherwise not?

Likewise: how are informal educators doing with them? 

In-person vs online professional development workshop for teaching hands-on skills: in 2004, the Astronomical Society of the Pacific, with a bunch of other partners, conducted AFGU = Astronomy from the Ground Up, which taught hands-on outreach activities to informal educators. We used an in-person model (weekend) and compared it with an equivalent partially asynchronous online workshop (3 weeks). It was fascinating to think about equivalency here, but here a few nuggets I remember: Both worked, but only because we constructed them both from a clear conceptual framework. We knew precisely why we did what. And probably not a surprise: Since in the online version educators practiced at their home institution with real audiences, rather than with colleagues at some remote workshop, transfer was better. They used it more.

Thank you for such rich and comprehensive conversation. You have done so much to develop and facilitate activities that do make a difference to visitor attitudes about chemistry. Have you thought about following up with visitors after a few weeks or months? How is this project informing your next inquiry? 

Thank you for developing these resources.  I use them extensively as part of our visits to elementary schools and for public events! https://www.instagram.com/fresnochemclub/

Dermot, I am glad that they are so useful for you.  For those who haven't found them yet, the digital version of all of the resources developed in this project can be found at https://www.nisenet.org/chemistry-kit.

Love this beautifully filmed video. This would get anyone and everyone interested in chemistry. We're doing work on STEM stereotypes that dissuade young girls from fully engaging on STEM topics. Are you finding that your resources and approaches can also help regarding gender stereotype issues that plague our society?

Andrew, I suppose you can say that simply because the project design focused (in part) on self-efficacy, and the groups with which the activities were tested included both boys and girls mixed, the activities were developed in a way that would support girls perceptions that chemistry is something they could do. I'll let others on the project talk more about how that played out in the way the research was conducted. But I will share a few short anecdotes that recipients of a Let's Do Chemistry kit reported to us:

• We had a young girl attend the event dressed in a lab coat. Her mother was thrilled by the event and shared with us how few science events are available for young girls. At the end of the evening, one of our Dymax Volunteers came up to me and was in tears. She said that her night was made when she met her younger self. She was referring to the young girl in the lab coat.

• A student was intrigued with the notion that what she experienced was actually chemistry. She thought only lab coat-clad older men did chemistry. But now she can claim that she is as chemist too!

• The most memorable moment from our Explore Science: Let's Do Chemistry  event has to be when a group of Girl Scouts were participating in the Sublimation Bubbles station. They were having a blast learning all about the bubbles, dry ice, and the chemistry behind it all when one of them turned to their troop leader and said, "We're like real scientists now!"

Thank you Elizabeth and Larry: 64% is a big number and "We are like real scientists now!" is priceless. 

Your content rich video is great and tells the story of your project quite well--nicely done! As an informal educator/marine scientist, I really appreciate the iterative approach you took to designing your program's activities. About how long do you feel like that design process took? Any tips for others who might be looking to implement a similar approach to developing their STEM outreach activities?

I look forward to exploring the resources linked above and thinking about how things might be integrated into our chemical oceanography camps and after school programs in the future!

Holly and Liz, if I remember right, we did the 15 activities in batches of 5 at a time, and this wasn't just a formative evaluation process, which we use all the time. Along with improving each batch of activities, we were also trying simultaneously to build a generalizable theoretical framework supported by the project data. I think that resulted in added steps and time in the DBR process as compared to the steps and time needed for more traditional formative evaluation. Does that seem true to you, Liz? But in the end you don't just have the finished activities, you have the framework, which can be applied to all sorts of future activities.

Your work is wonderful and I enjoyed seeing how a this group identified a need and took a novel approach to find ways to help close the gap. 

• What are the best ways of reaching practitioners of informal education and public outreach broadly throughout a field like chemistry?

in response to this prompt from the first post, also learning from comments and concepts embedded in this showcase, are there ways you can tie the design based research approach to identity? Would elements of storytelling techniques such as place and emotion connected to a summation of personal events that prompted change or a dramatic event help drive these concepts home in the settings you’ve described?

Alex, thanks for your comment and question. In a way, including the girl in the video who says at the end "I wasn't expecting that," and a number of the stories that kit recipients sent us in their reports ("We're like real scientists now!") connect the interest, relevance, and self-efficacy piece with identity and, I think, show the power of storytelling. That together with the research work and outcomes, I hope will be compelling. We are just starting a pilot workshop with members of the chemistry community and we will see what they think in the next few weeks.

To add to what Larry is saying, I could see some of this in the "deepen understanding" piece of the facilitation framework component.

In that deepen understanding piece, the activity facilitator/educator is advised to consider strategies that 

"encourage participants to draw from their own experience, make observations, and test their answers", "encourage them to make connections to everyday life and societal issues", and "contribute ideas and information from your {the facilitator's} own experience."

I can see clear connections between identity, narrative, and these facilitation strategies.

Thanks Larry and David. One more follow up, is there room to tie subject matter exports to engage your audiences remotely in the current context? I see related concepts discussed in the thread above but wonder how we can engage deeply , with a sense of community, when sitting in front of a screen. What are your thoughts? 

Alex, your question is fascinating, and one of the interesting things here to me is to consider what "subject matter experts" actually means in this context. I think perhaps it could mean people who are really committed to an skilled at chemistry outreach. There a number of ways that I imagine these kinds of people could be involved in building community and sharing the learning that is embodied in our framework. The most direct is the train-the-trainer model we are employing. We have designed the workshops that participants who complete it will go back and train others that they know and are in a position of facilitating hands-on chemistry activities. In addition to this train-the-training, though, I see that these workshops are in and of themselves creating networks of subject matter experts in the realm of hands-on chemistry outreach and that these folks can serve as mentors for one another and build a collective community and a shared sense of self-efficacy.

Thank you Larry - you always tackle the hard challenges! It's great to see that you've made so much progress on the activity front - I wasn't at all sure that there would be rich kits of chemistry activities that could address the challenges of "safety and mess" but you've done it, and with a strong theoretical framework too. Congratulations!

One thing I wonder is whether you've looked at the role of parents/caregivers in the work? I'm thinking that as we all pivot more toward families spending time doing joint educational activities at home, it would be helpful to know what kinds of curiosity, concerns, experiences, and tolerances parents have for chemistry in particular. And maybe you already captured some of that in the family work you've done in the museum context... sorry if I missed this in the earlier discussion. Thanks.

Thanks, Sue. We didn't really focus on parents as facilitators in this project. Our model was based on museum educators and volunteers, and chemists and chemistry students doing National Chemistry Week activities, as the facilitators of the hands-on activities. The kits were distributed in 2018 and the prospect of a closing of everything was nowhere on our minds. That said, the guidelines for facilitation, which drew heavily from the Exploratorium's guidance on facilitation, can be applied by parents as well as by museum volunteers or chemists doing outreach. But your question, I think, goes more toward parent's attitudes and predispositions concerning chemistry. And I don't think I really have an answer about that.  Dietram Scheufele's group at UW-Madison did work as a part of this project with public audiences online in the absences of any of the hands on material, and their findings might provide some leads for research that specifically focuses on parents. Similarly, the research teams at the Museum of Science and the Science Museum of Minnesota observed and interviewed groups interacting with museum facilitators, and perhaps comments by the adults in the groups might also point to some further questions related to what you are asking. David and his team have done some online programing in the last couple of weeks and Liz's group is working on evaluation. Maybe they have something to contribute to this discussion.

Larry's discussion about the research work led by Liz's and Dietram's groups is spot-on but let me just add in something more anecdotal from my perspective. I have observed (again, anecdotally) that parents and adults may bring their children to large-scale wow-style chemistry demonstrations but need to be calibrated with relevance from the beginning - and throughout -- or else there is the risk of tuning out.  This doesn't seem to happen the same way with hands-on activities, when they are well-facilitated. Perhaps it's because of concerns that parents have about kids either messing things up or being unsafe - I don't know.  Obviously, this is part and parcel with the format, so it's not really revolutionary to say this, but there seems to be something that may inherently prime the adults for participation in relevance discussions with hands-on activities that may need more establishment with more demo-type interactions.  We didn't study this at all, but I am bringing it up because obviously our new normal is more like demos in that even hands-on activities can be modeled but it's harder to actually get people doing them online in real time. So something to think about - perhaps the relevance piece can be imparted to adults in training them how to do these activities with their kids, rather than just doing the activities demo-style, so that the adults get more out of the experience?  This obviously would need evidence to support it since I am just spinning my wheels here.

Sue, as Larry said, we really focused on the facilitation that educators gave to visitors as opposed to parent-child or within group interactions. We based this work on facilitation strategies established by the Exploratorium's Tinkering Studio, but gave them our own spin for our context and project. We focused on three phases of facilitation, invite participation, support exploration, and deepen understanding. We found that facilitators spent most of their time supporting exploration during our activities. Not too surprisingly, it seemed that supporting exploration facilitation helped visitors' self-efficacy, and that deepening understanding moves appeared to support relevance. 

Great project to bring more chemistry instruction to informal learning environments! 

 Thanks for your comment, Christine.

I wanted to thank everyone for coming to watch our video and for the thought-provoking comments and questions.  This discussion has sparked me to envision some potential extensions of this work that I likely would not have thought of otherwise.