Great topic, Nancy! We've been influenced by the Hecht & Crowley work (2019) and their use of measures of the health of ecological systems. They talk about youth as an "indicator species" but we've chosen to go a level up and think about educators and their sense of connectedness as an "indicator" of the health of these ecosystems. That's focused our evaluation team's attention away from youth for the time being, and instead doing surveys and interviews with educators. TBD whether that approach proves profitable!!

Nancy knows since she was first author: In our evaluation of Oregon STEM Hubs we used an online survey designed to assess the development and health of the local STEM communities. The questionnaire was based on the notion of healthy and productive collaboration for a common purpose, and it asked stakeholders who represented organizations that were part of the Hubs about progress, culture, voice, contribution, etc.  That is, we focused on the nature of the network. You can find it here:

Staus, N., Preston, K., Keys, B., & Storksdieck, M. (2019). Taking Stock of Oregon STEM Hubs: Accomplishments and Challenges: An Evaluation Report Prepared for the Oregon Department of Education. Oregon State University. https://doi.org/10.5399/osu/1123

Alas, each of the 13 Oregon STEM Hubs run their own specific evaluations, focused on different outcomes that they locally determined as important to them. I think one of the big questions is whether STEM Ecosystems are being conceptualized and realized as collective impact organizations, or simply assets for STEM engagement that more people ought to know more about and that should as best as possible coordinate their efforts.

Leigh, 

The interest-driven pathways change direction with the local ability to aid individuals to stretch to their understanding of future skills. I don't think that set missioned driven sites can "grease the wheels of educators" and effectively stretch to reach the fast transition necessary to fill this gap rural locations.  You are missioned focused and responsible to your boards.  Youth have grasped the "climate change and climate justice terms" to connect with long term sustainablity based on a learning pathway that they see relevent to  their experience...before they plan a visite beyond their locaton.   These pathways have to include online support that is flexible and scales constantly back to their topic of interest....not remain in the mission of sites.  

To reach the climate  change and climate justice topic at the sustainable requires relevancy and flexibility.  They will engage...but will they remain and develop agency?  This requires documenting that small steps with cohorts separate from the pressures of formal edcation are valuted by someone....who?   First consideration is to allow them the space to develop the timing.    Yes, this also means not just youth.   

I guess we have to agree to disagree! We are locally driven including definitions of relevance, definition of timing, etc. "Will they remain and develop agency" -- I'll report back in 4-5 years and let you know how it goes. 

I don't think we have to disagree at all.  I am working in a community and under that broader definition. I recognize that this topic is intended to work out the span between the "it takes a village" to the broader emerging linkages of a "learning Ecosystem".  Leaping from the smaller focus of village to the current "ecosystem" is hard enough. Finding an agreement on "interest" not to mention the new "ecosystems" with technology means folks that have been working a long time in one aspect have a unique opportunity...and obligation to refocus. How can that happen?

Why do we all have to wait 4-5 years to hear some views as "agency" or "relevance" are very flexible terms?  Has our design on recognizing them not moved with the idea of STEM?  Are we not able to take a discussion technology like this and find the opportunity to think this through?

Betsy, I am not entirely sure I understand your comment fully, but here are two points I might add: "How can that happen", you ask, and I think that is precisely why STEM Hubs and other forms of "managed ecosystems" exist, to help stakeholders (including educators) not only what all the possibilities are for connected learning, but also facilitate collaboration. That might often just involve a small subset of stakeholders, like we did when I did climate shows in our planetarium and worked with the local nature center on a combined program that first introduced the (global) science of climate change in the planetarium, and then had the students visit the nature center right afterwards to explore in hands-on fashion some strategies to address climate change locally.

In terms of waiting 4-5 years: clearly, there are more short-term outcomes one can focus on, but some behavioral ones (in terms of educational or career pathways) one indeed needs to wait until people had a chance to act. And yes, I think they are also assessing those. But it does provide us with a hefty question: how far out do we feel we might impact people?

Hi Tony, and thanks for joining the discussion. Can you say more about the idea of trade-offs and benefits?

It seemed like different funders would bring different priorities to the work. In the OR example, the state brings a lot of legitimacy to the work but the kinds of measures of success and two-year budget cycles in the political arena brought a new set of challenges to the work. I also imagine that it took some visible STEM hub success in OR before state was willing to financially support them. I know that there has been some effort to bring major philanthropic funders together around collective action such as the STEM Funders Network (http://stemfundersnetwork.org/) and philanthropic efforts have a lot more freedom to be creative and flexible around what they fund and the kinds of measures of success they use. However, the existing literature indicates the "fickleness" of outside funding and when it goes away so does the expertise to support the funded work. I am thinking about the role of the funder, their assets, and how the tensions different approaches to funding might bring to ecosystems work.

Thank you for clarifying, Tony: I couldn't agree more with your sentiment here. It was wonderful that the STEM Funders Network invested into STEM Ecosystems (and now there are 94 supported globally), but there are always issues attached to funding. Alas, the brilliance was (just like in the Oregon STEM Hub example) that someone invested into institutional or organizational infrastructure - yeah for recognizing that communities need backbone support!  The funding cycle for STEM Hubs did indeed cause some anxiety. When your backbone structure faces 2-year or so renewal cycles, you start wondering about sustainability too much. Interestingly, state funding continues even as the initial model was to use state funding to get the machinery in place, and then rely on local support for continuance. The ecosystem should fund itself... which doesn't work because it is hard to negotiate across many stakeholders your way out of the commons and public good problem.  So: stable support from enlightened parties seems to be essential. The question is: how much power to determine the agenda comes with public or philanthropic support?

Cultural brokers is a hard term to feed into an interest driven project. An effective system starts small.  The ecosystems for STEM have been driven by national perspectives on groupings that drive topics and economics.  Why to we keep trying to label them "learning" before allowing the system to become "eco"?  I can understand that " science near me" sounds like the way to progress from educational structure but does it struggle from isolation from the other domains and structures that support varied enties?

We struggled with how to call the portal: are we providing an exchange between those who offer activities and events and those who seek them only in the domain of science?  Should we call it STEM?  Even then, is this how people are looking for opportunities?  Or are they looking for something engaging and interesting to do, with only mild preferences around content?  At the end of the day, we opted to start from a core (science), in hope that people have a very generic and broad understanding of what this stands for, and maybe later we can expand.  

Martin,

  Thank you for expressing the process of moving into true STEM.  I’ve felt that I’ve been waging this battle for years and getting the answer from this type of discussion forum is a huge step forward for those of us who can’t wait for the 4-5 years for the evidence from science developed projects that Leigh offers.  

We’ve been working hard to look at were we integrate STEM to be fair and equitable to all which means looking at whatever we call a “portal” (used to be a pipeline) in a different way that you described so well.  How can we call “looking for something engaging and interesting to do” something positive?  When does this practice..that we all get into at some point in our lives…build agency with those involved?   How do we offer opportunities to find that “FLOW”(Csikszentmilhalyi) that includes science topics but not requires replication….or competition…to be valued?