7859 Views (as of 05/2023)
  1. Florence Sullivan
  2. http://people.umass.edu/florence/
  3. Associate Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. University of Massachusetts Amherst
  1. Ricardo Poza
  2. Research Assistant
  3. Presenter’s NSFRESOURCECENTERS
  4. University of Massachusetts Amherst

Microgenetic Learning Analytics

NSF Awards: 1252350

2016 (see original presentation & discussion)

Grades 6-8

We investigate the incidence of internalized stereotypes as girls participate in a robotics activity. We operationalize this hidden phenomenon by examining the prevalence of negative internal attributions uttered during activity in the robotics environment for an all-girl group, in contrast to an all-boy group and a mixed gender group. We utilize a modified form of sentiment analysis to examine student talk as they solve problems. Preliminary results indicate that the all-girl group made more negative internal attributions than both the all-boy and the mixed gender group. Further analysis indicates that the all-girl group used derogatory terms to describe their activity, in comparison to the all-boy and mixed gender groups. These results have implications for the professional development of teachers.

This video has had approximately 455 visits by 338 visitors from 94 unique locations. It has been played 221 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the NSF 2016 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the NSF 2016 STEM For All Video Showcase (29 posts)
  • Icon for: Isabel Huff

    Isabel Huff

    Program Outreach Coordinator
    May 16, 2016 | 05:42 p.m.

    This is a very interesting study! Do you have any ideas about why the all-girl group made more negative internal attributions than the other kinds of groups? Were all of the groups in the same room all day?

  • Icon for: Ricardo Poza

    Ricardo Poza

    Co-Presenter
    Research Assistant
    May 16, 2016 | 10:56 p.m.

    Dear Isabel. 

    Thank you for inquiring about our study. For this particular study all participants were girls aged 8 through 14. They were 6 teams in total, where 5 of the 6 groups consisted of 3 girl-participants and one group had just 2 participants. All six teams participated on the one day event at the same time (about a 7 1/2 hour day). 

    Negative attributions are not all necessarily bad, in fact there is an inherent amount of frustration when learning new tasks. The yellow team, for example, presented the largest number of negative internal attributions. Coincidentally this was also one of the teams that completed the least number of robotic challenges. Although we can only conjecture at this point, it appears that the  perception of internalized stereotypes in girls might hinder motivation and willingness to learn tasks in areas where members of the opposite gender are supposed to excel. The data appears to indicate that for those teams with the lowest number of internal negative attributions (dark gray and light blue) there was a higher level of motivation for excelling at robotic tasks. It is important to highlight that both the dark gray and light blue teams performed the greatest number of successful robotic challenges. 

    I think it is very hard to act upon a task when we have an internalized perception that we are not going to do very well.

    Our results are premilinary, but it does raise questions as to how educators can counter the effects of gender-based internalized stereotypes. 

  • Icon for: Tamara Ball

    Tamara Ball

    Facilitator
    Asst. Project Scientist
    May 17, 2016 | 01:37 p.m.

    Great to see more hard evidence documenting the reality of “stereotype threats” . My own experience in engineering is that there are still many disbelievers – i.e. those that don’t succeed just aren’t motivated enought etc.

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 02:11 p.m.

    Hmmm…it’s hard to understand how knowledgeable people fail to understand the impact of culture and society on the development of self, yet, there it is. We just need to keep doing our work Tamara!

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 11:09 a.m.

    Another point is that we collected survey data from the girls related to their own stereotypes about who typically does robotics (men or women), about their own confidence in doing robotics, their own feeling of threat or challenge in relation to robotics study, and their sense of personal identification with technology. The girls who reflected lower levels of confidence and identification in the survey also made a greater number of negative attributions. That said, most of our girl participants felt very confident and identified with technology – so, though our research is preliminary, for girls who opt into these experiences, negative stereotypes do not seem to be operating. We can’t say what might be happening for girls who did not choose to take part in this event. It may be a different story for them.

  • Icon for: Vivian Guilfoy

    Vivian Guilfoy

    Facilitator
    Senior Advisor
    May 17, 2016 | 11:28 a.m.

    This is very important work—making sense of internalized stereotypes in the STEM space. Could you say a little more about how the workshop was advertised or how the girls and boys were recruited to participate? Also, where was the event conducted? in school? in an informal learning environment? I am also interested in learning how you decided to place girls in the groups (random or based on their survey responses). I am thinking about the impact of peer pressure in the age group and how it can “sway” attitudes and behaviors. I am also interested in any plans you have to use the data to help teachers understand the dynamics of learning and gender differences that may be operating in STEM.

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 02:07 p.m.

    HI Vivian – Thanks for your questions. This was a one-day robotics workshop for girls (only) ages 8-14. A total of 17 girls participated in 6 teams. The girls were recruited from area schools – the teams were made up of girls from the same school, but, they were not, necessarily friend groups. The technology teachers from the participating schools helped us recruit participants. This would classify as an informal learning setting – the workshop took place in Holyoke, MA at the Massachusetts Green High Performance Computing Center (http://www.mghpcc.org/). We are currently working on another, related paper that looks at group dynamics and how those dynamics play out in terms of opportunities to learn within the group. The results from both of these, related, investigations should yield recommendations for teachers and curriculum developers.

  • Icon for: Lisa Samford

    Lisa Samford

    Facilitator
    Executive Director
    May 17, 2016 | 03:56 p.m.

    Were your groups mixed age? The experience of being 8 is very different than that of 13—I wonder about the social dynamic (13 year olds influence youngers, and can dominate; some studies have indicated that previously confident girls show a dramatic shift following puberty—and in theory social/peer pressure, etc.) Did you do any analysis/find any age differences?

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 08:59 p.m.

    No, we have not done that. Actually, only one group had younger girls (an 8 y.o. and a 9 y.o.). these two were sisters and were paired with a friend who was 10 y.o. It was hard for the 8 y.o. to stay engaged. Really, it was beyond her. That said, I have met 8 y.o. girls who can and do stick with this type of robotics activity. ideally the age range would be 12-13 y.o., which were the majority of the participants. However, this workshop has traditionally been open to ages 8 – 12, so we stuck with the “Girls Connect” model. They were our partners.

  • Icon for: Tamara Ball

    Tamara Ball

    Facilitator
    Asst. Project Scientist
    May 17, 2016 | 01:38 p.m.

    That is an interesting point to consider further: the fact that your sample was a sample of girls who already opted into the program.

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 02:10 p.m.

    Yes, Tamara, I do think that makes a difference. We had some serious contingents of self-identified “geeky” girls. It is also interesting to see how some of that played out within groups vis-a-vis control of devices, direction of activities, etc.

  • Icon for: Sophie Joerg

    Sophie Joerg

    Assistant Professor
    May 17, 2016 | 03:28 p.m.

    This is a very important topic and it is great to see that it is being investigated! In the abstract you mention that the all-girl group made more negative internal attributions than both the all-boy and the mixed gender group. Could you give us further details on where the all-boy and the mixed gender group events happened? I realize that the results are preliminary. Do you already know if the girls who were part of the mixed gender group make less negative attributions compared to the girls in the all-girl groups?

  • Icon for: Lisa Samford

    Lisa Samford

    Facilitator
    Executive Director
    May 17, 2016 | 03:58 p.m.

    I’d be interested to hear about this with an “age” filter as well—the 13 year old girls I know are acutely more interested in validation from their male counterparts than 8 year olds—and almost all “youngers” are influenced by older students

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 09:11 p.m.

    Sophie – we had a prior set of data that included an all-boy group and a mixed group from two six grade science classrooms. The data sets were different in that the prior data sets were collected over a 12-day period – in 90 minute blocks. Our one day event consisted of a total of about 4 hours of active building and programming (with the rest of the time spent on instruction, lunch, breaks and the final activity where the girls showed off what their bots could do). As to the mixed group, one girl was fairly negative and the other pretty positive. The positive one worked hard to build the confidence of the negative one, that was a very interesting dynamic. I do think that boys have the stereotype that they should be better at robots than girls, and so we have seen such dynamics (in terms of assertions to dominance) in the mixed groups. At some point, it is the personality of the girl that dictates her acquiescing or resisting that narrative from male members of the group. We wanted to see what happens in all-girl groups – do the stereotypes show up in a (seemingly) more equitable environment. Of course, as Lisa is pointing out, there were other factors that came into play that created some level of hierarchy. So, we are working on unpacking that, as well.

  • Icon for: May Jadallah

    May Jadallah

    Associate Professor
    May 17, 2016 | 06:46 p.m.

    Very exciting work. What are your future steps?

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 17, 2016 | 09:15 p.m.

    May – good question. At the moment, we are more focused on looking at the intra-girl group dynamics. We were surprised by the level of hierarchy that emerged in the space. Our research is leading us towards looking at the role of materiality in claiming dominant roles. So, we will work with adjusting the available materials to see how that might affect the group dynamics. We will also continue analyzing this data set as regards the variance in negativity and confidence for the task. We may replicate with a different set of girls next year and add a focus group component after the fact to understand more about these experiences for girls.

  • Emily Hamner

    Guest
    May 18, 2016 | 08:29 a.m.

    Hi, in your video you give some example words identified as having a negative attribution. One of the words is “smart”. Can you explain in more detail how that one is negative?

  • Icon for: Ricardo Poza

    Ricardo Poza

    Co-Presenter
    Research Assistant
    May 18, 2016 | 11:10 a.m.

    Hello Emily:

    This is a great question. After a few attempts at parsing the data, we realized that by having a lexicon that in only including utterances with negative attributions we missed other utterances that could possibly indicate a negative salience. We found that there were times where just one word did not determine the saliency but rather the entire sentence (context) As a result we expanded our lexicon to include words related to confidence as well as other indicators such as “we are not good at this” and “smart”.

    Once our analysis tool flagged an utterance, we manually checked the entire sentence and categorized based on investigators consensus.

    In the specific case of the word “ smart", we found that there were a few cases where it was used to compare the level of confidence between participants. So for example: “The girl from the light blue team is very smart” – The previous utterance by itself does not indicate a negative attribution, in fact it is positive. However, when used to compare one’s confidence with another participant it might represent feelings of inadequacy. It is important to look at the entire utterance and its context. Our lexicon purposely did not include words such as “not”, because after a few trials we realized that we had a large number of false positives.

  • Icon for: Tamara Ball

    Tamara Ball

    Facilitator
    Asst. Project Scientist
    May 18, 2016 | 02:45 p.m.

    I also wonder about the results from your different data sources (observational video vs. survey) Did the two data sets validate each other ? or did they reveal differences? Often we researchers learn the most where there are mismatches in the data…

  • Icon for: Ricardo Poza

    Ricardo Poza

    Co-Presenter
    Research Assistant
    May 19, 2016 | 01:18 p.m.

    Hello Tamara,

    The robotic tasks were based on the First Lego League 2011 “Food Factor” set of missions. Here you can access the Wikipedia page for the specific challenge: https://en.wikipedia.org/wiki/Food_Factor

    In addition, here is the actual document describing all missions included in the challenge:
    http://www.firstlegoleague.org/sites/default/fi...

    Each year the First Lego League creates a game table setup accompanied by a series of missions that need to be completed for points.

    Participants need to program their robots in order to complete missions such as: retrieval or placement of objects, and trigger mechanisms or props within the game table. Each team has the same amount of time to successfully perform as many missions as possible. Points are awarded (or deducted) based on how the robot performs each challenge.

    Normally teams would have months to prepare, for our research we did training and performance all in one day.

    The entire day was structured as follows:
    - Robot construction (Lego Mindstorms)
    - Programming instruction
    - Build + experimentation
    - Lunch
    - Build + experimentation
    - Cookie break
    - Build + experimentation
    - Competition
    - Awards

  • Icon for: Vivian Guilfoy

    Vivian Guilfoy

    Facilitator
    Senior Advisor
    May 18, 2016 | 11:11 a.m.

    Your efforts to dig deeper and understand the reality for these girls through careful data analysis is difficult but very important work. I noticed today that the Gates Foundation just committed $80 million to closing gender data gaps—around the world—addressing the lack of current comprehensive data.

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 19, 2016 | 04:21 p.m.

    Ah, that’s worth knowing about. And, yes, the careful work is very important and very time consuming. We are currently working on developing computational means for exploring these data – in hopes that we can find a way to still dig deep, but to do it with a large data set and with certain aspects of the analysis being automated. Stay tuned on that one!

  • Icon for: Tamara Ball

    Tamara Ball

    Facilitator
    Asst. Project Scientist
    May 18, 2016 | 02:43 p.m.

    It would be helpful to know more about the robotics problems they were expected to solve. From the video we only know that they are “engaging with robotics technology and computer science study” But Were they inquiry based / opened ended activities? or did they have pre-fixed solutions? What was the context for solving these problems? I wonder if you would find different kinds of gender related differences based on the type of problem they were working on.

  • Icon for: Vivian Guilfoy

    Vivian Guilfoy

    Facilitator
    Senior Advisor
    May 19, 2016 | 12:30 p.m.

    Good points, Tamara. Each of us has our own stereotypes about what “robotics” is or means so the examples would be great to examine for their inherent “nature.”

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 19, 2016 | 04:23 p.m.

    Tamara and Vivian – see Ricardo’s reply above to learn more about the robotics challenges students undertook and how our workshop day was structured.

  • Icon for: Vivian Guilfoy

    Vivian Guilfoy

    Facilitator
    Senior Advisor
    May 19, 2016 | 12:30 p.m.

    Look forward to seeing the research that grows out of this project— and the promise it holds to understand and reduce stereotyping at its core.

  • Icon for: Barbara Ericson

    Barbara Ericson

    Senior Research Scientist
    May 19, 2016 | 03:50 p.m.

    I wonder how this study would be different if the robotics task was art based (like with Artbotics).

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 19, 2016 | 04:23 p.m.

    Hi Barbara – Artbotics should be more cooperative – so, it might change the results for sure. That is one of the things I don’t love about the FIRST lego league (Girls Connect is an intro to FLL – we collected data at a Girls Connect event). I do think that the competition aspect may have a negative effect on participation or how one feels about small failures in development. This would be a good next step – looking at how competition may or may not be related to affect or the triggering of internalized stereotypes.

  • Emily Hamner

    Guest
    May 19, 2016 | 04:33 p.m.

    That is a really interesting question. Our group also does arts based robotics projects (Arts & Bots). It would be interesting to see how girls’ attitudes and stereotypes compare across different types of activities. I wonder, for instance, about what types of activities are most encouraging for students with varying levels of existing interest in STEM. Some start with very little interest in STEM or negative opinions of their abilities. Others are interested enough to sign up for a robotics program. How might the type of activity they participate in influence their beliefs about their own STEM abilities?

  • Further posting is closed as the event has ended.