807 Views
  1. Nigel Bosch
  2. https://pnigel.com
  3. Assistant Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. University of Illinois at Urbana-Champaign
  1. Carolyn Anderson
  2. Professor
  3. Presenter’s NSFRESOURCECENTERS
  4. University of Illinois at Urbana-Champaign
  1. Suma Bhat
  2. Assistant Professor
  3. Presenter’s NSFRESOURCECENTERS
  4. University of Illinois at Urbana-Champaign
  1. Michelle Perry
  2. https://education.illinois.edu/faculty/michelle-perry
  3. Professor
  4. Presenter’s NSFRESOURCECENTERS
  5. University of Illinois at Urbana-Champaign

Underrepresented Student Learning in Online Introductory STEM College Courses

R305A180211

2021 (see original presentation & discussion)

Undergraduate

This project explores the interaction among various characteristics of online instruction and college students’ outcomes in STEM courses. In particular, we explore how students underrepresented in STEM (e.g., women, first-generation college students, U.S. racial and ethnic minorities) benefit from or are impeded by online course features such as flexible scheduling and discussion forums. Online instruction has the potential to make course content more accessible to a larger number of students, thereby strengthening STEM education. However, instructors and administrators need to know if certain online approaches create barriers so that they can (re)design online courses to serve all students and provide better support for students who are taking online courses. 

We analyze a combination of outcome measures like grades, log file records of the actions that students perform in online course websites, and the text of their interactions with other students in course discussion forums. These data enable us to answer research questions related to self-regulated learning (how students schedule and organize their learning activities), metacognition (how much students are aware of what the know and don’t know), and other topics that are especially important in online learning. We then examine how learning behaviors differ for students from different demographic groups, and thus contribute to findings on how to effectively support successful online learning for all students.

This video has had approximately 198 visits by 153 visitors from 71 unique locations. It has been played 62 times.
activity map thumbnail Click to See Activity Worldwide
Map reflects activity with this presentation from the 2021 STEM For All Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2021 STEM For All Video Showcase (26 posts)
  • Icon for: Jeff Ginger

    Jeff Ginger

    Postdoctoral Associate
    May 10, 2021 | 02:52 p.m.

    Hi Nigel et al. - iSchool Alum here. Did you notice any major differences between the types of STEM classes? For instance I could imagine behavior, engagement and teaching varying by specific discipline (structured / formulaic areas like math vs. subjective / emergent ones like informatics) as well as resulting from the classic major impact factors like subject content relevance, instructors as role models and class size or length.

     
    1
    Discussion is closed. Upvoting is no longer available

    Renato Azevedo
  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 10, 2021 | 05:32 p.m.

    Hi Jeff, great question! There were indeed some large differences, especially in forum participation, though it is difficult to say for sure whether they are due to a particular STEM topic, student population, or other reasons like those you mention. But, I think our best guesses for drivers of differences, based on interviews with teachers and reading syllabi, come down to course design. Most notably, there are various strategies for structuring discussion forums and encouraging them, such as making it a large, small, or non-existent component of students' grades, or putting students into small group forums versus a free-for-all approach. There is some literature on how to engender discussion (including lots of things that don't work!), but it's far from a solved problem. Definitely an exciting area for future research!

     
    1
    Discussion is closed. Upvoting is no longer available

    Jeff Ginger
  • Icon for: Stephen Alkins

    Stephen Alkins

    Facilitator
    Diversity, Equity, and Inclusion Officer
    May 11, 2021 | 12:06 p.m.

    Good presentation.  Along those lines, what were your standards for detecting gendered language by professors or by students?

    Separately did you notice differences (e.g. more or less) in "help-seeking" behaviors for online versus in-person classes.  In some sense students might feel more engaged to seek help through networks out of necessity, but some may also be discouraged to do so because of the distance. So, the recipe for success is the same, but whether or not the engagement is the same could be different.  Also, do you see racial or ethnic differences in who sought out more support resources in online learning.  This would also be particularly useful for considering design of supports for students with disabilities as well.

     
    1
    Discussion is closed. Upvoting is no longer available

    Renato Azevedo
  • Icon for: Michelle Perry

    Michelle Perry

    Co-Presenter
    Professor
    May 11, 2021 | 10:27 p.m.

    We’re glad that you enjoyed our video!

    Now, to address your first question, to detect gendered language, we followed Newman et al.’s (2008), which used Linguistic Inquiry and Word Count (LIWC), which is a program for text analysis. The categories that Newman et al. used were based on Tannen’s classic (1990) work on gendered discourse, which classified text as report- (male) or rapport (female)-oriented language. The specific report categories that we examined were analytic, clout, number use, and certainty; the specific rapport categories that we examined were authenticity, pronouns, social processes, discrepancies, tentativeness, and affiliation. For more information on this analysis, here’s a link to the article recently accepted for publication: http://publish.illinois.edu/ilearngroup/files/2021/05/Hendricks-et-al.-2021.pdf

    Unfortunately, addressing your next point, although we agree that there should be both benefits and barriers to seeking help online compared to in-person classes, we can’t answer your question about comparing help-seeking across these contexts because we didn’t directly compare online and in-person classes. We also didn’t separate out racially minoritized students from other groups of students underrepresented in STEM, given our adherence to privacy protections. We did consider the role of student engagement in predicting student improvement in the course and found that engagement (operationalized as posting to the discussion forum) was not related to improvement in the course. For more details about these analyses, I recommend reading our recently published paper at https://dl.acm.org/doi/10.1145/3448139.3448159

    Moving forward, I think it will be important to understand the ways in which the online space provides both natural and instructor-created supports for students to get the help they need, when they need it. 

    And thanks for these great questions!

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 10, 2021 | 05:34 p.m.

    Hi everyone,

    Thanks for watching our video! This is an exploratory project examining the ways in which students from groups that are underrepresented in STEM education make use of the unique features of online college-level courses. In particular, we focus on the behaviors that might distinguish students from underrepresented groups and their peers, and whether these behaviors are effective for learning. Ultimately, we anticipate that results will inform the design of online college courses to ensure that they encourage behaviors that are especially effective.

    We would welcome any feedback or questions you have! We’d be particularly interested in hearing thoughts on things like:

    • What kind of affordances do you think online courses do (or should) have to improve equity in STEM education?
    • What might some effective ways be of communicating research findings to online college course instructors who would be interested in making changes based on this research?
    • Anything else you think is relevant!
  • Icon for: Nuria Jaumot-Pascual

    Nuria Jaumot-Pascual

    Facilitator
    Research Scientist
    May 11, 2021 | 10:17 a.m.

    It is interesting to hear that online courses level the playing field. Why do you think that is the case? What may be factors contributing to this leveling?

    I read a study where women of color opted to take online courses to avoid the microaggressions, invisibility, and general social discomfort that they typically experience in in-person classes. Do you think that this is one of the factors that may contribute to leveling the playing field for students in online STEM classes? 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 11, 2021 | 11:10 a.m.

    I do think that is a strong possibility, and one which bodes well for the potential of future online courses. We're hoping to study the role of flexible anonymity that online spaces bring in promoting in future work, especially as it relates to help-seeking and help-giving among peers. With reduced stereotype threat, students might feel more inclined to ask (or answer) questions without fear of fitting into stereotypes they or their peers hold about their place in STEM.

    By the way, do you remember the name of that study? I would love to read it!

  • Icon for: Nuria Jaumot-Pascual

    Nuria Jaumot-Pascual

    Facilitator
    Research Scientist
    May 11, 2021 | 04:44 p.m.

    It's a dissertation and it's available online (see the link below). The author doesn't address stereotype threat, but does talk about the other issues you mention. It may be an interesting reference for your work. 

    Ratnabalasuriar, S. (2012). Forging Paths Through Hostile Territory: Intersections of Women's Identities Pursuing Post-Secondary Computing Education (Doctoral dissertation, Arizona State University). https://core.ac.uk/download/pdf/79563797.pdf

     
    1
    Discussion is closed. Upvoting is no longer available

    Renato Azevedo
  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 11, 2021 | 08:39 p.m.

    Thank you! Looking forward to diving into this!

     
    1
    Discussion is closed. Upvoting is no longer available

    Nuria Jaumot-Pascual
  • Icon for: Renato Azevedo

    Renato Azevedo

    Graduate Student
    May 13, 2021 | 06:00 p.m.

    Thank you for the reference!!! 

     
    1
    Discussion is closed. Upvoting is no longer available

    Nuria Jaumot-Pascual
  • Icon for: Nuria Jaumot-Pascual

    Nuria Jaumot-Pascual

    Facilitator
    Research Scientist
    May 13, 2021 | 09:50 p.m.

    I was just rereading Nigel's initial comment and the concept of flexible anonymity caught my eye. Can you talk a little more about it? How do you conceptualize it? 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 14, 2021 | 09:31 a.m.

    Certainly! In the online courses we've explored in this project, students have relative anonymity. Names are shown to other students when they interact in discussion forums, but little else is. In some courses, students can upload a picture of themselves; however, the data show that few students do so. On the other hand, students do tend to share personal details about themselves in the discussion forums, such as their place of birth, class experiences, hobbies, and other interests. So, students have some control over how much others' see of themselves, which may lower barriers to participation for some.

    Not all courses provide such flexibility, however. All the courses we've examined show student names to each other in their online discussions, which may allow peers to infer some ethnicity or gender information, if imperfectly. So none of them allow full anonymity, but some more than others. A course we plan to examine in the near future shows peers their name, picture, and email address with every interaction, for example. It will be interesting in the future, I think, to see if there is a trade-off between building a sense of community by helping students get to know more about each other, versus increasing the barrier to entry for students who may not feel comfortable with this, and whether that affects different groups of students differently.

     
    1
    Discussion is closed. Upvoting is no longer available

    Kristin Flaming
  • Icon for: Kristin Flaming

    Kristin Flaming

    Higher Ed Faculty
    May 14, 2021 | 12:21 p.m.

    I try to encourage students to post a picture because it does seem to help me and them develop a bit deeper connection. I find that having the option to post anonymously in the discussion helps those that are not comfortable early on. Usually later in the semester there is very little posting anonymously. 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 14, 2021 | 12:30 p.m.

    That seems like an excellent idea! I had not thought much about the temporal dynamics of it, but indeed perhaps the anonymity vs. comfort decision doesn't have to be much of a trade-off at all if it can be a gradual acclimation process. Thanks for that thought!

  • May 11, 2021 | 04:14 p.m.

    As so many are looking to take their courses online or refine the courses that are already delivered in an online modality, this project is really important to STEM education. Thank you for this work and for setting such a wonderful example in collaboration as well.

    I'm very interested in the methods through which you evaluated the different variables of interest in your project. Your team has examined such important topics of gendered language, support seeking/offering, and metacognition in innovative ways. I'm a faculty member at a campus that primarily teaches asynchronous online courses, and I'd definitely love to leverage findings like this to improve teaching and learning--especially in terms of knowing the most important ways to measure the impact of different course features on student outcomes. 

    I have noticed that some students struggle a bit more with issues of procrastination and attention/energy management when they don't have a set class time or the physical presence of other learners around them. How (or to what extent) might some of your methods and findings generalize to students with disabilities related to executive dysfunction (e.g., ADHD, autism, PTSD, depression) who are taking online courses? 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 11, 2021 | 06:56 p.m.

    That is a really interesting question, which we have not considered much in this project -- partly because we don't have data on executive dysfunction (all data here are retrospective which makes it tough to ask students new questions) but also because we haven't considered that enough, and should! Some of the courses definitely engendered more participation and discussion than others based on how activities were structured, and I would suspect that students with disabilities related to executive dysfunction would benefit even more than average from an engaging course structure. The most engaging course sections tended to be those with a large amount of structure, including small-group forum discussions, frequent "reporting out" from groups to the whole class, and grades to go with each of those.

    I also 100% agree with you from observations in my own courses about some students struggling more than most with online, and especially asynchronous, courses. In my courses over the last year I've tried to adopt some of the practices that seem to be most effective, like requiring forum participation (often), providing several different types of participation, and contacting students at very regular intervals. It isn't a perfect substitute for coming to class at a specific time, but anecdotally it seems to be helping!

     
    2
    Discussion is closed. Upvoting is no longer available

    Nuria Jaumot-Pascual
    Chelsea LeNoble
  • Icon for: Anya Goodman

    Anya Goodman

    Higher Ed Faculty
    May 11, 2021 | 05:50 p.m.

    Great video - thank you for sharing! Were the courses analyzed for your study lower division or upper-division? Were students required to post to forums or ask for help? Where students instructed in metacognitive strategies?  Were there explicit strategies used for building community in the class? I can see that anonymity may facilitate asking for help, but inhibit community formation among students. 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 11, 2021 | 07:26 p.m.

    We specifically focused on early, introductory STEM courses where we expected college students' experiences would be most formative. In some courses, students were required to post to forums in different ways, but not necessarily asking for help (e.g., posting reflections, answering other students' questions). The requirement varied quite a bit though, from around 5% to 25% of students' grades, whereas some courses had no grade attached. It certainly seems to make a difference! However, it is also hard to distinguish from the community building strategies you mention, which were roughly correlated with the grade percentage: courses with more grade emphasis on forums were also courses where there were more types/variety of participation encouraged (e.g., small group and large group discussion, specific discussion topics).

    Regarding metacognition though, none of the courses instructed students to engage in metacognitive strategies or discourse, though there is quite a bit of evidence that some students do anyway. However, there seems to be a lot of opportunity for explicitly coaching this in the future, especially to promote learning in asynchronous courses and courses where students don't have as much support (i.e., where the need for self-regulation is even higher than usual).

    I also am very curious about the double-edged sword of anonymity that you mention. We don't have a lot of variation in anonymity in the current courses, but are hoping to explore this a bit more in future courses where this can be more explicitly explored.

     
    1
    Discussion is closed. Upvoting is no longer available

    Anya Goodman
  • Icon for: Overtoun Jenda

    Overtoun Jenda

    Facilitator
    Assistant Provost and Professor of Mathematics
    May 11, 2021 | 09:12 p.m.

    Very interesting study. How did you handle laboratory work? If you had virtual labs, did you measure if this has an impact on learning certain concepts? Did you you use a mixed methods approach to assess students' experiences and measure satisfaction levels aggregated by gender, race, class, etc for example? 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 12, 2021 | 09:43 a.m.

    Thank you for these questions!

    These courses don't have lab work in the typical sense (largely because they're asynchronous, I suspect). However, one course in particular does have somewhat related activities where the students brainstorm solutions to a thought experiment using the discussion forums to discuss and present their solutions, which appeared to be an effective approach for nearly all students based on what we can garner from graded outcomes and participation.

    The question of mixed methods is something that would be really good to get to, and will likely be a big part of our next iteration of this work. In the current project, we are focusing on retrospective examination of courses that are completed. This enables us to take a slightly higher-level view because these data are anonymized and archival, which means we can get data from 100% of students. However, there is a lot missing from this high-level picture. We would love to do interviews and focus groups to obtain students' perspectives on some of these issues more directly (e.g., when and why they express metacognitive evaluations of their process), especially as they relate to things like gender, race, and first-generation college student status.

  • May 12, 2021 | 02:57 p.m.

    This is a great collection of lines of inquiry within this project! I'm curious about the operational definition of "consistent" and "inconsistent" patterns of learning. Are you looking at patterns of regularity in time? Types of resources used? We do a fair bit of work trying to classify the resources that LMSs afford and instructors provide as they serve students' cognitive and metacognitive processes, and I'd love to learn more about how others are approaching this. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Renato Azevedo
  • Icon for: Suma Bhat

    Suma Bhat

    Co-Presenter
    Assistant Professor
    May 12, 2021 | 05:13 p.m.

    Excellent questions. We are indeed looking at patterns of regularity in time, specifically over observation periods through the duration of the course, operationalized as the time between each assignment's release date and its due date. The patterns themselves are derived from sequences of 'events,' where each event corresponds to the resource accessed (e.g., forum post, grade view). If you would like to know more about the details of our approach, we recommend reading our recent LAK paper: 

    Zhou, J., & Bhat, S. (2021, April). Modeling Consistency Using Engagement Patterns in Online Courses. In LAK21: 11th International Learning Analytics and Knowledge Conference (pp. 226-236).

  • Icon for: Rivka Glaser

    Rivka Glaser

    Higher Ed Faculty
    May 14, 2021 | 12:33 p.m.

    Very interesting work.  I was wondering about the overlap in your results - for example those students that formed larger networks, were they the consistent learners?  In addition, I would be interested in any literature about engendering discussion forums.  I found that my students created their own forums for group discussion outside of class, but those were not assessable forums.

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 14, 2021 | 03:08 p.m.

    Overlap at the intersection of these methodologies is something we've just begun to explore (e.g., improvement and social network characteristics). We have not yet looked at consistency and social network size, but I think that is a really interesting question since it seems quite plausible that a well-connected network of peers would encourage regular interaction. Certainly something for us to examine going forward!

    I think methods for engendering effective discussion in forums are still sorely needed. However, there are some previous projects in this area, showing mixed results; it is a tough problem! Here are a few of my favorites (many of those that cite these are also relevant):

    Mazzolini, M., & Maddison, S. (2007) paper on the role of instructors in discussion forums

    Kizilcec et al. (2020) paper on interventions intended to improve discussion forums (though in very large classes)

    Schworm & Gruber (2012) paper on methods to encourage help-seeking in discussion forums, and interesting (if troublesome) results regarding the psychological barriers to help-seeking

  • Icon for: Lindsay Palmer

    Lindsay Palmer

    Graduate Student
    May 14, 2021 | 02:44 p.m.

    Excellent work! It is so interesting that you found that the behaviors to be successful in face-to-face classrooms are also successful in online environments. 

  • Icon for: Nigel Bosch

    Nigel Bosch

    Lead Presenter
    Assistant Professor
    May 14, 2021 | 03:17 p.m.

    Thank you! And yes, I am both encouraged and intrigued by that myself, because it may suggest that our online courses are simply modeled very closely after face-to-face courses. Hopefully, there is a great deal of future work to do discovering ways in which we can make them much better by leveraging their own unique affordances.

  • Further posting is closed as the event has ended.

Multiplex Discussion
  • Members may log in to post to this discussion.