10872 Views (as of 05/2023)
Icon for: Florence Sullivan

FLORENCE SULLIVAN

University of Massachusetts Amherst

Microgenetic Learning Analytics

NSF Awards: 1252350

2015 (see original presentation & discussion)

Grades 6-8

Microgenetic learning analytics is a new research method we are developing with funds from the National Science Foundation. We are developing computational means for performing microgenetic analysis of discourse data. Microgenetic analysis seeks to understand conceptual change over short time periods (minutes, hours, days). Our discourse data corpus is derived from middle school aged girls working in collaborative problem solving groups to solve robotics problems. The challenge of our work is finding effective means for interpreting contextual discourse that widely features indexical and pronomial data, as opposed to a specific content-based vocabulary. Microgenetic analysis is viewed by educational researchers as one of the most robust methods for understanding how human learning happens. Microgenetic analysis is a labor intensive method that is usually performed in a case study format involving one or a few students. Our goal is to develop a method that will expand possibilities for performing microgenetic analysis over larger data sets, and to expand the scope of research questions that may be asked and answered with such data sets. Our video will provide information on the current state of our solution to this educational research problem using computational means.

This video has had approximately 466 visits by 407 visitors from 108 unique locations. It has been played 327 times as of 05/2023.
Click to See Activity Worldwide
Map reflects activity with this presentation from the 2015 Teaching & Learning Video Showcase website, as well as the STEM For All Multiplex website.
Based on periodically updated Google Analytics data. This is intended to show usage trends but may not capture all activity from every visitor.
show more
Discussion from the 2015 Teaching & Learning Video Showcase (11 posts)
  • Icon for: Debra Bernstein

    Debra Bernstein

    Senior Researcher
    May 11, 2015 | 03:07 p.m.

    Thanks, Florence. This sounds like very interesting work, and a creative solution to dealing with the limitations of microgenetic data analysis. Can you give an example of the types of questions you can answer with this method?

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 11, 2015 | 10:11 p.m.

    Hi Debra – Thanks for your question. At this point, we are hoping to identify cycles of activity within the student collaborative problem solving group that will help us answer questions about the algorithms and strategies they develop to solve problems. We may also be able to ask questions about group work negotiations and how teams organize themselves from the inside of the activity. Another possibility will be the visualization of the group effort in terms of problem-solving cycles over time. Variations in group outcomes along any of these dimensions will help us ask deeper questions about the practice of collaborative problem solving in computational environments.

  • Icon for: Debra Bernstein

    Debra Bernstein

    Senior Researcher
    May 13, 2015 | 10:14 p.m.

    Thanks, Florence. I could see this being a particularly powerful method for looking at patterns over time (as you describe above). I look forward to learning more as the work progresses!

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 14, 2015 | 09:13 a.m.

    Yes, Debra, we are excited about the potential for asking questions that would otherwise be difficult to ask. And, this does have to do with patterns related to both collaboration and to the types of argumentation or other problem solving activity students engage in over the course of the unit. Lots of potential here.

  • Icon for: Jessica Hunt

    Jessica Hunt

    Facilitator
    Assistant Professor
    May 11, 2015 | 03:14 p.m.

    Your work is very interesting; I appreciated your conversation of the challenges of finding effective means for interpreting contextual discourse. Can you talk more about how you determined the various types of student talk that accompany specific problem solving activities (i.e., problem definition, diagnostic evaluation, explanatory talk)?

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 11, 2015 | 10:21 p.m.

    Hi Jessica – Thanks for your question. Yes, these categories are drawn from our prior research (e.g. Sullivan, 2011). We have been working over the last 10 years to look more closely at student learning with computational media, such as robotics. In our 2011 paper, we defined a troubleshooting cycle that is characteristic of student activity when solving robotics problems. This cycle includes diagnostic evaluation and argumentation (explanatory talk). Problem definition is a characteristic activity of most collaborative problem solving groups – people need to agree on what the problem actually is before they can work on solving it. We are working with our prior microgenetic analyses of robotics problem solving to help guide this work.

  • Icon for: Jessica Hunt

    Jessica Hunt

    Facilitator
    Assistant Professor
    May 11, 2015 | 11:58 p.m.

    Thanks, Florence! I’ll take a look at the paper you referenced.

  • Icon for: Stephanie Teasley

    Stephanie Teasley

    Facilitator
    Research Professor
    May 11, 2015 | 06:39 p.m.

    I applaud your effort to create a more automated way of capturing and analyzing CSCL talk. However, the stacked bar chart did not simplify your data for me or make it easy for me to understand the talk that occurred. Good data representation can be very tough for complicated micro-level data such as this. Do you do work with “data viz” people to help you?

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 11, 2015 | 10:30 p.m.

    Hi Stephanie – Thanks for your question. At this point, the stacked bar chart is not a final representation of our work. Rather it is a tool for us in doing the analysis. The stacked bar chart represents utterances over time – clusters of utterances that have generated a large number of codes point us to places in the text that may be helpful in understanding student learning. For example, the chart in the video shows 7 clusters of utterances that generated a larger number of codes. Seeing this on the chart, we return to the actual text to perform close analysis of student talk. Therefore, the stacked bar chart is a step in the data analysis. We may use the stacked bar chart in a paper to explain the process and the tools, so, yes, we will need to think about how to present that in a more comprehensible way. Thanks for the tip on using data viz.

  • Icon for: Neil Plotnick

    Neil Plotnick

    Facilitator
    Teacher
    May 11, 2015 | 07:48 p.m.

    Has this technique for analyzing student discourse been explored in areas outside of robotic competition? I would be curious as to the types of words you selected in for this context as they would likely vary depending on what the lesson covered. In addition, I would be curious how specific word use maps to measurable learning. Do students that exhibit particular types of discourse have assessments that indicate growth?

  • Icon for: Florence Sullivan

    Florence Sullivan

    Lead Presenter
    Associate Professor
    May 11, 2015 | 10:41 p.m.

    Hi Neil – Thanks for your question. At this point, we have not used the technique on data other than what we have collected at robotics competitions (and in robotics units taught in science ed classrooms). Yes, the words that would be uttered would be different. Part of this method includes the analyst “training the data” using a small subset of his/her data set. So, a different context would entail the use of a different speech genre. However, the analyst, in training the data would indicate which terms were performing what work in the activity. Once a smaller sample of the data has been coded – the codes would then be applied computationally across the entire data set.Obviously, it is very important that the analyst have a high level of expertise with the phenomenon of interest in order to do a good job of coding/interpreting utterances. We are still in the development phase of this project – we may use inter-rater reliability to establish the coding system. At this point, two members of the research team have sat with the training data and discussed each parts-of-speech trigram and speech genre code. This mapping is the key to effective use of the method.

  • Further posting is closed as the event has ended.