Hi Barry! Nice to hear from you again! It was nice to see you at AERA, btw.

Thank you for your recognition of the theoretical foundation of this project. This is a great question, Barry. It is something that we are really digging into this year.

One of the things we know about collaboration research is that speech and discourse is a main component for how people understand the way people work together. While there are some studies that look at prominent gestures and behaviors that are associated with collaboration, there aren't a lot of studies that help us understand the way people interact with each other as part of group work and how they are associated with knowledge building, shared goals, shared plans, etc. we also know that a big part of what we say isn't just in the words but it's also in the mannerisms and the gestures that our company the words. So, we want to look deeper into that. Steinbock’s paper talks about the relational space of collaboration- which can be looked at from a behavioral lens. Something that we're looking into right now is how gesture and speech work together as participants communicate their ideas and negotiate knowledge and groups. Understanding behavior is part of what Steinbock calls being mindful of process which is where you attend to and develop the social relational space during- in addition to the subject matter development and negotiation. On a very practical note- we use vision to help us make sense of what is being said, and we are trying to understand how to potentially support teachers in using their resource of sight (in a loud and busy classroom, for example) when managing a classroom with multiple groups working simultaneously.

In terms of the small variation, yes. There are small variations that are present. We performed an extensive literature review to help us create an architecture of behaviors and a coding rubric to identify and capture those behaviors. We are using machine learning to help us organize them and determine how they relate to one another and whether there are indeed differences.

I feel like I am missing something in this questions- let me know.

Hi Andresse. Thank you for your question. For the study we presented, we worked with both adults and students. Our conceptual model for collaboration was initially made for middle school age students and was refined as we looked at more and more data. When we collected data from adults in online spaces, we revised our model and corresponding rubric to match the adults and the online work platform that we designed. We found that while there were some changes, there were not as many differences that we expected to see. There is some literature (I can't remember at the moment) that says that after 14 or so, behaviors start to even out and become more consistent. It is also worth mentioning that the study was done in a lab setting, so the students and adults were self selected and were more likely to behave similarly. We have yet to try this in authentic work spaces.

Hi Julie, Thank you for your question. I wrote a novel in response to Barry- so please take a look at what I wrote and see if you have an additional questions. I will be happy to expound on anything. Arif Rachmatullah was the lead analyst for the student perception work and developed the concept behind that. I will invite him to respond to your question.

Hi Julie, thank you so much for your question. We used a survey after collaboration for students to reflect on their participation. We selected, modified, and re-validated the items developed by Dewiyanti et al. Please let us know if you are interested in these survey items; we are happy to share them with you. 

Hi Arif, thanks for your reply. I am definitely interested in these survey items. While we continue to build out more behavioral metrics for collaborative problem solving at iSAT, we are looking to self-report items in the mean time. If you wouldn't mind sending along the survey, my email is julieharrison@gatech.edu. Thank you!

Hi Ekundayo. Thank you for your vote! This is such a great question. Measuring collaboration is so difficult- as I am sure you know, it is the hardest type of group work out there. We really had to break things down into smaller parts and look at things separately. We are in the process of bringing those parts together to see how they interact. Some of the surprising things that we saw last year were that higher completion rates of tasks among groups did not necessarily translate into better collaboration quality. We often see collaboration turn into more competitive sports in classrooms, where students are racing against the clock. With this, we saw the quality of their group artifacts were not affected by the quality of their collaboration- this was surprising. This was based on a small sample size. We decided to dig deeper into this by looking at the features of the tasks/activities to determine whether there are certain types of tasks that correlate with certain types of collaboration qualities. This is still being investigated. I will invite Hui Yang, the other lead analyst on the behavioral roles to say more about what she is finding there.

Hi Ekundayo, adding to what Nonye has said, we actually have some findings related to the relationship between groups' mixed-gender status and the artifact quality the groups generated. We hypothesized that mixed-gender groups would be a positive predictor of artifact quality; however, the results showed that it was not a significant predictor. The only group demographic factor that significantly predicted artifact quality was group ethnically diverse status (whether the group consisted of members from different ethnicities). Again, these findings were still based on small sample size, and we are still collecting more data to obtain a more robust statistical model. And thank you so much for the vote!

Hi IIana, thank you for your comment. The visualization of students' collaboration behaviors that you saw in our video is a part of our behavioral analytic technology for analyzing students’ collaboration quality. Our team developed a process to stratify our collaboration conceptual model for developing the machine learning models in our automated collaboration assessment tool which could analyze students' collaboration behaviors (including the body movements) and provides feedback on their collaboration quality. My colleague Anirudh Som, the task lead for the tool development, could provide more details on the development of the visualizations of students’ bodies.

Hi there- thanks for your question. I just want to add to Hui's response. To use of the visualization that you saw give a multidimensional view of the all of the students so that the machine learning models can identify movements from different angles. The human annotators were also able to see different angles of the students when coding the videos. Those visualizations provide a more detailed account of what students are doing so that the behavioral analytics can have more to work with.

Hi Janet, thank you for your question. The current work which Arif has been leading suggests two types of group configuration that can be utilized to promote the quality of collaboration. The first configuration is when all students in a group have relatively high levels of science confidence with relatively equal chances to contribute to the collaboration process. The second configuration is when group members have different levels of science confidence and different chances to participate during collaboration. In this situation, group members who have higher science confidence levels have more opportunities to clarify concepts to members with lower science confidence. Arif can provide more details on these findings if you would like to learn more. 

Hi Janet, thank you so much for your question. I want to add to Hui’s response, especially related to your question about the analysis of movement and gesture. We haven’t yet analyzed the relations between science confidence and types of movements/gestures. One of our in-progress work is looking into the relationship among types of gestures (pointing, representational, working), types of talk (explanation, questioning, suggesting, etc.), and the groups’ artifact quality. We’re finding fascinating insights based on these relationships, which actually bring in another factor (i.e., task features). Our preliminary findings, as well as literature, seem to show that students’ behavior and movements tend to be dependent on the tasks, such as whether the task is abstract enough so that talk only does not afford students to express all their ideas, so they need to offload the thoughts into gestures. We’re still digging into this topic and cannot wait to share our findings with others.