Thanks Bill! The entropy graph that I briefly show at the end is a bit technical in nature, but basically it is looking at the distribution of a given variable (e.g., the total duration of a students’ speech) across the group of three students. We were trying to produce a single value (the Shannon entropy) to describe the distribution, and tried entropy as that measure, but it seems to be turning out to only be useful for differentiating groups at the end of the spectrum and not in the middle (since the entropy value does not distinguish between different types of groups well there). We think we need two values, not one, to describe the distribution better and help distinguish the groups (which is still better than the original three values I suppose).

The items we are using are from the Cornerstone math curriculum (based on SimCalc) that Jeremy Roschelle and other colleagues at SRI have been developing for over a decade now. We had to do a bit of adaptation, but not too much. Basically, we created an iPad app that allowed students to use remote controls to register their answers. (Part of the reason why was so we could log their actions better and part was so that we could try to get each student to be in charge of their answer – in other forms, like just using a keyboard, some students would easily just take over answering for another student.) So, the form of their interaction was changed, but the content of the items was not.
Thanks for your comment!

I was interested to note that your items were designed for groups of 3. Was there a reason that you wanted 3 students to work together? Could adaption be done for groups of 2 or 4?

The items we were using were already designed for groups of 3, so that required the least amount of content adaptation. But, we did purposely want to avoid groups of 2 because that is a different dynamic in many cases. (We have some data on pairs using the system where they share the responsibility of the third response, but I don’t think we’ll have enough of that situation to make any strong claims about differences.) I think that groups of 4 would be more similar to the groups of 3 in how they distribute and spread out the intellectual work of the group, so it could probably be easily adapted to that size.

It is still early in the process, as we are currently working on full analysis of the first phase of data collection, but basically we are trying with this exploratory project to see whether or not there is any linkage or relationship between the machine-detectable features of language and the quality (and/or features) of collaboration. I’m hoping by the end of the project we can produce a usable prototype system that will be able to give a teacher some kind of sense of how students were collaborating in their small groups and whether or not they need to intervene. We might also, depending on the results, be able to give specific feedback about what a particular group could do to improve their collaboration (e.g., ask each other more questions or make sure all three people are contributing to the discussion). I think we will be able to say groups with more ‘good collaboration’ are characterized by [these] speech features and groups with less collaborative activity typically have [these] kinds of features.
Thanks for your question!

Fascinating! I’m really eager to hear what you learn from this.

Right now we are using it with middle school students.
We are collecting the log data from the application so we know every time a student changes their answer, as well as when the student presses their button to submit an answer choice. Additionally, one of the things we are manually coding is whether or not they are giving an explanation about their answer choices to the fellow students. Early results seem to show that groups that have more explanations tend to have better collaborations.

Thanks so much!

Thanks!

Yes, there is a big issue with being able to distinguish childrens’ voices. Even when we’re watching the video it is sometimes hard to tell who is talking. This is one of the reasons why we collected individual audio channels for each student. But we also collected a single audio stream of all three in each group, and so one thing we’ll be looking at is whether or not we can distinguish them and also whether or not we need to. In many cases, it seems possible that there are other features of speech, besides who is talking, that can give us enough information about the collaboration.
Thanks for your comment! Good luck with your project.