Hi Colleen,
Thank you for your questions! I separated your questions into three sections below.

Q1: I chose to not provide any additional help up front for students, but rather just have them do the discussion and then assess themselves. I knew full well (from other research and previous experience) that they were going to be in accurate at assessing themselves. My student assessed the first discussions and then gave them to the students to compare with their own assessments. They are always so perplexed by the fact that they usually are rated much lower than they think, but after looking at our evidence they come to agree with it. Many report going back into the system to try to understand our ratings. I think this initial confusion and added sense making is important. We spend too much time preventing students from being confused, but a little confusion can be good for learning.

Q2: In my previous work I developed four competencies, two focused on task management and two on discourse. The ones in this video focus on discourse: information synthesis and knowledge negotiation. The other two are Planning and Productivity/Quality control.

Q3: I’m not sure we can ever make online and resident equal as there is something to be said for being in a room with someone. However, each type of environment provides opportunities for different types of learning. Online environments can provide things that face-to-face environments cannot and that is what I think we need to develop for our students. The CREATE environment is not fancy, it’s quite simple really, but it provides a record of discourse that students can examine and evaluate for evidence of different processes. You can’t do that with face-to-face environments and so collaborative training of this sort is more difficult without the online experience. However, many of our students have reported thinking about the items in face-to-face settings after having done the activity. They find it funny, actually. They say they can “see” people being dysfunctional and imagine scoring them ☺

Here is a link to the assessment rubrics that are currently in the create system: http://sites.psu.edu/mborge/wp-content/uploads/...

You can find more resources on my web page: http://sites.psu.edu/mborge/helpful-resources/

Hi Marcela,

Thanks for the response! And sorry for the long list of questions!
Q1 – that makes sense – I think the fact that they can audit their discussion makes this all possible. I could imagine in a face-to-face discussion, it might be helpful to prime students with the rubric information because this auditing wouldn’t be possible.
Thanks again for your responses and the rubrics. I’ll tweet out the rubrics I think people will find them really helpful!
- Colleen

Never apologize for asking questions, especially when you ask so many good ones :) Thank you for tweeting!

I think that this is something that could definitely help classroom teachers and students improve their practice. It made me wonder, did the idea to evaluate the collaborative process come from an in-person class setting or was it entirely based off of data/experiences from online settings?

This project was motivated by my PhD work, way back in 2002! Back then I was working with a research group trying to develop students’ understanding of science inquiry. We were working in urban schools and the biggest problem I saw was a lack of collaborative abilities. I came up with short guides and an instructional model to help student groups develop their ability to monitor and regulate collaborative processes. The biggest weakness with this work was that I had no way to assess change in collaborative processes and so could not see whether or how they were improving. So I set out to try to find a way. This was difficult because collaboration is a broad term with no concrete observable measures of quality. So I had to draw on research in industrial and organizational psychology, business and innovation, as well as conduct my own research to try to determine the types of processes most associated with high performing, innovative, problem solving teams. Most of this later work was conducted with older students in undergraduate usability engineering (face-to-face), information sciences courses (online), and laboratory settings (face-to-face).

Hi Barry, thank you for reducing my tech envy; much appreciated. I have investigated transfer to some degree. i.e., whether process learning extends beyond the course activity to new contexts. However, this has only been investigated through self-report and I haven’t published these findings yet so I cannot give you the exact frequencies. I can say that there are noticeable patterns between who reports thinking about the assessment items in other group contexts outside the course (including face-to-face contexts) and specific ways that they interacted with the system. So we are seeing some evidence of internalization. Personally, I think this is really interesting because it presents the potential of using such simple online tools to train folks for face-to-face collaboration- but no studies have been conducted on this yet. Many middle school teachers I know say they see great potential for such a tool to teach younger students how to interact in digital and face-to-face environments.

Thank you, Sue. You are so right, not all students get the benefit of household environments with people who have the time, knowledge, or resources to guide important types of development. Sadly, school only focuses on forms of development that are not always useful for general daily life function. Collaboration encompasses important developmental processes like joint attention, socio-emotional regulation, and metacognition. It is also extremely important for individual success, because if you know how to socialize and share ideas with others in competent ways, people will be more drawn to you. So it makes sense to find ways of integrating tools into K-12 to help develop these skills. And It is people like you who can help to ensure that NSF projects reach more diverse populations! This work began in K-12 classrooms and so I would love to go back to that context. Interestingly, a friend of mine (who is a developmental psychologist) suggested that tools like CREATE could be useful for kids with autism, to help them learn better patterns of communication.