Triangulating Assessment of Online Collaborative Learning

The implementation and assessment of collaborative learning is both complex and messy. This complexity is furthered in contemporary online learning environments as collaboration plays an integral role in the construction of knowledge through the various learning tasks set within an online course. Instructors often create collaborative learning tasks to link knowledge building (e.g., a community learning culture) from individual learning. Such collaborative tasks may range from short, in week tasks, to larger scaffolded projects that are completed over several weeks. Even though the course content may be adequately designed, the learning process that can be embedded through formative and summative assessment of the collaborative task can easily be overlooked or underused. Understanding and using appropriate assessment processes must be part of the overall learning design; it is important to integrate how assessment will play a critical role in collaboration and contribute to knowledge creation.

The focus of this article is to examine the interconnected components for assessing online collaborative learning by way of establishing the essence of collaboration in the online environment. Set upon the understand ing that “knowledge building is the creation and improvement of knowledge of value to add to one’s community” (Scardamalia & Bereiter, 2010, p. 8) and aligned with the online collaboration research of Ingram and Hathorn (2004), we present an assessment model that examines the interactivity of stakeholders involved in assessment of collaborative learning in online environments. Specifically, the model identifies how the learner, peers, and instructor need to be engaged in the assessment of learning through collaboration within online environments. Assessment design, assessment transactions, and assessment for knowledge construction are discussed as part of the online collaborative learning assessment model (Figure 1).

The concepts of group work, cooperation, and collaboration may be similar, yet each is unique. Group work brings learners together for communicating about the subject matter without direct instructor facilitation (Smith et al., 2011). The democratic, interdependent nature of group work is an autonomous function (Jahng, Nielsen, & Chan, 2010). However, Jahng et al.’s description of group work leaves the function of interactivity between group members undefined.

Cooperation is defined as a “style of working, sometimes called ‘divide-and-conquer,” in which students split an assignment into roughly equal pieces to be completed by the individuals, and then stitched together to finish the assignment” (Ingram & Hathorn, 2004, p. 216). Similarly, Dillenbourg and Schneider (1995) described cooperative learning as 舠a protocol in which the task is in advance split into subtasks that the partners solve independently.” These definitions of group work and cooperation acknowledge that each learner in a group works to address or solve a portion of the problem. From this group effort, information is shared and a conclusion or product is created. As such, the interactivity of sharing information allows for knowledge experts to be formed within the group. The cooperative context of sharing information in the task engages students in knowledge clustering and partition of knowledge, in general. Dialogical exchanges that take place are specific to sharing information such that individuals complete a project”s components. The overall learning outcome of such tasks often results in learners having a deep contextual understanding of their own contribution, and a more surfacelevel understanding of all components in the project.

Collaboration, as defined by Dillenbourg (1999), is when two or more people learn together. Adding to that definition, the Galileo Educational Network (2008) noted that collaboration is 舠a structured, recursive process where two or more people work together toward a common goal—typically an intellectual endeavor that is creative in nature—by sharing knowledge, learning and building consensus” (p. 1). Collaboration as described by Hungwei, Ku, Wang, and Sun (2009) involves the 舠interdependence of individuals as they share ideas and research a conclusion or produce a product” (p. 196). In a collaborative learning environment, people are actively engaged in knowledge construction that is cocreated, not owned by one particular learner after obtaining it from the course materials or instructor (Brindley, Walti, & Blaschke, 2009).

It is important to define the interactions of individuals within a collaborative group to clearly identify how collaboration can assist in supporting, motivating, and sustaining learning. Haythornthwaite and Andrews (2011) argued, “a collaborative, cooperative community holds the promise of more satisfied, supportive, and active learners, committed to the group”s learning goals” (pp. 111–112). Collaboration involves a different relationship between its members than what occurs with group and cooperative work and maintains a common goal or purpose. The learning tasks are designed in such a manner that all members need to contribute to the key components of the group”s work (Graham & Misanchuk, 2004). In collaboration, “individuals cannot compete against one another, because they are accountable for the product as a group” (Ingram & Hathorn, 2004, p. 218). Ingram and Hathorn (2004) identified the following three elements of collaboration:

1. Participation: Requires approximately equal participation among the members.

2. Interaction: Requires members to actively respond to each other as part of the discussion dynamic.

3. Synthesis: The product developed by the members is representative of the synthesis of ideas and a result of input from all members.

The product of collaboration is a direct result of the interwoven contributions of the members. This knowledge collaboration results in deeper and richer learning experience than could occur by individuals working independently. The composites of collaboration (e.g., dialogical exchange, recursive process, and building consensus) suggest participants in collaborative experiences move beyond the sharing capacity of group work and cooperative work. Therefore, true collaborative learning tasks elicit complex learning outcomes that suggest critical dialogue and exchange of ideas are further constructed to develop knowledge synthesis as described by Ingram and Hathorn (2004).

Online collaborative learning “emerged with the invention of computer networking and the Internet, and the concomitant socioeconomic shift from the industrial society to the Knowledge Age” (Harasim, 2012, p. 12). As noted by Harasim, online collaborative learning has an “emphasis on student discourse and collaboration” (p. 87). Therefore, it can be deemed that through “discourse and knowledge building mediated by the Internet; learners work together online to identify and advance ideas of understanding, and to apply their new understanding and analytical terms and tools to solving problems, constructing plans or developing explanations for phenomena” (Harasim, 2012, p. 88).

The dialogue created during online collaborative learning is not focused on the use of the technology to complete the task or about posting a single comment. For example, a study on technology use by Dunleavy, Dexter, and Heinecke (2012) suggested that technology does not automatically facilitate additional learning with students and consequentially does not portray a positive financial cost to learning benefit in all situations. Technology enables the learners to share information allowing the building and maintaining of ideas over time among more than two people. Termed “technologies of participation” (p. 84) by Haythornthwaite and Andrews (2011), technology applications that promote dialogue (e.g., blogs, wikis, and social networking tools) provide the forum or environment in which the collaboration occurs. Instructors need to continue to ask themselves the pedagogical-forming question posed by Farwick Owens, Hester, and Teale (2002) “What does technology allow me to do better, not just differently” (p. 625) when looking to foster meaningful learning. The use of digital technologies as noted by the Galileo Educational Network (2000-2014), need to be “used in ways that mirror their use in the discipline(s), the world beyond the school, and extend, expand, and deepen student learning” (p. 5). As such, students, along with the instructor, need to use technology in meaningful and purposeful ways to support the collaborative learning.

Fundamental to constructivism is that learners “construct their own understandings of the world in which they live” (Sergiovanni, 1996,p. 38). Shallow constructivism, according Scardamalia and Bereiter (2003) results in students demonstrating “little awareness of the underlying principles that these tasks are to convey” (p. 1371). With deep constructivism, “practices such as identifying problems of understanding, establishing and refining goals based on progress, gathering information, theorizing, designing experiments, answering questions and improving theories, building models, monitoring and evaluating progress, and reporting are all directed by the participants themselves” (Scardamalia & Bereiter, 2003, p.1371).

Learning through a social constructivist approach “is essentially a social activity, that meaning is constructed through communication, collaborative activity, and interactions with others” (Swan, 2005, p. 5). In the online collaborative environment, the design and facilitation of the learning tasks need to provide opportunities for students to interact and work with others through questioning, sharing, discussing, and negotiating meaning that leads to co-construction of new knowledge. Effective collaboration in the online environment “involves interactions with other people, reciprocal exchanges of support and ideas, joint work on the development of performances and products, and coconstruction of understandings through comparing alternative ideas, interpretations, and representations” (Wiske, Franz, & Breit, 2005, p. 105).

When students are given opportunity to cocreate knowledge, according to Jacobsen, Lock, and Friesen (2013), the instructor”s role changes. The role is no longer that of a transmitter or deliverer of information. Rather, the instructor becomes a “designer who is intentional about the work he or she asks students to do.” They further argued that, with this change of role, the instructor needs to also be “mobile and responsive to individual and group learning needs, and to providing ongoing feedback to help all learners continually improve their work.” Various forms of assessment play a critical role in the collaborative learning process in support of the co-construction of knowledge.

Online collaborative learning tasks need to be developed using a gradual and graduated approach. Each new learning component builds upon the previous in support of achieving the learning outcome. This supportive approach requires an iterative design supported through effective scaffolding by the instructor. With each component in the collaborative learning task, the instructor should ask him/herself, how will the student demonstrate understanding of x? Addressing the current practice of iterative assessment, Taras and Davies (2013) noted postsecondary instructors are not aware of the implications and procedures for differing types of assessment. Their study lends itself to the need for further understanding of assessment is warranted.

A consideration for enhancing assessment practices is Kaslow et al.”s (2007) principles or guidelines designed to enhance the student learning experience through an assessment of competence. To exemplify how assessment of competence is achieved and sustained, they suggested such items as: inclusion of collaboration across constituencies, use of self-assessment and self-reflection, inclusion of comprehensive assessments, and continued training on assessment.

Online collaborative learning can involve both summative and formative assessment to assess competence. Each form of assessment has its specific purpose and place within the assessment design. The purpose of summative assessment “is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark” (Eberly Center, n.d.). Black and Wiliam (1998) defined formative assessment “as encompassing all those activities undertaken by teachers, and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged” (p. 7). Strategically used together, both formative and summative assessment can advance and strengthen learner outcomes.

A key component to this strategic interworking is how the feedback is used to inform the next steps in student learning.

Formative assessment in online collaborative learning can be designed to engage the learner in self-assessment of his/her work in various ways. For example, when using blogs, the learner can review the work in succession. Reading in progression allows the learner to reflect on his/her learning progress and process of learning. Furthermore, the online environment allows for this learner to compare his/her work to that of others who blog.

Within the online environment, peers can easily review each other”s work and provide feedback. This sharing further integrates the construction of knowledge among collaborative learners. In a study of online collaboration among music learners, Waldron (2013) explored the interactions from the initial stages of a learner posting a video and how the learning outcomes were achieved through the collaborative interaction of peers. Waldron suggested outgrowths of student learning through online sharing may include the opportunity for positive debate—and its ability to help further peer understanding. While a blog provides opportunity for the student to develop his/her ideas and thoughts, peer assessment through interactive dialogue becomes a secondary assessment process outgrowth. Learners continue to self-reflect as peer-discussion and debate contribute to the deeper contextual knowledge sharing. As the learner-learner interaction continues, the initial task may be altered or adapted to adjust to the new learning afforded by the peer interactions. With an online collaborative task, formal peer assessments may be created by subsequent tasks that require learners to respond to peer discussion posts or complete peer-reviewed evaluations. The integral part of formative assessment is the provision of opportunity by the instructor to allow for the learner to use the feedback for application to his/her learning (Adams & McNab, 2013).

Another means of formative assessment involves the instructor providing feedback to the individual student. Linking back to Ingram and Hathorn”s (2004) work on participation, interaction, and synthesis, feedback needs to be timely, specific to the learning task outcomes, and demonstrates analyses for possible opportunities and challenges. Online environments provide the instructor with various means to communicate and provide feedback to the learner through such things as e-mail, discussions boards, and video conferencing. There is a vast array of communication collaboration technologies that can be used to support the achievement of the learning outcomes.

Through self-, peer-, and instructor formative assessment, assessment can be a continual process that positions the learner for further development of their learning and can provide multiple opportunities for refining of the work. Eddy and Lawrence (2013) further explored this notion of iterative assessment as they identified four stages for authentic assessment: continued evaluation, assessment through learning experiences, use of multiple assessors, and the opportunity for learner choices. The notion of iterative process plays a critical role in assessment.

Formally termed learning and knowledge analytics, activities completed and viewed by learners can be observed from sourcing, managing and analyzing their data within the learning management system (Tempelaar, Heck, Cuypers, van der Kooij, & van de Vrie, 2013). Instructors can use the data to view learner progress, identify possible content or assessment concerns, as well as sees patterns of use among learners to aid in the understanding of learning outcomes (Verbert, Manouselis, Drachsler, & Duval, 2012). Verbert et al. further identified how data sets are parsed to assist in the predictability of learner outcomes and aid in identifying additional resources that will be helpful for learners.

The University of Saskatchewan began exploring the use of learning and knowledge analytics in their evolution of the Comtella article resource mechanism (Vasseliva, 2008). This study demonstrated the notion of learners being motivated to participate in the learning activities through the visual incentives provided by Comtella”s inclusion of visualization. The visualization component builds upon the idea that learners will participate when given visual incentives, or social gratification. From this particular study in the use of learning analytics for the development of learner participation and increased motivation with their learning, it was noted that various forms of incentives (e.g., visual, intrinsic, and social incentives) are points of impetus for motivating students toward achieving learning outcomes.

The affordances of specific technologies can be used as means of providing information on learning. This type of information may be helpful to the individual, to the collaborative group, as well as to the instructor in terms of using it to inform next steps in the learning and teaching. There is a need to further explore the potential role of technology in assessment for collaborative learning in online environments.

An online collaborative learning environment is a complex space involving dynamic interactions between students (peer assessment), students and instructor (instructor assessment), and each student within themselves (selfassessment). The assessment strategies employed to assist learning in reaching deep levels of understanding can be unique to each learning task. In this assessment model, three main levels need to be addressed in support of assessing online collaborative learning: (1) assessment design, (2) assessment transactions, and (3) assessment for knowledge construction. Within each of three levels, online collaborative learning is supported through various integrated assessment components that influence the advancement of knowledge construction (see Figure 1).

The foundation of online collaborative learning is built upon the assessment design. Careful attention needs to be given to the alignment how the learning outcomes will be achieved through the particular learning task and types of assessment used (formative and summative). The overall design of the learning, which includes assessment and interaction among learners and instructors, is influential in moving the learner toward self-regulation and motivation for learning (Zusho, Karabenick, Bonney, & Sims, 2007).

In designing the collaborative learning task, learners need to be given opportunities to demonstrate their knowledge and skills through various methods. For example, the task can be designed so that it can be assessed whether the work is in text, oral, video, or graphic format. This form of multiple engagement in learning is based on the work of Meyer, Rose, and Gordon”s (2013) universal design for learning. Universal design for learning is built upon the premise that learners come to their learning through unique means (e.g., cultural, neurological, etc.) and they should be able to engage in their learning using a variety of means and methods.

Another element in the assessment design is the incorporation of trans-discipline approach that allows learners to create a broader understanding of “knowledge of knowledge” rather than knowledge fragmentation (Quattrone, 2000). Quattrone argued that the inclusion of trans-disciplinary approach at a philosophical level permits the various academic disciplines to experience “cross-fertilization” (p. 3) and decreases the limitations set within contextual hierarchical discipline structures. The purposeful inclusion of trans-disciplinary processes allows for students to interact with the subject matter in ways that permit discovery of domain specific ideas while tapping into practices that inform fundamental learning skills and habits of mind. Focused on practices that transcend one specific discipline, trans-disciplinary learning involves cognitive thought processes that can assist students in applying previous knowledge experiences to solve current problems, create novel ideas, and understand various viewpoints (Mishra, Koehler, & Henriksen, 2011). Set within contemporary teaching and learning practices and everchanging technologies, the ability to make both connections in and across disciplines and to the broader context of knowledge becomes of substantial consequence for learners. From its constructivist mooring, the experiences developed during the interactions of trans-disciplinary processes between subject-knowledge and metacognitive knowledge, unique learning experiences are explored and developed. These cognitive connections are “metalevel thinking skills, which respects the importance of disciplinary knowledge while allowing for novel connects” (Mishra, Henriksen, & the Deep-Play Research Group, 2012, p. 19).

This includes self-assessment, peer-assessment, and instructor assessment. The learners” progress and performance with collaboration, as noted by Palloff and Pratt (2013), should not only be for the instructor to monitor and report. They argued that learners “should be encouraged to comment on one another”s work, and self-assessment should be embedded in the final performance evaluation of each student” (p. 42). The ability to learn from self and peerassessment is a critical component in collaborative work. Further, technology can play a role in how the assessment is captured (e.g., audio, video), as well as how it can be used to inform next steps and/or to present/re-present the final product. The instructor working with the learners needs to create and sustain a culture of self, peer and instructor assessment that informs the collaborative learning experience.

The nature of providing opportunities for meaningful online collaboration influences knowledge construction. This type of knowledge is viewed as multidimensional. Described by van Aalst (2009, p. 261) as “effortful, situated, and reflective,” he also associated it with deep content understanding, meaning-making and activating a variety of meta-cognitive processes and connections. Gained through interactions between members (e.g., knowledge exchanged by various means of discourse), learners” construction of knowledge is dependent on the interactions within the design and assessment of the online collaboration. Furthermore, the development of intentional ways in fostering greater engagement through active learning requires the learner to “make information or a concept their own by connecting it to their existing knowledge and experience” (Barkley, 2010, p. 17). With the creation of intentional discourse in online collaborative learning tasks, learners have increased opportunities to connect with peers to extend their learning community. As these interactions and activities are occurring through collaborative learning process, assessment plays a key role in the construction of knowledge. As such, careful consideration needs to be given to how the assessment practices are used to foster knowledge construction.

The online collaborative learning assessment model highlights the relationship and interconnectedness of the components. Self-, peer-, and instructor assessment is needed in support of robust collaborative learning. The central focus of the model is to support learners in their knowledge creation through meaningful online learning experiences. Learners and the instructor in online collaborative environments need to share in the active learning process and in the assessment of and for learning. The product of using this model is to support the design and implementation of a shared learning experience that fosters a critical learning exchange that supports transformational learning.

Online collaborative learning that fosters knowledge construction is complex. Within this complexity, a critical element is that assessment forms an integral component woven into the design of the online collaborative learning experience. Furthermore, online collaborative learning assessment addresses the nature of collaborative learning—that which is participatory, interactive, and develops a synthesis of knowledge ideas (Ingram & Hathorn, 2004). Together, the stakeholders (e.g., the learner, the peers, and the instructor) need to purposefully and actively embrace in an assessment practice that both informs and furthers the ongoing improvement of the work (formative assessment), as well as in assessing the final product (summative assessment). The fostering of knowledge construction can be encouraged through robust collaborative learning and intentional, participatory assessment practices.