This study used an online concept mapping activity (CMA) featuring peer learning to enhance learning achievement in concept application. Ninety-seven graduate students participated in this study. The students who participated in the online CMA could later apply the concepts with significantly higher performance and greater fidelity than those who did not. Findings of this study advance CMA for concept learning and application by integrating new features into CMA that motivate students to adopt deep learning approaches and develop effective cognitive information processing ability for better concept application
Graduate students in an online course reported difficulties in applying abstract concepts of learning theory into practical instructional settings, the most important goal of the course. Meanwhile, the instructor of the course found that a large number of students used learning approaches that engaged very little cognitive information processing. For example, some students completed their assignments by putting together disparate pieces of information from the textbook without logically integrating the information; other students failed to apply concepts into instructional settings with consistent meanings and implications of the concepts.
Research has identified two major ways in which students approach learning: surface and deep learning approaches (Biggs, 1999; Hall, Ramsay, & Raven, 2004; Marton, Dall’Alba, & Beaty, 1993). Surface learning approaches mainly rely on memorization and copying information from learning material without seeking connections, meanings, or the implications of what is learned. By using surface learning approaches, students are unlikely to achieve high-quality learning outcomes, develop high-level knowledge acquisition and competencies, or accomplish concept applications (Prosser & Trigwell, 1999). In contrast, deep learning approaches seek meanings, search for relationships among the concepts, and integrate the newly learned knowledge into existing knowledge structures and prior experiences. By using deep learning approaches, students are more likely to result in better retention and application of knowledge (Booth, Luckett, & Mladenovic, 1999; Ramsden, 1992).
Two reasons may explain why students did not choose deep learning approaches: ineffective cognitive information processing (CIP) ability and low learning motivation. CIP refers to the process of seeking the meaning of learning material that is an essential skill for accomplishing concept applications (Hall et al., 2004; Laight, 2004). Students might have less-developed CIP ability to draw sufficient meaning from learning material for concept application. On the other hand, student motivation may affect their choice of surface or deep learning approaches (Hall et al., 2004). Students with less learning motivation, regardless of their CIP ability, might choose to acquire a minimum level of knowledge for completing assignments and engage little or no concept application. Thus, for students to adopt deep learning approaches, learning activities must be able to facilitate students to develop effective CIP ability and, at the same time, motivate students to practice concept application.
Concept Maps And Concept Mapping Activities
A concept or knowledge map is a two-dimensional, hierarchical, and node-link diagram that displays the relationships of a set of concepts (Novak, 1998; O’Donnel, Dansereau, & Hall, 2002). The nodes present the concepts and the links describe the relationships among concepts. As instructional media or graphic organizers, concept maps facilitate students to acquire conceptual knowledge with deep comprehension of learning materials (Alpert, 2003; Sowa, 2000).
Concept mapping activities (CMAs) are learning activities in which students draw concept maps to represent the conceptual knowledge they read from learning materials (Chang, Sung, & Chen, 2001; DeSimone, Schmid, & McEwen, 2001; Kinchin, 2001). Through concept mapping, students build a solid schema and visual representations of a set of abstract concepts that are ready to be applied to various application situations (Beveridge & Parkins, 1987; Catrambone & Holyoak, 1989; Cooper & Sweller, 1989). The process of translating information about the concepts to be learned from text to a node-link graphic requires students to process the meaning of the text more thoroughly than they normally do when reading text. Also, students must make appropriate judgments in order to describe and classify the nature of the relationships among concepts (DeSimone et al., 2001; Jonassen, Beissner, & Yacci, 1993; Kinchin, 2001). While participating in CMAs, students must comprehend the definitions of individual concepts to search for the fundamental essences, differences, and similarities among concepts. The process of searching deep comprehension associated with CMAs engages students in substantial CIP for conceptual knowledge acquisition and application that promotes deep learning approaches.
Empirical studies reported the effectiveness of CMAs for learning concept application with deep learning approaches. More than 250 science students used CMAs; one third of them reported that CMAs made them to think deeply; half of them stated that CMAs helped in understanding the relationships between concepts (Santhanam, Leach, & Dawson, 1988). A study involving 187 college students found that students who participated in CMAs had higher capability in concept application than those who did not (Parkes, Zimmaro, Zappe, & Suen, 2000). Furthermore, CMAs that focus on constructing cognitive structure of knowledge are reported to better facilitate concept applications, compared to other learning activities that focus on memorizing information (Gentner, 1989; Paas, 1992). A meta-analysis of 5,818 participants learning with concept maps in 55 studies found that activities related with concept maps, either in face-to-face or online instructional settings, were more effective for knowledge application, compared with other learning activities, such as reading text passages, attending lectures, or participating in class discussion (Nesbit & Adesope, 2006).
In computer-based learning environments, research studies reported advantages of using collaborative CMAs for concept learning. Twenty-six graduate students, in two groups, participated in computer-based CMAs and showed that both groups continuously altered their group concept maps, which indicated the acquisition of new knowledge by individual students and concept negotiation among group members (DeSimone et al., 2001. In another study, 17 college freshmen in small two- to three-member groups, used CMAs within a Web-based collaborative inquiry learning system to solve instructor-given problems, and 81% of the students reported that CMAs were helpful for constructing group concept maps and reconciling the differences in conceptual knowledge among group members (Chang, Sung, & Lee, 2003).
Although studies documented the benefits of CMAs as facilitating the development of student CIP ability, using CMAs may incur two drawbacks and influence student motivation (Chang et al., 2003; Chang et al., 2001;
DeSimone et al., 2001. One drawback is that constructing concept maps is a difficult task for most students and training is often recommended. Students need specific instruction to learn how to extract meaning from text to create concept maps individually and how to create a collaborative concept map in group-activities. Such training requires extra effort from both students and instructors in addition to learning and teaching the scheduled content. The other drawback is that concept mapping is a time-consuming long process. Students must extract meaning from text, integrate feedback, and revise concept maps multiple times. It also takes additional time for instructors to demonstrate, coach, and provide feedback on student concept maps. For collaborative CMAs, even more time will be needed for group coordination, communication, and negotiation.
While CMAs used in existing studies may share the same core activity of concept map drawing, they hardly addressed student motivation affected by the above-mentioned drawbacks. CMAs differ than other learning activities in many design features, such as, the nature and complexity of the task, the concepts to be mapped, the learning context, and the presence of examples. By incorporating appropriate design features, CMAs could be designed to alleviate the drawbacks and motivate students toward deep learning and concept application. This study proposed a CMA featuring peer learning, providing examples of concept maps and applications, and using a limited set of specified concepts. This study investigated whether students participating in this type of CMA would better apply the concepts than those who did not. It was expected that students who participated in this type of CMA would actively engage effective CIP and have higher motivation for deep learning and, subsequently, perform better in course assignments of concept applications, compared to students who did not participated in the CMA. The effectiveness of this type of CMA was examined by measuring learning achievement in concept application, rather than students’ subjective evaluation of their learning motivation.
Method
Ninety-seven volunteer graduate students (81 females and 16 males) at a Southwest public university participated in this study. Participants had an average age of 34 and completed an average of 1.52 online courses. Participants were solicited from six intact classes, one each semester, of one online course taught by the same instructor. The first three intact-classes did not receive the concept mapping activity (CMA) whereas the second three intact-classes received the CMA. The no-CMA group included 57 participants, 50 females and 7 males, who had an average age of 34 and completed an average of 1.56 online courses. The CMA group included 40 participants, 31 females and 9 males, who had an average age of 35 and completed an average of 1.45 online courses. This study compared participant’s scores of concept application assignments between the no-CMA (control) and CMA (treatment) groups.
The Course
The course was a required graduate course and delivered entirely online. Although there were no scheduled face-to-face class meetings, students were encouraged to request individual meetings with the instructor. The course introduced 10 learning theories and the course goal was for students to be able to design instructional strategies and competently apply the concepts of learning theories to practical instructional settings. Students were required to reading textbook chapters and online materials, participate in weekly online learning activities, and complete four concept application assignments. The weekly online learning activities included composing and posting applications of the theory onto a bulletin board and completing online quizzes. The four concept application assignments required students to design instructional strategies applying concepts of learning theories into the instructional settings specified by the instructor.
Variables
Concept application, the dependent variable in this study, was the total score of the four concept application assignments. The possible concept application scores ranged from 0 to 100 points, hence the highest score of each concept application assignment was 25 points.
Each assignment specified a learning goal for a specific group of learners and a learning theory that was different from the learning theory practiced in the CMA. It asked students to identify at least three concepts of the given learning theory and apply these concepts to design instructional strategies. About every 4 weeks, students completed one of the four assignments with different learning theories. The instructor, who was one of the researchers, graded concept application assignments strictly based on three criteria: the instructional strategies were clearly described and appropriate for the given learners to achieve the learning goal (40%), the instructional strategies were consistent with the concepts of the given learning theory (40%), and the rationale for using the instructional strategy was logical and consistent with theory (20%). Appendix A presents the grading criteria with detailed rubrics for concept application assignments.
The CMA, the instructional treatment in this study, asked each student to draw a concept map with nodes and links. Students were given three to five concepts of a particular learning theory, three instructional strategies appropriate for applying these concepts, and an example of concept map. Students were instructed to use nodes for the given concepts and instructional strategies and links for verbs that indicate the relationship between concepts and instructional strategies. All students were required to post their concept maps on the public bulletin board on the course Web site and were strongly encouraged to view other students’ concept maps. Concept maps were not graded and no feedback on concept maps was provided to participants. Two CMAs, each given concepts and instructional strategies of a different learning theory, were used in this study. Appendix B presents one of the CMAs used in this study.
The CMA incorporated three instructional features to enhance learner motivation and deep learning. First, the CMA offered an opportunity for students to learn from peers. Every student drew concept maps from his or her own perspective. Viewing other students’ concept maps allowed students to compare their own perspectives with others. Such comparisons required students to engage effective CIP and develop more meaningful knowledge from multiple perspectives (Dornisch & Land, 2002). Peer review and comparison also motivated deep learning; students who participated in Web-based peer-review learning environments reported strong intrinsic motivation in problem solving (Song, 2005); students who used a Web-based peer-reviewed system to construct, assess, and review questions of learning material reported high intrinsic motivation on the learning tasks (Yu, Liu, & Chan, 2005).
Second, the CMA provided examples of a concept map and three instructional strategies. Providing application examples is very important for learning how to apply theories into practice situations (Driscoll, 2005; Merrill, 2002). The given concept-map example served as a cognitive stimulus for students to develop effective CIP for judging proper relationships among concepts. The examples of instructional strategies served as a guide for students to create their own instructional strategies appropriate for various settings. Both the concept map and the application examples also made it easier and shortened the time needed for students to draw their own concept maps. Third, the CMA used a limited set of specified concepts, three to five concepts, to circumscribe the task of concept mapping. With only three to five concepts, the task of learning concept mapping became simpler and easier for students to manage and accomplish within a shorter time, compared to using a large number of concepts.
Due to the special features of the CMA in this study, the task of constructing concept maps became easy and could be accomplished within a short time. The special features of the CMA, including learning from peers, providing examples, and using a small set of specified concepts, were designed to overcome the drawbacks of the existing CMAs so that students were able to develop effective CIP and to increase motivation for deep learning.
Procedure
The instructor-researcher sent an e-mail to each student in all six intact classes to explain the purpose of this study and ask for volunteers. The informed consent form and demographic data survey were attached. The participants volunteered by replying with the completed form and survey in an attachment. Both no-CMA (control) and CMA (treatment) groups received identical online instruction and completed the same online learning activities. Only the CMA group received two CMAs, the initial CMA in the third week, one week before the first concept application assignment, and the second CMA in the fifteenth week, one week before the fourth and last concept application assignment. The timing allowed students to complete the initial CMA prior to all assignments and the second CMA to refresh their memory before the last assignment.
Data Analysis
ANOVA tests determined whether the no-CMA and CMA groups had significant differences in Concept Application scores, participant age, and online learning experience. A homogeneity test determined whether there was a significant difference in the variances of Concept Application scores between the no-CMA and CMA groups. Pearson correlation coefficient tests were used to verify if there were unexpected relationships among observed variables. A chi-square test determined whether the no-CMA and CMA groups had significant differences in gender. An a priori .05 significance level was used for all statistical analyses in this study.
Results
Concept application’s average score was 84.74 points out of the potential maximum score of 100 points (SD = 10.86, range 49-100 points). The CMA group achieved an average of 90.93 points (SD = 5.55, range 70-100 points) whereas the no-CMA group achieved an average 80.40 points (SD = 11.58, range 49-97 points). Figure 1 illustrates the ranges of the concept application scores between the CMA and the no-CMA groups.
ANOVA tests indicated that the CMA group significantly outperformed the no-CMA group in Concept Application scores (F = 28.42, p = .00). A homogeneity test on concept application scores indicated a significant difference between the two groups’ variances of concept application scores (Levene = 26.14, p = .00). Furthermore, concept application scores were not significantly correlated to participant online learning experience (r = -.10, p = .33) or age (r = -.00, _p = .99). Pearson correlation tests indicated a significant correlation between participant online learning experience and age (r = .22, p = .03). However, ANOVA tests found no significant differences between the CMA and the no-CMA groups in online learning experience (F = .10, p = .76) and age (F = .28, p = .61). A chi-square test also found no significant difference in gender between the CMA and no-CMA groups (df = 1, p = .18).
Discussion and Conclusion
The CMA group significantly outperformed the no-CMA group in concept application, which is consistent with previous findings that CMAs effectively enhance concept learning (Parkes et al., 2000; Santhanam et al., 1988). In addition, a significant difference between the two groups’ variances of concept application scores was found. The minimum score of concept application of the CMA group (70) was considerably higher than that of the no-CMA group (49), which signified that low achievers learned and benefited remarkably from participating in CMA. this finding indicated that the low achievers benefited the most by participating in CMA. These findings indicated that the CMA used in this study substantially not only increased the average of learning outcome but also narrowed the learning outcome differences among students. These findings suggest that the specific features of the CMA effectively engaged learners in substantial cognitive information processing for deep learning. An implication for instructional design to be drawn from these findings is that learning activities to be effective for less-prepared learners may incorporate the specific features of the CMA used in this study, such as learning from peer, providing application examples, and using a limited set of concepts.
These findings further suggest that CMAs could be designed to be an efficient online individual activity that requires less time to complete, although CMAs were often used as collaborative activities in computer-based learning environments (Chang et al., 2003; DeSimone et al., 2001). In this study, the CMA was designed as an individual activity that eliminates the additional time needed for collaborative coordination, communication, and negotiation. As an individual activity, the CMA has advantages for less-experienced online students, most of whom may still be learning how to telecommunicate effectively with their peers and instructor in online learning environments. Thus, students can focus on developing their cognitive information process (CIP) to construct their own concept maps, instead of spending time in negotiating a group work with peer students. Furthermore, the advantages associated with collaborative learning were largely retained by the design of the peer review process in the CMA. For example, the process of posting individual concept maps and viewing others can achieve collaborative learning but engage in little or no group meetings.
The findings of this study suggest several future studies. Since using CMAs required extra learning time of students and the higher achievement of the CMA group might be attributed to the extra learning time instead of participating in the CMA, future studies may control the learning-time effect associated with the CMAs. For example, a research design may compare the effects of two learning activities, one with the CMA and another one with other activity, that take about the same amount of time from students. Future research may examine an individual or the combination of design features to identify optimal CMA features in different learning situations. The research in online concept mapping may be expanded to include the effects of concept complexity, the role of peer learning, and the impacts in various content domains.
These results must be interpreted with caution given three limitations of this study: non-random group assignment, potential grading bias, and extra learning time attribution. First, intact-classes, instead of random individual students, were assigned to one of the two groups in this study. Although non-random group assignment may lead to biased results (Krathwohl, 1997), because that the students in the two groups showed no significant statistical differences in age, gender, and online learning experience, the potential bias caused by distinct differences between groups should have been minimized to a great extent and most likely would not mislead the experimental results in this study. In addition, to randomly assign individual students in the same class to different groups, some of which would receive the treatment and some of which would receive no treatment, may violate the regulation of fair instruction to all students in the same class and may not be a feasible method. Second, the limitation concerns the potential grading bias resulting from the rater, or the instructor-researcher, already knowing which students received treatment or not. To remedy the potential grading bias, the instructor-researcher followed a set of grading criteria with detailed rubrics (Appendix B) as carefully and fairly as possible while scoring concept application assignments, the dependant variable. Thus, the potential grading bias should be largely reduced. Lastly, the CMA group spent additional learning time in CMA, while the no-CMA group did not. The higher learning achievement of the CMA group might be due to the additional learning time, rather than the effectiveness of the CMA. The authors were fully aware of this limitation and suggested further research in the previous section to investigate the relationships among CMAs, additional learning time, and learning achievement. Accordingly, generalizations of the findings from this study should be applied with caution.
In conclusion, this study tested the effectiveness of an online concept mapping activity (CMA) featuring peer learning and concept application examples with a limited set of specified concepts. Students participated in the online CMA achieved significantly higher scores and greater fidelity in concept application than those who did not. Findings of this study lend empirical support to the effectiveness of online CMAs and concur with Scott and Baker (2003) that CMAs should be incorporated into most conceptual learning events. The findings of these study suggest that, first, CMAs with specific features can facilitate effective cognitive information processing and enhance learning motivation for deep learning, especially for less-prepared learners; second, CMAs with specific features can be used as individual activities if collaborative CMAs are too time-consuming.
Appendix A: Grading Criteria for Concept Application Assignments
Each concept application assignment had 25 points and there were four concept application assignments in the course. The possible score of Concept Application ranged from 0-100 points.
Criteria 1: (10 points)
The instructional strategies were clearly described and appropriate for the given learners to achieve the learning goal (40% out of 25 points for each concept application assignment).
| Point | Rubric |
|---|---|
| 10 | Clear description of 3 or more instructional strategies for achieving the learning goal; All instructional strategies appropriate to the given learners |
| 9 | Clear description of 3 or more instructional strategies for achieving the learning goal; Only 2 instructional strategies appropriate to the given learners |
| 8 | Clear description of 3 or more instructional strategies for achieving the learning goal; Only 1 instructional strategy is considered appropriate to the given learners |
| 8 | Clear description of 2 instructional strategies for achieving the learning goal; All instructional strategies appropriate to the given learners |
| 6 | Clear description of 2 instructional strategies for achieving the learning goal Only 1 instructional strategy appropriate to the given learners |
| 4 | Clear description of only 1 instructional strategy which is considered appropriate to the given learners to achieve the learning goal |
| 2 | Clear description of only 1 instructional strategy for achieving the learning goal; No instructional strategy appropriate to the given learners |
| 0 | No clear description of any instructional strategy for achieving the learning goal; No instructional strategy appropriate to the given learners |
Criteria 2: (10 points)
The instructional strategies were consistent with the concepts of the given learning theory (40% out of 25 points for each Concept Application Assignment).
| Point | Rubric |
|---|---|
| 10 | All 3 or more instructional strategies were consistent with the concepts of the given learning theory |
| 8 | All 3 or more instructional strategies were consistent with the concepts of the given learning theory but with a few lapses |
| 5 | Most instructional strategies were consistent with the concepts of the given learning theory, a few instructional strategies were not |
| 3 | Some instructional strategies were somewhat consistent with the concepts of the given learning theory, others were not clear |
| 1 | Instructional strategies were proposed but not consistent with the concepts |
| 0 | No instructional strategies were proposed |
Criteria 3: (5 points)
The rationale for using the instructional strategy was logical and consistent with theory (20% out of 25 points for each concept application assignment).
Appendix B: One of the Concept Mapping Activities in this Study
Purpose
Demonstrate your knowledge by drawing a concept map about the relationships between concepts and instructional strategies concerning the scenario below
Scenario
Learning goal: Compare and contrast three types of government: democracy, oligarchy, and monarchy
Learning theory: Gagne’s 9 events of instruction
Learners: 5th graders. This is an introductory lessen about types of government to these learners.
Instruction
Draw a concept map with nodes and links. See an example of a concept map on the next page. Nodes must include all concepts and instructional strategies below. Every link must contain a verb indicating the relationships between nodes.
Nodes:
Concepts
Presenting the content
Eliciting performance
Gaining attention
Providing feedback
Assessing performance
Instructional Strategies
The instructor divides the class into three small groups and each group is assigned to one type of government. Using all nonhuman resources (e.g. books in the library and Web pages on the Internet), each group conducts a research to find countries that are currently employing the type of government assigned to the group. Then, each group reports their findings to the class and the instructor provides his/her comments on each group’s report.
The instructor shows a short video clip of people voting for their mayor and asks learners to identify the type of government these people have. Then, the instructor gives a lecture of the election process of democracy government.
The instructor asks learners to write an essay about the strengths and weaknesses of one type of government.
Example of a Concept Map
Nodes: Dogs, loyal, bark, home, legs, tail, other dogs, people, shoes, holes, running (see Figure 1).
