INVESTIGATING THE USE OF ADVANCE ORGANIZERS AS AN INSTRUCTIONAL STRATEGY FOR WEB-BASED DISTANCE EDUCATION Free

The present study investigated short-term and long-term effects of two kinds of advance organizers (AOs) in a fully Web-based course. A concept map was used as a visual AO, and an outline was used as a text AO. Students’ learning achievement in knowledge acquisition and application was tested both immediately and 4 weeks after the experiment. Specifically, this study is designed to explore the effectiveness of AOs in improving students’ learning performance in a fully Web-based course.

The rationale for using AOs is rooted in cognitive learning theories. Cognitive theories state that learning performance depends on processing capacity and prior knowledge (Driscoll, 1999). With the aid of AOs, learners are able to link what they already know to new information and apply it to new contexts.

Ausubel first introduced the concept of advance organizers in his assimilation theory of meaningful learning and retention. Based on Ausubel’s theory and the later studies on AO (Ausubel, 1968; Kenny, 1993; Mayer, 1979; Stone, 1983), a framework has been synthesized to predict the effectiveness of AOs. The theoretical framework includes the following key propositions: students given advance organizers should perform better in tests on the material to be learned than students in control groups; the advance organizer effect should be at least as great in longer studies as in shorter ones; the graphic advance organizers should be at least as effective as the text advance organizers.

Extensive research was conducted on the effectiveness of using AOs in classroom teaching from the 1960s to the 1990s. The research evidence concerning any facilitative effect of AOs upon learning and retention is variable, but positive in general. Although Ausubel’s early experiments supported the effectiveness of AOs with significant increasing learning achievement (Ausubel, 1968), later studies failed to show a consistent positive facilitative effect (Barnes & Clawson, 1975; Luiten, Ames, & Ackerson, 1980; Mayer, 1979; Stone, 1983).

In the 1990s and 2000s, many researchers began to conduct studies on AOs in a variety of formats, such as visual AOs (DaRos & Onwuegbuzie, 1999; Herron, Hanley, & Cole, 1995; Hirumi & Bowers, 1991; Millet, 2000) and multimedia AOs (Calandra, Lang, & Barron, 2002; Hale, 2003; Minchin, 2004; Tseng, Wang, Lin, & Hung, 2002; Yeh & Lehman, 2001). Failing to generate statistically significant results on effectiveness of AOs, most researchers continued to suggest a mild but positive effect of AOs on learning and retention. The statistical nonsignificance of the research might be attributed to imprecise construction of organizers, short duration of treatment, inadequate research control, and insufficient instruction on how to use AOs (Kenny, 1993; Luiten et al., 1980; Mayer, 1979).

Two hypotheses were posited for this study.

Hypothesis I. There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the concept map, outline, and control groups.

Hypothesis II. There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the concept map, outline, and control groups.

The population of this study consisted of 164 undergraduate students enrolled in a fully Web-based, health-related ethics class at the University of Central Florida. The students were largely between the age of 21-23, in either their junior or senior years. The enrolled students were asked to participate in this study on a voluntary basis. Sixty-three of the total 164 students voluntarily participated in the experimental activities.

This study used an experiment-control posttest-only design with random assignment to examine the effects of AOs on learning achievements, as illustrated in Figure 1.

R indicated that all participants were randomly assigned to three groups, two treatment groups (E₁ and E₂) and one control group (C). AOs were the intervention in this experimental design. The experimental group (E₁) reviewed a concept map, a form of graphic AO, before reading the textbook. The comparison group (E₂) reviewed a text AO, and the control group (C) did not read any AO before textbook reading. During the course of the study, all three groups completed an immediate posttest (O₁, O₃, and O₅) and a delayed posttest (O₂, O₄, and O₆).

One of the dependent variables in this study is students’ learning achievement, encompassing their short-term (O₁, O₃, and O₅) and longterm knowledge acquisition and application learning achievements (O₂, O₄, and O₆). The short-term and long-term knowledge acquisition was tested with two corresponding 12-item knowledge quizzes. The short-term and long-term knowledge application was tested with problem-based scenario essay questions.

The independent variable is the treatment of AOs (X₁ and X₂). The three groups had the same instruction, except for the treatment of AOs. The experimental group was intervened with a concept map (X₁); the comparison group was intervened with a textual outline (X₂); and the control group had no AO exposure before textbook reading.

Time is a confounding variable for the research which has been used to distinguish the short-term and long-term impacts of AOs. It is assumed that the time factor might influence students’ learning achievement over a period of 4 weeks’ time.

Two forms of AOs were designed respectively for the experimental and comparison groups. The construction of the AOs followed a series of research-based procedures (Bricker, 1989; Mayer, 1979; West, Farmer, & Wolff, 1991). Students were instructed to review the AOs before they read the textbook. The graphic AO is a flash-based interactive concept map. The text AO presents the same concepts and explanation as the concept map. Both AOs are linked to the instruction page of module 2. The only difference between the two organizers is the presentation of the relationship among the concepts. The concept map illustrates the relationship visually in a nonlinear way, and the textual outline presents it textually in a linear way. The validity of these AOs was tested and confirmed by expert reviews from both the instructor and the outside instructional designers, and modifications were made based on their suggestions.

This study utilized three major instruments: posttest I, posttest II, and a student survey. Posttest I and II are parallel in content and format, with a 12-question, multiple-choice quiz examining concept acquisition and three open-ended questions based on a scenario, testing knowledge application. Posttest I measured students’ short-term learning achievement, and posttest II measured their long-term learning achievement. The student survey contained 18 multiple-choice questions on students’ prior online learning experience, study environment, use of concept maps, use of quiz, and demographic information. One additional open-ended question collected further comments from student participants.

This study lasted for 6 weeks. During the first week of the 2006 Spring semester, participants were randomly assigned to three groups. Each group was provided with one version of module 2 during week two. In the course module, the students were suggested to first review the AOs to gain an overall idea of the key concepts and issues covered in this module, if they had one available in their group. The students in the experimental group reviewed the multimedia concept map before reading the book. The students in the comparison group reviewed the text outline before reading the book. The students in the control group were not given an AO, and they proceeded directly to textbook reading.

After textbook reading, the students were instructed to complete all the assignments on the assignment page, including the two parts of the posttest I. The knowledge quiz of posttest I was a timed WebCT quiz. The students had 15 minutes to complete 12 multiple-choice questions and they could access and submit the quiz only once. However, as this was a fully Web-based course, the quiz was not proctored and students had the flexibility to do the quiz at their convenience during the instruction week. For the second part of posttest II, the students completed three questions based on a scenario using Microsoft Word and submitted the assignment to the WebCT Assignment tool by the next Monday morning. Also during this week, students filled out an online survey to report their background information and their uses of AOs.

Four weeks after module 2, in week six, posttest II was administered through WebCT. Together with all the other assignments for module 6, posttest II, including a quiz and three scenario questions, was open for the students. It was stated in module 6 that both the quiz and scenario questions of posttest II were part of the voluntary research. The students completed this posttest with the knowledge they had learned in module 2.

Statistic procedures, including descriptive analysis, one-way analysis of variance (ANOVA), and repeated-measure regression were performed to study the research findings. Descriptive analysis was used for scores in the posttests. Means, standard deviations, and effect sizes of students’ learning achievement scores were computed for each quiz and scenario questions of posttest I and II. The assumptions of the analysis, including the homogeneity of variance and the normality of population distributions, were examined using the Levene’s test and the Q-Q plot procedures. Descriptive analysis was also used to calculate the frequencies which describe how students had used and liked the advance organizers.

To test the two hypotheses, ANOVA was used to compare the mean scores of the posttests among the three groups. In combination with the AO effects, this study examined other factors that might influence students’ learning achievement using repeated-measure analysis of covariance (ANCOVA). These factors include students’ weekly study time, study place, gender, academic status, ethnicity, grade point average, and so forth.

In posttest I, students of group 1 using a concept map had the highest mean score (36.25) in the knowledge quiz 1, compared with those of the other two groups. In the performance-based scenario questions, there was little difference in the mean scores of the three groups. Table 1 illustrates the detailed means and standard deviations of students’ learning outcomes in posttest I.

Hypothesis I suggests that students who were exposed to a concept map AO or an outline AO would show no difference, in both the short-term knowledge-based and performance-based learning achievements, from those who were not exposed to an AO. Table 2 shows that there is no statistically significant difference among the three groups in either the knowledge quiz (F_{2, 122}= 1.130, α > 0.05) or the performance scenario questions (F_{2, 137} = 0.412, α > 0.05). Also, the effect sizes for AOs in both tests are relatively low. Only 1.8% of the differences in quiz 1 scores can be explained by the treatments of AO among the groups. Less than 1% of the difference in scenario 1 scores can be explained by the use of AO.

In posttest II, there are little variations in the mean scores in either quiz 2 or scenario 2 questions. The control group outscored the AO treatment groups by less than 1 point in both tests. Table 3 shows the means and standard deviations of the students’ learning outcomes in the delayed posttest.

Similar to the findings in the short-term learning achievement posttest I, the difference in posttest II is not statistically significant in either the knowledge-based quiz (F_{2, 95} = 0.412, α > 0.05) or the performance-based scenario questions (F_{2, 60} = 1.051, α > 0.05). The effect size of AO in quiz 2 is below 1%. The effect size of AO in scenario questions 2 is 3.4%, far below 20%, which is indicative of a small effect by the Cohen’s convention.

The research findings show that there is no difference in either the short-term or long-term learning achievements among students in the concept map, outline, and control groups. According to Ausubel’s assimilation theory, students given AOs should perform better on tests on the material to be learned than students in control groups (Ausubel, 1968). In the current study, students in the concept-map group outscored the other groups by 3-4 points out of a total 60 points on average in the immediate knowledge quiz (posttest I). The difference in performance-based scenario questions was small among the three groups. The full score for the scenario questions was 25, and the mean scores for all of the groups were around 22.5, indicating a ceiling effect that the assessment instrument may lack sensitivity and discrimination in measuring learning outcomes. The control group scored at an average of 22.28 out of 25 on the scenario questions. There was less than 3 points (12%) of improvements for the treatment groups to achieve. Similar to the historical studies on AOs in face-to-face classes, no statistically significant difference was found in either the knowledge-based or the performance-based tests in this Web-based AO study.

It is speculated that the AO effect might be more observable in longer period of study time, especially in the ones over 10 days, than in shorter ones (Ausubel, 1968; Luiten et al., 1980; Stone, 1983). However, this study failed to prove a greater AO effect on students’ learning achievements in a delayed posttest (post-test II) 4 weeks after the AO intervention. The differences in both the knowledge-based and the performance-based tests were trivial, and the effect sizes were considered to be small and below 0.05 by the Cohen’s convention. Similar to posttest I, a lack of differentiation might be one of the reasons that attribute to the nonsignificant result in posttest II. Also, in the posttest, students reported a shortage of time during the quiz, and over 50% of the students made errors in the last two quiz questions. Therefore, speeding effect, where students did not have sufficient time to answer all questions, might be another factor that seriously affected the measuring error of the quiz instrument.

Most of the students found using AOs, especially the concept map, helped them scaffold the learning materials. Their feedbacks in the survey indicated how they used AOs in learning. The majority of the students would read AOs before they read the textbook. They spent, on average, 6-10 minutes reading the concept map, and would usually refer back to the concept map during or after they read the textbook. For the text outline, the student would spend 1-5 minutes reading, and read it only once. According to the survey results, this study demonstrated how AOs could be integrated in Web-based distance learning, and the concept map was better received by the students compared with the text outline.

This study is an attempt to validate Ausubel’s AO theory in fully Web-based learning environment. However, the findings did not support Ausubel’s proposition that students given AOs performed better in tests than students in the control group. Moreover, the AO effect in long-term tests of this study was not as great as in shorter ones, as Ausubel had predicted. On the other hand, the graphic AO was as effective as the text AO in this study, which evidenced one of Ausubel’s propositions. Specifically, students using a concept map (graphic AO) consistently achieved higher scores than those using a text outline (text AO). Students and the instructor also preferred the concept map to the outline, despite that the contents for both AOs were identical. The visual elements and interactivity of the concept map were favored by students in Web-based learning, and the outline was regarded as static and linear.

The practical significance of this study lies in that it has updated and improved the AO conceptual framework to fit the new Web-based learning environment. The original Ausubel’s model was first developed for the face-to-face classroom setting where the blackboard is the main teaching medium. The framework had been constantly modified by later researchers to further investigate the methods for constructing and applying an AO in a computer-based instruction environment in the late 1980s and early 1990s. In the new century, school learning is enhanced and optimized with the explosive development of emerging Internet technologies and diversified digital media. However, the research on AOs in fully Web-based learning is very limited. The current study expands the AO framework to a fully Web-based environment. The use of advance organizers is a good teaching and learning practice in the context of self-paced online learning. Students will benefit from using AOs not only in a traditional classroom, but also in the ever-growing Web-based learning environment.

In retrospect, this study might have been improved. The statistical insignificance of students’ performance between AO groups and the control group might be due to several reasons. One of the issues that the researcher found is that an online quiz is difficult to monitor. Though the quizzes had been designed as closed-book tests and test questions were randomized, it was possible that students still referred to their lecture notes and the textbook while they took the quizzes. This might seriously threaten the validity of the test instruments. An important implication for further research is to develop measures to prevent students from referring to other assistant materials. Another reason for the nonsignificant result might be the lack of measurement of students’ analytical and critical thinking abilities. Future studies should develop stricter rubrics to differentiate students’ learning outcomes in the assessment instruments. Also, the limited intervention time might be a factor that negatively influenced the effectiveness of AO. The current AO intervention was one week. For future studies, longer intervention time is highly recommended.

The participatory AO (student-generated AO) is the new direction for future studies on AOs in Web-based learning. Students might interact with the material to be learned more in depth with the help of participatory AOs, thus making the materials easier for them to comprehend and use. Recently, new instructional concept mapping tools have become available for instructors and students to create digital organizers in computer-assisted instruction and online education. For example, the Visual Understanding Environment (VUE) and the C-Map are two free information management applications that provide an interactive concept mapping interface. Future Web-based AO research studies can take advantage of these free AO tools, focus on helping students generate their own advance organizers, and measure the effectiveness of participatory AOs in both face-to-face and Web-based educational settings.