Effects of Interactivity on Student Achievement and Motivation in Distance Education Free

This study investigated the effects of three levels of interactivity on student achievement and motivation of undergraduate college students taking a Web-based course. The proliferation of distance education has brought concerns regarding student achievement and motivation. Of specific concern has been the level of interactivity offered by Web-based instruction. Interactivity in Web-based instructional environments is considered to play a significant role in student learning. Although there has been a significant amount of research in distance education, few studies have looked at the effectiveness of instructional methods for Web-based instruction. Simonson (1995) argued that educators must strive “to make the experience of the distance learner as complete, satisfying, and acceptable as that of the local learner” (p. 12). A main goal in education should be the development of learners who will find learning to be an enjoyable experience and who will be motivated to learn.

Although interactivity is often referred to as a significant component for successful online learning, empirical evidence of its importance is lacking. Several studies have examined the effects of interactivity on motivation in distance learning environments and have found it to have a positive effect (Huang, Diefes-Dux, Imbrie, Daku, & Kallimani, 2004; Kennedy, 2004; Su Bonk, Magjua, Liu, & Lee, 2005; Thurmond, 2003; Visser, Plomp, Amirault, & Kuiper, 2002). Motivating students to learn should be a top priority of instructional designers when developing online learning materials. Hodges (2004) concluded that regardless of the model or theory employed when developing instruction, the main goal is to incorporate strategies that will motivate students to learn. He pointed out that there is no cookbook that provides a recipe for motivating students. Motivation is important in distance education because students should be able to work independently without face-to-face interaction, to be good at time management, and to be responsible for their own learning. This requires a higher degree of motivation to learn than it would in a traditional setting.

This study employed the ARCS model of motivational design developed by Keller (1987a, 1987b, 1987c). ARCS has been described as a model for designing instruction to motivate students. In this model, specific motivational requirements for learners to acquire desired outcomes are emphasized. ARCS is an acronym for the four motivational categories of the model: attention, relevance, confidence, and satisfaction. The attention component includes strategies that stimulate and sustain learners’ curiosity. The relevance component includes strategies that link learners’ needs and motives. The confidence component includes strategies that provide learners with a positive expectation to achieve. The satisfaction component includes strategies that provide intrinsic and extrinsic reinforcements for learners’ efforts (Keller, 1987a, 1987b, 1987c). The ARCS’ model validity and reliability have been tested in numerous studies, including classroom-based and Web-based instruction (Maushak, Lincecum, & Martin, 2000; Small, 1997; Visser & Keller, 1990; Visser, Plomp, & Kuiper, 1999; Visser et al., 2002).

This study employed activities that provided three levels of interactivity based on Keller's ARCS model. It also employed Keller's (1993) Instructional Materials Motivation Survey (IMMS) which was based on his ARCS model of motivation to measure the level of students’ attention, relevance, confidence and satisfaction. Keller suggested that the instructional design of the learning experience be evaluated based on motivational as well as learning outcomes (Keller, 1987a, 1987b, 1987c). Keller further postulated that the four motivational categories must be met for a learner to be motivated to learn and suggested that the ARCS model conditions are a sequential process leading to a successful learning experience (Keller, 1987a, 1987b, 1987c). Therefore, in this study student outcomes, in addition to motivation, were also measured.

The main hypothesis for this study was that the higher the level of interactivity the higher the level of motivation and increased students’ outcomes. To improve the level of interactivity, this study made use of generative activities and also activities that provided elaborate immediate feedback as an interactive tool to motivate students. A reactive interaction group received a medium level of interactivity which included elaborate immediate feedback, and a proactive interaction group received a high level of interactivity which incorporated a generative activity. There are many applications of generative learning that can be used in instruction of online courses. Sharp, Knowlton, and Weiss (2005) described several categories of generative learning strategies and discussed their merits. Studies have also supported feedback as an interactive process that serves as a vehicle for increasing learning and satisfaction in Web-based courses (Atack & Rankin, 2002; Restauri, King, & Nelson, 2001; Schoenfeld-Tacher, McConnell, S., & Graham, 2001; Soon, Sook, Jung, & Im, 2000; Thurmond, Wambach, Connors, & Frey, 2002; Volery, 2001; Vrasidas & McIsaac, 1999).

Interactivity is one instructional method that has been linked to increased levels of motivation leading to students’ positive outcomes, but empirical evidence of its importance is lacking. Therefore this study focused on looking at the effects of different levels of interactivity on students’ motivation and success in Web-based courses. Because the main goal in education should be to motivate students to learn, interactivity should be a topic of further research. It is important for instructors to incorporate interactivity into Web-based courses to motivate students to learn in a distance learning environment.

This study was based on a study conducted by Gao and Lehman (2003), and employed the ARCS model of motivational design and the Instructional Materials Motivation Survey (IMMS) developed by Keller (1987a, 1993). Data were collected and analyzed from students enrolled in one of four undergraduate web-based health administration courses at a major Southern university during a regular semester. Student volunteers were assigned to one of three groups: a low-level interaction group, a reactive interaction group, and a proactive interaction group. Each group was administered a different treatment that included a low, medium, and high level of interactivity. The lesson's objectives and reading material for all three groups was identical; only the activities differ based on the level of interactivity. The low interaction group received the lowest level of interactivity, consisting of static reading material and hyperlinks with relevant questions to answer. The reactive group received a treatment consisting of a medium level of interactivity. This activity consisted of having students answer questions related to the reading material assigned. When students typed their responses, they were provided with immediate elaborate feedback that allowed them to also evaluate their responses. The proactive group received a treatment consisting of the highest level of interactivity. This treatment incorporated a generative activity that asked students to generate their own scenario based on the knowledge they acquired from the reading material.

Prior to completing the assigned activity, participants were asked to complete a pretest to determine their level of knowledge on the subject. The purpose of administering this pretest was to eliminate any volunteers who had a significantly higher level of knowledge of the lesson's topic. Participants took a test immediately after completing the experimental learning unit to measure their level of achievement; and upon completion of the course, participants took a delayed posttest, to determine their level knowledge retention. Participants also completed the IMMS survey to determine their level of motivation. The IMMS had been converted into a Web-based format to allow students to complete it online. Each question of the IMMS has a choice of five answers: A = not true, B = slightly true, C = moderately true, D = mostly true, and E = very true. Participants responded based on a Likert-type scale and 1 to 5 points were assigned for each answer. The IMMS survey consists of a total of 36 questions, the minimum score is 36 and the maximum score is 180. The individual questions fit into one of four categories: attention, relevance, confidence and satisfaction. The reliability estimates of the IMMS survey based on Cronbach's alpha for each subcategory of the survey were previously determined to be: attention .89, relevance .81, confidence .90, satisfaction .92 and the total scale .96 (Keller, 1993).

This study examined the effects on achievement, knowledge retention, and motivation of various levels of interactivity in Web-based instruction. The results of the instruments employed were evaluated to measure the effects of the treatments on students’ motivation and performance and related variables, which included the level of knowledge retention and overall satisfaction with the learning experience.

A summary of the results of the three tests administered is provided below in Tables 1-4. The pretest was scored on a 5-point scale, with 1 being the lowest level of knowledge and 5 being the highest. The ANOVA was conducted with the factor being level of interactivity and the dependent variable being the student score on a test. The results for the ANOVA indicated no statistically significant differences among the mean scores for each level of interactivity, F_{1, 113} = 1.515 and p = 0.224 (Table 1). These results showed that there were no statistically significant differences in the pretest among students in all groups, which qualified them to participate in the study.

The test that was administered immediately after completion of the experimental lesson was scored on a 10-point scale with 1 being the lowest and 10 being the highest level of knowledge. A Kruskal-Wallis test was conducted, with the factor being level of interactivity and the dependent variable being the student score on the test. This test was conducted to evaluate differences in median scores among the three interactivity levels: low, medium, and high (Table 2). The test was statistically significant (χ_{2, 124} = 10.080, p = 0.006). These results supported the hypothesis that different levels of interactivity had an effect on participants’ level of achievement. Table 3 shows the results of a Mann-Whitney test. For the first comparison of low- and medium-interactivity groups the results were statistically significant (z = −2.409, p = 0.008). A comparison between the low-and high-interactivity groups was also statistically significant (z = −2.922, p = 0.002). These results support the hypothesis that the increase in interactivity had an effect on students’ outcomes on the test.

The delayed posttest administered at the end of the course was scored on a 5-point scale with 1 being the lowest level of knowledge and 5 the highest. A Kruskal-Wallis test was conducted with the factor being level of interactivity and the dependent variable being the student score on a delayed test. The results are shown on Table 2 and were statistically significant (χ_{2, 101} = 12.1900, p = 0.002), indicating differences in median scores on the delayed posttest for all levels of interactivity.

To determine where the differences were in the delayed posttest results, follow-up tests (Mann-Whitney tests) were conducted to evaluate pair-wise differences among the three levels of interactivity. A Mann-Whitney test for the first comparison of the low- and medium-interactivity groups was performed to evaluate the hypothesis that the students in the lowinteractivity group would score lower on average on a delayed test than the students in medium-interactivity group. These results are shown on Table 3 and were statistically significant (z = −3.431, p = 0.001).

The results of the IMMS survey are shown on Table 5 and were based on a 5-point Likert-type scale with 1 being the lowest and 5 the highest. A Kruskal-Wallis test was conducted with the factor being level of interactivity and the dependent variable being the student score for each subscale category.

The attention survey subcategory results were not statistically significant (χ²_{2, 1082} = 3.807, p = 0.149), indicating no differences in median scores for the different levels of interactivity groups. Therefore, interactivity had an effect on students’ level of attention to the learning material.

The confidence survey subcategory the results did not initially appear to be statistically significant (χ²_{2, 810} = 5.263, p = 0.072). However, when follow-up tests were conducted to evaluate pair-wise differences mong the three groups controlling for Type I error across tests using the least significant difference procedure, the results indicated a significant difference in median scores between the low- and high-level interactivity groups (U = 32512.500, p = 0.0105). These results indicated that students that received the high-level interactivity treatments had a high level of confidence with the learning materials.

The relevance survey subcategory results were not initially statistically significant (χ²_{2, 947} = 4.739, p = 0.094). However, when follow-up tests were conducted to evaluate pair-wise differences among the three groups controlling for Type I error across tests using the least significant difference procedure, the results indicated a significant difference in the median scores between the low- and medium-level interactivity groups (U = 42312.500, p = 0.016). Similar results were found in the confidence subcategory and strengthen the hypothesis that higher levels of interactivity lead to higher levels of motivation.

The satisfaction survey subcategory the results were not statistically significant (χ_{2, 656} = 7.561, p = 0.072). However, when follow-up tests were conducted to evaluate pair-wise differences among the three groups controlling for Type I error across tests using the least significant difference procedure, the results indicated a significant difference in median scores between the low- and medium-level interactivity groups (U = 42312.500, p = 0.016). This result supports the original hypothesis that increased levels of interactivity will have a positive influence on students’ level of satisfaction with the learning materials.

To summarize the results, the test and delayed posttest results indicated that participants who received the treatments with high levels of interactivity outperformed and had higher retention levels than those in the group that received the treatment with the low level of interactivity. The IMMS survey results showed that participants were generally more motivated with the higher levels of interactivity. Participants with the proactive level of interactivity showed a significantly higher level of confidence, relevance, and satisfaction. However there were no statistically significant differences for the attention subcategory of the survey. Reactive-type activities with a medium level of interactivity had a positive effect on students’ outcomes and retention. They also had an effect on how relevant students felt the material was. Proactive-type activities also had a positive effect on students’ outcomes but had no significant effect on the level of students’ retention. The proactive-type activities had a positive effect on the students’ level of confidence and satisfaction leading to the conclusion that the higher the level of interactivity the more confident and satisfied students were with the learning material.

Although the literature makes reference to the importance of motivation on improving outcomes, there is not much empirical research that has explored this area in Web-based courses. The results of this study indicated that instructional strategies that incorporate interactivity improve motivation and lead to improved outcomes in Web environments. While activities that incorporated elaborate immediate feedback and generative activities increased motivation and improved outcomes, elaborate immediate feedback was more effective in motivating students. This supports the prior findings by Gao and Lehman (2003). The delayed posttest that was administered indicated that both immediate feedback and generative activities improved knowledge retention.

The study's results lead to the implications that Web-based instructional materials that incorporate immediate feedback and generative activities allow students to increase their interaction with the learning material, and that this interaction improves students’ motivation, performance, and knowledge retention.

This study showed that higher levels of interactivity have a positive effect on students’ motivation and outcomes in a Web environment. Students who received a proactive treatment, which included a higher level of interactivity, had significantly higher scores than students who received a treatment with low-level of interactivity. However, no significant differences in students’ scores were found when the reactive, medium level and proactive levels of interactivity were compared. This is consistent with the findings of Gao and Lehman (2003) suggesting that once a certain level of interactivity is reached, students will be motivated and outcomes will improve.

The results of the posttest that was administered at the end of the course were statistically significant. These results can be interpreted as an indication that interactivity has an effect on knowledge retention. The four IMMS survey subcategories were examined individually. Although the results did not support that interactivity has an effect on students’ level of attention, this could have been due to faults in the design of the treatments when incorporating the recommendations of the ARCS model. The results of the other three subcategories that comprise level of motivation: relevance, confidence, and satisfaction, were statistically significant. This is an important result because motivational theories have suggested that relevance is needed to fulfill personal needs or goals, which in turn enhance effort and performance (Means, Jonassen, & Dwyer, 1997). These results supported the ARCS model and the original hypothesis that increased levels of interactivity will have a positive influence on students’ level of satisfaction with the learning materials.

Activities in the reactive group had a greater effect on increasing the level of relevance of the instruction than the generative activities. Feedback, which was incorporated into the reactive activities, is an important component of interactivity and plays a crucial role in distance education as part of the interactive process (Kenny, 2002). This leads to the conclusion that these activities had a positive effect on students’ retention of knowledge.

Today's Web environments make it possible for instructional materials to be more sophisticated and interactive. The Web also allows for students to obtain immediate or rapid feedback. This interaction can address diverse learning styles and help improve outcomes. Although this study provided useful insights that could help educators develop strategies for effective Web-based instruction, more empirical research is needed in this area. Because the main goal in education should be to motivate students to learn, interactivity should be a topic of further research. As instructors are able to adapt their instructional methods to incorporate interactivity into their Web environments, they should expect a greater level of student motivation and thus be in a better position to meet the needs of students.