Effectiveness of Simulation in a Hybrid and Online Networking Course

Computer networking, defined as the interconnection of computers and computing equipment using either wires or radio waves over small or large geographic areas (White, 2002), has long been regarded as one of the more difficult technology-related subjects to teach. Historically, this type of course was thought to require much hands-on interaction with the instructor and was not viewed as a good candidate for an online course. Due to recent advances in network simulation technologies, this view is being challenged. New simulations allow complex networking systems to be easily modeled on desktop computers. The effectiveness of simulations and the challenges of distance learning environments are described next, followed by the research design, methods, and results.

Simulations have been an effective tool in learning environments (Lindström, et al., 1993; Williamson & Abraham, 1995). Simulation research can be divided into three subsets: cognitive studies, attitudinal studies, and retention studies (Dekkers & Donnatti, 1981). Cognitive studies are concerned with the simulation's effect on the cognitive development of the student. Attitudinal simulation studies are concerned with student attitude formation, and retention studies are concerned with information retention. This project touched on all three of these areas of simulation research. Comparisons were made of the cognitive development of the two course sections through different networking labs and team problems. Attitudinal development was accessed through a course survey given at the conclusion of the course. Retention was accessed through tests given at specified points in the course.

While computers afford the design of highly interactive open-ended learning environments such as simulations, decisions about how to design the instruction used in conjunction with the simulation are often made with little understanding of how the user will perceive, process, and interpret the resulting feedback that the simulation provides. Tailoring the simulation to the needs and goals of a particular course requires an understanding of the capabilities of the simulation and the ways it introduces and enforces various networking concepts. Research demonstrates that the way information is represented matters greatly in the learning process (Rieber, 1996).

However, these studies of the effectiveness of simulations in learning environments compared simulations with traditional classroom instruction. Web-based distance education courses offer a unique challenge to teaching traditionally hands-on courses like computer networking.

Lack of motivation and low student satisfaction have been cited as factors relating to the high drop-out rate in online education. There are a number of studies indicating that simulations can create considerable motivation in their participants (Brawer, 1982). Until recently, only simple simulations, with a low level of complexity, were possible in an online learning environment. The ability of this new online delivery mode for simulation technology has great promise for many types of online courses. A key factor cited in improving student motivation when using simulations in instruction is the relevancy or realism of the simulated environment (Orbach, 1979). Advances in commercially available network simulations have produced products that emulate real-world network operations.

The purpose of this research was to compare students’ performance in simulation-based courses and static graphic representational teaching of the same course content in an online learning environment.

Eighty-five freshman and sophomores enrolled in two sections of a new online introductory networking and telecommunications course were compared. One section utilized a network simulation package, created by NetCracker Technologies, Inc., that allows the student to build and send data through different network configurations. The second section used Microsoft Visio, a static network diagramming software package. The two groups were compared and assessed on a variety of levels.

NetCracker Technologies offers a simulation package that allows students to design local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs) utilizing a wide variety of networking components. A wide variety of data types may be used to test the functionality of the network design.

Static network diagramming software packages like Microsoft Visio have been used in networking courses for decades to teach networking concepts and develop visual representations of networks.

An undergraduate-level course, Introduction to Networking and Telecommunications, is taught at a national university. The course is comprised of online instructional materials and assignments. Most interactions with the instructor were online. The only face-to-face instructor interactions were in the form of team project meetings and team presentations.

One course section consisting of 44 students was asked to use the network simulation package for all course assignments, and the second course section, consisting of 40 students, was asked to use Microsoft Visio for all course assignments. Basic instruction was given on both packages.

Assessment of learning outcomes was conducted using multiple choice tests, project results, and a survey of students. The students completed four individual networking labs and two team problems using their respective software packages. The labs required students to demonstrate knowledge of networking rules and concepts by devising networking solutions to given scenarios. The team projects required students to synthesize factual, procedural, and conceptual knowledge about computer networks in order to solve a complex problem. Mid-term and final exams, consisting of 50 multiple-choice questions, were given to both course sections. The exams focused primarily on factual knowledge. A qualitative course survey was administered to both course sections at the end of the course.

Standard statistical analysis procedures were used to compare differences among the two conditions. Results from ANOVA showed the network simulation condition outperformed their counterparts in the static simulation condition. The mean on the test performance was significantly (p < .05) lower for the static simulation condition.

Table 1 shows the descriptive statistics for each measure used in the analysis. The course section that used that network simulation software demonstrated better understanding of networking concepts and better retention of the course information than did the course section the did not use the simulation. Scores on the individual lab assignments were all significantly (p = .000) higher for students who were in the network simulation group. Team projects scores were significantly (p = .001) higher for the simulation group. Mid-term and final exam scores were also significantly (p = .008 for final and p = .017 for mid-term) higher for the students who used the simulation.

No students in the course section that used the simulation withdrew from the course due to failing grades, while three students in the course section that did not use the simulation withdrew from the course with failing grades.

Students who used the simulation reported that they spent more time on course assignments than did the students who used Microsoft Visio (3.5 hours versus. 2 hours) and reported that they spent more time on assignments primarily because the simulation allowed them to experiment with different network configurations and verify the functionality of their network designs. Many students reported that they felt that the simulation helped them to understand complex networking concepts because they could actually build and test the different components of a network and try different network configurations. Many of the students who used Microsoft Visio reported that they had no way to verify that their network designs functioned correctly prior to submitting them to the instructor and, as a result, did not experiment with different network configurations.

Online learning environments have been plagued with student motivational problems, leading to high drop-out rates. This project suggests that the use of interactive learning tools, such as simulations, have the potential to increase student motivation and learning in an online learning environment. Simulations offer immediate feedback and allow the learner to explore different alternatives at will. Simulations enable knowledge application through multidimensional problem solving (de Mesquita, 1992). Simulations have been found to significantly improve knowledge transfer (Kozma, 1992, White, 1994). This type of discovery-based learning using simulations has been shown to increase understanding of abstract concepts (Rieber, 1996) and increase student motivation (Brawer, 1982).

The network simulation utilized in this study can be used for a variety of learning objectives targeted at differing levels of learning. The intellectual skills that contribute to challenging classroom learning, as identified by Gagne (1977, 1985), are discrimination, concept learning, and rule utilization. Simulation affords the instructor the opportunity to develop instructional events for all of these levels of learning.

For example, at the factual level, the simulation can be used to identify the parts of a networking system. The package can also be used as a type of procedural simulation, described by Reigeluth and Schwartz (1989), and teach network diagramming and documentation procedures. At the conceptual and rule levels, the deep structures made possible by the diagnostic capabilities of the simulation allow for evolving problems that require sequential and interrelated decisions. Therefore, as in real situations, errors may be compounded on top of errors as nonproductive diagnostic and solution procedures are pursued (Berven & Scofield, 1980).

While the use of simulation in education has been studied for decades, its use in an online learning environment has not been widely explored. Most simulation research compares simulations to regular classroom instruction. The instructional goals for which each can be most effective differ and more research needs to be compiled on the use of simulation as a supplement to traditional and online instruction. Traditional face-to-face instruction and online instruction can be effective at transmitting items of information, while simulations have the potential to develop students’ mental models of complex systems and problem-solving strategies. Further research on the effectiveness of differing combinations of these instructional methods is needed.

Another area of simulation research that requires additional study is the use of feedback. Schimmel (1983) identified three types of feedback; confirmation, corrective, and explanatory. The simulation used in this study provided only confirmation feedback. Research on the information content of feedback has been inconsistent and has provided little guidance for designing feedback. More research is needed on the effectiveness of different types of feedback in simulation and in online learning environments.