Does It Matter?: Analyzing the Results of Three Different Learning Delivery Methods Free

Profiles of college students can be very diverse. Some students are the traditional 18-year-old first time incoming freshmen. Families support some students entirely, while others are on their own and have to work various part-time jobs or hold a full-time position to pay bills. There are also others who are married. Some may have young dependents. There are still others who come back to college after having been in the work force. With all the different types of learners, contemporary college instructors are faced with using various methods, both inside and outside the classroom, to impart knowledge and help students learn and master subject matter.

At the same time, the impact of a new emphasis on instructional design and technology has generated great interest in applying new or different ways of delivering information and instruction. Education in the traditional classroom has been enhanced and supplanted by the use of instructional television (ITV) and various other forms of distance learning delivered by computer and the Internet (online instruction).

“Instructional designs serve as mediators between the realms of learning theory and instructional practice, providing a means of developing interventions through which changes in learned capabilities can occur” (Wagner, 1994, p. 20). Instructional designers must consider learner-centered and selfdirected approaches. Learner-centered instruction considers a myriad of characteristics, processes, interactions, and delivery methods that result in effective teaching and learning. For example, asynchronous delivery strategies allow learners to complete work in their own time and location rather than being in the classroom at a specified time.

Related to learner-centered instructional design is the notion of self-directed learning. “As a person matures, his or her self-concept moves from that of a dependent personality toward one of a self-directing human being” (Merriam & Caffarella, 1999, p. 272). Adults prefer self-directed or self-designed activities, more than one medium for learning, and to control the learning pace (Zemke & Zemke, 1984). Self-directed learning does not mean isolation but, in fact, involves several resources, professionals, lectures, seminars, and face-to-face interactions. According to Gerald Grow (1991), adult learners progress from dependency to self-direction. “Some features of self-direction are distinctly situational: Few learners are equally motivated toward all subjects. Some features appear to be deep, familial, perhaps even genetic traits of individual personalities—such as persistence” (p. 128).

Instructional designers have introduced new ways in which instructors can work with students. There has been a paradigm shift from the mode where a teacher delivers information to a mode where responsibility for learning and learning activities is shifted to the learners. While these are all happening, the ubiquitous appearance of generally affordable personal computers that are interconnected by the Internet has made possible the dispersion of learning and a paradigm shift to distributed learning and distance learning.

While there is no single best technology for distance delivery of instruction, instructional designers are interested in making the technology as seamless and transparent as possible. Distance education technologies expand options for interaction among faculty and students. Kochery (1997) noted that by creating an environment that advocates peer interactions, social support, and interpersonal communications, cooperative learning models can help attain the sense of a learning community which is frequently lacking in distance education experiences.

Born and Miller (1999) noted that faculty were concerned about the lack of studentteacher interactions in Web delivered courses. Lindner, Kelsey, and Dooley (2002) found that students were happy with their relationships with their professors, stating that interactions were generally adequate with a few exceptions. Other studies, however, have shown that distance education students were dissatisfied with isolation, inaccessible resources and educational materials, registration and technology problems, and amount of time required to complete course requirements (Kelsey, Lindner, & Dooley, 2002).

Skeptics and scholars alike would like to know whether the shifting paradigms and the new delivery tools have had an impact on the outcomes of the educational process, especially in the higher education arena. Learners have their own perceptions. While some find these new delivery tools beneficial, enhancing the learning experience, others do not rank effectiveness as high. Since evaluation in distance education efforts can identify program effects, helping staff and others to find out whether their programs have an impact on learners’ knowledge, skills or attitude (Nickols, 1999), it is prudent to collect data on how these new tools might affect the level of learning.

The increasing diversity, needs, and preferences of learners, coupled with increased access to knowledge via the Internet, prompted the need for comparing various ways that knowledge can be delivered and learned effectively.

This study compared multiple sections of a college course at the junior level, hospitality human resource management, which was offered by the same instructor, to the same student pool, in one semester, with the same text and common examinations, but using three different delivery methods: traditional classroom, instructional television, and Internet. Specifically, the researchers were interested in the following:

1. Are the three different delivery methods equally effective?

2. Will the different methods affect the learners feelings regarding being “part of the class,” communicating and interacting with each other?

3. Will the different methods affect the learners’ access to learning resources?

This is an exploratory study, which was conducted with one class that had three different sections with the same instructor during a semester. This study would be greatly enhanced if a longitudinal study could be performed and/or more data collected either with different instructors from different types of classes, and with a larger number of learners. Students chose the course section (delivery method) themselves rather than by random assignment to a particular delivery method. Future studies would be enhanced by using random assignment.

In the last 10 years or so, distance education has been accepted into mainstream education. More than a third of colleges in the United States are offering degrees and/or courses on the Internet, including universities such as Columbia, Stanford, the London School of Economics, and the 181 year old University of Virginia (Eggen, 2000). Current literature suggests that some 54,000 courses enrolled 1.6 million students in distance education programs in 1998 (National Center for Educational Statistics, NCES, 1999;,Merisotis, 1999), and was expected to exceed 2 million by the year 2002 (Stewart, 2001). Between 1994 and 1998, the proportion of colleges and universities offering online distance education rose from about 11 to 44% (NCES Fast Facts, 2001).

The reason for this trend is simply the demand from a particular segment of the student population: nontraditional students. Is distance education the cure-all for education? No, but it does offer alternatives. Kathleen Burke stated that undoubtedly, distance education, in its various forms, opens opportunities to many, especially to women with children and military personnel who are moved often (Brown, 1999). Distance education offers learners the convenience and flexibility to learn at anytime, anywhere. Knowledge is available 24-7 for those who want it. For those who cannot travel long distances, it also opens the door for knowledge (Owston, 1997).

Some reports have been published with encouraging results that student dropout rates can been decreased using online education (Carnevale, 2000, January), while others (Johnstone & Krauth, 1996) mention no significant learning outcome differences in distance education compared to traditional classroom methods (Russell, 2001; Young, 2000).

Distance education has experienced some growing pains and is not for everyone. The success stories may come from those who are more disciplined and able to follow a schedule on their own. Because learners are not required to be in a class at a certain time to listen to the lecture, the less-disciplined groups may soon fall behind the class progress. Simply learning to use the VCR to tape classes when they are being broadcast or using programs on the Internet to communicate with the professors and fellow classmates can be challenges for the distance learners (Carr, 2000).

Educators are coping with the new delivery techniques to ensure that the quality of education is not being compromised. Numerous articles and textbooks explore such concerns in detail. Solloway and Harris (1999) discuss the importance of creating a learning community online using Lotus Notes threaded discussion group, and document their success in working with students through this discussion community. Carnevale (2000, October) supports the same idea stating that a social bond is pertinent in the success of distance education. Others view this community concept as important but they extend the idea of a physical community in a traditional classroom to a cyber-community, where people do things together without regard to location (Wellman, 1999).

Besides the sense of community, educators need to modify their traditional classroom teaching techniques to the new distance education environment. Abbey (2000) compiled 16 insightful discussions on topics ranging from designing technology enhanced learning environments to cognitive effects of Web page design. Lau (2000) explored methods of evaluation, digital video usage, and some essential conditions to success in distance education. White and Weight (2000) put together an online teaching guide to discuss how to communicate online, facilitate individual and group learning and even adapt the regular syllabus to the distance learning mode. Educators need to ensure that there is a sense of community within the class and provide a means of communication among participants.

A pretest, posttest quasi-experimental design, and a survey instrument were used to compare how traditional classroom, instructional television, and the Internet affect the academic achievement of college students in a hospitality human resource management class, and to assess the learners’ perceptions of these delivery methods.

The model considered the independent variables as: the traditional classroom instruction, instructional television, and the Internet. At the beginning of the semester, each student was administered a common pretest to assess the baseline knowledge in hospitality human resource management. At the end of the semester, the same set of questions was administered and evaluated as part of a common final examination for all sections.

The traditional face-to-face class consisted of students meeting in a classroom. Traditional interactions took place between the learners and the instructor and between the learners themselves. In addition, the face-to-face course included a threaded discussion and PowerPoint slides, which were available online.

The instructional television group met in the television studio. The studio sessions were broadcasted at three other remote sites in the city. Further distribution was made through delayed broadcast on the citywide cable system on an educational channel, where many students viewed the televised class from home or videotaped the classes for later viewing. Students enrolled in the ITV course were required to participate in a threaded discussion that was available online.

Students in the online course were asked to connect to a course-related Web site that provided access to the course content. The online course was developed using the software IntraKal, an online course management and delivery package, adopted by the university at that time. Students enrolled in the online section only met face-to-face for an orientation meeting prior to the beginning of the course, and for a final examination. Like the other two sections, students in the online delivered course also participated in a threaded discussion.

The dependent variable in this research design was the achievement of the learners. Analysis of covariance was used to analyze the results. The pretest was covaried with the outcome variable, or posttest, in order to remove variability or noise.

An assessment of the perceptions of the learners with respect to the quality of interaction was examined through three surveys, one for each respective delivery method. Each survey contained common questions but also included questions that were related to the specific delivery mode. All learners were asked to provide demographic information such as age, gender, classification, and the like. Each was asked to determine the effectiveness of the course and the amount of interaction with the instructor and other learners in the course. They were asked to rate the effectiveness of course-related resources and indicate if they felt they were part of the class. The data were entered into SPSS to run descriptive statistics such as frequency and percentages.

“Classes” were for one full semester with the same content delivered and common assessment measurements administered to the groups in the three different sections based on delivery method. A total of 83 students participated in this study.

Of the 83 participants, 34 were in the traditional class, 31 were in the ITV class, while 18 were in the Internet class. In the traditional classroom where lecture-discussion was the norm, 72% of the group was 23 years of age or younger, while 28% fell between the age group of 24-29. The female to male ratio was 59% to 41%. In the ITV class, the 23 and younger age group was only 56%, with 33% in the 24-29 age group, 6% in the 30-35 group, and another 6% in the 42-47 group. Once again, the female group had the higher percentage in the ITV mode, with 61%. These students could participate in the studio, at remote sites, or in the comfort of their own home. Finally, the online group had half of its learners below age 23, 33% between age 24-29, and 17% in the group of 30-35. The female-male ratio was 58%-42%, again with the female group being the dominant (see Table 1).

The analysis of covariance on the pre- and posttest showed an F ratio of 2.677 and a p value of 0.075, signifying that though there might be differences among the three groups, such difference was not statistically significant (see Table 2). Thus, the delivery method did not contribute to any difference in the learners’ achievement level.

While it is interesting to note that the three methods did not contribute to any difference in learning achievement, the researchers were also interested in measuring the perception of the learners about the efficacy of each of the methods. When the learners of the ITV group were asked about their “comfort” with the instructional television method, 39% of them stated that they were not very comfortable at the beginning, although this number dropped to only 6% at the end. In the online group, 25% of the learners rated themselves “not comfortable” at the beginning, and this remained unchanged. To record their preferences and see if there was any difference in the preference between the groups on the traditional method and distance education methods such as ITV and online, the learners were asked how they would rate the method they chose against the others. Seventy-nine percent of the traditional classroom group reported that they felt their own chosen method was superior to the distance education method. In contrast, only 66% of the ITV group thought that their delivery methods was superior to the traditional classroom method, and only 50% of the online group shared this same view. Thus, it appeared that this group of learners still slightly favored the traditional classroom delivery method (see Table 3).

Questions were asked of the learners regarding their feeling of being part of a “group” in their respective delivery environments. When asked if they “felt as part of a class,” 76% of the traditional classroom group agreed or strongly agreed with such statements, while only 44% of the ITV class and 34% of the online class shared the same perspective.

When asked about the ability to communicate easily with other learners, 66% of the traditional classroom group agreed or strongly agreed that this was true. The other two groups both reported at a level of about 50%. Thus, while half of the participants in both distance education methods felt that they could communicate easily with others, the other half did not.

Besides communicating, the learners were also asked about general interaction with classmates. Not surprisingly, the traditional classroom group posed the highest satisfaction level with 80% reporting a good level of interaction, and only 44% and 33% reporting positive perceptions for the ITV and online group, respectively.

The researchers then asked about the learners’ experience in sharing thoughts and opinions through classroom discussion or postings on the Internet. Since all three groups had the chance to use the Internet to post discussions, the response to this question was quite even, with all three groups reporting “agree or strongly agree” in the range of 45% to 51%. Since the online group used the Internet as their primary communication method, it was expected that this group would view this question more positively than the other two groups. It was interesting that, although the preferences of the learners differed, the learning achievement level did not. Thus, the outcome suggested that the use of various methods in delivery would provide choices needed for the varied learners and yet would not compromise the learning results. However, it was quite unsettling to see that the comfort level and communication among the learners were not high in the distance education delivery methods.

So where do we go from here? This question can be divided into three areas: implications for instructors, implications for instructional designers, and applications of technology in general.

If instructors view learners as customers, the results of this study were not positive. For example, some learners did not feel comfortable in communicating with the class. Therefore, instructors may wish to pay more attention to learners’ needs in all delivery methods, especially with distance learning methods, and try to establish closer contacts and interactions with the learners. Instructors may also want to encourage and monitor interactions among the learners so that the learners feel more comfortable using the new technology.

Referring back to Table 3, the comfort level was 76% for the traditional classroom, 44% using ITV and 34% online. The interface is even more dramatic, 80%, 44%, and 33% respectively. With better interface design and an increased vigilance of instructors who teach online, we can further explore how to increase the comfort level and interface to improve instruction and learning.

Instructional designers may need to work with instructors more closely to develop learner-centered and learner-driven activities. In most cases, the instructional designer and the instructor are the same person. Thus, instructors need to develop skills or have them provided by other means. Many major publishers have their own instructional design personnel to create compact disks and other tools to enhance a traditional text. Perhaps these services can complement online instruction.

It is true that some view online education as a “glorified correspondence course,” with interactions being few and far between. However, a well-designed distance education course does not fit such descriptions. The instructional designer can put in real-time chats with instructors, threaded discussions among the learners and the instructors, and even public or private e-mails to the other learners in the class and to the professors. One can tape a segment of a guest speaker to be streamed online or, better yet, have a real-time guest speaker sign on to the class to chat with the students. The advantage of this strategy in a distance environment is that this guest may be in a totally different state or country. As long as the time is coordinated, all students, instructors, and guests can log on and learning can begin.

The idea of synchronous and asynchronous learning plays a big part in distance education. Learners and instructors do not have to engage in a conversation at the same time. Instructional designers will need to keep this mind so that the instructors can provide various learning experiences to many different learning styles. Thus, there are many strategies that instructional designers can use to assist firsttime or novice instructors to “reach out and touch” the learners.

Of course, one can never forget technology, as it is the facilitating force behind distance education, whether it is ITV or online. First and foremost, the comfort level of the learners must be raised. Most learners reported the comfort level low as they experienced difficulties with the technology. They were intimidated to ask questions or simply assumed that it was the “computer” that caused the delays or problems.

The software used in this study was IntraKal. Since the conclusion of this study, IntraKal has been improved, and the university and the individual colleges have begun using different software, such as WebCT and Blackboard. Each of these software programs has its advantages and disadvantages, from the ease of use, to the ease to learn, and the cost involved. In general, classroom support software has become much more user friendly and should elicit higher future responses to the comfort question. Educational institutions should spend ample time to assess their own needs before investing in particular software. Many of the programs available contain or facilitate a posting or bulletin board for threaded discussions, e-mails, and chat rooms. Some even have built-in functions such as an academic calendar and icons for assignments, texts, syllabi, and other course elements.

Further, studies suggest that distance-delivered education can be more effective than traditional face-to-face classroom activities (Russell, 2001). The data in this study and knowledge of the degree of sophistication and development of the distance-learning activities gives credence to that notion. Improvements in software, telecommunications tools, knowledge of instructional design and learning principles, and greater interaction suggest that the future of distance learning may result in better learning for students and a more comfortable learning environment.