In this study we sought to develop assessment tools to measure students’ perceptions of two-way audio/video distance classrooms, as well as self-efficacy and satisfaction with the distance learning experience. These measures were subsequently used to assess the relationship between these variables. Initially 222 students in two-way audio/video distance classes completed inventories measuring their self-efficacy, satisfaction, and perception of key elements in the learning environment. Initial data collection along with ratings of individual items of experts and novices were used to refine the measures. A second data collection using the refined measures was completed with 187 students in two-way audio/video distance classes. Measures were correlated to determine the relationship between factors. The results were mixed. Findings offer two possible indications. Students do not discriminate between the concepts measured or the factors are very highly related. Implications and interpretations are explored in the discussion section.
Distance education is a popular venue for instruction and is rapidly becoming more popular in postsecondary education. A report from the National Center for Educational Statistics finds that during the 1997-1998 school year approximately 44% of all institutions of postsecondary education were offering distance education courses (Lewis, Snow, Farris, Levin & Green, 1999). An additional 22% of all postsecondary educational institutions plan on providing distance education courses in the next three years. The same report states that of those institutions surveyed who offer distance education, 56% used two-way audio/video (2WAV) technologies as their primary mode of delivery (Lewis et al., 1999). Much of the extant research on distance education has focused on student outcome (Ahern & Repman, 1994). In focusing on this area, affective aspects have often been ignored. With the expectation that institutions will continue to invest a large number of resources in distance education programs, it is important to understand satisfaction with distance learning courses. Given the increase in the number of educational opportunities and diminishing boundaries, schools must provide their learners with a satisfying experience if they wish to maintain and increase retention.
The hypothesis that learning outcomes in distance learning classes is equivalent to or better than traditional classrooms has been well established (Moore & Kearsly, 1996). Even though the learning outcomes from distance education classrooms are equivalent to or better than traditional classrooms, there is still approximately a 1 in 3 attrition rate in distance education courses (Moore & Kearsly, 1996). Because of this, the authors believe that it is important to investigate distance learners’ satisfaction with learning in order to assist distance educators, designers and policy makers with improving the attrition rate of distance programs.
This study investigates the most fundamental difference between a traditional classroom and a 2WAV classroom—the environment in which the instruction takes place. It is this pivotal difference and its relationship between students’ perceptions of self-efficacy and satisfaction with the learning experience on which this study focuses. The logic behind this focus is two-fold.
First, the authors believe that there is a positive relationship between the students’ perception of the 2WAV learning environment and the students’ satisfaction with the learning experience. Given this relationship, it is imperative to determine which elements can be modified to create a more positive atmosphere for the learner. Following this logic, it is probable that student satisfaction, and thus retention, will increase as a result.
Second, and equally important, is the learners’ own expectations of success in a 2WAV classroom. Specifically, it is hypothesized that there exists a positive relationship between students’ satisfaction with the learning experience and their self-efficacy. Support for this hypothesis has implications for improving student satisfaction, learning, and retention. Support for the hypothesis will encourage educators to develop specific strategies to improve self-efficacy and can combat the problem of dissatisfaction and retention in distance education.
While there is a paucity of research that directly addresses this issue, there is support for the idea that many people are simply not comfortable with new technologies or that they believe it will hinder their ability to perform tasks (Rosen & Maguire, 1990; Mahmood & Medewitz, 1989). We extend the work of prior authors by examining 2WAV distance education environments.
Students’ Perception of the Environment and their Satisfaction with the Learning Experience
Although 2WAV classrooms have been around for over 40 years, there has been little empirical examination of physical environmental variables and, in particular, students’ perceptions and satisfaction with them (Biner, Dean, & Mellinger, 1994). According to McIsaac and Gunawardena (1996) “… research is needed to identify how technology interacts with students and how it affects teaching and learning.” Additionally, research on how the technology affects learners as well as the environmental conditions necessary for its implementation should be ongoing (McIsacc & Gunawardena, 1996.) This paper attempts to address these research needs.
The physical environment is one of the foundations on which a comfortable classroom is built. Careful planning of a 2WAV classroom can ease communication and interaction; failure to do so can be detrimental (Cape, 1996). Although there are a plethora of techniques to facilitate communication and interaction in the classroom, use of any of these methods in the absence of a sound foundation is analogous to a well-conceived building placed upon a fault line. Careful planning and judicious support in a 2WAV classroom plays an important part in establishing a solid foundation for distance learning.
To produce a solid foundation, factors such as lighting, audio quality, and table and monitor arrangement, all of which affect a student’s perception of the environment, must be addressed. In addition, unintelligible audio or video portions of the class can create a whole new set of stressors that may hinder learning. Seeing oneself on a video monitor can be unnerving to some people, while to others it’s a curiosity (Bruce & Shade, 1995). For some, the technology in this environment can be particularly disconcerting. As a result, the availability and response time of technical support staff in this environment is of particular importance. If a problem with the technology arises and it is not resolved quickly, the frustration level of all participants may increase while satisfaction decreases. It is essential that the technology, environment, and support work together seamlessly, lest the environment become intrusive, provoke anxiety, and ultimately compromise the students’ satisfaction with the learning experience.
Self-Efficacy and Satisfaction With Learning
For many people, learning in a 2WAV distance education environment is a new experience. New experiences by nature can be anxiety-provoking. Coupling this experience with new technology can heighten anxiety. Our study examines the relationship between students’ perception of their ability to cope with the technology and the environment. According to Bandura (1977) an individual’s ability to cope with anxiety-provoking situations is an issue of self-efficacy. If students are not able to cope with the learning environment, students will expend less effort and learning is unlikely to occur (Schunk, 1985).
One cause for concern regarding self-efficacy is the physical distance between classrooms, the students and the teachers can strip the class participants of the physical presence and comfort that can be found in a traditional classroom (Moore & Kearsley, 1996; Bruce & Shade, 1995). 2WAV technology was originally seen as a solution for problems associated with the dispersion of members of the classroom. The technology was supposed to increase a sense of presence and foster increased interaction, thus creating a better and more satisfying learning experience for the students. To a certain extent, 2WAV has met these expectations (McIsacc & Gunawardena, 1996).
Support for 2WAV’s ability to provide a medium that is equivalent to face-to-face in terms of interaction has found support in communications research. Researchers in this area found, when examining interactions between people over 2WAV versus face-to-face situations, that people tend to communicate equally well in either situation. With respect to communication, very little difference has been found between the two mediums (Sellen, 1995; Heeter, 1992). Unfortunately, once technology is incorporated into educational situations, issues of communication tend to lose their simplicity. Realizing this, Hillman, Willis and Gunawardena (1994) pointed out that although the 2WAV environment has the potential to provide real-time interaction, significant impediments might result if the student is not comfortable with the technology or if the technology is poorly implemented.
There are numerous ways to promote higher levels of self-efficacy. One important way is to establish a safe and comfortable environment. A safe environment is likely to increase a person’s level of self-efficacy and decrease anxiety, while the opposite will have an equal and opposite effect on their self-efficacy and anxiety. In a traditional classroom the number of environmental variables are minimal yet familiar to the student. However, in a 2WAV classroom there are numerous variables which are often less familiar. In addition, the use of technology itself might lead to an increase in anxiety and a decrease in self-efficacy (Rosen & Maguire, 1990; Mahmood & Medewitz, 1989).
This study seeks to develop a method by which physical and environmental aspects as well as students’ perceptions of self-efficacy in this type of environment and their overall satisfaction with the learning experience can be assessed. The authors believe that students’ satisfaction with the learning experience is related to environmental variables and students’ self-efficacy.
Method
Participants
The participants for the study consisted of three groups. The first group was made up of students enrolled in 12 separate 2WAV classes at two major Midwestern universities and two community colleges. The sample consisted of 222 subjects (146 female, 54 male, 22 non-reported; 93% Caucasian, 4% African American, 1% Hispanic, 1% Asian, 1% other). The age range was 18-64, with a modal age of 19 and a mean age of 31. Ninety-seven percent of the participants were native English speakers. The second group consisted of 3 experts in distance education and 97 novice raters from a major Midwestern university. The final group were students enrolled in 22 separate 2WAV classes at two major Midwestern universities and two community colleges. The sample consisted of 187 subjects (122 Female, 63 male, 2 non-reported; 89% Caucasian, 1% African American, 2% Hispanic, 1% Asian, 7% non-reported). The age range was 16-55, with a modal age of 18 and a mean age of 26. Ninety-nine percent of the participants were native English speakers.
Instruments
The instruments used in this study included the following measures: general satisfaction with the distance learning experience, perception of self-efficacy in a 2WAV education environment, and student perception of environmental elements of the 2WAV classroom. Environmental variables were subdivided into three categories: perception of the physical environment, perception of the physical layout, and perception of the management of the distance learning environment.
Due to the lack of empirically validated instruments, measures were developed to assess these constructs. Developing these measures was a multi-step process. The initial items were developed through a combination of techniques.
The measure of satisfaction with the learning experience was developed based on measures assessing similar concepts. Consumer satisfaction has been an important area of research in many fields (Ware, 1978; Lebow, 1982; Locke, 1976). In the past decade this area of research has gained interest in the area of education (Chadwick & Ward, 1987). Although the fields studied in satisfaction research vary, each seems to tap a certain unidimensional factor in its assessment of consumer satisfaction. Larsen, Attkisson, Hargreaves, and Nguyen (1980), in their development of Services Evaluation Questionnaire, found that three items defined a unidimensional measure of satisfaction.
To what extent has our program met your needs?
In an overall general sense, how satisfied are you with the services you received?
If you were to seek help again, would you come back to our program?
These three items were used as a guideline for the initial development of items to assess satisfaction with learning experience.
In the case of the self-efficacy measure, items were developed based on a variety of self-efficacy measures whose individual items were based on Bandura’s (1977) original theory of self-efficacy. Items related to aspects that reduce self-efficacy, such as anxiety, were reverse scored. These items were then specifically tailored to address the learning environment in question, the 2WAV classrooms.
Finally, in developing a measure to assess environmental aspects of a 2WAV classroom, ten distance education educators and four technicians from two large Midwestern universities were interviewed. Specifically we asked them questions regarding their own experiences and their students’ perceptions of the distance education environment. This information was then summarized and categorized into different environmental issues for 2WAV environments. Individual items were derived from these categories.
From these procedures, a base of 64 items was developed. Using a combination of techniques these 64 items were further refined to develop the final measures. The refinement procedure consisted of three pieces. In the first step 222 subjects in 2WAV classes were given all of the 64 items. The students were asked to rate how strongly they agreed with each of the items. Ratings were given using a six point rating scale with one being strongly disagree and six strongly agree. Item analysis was performed using coefficient alphas. Next, three experts in the field of distance education were given the items and were asked to rate how accurately the items measured each of the constructs. Last, a rating procedure was used in which 97 novices, students in educational psychology courses, were given definitions for each construct, and asked to identify which construct or constructs each of the items seemed to measure.
The following definitions were used in each rating procedure:
Satisfaction with the Learning Experience: How satisfied one was with their learning experiences in a two-way audio/video classroom.
Self-Efficacy: Self-assessment of one’s ability to be successful in a two-way audio/video classroom.
Physical Environment: Perceptions of various physical characteristics of the two-way audio/video environment, such as audio, video, and so on.
Physical Layout: Perceptions of the physical layout of a two-way audio/video classroom.
Management of the two-way audio/video Classroom: Perceptions of how well the instructor and/or technology support team was able to successfully manage a distance learning environment. Management includes the support and use of the two-way audio/video technology, as well as regular classroom management.
Using these definitions, the novice raters were asked to categorize each item. Participants were allowed to choose one, multiple, or none of the constructs when making this decision. Next, concordance rates among these students were calculated. The experts followed a similar procedure.
Each of these three methods was then used to refine each of the scales. The items eventually chosen for each of the scales had a wide range of frequency ratings from the novices. When examining the results of the ratings, we looked at two specific areas. First we examined the percentage of identifications for the selected definition alone and then we looked at the number of identifications for something other than the selected definition. For the satisfaction with learning experience there was a 26%-79% selection of the satisfaction with learning experience alone. The two items with the smaller confirmation rates had confirmations of 25% and 17% for the no matches choice which detracted from their overall concordance rate. For the self-efficacy items there was a 72% - 90% selection of the self-efficacy definition alone for the items with no other choices or combinations above a 25% selection rate. The environmental variables, almost uniformly, were confirmed as one or a combination of the three environmental choices. The overall percentages for confirmations as one or a combination of the three environmental variables ranged from 48% - 98%, with only two of these items below the 75% rate. However, when these items were broken down into the three individual environmental categories the novices’ confirmation rate dropped considerably. The only category with reasonable sole confirmation rates was the physical layout, with confirmation rates ranging from 60%-85%.
The concordance rates from the novices were examined in conjunction with the expert ratings as well as the item analysis from the initial sample. Based on these results, several of the initial items were deleted and some of the wording on a few items was changed. Last, based on the responses from the expert raters, four additional items were added to flesh out the newly refined scales.
The final measures included items such as; “I would take another 2WAV class” (Satisfaction with learning experience [SLE]: 6 items); “Interacting via 2WAV is probably something I will be good at.” (Perception of Self-Efficacy [SE]: 13 items); Environment Perception [EP] (19 items) which was divided into three subscales; “The lighting in the room is good” (Physical Environment [PE]: 7 items); “The layout of the room makes interacting with people at my site easy” (Physical Layout [PL]: 6 items); “The technology support for this course is good” (Management of the 2WAV classroom [MTAV]: 6 items).
These items were given to a second group of students (n=187) in 2WAV classrooms. Reliability analysis was performed on each of the new scales. Each measure was found to have acceptable internal consistency (SLE = 0.88; SE = 0.91; EP = .89; PE = 0.74; PL = 0.83; MTAV = 0.67).
Procedures
The initial data collection took place over two semesters. Twelve classes participated in this data collection. The second data collection took place during the fall consisting of 97 students from educational psychology classes. The final data collection took place during the following summer semester. Twenty-two different classes participated in this round. In the first and third instances, during the last two weeks of class, a questionnaire was administered to each participant. In all instances, prior to the administration of the questionnaires, the participants received an informed consent form, and were asked to participate voluntarily in the study. Standard human subject procedures were followed as designated by each school’s human subjects committee.
Analysis and Results
Scores on each of the three measures in question (SLE, SE, EP) and the three sub-scales (PE, PL, MTAV) were obtained for each of the students. In order to determine if the instruments were measuring different and distinct constructs, bivariate correlations were run. Each of the measures were significantly correlated at the alpha = .01 level. The results of the analysis are displayed in Tables 1 and 2.
The results of the bivariate correlations indicate that all the variables are highly correlated with each other. Given such a high correlation level it was determined that additional analysis examining the relationship between these instruments was unwarranted (Huberty and Morris 1989).
Discussion
The results of the analysis lend themselves towards two possible interpretations. One possible interpretation, and one that needs to be investigated further, is that the measures are assessing the same construct and not individual unique factors. This interpretation is based on the relatively high levels of correlation between the individual measures. It would seem, following this line of reasoning, that the students in question perceived the items in the same way.
A second interpretation of the data is that each of the variables is a function of the other variables. The interesting, albeit non-conclusive interpretation, is that satisfaction is highly related to perceptions of the environment and self-efficacy. Thus, high positive perceptions of 2WAV environmental issues in a class would mean that students would most likely have high satisfaction for that class. Additionally, high perceptions of self-efficacy would also mean that students would most likely have high satisfaction for the class. Therefore, students’ satisfaction with the course will be directly related and intimately tied to their perceptions of self-efficacy and perceptions of the environment. This interpretation is supported by the fact that both experts and novices, who were not in a distance education class, interpreted the items for each of these measures as different, while participants in the 2WAV classroom answered them in a very similar direction and magnitude.
The results of this study provide an important first step. It appears that, at the very least, designers should consider students’ perceptions of environmental elements and the relationship to students’ self-efficacy and satisfaction when designing a 2WAV classroom. Additionally, distance educators should consider techniques for improving students’ levels of self-efficacy in the 2WAV classroom. Simple exercises in the beginning of the course that provides the learners with positive experiences could improve the students’ levels of self-efficacy and increase their satisfaction in the course. However, the results of this study are far from conclusive and in order to make specific recommendations further research needs to be done.
Finally, the implications for this study extend beyond two-way audio/video classrooms. In a recent study examining environmental variables in an online web-based course it was found that students’ satisfaction was intimately tied to environmental variables that led to increases in frustration (Hara & Kling, 1999). Distance learning in all forms is rapidly growing. The questions we ask of 2WAV classes are some of the same ones being asked in asynchronous and synchronous web-based courses. If we are to successfully pull students into a novel environment, we must not forget the affective aspects of doing so. Furthermore, it is essential that educators and designers take these elements into account when developing and conducting distance education classes. If we want distance education programs to excel, it is important to tune in to students’ satisfaction with distance learning experiences.
The authors wish to thank Charles Evans, Terence J. Tracey and the late Richard C. Pugh for their advice and assistance. Additionally, different portions of this paper were presented at the following conferences in 1998, AECT, AERA and APA and the authors would like to thank the conference participants who provided comments and insight into the study.
This study was supported in part from a grant from the Office of Statewide Programming, University of Illinois at Urbana.
