Widespread availability of computers and the Internet provide considerable enrichment in terms of variety of material and formats for presentation over what was possible with the old correspondence courses. As a result, a large number of universities have begun to offer an extensive list of online courses in various programs. We set out to answer the question about student satisfaction with online distance learning by surveying the undergraduate and graduate students taking online courses in the College of Liberal Studies of the University of Oklahoma. The survey was sent to a total of 767 students who had participated in an online course lasting either 8 or 16 weeks in the College of Liberal Studies at the University of Oklahoma. Students were contacted and requested to participate in the survey at the end of each 8-week and 16-week course. A total of 304 students participated in this survey producing a participation rate of 40%. Overall the results of this survey indicated that the students were very satisfied with the format, the nature and amount of the work required, as well as with the assessment of their work and grading.
INTRODUCTION
Distance education, in one form or another, has been around at least since the nineteenth century, when colleges began to offer instruction through correspondence courses. Over the years various media have been employed to facilitate distance education, including radio, television, and more recently computers and the Internet (Phipps & Merisotis, 1999). While design flaws have frequently been pointed out in the research in this area, there is a pretty substantial accumulation of evidence indicating that distance education produces results equivalent to face-to-face classroom instruction in many areas (e.g. Russell, 1999). While there are advantages and disadvantages to online instruction, student satisfaction is generally good (e.g., Johnson, Killion, & Oomen, 2005). While barriers and problems with online teaching have been investigated (e.g., Berge, 1998), widespread availability of computers and the Internet provide considerable enrichment in terms of variety of material and formats for presentation over what was possible with the old correspondence courses. As a result, a large number of universities have begun to offer an extensive list of online courses in various programs. It is now possible at many institutions to obtain training through the doctoral level entirely online. Many faculty providing instruction in these courses have been doing so for 10 or 15 years (Bonk, Maher, & Halpenny, 2001). A question of concern to such faculty is, “How are we doing?” There are two ways to answer this question. One is to evaluate student achievement using various assessment procedures to determine the amount learned in such courses and, when possible, to compare this achievement with that of students in on campus face-to-face classes. A second approach to answering this question is the topic of the present article. This approach has to do with student satisfaction. What factors in online courses do students consider important in enhancing their learning? What characteristics of online instruction do they specifically single out as the most important benefits of such instruction and what are their top pet peeves about online instruction? We set out to answer the above questions by surveying the under-graduate and graduate students taking online courses in the College of Liberal Studies of the University of Oklahoma. The University of Oklahoma has been a pioneer in distance edu-cation. In 1960, the College of Liberal Studies at the University of Oklahoma was established and, over the years, had extensive offerings of courses leading to a bachelor’s and master’s degree (begun in 1967) in liberal studies. As computers and the Internet became readily accessible, the university switched its format from correspondence with intensive on-campus seminars to totally online offerings for most courses and programs. The first online courses in this program were offered in 1997. It seemed appropriate at this point to survey students regarding their attitudes regarding effectiveness of the program.
METHODS
Following a review of the literature and a careful examination of the format employed for most online courses in the College of Liberal Studies, a survey was prepared for administra-tion to students taking such courses. It was determined that the survey must be brief and easily accessible in order to obtain student cooperation. Previous attempts at routine eval-uation of these courses had resulted in a very low rate of participation by students. The survey employed appears as Appendix A at the end of this article. The majority of the courses offered by the College of Liberal Studies are organized in four units that the student completes in either 8 or 16 weeks. Each unit typically has a required reading from the course textbook, as well as library and Internet research. Based on this, the student writes two to three essays on assigned topics covering the material. Each unit also involves one phase of preparation of an overall paper called the “course spanning paper” that is due at the end. In most of the courses, no standardized or objective tests are used, rather each of the essays and papers receives a grade and the final grade for the course is dependent on performance on these writing tasks. The survey employed in this investigation was specifically prepared to mirror the course structure employed in the College of Liberal Studies.
The survey was first administered as a pilot study to summer school students in 2005. Stu-dent response to the survey was very good and there did not appear to be any problems with ambiguity or understanding the questions. Thus, it was determined to use the same instrument in a more extensive survey for the fall semester of 2005. Two additional questions were added at this point to facilitate data anal-ysis—students were asked to indicate their gender and whether they were undergraduate or graduate students. Students were contacted and requested to participate in the survey at the end of each 8-week and 16-week course. A total of 48 different undergraduate and 24 graduate courses, many with multiple sections, were included in the survey. Responses to the survey were anonymous. Trent Gabert, associate dean of the College of Liberal Studies, sent an e-mail letter to all students informing them of the survey and encouraging them to partici-pate in order for instructors in the college to understand student attitudes better and improve the quality of courses. The instructors in the courses were also requested to encourage their students to participate. Two followup reminders were sent via e-mail to all students encouraging them to participate if they had not already done so.
ANALYSIS
Data for questions 1-15 were expressed in per-centages. In addition, the raw data for these items were analyzed using a MANOVA. The data were also subjected to a regression analysis which was performed between overall sat-isfaction with the online program (question 15) as the dependent variable and the 12 questions regarding specific student satisfaction as the independent variables. The open-ended ques-tions: “16. The best thing (kudos) about OU online courses is (list 2 or 3)” and “17. The worst thing (pet peeves) about OU online courses is (list 2 or 3)” were subjected to a qualitative analysis. Three graduate student judges independently read all of the responses for questions 16 and 17. As they read the responses, they made a list of categories into which they thought it would be possible to classify the comments. After each had read the comments and created categories indepen-dently, they met and in conference settled on a final list of categories for each question. They then reread each item classifying it into the appropriate category. Items for which two or three of the raters agreed were considered clas-sified. Comments not so classified were dis-cussed in conference and classified by consensus. Only 4 of the 782 total responses had to be classified by conference.
RESULTS
The survey was sent to a total of 767 students who had participated in an online course lasting either 8 or 16 weeks in the College of Liberal Studies at the University of Oklahoma. Responses were anonymous and were made via a specially prepared website. A total of 304 students participated in this survey producing a participation rate of 40%. While we had hoped for a better participation rate, 40% was considered acceptable for a strictly voluntary and anonymous survey.
A MANOVA was conducted on questions 3-15 regarding student satisfaction using data from questions 1 and 2 to categorize respon-dents. Questions 7, 8, 9, and 13 had significant MANOVA models. Question 7 had a significant MANOVA of F(3, 241) = 10.53, p < .0001. Review of the analysis of variance indi-cated that degree was a significant contributor of differences, F(1, 241) = 23.86, p < .0001. The Bonferroni t test revealed that undergraduate students (M = 4.2831) enjoyed sharing their work with other students more than did graduate students (M = 3.1139). Question 8 had a significant MANOVA of F (3, 241) = 10.76, p < .0001. Examination of the analysis of variance indicated that degree was a significant contributor of differences, F(1, 241) = 31.85, p < .0001. The Bonferroni t test revealed that undergraduate students (M = 3.1265) ideally preferred feedback in a shorter amount of time than did graduate students (M = 4.6203). Question 9 had a significant MANOVA of F(3, 241) = 18.28, p < .0001. Inspection of the analysis of variance indicated that degree was a significant contributor of differences, F(1, 241) = 47.11, p < .0001. The Bonferroni t test revealed that undergraduate students (M = 5.1386) realistically preferred feedback in a shorter amount of time than did graduate students (M = 7.3544). Question 13 had a significant MANOVA of F(3, 241) = 2.85, p = 0.0382. A look at the analysis of variance indicated that degree was a significant contributor of differences, F(1, 241) = 6.46, p = 0.0117. The Bonferroni t test revealed that graduate students (M = 2.3165) were less satis-fied with an 8-week-long course than were undergraduates (M = 2.0482). In addition, there was a significant interaction between degree and gender, F(1, 241) = 4.12, p = 0.0434. Examination of the means revealed that undergraduate females (M = 2.1885) and graduate females (M = 2.1875) were equivalent in their satisfaction with the 8-week courses. However, male undergraduates (M = 1.8545) were more satisfied with the 8-week courses than were male graduate students (M = 2.5143).
Table 1 presents the percentages of students selecting each option for each of questions 115. Figures 3 and 4 display the frequencies for the categories employed to classify the responses having to do with what was the best thing (kudos) about online courses and what was the worst thing (pet peeves) about such courses.
A standard multiple regression was per-formed between overall satisfaction with the online program (question 15) as the dependent variable and the 12 questions regarding specific student satisfaction as the independent variables. The linear combination of the specific satisfaction variables were significantly related to the overall satisfaction variable, F(12, 138) = 7.19, p < .0. The multiple correlation coefficient was .62, indicating that approximately 38% of the variance of the overall satisfaction with the program in the sample can be accounted for by the linear com-bination of specific satisfaction measures.
Only four of the independent variables (question 3, 8, 9, and 13) contributed signifi-cantly to prediction of overall satisfaction. Satisfaction with reading assignments (question 3) contributed significantly to the prediction of overall satisfaction (β = 0.21, p = 0.01). Ideal time for feedback (question 8) contributed significantly to the prediction of overall satisfaction (β = 0.05, p = 0.06). Realistic time for feedback (question 9) contributed significantly to the prediction of overall satisfaction (β = -0.06, p = 0.02). Satisfaction with the 8-week program (question 13) contributed significantly to the prediction of overall satisfaction (β = 0.10, p = 0.05).
Regressions analyses examining the predic-tive qualities of gender, as well as degree were also conducted. Gender was not a significant predictor of overall satisfaction, F(1, 297) = 1.88, p = 0.1709. Neither was degree a significant predictor of overall satisfaction, F(1, 297) = 0.88, p = 0.3501.
DISCUSSION
The main effects of the MANOVA reveal rela-tively minor differences in which undergraduates enjoyed interaction with their classmates slightly more and wanted quicker feedback on their assignments than did graduate students. The significant interaction on question 13 dealing with 8-week courses reveals that undergraduate males were more satisfied with 8-week courses than were graduate student males. This result must be interpreted with caution because very few 8-week courses are offered for graduate students and these are primarily during summer sessions. Overall, students were very satisfied with 8-week courses. Given the small number of minor differences revealed by the MANOVA, we determined to present the data for the individual items on the survey for the whole group rather than breaking it down by gender or class level.
Data for survey items 1-15 are shown in Table 1. Questions 1 and 2 were included to obtain data for classification purposes in per-forming the MANOVA. The data indicate good participation in the survey by males and females as well as by undergraduate and graduate students
Examination of questions 3, 4, 5, and 6 indicated that students found the basic components and format of the courses useful in enhancing their learning. They were somewhat more positive with respect to the reading assignments and the short essay assignments than they were for the Internet assignments and the major course spanning term paper, however, they were basically positive about all of these activities.
One of the disadvantages cited with respect to online instruction is that there is no opportu-nity for interaction with classmates. To make up for this, some instructors use chat sessions, discussion boards, or encourage class members to e-mail selected assignments to each other for comment and interaction. Question 7 indicates that the students were only mildly pleased with such opportunities. Recall that earlier results from the MANOVA indicated that undergraduate students were slightly more positive about such interactions. Anecdotally, however, some students commented that they were too busy and did not want to hear from other students.
A crucial aspect of online courses has to do with the promptness of feedback to students. Question 8 indicated that most students would prefer to have feedback within 2 to 3 days. However, question 9 indicated 5 to 7 days is acceptable. Recall that the MANOVA revealed that undergraduates, who tend to take courses in 8 weeks, prefer slightly quicker feedback than students in general.
The majority of the courses offered by the College of Liberal Studies employ a series of essays and papers for evaluation and assigning grades. No objective tests are administered. Since grading of student essays is a highly subjective endeavor, we were interested in surveying the students regarding their attitude toward this aspect of online courses. Examination of the results for questions 10 and 11 indicated that students did not regard the grades assigned to them as being overly subjective or unfair. In addition, question 12 indicated that they did not think an objective test would be desirable as a part of the course.
Eight weeks is relatively short period of time in which to complete an entire course online. Our concern with this issue led us to include question 13. In spite of the panic that students often experience with deadlines in courses of that duration, the majority of the students in the survey were of the opinion that eight weeks was long enough to adequately cover the course material.
Question 14 dealt with the amount of work required for the online courses. Overwhelmingly, these students felt that the amount of work demanded was about right. Interestingly, none thought that too little work was required.
Responses to question 15 indicated an extremely high level of satisfaction with the online experience the students had. Thus, overall the results of this survey indicated that the students were very satisfied with the format, the nature and amount of the work required, and with the assessment of their work and grading.
Figures 1 and 2 present summary data for the open-ended questions (16 & 17) on the survey. Over 99% of the responses were assigned to a category on the first attempt and all were successfully classified in conference. Examination of Figure 1 indicates that the flexibility of online courses was the most attractive feature. Typical responses in this category were, “Online courses work very well with people working full time” and “I can do the work in my own time.”
Instructors and materials were rated very highly. Typical comments included “Great instructors,” “The interaction with OU professors is top notch!” and “The books are not things I would read for pleasure but I find them very engaging once I read them for the courses.”
Convenience was also very high on the list. Typical responses in this category were “I can do it all from home and as a single mom that is my most important factor” and “I get to attend OU and still live in my hometown!”
Finally students mentioned the use of tech-nology. However, this was much less important than the others. Items mentioned were the course Web site, e-mail, and message boards.
Examination of Figure 2 indicated that per-sonal issues topped the list of pet peeves. This category consisted of specific complaints about a particular instructor or a problem with a particular assignment.
The second most frequent pet peeve involved the quality of interaction. This had to do with the students feeling isolated because they were not able to meet their professor or classmates face-to-face. Typical comments were, “I would like to personally meet the professors” and “I wish there was more interaction among students.” It should be recalled from the data earlier presented that there was not major dissatisfaction with this aspect of the course. However, for those who had pet peeves, this was one that was frequently mentioned. The same could be said for the third most common pet peeve which refers to too much material. Typical responses in this category were, “Way too much reading and essays. It’s more than a full time job” and, “Having to write so many darned papers!” However again, it should be recalled that the majority of students felt the amount of material was “about right” in the data presented earlier.
Quality and timeliness of feedback appeared as significant pet peeves. Typical responses in this category were, “Feedback wasn’t constructive enough to improve” and, “The feedback is minimal.”
Technology was a significant source of pet peeves. This category referred mainly to difficulties of dealing with computer programs, Internet sites, broken links, and similar problems.
Finally, 30% of the respondents insisted that they had no pet peeves regarding the online experience. It is rather surprising that such a substantial number would have no complaints at all, however, this agrees with the overall satisfaction data presented earlier. Typical comments were “I cannot honestly think of one negative thing about the online courses” and “I’m pleased.”
Examination of the results of the regression analysis indicates that this analysis did not pro-vide significant insight beyond what was obtained via the MANOVA; however, the results did confirm the earlier results in terms of general outcome.
The data in this report provide evidence of a high level of satisfaction with online courses as well as provide insight into the features that students find particularly helpful or detrimental. It should be pointed out that the current data are limited to the particular courses studied at one university. They may or may not generalize to other settings, though as pointed out in the introduction, students in other settings have positively endorsed distance and online instruction.
Acknowledgment
This investigation was supported by a grant from the College of Liberal Studies of the University of Oklahoma.
APPENDIX A: OU COLLEGE OF LIBERAL STUDIES ONLINE COURSE SURVEY
1. Please indicate whether you are:
Female__ Male __
What degree are you obtaining:
Undergraduate__ Graduate __
3. I found the reading assignments useful in enhancing my learning.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree __ Not applicable__
4. I found the essay assignments useful in enhancing my learning.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__ Not applicable__
5. I found the Internet assignments useful in enhancing my learning.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__ Not applicable__
6. I found the course spanning final paper assignment useful in enhancing my learning.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__ Not applicable__
7. I enjoyed sharing my work with and getting comments from other students in courses.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__ Not applicable__
8. In an online course, the professor should provide prompt feedback on completed assignments. Ideally feedback should be received within how many days?
1__ 2__ 3__ 4__ 5__ 6__ 7__ 8__ 9__ 10__ Other__
9. Realistically, feedback should never be later than how many days?
1__ 2__ 3__ 4__ 5__ 6__ 7__ 8__ 9__ 10__ Other__
10. Overall the instructor grading in the courses I have taken online has been fair.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__
11. Evaluation in the online courses I have taken has been too subjective.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__
12. I would prefer an objective test in addition to the more subjective forms of evaluation.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__
13. If you were in an 8 weeks course, was it long enough to adequately cover the course material.
Strongly agree__ Agree__ Neutral__ Disagree__ Strongly disagree__ Not applicable__
14. The amount of work required for the online courses I have taken has been.
Very excessive__ Excessive__ About right__ Too little__ Far too little__
15. Overall my experience with online courses at OU has been.
Very positive__ Positive__ Neutral__ Negative__ Very negative__
16. The best thing (kudos) about OU online courses is (list 2 or 3):
17. The worst thing (pet peeves) about OU online courses is (list 2 or 3).
Thank you very much for your participation. College of Liberal Studies, University of Oklahoma.
