The purpose of course evaluations is to allow students the opportunity to evaluate the course and the instructor. As a reflective teacher, I take the 20 years’ worth of comments and numerical ratings on traditional, seated course evaluations to heart. When compared to newer evaluations from the online environment, a dichotomy exists. The purpose of the study is to explore why the evaluations are so different and to identify possible solutions to unify to chasm between seated and online perceptions to the same statistics course.
As a reflective teacher, I take the comments and numerical ratings of my course evaluations to heart. It is necessary to think long and hard about the Likert scale rankings and the constructive comments that students provide and make the necessary changes the next time the course is taught. However, I have recently found myself in a state of cognitive dissonance with respect to the student ratings in the seated versus the online environment.
The purpose of course evaluations is to allow students the opportunity to evaluate the course and the instructor. Research has been done on student evaluations with respect to student satisfaction criteria (Kirk & Spector, 2009; Rothman, Romeo, Brennan, & Mitchell, 2011), comparisons of full-time and adjuncts (Kirk & Spector, 2009; Mueller, Mandernach, & Sanderson, 2013), and quantitative course evaluations (Constand & Pace, 2014; Royal & Stockdale, 2015) among many other facets of course evaluation. Of particular interest and relevance to this study is the student perception of synchronous versus asynchronous online education models (Offir, Lev, & Bezalel, 2008; Parenti, 2013) and the direct application to evaluation systems (Allen, Bourhis, Burrell, & Mabry, 2002; Baker, 2010; Lyke & Frank, 2012; Rothman et al., 2011; Swan, 2002).
One particular type of course evaluation, and specifically employed at the university of study, is called the IDEA student ratings inventory. The instrument is powered by Campus Labs for ease of student use in its interactive online platform. The system controls for extraneous circumstances (e.g., class size, student motivation), and provides comparative scores (IDEA, n.d.). This type of course evaluation is different than many homemade evaluation instruments in that it is not just the proverbial faculty popularity contest. It is a standardized system for all universities which also allows for converted ratings compared at both the institutional and content discipline levels, including customized reports with system feedback on strengths and areas of improvement. Similar to many other evaluation instruments, it measures the summary of excellent teacher and excellent course. However, our institution chose the normative system as it also measures students’ perceptions of their progress on relevant objectives, therefore attempting to measure the student learning captured the course. Our university has opted to use this same system of evaluation across both seated and online environments, as directed by Berge and Meyers (2000) and Clark (1989).
In teaching math in the seated environment at the same school for the last 20 years, I have found very positive evaluations of my courses, teaching pedagogy, and student progress on meeting the learning objectives. Because of the identical nature of IDEA evaluations across modality and a fairly newfound interest in teaching online, I found some interesting descriptive statistics in the comparison of teaching evaluations differed only by modality (i.e., same course, content, assignments, etc.). Using a Likert scale with low = 1 and high = 5, my 3-year average for seated coursework ratings was 4.76 for excellent teacher (n = 227), with a very small range of .4. The mean for excellent course rating was 4.29, with response rates hovering around 75%. Common qualitative statements included statements such as “you are perfect!” “She was an awesome teacher that I enjoyed having as a professor” and finally, is by far one of the best professors. She is extremely active and engaging with students and course material. It is obvious that she enjoys teaching and has a great passion for learning. She makes students WANT to do well.
In contrast, my online ratings provided a very different vision. Using the same Likert scale with 1 = low and 5 = high, my average was 3.78 for excellent teacher, while the excellent course rating average was 3.2 for the mere 19 students who filled out the evaluations across the three summer semesters. The response rate average for the online courses was 27%. The discrepancies in my evaluations between modalities were obvious. Although the decrease in ratings online matched with the propositions found in some other studies (Baker, 2010; Lyke & Frank, 2012; Rothman et al., 2011), it was important to identify the potential cause for the discrepancies and to explore solutions.
Shortly after receiving the poor course evaluations one summer, an unscheduled meeting with a student caused an epiphany. After spending the better part of an hour with “Ann,” the name of the student finally clicked: she was one of the students who had just completed the online statistics class. When I met her in the live setting, I found her to be completely dynamic, full of energy and vitality, with lots of personality. That is when I realized that I never “got” Ann; her personality was not allowed to shine in its bright capacity in the asynchronous online environment. When I met her in the live setting, she was as articulate as in her online persona, but so much more. My epiphany was twofold, as I also hypothesized that neither Ann nor her classmates “got” me either. I too, am a vivacious person with a passion for student learning. The asynchronous environment again squashed the students’ perception of how much I care, despite multiple emails stating such facts.
With this story and general context in mind, I hoped to circumvent the disconnection by holding synchronous sessions to teach, answer questions, build rapport, and otherwise bond with students in an upcoming online statistics class. The literature supported my hypothesis that increasing the number and quality of student/faculty interactions and responses into real time would provide for a greater sense of engagement, satisfaction, and perceived learning (Allen et al., 2002; Falloon, 2011; Hrastinski, 2008; Swan, 2002). Schutt, Allen, and Laumakis (2009) encouraged the addition of synchronous activities, and others offered its enhancement of student outcomes (Dickey, 2003; Shotsberger, 2000). Additionally, Parenti (2013) listed student preferences for live synchronous class time sessions, live synchronous chat messaging, as well as keeping the standard asynchronous e-mail communications. I adhered to all such advice with the thought that by adding synchronous learning to the course structure, I could get to know the students better and that they would truly see how much I care and my passion for their learning. I hoped that this would translate into greater degrees of learning and associated satisfaction with the course and my teaching, per general common sense as well as the literature.
Unfortunately, once mentored through using the synchronous materials and implementing them during the online summer session, I found my results to be similar to previous semesters. The actual results were 3.4 for excellent teacher and 3.3 for the progress on relevant objectives. In short, the excellent teacher rating went down, and the excellent course rating increased, though only slightly. Holding the live synchronous sessions might have helped, but not greatly. One student commented that “The coursework was challenging but the weekly live sessions definitely helped.” For others, however, the live sessions did not change the students’ perceptions of learning, nor their perceptions of me:
[She] is quite possibly the worst professor I have come across in my academic and professional career. She made the subject dry and difficult and outside of setting up a 1 hour tutor session every Thursday, she hardly helped in getting through the subject … made this subject so difficult that I had to take time from other courses just to balance her workload. She overly criticized every assignment I completed and made an entry level course into the most difficult subject I have ever come across. I would not recommend her to anyone.
I could write a whole dissertation defending myself from the student above, but suffice it to say that provisions were made to support the student in every capacity available: extended deadlines, mastery learning approaches, frequent encouraging messages through the class announcements page, a daily answer to questions posted in the specific help forum, and so forth. However, the notion that I must defend my actions just further exemplifies my position that the students did not feel the support that I was offering. Despite the live sessions, it was not enough to build the support, rapport, and trust necessary for optimal student success in the difficult content area.
While the results of my synchronous experiment neither demonstrated any evaluative improvement in the sample nor could they be inferred to the population due to the very limited sample sizes, they are still results worthy of reflection. The literature is fairly clear that the content area is generally rated lower than other coursework (Cashin, 1990, 1995; Constand & Pace, 2014; Felton, Koper, Mitchell, & Stinson, 2008; Royal & Stockdale, 2015), but as the content remained identical in level, instructor, assignments, and so forth, the validation of quantitative subject matter is moot. However, a reflective extension is to call for further research to compare all quantitative courses taught online at the university and/or the comparison of teaching evaluation scores of all professors who use the live, synchronous component available in the learning management system. A larger group of students might show that teaching synchronously is a benefit to students’ perceptions of learning.
In conclusion, the changes to the class learning platform from asynchronous to synchronous, are inconclusive due to the low response rate and the small sample size. However, when I teach online again, I will continue the use of the synchronous component as I do feel that students gain greater benefits exemplified by a student comment on a course evaluation: “You can’t ask the video a question.” The synchronous format cannot be mandated due to our current marketing strategy for online education, but it is still the right thing to do, even if only on a voluntary student basis. For the time being, I will keep trying to enhance the connections between students, mathematical content, and myself in a synchronous environment, and spread the word of its potential benefits, that “when students are more active in the learning process, the material becomes more relevant and more significant for them, they remember it better, understand it and as a result their achievements improve” (Offir et al., 2008, p. 1181)
