This teacher action research study focuses on causes for a decrease in course satisfaction by students enrolled in a large-scale, compressed timeline online course compared to satisfaction in previous courses. To determine modifications to the large-scale, compressed timeline online course, I analyzed course evaluations and the structures of this course and small-scale online courses. Based on evidence and self-determination theory, I increased the frequency and supportive nature of student-faculty communication and slightly reduced the number of assignments in order to strengthen feelings of relatedness, autonomy, and competence. Evaluation scores and comments from the second large-scale, compressed timeline online course indicated that satisfaction is influenced by these changes.
Introduction
During the migration from traditional brick and mortar classrooms to virtual online platforms, thinking about e-teaching has expanded to include new perceptions of students’ engagement with course content, curricular and pedagogical modifications, instructional roles, formative and summative assessment, and the many factors that influence learning. One useful strategy for gathering information about how to strengthen online programs to better meet local programmatic needs is through the satisfaction ratings of the student course evaluations.
Filak and Sheldon (2003) argue that the many factors investigated as possibly impacting students’ satisfaction, or lack thereof, with university classes can be understood when analyzing student satisfaction through the lens of self-determination theory (SDT). SDT posits that humans possess innate psychological needs for competence, autonomy, and relatedness, and that experiences in social contexts will support or deter fulfillment of these needs. People gain a sense of competence when their experiences promote feelings of confidence, effectance, and success when they employ their capacities attempting challenges. Autonomy is associated with feeling self-determined in that people are the source of their actions and behavior, and they value their actions. Relatedness results from feeling care for others, cared for by others, connected to others in the social context, satisfied, safe, and understood (Deci & Ryan, 2002,2008; Deci, Vallerand, Pelletier, & Ryan, 1991; Filak & Sheldon, 2003; Ryan & Niemiec, 2009).
Ryan and Deci (2009) acknowledge that, while SDT is a theory based on the assumption that there is a common human nature and a set of basic human psychological needs (the need for autonomy, competence, and relatedness), people's diverse cultural values influence their individual experiences and the meaning made from these experiences. Reinforcing this notion of the dialectical relationship between basic human needs and the social environment, Deci and Ryan (2002) emphasize the interaction between the social context and human nature, and the potential for experiences within the social context to support or interfere with meeting basic needs. When these needs are fulfilled, people experience growth, engagement, development, wellbeing, motivation, and integrity. These feelings are forestalled when the needs remain unfulfilled. Thus, people prefer social contexts that promote meeting these needs and are dissatisfied with social environments that interfere with fulfilling these needs. Regarding educational settings, dissatisfaction is associated with contexts in which students have a sense that achievement success, and individual potentials and capacities are beyond reach; lack the support and tools to be successful; lack voice, choice, power, and understanding of the rationale for learning experiences; and feel their environments are controlling, unfriendly, cold, and uncaring, and where they do not feel valued, respected, or liked (Deci & Ryan, 2002; Jang, Reeve, Ryan, & Kim, 2009; Niemiec & Ryan, 2009; Ryan & Niemiec, 2009). Filak and Sheldon (2003) demonstrated this relationship in their study that underscored that feelings of competency, autonomy, and relatedness predicted course and instructor evaluation scores.
Because of a considerable decrease in graduate student course evaluation scores when employing a new large-scale online model compared to evaluation scores from previous classes taught using other models (face-to-face, small-scale online, and compressed video), I investigated factors that may have influenced student satisfaction with the new model. The research questions that guided this study were (1) How does the large-scale, compressed timeline online model compare in structure to the small-scale online model? (2) How do student course evaluations for the large-scale, compressed timeline online class compare to other course evaluations? (3) What factors may have contributed to reducing students’ satisfaction with a large-scale, compressed timeline online class compared to previous classes? (4) What changes can I make to future large-scale, compressed timeline online classes to promote greater student satisfaction without compromising course integrity? (5) How do these enacted revisions impact student satisfaction for a second large-scale, compressed timeline online class?
Background
In 2008, a midsouth university school of education began replacing some face-to-face, online, and synchronous classes (offered at several different sites via compressed video) with a large-scale, compressed timeline online model supported by Educational Alliance (a pseudonym; EA), a branch of a private, for-profit company. As was true with previous classes I taught, the first EA course (EA1) was designed for experienced educators interested in licensure as school leaders. Two hundred thirty-nine educators working towards the Master of Science in Education (Educational Theory and Practice) enrolled in the class. Four teaching assistants (TAs) were assigned to interact with students, answer questions, and grade assignments. The TAs were full-time teachers or administrators with advanced degrees related to the program. I had previously taught this course five times during small-scale online classes during 5-week summer sessions and as a semester-long synchronous, compressed video class. At the end of teaching my first course using the EA model, I received low evaluation scores for all course evaluation items. For example, the mean score for the item “I would highly recommend this professor to other students” was 2.5 (5.0 = “strongly agree;” 1 = “strongly disagree”). This mean score is more than 2.0 points below the score for the item related to overall satisfaction of me as the instructor on all course evaluations during my previous five semesters teaching graduate education courses. Clearly, I needed to seriously and systematically reflect on the possible causes of the divergent EA1 course evaluation scores and make changes to future EA courses.
General Methods
To launch my thinking about how I could improve student satisfaction, I turned to teacher action research. I chose Mills’ Practical Action Research model (Mills, 2011) because I was interested in analytically examining my practice, investigating practical questions, and developing a plan for effecting change. Figure 1 illustrates the stages of my action research study. During the first stage of my action research, I gathered data (features of online course models and results from student course evaluations) to compare my first large-scale, compressed timeline online course with previous courses. Evidence from these data, in conjunction with consideration of self-determination theory, guided modifications to my second large-scale, compressed timeline course. During the second stage of my inquiry, I gathered data from student evaluations for the second large-scale, compressed timeline online class and compared these student evaluations to those for EA1 to determine if the changes improved student course satisfaction.
Modified Dialectic Action Research Spiral Summarizing Action Research Methodology
Modified Dialectic Action Research Spiral Summarizing Action Research Methodology
Stage 1
The purpose of Stage 1 was to make changes in a second large-scale, compressed timeline online class based on an investigation of the first large-scale, compressed timeline online course and previous courses, including small-scale, online courses.
Method
Since I was focused on making changes in my practice, I first generated a list of the features, including my practices, of EA1 and other online classes I had taught. A second data source was students’ ratings on course evaluation items common to the evaluation instrument for the EA model course and all other courses offered for which I received evaluation summaries.
Last, I collected and reviewed comments on all evaluations. While it would be most informative to compare comments from EA and non-EA online courses only, it was necessary to review comments for all courses because comments provided to me were not necessarily separated by course.
Results and Discussion
The online courses offered via two different models shared features but also differed in some respects as summarized in Table 1. Similarities included the expectations of the students’ time commitment and the structure and number of assignments. One major difference between the non-EA and EA online models was that students enrolled in EA1 had less time to complete course work because they enrolled in the course during 5 weeks while they were working full time, while the non-EA students either enrolled in semester-long classes during the school year or a 5-week session during their summer vacation. A second major difference between the models was the student enrollment: approximately 20 students enrolled in the non-EA online classes; 329 enrolled in EA1. A third major difference was that I interacted a great deal more and in different ways with students in the non-EA classes. Because of the intentional EA model design, I had very little interaction with EA1 students.
Features of Small-Scale, Online Courses and Large-Scale, Compressed Timeline Online Courses
| Feature | Non-EA Online Classes | EA Online Class |
|---|---|---|
| Student enrollment |
|
|
| Class schedule |
|
|
| Student time commitment |
|
|
| Student assignments |
|
|
| Faculty responsibilities |
|
|
| Feature | Non-EA Online Classes | EA Online Class |
|---|---|---|
| Student enrollment | Approximately 20 | No cap |
| Class schedule | Semester-long school year courses Five-week summer courses | Five-week course during school year |
| Student time commitment | Approximately 150 hours; 10-12 hours per week | Approximately 150 hours per course; 30 hours per week |
| Student assignments | Readings, reaction papers (to readings) interviews with practicing educators, literature review, analysis of school curriculum discussion board contributions | Readings, reaction papers (to readings), interviews with practicing educators, literature review, analysis of school curriculum, discussion board contributions |
| Faculty responsibilities | Develop and update curriculum, instruction and assessments; Assess student work; provide substantive feedback; re-assess work; Interact with whole class, in small discussion groups, and individually; Interact with students via e-mails, online discussion boards, precourse sessions, live community/school visits, phone calls; Inform students about plagiarism tool; facilitate students’ submissions to plagiarism website | Develop and update curriculum, instruction, and assessments; Train TAs to assess student work and supervise grading; Interact with TAs via weekly phone conferences and e-mails; Interact with students via e-mails forwarded by TAs; Inform students about weekly focus using video; Inform students about plagiarism tool; Facilitate students’ submissions to plagiarism website |
Next, I turned to the results of evaluations for 18 classes in which 668 students were enrolled. For the combined EA1 and non-EA courses, 52% of the students completed evaluations. Course evaluations were not available for four non-EA classes for unknown reasons. Three different evaluation forms were used by the department thus a direct comparison of all instrument items across all courses was impossible. However, I did identify items that were the same or similar for the different evaluations and compared ratings for these common items. The means for similar items are found in Table 2. The EA1 students’ mean scores were lowest for items related to overall rating of professor (M = 2.5), clear explanations/exam ples (M = 2.7), appropriate assignments (M = 2.8), useful feedback (M = 2.8), and effective course design (M = 2.8). EA1 students ranked all similar items lower than did students in non-EA classes. For these same items, the mean scores for non-EA classes fell between 4.5 and 4.9 including the scores for the same non-EA course taught online during the 5-week summer session. The EA1 students’ mean scores were highest for items rating clear course objectives (M = 3.4), clear grading criteria (M =3.3), effective use of technology (M = 3.3), appropriate materials/resources (M = 3.2), effective student-to-student interactions (M = 3.2), and demonstrated instructor content knowledge (M = 3.2). The mean scores for these same items on non-EA student evaluations fell between 4.4 and 5.0 including the same course taught online during the 5-week summer session. While these scores were informative, I felt more in-depth information from which to identify potential changes to make in the course was needed.
Mean Scores for Course Evaluation Items
| Evaluation Item Related to | First EA Course; (N = 172-175) | Small-Scale Online; Five-Week; During Summer; Two Classes;(N = 12-13) | Compressed Video; Semester Long; During School Year; Four Classes; (N = 48-54) | Small-Scale Online; Semester-Long; During School Year; Five Classes; (N = 41-44) | Face-to-Face; Semester-Long; During School Year; Six Classes (N = 106-108) |
|---|---|---|---|---|---|
| Overall rating of professor | 2.5 | 4.8 | 4.6 | 4.6 | 4.9 |
| Demonstrated instructor content knowledge | 3.2 | 5.0 | 4.7 | 4.8 | 4.9 |
| Clear explanations/examples | 2.7 | 4.7 | 4.5 | 4.7 | 4.8 |
| Clear course objectives | 3.4 | 4.7 | 4.7 | 4.8 | 4.8 |
| Appropriate assignments | 2.8 | 4.7 | 4.7 | 4.7 | 4.8 |
| Clear grading criteria | 3.3 | 4.8 | 4.7 | 4.6 | 4.8 |
| Useful feedback | 2.8 | 4.8 | X | 4.9 | X |
| Effective course design | 2.8 | 4.7 | X | X | X |
| Appropriate materials resources | 3.2 | 4.8 | X | X | X |
| Effective use of technology | 3.3 | 4.4 | X | X | X |
| Effective student-to-student interactions | 3.2 | 4.6 | X | X | X |
| Evaluation Item Related to | First EA Course; (N = 172-175) | Small-Scale Online; Five-Week; During Summer; Two Classes;(N = 12-13) | Compressed Video; Semester Long; During School Year; Four Classes; (N = 48-54) | Small-Scale Online; Semester-Long; During School Year; Five Classes; (N = 41-44) | Face-to-Face; Semester-Long; During School Year; Six Classes (N = 106-108) |
|---|---|---|---|---|---|
| Overall rating of professor | 2.5 | 4.8 | 4.6 | 4.6 | 4.9 |
| Demonstrated instructor content knowledge | 3.2 | 5.0 | 4.7 | 4.8 | 4.9 |
| Clear explanations/examples | 2.7 | 4.7 | 4.5 | 4.7 | 4.8 |
| Clear course objectives | 3.4 | 4.7 | 4.7 | 4.8 | 4.8 |
| Appropriate assignments | 2.8 | 4.7 | 4.7 | 4.7 | 4.8 |
| Clear grading criteria | 3.3 | 4.8 | 4.7 | 4.6 | 4.8 |
| Useful feedback | 2.8 | 4.8 | X | 4.9 | X |
| Effective course design | 2.8 | 4.7 | X | X | X |
| Appropriate materials resources | 3.2 | 4.8 | X | X | X |
| Effective use of technology | 3.3 | 4.4 | X | X | X |
| Effective student-to-student interactions | 3.2 | 4.6 | X | X | X |
Note: 5 = strongly agree; 4 = agree; 3 = neutral; 2 = disagree; 1 = strongly disagree. X = Similar question not included on student course evaluation.
Before developing an action plan for EA course modifications, I conducted a qualitative analysis of the students’ comments to help me narrow the focus of factors that may have contributed to the low EA1 evaluation scores. Before examining EA1 evaluation comments, I reviewed non-EA evaluations for which I earned high overall instructor ratings (M ≥ 4.5). This included all courses except EA1. I was unable to distinguish comments by students enrolled in online courses from comments by students enrolled in face-to-face and compressed video classes because, for some semesters, comments reported by the university to me were not separated by course. Fiftyeight students (17%) enrolled in these classes contributed comments.
After reading comments from all non-EA evaluations, I determined that the unit of analysis would be a passage that included only one descriptive opinion. After separating the non-EA course comments into 184 statements, each describing one opinion, I grouped the statements into categories based on similar themes (Merriam, 2009). The categories of students’ views included those related to their personal and professional growth because of the class assignments, their time and effort commitment to the course, the instructor's professional content knowledge, the instructor's knowledge and ability to design and facilitate a course (pedagogy), the instructor's ability to provide a caring and supportive environment, and general descriptive comments about quality of instructor and course.
Using the same categories, I deductively analyzed comments submitted by EA1 students (244 passages from 104 students, or 31.6% of those enrolled in the EA course) to identify what the two sets of comments (EA1 evaluations and non-EA evaluations) had in common and to determine what was missing that might be related to students’ views of the course and instructor (Merriam, 2009). I identified the passages within each category as positive, negative, or neutral/recommendations lacking a positive or negative tone. For the non-EA courses, 97% of the comments were positive. Three categories received most of the positive comments with general descriptions about the quality of the course and instructor receiving the greatest number (37%), perceptions of the instructor's professional content knowledge and ability to design and facilitate the course receiving the next highest number of positive comments (24%), and perceptions of the instructor's ability to provide a caring and supportive learning environment receiving the third highest number of positive comments (22%). Table 3 summarizes the percent of comments that were positive, negative, and neutral or recommendations.
Percentages of Positive, Negative, Neutral/Recommendation Comments for Each Theme on Student Evaluations for Non-EA Courses and EA1
| Categories of Students’ Views | Non-EA Courses (184 passages from 58 students) | First EA Course (EA1) (244 passages from 104 students) | ||||
|---|---|---|---|---|---|---|
| Positive | Negative | Neutral/Recommendations | Positive | Negative | Neutral/Recommendations | |
| Their personal and professional growth because of the class assignments | 7% | 0% | 0% | 2% | 4% | 0% |
| Their time and effort commitment to the course | 0% | 1% | 0% | 0% | 18% | 1% |
| The instructor's professional content knowledge | 7% | 0% | 0% | 0% | 0% | 1% |
| The instructor's knowledge and ability to design and facilitate a course (pedagogy) | 24% | 0% | 1% | 5% | 57% | 6% |
| The instructor's ability to provide a caring and supportive environment | 22% | 0% | 0% | 0% | 6% | 0% |
| General descriptive comments about quality of instructor and course | 37% | 0% | 1% | 0% | 0% | 0% |
| Categories of Students’ Views | Non-EA Courses (184 passages from 58 students) | First EA Course (EA1) (244 passages from 104 students) | ||||
|---|---|---|---|---|---|---|
| Positive | Negative | Neutral/Recommendations | Positive | Negative | Neutral/Recommendations | |
| Their personal and professional growth because of the class assignments | 7% | 0% | 0% | 2% | 4% | 0% |
| Their time and effort commitment to the course | 0% | 1% | 0% | 0% | 18% | 1% |
| The instructor's professional content knowledge | 7% | 0% | 0% | 0% | 0% | 1% |
| The instructor's knowledge and ability to design and facilitate a course (pedagogy) | 24% | 0% | 1% | 5% | 57% | 6% |
| The instructor's ability to provide a caring and supportive environment | 22% | 0% | 0% | 0% | 6% | 0% |
| General descriptive comments about quality of instructor and course | 37% | 0% | 1% | 0% | 0% | 0% |
Next, I deductively analyzed the comments contributed by EA1 students applying the same categories. Seventeen comments (7%) were positive and fell into two categories: students’ views of the impact of course assignments on their professional and personal growth and views of the instructor's pedagogy. Eighty-five percent of the comments were negative with two categories receiving the highest proportion of negative comments: students’ perceptions of the instructor's knowledge and ability to design and facilitate a course (57%), and perceptions of their time and effort commitment to the course (18%). These two groups of students held different views for every category with the most contrast between views about the time and effort they needed to invest to successfully complete my courses (i.e., their workload for the classes), the instructor's abilities to provide a caring and supportive learning environment, and the instructor's knowledge and abilities related to designing and facilitating courses.
I contend that students’ levels of satisfaction with their graduate education courses are strongly linked to their basic needs of autonomy, relatedness, and competency being supported or thwarted during their class experiences. I also propose that two major features of the EA model that differed from the non-EA model inhibited these needs from being met: (a) the compressed course timeline during the school year, and (b) the nature and frequency of faculty interaction with students. Paired with this compressed timeline was the EA program marketing which emphasized a course design for working students leading very busy lives. EA1 students may have interpreted this advertisement to mean that their invested work and time required for success with the courses would be reduced from that required for 14-week or summer 5-week courses. This structural difference may have led, in part, to students’ dissatisfaction because they were unable to complete assignments and meet academic challenges to their level of competence, thus thwarting their need for competency. Student comments reflect this frustration with being unable to competently meet the time and effort requirements of the course. For example: This course was not designed for working teachers. The work load was unreasonably overwhelming. My teaching profession suffered because I normally work after school hours on lessons, grading, etc.
This impression was not reported by students enrolled in the 14-week class or the summer 5-week non-EA online classes. While these students made a few comments about the amount of work and effort required for their graduate courses, they paired these comments with positive feedback about the course such as: This course was excellent and wellplanned. It also required a tremendous amount of work, but the learning experience was fantastic.
The frequency and nature of student-faculty interactions also differed in the two sets of classes. Since the student enrollment for the EA courses technically has no limit and enrollment has been over 1,000 in some classes (facilitated by one professor), a major consideration with course design is the scalability. This scalability hinders faculty from interacting with students, so EA courses are structured such that faculty primarily interact with TAs who conduct the majority of the course communication with students. I argue that this lack of interaction, in conjunction with the nature of these interactions between students and me, contributed to their dissatisfaction because their needs for feeling valued, understood, and cared for were unmet during the course. In addition, the EA1 students’ feelings of relatedness may have been thwarted because the volume of work aligned with expectations of a three-credit hour graduate-level course assigned during a compressed 5-week schedule contradicted the marketing message to working students who lead busy lives. Student comments demonstrate their views that the social context fell short of meeting their needs for relatedness. For example: Part of the requirements to be admitted to the program was that applicants had to be currently teaching. Did [professor] forget this or was it that she cared not? On the contrary, I propose that the frequency and nature of the interactions during non-EA courses met students’ need for relatedness. Evidence of this support can be found in a student's comment: Very … helpful.
Great with understanding that we all work and needed some flexibility. She cared that we learned.
Evidence for meeting or thwarting EA1 students’ need to feel they are the source of their own actions and value their own actions (autonomy) was less obvious when reviewing the comments. Based on their comments, students’ dissatisfaction with the course may have been influenced by feeling they lacked power with making choices about when they completed assignments, how much time they spent on assignments, and how they spent their nonclass time because of the compressed timeline during the school year. The students may have felt they had no voice because of the limited communication with me. The following student comment reflects this dissatisfaction with their lack of control of their work during the course and during time outside of the course: I like many other teachers had parent/teacher and CEA conferences during this week; we were required by our district to stay until 7:30 two different nights. The following two weeks we had Benchmark testing and SAT-10 testing. On the contrary, the traditional timelines and direct lines of frequent and varied communication promoted feelings of autonomy for the non-EA student. These students’ had choices regarding when to complete assignments, voices because of their interactions with the professor, and power because of the value they perceived from assignments they felt prepared them for future success. These feelings are reflected in comments such as: The materials used were informative and opened me up to new ideas I will use in the school environment.
My next step was to develop an action plan for making changes to the second EA course. Keeping in mind that Mills (2011) recognizes that action researchers can only make changes in what they have control over, I excluded making changes in the course lengths, much of the required coursework, and course time investment because course lengths are decided on by university administration rather than being under my discretion, and it is important to me that my courses preserve the academic integrity of three-credit hour graduate-level courses. Based on the literature about student satisfaction ratings of their instructors and courses (Best, 2000; Chang & Smith, 2008; Endres, Chowdhury, Frye, & Hurtubis, 2009; Espasa & Meneses, 2010; Frick, Chadha, Watson, Wang, & Green, 2009; Heiman, 2008; House, 2006; Marks, Sibley, & Arbaugh, 2005; Walker & Kelly, 2007), self-determination theory (Deci & Ryan, 2002; Filak & Sheldon, 2003; Jang et al., 2009; Ryan & Niemiec, 2009; Niemiec & Ryan, 2009) and findings from my collected data, I decided to make changes with student-faculty communication during my next EA class, and slightly reduce the number of course assignments.
While Filak and Sheldon (2003) determined that when a social environment meets all three needs for competence, autonomy, and relatedness, students are most satisfied with their instructors, I felt I could influence student satisfaction the most by concentrating on meeting one of these needs, relatedness, by increasing the frequency of my interactions with students and changing the nature of these interactions to make them more supportive. In addition, by changing the quality of the interactions to include personal interactions, rationales for assignments, reasons I valued student learning from assignments, and my understanding of the immense workload, I hoped that students would value their work and feel that they had a louder voice related to the course (i.e., feel more autonomous). Selected changes to faculty-student interactions included sending precourse welcome letters, including my professional background, the course syllabus, textbook names, book sources, and detailed directions for all assignments; communicating more frequently (at least weekly) via e-mails, including assignment descriptions and rationales, solicitation for assignment feedback, reassurances that I understood the challenges of the compressed schedule, appreciation for students’ commitment to quality work, my commitment to maintaining the integrity of the course, invitations to respond via e-mail to engage in discussions with any student who was interested in continuing e-mail discussions; responding to students’ e-mails promptly 7 days a week until at least 10:00 P.M. since students completed much of their coursework on weekends and at night; responding once a week to students’ questions collected by the TA; and when communicating, using a more personal and conversational tone, students’ names, including additional background and rationales for assignments or decisions I had made, and encouraging students to continue the conversations with me until they felt their questions or concerns were addressed to their satisfaction.
With regard to making changes to increase students’ feelings of competence, which Filak and Sheldon (2003) determined had the greatest impact on course approval, since I was not ready to significantly reduce the workload, I decided to postpone considering this dimension of self-determination theory. However, because of the concerns about the EA course discussion board assignments (which were not mentioned in non-EA class evaluations including those for 5-week online summer classes), I decided to eliminate most of the discussion board discussions until I could conduct a separate study to guide me in what changes to make.
With Stage 1 nearly complete, I was anxious to enact changes in the second EA class. Unfortunately, I was assigned a different EA course (EA2), but rather than set the action research aside, I moved forward with the study because EA2 was similar to EA1 in that both courses were structured using the EA model except for modifications to EA2 resulting from the action research; enrolled educators interested in earning master's degrees in education focusing on classroom and/or school leadership; included assignments focused on complex ideas about curriculum, curriculum change, and curriculum processes connected to students’ prior knowledge, skills, and understandings, with assignments targeting recommendations for reform; were designed to retain the integrity of a three credit hour graduate-level course regarding the number and complexity of assignments; were supported by the same distance education platform; had similar student enrollment numbers (EA1 = 329 students; EA2 = 338 students); and were supported by four TAs who worked full time as K-12 educators and held at least master's degrees in relevant fields. However, the students enrolled in EA1 (Curriculum Management) were working toward their National Board certification while those enrolled in EA2 (Secondary Curriculum) were meeting course requirements for their school administration license. I had previously taught the EA2 course online during 14-week (school year) and 5-week (summer) sessions.
Stage 2
The purpose of Stage 2 was to gather data from student evaluations of the second large-scale, compressed timeline online class to compare with previous classes to determine if the modifications contributed to greater course satisfaction.
Method
At the conclusion of EA2, I reviewed the course evaluation scores for items similar to all other courses. Table 4 includes the mean scores for end-of-course evaluation items for EA courses and the mean of means of non-EA course evaluation items. I also analyzed students’ comments on course evaluations with the same six categories employed when analyzing other evaluations (students’ personal and professional growth because of the class assignments, their time and effort commitment to the course, the instructor's professional content knowledge, the instructor's knowledge and ability to design and facilitate a course, the instructor's ability to provide a caring and supportive environment, and general descriptive comments about quality of instructor and course).
Mean Scores for EA Student Course Evaluation Questions and Mean of Mean Scores for all Other Evaluations
| Evaluation Item Related to | Course and Mean of Item Scores | Change in Mean of EA Scores | ||
|---|---|---|---|---|
| Mean of Mean Scores for all Courses Except EA Courses (N=207-219) | Mean Scores for First EA Course; Stage 1 of Research (N=172-175) | Mean Scores for Second EA Course Stage 2 of Research (N=183) | ||
| Overall rating of professor | 4.7 | 2.5 | 4.2 | +1.7 |
| Demonstrated instructor content knowledge | 4.7 | 3.2 | 4.3 | +1.1 |
| Clear explanations/examples | 4.7 | 2.7 | 4.3 | +1.6 |
| Clear course objectives | 4.8 | 3.4 | 4.6 | +1.2 |
| Appropriate assignments | 4.7 | 2.8 | 4.3 | +1.5 |
| Clear grading criteria | 4.7 | 3.3 | 4.5 | +1.2 |
| Useful feedback | 4.9 | 2.8 | 3.8 | +1.0 |
| Effective course design | 4.7 | 2.8 | 4.2 | +1.4 |
| Appropriate materials/resources | 4.8 | 3.2 | 4.1 | +0.9 |
| Effective use of technology | 4.4 | 3.3 | 4.1 | +0.8 |
| Effective student-to-student interactions | 4.6 | 3.2 | 3.2 | 0 |
| Evaluation Item Related to | Course and Mean of Item Scores | Change in Mean of EA Scores | ||
|---|---|---|---|---|
| Mean of Mean Scores for all Courses Except EA Courses (N=207-219) | Mean Scores for First EA Course; Stage 1 of Research (N=172-175) | Mean Scores for Second EA Course Stage 2 of Research (N=183) | ||
| Overall rating of professor | 4.7 | 2.5 | 4.2 | +1.7 |
| Demonstrated instructor content knowledge | 4.7 | 3.2 | 4.3 | +1.1 |
| Clear explanations/examples | 4.7 | 2.7 | 4.3 | +1.6 |
| Clear course objectives | 4.8 | 3.4 | 4.6 | +1.2 |
| Appropriate assignments | 4.7 | 2.8 | 4.3 | +1.5 |
| Clear grading criteria | 4.7 | 3.3 | 4.5 | +1.2 |
| Useful feedback | 4.9 | 2.8 | 3.8 | +1.0 |
| Effective course design | 4.7 | 2.8 | 4.2 | +1.4 |
| Appropriate materials/resources | 4.8 | 3.2 | 4.1 | +0.9 |
| Effective use of technology | 4.4 | 3.3 | 4.1 | +0.8 |
| Effective student-to-student interactions | 4.6 | 3.2 | 3.2 | 0 |
Note: 5 = strongly agree, 4 = agree, 3 = neutral, 2 = disagree, 1 = strongly disagree.
Results and Discussion
The EA2 evaluation was completed by 183 of the 338 (54%) students enrolled in the course. The highest mean scores were documented for items related to clear course objectives (M = 4.6), and clear grading criteria (M = 4.5). The lowest earned score for the items related to useful feedback (M = 3.8) and effective student-to-student interactions (M = 3.2). Neither of the low scores was unexpected since the EA model requires TAs, rather than faculty, to complete all grading, and TAs typically complete an assignment rubric for each student, providing very little other feedback. Since I had eliminated most discussions among students, it was expected that the item related to the usefulness of student interactions would be low. The remaining EA2 items earned mean scores from 4.1 to 4.3. Except for the item regarding the effectiveness of student-to-student interaction which stayed the same, students enrolled in EA2 ranked all end-of-course items higher than items similar to those for the EA1 evaluation with an increase in item mean score from 0.8 to 1.7 mean points.
The item score reflecting the overall faculty rating increased from 2.5 to 4.2. EA2 mean scores were within a range of 0.3 to 0.7 mean points of the non-EA courses’ mean of means (not including the scores for the items about student interactions and feedback on assignments). Comments from 42 students on the end-of-course evaluation provided additional insight into their perceptions of the quality of the EA2 course and instructor. Of the 131 comments, 51% were positive, 34% were negative, and 15% were neutral or recommendations. The greatest number of positive comments addressed views of the instructor's knowledge and ability to design and facilitate the class (23%), with the next highest percent earned by the item seeking feedback about the instructor's ability to provide a caring and supportive environment (14%), followed by 8% describing general impressions of the course and instructor. The highest number of negative perceptions (21%) addressed the instructor's ability to design and facilitate the course, with 10% of the comments relating to the time and effort required for the course. While the percent of positive comments for EA2 (51%) is far below that for non-EA courses (97%), there was a large improvement when compared to EA1 (7%). Notably, the same two categories earned the most positive comments on the EA2 and non-EA courses: the instructor's ability and knowledge to design and facilitate a course and the instructor's ability to provide a caring and supportive environment. Regarding negative comments, the percent was much higher for EA2 (34%) compared to non-EA course (1%), and the percent of negative comments for EA2 dropped by more than half compared to the EA1 (85%).
Summary and Concluding Discussion
University faculty continually seek ways to improve their courses in order to promote student academic success, increase student interest in coursework, and raise course evaluation scores. When student evaluation scores for my first large-scale, compressed timeline online course were much lower than scores for previous courses, including small-scale online classes I directly taught, I was highly motivated to conduct teacher action research to gain insight into what I could do to raise student satisfaction with my second large-scale, compressed timeline online course.
A close examination of the structures of the two online course models indicates that students in the non-EA courses had more time to complete coursework, interacted more with the faculty, and shared the course with many fewer students. Course evaluations from students enrolled in the non-EA courses and EA1 reveal divergent views particularly regarding the time and effort needed to successfully complete the course, the instructor's abilities to provide a caring and supportive learning environment, and the instructor's knowledge and abilities related to designing and facilitating courses. Based on the contrast between the two online models, student evaluation scores and comments, and a review of the literature about online teaching and self-determination theory, I determined that the compressed course timeline and the nature and quantity of faculty interactions with students were the major factors that contributed to low student evaluations for EA1. Based on these comparisons, I decided to slightly reduce the number of virtual discussion assignments, increase the frequency of my interactions with the students, and change the nature of my communications with students.
The scores and student comments from evaluations for my second large-scale, compressed timeline online course showed great improvement over evaluations for EA1. I propose that increasing the quantity of the student-faculty interactions and altering the nature of those interactions promoted students’ satisfaction with the course because these changes enhanced students’ feelings of relatedness. In other words, they felt more connected to others and cared for by others because of the changes in faculty-student interactions. In addition, I contend that the reduction in discussion board assignments contributed to students’ feeling of competence, as evidenced by the increase in the scores for the evaluation items related to the instructor providing appropriate assignments and an effective course design. This is also evidenced from the increased percentage of comments about the instructor's knowledge and ability to design and facilitate the class. To a lesser extent, these changes increased students’ feelings of autonomy, of having a voice and control over the quality of their work.
This connection between online course satisfaction and student-faculty interactions has strong support in the literature. The findings of this study align with research of large- and small-scale online language classes from which Russell and Curtis (2013) concluded that the reduced quality and quantity of faculty-student interactions in large online classes may be related to students’ dissatisfaction with the course; Walker and Kelly's (2007) study which determined that many students enrolled in online courses expressed dissatisfaction with the quantity and quality of interactions with faculty; Endres et al.’s (2009) study in which they determined that faculty practices, including interactions with students, correlated with student course satisfaction; and Chang and Smith's (2008) findings which noted a significant and positive relationship between student-faculty interactions and student course satisfaction.
The purpose of teacher action research is for faculty to improve their teaching and student learning by systematically investigating practices, making changes based on data collection and analysis, and investigating the resulting changes in practice. Within the context of practical action research as applied to a study of student course evaluations, there were limitations to my study. One limitation was that I studied two different groups of students rather than applying an intervention to one group. Unfortunately, the 5-week, compressed timeline of the courses did not permit a more in-depth study investigating the results of an intervention part way through one course. A second limitation to the findings is that I studied students who were enrolled in two different courses rather than students enrolled in the same course. However, students in both classes were very similar in that they were practicing teachers seeking master's degrees in education programs with overlapping curricula and from the same department at the same university. Because I was not scheduled to teach the EA1 course immediately after teaching it the first time, I chose to apply the interventions to the next course I taught rather than delay investigating how to improve my large-scale online course. I felt a sense of immediacy to determine how to improve my courses to effect student satisfaction because delaying the study would have impacted thousands of students enrolled in courses before I taught the EA1 course the next year. Finally, typical of studies involving student course evaluations, the investigation is limited by the number of students returning their course evaluations.
Since the primary purpose of teacher action research is for faculty to learn more about their teaching and student learning in their classrooms, I argue that these limitations do not detract from what I have learned about my students’ satisfaction with my course and how I can improve my online courses. While it is not the intention for teacher action research results to be generalized across large populations, the findings are also very informative for other university faculty teaching online courses.
Despite the limitations of teacher action research, my study contributes to the evolving body of literature that describes the results of interventions intended to promote student satisfaction with online courses. In addition, the study contributes to the nascent body of literature about large-scale, compressed timeline online courses. Equally important, this study expands the research literature that applies self-determination theory to understanding student course satisfaction and changes to improve students’ satisfaction with online courses.
As a result of my study, I am confident about changes I will make in my next large-scale, compressed timeline online course to promote students’ feelings of relatedness, competency, and autonomy that result in improved student course satisfaction. I strongly urge college faculty and administrators who design large-scale, compressed timeline online courses to include frequent opportunities for supportive student-faculty interactions. Leveraging new communication technologies can support this communication; however, even with leveraging new technologies, building in frequent and supportive interactions between faculty and students enrolled in online courses is not practical with one faculty member and thousands of students. Second, in order to maintain the integrity of graduate-level courses that include complex assignments linking theory with practice with opportunities for student-student discussions, I urge university administrators to avoid a 5-week course length for online classes.

