Online education has become more and more prevalent among universities all over the world, especially since the COVID-19 pandemic. An important question to ask is whether this non-traditional teaching modality is effective. The current study is unique in its focus on a department instead of a whole university or a single class or two. We compared student performance in psychology classes in three teaching modalities: in-person, hybrid and online courses between 2015 and 2019. Stringent inclusion criteria were used to reduce the large dataset. Results from statistical analyses are presented and discussed to answer the four research questions. Overall, we find significant differences among the three teaching modes, with hybrid outperforming in person and online when measuring students’ performance indicated by the course grades. However, this impact is more nuanced depending on the course level. Upper-level courses (3,000–4,000 level) benefit more from in-person and hybrid mode than online, whereas the introductory course (1,000 level) benefits more from hybrid mode than in person or online. The most significant difference is in the 2,000-level courses, primarily the research methods course, where online students’ course grades are notably trailing behind in-person or hybrid courses. Further, consistent with findings from other researchers, students’ previous grade point average has the strongest association with their course grades in all teaching modalities. These findings have important implications for different academic departments in that they may want to consider these results when scheduling classes to serve student needs best and achieve the best student success outcome.
Introduction
In the last decades or so, online education has spread widely among universities and colleges in the United States of America and other parts of the world. Described as a revolutionary innovation in educational technology, online education offers students the opportunity and flexibility to navigate their work and life while pursuing higher education, and it also helps educational institutions reduce their reliance on shrinking on-campus resources such as available classrooms. Before the COVID-19 pandemic hit, there were three primary teaching modalities in many universities: in person, hybrid and online. Whereas in-person learning is the traditional classroom teaching format, the online format is teaching and learning exclusively via the Internet, where students access learning materials, engage in course activities and submit their assignments online. The hybrid format consists mainly of 50% in person and 50% online delivery, although the percentages vary sometimes. Dating back to the 1990s, many academic programs have been offering entire or partial online degrees, and these online programs have become even more prevalent now since the COVID-19 pandemic. It will be easy to believe that online education is here to stay.
Since the outbreak of COVID-19, online learning has taken on an even more prominent role in education at all levels, resulting in many creative delivery methods. For example, different formats became commonly used, such as synchronous (students attend class sessions via Learning Management Systems such as D2L and Blackboard using Zoom, Teams, etc. at fixed class time), asynchronous (students access class at any time), hybrid (a combination of in-person and online sessions), Hyflex (online and in-person class taught simultaneously and students can choose which one to attend) or flipped (lectures delivered online and activities and discussions conducted in person). All these formats played an essential role in maintaining teaching and learning during the COVID-19 pandemic.
Many educators and researchers have sought to answer the fundamental question of whether the online format is an effective teaching and learning method. A comprehensive meta-analysis published by the Department of Education (<xref ref-type="bibr" rid="ref003">Means, Toyama, Murphy, & Bakia, 2009</xref>) found that, overall, students in online classes performed better than students in in-person classes to a moderate effect. Although their analysis was based on K-12 students and not college students, the findings support that the online format is an effective teaching method. Nguyen’s (2015) thorough review of the literature on outcomes (positive, unfavorable, mixed and no difference) of online education concluded that there was robust evidence to support the notion that online modality provides at least the same level of effectiveness as the in-person format. This conclusion is also supported by similar findings from Biel and Brame’s review (2016). Another comprehensive meta-analysis (Bernard et al., 2004) examined 232 university student achievement studies between distance education and traditional classroom instructions. The results showed that, overall, the online teaching modality is an effective teaching tool, although the variability among studies is large, with many studies showing better online outcomes than in-person format, whereas other studies show the opposite. These findings suggest that the success and effectiveness of online teaching and learning depend on various situations and contexts. Many factors contribute to the effectiveness and success of online teaching and learning. For example, <xref ref-type="bibr" rid="ref006">Sun and Chen , 2016</xref> identified several aspects of effectiveness due to faculty factors such as course design, instructor training, technology use, course materials and building learning community as contributing factors. Darius, Gundabattini, and <xref ref-type="bibr" rid="ref009">Solomon, 2021</xref> identified student-reported positive factors related to course delivery, such as animations, instructor video lectures, online multiple-choice quizzes and interaction with faculty. <xref ref-type="bibr" rid="ref012">Stern, 2004</xref> identified important student factors such as time management, motivation, organizational skills and awareness of learning styles as contributing variables.
Similarly, Kedia & <xref ref-type="bibr" rid="ref015">Mishra, 2023</xref> also identified student-related factors such as instructor-student interaction, constructive use of social media, family support and more technical support. Most educators agree that online teaching mode is an effective learning tool overall, though most also acknowledge that online education carries with it challenges for students. For example, Baum & <xref ref-type="bibr" rid="ref018">McPherson, 2019</xref> have expressed well-founded concerns about the challenges faced by students of lower socioeconomic status backgrounds. Most researchers agree that the importance of online education warrants ongoing attention, discussion and research.
To answer whether online modalities are effective teaching and learning methods, researchers have compared the effectiveness outcomes of traditional in-person classes with online and hybrid classes. They frequently examine the effectiveness through student performance in the class as indicated by their final course grades. In general, there are more studies that compare in person with online courses in the literature review than those that also include a hybrid format in the research. However, most studies have included hybrid mode in the analyses as well. In the following section, we attempt to review, summarize and compare the literature on college students’ academic performance in in-person, hybrid and online modalities.
Literature review
In-person and online performance review
Many research educators have compared students’ performance in online vs in-person courses. In most studies, the course grade indicates academic performance, although some use quiz scores, project grades and final exams. Earlier meta-analytical research showed that online students slightly outperformed in-person students in K-12 education (Means, Toyama, Murphy, & Bakia, 2009). Online college students often outperformed classroom students (Bernard et al., 2004). A review of recent in-person and online student performance studies suggests no statistical difference between online and in-person course grades. A few studies reported higher in-person course grades than online (Van Wart, Ni, Rose, McWeeney, & Worrell, 2019).
Research has examined grades at an institutional level as well as at a variety of different academic disciplines. Cavanaugh and Jacquemin , 2015 used a large dataset of over 5,000 courses over 10 academic terms from a large university, and their analyses indicated that, overall, the differences in students’ grades across the courses were not significant between the two teaching modalities. Similarly, studies in different disciplines, such as sciences and social sciences, found the same results. For example, comparisons in environmental science courses (Paul & Jefferson, 2019), statistics courses (Summers, Waigandt, & Whittaker, 2005), introductory biostatistics courses (Hoffman & Elmi, 2021), technology courses (Kelly, 2009), research methods courses (Ni, 2021) and psychology courses (Kemp & Grieve, 2014; Lupercio, 2020; Waschull, 2001) all showed similar findings.
Studies that found significant grade differences between the two teaching modalities usually report lower online course grades than in-person courses. A large-scale study (Xu & Jaggars, 2014) examined nearly 500,000 courses in community and technical colleges in Washington State and found that online students in all disciplines achieved lower academic performance measured by course grades than in-person students. Similarly, another study from community colleges and four-year institutions in Florida showed that online course grades were significantly lower than in-person courses (Amparo & Smith, 2018). These results were further supported by other studies in psychology courses (Helms, 2014; Zhan & Moodie, 2021), an economics course (Arias, Swinton, & Anderson, 2018), writing courses (Sapp & Simon, 2005) and an algebra course (Amro, Mundy, & Kupczynski, 2015).
In sum, more recent available evidence indicates that online students perform at a comparable level to in-person students or at a lower level, which is measured by course grades.
In-person, hybrid and online performance review. As Means, Toyama, Murphy, and Bakia, 2013 noted, the blended learning mode is becoming more prevalent in higher education. Therefore, combining hybrid teaching to compare student achievement between online and in-person instruction is essential. A literature review suggests plenty of evidence indicating that the hybrid teaching mode is an effective way of teaching. For example, Means, Toyama, Murphy, and Bakia, 2013 concluded that the hybrid courses in their meta-analysis outperformed in-person modes, whereas the fully online and in-person outcomes were not significantly different. Similarly, Bernard, Borokhovski, Schmid, Tamim, and Abrami, 2014 also reported that students in hybrid teaching modality significantly outperformed those in in-person classes.
Along the same line, a more recent study (Moodie, 2021) reported significant differences in student performance among the three modes. The researcher did an extensive statistical comparison of student performance in all business courses at a comprehensive university from 2015 to 2019. Course grades were used as indicators of student performance outcomes. The results showed that grades in hybrid courses were significantly higher than in in-person or online sections. The same researcher also compared the data from the entire university and found that students in hybrid sections obtained higher final course grades than those in in-person or online sections (2022). Similarly, Zhan and Moodie (2021b) compared student performance in an introductory psychology course and found that student course grades were significantly higher in the hybrid mode than in in-person or online formats, supporting Moodie’s (2021) findings.
On the other hand, some researchers report no significant differences in student performance among the three teaching modalities. For example, Larson and Sund , 2009 compared students’ academic performance in an introductory management information system course taught by the same instructor in all three modes. Students’ exams and final grades were used as indicators of academic performance. Their results showed no significant differences in student performance among the three delivery modalities. Similarly, Lovern, 2010 examined student performance in a teacher education program, using midterm, final exam, project grade and final course grade as measures of academic performance. The results showed no significant differences in the measures among the three teaching modes. Price, Arthur, and Pauli, 2016 also reported no correlation between teaching modes and student performance assessed by their final course grades.
However, we also found a study examining students’ outcomes in a developmental psychology course using different teaching modalities (Zhan & Moodie, 2021). The results showed that students in in-person sections actually obtained significantly higher course grades than those in online and hybrid sections.
In sum, the results on student academic performance in in-person, hybrid and online modalities suggest plenty of evidence that online teaching, especially in hybrid mode, is an effective instructional method. The overall effectiveness of online teaching varies depending on the contexts and situations related to students, instructors and disciplines.
Student satisfaction and preference review. Student satisfaction with and perceptions of a course may affect their attitudes and performance. Larson and Sund , 2009 compared students’ satisfaction levels and perceived learning effectiveness in an in-person, hybrid and online introductory management information systems course. The researchers found that the satisfaction level was similar in all three modalities, and students expressed comparable satisfaction with the course. However, students in hybrid and online classes reported increased critical thinking and higher motivation than the face-to-face students. Similarly, Price et al., 2016 also reported no significant difference in student satisfaction among teaching modalities for business courses. The no-difference finding was also supported by a case study (Goerke, 2018) that examined military school students’ satisfaction with mission accomplishment, course management, course instruction and course value. There were no differences in student satisfaction among the in-person, hybrid and online teaching formats.
Several studies explored students’ preferences for teaching modality. Kemp and Grieve , 2014 examined Australian undergraduate students’ preference for in-person or online courses. It was reported that overall, students had similar preferences, although they preferred to participate in classroom activities rather than online. The researchers further reported that this preference was not related to the student’s academic success as measured by grades. Likewise, Weldy, 2018 surveyed business students’ preferences for teaching formats. It was found that significantly more students (76%) preferred the in-person mode than either hybrid (12%) or online (12%). However, no analysis has been reported that links preference to performance.
In sum, available evidence suggests that, overall, students’ views on the three teaching modalities are comparable, and their satisfaction levels are similar across these three modes.
Student variables review
Overall, research on the relationship between student variables such as age, gender, ethnicity, learning styles and their academic outcomes in different teaching modes shows no conclusive evidence. Moodie’s (2022) research indicates that students of different demographics may benefit differently from hybrid classes than from in-person or online classes. Overall, students’ previous grade point average (GPA) has been shown consistently to be significantly correlated with their academic performance regardless of teaching modalities (Cavanaugh & Jacquemin, 2015; Moodie, 2021; Xu & Jaggers, 2014; Zhan & Moodie, 2021). Previous GPA is a reliable predictor of student learning outcomes in all classes. One variable not commonly included in the study is students’ previous experience with online learning. Although Weldy’s (2018) study did include prior exposure to online classes, an analysis report of the relationship between this factor and the learning outcomes needs to be made.
The current study
The current study explored the effectiveness of the teaching modality of psychology courses from 2015 to 2019 in a comprehensive university in the southeastern United States. The Department of Psychological Science has over 50 full-time and many part-time faculty, supporting over 2,500 majors and 800 minors. Previous studies examining the effectiveness of teaching modalities tended to be either large-scale, focusing on institutional and cross-disciplinary comparisons, or a single course or two, focusing on a particular discipline. Therefore, the results have been either generally sweeping or narrowly applied. The current study took an in-between approach, focusing on all the courses taught in a single discipline and setting stringent criteria for the courses to be included in the analyses. We also carefully considered course levels (lower vs upper) and student variables such as prior experience with online learning and previous GPA.
This study attempted to address the following four research questions:
Are there any significant differences in student performance in psychology courses among the three teaching modalities?
Do course levels significantly impact the effect of teaching modalities on student learning outcomes?
Does the impact of previous GPA influence student academic performance differently with different teaching formats?
What role does previous online learning experience play in current student performance in corresponding modalities?
Method
The dataset
The data were obtained from the university’s institutional record office. The data period ranges from the fall of 2015 to the fall of 2019. The analyses excluded data from the more recent semesters since the beginning of the COVID-19 pandemic due to the sudden changes in teaching modes, causing many unexpected challenges for faculty and students. Therefore, that period’s grades may differ from those of regular semesters.
The original data set contained a total of 54,453 course-student data points. Among those, 31,251 were lower-level courses (1,000–2,000 level), and 23,202 were upper-level courses (3,000–4,000 level).
Data reduction. To ensure data comparability and the integrity of the comparison, we carefully reviewed all relevant information and set perimeters for what courses to include and exclude. The dataset was reduced by removing courses that met the following criteria:
Courses that were individually based, such as directed studies or independent studies (these courses usually had a small number of students, ranging from one to several);
Courses that were internship and/or practicum-based (these courses were primarily site-based and students had hands-on experience at the place);
Courses that were taught during study abroad trips (these courses usually had small numbers and short duration);
Courses that had not been taught in all three modalities during the study period (these courses had yet to be offered in all three modalities) and
Courses taught in summer (these courses had variable duration ranging from ten days to eight weeks).
The reduced dataset resulted in 45,614 course-student data points. Among those, 19,873 were lower-level courses, and 25,740 were upper-level courses. Descriptions of the characteristics of the reduced dataset are presented in Tables 1 and 2.
Statistical analysis. Several types of statistical analyses were performed. First, the overall means and standard deviations for course grades in all three teaching modalities were calculated for all psychology courses. T-tests were conducted to examine whether any differences in various configurations of teaching modalities were statistically significant. Second, the means and standard deviations for all courses were calculated, breaking down by different levels. Again, t-tests were conducted to determine if any differences were statistically significant. Then ANOVA tests were performed to confirm the results among different categories and levels of the courses. A regression test was performed to determine whether the teaching modality by different course levels has predictive power for students’ class performance. Third, means and standard deviations for students’ previous GPAs were calculated, followed by a correlation analysis to determine the relationship between their GPA and current course grade. A regression analysis examined the predictive power of students’ previous GPA on their course grades. Last, means and standard deviations for the number of online and hybrid courses students have taken in the past five years were calculated for each course level by teaching modality. A correlation analysis was carried out to determine the relationship between prior online learning experiences and current course grades.
Results
The findings are presented in the order of research questions.
RQ1. Are there any significant differences in student performance in psychology courses among the three teaching modalities?
Table 3 shows that hybrid courses have the highest grade mean for in-person and online courses. T-tests of different combinations were conducted to determine whether these differences were significant and are presented in Table 4. It was confirmed that there are significant statistical differences in course grades by different configurations of the three modalities, all at a p-value smaller than 0.001. As shown in Table 4, the hybrid mode is associated with the highest mean grade, followed by the in-person format and then the online mode. The answer to the first research question is affirmative. For the psychology courses in this study, students’ course grades are significantly different in the three teaching modalities, with the hybrid modality linked to the highest academic performance, followed by the in-person and the online formats.
Characteristics of the variables in the reduced dataset Distance Learning
| Variable | Mean | Sd. |
|---|---|---|
| Course grade | 2.959 | 1.107 |
| Previous GPA | 3.070 | 0.615 |
| Age | 22.038 | 4.797 |
| Course head count | 58.883 | 41.336 |
| % Students passing | 89.8% | |
| Modalities | Percentage | |
| F2F | 53.2% | |
| Hybrid | 24.5% | |
| Online | 22.3% | |
| Student ethnicity | Percentage | |
| White | 52.4% | |
| African American | 25.8% | |
| Hispanic | 10.3% | |
| Asian | 3.5% | |
| International | 1.3% | |
| # Courses have taken | Mean | Sd. |
| Total term course count | 4.501 | 1.064 |
| F2F courses | 2.837 | 1.671 |
| Hybrid courses | 0.730 | 0.896 |
| Online courses | 0.934 | 1.231 |
| Total in last five years | 23.994 | 14.624 |
| F2F | 16.285 | 11.548 |
| Hybrid | 4.257 | 4.272 |
| Online | 3.453 | 4.669 |
RQ2. Do course levels significantly impact the effect of teaching modalities on student learning outcomes?
The students’ mean course grades were compared across lower-level and upper-level courses and modalities. Table 5 and Figure 1 present the results. Each course level (1,000s–4,000s) by modalities was examined to examine any effects of course levels further. Table 6 and Figure 2 present the comparison.
To examine whether these differences were statistically significant, t-tests were performed, and they are presented in Table 7. ANOVA and a regression test were also conducted to confirm the results further. Tables 8 and 9 present the results. All the results suggest that the course levels (lower vs. upper) had a significant relationship with course grades. As shown in Tables 7–9, the upper-level courses have significantly higher grades than the lower-level courses (p 0.001). However, the impact of teaching modalities differs depending on the course level.
There are significant differences in course grades between lower and upper-division courses by teaching modalities. In the lower-level courses overall, the pattern is the same as the overall findings: the mean course grades are highest in hybrid courses, followed by in-person and online courses. However, if we only examine the 1,000-level courses, in-person and online modalities are the same. The most significant difference in mean course grades by modalities appears in the 2,000-level courses, where the online format significantly lags behind in person or hybrid. In the upper-level courses, there is no significant difference between in-person and hybrid courses, but they are significantly higher than online. These results suggest that the answer to Research question 2 is affirmative. In this study, the course levels significantly impact the effect of teaching modalities on student learning outcomes as measured by course grades. However, the effect differed depending on the course level.
Characteristics of the variables by three teaching modalities
| Variable | F2F | Hybrid | Online | All |
|---|---|---|---|---|
| Course grade (mean) | 2.971 | 3.068 | 2.812 | 2.959 |
| Previous GPA (mean) | 3.078 | 3.096 | 3.023 | 3.070 |
| Age (mean) | 21.278 | 21.900 | 23.999 | 22.038 |
| Course head count | 66.760 | 55.188 | 44.186 | 58.883 |
| Students passing (%) | 90.1% | 91.9% | 86.8% | 89.8% |
| Student ethnicity | ||||
| White (%) | 52.0% | 51.8% | 53.8% | 52.4% |
| Black (%) | 25.3% | 26.9% | 25.6% | 25.8% |
| Hispanic (%) | 10.6% | 10.6% | 9.3% | 10.3% |
| Asian (%) | 3.8% | 3.2% | 3.1% | 3.5% |
| International (%) | 1.4% | 1.2% | 0.9% | 1.3% |
| Mixed (%) | 3.5% | 2.6% | 0.6% | 6.7% |
| # Courses have taken | ||||
| Total in last five years | 21.791 | 27.634 | 25.247 | 23.994 |
| F2F | 16.738 | 17.562 | 13.806 | 16.285 |
| Hybrid | 3.224 | 7.042 | 3.663 | 4.257 |
| Online | 1.830 | 3.030 | 7.779 | 3.453 |
Means and standard deviations for all psychology course grades in three modalities (N = 45,6l4)
| In-person | Hybrid | Online | All | |
|---|---|---|---|---|
| Mean | 2.971 | 3.068 | 2.812 | 2.959 |
| Sd. | 1.222 | 1.082 | 1.229 | 1.228 |
T tests for course grades in different modality configurations
| T-tests | In-person vs hybrid | In-person vs online | Hybrid vs online |
|---|---|---|---|
| Difference in means | -0.11229 | -0.22281 | -0.11052 |
| t stat | -9.07912 | -17.8805 | -7.90701 |
| P(T = t) one-tail | 5.88E-20 | 2.65E-71 | 1.38E-15 |
| t critical one-tail | 1.644916 | 1.644923 | 1.644925 |
| P(T < = t) two-tail | 1.18E-19 | 5.3E-71 | 2.76E-15 |
| t critical two-tail | 1.960062 | 1.960071 | 1.960075 |
Means and standard deviations for course grades by modalities and course levels
| Modality | Course Lower | Levels Upper | All |
|---|---|---|---|
| In-person | 2.838 (1.159) | 3.106 (1.005) | 2.971 (1.094) |
| Hybrid | 3.002 (1.128) | 3.094 (1.02) | 3.068 (1.053) |
| Online | 2.704 (1.125) | 2.896 (1.105) | 2.812 (1.178) |
| All | 2.834 (1.181) | 3.055 (1.036) | 2.959 (1.107) |
Mean course grades in lower- vs upper-level courses by modality Distance Learning
Mean course grades in lower- vs upper-level courses by modality Distance Learning
Means and standard deviations for course grades by modalities and four course levels
| Modality | Course 1,000 | 2,000 | Levels 3,000 | 4,000 | All |
|---|---|---|---|---|---|
| In-person | 2.804 (1.179) | 2.911 (1.112) | 3.076 (1.029) | 3.166 (0.951) | 2.971 (1.094) |
| Hybrid | 2.909 (1.217) | 3.038 (1.089) | 3.079 (1.032) | 3.137 (0.982) | 3.068 (1.053) |
| Online | 2.861(1.233) | 2.553 (1.253) | 2.897 (1.111) | 2.891 (1.092) | 2.812 (1.178) |
| All | 2.823 (1.193) | 2.761 (1.164) | 3.038 (1.051) | 3.093 (1.001) | 2.959 (1.107) |
T-test results for modalities by course levels
| Course levels | Modalities In-person vs hybrid | In-person vs online | Hybrid vs online |
|---|---|---|---|
| Lower | |||
| Difference | -0.16319 | 0.135294 | 0.298485 |
| t stat | -7.22991 | 6.266853 | 10.85475 |
| P (T < = t) one-tail | 2.78E-13 | 1.95E-10 | 1.53E-27 |
| t critical one-tail | 1.645154 | 1.645061 | 1.645064 |
| P (T < = t) two-tail | 5.55E-13 | 3.89E-10 | 3.06E-27 |
| t critical two-tail | 1.960432 | 1.960287 | 1.960291 |
| Upper | |||
| Difference | 0.011698 | 0.209758 | 0.198059 |
| t stat | 0.799065 | 12.18439 | 10.70371 |
| P (T < = t) one-tail | 0.212132 | 3.2E-34 | 6.48E-27 |
| t critical one-tail | 1.644944 | 1.645 | 1.644983 |
| P (T < = t) two-tail | 0.424264 | 6.4 E -34 | 1.3E-26 |
| t critical two-tail | 1.960104 | 1.960192 | 1.960165 |
ANOVA results for modalities and course levels
| ANOVA for course levels Source of variation | SS | df | MS | F | p-value | F crit |
|---|---|---|---|---|---|---|
| Between groups | 245139.1 | 4 | 61284.78 | 147437.3 | 0 | 2.371971 |
| Within groups | 94799.06 | 228065 | 0.415667 | |||
| Total | 339938.2 | 228069 |
Regression results for modalities and course levels
| Regression statistics for course levels | |
|---|---|
| Multiple R | 0.123252 |
| R square | 0.015191 |
| Adjusted R squared | 0.015104 |
| Standard error | 1.098414 |
| Observations | 45,614 |
RQ3. Does the impact of previous GPA influence student academic performance differently depending on teaching formats?
First, the means and standard variations of students’ previous GPAs were calculated by four course levels for each teaching modality. The results are presented in Table 10. Overall, students in hybrid courses have higher previous GPAs than in-person or online courses, although these differences are not statistically significant. Across the four course levels, however, there are significant differences in students’ previous GPA (F = 64.42 AND p < 0.01). T-tests show that the previous GPA in 4,000-level courses is significantly higher than the previous GPA in the 2,000-level courses, but there are no significant differences for 1,000-level or 3,000-level courses.
Then, a correlational and regression analysis was performed to examine whether and how students’ previous GPA affected course grades differently. This resulted in 39,557 course-student data sets. Table 11 shows the results of correlations, and Table 12 shows the results from the regression test.
Previous GPA means and standard variations for different modalities by course level
| Modality | Course 1,000 | 2,000 | Levels 3,000 | 4,000 | All |
|---|---|---|---|---|---|
| In-person | 3.017 (0.724) | 3.068 (0.633) | 3.091 (0.595) | 3.145 (0.519) | 3.078 (0.626) |
| Hybrid | 3.034 (0.673) | 3.064 (0.664) | 3. l (0.568) | 3.134 (0.522) | 3.096 (0.586) |
| Online | 2.964 (0.737) | 2.945 (0.701) | 3.062 (0.555) | 3.078 (0.522) | 3.023 (0.62) |
| AH | 3.006 (0.724) | 2.992 (0.662) | 3.087 (0.578) | 3.126 (0.522) | 3.070 (0.615) |
Significant correlations (p < 0.01) exist between students’ previous GPAs and their course grades for all modalities, indicating that a student’s previous GPA is positively related to their course grade regardless of teaching modality.
The regression results indicate that while the previous GPA plays a significant part in predicting students’ course grades, the adjusted R-squared differs for the three modalities, with hybrid being more robust than in person or online.
RQ4. What role does previous online learning experience play in current student performance in corresponding modalities?
Descriptive information was first obtained to answer this question. Means and standard deviations of several previous online and hybrid courses in the last five years were calculated for all courses by course levels. The hybrid courses were included because they also offered an online learning experience. The result confirmed the expectation that students in lower-level courses had less online learning experience than those in upper-level courses. Students in 4,000-level courses would have had significantly more online learning experience (p < 0.01). An interesting observation was that students in 2,000 and 3,000-level courses did not differ much from each other in their prior online learning experience. Both had significantly less experience than students in 4,000-level courses (p < 0.01). Results are presented in Table 13 and Figure 3.
Correlations between previous GPA and course grade for different modalities
| Modality | Correlations |
|---|---|
| All | 0.3991975 |
| F2F | 0.48339033 |
| Hybrid | 0.51485425 |
| Online | 0.48497409 |
Regression results for previous GPA and course grade in different modalities
| Modality | Adjusted R squared | Coefficients |
|---|---|---|
| All | 0.1593302 | 0.9814897 |
| F2F | 0.23362801 | 0.83700337 |
| Hybrid | 0.26500264 | 0.9150023 |
| Online | 0.23511776 | 0.9134271 |
Means and standard deviation for prior online and hybrid number by course levels
| Level | Hybrid Mean | Hybrid Sd | Online Mean | Online Sd |
|---|---|---|---|---|
| 1,000 | 1.79972 | 2.300931 | 1.392134 | 2.614613 |
| 2,000 | 3.53921 | 3.451257 | 3.009708 | 4.002424 |
| 3,000 | 4.882464 | 4.342952 | 4.165146 | 5.098271 |
| 4,000 | 6.845902 | 5.564105 | 4.883847 | 5.664834 |
| All | 4.100425 | 4.189486 | 3.412912 | 4.679444 |
Then we examined the number of online and hybrid courses students took in those two modalities during the study period, excluding in-person courses. Students in each modality appeared more likely to have taken classes in the same modality than a different one. For example, students in hybrid classes were more likely to report that they had taken more hybrid courses than online courses. The same was true with online courses. Those numbers are presented in Table 14.
Next, statistical analysis was conducted to examine whether and how previous online learning experiences have impacted the current course grades in corresponding teaching modalities. Correlation analysis shows no significant relationship between students’ course grades and their previous online and hybrid course experience. The results are presented in Table 15.
Discussion
We focused on a single department, psychological science and selected courses that met the inclusion criteria in the analyses. Through statistical analyses, we addressed and answered our four research questions.
| Course modality | Course level | Online mean# | Online Sd | Hybrid mean# | Hybrid Sd |
|---|---|---|---|---|---|
| Hybrid | 1,000 | 1.934871 | 2.59556 | 3.902655 | 2.249014 |
| 2,000 | 2.624464 | 3.174921 | 5.671882 | 3.360465 | |
| 3,000 | 4.132894 | 4.21848 | 7.17724 | 4.330567 | |
| 4,000 | 5.627946 | 4.876483 | 9.194344 | 4.91829 | |
| Online | 1,000 | 4.650384 | 3.908035 | 0.987832 | 1.826005 |
| 2,000 | 6.340773 | 5.105716 | 1.911343 | 2.693141 | |
| 3,000 | 9.158429 | 6.432409 | 3.291811 | 3.735718 | |
| 4,000 | 10.47419 | 6.779114 | 4.069409 | 4.047406 |
Correlations between previous online and hybrid course numbers and current corresponding course grades by course levels
| Level and modality | Previous hybrid # | Previous online # |
|---|---|---|
| 1,000 hybrid | -0.0053 | 0.0024 |
| 1,000 online | 0.0388 | -0.0768 |
| 2,000 hybrid | 0.0262 | -0.0738 |
| 2,000 online | 0.0797 | -0.2662 |
| 3,000 hybrid | -0.0044 | -0.0624 |
| 3,000 online | -0.0437 | -0.0642 |
| 4,000 hybrid | -0.0671 | -0.0542 |
| 4,000 online | 0.0177 | -0.0335 |
First, the findings indicate significant statistical differences in course grades among the three modalities, with hybrid outperforming in-person and online courses and in-person outperforming online courses. These findings support the Department of Education’s assertion that hybrid teaching may be more effective than in person or online (Bernstein, 2021). Our result is also consistent with the previous research that showed hybrid teaching is associated with better academic performance, as indicated by course grades in all courses in a comprehensive university setting (Moodie, 2021). The current findings suggest that the hybrid format is an effective teaching method in psychology courses, sometimes even more effective than other formats. However, a literature review still shows mixed and inclusive evidence because the contrasting findings were reported by other researchers in the past (Larson & Sund, 2009; Price et al., 2016). One possible explanation could be that the effectiveness of hybrid teaching may depend on academic disciplines. The previous studies that found no significant differences among teaching modalities did not examine psychology courses specifically. These discrepancies exist due to the nature of different academic subjects. Based on the current findings, hybrid teaching is an effective method and should be encouraged among faculty as much as possible. Hybrid courses should be offered as often as possible. It may mean that the universities would need to provide more technical support to those faculty who need training in teaching hybrid classes, but it is worth the effort in terms of student performance.
Our second research question asks whether the course level would impact the effect of teaching modes on students’ performance. The results indicate that, overall, course level significantly impacts the effect of teaching modalities on students’ performance. However, it is not a linear pattern, and the results do not hold true for every course level. Several nuances are worth discussing here. First, the hybrid advantage is no longer present in the upper-level courses. Instead, in-person and hybrid modes are associated with similar student course grade outcomes. However, online courses still need to catch up. The implication may be helpful in planning and scheduling because the psychological science department may wish to schedule more in-person and hybrid upper-level courses than online courses to enhance maximum student performance outcomes. Second, in the 1,000-level course, i.e. the introduction to psychology course, while the hybrid modality still holds the lead, in-person and online modalities do not differ in the course grades. Third, the most significant differences in course grades by teaching modalities appear in the 2,000-level courses, with online courses trailing significantly behind. The context of the curriculum is relevant to the discussion here. The research method course, a 2,000-level course, is one of the core courses in the psychological science department and is required for all majors. Many sections of this course are offered each semester. It is generally considered a more challenging subject for students, especially those with a weak quantitative background. The findings suggest that online sections have the lowest course grades compared to in-person or hybrid formats. More problematic concepts involved in research methods are likely better explained and taught in face-to-face settings than online.
It may be worth revisiting how best to help students succeed in this course by offering more in-person and hybrid formats and reducing the online offerings. However, we need to find out if a synchronous online format would help because the online courses in the dataset were asynchronous. In a synchronous online class, students may still benefit from in-person interaction, albeit in a virtual setting.
Our third research question asks whether the impact of students’ previous GPA on their current course grades would differ due to differences in teaching modes. The findings suggest a strong correlation between students’ GPAs and course grades regardless of teaching modalities. Students with higher GPAs do better in all teaching modalities than those with lower GPAs. This result is consistent with previous research findings (Cavanaugh & Jacquemin, 2015; Moodie, 2021) that students’ previous GPA is a reliable predictor of their current class performance.
A variable that has yet to be included in previous studies by other researchers is students’ experience with online classes before taking the current course. We included this factor in our analyses but found no significant relationship between the number of prior online and hybrid courses and current performance outcomes in those classes. It is somewhat puzzling because it is reasonable to expect that students’ familiarity with the online learning format would benefit them in their performance in class. One possible explanation is that student success in online or hybrid classes is independent of prior online learning, and other factors impact their performance rather than familiarity with the online or hybrid format. This explanation seems plausible given that in our analysis, prior learning experience, as indicated by the number of courses taken regardless of modalities, has no bearing on the current course grades for any course levels or modalities.
The dataset for the current study may have come from a psychological science department, but it also has broader implications for other departments. Our findings add to the literature on teaching modality effectiveness in the context of student performance, as indicated by the course grades, showing that the hybrid teaching method is as effective as the in-person method and, in some cases, even more so. The findings show several factors associated with how different teaching modalities are effective when compared to one another. One factor is the course level, and the other is the type of the course. For example, if the course has more technical and complex content, in-person teaching might be more advantageous than online mode (Van Wart et al., 2019). Our findings support this position. When making decisions on course teaching modalities, department administrators should review these results and strategically plan the course offerings by the course level and the course type. Students in more challenging courses such as research methods and other upper-level courses may benefit more from in-person or hybrid settings than asynchronous online settings.
Conversely, the in-person format does not differ from online in students’ performance outcomes for the introductory course. If physical space is lacking, more online sections can be offered instead of an in-person format for teaching the introductory course without jeopardizing benefits to students’ performance outcomes. These implications are relevant and can be applied to other disciplines and departments as well.
Non-traditional teaching and learning methods are spreading rapidly in universities and colleges worldwide. They are here to stay for decades to come. Our challenge is ensuring these new formats are at least as effective, if not more, than the traditional in-person classroom teaching. The current study adds to the literature in this field and joins the others in seeking answers.
Limitations and future directions
First, we acknowledge that students’ academic performance has many aspects beyond the course grades. Only using the final grade is a simplistic way of measuring students’ achievement in the class, and it does not reflect the totality of their academic achievement. In future studies, we plan to include other indicators such as assignments, quizzes and project grades.
Second, students’ perceptions and preferences of teaching modalities should be considered. If students favor a particular teaching and learning mode, they may be more motivated to do well in the class than if they had no choice but to select the course modality. In future studies, we plan to add open-ended preference questions to link to course grades.
A related factor is students’ interest in the subject. It should be considered in future studies examining teaching modalities and student performance. Interest in the course would provide motivation to students to make their best effort and do well in the course.
Last, the instructor factor may be relevant to how effective a particular teaching method is. Different instructors have varied teaching experiences with modalities and have different strengths and weaknesses. In future studies, we plan to focus on the same instructors who teach all three modalities of the same course to control the variabilities of different instructional personnel factors.
