In the Republic of Korea, the COVID-19 pandemic coincided with the start of the 2020 academic year and saw emergency remote teaching (ERT) emerge as a way of maintaining educational continuity for millions of students. While ERT was new and unplanned at the time, the practice became sustained over the semesters that followed, marking a shift from ERT to sustained remote teaching (SRT). Questions remain, however, whether students’ experiences and perceptions with learning remotely would improve as a result of institutional preparedness and faculty experience. Given this, we investigated exchange students’, a unique group of students who are historically interested in having place-based residential education, experiences, and perceptions with SRT while attending college in Korea. We administered a survey to 140 (spring 2020), 93 (fall 2020), 141 (spring 2021), and 143 (fall 2021) exchange students where they rated their perceptions of teaching and learning processes, student support, and course structure with their SRT learning experiences. Independent-samples one-way ANOVAs comparing perceptions between Semester 1 and 2, Semester 2 and 3, Semester 3 and 4, and Semester 1 and 4 indicated several statistically significant mean score increases, though the scope and degree of the changes are ultimately minor improvements and interpreted as insignificant. Implications for SRT policy and future research are discussed.
Introduction
In early 2020, educational institutions worldwide suddenly transitioned courses to a remote or online learning format in response to the COVID-19 pandemic (Hodges et al., 2020). In the Republic of Korea (hereafter Korea), more than 10,000 cases occurred by the end of March 2020 (Ministry of Health and Welfare, n.d.); Korea was considered the worst epicenter outside of China at the time. Like the rest of the world, the majority of faculty in Korea had little to no prior training and/or experience in teaching at a distance, and these universities were largely unable to support faculty the way that universities with traditional online programs do: with dedicated support staff, proper hardware, software, services, et cetera (Means et al., 2014). However, the Korean government’s initial pandemic response efforts led to a dramatic reduction in cases, which in turn influenced the decision to allow national borders to remain open, allowing for international travel and tourism to a degree. This decision also allowed for international students to enter the country, as well as for long- and short-term student mobility programs (such as academic exchange/study abroad programs) to continue operating. Other countries such as Canada, China, and the United States, by contrast, had closed both borders and/or suspended their mobility programs (Veerasamy & Ammigan, 2021). While much praise was given to the Korean government at this time for its apparent success in mitigating the spread of the SARS-CoV2 virus (Lim et al., 2021), the optimism of returning back to normal was ultimately short-lived if not premature.
A few weeks prior to the start of the fall 2020 semester, a COVID-19 cluster infection in the Seoul metropolitan area (which is home to roughly half of the national population) led to a second wave of COVID-19 cases which was significantly more severe than the first (Kim, 2020). This wave would ultimately never abate and continue into and throughout 2021. The initial 10,000 cases that occurred between late January and the end of March 2020 would seem miniscule compared to approximately 60,000-180,000 cases per month throughout fall 2021 (see Yonhap News Agency, 2021); the monthly case trend is illustrated in Figure 1. Even though the national vaccination rate at this time exceeded 80%, COVID-19 hospitalizations and deaths began increasing to critical levels (see The Korea
Times, 2021). Thus, the Korean government’s plan for a phased return to normal life was halted with numerous restrictions (e.g., business operation curfews, take-out only dining, closure of “high risk” facilities, etc.) being reimposed (Yoon, 2021). While courses were poised to return to face-to-face delivery in November 2021, remote teaching learning remained in effect. Even in early 2022, cases began increasing to more than 100,000 per day due to the CO VID omicron variant (Yonhap News Agency, 2022a), ultimately nearing 200,000 per day a week before the start of the spring 2022 semester (Yonhap News Agency, 2022b). From the first week of the semester, cases began approaching 300,000, representing some of the highest reported case loads in the world (Yonhap News Agency, 2022c).Since many Korean universities required students to enter the country in case of resuming face-to-face courses or certain activities (i.e., exams; Stewart & Kim, 2021), seemingly “paradoxical” residential distance education programs for international students emerged where, although students had checked into campus and were living in dormitories, courses were primarily delivered online for the duration of the semester (Stewart & Lowenthal, 2021, 2022). Further, many campus facilities had been closed as a health and safety measure to mitigate the spread of COVID-19 on campus. Nevertheless, each subsequent semester saw the increase in faculty experience with remote teaching, as well as having time to plan and prepare for conducting their courses in a SRT format in advance. Thus, since educational institutions, instructors, and students now possessed experience with remote teaching and learning, we thought it pertinent to longitudinally investigate students’ experiences and perceptions of SRT. More specifically, we focused our inquiry on how the experiences and perceptions of international exchange students taking SRT in Korea changed over time. In this paper, we present the results of a four-semester study grounded in performance improvement theory and discuss implications and areas of future research and practice.

COVID-19 Cases in Korea January 2020–December 2021

COVID-19 Cases in Korea January 2020–December 2021
Related Literature
Formal internet-based distance education is a common, modern activity (Dunlap & Lowenthal, 2018). Prior to COVID-19, over a third of students took at least one internet-based course in a given year in the United States alone (Seamen et al., 2018). Formal distance education itself dates back to postal correspondence courses in the early 1800s (Bower & Hardy, 2004) and ongoing technological developments have simply expanded the scope and degree of the practice worldwide (Moore & Kearsley, 2012; Saba, 2011). Emergency remote teaching (ERT), like distance education, is also not new but ERT was obscure prior to the pandemic, appearing briefly in response to various local crises (Davies & Bentrovato, 2011; Hodges et al., 2020). The global scale and ongoing nature of the COVID-19 pandemic have turned the obscure practice of ERT into a universal, sustained experience, or what Stewart et al. (2022) have called sustained remote teaching (SRT). Student experiences with SRT, for better or worse, are likely to not only influence current perceptions of distance education, but broadly mischaracterize formal distance education for many (Stewart & Lowenthal, 2021, 2022). Thus, the dominant experience with remote teaching in the pandemic is, in fact, not ERT, but rather SRT. While ERT was assumed to be short-lived early in the pandemic (Stewart, 2021), the continued reliance on remote teaching is now spanning the majority of (if not entire) academic programs such as master’s degrees, graduate certificates, exchange programs, et cetera. International students are no exception in this regard, but they do face additional challenges (Bond et al., 2021; Forbes-Mewett, 2019).
International Students and Remote Teaching
When international students take traditional online courses, many find themselves exposed to certain hardships related to language proficiency and sociocultural norms which are intrinsic to this student group (Zhang & Kenny, 2010). International students also often find themselves studying virtually alongside diverse peers in heterogeneous sociolinguistic/ cultural learning environments which are likely to affect students’ experiences in unexpected ways (Harrison et al., 2018). While positive experiences are definitely had when international students take online courses (Gemmell et al., 2014), negative ones are also common (Lee, 2011). For example, international students can have a more difficult time navigating and interacting in virtual learning environments (Habib et al., 2014). Further, given the asynchronous nature of many conventional online courses, international students can also be prone to more isolation and loneliness than their noninternational and/or face-to-face counterparts in their host countries (Erichsen & Bolliger, 2011). This isolation has been amplified with exchange students in Korea as a result of ERT (Stewart & Lowenthal, 2021, 2022). Further complicating matters is that certain students are predisposed to struggle with formal online learning (Means et al., 2014; Xu & Jaggars, 2014) as a result of factors such as socioeconomic status (Stoessel et al., 2015). In the case of international students, learning online can also compound mental health issues which are often more prevalent among international students (Forbes-Mewett, 2019). These more common mental health issues have been simply added to with pandemic-related mental health issues in numerous capacities ranging from stress, decreased motivation, confusion/disorientation, anxiety, et cetera (Bal et al., 2020; Gao, 2020; Green et al., 2020; Kapasia et al., 2020; MacIntyre et al., 2020; Petillion & McNeil, 2020). Despite the overwhelming use of synchronous teaching tools and hypothetically increased real-time interactions mitigating feelings of isolation (Bond et al., 2021; Lowenthal et al., 2021), teachers and students have still reported feelings of isolation and loneliness (Green et al., 2020). Synchronous course delivery does not itself address presence or engagement if not deliberately designed for and fostered (Trespalacios & Uribe-Florez, 2020). While many scholars, teachers, and students reasonably expected that the quality of the initial ERT courses would be low given their make-shift nature (Hodges et al., 2020), there also seems to be an implicit assumption that ERT course quality would improve over time; such an expectation, however, is questionable (Shattuck, 2021). Further, much ERT research to date has primarily sampled regular degree/local students, glossing over vulnerable student populations such as international students (Bond et al., 2021). More critically, empirical research to date has shown ambiguous results in terms of student performance in numerous areas.
Ambiguous Performance
Peters et al. (2020) found that live lectures did not necessarily increase attendance rates or student engagement during the first semester of the pandemic. Gillis and Krull (2020) reported that even with pass-fail grading, student motivation had decreased. Wilcox and Vignal (2020) noted that teachers did not perceive decreased student workloads as having any positive effect on student performance. Faize and Nawaz (2020) found an increase in student satisfaction as a result of changes to teaching practices during their initial ERT period, but they realized that this change could also have been due to students and instructors simply having more experience teaching and learning remotely toward the end of the semester. By extension, it is reasonable to hypothesize that remote teaching/learning experience semester over semester could possibly result in no particularly obvious changes, which Stewart et al. (2022) found when comparing exchange students’ perceptions of remote teaching between the first and second semester of 2021 (i.e., perceptions did not improve in any significant way). Aldhahi et al. (2022) similarly found that while self-efficacy was highly correlated with e-learning satisfaction, the possibility existed that institutional preparedness could also be responsible for their positive results to some degree rather than self-efficacy alone. Abdulrahim and Mabrouk (2020) found that digital learning had improved student learning outcomes, however, noted that participants predominantly came from majors in the humanities, suggesting the possibility that the field of study could be responsible for the positive outcome. In short, improvement in remote teaching practices is not a given over the course of a semester (Alqurshi, 2020; Jandric et al., 2021; Schlesselman, 2020; Shattuck, 2021; Shim & Lee, 2020; Stewart et al., 2022). In fact, significant change may not manifest in a single semester or even two, and a tough yet critical question to ask is if the indefinite duration of SRT and faculty experience reasonably improves remote teaching quality in terms of scope (i.e., numerous benchmarks) and degree (i.e., significant/meaningful changes). This is particularly pragmatic in terms of evaluating whether or not faculty support, training, and experience has resulted in improved performance, as well as if the financial and human resource development costs of doing so have been beneficial as institutions have invested considerable resources to maintain educational continuity.
Sustained Remote Teaching Issues
The COVID-19 pandemic has caused the first involuntary global exercise in remote teaching and learning to date. While many institutions, educators, and students were no doubt hopeful that ERT would be short-lived, the complexities of (and differences in) the human and governmental responses to the pandemic have forced the hands of many institutions and educators; remote teaching and learning continues to be necessary as a health and safety measure. The introduction of improved public health and safety measures, rapid polymerase chain reaction testing, COVID-19 vaccines, and treatments for the disease have started to reduce the need for social distancing at schools around the world, however, this is not uniform due to heterogeneous vaccination rates and paradoxically increasing COVID-19 case rates even where such rates are high. In Korea, infections have increased dramatically (when compared to the start of the pandemic) despite the government achieving more than an 80% vaccination rate nationally (see The Korea Times, 2021). Thus, the reliance on SRT raises multiple issues that are not currently addressed in the literature.
First, when remote teaching is maintained beyond the onset of a crisis and sustained indefinitely, we are witnessing a distinctly different remote teaching phenomena and it is not clearly known if the ongoing institutional support-interventions to train faculty and assist emergency transitions ultimately improve student experiences and perceptions with remote teaching and learning (Alqurshi, 2020; Jandric et al., 2021; Schlesselman, 2020; Shim & Lee, 2020; Stewart et al., 2022). For better or worse, the worlds’ SRT experiences, rather than ERT, are likely to (mis)characterize how students and instructors view learning online since the health and safety threat posed by the pandemic has not ended. While institutions continue to teach remotely, research to date (e.g., Bingimlas, 2021; Choi et al., 2021; Dulohery et al., 2021; Kawasaki et al., 2021; Müller et al., 2021; Perets et al., 2020; Petillion & McNeil, 2020; Sofianidis et al., 2021; Stewart & Lowenthal, 2021; Tabatadze & Chachkhiani, 2021; Wilcox & Vignal, 2020) largely documents changes in behaviors and perceptions of ERT only during the first semester of the pandemic. There is a significant blind spot in terms of whether or not the quality of remote teaching, or at least the perception thereof, has reasonably improved over time. Given this, we investigated the issue of SRT improvement through the lens of performance improvement theory.
Performance Improvement Theory
Performance broadly refers to the manner in which an agent, units, and/or processes function such as teams, groups, departments, institutions, et cetera (Elger, 2007). When performance is considered to have improved, this change is hypothesized as the result of knowledge and skill acquisition (Vits & Gelders, 2002). That is, new knowledge is applied, resulting in outcomes that can be measured such as the faster production of tasks, the more frequent use of more refined techniques or tools, and/or the more efficient/effective use of resources, et cetera. Performance is also affected by other factors such as the economic, political, social and cultural contexts in which people and groups are situated (Swanson, 1999). For example, a school located near a university with a large teaching program may be able to acquire higher skilled teachers more quickly due to proximity to graduates and/or having a direct relationship with the college compared to one that is located in a rural area. Further, performance occurs across multiple knowledge/skill domains (Elger, 2007) and what may successfully work in one setting, such as interventions at an institution with experienced faculty, may not manifest the same way at one with novice faculty. In any case, interventions to change or improve performance are generally designed to meet indicators of a target behavior (Burrow & Berardinelli, 2003). In educational settings, this might be changing an instructional method by making related structural changes (e.g., implementing new evaluation criteria) to modify instructional behavior long term (Morrison et al., 2011). However, when considering ERT and SRT, a performance paradox exists where permanent remote delivery is not the intended goal (see Hodges et al., 2020).
Performance improvement theory suggests that the original crisis-based interventions used to help faculty transition to ERT, in addition to ongoing training, support, and experience, should produce an improvement in teaching behavior that is measurable. Since the COVID-19 pandemic endures around the world, SRT continues to be relied upon as the primary method of educational continuity, requiring teaching and learning to still be done online involuntarily. Further, since SRT (like ERT) is not meant to be permanent, it is likely unreasonable to expect institutions to change their course and faculty evaluation criteria. It is also worth noting that even once the most serious health risks subside, some students and instructors may continue to teach and learn remotely by choice, creating a more refined conceptual practice of Sustained ERT (S-ERT) versus SRT, which is not a crisis-based practice. In this article, we investigated longitudinally how instructors’ remote teaching experience, knowledge, and skill (i.e., new inputs) would change instructor performance (i.e., new processes) and result in hypothetically changed student perceptions of SRT (i.e., new outcomes [improved performance]) (Swanson, 1999).
Context of Study
This study was undertaken at a large, private research institute in northern Seoul over four consecutive regular academic semesters from spring 2020 to fall 2021. The university has a student population of approximately 20,000 students; 3,300 are international degree-seeking, exchange, and language center students. The university has been conducting the overwhelming majority of its courses online as a health and safety measure against COVID-19 since spring 2020. While spring 2020 initially saw the emergence of ERT as an unplanned crisis response to COVID-19, this practice became planned and sustained each consecutive semester to date due to persistent COVID-19 cluster infections (see Kim, 2020). These cluster infections have often coincided with the start of academic semesters around March and August and/or spiked significantly during them (see Yonhap News, 2021).
Key Research Objectives
While exchange students may not be the most numerous among the total student population, exchange students at our university are very diverse (usually representing more than 130 universities from a hundred different countries). They also have the ability to crossenroll across almost all colleges and departments. Thus, their experiences cover a broad area of subject matters, courses, faculty, teaching methods, majors, et cetera despite being proportionally smaller than degree students as a whole. Since university faculty have gained roughly 2 years of experience in teaching remotely from the onset of the pandemic, this study was guided by the following research questions:
Have exchange students’ perceptions of teaching and learning processes, student support, and course structure change when participating in SRT?
Do SRT teaching and learning processes, student support, and course structure improve over time?
Methodology
Data was collected via an electronic survey around the middle to the end of four consecutive remote teaching semesters starting spring 2020 (at the start of the COVID-19 pandemic) then in fall 2020, spring 2021, and finally fall 2021. After completing an informed consent statement, students were asked basic demographic items, questions about the characteristics of their courses, and to rate their perceptions (using a 5-point scale [1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree]) of learning online on three dimensions (teaching and learning processes, student support, and course structure) from the Institute for Higher Education Policy (iHEP) benchmarks for success in internet-based distance education (Phipps & Merisotis, 2000). At the end of the survey, an optional openended question invited students to share their experiences with learning remotely in their own words. The survey remained open for around 4 to 5 weeks at each data collection point with automated email reminders at weekly intervals. No incentives were offered for participation.
Participants
Participants came from the spring 2020 (263), fall 2020 (167), spring 2021 (287), and fall 2021 (444) exchange student body. In Semester 1 (spring), there were 140 responses yielding a 53.23% response rate. In Semester 2 (fall 2020), there were 93 responses yielding a 56.02% response rate. In Semester 3 (spring 2021), there were 141 responses producing a 45.6% response rate and lastly in Semester 4 (fall 2021), there were 143 responses yielding a 32.2% response rate (detailed respondent demographics are presented in Table 1). Students’ nationalities were representative of the population as a whole with variations between population totals and survey respondents by nationality ranging from 0-5% each semester. Other demographic characteristics were confirmed to be congruent with the exchange program as a whole by the Office of International Affairs. Notable (but not unexpected) among the respondents was the continual increase in prior online course experience. We suspect that where these numbers decrease or when students were answering “no”, they may have been assuming the survey item referred to online course experience at the host versus their home universities where they did, in fact, have experience of remote teaching and learning.
Respondent Demographics and Exchange Characteristics
| Survey Item | Characteristic | Spring 2020 53.23% (n = 140) | Fall 2020 56.02% (n = 93) | Spring 2021 45.60% (n = 131) | Fall 2021 32.20% (n = 143) |
|---|---|---|---|---|---|
| Age |
| 62.83% | 65.6% | 69.5% | 69.2% |
| 37.17% | 34.4% | 30.5% | 30.8% | |
| Gender |
| 13.6% | 16.1% | 18.3% | 13.3% |
| 86.4% | 83.9% | 81.7% | 86.7% | |
| Primary study level |
| 72.2% | 81.1% | 78.6% | 79.7% |
| 16.4% | 13.5% | 17.6% | 11.9% | |
| 11.4% | 5.4% | 3.8% | 8.4% | |
| Exchange |
| 47.1% | 41.4% | 54.4% | 53.8% |
| length |
| 11.4% | 7.2% | 13.0% | 7.7% |
| 32.1% | 40.5% | 22.1% | 28.7% | |
| 9.4% | 10.8% | 11.5% | 9.8% | |
| Campus |
| 90% | 95.5% | 87% | 83.9% |
| 10% | 4.5% | 13% | 16.1% | |
| Prior online course experience |
| 17% | 82% | 68% | 79.7% |
| 83% | 18% | 32% | 20.3% |
| Survey Item | Characteristic | Spring 2020 53.23% (n = 140) | Fall 2020 56.02% (n = 93) | Spring 2021 45.60% (n = 131) | Fall 2021 32.20% (n = 143) |
|---|---|---|---|---|---|
| Age | 18-22 | 62.83% | 65.6% | 69.5% | 69.2% |
23-30 | 37.17% | 34.4% | 30.5% | 30.8% | |
| Gender | Male | 13.6% | 16.1% | 18.3% | 13.3% |
Female | 86.4% | 83.9% | 81.7% | 86.7% | |
| Primary study level | Undergraduate | 72.2% | 81.1% | 78.6% | 79.7% |
Graduate | 16.4% | 13.5% | 17.6% | 11.9% | |
Certificate | 11.4% | 5.4% | 3.8% | 8.4% | |
| Exchange | 4 months | 47.1% | 41.4% | 54.4% | 53.8% |
| length | 6 months | 11.4% | 7.2% | 13.0% | 7.7% |
10 months | 32.1% | 40.5% | 22.1% | 28.7% | |
12 months | 9.4% | 10.8% | 11.5% | 9.8% | |
| Campus | Seoul | 90% | 95.5% | 87% | 83.9% |
Satellite | 10% | 4.5% | 13% | 16.1% | |
| Prior online course experience | Yes | 17% | 82% | 68% | 79.7% |
No | 83% | 18% | 32% | 20.3% |
Note: Total exchange student enrollment for spring 2020 = 263, fall 2020 = 167, spring 2021 = 287, fall 2021 = 444.
Certificate refers to an intensive Korean language program.
Results
Course Characteristics
Course characteristics remained similar over the four semesters of data collection, though the most notable item was Course Type. Each semester, students described an increase in the prevalence of synchronous courses (i.e., live lessons through video conferencing) from 28.6% in spring 2020 to 51.6% in fall 2021. Since students also noted taking courses offered in both synchronous and asynchronous formats, the predominance of synchronous online courses is striking when compared to formal distance education which is predominantly asynchronous in upper secondary and tertiary education (Means et al., 2014).
Emergency/Sustained Remote Course Characteristics
| Survey Item | Characteristic | Spring 2020 53.23% (n = 140) | Fall 2020 56.02% (n = 93) | Spring 2021 45.60% (n = 131) | Fall 2021 32.20% (n = 143) |
|---|---|---|---|---|---|
| Course load |
| 26.42% | 35.48% | 27.5% | 33.3% |
| 60.71% | 49.46% | 56.5% | 49.0% | |
| 12.84% | 15.05% | 16.0% | 17.6% | |
| Course size |
| 39.5% | 36.6% | 33.6% | 35.0% |
| 44.3% | 41.9% | 43.5% | 29.4% | |
| 15.7% | 17.2% | 19.9% | 35.0% | |
| 0.5% | 4.3% | 3.1% | 0.6% | |
| Course type |
| 8.6% | 9.7% | 4.6% | 2.0% |
| 28.6% | 31.2% | 46.6% | 51.6% | |
| 62.9% | 59.1% | 48.9% | 46.4% | |
| Course activities |
| 10.7% | 7.2% | 7.3% | 7.1% |
| 12.9% | 16.6% | 14.9% | 17.0% | |
| 19.7% | 18.9% | 18.3% | 18.4% | |
| 9.2% | 7.2% | 7.0% | 9.0% | |
| 27.9% | 29.1% | 28.5% | 28.5% | |
| 19.7% | 21.1% | 24.0% | 20.0% | |
| Location of course engagement |
| 66% | 62.5% | 55.2% | 61.5% |
| 14.5% | 26.9% | 29.8% | 25.2% | |
| 13% | 7.6% | 7.8% | 6.3% | |
| 4.4% | 3% | 5.5% | 5.6% | |
| 2.1% | 0% | 1.7% | 1.4% |
| Survey Item | Characteristic | Spring 2020 53.23% (n = 140) | Fall 2020 56.02% (n = 93) | Spring 2021 45.60% (n = 131) | Fall 2021 32.20% (n = 143) |
|---|---|---|---|---|---|
| Course load | 1-2 | 26.42% | 35.48% | 27.5% | 33.3% |
3-5 | 60.71% | 49.46% | 56.5% | 49.0% | |
6-9 | 12.84% | 15.05% | 16.0% | 17.6% | |
| Course size | 1-20 | 39.5% | 36.6% | 33.6% | 35.0% |
21-40 | 44.3% | 41.9% | 43.5% | 29.4% | |
41-60 | 15.7% | 17.2% | 19.9% | 35.0% | |
61+ | 0.5% | 4.3% | 3.1% | 0.6% | |
| Course type | Asynchronous | 8.6% | 9.7% | 4.6% | 2.0% |
Synchronous | 28.6% | 31.2% | 46.6% | 51.6% | |
Both types | 62.9% | 59.1% | 48.9% | 46.4% | |
| Course activities | Discussion forums | 10.7% | 7.2% | 7.3% | 7.1% |
Small-group projects | 12.9% | 16.6% | 14.9% | 17.0% | |
Self-study assignments | 19.7% | 18.9% | 18.3% | 18.4% | |
Live (text) chats | 9.2% | 7.2% | 7.0% | 9.0% | |
Video conferencing | 27.9% | 29.1% | 28.5% | 28.5% | |
Prerecorded lectures | 19.7% | 21.1% | 24.0% | 20.0% | |
| Location of course engagement | Dormitory | 66% | 62.5% | 55.2% | 61.5% |
Apartment | 14.5% | 26.9% | 29.8% | 25.2% | |
Cafe | 13% | 7.6% | 7.8% | 6.3% | |
Goshiwon | 4.4% | 3% | 5.5% | 5.6% | |
Study room | 2.1% | 0% | 1.7% | 1.4% |
Note: Total exchange student enrollment for spring 2020 = 263, fall 2020 = 167, spring 2021 = 287, fall 2021 = 444.
Goshiwon is a common housing option available to students unique to Korea.
iHEP Dimensions
The overall scores for the survey’s three dimensions and scale reliability are presented in Appendix A. Each dimension’s Cronbach’s alpha score (with the exception of course structure for fall 2020) was greater than 0.7 representing internal data consistency overall. Students’ perceptions of the three iHEP dimensions can be characterized as neutral or slightly positive with the mean scores ranging between 3 (neutral) and 4 (agree) each semester. The mean scores for each dimension increased by approximately 2-7% from spring 2020 to fall 2021 despite some fluctuations. Meanwhile, the standard deviation scores generally decreased over time with some dimensions seeing minor (i.e., teaching and learning processes) or consistent (i.e., course structure) fluctuations.
Teaching and Learning Processes
Among the three iHEP dimensions, the teaching and learning processes dimension shows the most variation semester over semester among its benchmarks. What stands out is the sudden increases and then subsequent decreases in benchmarks such as “Feedback to students is provided in a manner that is constructive and helpful,” “Student interaction with other students is facilitated through a variety (e.g., 1:1 group activities, projects, discussion, etc.) of ways,” or “Course materials (i.e., books, PowerPoints, videos, software, etc.) promote collaboration among students.” The mean scores generally increase over time (or are relatively consistent) with the exception of “Feedback to students is provided in amanner that is constructive and helpful.” which decreases throughout the pandemic.
Teaching and Learning Processes’ Mean Scores Over Time
Student Support
Students’ perceptions of student support fluctuated throughout the pandemic. Three out of the five benchmarks’ mean scores increase over time while two (“Information [e.g., syllabus, software guides, tutorials, etc.] is supplied to students about their courses” and “Students can obtain assistance to help them use the course software [e.g., E-Class, WebEx, Zoom, etc.]“) decrease.
Course Structure
Among all of the iHEP dimensions, Course Structure saw the most stability among its benchmarks. Four benchmarks’ mean scores generally increase over time, and only one (Specific expectations are set for students with respect to a minimum amount of time per week for study and homework assignments) decreases noticeably by 6.5% in spring 2021 (Semester 3) but then increases again in fall 2021.
Inferential Analyses
Since the majority of exchange students only stay for 4 to 6 months (see Table 1) in addition to unpredictably shortening or extending exchange periods (Stewart, 2020a), paired tests are not feasible. Given this characteristic of the target population, we decided to measure potential differences in perceptions of the iHEP dimensions and benchmarks through independent samples one-way ANOVAs. We used SurveyMonkey each semester to send out unique email-based invitations and thus were able to cross reference complete responses for independence of observations. Further, since characteristics of the data matched the four assumptions of a Kruskwal-Wallis independent samples one-way ANOVA, we conducted the nonparametric test for each iHEP benchmark and overall iHEP dimension score using the statistics software Jamovi. Common responses that were removed prior to inferential analysis are noted under each respective semester comparison along with adjusted M and SD calculations (as well as a scores where applicable).
Semester 1 ERT Versus Semester 2 SRT
When comparing the first ERT semester (spring 2020) with the first SRT semester (fall 2020), there were four benchmarks with statistically significant results (two in teaching and learning processes, two in course structure) as well as the two dimensions teaching and learning processes and course structure. the improvement from ERT to SRT is relatively minor (if not insignificant in terms of real-world changes) with only 20% (4/20) of benchmarks showing improvement increases in their mean scores from 5-11%. Nevertheless, it is the only baseline for comparison in the subsequent SRT semesters in this study. Results are presented in Table 3. Note: The adjusted semester response count for independent samples analysis are 133 (spring) and 86 (fall). CI = 95%.
Independent-Samples Analysis of Differences Between Spring 2020 and Fall 2020 (Semester 1 to 2)
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| Teaching and learning processes |
| .847 .884 | 3.27 3.47 | .646 .662 | 102.69 121.30 | 4.51900 | 1 | .034 |
| Student interaction with other students is facilitated through various ways (e.g., 1:1, group activities, projects, and discussions). |
| 2.81 3.16 | 1.156 1.146 | 102.84 121.07 | 4.58646 | 1 | .032 | |
| Course materials (i.e., books, PowerPoints, videos, and software) promote collaboration among students. |
| 2.63 3.15 | 1.062 1.00 | 98.38 127.97 | 12.2323 | 1 | <.001 | |
| Course structure |
| .754 .690 | 3.49 3.73 | .754 .534 | 102.48 121.62 | 4.82568 | 1 | .028 |
| Faculty are required to grade and return all assignments within a specific period. |
| 3.34 3.69 | 1.10 .815 | 102.98 120.85 | 4.61844 | 1 | .032 | |
| Learning outcomes for each course are summarized in clearly written, straightforward statements. |
| 3.30 3.58 | .977 .789 | 103.38 120.24 | 4.21368 | 1 | .040 |
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| Teaching and learning processes | Spring 2020 Fall 2020 | .847 .884 | 3.27 3.47 | .646 .662 | 102.69 121.30 | 4.51900 | 1 | .034 |
| Student interaction with other students is facilitated through various ways (e.g., 1:1, group activities, projects, and discussions). | Spring 2020 Fall 2020 | 2.81 3.16 | 1.156 1.146 | 102.84 121.07 | 4.58646 | 1 | .032 | |
| Course materials (i.e., books, PowerPoints, videos, and software) promote collaboration among students. | Spring 2020 Fall 2020 | 2.63 3.15 | 1.062 1.00 | 98.38 127.97 | 12.2323 | 1 | <.001 | |
| Course structure | Spring 2020 Fall 2020 | .754 .690 | 3.49 3.73 | .754 .534 | 102.48 121.62 | 4.82568 | 1 | .028 |
| Faculty are required to grade and return all assignments within a specific period. | Spring 2020 Fall 2020 | 3.34 3.69 | 1.10 .815 | 102.98 120.85 | 4.61844 | 1 | .032 | |
| Learning outcomes for each course are summarized in clearly written, straightforward statements. | Spring 2020 Fall 2020 | 3.30 3.58 | .977 .789 | 103.38 120.24 | 4.21368 | 1 | .040 |
Note: The adjusted semester response count for independent samples analysis is 133 (spring) and 86 (fall). CI = 95%.
Semester 2 SRT Versus Semester 3 SRT
When comparing the first SRT (fall 2020) with the second SRT semester (spring 2021),changes in students’ perceptions of performance improvement are minimal. Only a single benchmark was shown to have a statistically significant result between the two semesters with a 4.68% increase in the mean score equating to 5% of all benchmarks. The change in perceptions is negligible and the results are presented in Table 4.
Independent-Samples Analysis of Differences Between Fall 2020 and Spring 2021 (Semester 2 to 3)
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| A system is in place to address student complaints or difficulties with the course. |
| 3.67 3.85 | .822 .975 | 95.23 110.91 | 3.9202 | 1 | .048 |
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| A system is in place to address student complaints or difficulties with the course. | Fall 2020 Spring 2021 | 3.67 3.85 | .822 .975 | 95.23 110.91 | 3.9202 | 1 | .048 |
Note: The adjusted semester response count (and adjusted M and SD) for independent samples (8 common responses removed) analysis are 85 (fall 2020) and 123 (spring 2021). CI = 95%.
Semester 3 SRT Versus Semester 4 SRT
When comparing the second SRT (spring 2021) and third SRT (fall 2021) semesters, there were only two benchmarks that saw statistically significant improvement. Both benchmarks were from the teaching and learning processes dimension and saw 8.71-10.13% increases in their mean scores, representing 10% of all the benchmarks. The change in perceptions is minimal/negligible. Results are presented in Table 5.
Independent-Samples Analysis of Differences Between Spring 2021 and Fall 2021 (Semester 3 to 4)
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| Courses are well organized into units and allow students to master objectives before moving on to the next unit. |
| 3.46 | .781 | 118.84 | 5.3302 | 1 | .021 | |
| 3.79 | .882 | 139.21 | |||||
| Student interaction with faculty is facilitated through various ways (e.g., chat, email, office hours, and class postings). |
| 3.46 | 1.06 | 120.26 | 4.1192 | 1 | .042 | |
| 3.85 | .806 | 137.91 |
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| Courses are well organized into units and allow students to master objectives before moving on to the next unit. | Spring 2021 | 3.46 | .781 | 118.84 | 5.3302 | 1 | .021 | |
Fall 2021 | 3.79 | .882 | 139.21 | |||||
| Student interaction with faculty is facilitated through various ways (e.g., chat, email, office hours, and class postings). | Spring 2021 | 3.46 | 1.06 | 120.26 | 4.1192 | 1 | .042 | |
Fall 2021 | 3.85 | .806 | 137.91 |
Note: The adjusted semester response count (and adjusted M and SD) for independent samples (8 common responses removed) analysis are 123 (spring 2021) and 135 (fall 2021). CI = 95%.
Semester 1 ERT Versus Semester 4 SRT
When comparing Semester 1 (the first semester in the pandemic and the only ERT semester) with the last SRT semester (fall 2021) three semesters later, changes in students’ perceptions of SRT teaching and learning processes and course structure are more noticeable. Eight benchmarks saw statistically significant results with one significantly worse result and seven items with improvement (representing roughly 30% of total benchmarks), in addition to teaching and learning processes and course structure dimensions showing statistically significant improved perceptions. The increases in theseitems’ mean scores ranged from 7—15%. These two dimensions similarly saw statistically significant improvement from the first semester of ERT (spring 2020) and the next semester of SRT (fall 2020). The overall improvement over 2 years shows improved scope (i.e., the total number of improved benchmarks), however the degree (i.e., increase in mean scores) of improvement is still relatively minor. Interestingly, there were no statistically significant changes in students’ perceptions of the student support dimension or its benchmarks across any of the four semesters. Results are presented in Table 6.
Independent-Samples Analysis of Differences Between Spring 2020 and Spring 2021 (Semester 1 to 4)
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| Teaching and Learning Processes |
| .839 .876 | 3.29 3.54 | .624 .617 | 126.19 157.48 | 10.3744 | 1 | .001 |
| Feedback to students is provided in a manner that is constructive and helpful. |
| 3.64 3.10 | .983 .803 | 152.25 131.53 | 4.8242 | 1 | .028 | |
| Student interaction with faculty is facilitated through various ways (e.g., chat, email, office hours, and class postings). |
| 3.56 3.85 | .815 .813 | 133.04 150.78 | 3.8833 | 1 | .049 | |
| The course units are of varying lengths, determined by the complexity of the learning objectives. |
| 3.47 3.87 | .917 .816 | 128.95 154.78 | 8.3774 | 1 | .004 | |
| Student interaction with other students is facilitated through various ways (e.g., 1:1, group activities, projects, and discussions). |
| 2.82 3.32 | 1.05 .939 | 128.46 155.25 | 8.2851 | 1 | .004 | |
| Course materials (i.e., books, PowerPoints, videos, and software) promote collaboration among students. |
| 2.66 3.14 | 1.05 1.00 | 123.28 160.33 | 15.6144 | 1 | <.001 | |
| Easily accessible technical support is available to students throughout the course. |
| 3.09 3.32 | .925 .893 | 131.66 152.13 | 5.0021 | 1 | .025 | |
| Course Structure |
| .746 .805 | 3.50 3.77 | .656 .633 | 124.68 158.96 | 12.5990 | 1 | <.001 |
| Students are provided with basic course information that outlines course objectives, concepts, and ideas. |
| 3.86 4.06 | .721 .789 | 130.11 153.64 | 7.19684 | 1 | .007 | |
| Specific expectations are set for students concerning a minimum amount of time per week for study and homework assignments. |
| 3.46 3.64 | .932 .891 | 132.02 151.77 | 4.62519 | 1 | .032 | |
| Learning outcomes for each course are summarized in clearly written, straightforward statements. |
| 3.35 3.72 | .936 .851 | 125.40 158.26 | 13.0170 | 1 | <.001 |
| iHEP Dimensions/Benchmarks | Semester | α | M | SD | Mean Rank | χA2 | df | p |
|---|---|---|---|---|---|---|---|---|
| Teaching and Learning Processes | Spring 2020 Spring 2021 | .839 .876 | 3.29 3.54 | .624 .617 | 126.19 157.48 | 10.3744 | 1 | .001 |
| Feedback to students is provided in a manner that is constructive and helpful. | Spring 2020 Spring 2021 | 3.64 3.10 | .983 .803 | 152.25 131.53 | 4.8242 | 1 | .028 | |
| Student interaction with faculty is facilitated through various ways (e.g., chat, email, office hours, and class postings). | Spring 2020 Spring 2021 | 3.56 3.85 | .815 .813 | 133.04 150.78 | 3.8833 | 1 | .049 | |
| The course units are of varying lengths, determined by the complexity of the learning objectives. | Spring 2020 Spring 2021 | 3.47 3.87 | .917 .816 | 128.95 154.78 | 8.3774 | 1 | .004 | |
| Student interaction with other students is facilitated through various ways (e.g., 1:1, group activities, projects, and discussions). | Spring 2020 Spring 2021 | 2.82 3.32 | 1.05 .939 | 128.46 155.25 | 8.2851 | 1 | .004 | |
| Course materials (i.e., books, PowerPoints, videos, and software) promote collaboration among students. | Spring 2020 Spring 2021 | 2.66 3.14 | 1.05 1.00 | 123.28 160.33 | 15.6144 | 1 | <.001 | |
| Easily accessible technical support is available to students throughout the course. | Spring 2020 Spring 2021 | 3.09 3.32 | .925 .893 | 131.66 152.13 | 5.0021 | 1 | .025 | |
| Course Structure | Spring 2020 Spring 2021 | .746 .805 | 3.50 3.77 | .656 .633 | 124.68 158.96 | 12.5990 | 1 | <.001 |
| Students are provided with basic course information that outlines course objectives, concepts, and ideas. | Spring 2020 Spring 2021 | 3.86 4.06 | .721 .789 | 130.11 153.64 | 7.19684 | 1 | .007 | |
| Specific expectations are set for students concerning a minimum amount of time per week for study and homework assignments. | Spring 2020 Spring 2021 | 3.46 3.64 | .932 .891 | 132.02 151.77 | 4.62519 | 1 | .032 | |
| Learning outcomes for each course are summarized in clearly written, straightforward statements. | Spring 2020 Spring 2021 | 3.35 3.72 | .936 .851 | 125.40 158.26 | 13.0170 | 1 | <.001 |
Note: The semester response count for independent samples analysis (no common responses) is 140 (52.23% response) and 143 (32.2% response). CI = 95%.
Discussion
Semester over semester, we saw minor changes in students’ perceptions of remote teaching course quality. Among the changes in student perception that did occur, these were limited in scope (i.e., only 5—30% percent of benchmarks) and degree (i.e., percent increases ranging from 5—15%) over the 2-year period. We interpret this, ultimately, as lacking practical significance in terms of real-world performance given the length of time involved. Like other studies of ERT/SRT since the start of the pandemic (e.g., Alqurshi, 2020; Jandric et al., 2021; Schlesselman, 2020; Shim & Lee, 2020; Stewart et al., 2022), the results of this study, though taking a much longer longitudinal view, show negligible and/or ambiguous improvement. On the surface, it would seem that university interventions, faculty support, and experience are not improving students’ perceptions of ERT/SRT. The lack of change, however, could be occurring for a number of other reasons, the most salient of which we outline below in terms of alignment.
First, when students and instructors begin conducting online courses, there can be a misalignment between roles and/or responsibilities in the digital environment, leading to frustration and/or confusion which results in subpar experiences (Bork & Rucks-Ahidiana, 2013). For example, students or instructors may not take the initiative to actively communicate directly with one another or the class as a whole, students may not proactively ask for clarification, assistance, et cetera. Moreover, the sociocultural alignment between exchange students and the host institution is different (Gunawardena & Jung, 2014), and institutions in Korea have not often taken such differences into account when delivering courses online (Lee, 2011). Similarly, given different experiences (and familiarity) with online learning environments, international students often face more obstacles interacting with the course itself (Habib et al., 2014). Further, after 2 years of experience with crisis-based remote teaching and learning, students still do not have the same prior experience and comfort with ERT as they have with in person face-to-face learning. Along those same lines, many instructors likely find themselves in a similar situation to their students—lacking sufficient training and experience teaching from a distance. As a result, many are continuing to simply replicate face-to-face teaching practices despite the change in the learning environment (Bozkurt et al., 2020; Chatziralli et al., 2020; Van Heuvelen et al., 2020). Another example of misalignment occurs typologically (i.e., international exchange versus local degree) by student.
In the case of exchange students, they are perpetually new to the university (Stewart et al., 2022), have different sociocultural norms of learning and teaching in the host country (Gunawardena, 2003, 2014; Gunawardena & Jung, 2014; Gunawardena & LaPointe, 2008), and are unaccustomed to the local virtual remote teaching and learning environment (Habib et al., 2014; Lee, 2011). As short-term educational migrants, exchange students lack a local support structure such as friends, family, and even faculty members (Stewart, 2020b). Further as minorities in the local society, exchange students are also statistically prone to perform worse in online courses (Stoessel et al., 2015). Thus even when instructors might be improving their ability to teach from a distance each semester, these changes might not even be noticeable by exchange students. Another reason for ambiguity may be an increase in course size due to the online delivery necessitated by the pandemic.
While physical classrooms are limited by size to some extent, virtual classrooms are not. Nevertheless, the ability for faculty to scale up teaching ability is finite (Sithole et al., 2019) and complicated by the involuntary nature of ERT/SRT (Stewart et al., 2022). Undergraduate courses where this study was conducted can easily see enrollment from 60-70 students. In terms of performance improvement theory, there may be a misalignment between the scalability of teaching ability and course enrollment numbers (Bork & Rucks-Ahidiana, 2013). Similarly, given that exchange students have been taking online courses in their home countries both before and during the pandemic, improved faculty performance could be coinciding with higher student expectations. Their exposure to more approaches to remote teaching may have provided them with better reference points for what a “good” remote course can look like, raising expectations at the host university. Nevertheless, a lack of formal changes to course evaluations (at least at the institution where the study was conducted) may also be playing a part.
It is possible that the lack of formal changes in evaluation (i.e., faculty/course evaluation criteria) has resulted in a status quo of teaching performance (Morrison et al., 2011). Other research to date, however, contrasts with this finding to some degree. For example, Jandric et al. (2021) noted how images of educators’ remote workspaces changed from the onset of the pandemic and their working environments 1 year later. They saw a shift away from chaotic and ill-prepared ERT-based working conditions to more dedicated and sophisticated distance teaching (i.e., SRT) preparation and organization. Improvement in this case may be related to institution and context (Swanson, 1999) and not felt broadly across all institutions by all student populations.
Thus, given how there are multiple factors influencing the lack of change in students’ perceptions of SRT, the need for more formal remote teaching plans is evident. In this sense, the SRT performance paradox as seen through exchange students’ perceptions and experiences is not a new problem nor one that is unique to this particular demographic. This paradox has a lengthy history in educational technology; computer hardware/software does not solve educational problems by itself, nor will access to technology alone suddenly change instructors’ beliefs about educational technology or distance education (Cuban et al., 2001; Ertmer, 1999). Thus, the lack of change can be considered unsurprising from this perspective. Further, the misalignment between traditional face-to-face courses and teaching practices and the sudden necessity of remote delivery was expected at the onset of the pandemic (Hodges et al., 2020). Based on our data, it would seem that the behavioral patterns established at the beginning of such crises are likely to persist for the duration of any given crisis absent a proactive guide for behavioral change and hopefully consequent improvement in practice.
Thus, institutions (and by extension, students) would likely benefit from at least the following two strategies: a) making formal plans for ERT/SRT scenarios in order to produce a higher quality of sustained remote teaching under duress of long-term crises, and b) having basic distance education courses added to preservice education programs, in addition to in-service professional development as a way of preemptively and proactively developing the expertise needed to effectively facilitate courses remotely. Given the unprecedented global crash course in distance education due to the pandemic, we suspect that online teaching will only expand into more traditional face-to-face learning environments (Lowenthal et al., 2021) given the hands-on experience with the praxis that millions of educators now possess.
Conclusion
The COVID-19 pandemic is far from over, however, future crises will present numerous short- and long-term challenges for educators and institutions. While distance education in many ways was ubiquitous before the pandemic (Dunlap & Lowenthal, 2018; Means et al., 2014; Stewart, 2019), this was largely undertaken voluntarily by students; their educators also had the requisite training and experience in teaching at a distance. However, assuming SRT will improve overtime as a function of crisis experience is tenuous and problematic. Given that we saw little reasonable change in scope and degree over a 2-year period in our data, it is paramount to develop ERT/SRT plans in order for educators to be more effective at remote teaching; merely providing access to webcams and videoconferencing software is simply not good enough. Moreover, without policy change to induce changes in performance during a crisis (Swanson, 1999), it is possible that SRT improvement will not occur. Nevertheless, the findings in this paper are not without limitations. First, the sampling was limited to one specific type of student at one university and it is likely that other types of students (i.e., degree students, local Korean students, graduate students, etc.) would rate these aspects of ERT/SRT courses differently. Further the ERT/SRT experience at other institutions may also be different due to different institutional capacity and faculty know-how. In that same vein, the dynamics and context of the study are set in Korea with international exchange students during the pandemic; other locations and other host university-student dynamics may present different results. The results and findings need to be considered judiciously when making comparisons to other contexts, settings, and populations. Nevertheless, as the pandemic continues, there is a clear need for research about the quality of SRT as its use as a remote teaching method continues. Future studies should collect data from multiple institutions and at larger scales to have broader generalizability to inform future practice and policy. While COVID-19 may be the crisis of today, there will, no doubt, be future crises that cause us to collectively rely on remote teaching; the question is whether or not we are willing to hold ourselves to higher standards of teaching and learning under duress rather than excusing our responsibilities as an unavoidable casualty of the crisis du jour.
References
Appendix A
iHEP Dimensions Scores by Semester
| α | M | SD | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iHEP | Spring 2020 | Fall 2020 | Spring 2021 | Fall 2021 | Spring 2020 | Fall 2020 | Spring 2021 | Fall 2021 | Spring 2020 | Fall 2020 | Spring 2021 | Fall 2021 |
| TLP | .839 | .877 | .908 | .876 | 3.294 | 3.46 | 3.40 | 3.54 | .624 | .644 | .775 | .617 |
| SS | .728 | .814 | .764 | .810 | 3.270 | 3.24 | 3.30 | 3.34 | .841 | .706 | .702 | .664 |
| CS | .746 | .678 | .711 | .805 | 3.508 | 3.72 | 3.69 | 3.77 | .656 | .525 | .606 | .633 |
| α | M | SD | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iHEP | Spring 2020 | Fall 2020 | Spring 2021 | Fall 2021 | Spring 2020 | Fall 2020 | Spring 2021 | Fall 2021 | Spring 2020 | Fall 2020 | Spring 2021 | Fall 2021 |
| TLP | .839 | .877 | .908 | .876 | 3.294 | 3.46 | 3.40 | 3.54 | .624 | .644 | .775 | .617 |
| SS | .728 | .814 | .764 | .810 | 3.270 | 3.24 | 3.30 | 3.34 | .841 | .706 | .702 | .664 |
| CS | .746 | .678 | .711 | .805 | 3.508 | 3.72 | 3.69 | 3.77 | .656 | .525 | .606 | .633 |
Note: TLP = teaching and learning processes, SS = student support, CS = course structure.
Appendix B
Perceptions of Teaching and Learning Processes
| Benchmarks | Semester | 1 | 2 | 3 | 4 | 5 | M | SD |
|---|---|---|---|---|---|---|---|---|
| Faculty provide feedback on student assignments and answer questions in a timely manner. | Spring 2020 | 3 (2.1%) | 19 (13.6%) | 27 (19.3%) | 68 (48.6%) | 23 (16.4%) | 3.64 | .983 |
| Fall 2020 | 5 (5.4%) | 5 (5.45) | 27 (29.0%) | 37 (39.8%) | 19 (20.4%) | 3.65 | 1.04 | |
| Spring 2021 | 6 (4.6%) | 5 (3.8%) | 44 (33.6%) | 63 (48.1%) | 13 (9.9%) | 3.55 | .896 | |
| Fall 2021 | 2 (1.4%) | 6 (4.2%) | 47 (32.9%) | 70 (49.0%) | 18 (12.6%) | 3.67 | .803 | |
| Feedback to students is provided in a manner that is constructive and helpful. | Spring 2020 | 4 (2.9%) | 13 (9.3%) | 34 (24.3%) | 68 (48.6%) | 21 (15.0%) | 3.64 | .946 |
| Fall 2020 | 3 (3.25) | 6 (6.5%) | 21 (22.6%) | 50 (53.8%) | 13 (14.0%) | 3.69 | .909 | |
| Spring 2021 | 14 (10.7%) | 35 (26.7%) | 29 (22.1%) | 42 (32.1%) | 11 (8.4%) | 3.01 | 1.167 | |
| Fall 2021 | 12 (8.4%) | 31 (21.7%) | 43 (30.1%) | 44 (30.8%) | 13 (9.1%) | 3.10 | 1.105 | |
| Courses are well organized into units and allow students to master objectives before moving on to the next unit. | Spring 2020 | 4 (2.9%) | 11 (7.9%) | 39 (27.9%) | 72 (51.4%) | 14 (10.0%) | 3.58 | .882 |
| Fall 2020 | 1 (1.1%) | 4 (4.3%) | 30 (32.3%) | 45 (48.4%) | 13 (14.0%) | 3.81 | .770 | |
| Spring 2021 | 8 (6.1%) | 16 (12.2%) | 35 (26.7%) | 51 (38.9%) | 21 (16.0%) | 3.47 | 1.091 | |
| Fall 2021 | 2 (1.4%) | 7 (4.9%) | 37 (25.9%) | 67 (46.9%) | 30 (21.0%) | 3.81 | .872 | |
| Student interaction with faculty is facilitated through a variety (e.g., chat, email, office hours, class postings) of ways. | Spring 2020 | 2 (1.4%) | 11 (7.9%) | 45 (32.1%) | 70 (50%) | 12 (8.6%) | 3.56 | .815 |
| Fall 2020 | 1 (1.1%) | 3 (3.2%) | 23 (24.7%) | 52 (55.9%) | 14 (15.1%) | 3.61 | .860 | |
| Spring 2021 | 6 (4.6%) | 14 (10.7%) | 31 (23.7%) | 57 (43.5%) | 23 (17.6%) | 3.59 | 1.044 | |
| Fall 2021 | 1 (.07%) | 7 (4.9%) | 32 (22.4%) | 75 (52.4%) | 28 (19.6%) | 3.85 | .813 | |
| The course units are of varying lengths determined by the complexity of the learning objectives. | Spring 2020 | 5 (3.6%) | 13 (9.3%) | 46 (32.9%) | 63 (45%) | 13 (9.3%) | 3.47 | .917 |
| Fall 2020 | 1 (1.1%) | 4 (4.3%) | 37 (39.8%) | 44 (47.3%) | 7 (7.5%) | 3.56 | .744 | |
| Spring 2021 | 5 (3.85) | 4 (3.1%) | 21 (16.0%) | 78 (59.5%) | 23 (17.6%) | 3.84 | .884 | |
| Fall 2021 | 2 (1.4%) | 4 (2.8%) | 34 (23.8%) | 74 (51.7%) | 29 (20.3%) | 3.87 | .816 | |
| Each unit requires students to engage themselves in analysis, synthesis, and evaluation as part of their course assignments. | Spring 2020 | 6 (4.3%) | 11 (7.9%) | 48 (34.3%) | 62 13 (44.3%) (9.3%) | 3.46 | .924 | |
| Fall 2020 | 1 (1.1%) | 8 (8.6%) | 29 (31.2%) | 43 12 (46.2%) (12.9%) | 3.61 | .860 | ||
| Spring 2021 | 4 (3.1%) | 13 (9.9%) | 31 (23.7%) | 64 19 (48.9%) (14.5%) | 3.62 | .956 | ||
| Fall 2021 | 2 (1.4%) | 10 (7.0%) | 42 (29.4%) | 73 16 (51.0%) (11.2%) | 3.64 | .827 | ||
| Class voice mail, video conferencing, or email systems are provided to encourage students to work with each other and their instructor(s). | Spring 2020 | 7 (5.0%) | 32 (22.9%) | 41 (29.3%) | 53 7 (37.9%) (5.0%) | 3.15 | .996 | |
| Fall 2020 | 5 (5.4%) | 12 (12.9%) | 33 (35.5%) | 36 7 (38.7%) (7.5%) | 3.30 | .976 | ||
| Spring 2021 | 6 (4.6%) | 8 (6.1%) | 41 (31.3%) | 57 19 (43.5%) (14.5%) | 3.57 | .969 | ||
| Fall 2021 | 1 (0.7%) | 13 (9.1%) | 48 (33.6%) | 62 19 (43.4%) (13.3%) | 3.59 | .858 | ||
| Courses are designed to require students to work in groups utilizing problemsolving activities to develop an understanding of the topic. | Spring 2020 | 11 (7.9%) | 41 (29.3%) | 34 (24.3%) | 50 4 (35.7%) (2.9%) | 2.96 | 1.04 | |
| Fall 2020 | 6 (6.5%) | 20 (21.5%) | 38 (40.9%) | 22 7 (23.7%) (7.5%) | 3.04 | 1.01 | ||
| Spring 2021 | 9 (6.9%) | 29 (22.1%) | 35 (26.7%) | 44 14 (33.6%) (10.7%) | 3.19 | 1.110 | ||
| Fall 2021 | 2 (1.4%) | 24 (16.8%) | 48 (33.6%) | 58 11 (40.6%) (7.7%) | 3.36 | .900 | ||
| Student interaction with other students is facilitated through a variety (e.g., 1:1, group activities, projects, discussions) of ways. | Spring 2020 | 21 (15.0%) | 37 (26.4%) | 37 (26.4%) | 36 9 (25.7%) (6.4%) | 2.82 | 1.05 | |
| Fall 2020 | 7 (7.5%) | 21 (22.6%) | 26 (28.0%) | 28 11 (30.1%) (11.8%) | 3.61 | 1.14 | ||
| Spring 2021 | 13 (9.9%) | 25 (19.1%) | 34 (26.0%) | 47 12 (35.9%) (9.2%) | 3.15 | 1.140 | ||
| Fall 2021 | 1 (0.7%) | 31 (21.7%) | 45 (31.5%) | 53 13 (37.1%) (9.1%) | 3.32 | .939 | ||
| Course materials (i.e., books, PowerPoints, videos, and software) promote collaboration among students. | Spring 2020 | 20 (14.3%) | 46 (32.9%) | 39 (27.9%) | 32 3 (22.9%) (2.1%) | 2.66 | 1.05 | |
| Fall 2020 | 5 (5.4%) | 18 (19.4%) | 35 (37.6%) | 29 6 (31.2%) (6.5%) | 3.14 | .985 | ||
| Spring 2021 | 18 (13.7%) | 27 (20.6%) | 35 (26.7%) | 42 9 (32.1%) (6.9%) | 2.98 | 1.167 | ||
| Fall 2021 | 6 (4.2%) | 35 (24.5%) | 45 (31.5%) | 47 10 (32.9%) (7.0%) | 3.14 | 1.004 |
| Benchmarks | Semester | 1 | 2 | 3 | 4 | 5 | M | SD |
|---|---|---|---|---|---|---|---|---|
| Faculty provide feedback on student assignments and answer questions in a timely manner. | Spring 2020 | 3 (2.1%) | 19 (13.6%) | 27 (19.3%) | 68 (48.6%) | 23 (16.4%) | 3.64 | .983 |
| Fall 2020 | 5 (5.4%) | 5 (5.45) | 27 (29.0%) | 37 (39.8%) | 19 (20.4%) | 3.65 | 1.04 | |
| Spring 2021 | 6 (4.6%) | 5 (3.8%) | 44 (33.6%) | 63 (48.1%) | 13 (9.9%) | 3.55 | .896 | |
| Fall 2021 | 2 (1.4%) | 6 (4.2%) | 47 (32.9%) | 70 (49.0%) | 18 (12.6%) | 3.67 | .803 | |
| Feedback to students is provided in a manner that is constructive and helpful. | Spring 2020 | 4 (2.9%) | 13 (9.3%) | 34 (24.3%) | 68 (48.6%) | 21 (15.0%) | 3.64 | .946 |
| Fall 2020 | 3 (3.25) | 6 (6.5%) | 21 (22.6%) | 50 (53.8%) | 13 (14.0%) | 3.69 | .909 | |
| Spring 2021 | 14 (10.7%) | 35 (26.7%) | 29 (22.1%) | 42 (32.1%) | 11 (8.4%) | 3.01 | 1.167 | |
| Fall 2021 | 12 (8.4%) | 31 (21.7%) | 43 (30.1%) | 44 (30.8%) | 13 (9.1%) | 3.10 | 1.105 | |
| Courses are well organized into units and allow students to master objectives before moving on to the next unit. | Spring 2020 | 4 (2.9%) | 11 (7.9%) | 39 (27.9%) | 72 (51.4%) | 14 (10.0%) | 3.58 | .882 |
| Fall 2020 | 1 (1.1%) | 4 (4.3%) | 30 (32.3%) | 45 (48.4%) | 13 (14.0%) | 3.81 | .770 | |
| Spring 2021 | 8 (6.1%) | 16 (12.2%) | 35 (26.7%) | 51 (38.9%) | 21 (16.0%) | 3.47 | 1.091 | |
| Fall 2021 | 2 (1.4%) | 7 (4.9%) | 37 (25.9%) | 67 (46.9%) | 30 (21.0%) | 3.81 | .872 | |
| Student interaction with faculty is facilitated through a variety (e.g., chat, email, office hours, class postings) of ways. | Spring 2020 | 2 (1.4%) | 11 (7.9%) | 45 (32.1%) | 70 (50%) | 12 (8.6%) | 3.56 | .815 |
| Fall 2020 | 1 (1.1%) | 3 (3.2%) | 23 (24.7%) | 52 (55.9%) | 14 (15.1%) | 3.61 | .860 | |
| Spring 2021 | 6 (4.6%) | 14 (10.7%) | 31 (23.7%) | 57 (43.5%) | 23 (17.6%) | 3.59 | 1.044 | |
| Fall 2021 | 1 (.07%) | 7 (4.9%) | 32 (22.4%) | 75 (52.4%) | 28 (19.6%) | 3.85 | .813 | |
| The course units are of varying lengths determined by the complexity of the learning objectives. | Spring 2020 | 5 (3.6%) | 13 (9.3%) | 46 (32.9%) | 63 (45%) | 13 (9.3%) | 3.47 | .917 |
| Fall 2020 | 1 (1.1%) | 4 (4.3%) | 37 (39.8%) | 44 (47.3%) | 7 (7.5%) | 3.56 | .744 | |
| Spring 2021 | 5 (3.85) | 4 (3.1%) | 21 (16.0%) | 78 (59.5%) | 23 (17.6%) | 3.84 | .884 | |
| Fall 2021 | 2 (1.4%) | 4 (2.8%) | 34 (23.8%) | 74 (51.7%) | 29 (20.3%) | 3.87 | .816 | |
| Each unit requires students to engage themselves in analysis, synthesis, and evaluation as part of their course assignments. | Spring 2020 | 6 (4.3%) | 11 (7.9%) | 48 (34.3%) | 62 13 (44.3%) (9.3%) | 3.46 | .924 | |
| Fall 2020 | 1 (1.1%) | 8 (8.6%) | 29 (31.2%) | 43 12 (46.2%) (12.9%) | 3.61 | .860 | ||
| Spring 2021 | 4 (3.1%) | 13 (9.9%) | 31 (23.7%) | 64 19 (48.9%) (14.5%) | 3.62 | .956 | ||
| Fall 2021 | 2 (1.4%) | 10 (7.0%) | 42 (29.4%) | 73 16 (51.0%) (11.2%) | 3.64 | .827 | ||
| Class voice mail, video conferencing, or email systems are provided to encourage students to work with each other and their instructor(s). | Spring 2020 | 7 (5.0%) | 32 (22.9%) | 41 (29.3%) | 53 7 (37.9%) (5.0%) | 3.15 | .996 | |
| Fall 2020 | 5 (5.4%) | 12 (12.9%) | 33 (35.5%) | 36 7 (38.7%) (7.5%) | 3.30 | .976 | ||
| Spring 2021 | 6 (4.6%) | 8 (6.1%) | 41 (31.3%) | 57 19 (43.5%) (14.5%) | 3.57 | .969 | ||
| Fall 2021 | 1 (0.7%) | 13 (9.1%) | 48 (33.6%) | 62 19 (43.4%) (13.3%) | 3.59 | .858 | ||
| Courses are designed to require students to work in groups utilizing problemsolving activities to develop an understanding of the topic. | Spring 2020 | 11 (7.9%) | 41 (29.3%) | 34 (24.3%) | 50 4 (35.7%) (2.9%) | 2.96 | 1.04 | |
| Fall 2020 | 6 (6.5%) | 20 (21.5%) | 38 (40.9%) | 22 7 (23.7%) (7.5%) | 3.04 | 1.01 | ||
| Spring 2021 | 9 (6.9%) | 29 (22.1%) | 35 (26.7%) | 44 14 (33.6%) (10.7%) | 3.19 | 1.110 | ||
| Fall 2021 | 2 (1.4%) | 24 (16.8%) | 48 (33.6%) | 58 11 (40.6%) (7.7%) | 3.36 | .900 | ||
| Student interaction with other students is facilitated through a variety (e.g., 1:1, group activities, projects, discussions) of ways. | Spring 2020 | 21 (15.0%) | 37 (26.4%) | 37 (26.4%) | 36 9 (25.7%) (6.4%) | 2.82 | 1.05 | |
| Fall 2020 | 7 (7.5%) | 21 (22.6%) | 26 (28.0%) | 28 11 (30.1%) (11.8%) | 3.61 | 1.14 | ||
| Spring 2021 | 13 (9.9%) | 25 (19.1%) | 34 (26.0%) | 47 12 (35.9%) (9.2%) | 3.15 | 1.140 | ||
| Fall 2021 | 1 (0.7%) | 31 (21.7%) | 45 (31.5%) | 53 13 (37.1%) (9.1%) | 3.32 | .939 | ||
| Course materials (i.e., books, PowerPoints, videos, and software) promote collaboration among students. | Spring 2020 | 20 (14.3%) | 46 (32.9%) | 39 (27.9%) | 32 3 (22.9%) (2.1%) | 2.66 | 1.05 | |
| Fall 2020 | 5 (5.4%) | 18 (19.4%) | 35 (37.6%) | 29 6 (31.2%) (6.5%) | 3.14 | .985 | ||
| Spring 2021 | 18 (13.7%) | 27 (20.6%) | 35 (26.7%) | 42 9 (32.1%) (6.9%) | 2.98 | 1.167 | ||
| Fall 2021 | 6 (4.2%) | 35 (24.5%) | 45 (31.5%) | 47 10 (32.9%) (7.0%) | 3.14 | 1.004 |
Note: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree, M = mean, SD = standard deviation.
Appendix C
Student Perceptions of Student Support
| Benchmarks | Semester | 1 | 2 | 3 | 4 | 5 | M | SD |
|---|---|---|---|---|---|---|---|---|
| Information (e.g., syllabus, software guides, tutorials) is supplied to students about their courses. | Spring 2020 | 2 (1.4%) | 12 (8.6%) | 29 (20.7%) | 79 (56.4%) | 18 (12.9%) | 3.71 | .852 |
| Fall 2020 | 1 (1.1%) | 4 (4.3%) | 30 (32.3%) | 45 (48.4%) | 13 (14.0%) | 3.70 | .805 | |
| Spring 2021 | 5 (3.8%) | 12 (9.2%) | 54 (41.2%) | 46 (35.1%) | 14 (10.7%) | 3.40 | .934 | |
| Fall 2021 | 2 (1.4%) | 14 (9.8%) | 66 (46.2%) | 50 (35.0%) | 11 (7.7%) | 3.38 | .821 | |
| Students can obtain assistance to help them use the course software (e.g., E-Class, WebEx, Zoom). | Spring 2020 | 4 (2.9%) | 18 (12.9%) | 42 (30.0%) | 69 (49.3%) | 7 (5.0%) | 3.41 | .881 |
| Fall 2020 | 2 (2.2%) | 14 (15.1%) | 33 (35.5%) | 37 (39.8%) | 7 (7.5%) | 3.35 | .905 | |
| Spring 2021 | 13 (9.9%) | 31 (23.7%) | 44 (33.6%) | 34 (26.0%) | 9 (6.9%) | 2.96 | 1.084 | |
| Fall 2021 | 8 (5.6%) | 35 (24.5%) | 53 (37.1%) | 41 (28.7%) | 6 (4.2%) | 3.01 | .964 | |
| A system is in place to address student complaints or difficulties with the course. | Spring 2020 | 5 (3.6%) | 29 (20.7%) | 48 (34.3%) | 53 (37.9%) | 5 (3.6%) | 3.17 | .921 |
| Fall 2020 | 7 (7.5%) | 14 (15.1%) | 38 (40.9%) | 28 (30.1%) | 6 (6.5%) | 3.13 | 1.00 | |
| Spring 2021 | 4 (3.1%) | 8 (6.2%) | 27 (20.6%) | 62 (47.3%) | 30 (22.9%) | 3.81 | .962 | |
| Fall 2021 | 2 (1.4%) | 10 (7.0%) | 30 (21.0%) | 76 (53.1%) | 25 (17.5%) | 3.78 | .865 | |
| Easily accessible technical support is available to students throughout the course. | Spring 2020 | 6 (4.3%) | 29 (20.7%) | 59 (42.1%) | 39 (27.9%) | 7 (5.0%) | 3.09 | .925 |
| Fall 2020 | 7 (7.5%) | 10 (10.8%) | 44 (47.3%) | 29 (31.2%) | 3 (3.2%) | 3.12 | .919 | |
| Spring 2021 | 5 (3.8%) | 16 (12.2%) | 58 (44.3%) | 40 (30.5%) | 12 (9.2%) | 3.29 | .932 | |
| Fall 2021 | 5 (3.5%) | 14 (9.8%) | 66 (46.2%) | 46 (32.2%) | 12 (8.4%) | 3.32 | .893 | |
| Students are provided with training or information to help them use course software, digital tools, applications, electronic databases, and websites. | Spring 2020 | 14 (10.0%) | 33 (23.6%) | 40 (28.6%) | 48 (34.3%) | 5 (3.6%) | 2.98 | 1.06 |
| Fall 2020 | 9 (9.7%) | 24 (25.8%) | 28 (30.1%) | 31 (33.3%) | 1 (1.1%) | 2.90 | 1.01 | |
| Spring 2021 | 10 (7.6%) | 22 (16.8%) | 55 (42.0%) | 38 (29.0%) | 6 (4.6%) | 3.06 | .975 | |
| Fall 2021 | 4 (2.8%) | 22 (15.4%) | 62 (43.4%) | 49 (34.3%) | 6 (4.2%) | 3.22 | .857 |
| Benchmarks | Semester | 1 | 2 | 3 | 4 | 5 | M | SD |
|---|---|---|---|---|---|---|---|---|
| Information (e.g., syllabus, software guides, tutorials) is supplied to students about their courses. | Spring 2020 | 2 (1.4%) | 12 (8.6%) | 29 (20.7%) | 79 (56.4%) | 18 (12.9%) | 3.71 | .852 |
| Fall 2020 | 1 (1.1%) | 4 (4.3%) | 30 (32.3%) | 45 (48.4%) | 13 (14.0%) | 3.70 | .805 | |
| Spring 2021 | 5 (3.8%) | 12 (9.2%) | 54 (41.2%) | 46 (35.1%) | 14 (10.7%) | 3.40 | .934 | |
| Fall 2021 | 2 (1.4%) | 14 (9.8%) | 66 (46.2%) | 50 (35.0%) | 11 (7.7%) | 3.38 | .821 | |
| Students can obtain assistance to help them use the course software (e.g., E-Class, WebEx, Zoom). | Spring 2020 | 4 (2.9%) | 18 (12.9%) | 42 (30.0%) | 69 (49.3%) | 7 (5.0%) | 3.41 | .881 |
| Fall 2020 | 2 (2.2%) | 14 (15.1%) | 33 (35.5%) | 37 (39.8%) | 7 (7.5%) | 3.35 | .905 | |
| Spring 2021 | 13 (9.9%) | 31 (23.7%) | 44 (33.6%) | 34 (26.0%) | 9 (6.9%) | 2.96 | 1.084 | |
| Fall 2021 | 8 (5.6%) | 35 (24.5%) | 53 (37.1%) | 41 (28.7%) | 6 (4.2%) | 3.01 | .964 | |
| A system is in place to address student complaints or difficulties with the course. | Spring 2020 | 5 (3.6%) | 29 (20.7%) | 48 (34.3%) | 53 (37.9%) | 5 (3.6%) | 3.17 | .921 |
| Fall 2020 | 7 (7.5%) | 14 (15.1%) | 38 (40.9%) | 28 (30.1%) | 6 (6.5%) | 3.13 | 1.00 | |
| Spring 2021 | 4 (3.1%) | 8 (6.2%) | 27 (20.6%) | 62 (47.3%) | 30 (22.9%) | 3.81 | .962 | |
| Fall 2021 | 2 (1.4%) | 10 (7.0%) | 30 (21.0%) | 76 (53.1%) | 25 (17.5%) | 3.78 | .865 | |
| Easily accessible technical support is available to students throughout the course. | Spring 2020 | 6 (4.3%) | 29 (20.7%) | 59 (42.1%) | 39 (27.9%) | 7 (5.0%) | 3.09 | .925 |
| Fall 2020 | 7 (7.5%) | 10 (10.8%) | 44 (47.3%) | 29 (31.2%) | 3 (3.2%) | 3.12 | .919 | |
| Spring 2021 | 5 (3.8%) | 16 (12.2%) | 58 (44.3%) | 40 (30.5%) | 12 (9.2%) | 3.29 | .932 | |
| Fall 2021 | 5 (3.5%) | 14 (9.8%) | 66 (46.2%) | 46 (32.2%) | 12 (8.4%) | 3.32 | .893 | |
| Students are provided with training or information to help them use course software, digital tools, applications, electronic databases, and websites. | Spring 2020 | 14 (10.0%) | 33 (23.6%) | 40 (28.6%) | 48 (34.3%) | 5 (3.6%) | 2.98 | 1.06 |
| Fall 2020 | 9 (9.7%) | 24 (25.8%) | 28 (30.1%) | 31 (33.3%) | 1 (1.1%) | 2.90 | 1.01 | |
| Spring 2021 | 10 (7.6%) | 22 (16.8%) | 55 (42.0%) | 38 (29.0%) | 6 (4.6%) | 3.06 | .975 | |
| Fall 2021 | 4 (2.8%) | 22 (15.4%) | 62 (43.4%) | 49 (34.3%) | 6 (4.2%) | 3.22 | .857 |
Note: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree, M = mean, SD = standard deviation.
Appendix D
Student Perceptions of Course Structure
| Benchmarks | Semester | 1 | 2 | 3 | 4 | 5 | M | SD |
|---|---|---|---|---|---|---|---|---|
| Students are provided with basic course information that outlines course objectives, concepts, and ideas. | Spring 2020 | 1 (0.7%) | 3 (2.1%) | 32 (22.9%) | 82 (58.6%) | 22 (15.7%) | 3.86 | .721 |
| Fall 2020 | 0 (0%) | 2 (2.2%) | 16 (17.2%) | 56 (60.2%) | 19 (20.4%) | 3.99 | .684 | |
| Spring 2021 | 2 (1.5%) | 5 (3.8%) | 13 (9.9%) | 73 (55.7%) | 38 (29.0%) | 4.07 | .825 | |
| Fall 2021 | 2 (1.4%) | 2 (1.4%) | 22 (15.4%) | 76 (53.1%) | 41 (28.7%) | 4.06 | .789 | |
| Sufficient resources are available to the students to complete class assignments, tasks, and projects. | Spring 2020 | 5 (3.6%) | 13 (9.3%) | 37 (26.4%) | 71 (50.7%) | 14 (10.0%) | 3.54 | .924 |
| Fall 2020 | 0 (0%) | 10 (10.8%) | 23 (24.7%) | 48 (51.6%) | 12 (12.9%) | 3.67 | .838 | |
| Spring 2021 | 3 (2.3%) | 10 (7.6%) | 38 (29.0%) | 66 (50.4%) | 14 (10.7%) | 3.60 | .866 | |
| Fall 2021 | 3 (2.1%) | 6 (4.2%) | 43 (30.1%) | 70 (49.0%) | 21 (14.7%) | 3.70 | .848 | |
| Specific expectations are set for students with respect to a minimum amount of time per week for study and homework assignments. | Spring 2020 | 2 (1.4%) | 25 (17.9%) | 31 (22.1%) | 70 (50%) | 12 (8.6%) | 3.46 | .932 |
| Fall 2020 | 1 (1.1%) | 7 (7.5%) | 27 (29.0%) | 44 (47.3%) | 14 (15.1%) | 3.68 | .862 | |
| Spring 2021 | 7 (5.3%) | 15 (11.5%) | 39 (29.8%) | 54 (41.2%) | 16 (12.2%) | 3.44 | 1.024 | |
| Fall 2021 | 3 (2.1%) | 11 (7.7%) | 40 (28.0%) | 69 (48.3%) | 20 (14.0%) | 3.64 | .891 | |
| Learning outcomes for each course are summarized in clearly written, straight forward statements. | Spring 2020 | 5 (3.6%) | 20 (14.3%) | 46 (32.9%) | 59 (42.1%) | 10 (7.1%) | 3.35 | .936 |
| Fall 2020 | 0 (0%) | 10 (10.8%) | 27 (29.0%) | 50 (53.8%) | 6 (6.5%) | 3.56 | .773 | |
| Spring 2021 | 2 (1.5%) | 10 (7.6%) | 34 (26.0%) | 69 (52.7%) | 16 (12.2%) | 3.66 | .847 | |
| Fall 2021 | 3 (2.1%) | 6 (4.2%) | 41 (28.7%) | 71 (49.7%) | 22 (15.4%) | 3.72 | .851 | |
| Faculty are required to grade and return all assignments within a certain time period. | Spring 2020 | 9 (6.4%) | 25 (17.9%) | 35 (25.0%) | 54 (38.6%) | 17 (12.1%) | 3.32 | 1.10 |
| Fall 2020 | 0 (0%) | 7 (7.5%) | 27 (29.0%) | 46 (49.5%) | 13 (14.0%) | 3.70 | .805 | |
| Spring 2021 | 3 (2.3%) | 8 (6.1%) | 32 (24.4%) | 69 (52.7%) | 19 (14.5%) | 3.71 | .873 | |
| Fall 2021 | 1 (0.7%) | 11 (7.7%) | 34 (23.8%) | 74 (51.7%) | 23 (16.1%) | 3.75 | .843 |
| Benchmarks | Semester | 1 | 2 | 3 | 4 | 5 | M | SD |
|---|---|---|---|---|---|---|---|---|
| Students are provided with basic course information that outlines course objectives, concepts, and ideas. | Spring 2020 | 1 (0.7%) | 3 (2.1%) | 32 (22.9%) | 82 (58.6%) | 22 (15.7%) | 3.86 | .721 |
| Fall 2020 | 0 (0%) | 2 (2.2%) | 16 (17.2%) | 56 (60.2%) | 19 (20.4%) | 3.99 | .684 | |
| Spring 2021 | 2 (1.5%) | 5 (3.8%) | 13 (9.9%) | 73 (55.7%) | 38 (29.0%) | 4.07 | .825 | |
| Fall 2021 | 2 (1.4%) | 2 (1.4%) | 22 (15.4%) | 76 (53.1%) | 41 (28.7%) | 4.06 | .789 | |
| Sufficient resources are available to the students to complete class assignments, tasks, and projects. | Spring 2020 | 5 (3.6%) | 13 (9.3%) | 37 (26.4%) | 71 (50.7%) | 14 (10.0%) | 3.54 | .924 |
| Fall 2020 | 0 (0%) | 10 (10.8%) | 23 (24.7%) | 48 (51.6%) | 12 (12.9%) | 3.67 | .838 | |
| Spring 2021 | 3 (2.3%) | 10 (7.6%) | 38 (29.0%) | 66 (50.4%) | 14 (10.7%) | 3.60 | .866 | |
| Fall 2021 | 3 (2.1%) | 6 (4.2%) | 43 (30.1%) | 70 (49.0%) | 21 (14.7%) | 3.70 | .848 | |
| Specific expectations are set for students with respect to a minimum amount of time per week for study and homework assignments. | Spring 2020 | 2 (1.4%) | 25 (17.9%) | 31 (22.1%) | 70 (50%) | 12 (8.6%) | 3.46 | .932 |
| Fall 2020 | 1 (1.1%) | 7 (7.5%) | 27 (29.0%) | 44 (47.3%) | 14 (15.1%) | 3.68 | .862 | |
| Spring 2021 | 7 (5.3%) | 15 (11.5%) | 39 (29.8%) | 54 (41.2%) | 16 (12.2%) | 3.44 | 1.024 | |
| Fall 2021 | 3 (2.1%) | 11 (7.7%) | 40 (28.0%) | 69 (48.3%) | 20 (14.0%) | 3.64 | .891 | |
| Learning outcomes for each course are summarized in clearly written, straight forward statements. | Spring 2020 | 5 (3.6%) | 20 (14.3%) | 46 (32.9%) | 59 (42.1%) | 10 (7.1%) | 3.35 | .936 |
| Fall 2020 | 0 (0%) | 10 (10.8%) | 27 (29.0%) | 50 (53.8%) | 6 (6.5%) | 3.56 | .773 | |
| Spring 2021 | 2 (1.5%) | 10 (7.6%) | 34 (26.0%) | 69 (52.7%) | 16 (12.2%) | 3.66 | .847 | |
| Fall 2021 | 3 (2.1%) | 6 (4.2%) | 41 (28.7%) | 71 (49.7%) | 22 (15.4%) | 3.72 | .851 | |
| Faculty are required to grade and return all assignments within a certain time period. | Spring 2020 | 9 (6.4%) | 25 (17.9%) | 35 (25.0%) | 54 (38.6%) | 17 (12.1%) | 3.32 | 1.10 |
| Fall 2020 | 0 (0%) | 7 (7.5%) | 27 (29.0%) | 46 (49.5%) | 13 (14.0%) | 3.70 | .805 | |
| Spring 2021 | 3 (2.3%) | 8 (6.1%) | 32 (24.4%) | 69 (52.7%) | 19 (14.5%) | 3.71 | .873 | |
| Fall 2021 | 1 (0.7%) | 11 (7.7%) | 34 (23.8%) | 74 (51.7%) | 23 (16.1%) | 3.75 | .843 |
Note: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree, M = mean, SD = standard deviation.




