Evaluation of a technical information systems module for distance learning during the COVID-19 pandemic

While there is optimism about the end of the COVID-19 pandemic in some parts of the world, other areas are still experiencing several waves and variants. Even after the significant advancement of vaccination, recovery from the pandemic lingers on (Alwan, 2021). As new virus variations are discovered and COVID-19 cases show seasonality (Merow and Urban, 2020), schools cannot just transition to the pre-pandemic mode. The educators and researchers are uncertain of the best teaching modality. Some research suggests that effective and efficient learning management systems (LMS) would be a key reason for sustainable education (Cavus et al., 2021). Other research finds positive relationship between massive open online courses and students’ learning performance during the pandemic (Yang and Lee, 2021). Institutes can better prepare for the uncertain future by learning from the successful strategies during the pandemic.

The COVID -19 pandemic quickly led to the closure of schools worldwide. The schools were hoping that public health officials' advice of social distancing could help flatten the infection curve and reduce total fatalities from the disease (Murphy, 2020). However, as the world struggled to control the pandemic spread, in-person classes were suspended for the long-term. As a substitute for classroom instruction, many universities promoted and relied on online learning (Brady and Pradhan, 2020).

This change in teaching modality created significant disruptions in all courses, including the technical courses. While technical classes in information systems (IS), such as databases, web development and app development were often delivered via diverse modalities (Alavi and Gallupe, 2003), the pandemic posed different challenges. Prior to the pandemic, most of the technical classes were already carefully planned for different modalities. The schools needed to ensure that the information technologies were accessible to everyone. Stating differently, most schools and students were both prepared for online classes. During the pandemic, schools had less time to prepare, and students essentially did not have any choice but to adhere to these changes (Sahlberg, 2020).

Within a short span of the COVID-19 pandemic, many researchers have contributed their works on teaching and learning. Studies show that a flexible approach in designing and delivering courses strive to prepare the students for the changing dynamics of an unpredictable world (Johnson-Eilola and Selber, 2021). The schools need to address different learning styles by encouraging educators to present their curriculum and learning outcomes using various modalities and exercises.

For technical courses, students often need to access specialized software. During the pandemic, school resources are strained, and many students may not have access to these critical technical resources. To address these issues, this research paper focuses on: How can diverse learning modalities be utilized to support the mission of IS education during a pandemic?

While IS programs are taught differently across universities, some are more technocentric, while others are business oriented. The IS 2010 curriculum guidelines proposes program-level learning outcomes for IS programs (Topi et al., 2010). This model curriculum states that the intended student success outcome is to prepare students for career tracks such as application developer, business analyst, business process analyst, database administrator, etc. (Topi et al., 2010). Core competencies for these careers include (Topi et al., 2017) Data, information, and content management, IT infrastructure, IS management and operations and systems development.

To develop these competencies, the students need to gain hands-on experiences with tools and technology. The pandemic poses challenges in providing access and supporting these technologies to students. We examine the delivery of an IS course and evaluate the student assessment to address the following research questions: RQ1: What different tools and technologies are suitable for synchronous and asynchronous modalities? RQ2: What modalities and modules are impacted more during the pandemic?

We utilized some prominent sources such as EBSCO host and Business Source Premier Search to conduct the literature review. We excluded the articles that concentrated on specific training and challenges related to the medical, engineering and other specialized fields.

The articles retrieved in the search results focused on public health policies and the response of the universities to these initial closures. The primary reason for the closure of schools around the world was to contain the pandemic spread. The government agencies, including public health officials, promoted social distancing to flatten the infection curve and reduce total fatalities from the disease (Murphy, 2020). The US government created guidelines for back-to-school plans and high-risk exposure for teachers and students (Bailey et al., 2020). However, the pandemic did not subside, and educators became hesitant to return to the classroom. For example, 79 educators in Utah have retired or resigned over COVID-19 concerns (Bogus and Lowe, 2020). As returning to the classroom became difficult, the schools began offering online learning to substitute for classroom instruction (Dhawan, 2020).

Spring 2020 was a tumultuous time for educators and learners. The risk and uncertainty caused by the COVID-19 outbreak forced higher education administrations to close campuses and sent faculty members scrambling to move their courses online (Chen et al., 2005). The various stakeholders in the educational institutes became concerned about the extraordinary challenge of educating students and keeping everyone as safe as possible during the COVID-19 pandemic in the US. Moreover, there are other pedagogical issues related to teaching during the pandemic in addition to dealing with the teaching modalities issues. For example, many universities rely on school libraries to provide critical support for student learning outcomes. The academic libraries need to expand on their role and responsibilities as a result of the outbreak of Covid-19 (Ifijeh and Yusuf, 2020).

Along with pedagogical issues, psychological and behavioral health policies are critical in the successful public health responses to the COVID-19 pandemic (Glassgold and Wolff, 2020). Given the social, economic and emotional strain of COVID-19 to families and communities, it is apparent that the students’ focus on other things outside studies is understandable. It has been found that students who undertake coursework in such emergency contexts may deprioritize their studies due to the challenges they face outside the classroom (Banki, 2021).

While the universities are trying to cope with new changes due to the pandemic, they also need to devise new strategies for a secure future. Currently, the growth of online education during the pandemic has put extraordinary stress on campus technologies. Many schools struggled to expand broadband to support online education (Garbe et al., 2020). A study suggests employing more open-source technology in an online instruction during the lockdown to alleviate some of the stress on campus technology (Delaney et al., 2020).

According to Chakraborty et al. (2020), professors typically use presentation programs and note-taking programs to distribute information. Chakraborty et al. (2020) mentions that PowerPoint is found suitable to deliver a lecture following a predetermined sequence, whereas Prezi is used to make the session more interactive instead of following a fixed sequence of topics. OneNote can help a professor and students to collaborate. Furthermore, devices like digital pens are found useful for lectures.

The education institutions need to realize that the mobility of people during a lockdown is hampered (Fogarty, 2020). The traditional, large, classroom-based instructions on campuses or structured buildings are closed, presenting the universities with and opportunities to innovate for the future (Seke, 2020). Even before the pandemic, issues such as uninvolved learning, and one-size-fits-all teaching were issues for institutions. The challenges of traditional schooling have become more prominent during the pandemic (Lim-Lange, 2020). If institutions are successful in addressing all the concerns of pedagogical issues during pandemic, COVID-19 can be seen as “a new educational revolution” (Seke, 2020).

To get to this “new educational revolution”, schools first need to adopt a flexible teaching approach to better prepare for the changing dynamics of an unpredictable world (Johnson-Eilola and Selber, 2021). A recent study showed that during online sessions, students faced difficulty accessing school computer lab remotely to complete the assignments (Boton, 2020).

In Figure 1, we discuss our research framework. Figure 1 illustrates the two types of learning techniques that are mostly implemented during a pandemic. These include synchronous and asynchronous online learning methods. During a pandemic, a school may provide limited access to technology and other campus resources. Some students may thus have limited access to technology such as laptops and the Internet due to the affordability issue causing a learning disruption.

Online education has been gaining attention since it provides flexibility in student access to instruction and lowering costs associated with education (Graham, 2006; Taplin et al., 2013). Online education has now become a popular topic of discussion and research literature related to higher education (Young, 2011).

The mode of interaction for online instruction can be categorized into asynchronous and synchronous. In the asynchronous learning model, an instructor interacts with the students at different times (Chen et al., 2004). Synchronous learning requires the presence of both the instructor and students at the same time for the learning to take place (Chen et al., 2005).

During the pandemic, the minority population in most locations was impacted more severely (Tai et al., 2021). These students faced greater challenges regarding family and work obligations, anxiety and stress of pandemic and maintaining good academic standing (García and Weiss, 2020).

Much of the current scholarly discussions related to designing online learning environments are focused primarily on asynchronous communications (Yamagata-Lynch, 2014). Previous literature has found that the asynchronous learning model leads to improved educational outcomes compared with no educational intervention (Brady and Pradhan, 2020). Moreover, Borthick and Jones (2000) argue that learning method called “collaborative discovery” that is a combination of synchronous and asynchronous learning, is a better form of learning compared to lecture-based instruction for students to develop problem-solving skills. The authors apply this form of learning strategy to an IS assurance course. The learning objectives for the course are for students to learn to identify risks, evaluate internal control, develop and implement assurance plans for automated IS. All the resources for the course, organized by class session, are available from websites without using any specific textbook for the course. These resources include explaining the activity that students had to work on for each class session (Borthick and Jones, 2000).

A study in 2013 identified several technical skills for IT professionals, including cybersecurity, mobile app development, data analytics, social networking technologies, network infrastructure, to name a few (Legier et al., 2013). These learning outcomes align with the career workforce objectives recommended in the IS 2010 curriculum guide (Topi et al., 2010). The model in this study is applied to the delivery of a technical module in the IS program, specifically mobile app development using the Mendix platform.

Transactional distance is a pedagogical concept that learners at a distance from their instructors and peers experience through their interactions with each other (Moore, 1993). Within the literature related to synchronous communications, there is little discussion regarding how to support learners in synchronous online learning environments effectively (Asterhan and Schwarz, 2010). According to Chakraborty et al. (2020), videoconferencing tools, like Google Meet, Zoom and Microsoft Teams help in organizing online lectures and discussion sessions.

In terms of video conferencing in synchronous instruction, Han (2013) examined the effects of instructor video casting on the students’ sense of connection to the instructor. Han found that in courses that included instructor video casting, compared to courses that did not use video casting, students were able to overcome the feeling of being at a distance from the instructor.

Content can be delivered synchronously via several available videoconferencing platforms (Zoom, Webex, GoToMeeting and others). Table 1 discusses the various synchronous videoconference platforms. Zoom, Webex and GoToMeeting platforms allow for limited synchronous meeting time even with free user accounts. These synchronous tools are secure given the security features they use which are listed in Table 1. PowerPoint slide presentations work well on any of the major platforms. Some video conferencing platforms have a “whiteboard” function that will allow for creating a virtual chalk talk.

Studies about asynchronous online learning state that students will experience meaningful learning when they are in learning environments that have opportunities of engagement (Palloff and Pratt, 2007). These environments are designed to help participants develop a sense of engagement in collaborative discussions (Conrad and Donaldson, 2011; Lehman and Conceição, 2010).

A study compared two offerings of a business communications course including only asynchronous communication and one that also included synchronous chat. Analyzing synchronous chat, the author found two non-overlapping types of students: those that frequently used the chat and those that did not. The subset of students that were chat adopters felt a strong sense of connection (Hrastinski, 2006).

Chakraborty et al. (2020) state that online educational platforms such as Google Classroom and Blackboard allow professors to share notes and other resources related to their courses with students. The online educational platforms also allow students to turn in their assignments and in turn help professors to keep track of the student assignments. Some universities are also sharing course material through their websites (Chatterjee and Chakraborty, 2021) and their own LMS (Mishra et al., 2020).

Table 2 shows the comparison of some popularly used LMS, namely, Blackboard, Moodle and Canvas. All the three platforms are mobile accessible. It is possible to download and read the study material offline in all the three platforms.

When the school shifted to an online teaching modality, many of the students did not have adequate access to computer hardware and software. Against this challenging backdrop, the IS instructors had to adapt delivering courses that help students grow professionally and gain industry-relevant skills.

The IS course we discuss in this paper contains several applied components such as design, analysis and prototype development using a group project. The course component includes students working on hands-on exercises using a low code programming platform called Mendix.

Assessments are an important part of the educational process. However, there are challenges to performing assessment effectively in the online environment (Liang and Creasy, 2004), as online assessment requires a more methodical approach than traditional ones (Robles and Braathen, 2002).

Online assessment techniques are found to be effective if realistic scenarios are used in setting learning objectives (Boyle et al., 2003). Examples of types of assessments that are indicated as effective in the online environment are exams, timed tests/quizzes, self-tests, weekly review questions, homework/written assignments/field work, case study responses, peer evaluation with feedback, projects, short essays, group work, presentations and involvement in discussion forums, to name some (Robles and Braathen, 2002; Gaytan and McEwen, 2007; Kearns, 2012; Huang et al., 2020).

In this paper, we discuss the delivery of an app project (using Mendix) via remote delivery. The implementation of the project also involves students to complete online training modules, participate in course discussion forums and to present their project work in synchronous class presentations.

The IS course discussed in this paper is a three-credit undergraduate course on Introduction to IS for the business majors. We examined the student performance before and during the pandemic in the same course. For the study, we considered the student performance in the spring semester of 2019 which was the term prior to the pandemic and the spring semester of 2020 which was the term during the pandemic. During the spring semester of 2019, 44.4% were first-generation college students in comparison to 27.8% in the spring semester of 2020. The students were above 18 years of age in both the terms we considered for our study. The ages of students ranged between 19 and 44 for both the terms, with an average age of 26.1 years for the spring semester of 2019 and an average age of 25 years for the spring semester of 2020.

To prepare for the applied portion of the course, students were assigned to watch Mendix training video and complete the online Mendix training. Students completed this part of the assignment on their own time. The instructor could track the student progress in the Mendix platform.

Once the students completed the online training and certification process, students began working on their course projects in Mendix. Students were placed in groups for the entirety of the semester to work on the semester long project. Students started with the online Mendix training that contains an exemplar app. Students need to modify this app to fit the project instructions that are detailed below.

You are a new employee at a non-profit organization that helps children get a better education by sponsoring workshops and lectures across the country. The events are growing at a rapid pace, and it’s been challenging to keep track of all the logistics. In addition, the donors for the organizations are very generous and have been willing to give more. However, it’s been challenging to keep track of all the donations. Your manager is asking you to come up with a solution to keep track of all the information and automate/improve the processes.

Develop a mobile app (iPhone or android and/or tablet) app for the scope defined.

App desired features are provided below:

1. Donors’ portal where they can login and see all of their donations and how their money is being spent.

   • You can create a generic donor user role and a generic employee to showcase the features and functionalities.

2. Employee portal where they can login and do the following:

   • Track the donors, their donations and update any relevant information.

   • Track the events they do and the benefits those events provide.

   • Track the cost of the events and the donors funding for those events.

3. Attendees can register for upcoming events hosted by the organization:

   • Students do not need to set up a user account for the attendees to be able to login.

4. All stakeholders want to know everything on their phone so they can access it on the go.

The first deliverable for students was to develop their use cases or user stories in Mendix. The students started with the requirements for application functionalities, and then these requirements were translated into the user stories into Mendix. In prior semesters, the students needed to download the Microsoft Visio software or work in the campus lab to create a story map in Visio. Creating the user stories directly into the Mendix simplified the process for students.

Next, the students created a domain model, the equivalent of the database tables in Mendix. However, during the pandemic, students did not have access to the campus labs and ensuring that every student has access to the proper database tools were not feasible. Mendix cloud platform simplified the technical aspects of the project. Once the domain model was built, student completed the design of their prototype in Mendix.

Finally, the students delivered well-structured, concise and professional presentations providing details of the application development process and lessons learned. They all agreed that use of Mendix contributed positively to their learning experience. After the project was completed, the instructor asked students to reflect on whether they found the hands-on exercise on Mendix useful and would they use Mendix again in other courses if we offer it.

The event app was graded based on the categories of program utility, appropriateness, usability and graphical design. Program utility and appropriateness refer to the software functionalities that solve a business problem, look and feel of the prototype suitable for the target users and feasibility of being deployed in target users’ environment. Usability and graphical design relate to designing for a greater user experience.

By adapting to a low-code platform-independent development tool such as Mendix, IS course can address the significant concerns raised by abrupt online modality. More specifically, our research question concerns two facets of IS education: designing an IS course that aligns with the Model IS curriculum and delivering an IS course during the pandemic. We report that by utilizing Mendix, we aligned the course assignments with the model IS curriculum. The results are summarized in Table 3. We then compared the student performance via assessment data during the semester disrupted by pandemic to the pre-pandemic.

The first objective of the model IS curriculum is data and information management that seeks to develop perspectives related to the “design and utilization of information and organizational processes across functional areas (Topi et al., 2010, p. 28)”. The course assignments require students to examine several functional areas of an organization, such as employee functionalities and donor roles. The students also analyze the organizational processes such as fundraising and event management. The students finally develop entities and attributes in Mendix domain model by analyzing the organizational processes across functional areas.

The second objective of the model IS curriculum relates to this research is IT Infrastructure. The model IS curriculum divides infrastructures into two broad categories. The first states that “the classrooms equipped with computer projection, Internet and local network access, and appropriate computing and software infrastructure, so that the entire curriculum can be adequately delivered (Gorgone et al., 2003, p. 2). Since all the classes were held online, the students could not rely on classroom technologies. By utilizing various synchronous and asynchronous technologies, the instructors were able to deliver the technical module. The second relates to the core topic areas for the IT infrastructure are computers and communication networks that include cloud computing. By utilizing Mendix, students learned about cloud hosting, infrastructures needed for low-code development platform. In addition, the students completed a full software development life cycle.

The third and the fourth objectives of the model IS curriculum deal with Systems Development and IS Management and Operation. The issues related to this objective seek to develop “approaches in managing the information systems function in organizations and how the IS function integrates/supports/enables various types of organizational capabilities (Topi et al., 2010, p. 54)”. In this regard, the course assignment was designed to plan, analyze and implement their prototypes. During the planning stage, the students examine the organizational goals (engaging donors, managing events, etc.). In analysis, students examined how to integrate various organizational processes, such as, fundraising, managing donors and others. Utilizing synchronous and asynchronous modalities, we were able to deliver the technical module to the students adhering to the model IS curriculum. Low code development platform Mendix fits well with these modalities as well as the curriculum.

The second part of the research question examines how we deliver IS education during the pandemic. In our research, we examined two methods of course delivery: asynchronous and asynchronous. Using Mendix as an example, we summarize how the proposed framework facilitates course delivery during the pandemic in Table 4.

For the asynchronous part, the students were directed to complete the self-paced online training provided via the Mendix academic portal. There are more than 25 separate learning paths available for students. For the asynchronous part, students create their own prototype ideas based on their synchronous training.

Finally, we used assessment data to compare the student performance on these modules during the pandemic to the pre-pandemic semester. We compared the results from the same IS course. All IS students who completed this teaching module demonstrated their understanding of the key concepts of no-code business app development. Table 3 exhibits the course assignments designed to match model IS curriculum. In Table 4, we describe how synchronous and asynchronous modalities were utilized.

As indicated in Table 5, there were 29 students were enrolled in pre-pandemic semester (Spring semester of 2019) vs. 27 during the pandemic (Spring semester of 2020). During both the semesters, five modules, video training, domain model, SDLC (systems development life cycle) development, software design process and prototype development utilized asynchronous modality. During these semesters, presentations and participations were recorded in synchronous sessions. The students earning 80% or better grades were identified as students performing satisfactorily in both the pre-pandemic and pandemic semesters. The numerical grades were converted to a five-point range to report the average grades. The student performance is assessed on a 5-point scale (1 being low and 5 being high) for the synchronous and the asynchronous tasks related to the course.

Results indicate that the students’ performances were lower for all modules during the pandemic compared to the pre-pandemic semester. There were some differences between the synchronous asynchronous modality. The synchronous modality saw the biggest drop in the number of satisfactory student performance from the non-pandemic semester to the pandemic semester. During the pandemic, the performance in both synchronous modules, in-class presentations and participations and discussion dropped by 15%. Furthermore, the average score for in-class presentations dropped by 10% and that of the participation and discussions lowered by 8% points.

The drop in the performance of the students and the average points were not as severe in asynchronous modules. For instance, for the on-demand video training and domain modules, the performance of the students during the pandemic declined by 3% and the average score dropped by 0.2 points. The bigger drop (10%) occurred for the software design and prototype development modules. Furthermore, it was observed during the pandemic that the average score also dropped by a range of 6%–10% in these SDLC and prototype development modules.

Over the past 10 years, universities and business schools have increased online classes. The number of students that are taking online classes is also increased. During the “normal” time, students can choose in-person or online classes. However, in this pandemic, educators and students did not have a choice. Both the synchronous and asynchronous modes are affected by the pandemic. One of the key factors considered in this study are the different modalities. However, the attributes of different learners could also explain the drop in the assessment scores. The future study should consider other variables such as learner’s attributes. Several studies suggest that social presence is higher in synchronous communication than in asynchronous discussion. For example, Schwier and Balbar (2002) discuss and compare asynchronous discussion to synchronous chat in a graduate education course. The authors describe that synchronous chat contributed to the continuity of the class, helped sustain regular contact, and created a sense of urgency and immediacy. Students perceived more social presence in synchronous chat than in asynchronous media such as email. Successfully delivering an IS course requires institutions to address concerns outlined in our research framework (Schwier and Balbar, 2002).

The reality of the new normal, disrupted by COVID effects, has involved a major transformation of education. The sudden closure of face-to-face teaching led academics and students into “unfamiliar terrain” with the need to adapt rapidly to fully remote (online) settings (Carolan et al., 2020). This sudden change required universities to evolve toward online teaching utilizing the resources available.

During the pandemic, both the students and instructors, were forced to adapt to the new teaching modality delivered online. Mendix allowed us to provide education similar to the in-person lab course. Although the performance was a little lower during the pandemic, Mendix alleviates the learning disruptions because the platform is available online and offline. Moreover, many of the learning content can be downloaded and read separately. During a pandemic, universities may struggle to offer technical support to all the students. By offloading some of the campus-based technology to academic-supported platforms such as Mendix, we were able to deliver the IS course without interruption.

Despite the challenges, universities in the United States were able to deliver the courses and help students to graduate on time. While this pandemic seems to be coming to an end in the United States, other parts of the world are still experiencing new waves of the virus. The lessons learned from transitioning to the online modality can help universities better prepare for the future. Previously, e-learning and distance education were popularly considered as part of non-formal education (Mishra et al., 2020). However, with the aftermath of the COVID-19 crisis, online education has now seen a pedagogical shift from the traditional method of teaching face-face classes to teaching on Zoom and other online platforms (Mishra et al., 2020).

This study focused mainly on delivering an IS module aligned with the IS curriculum model during the pandemic. The issue of the digital divide and equity needs further examination. Our study affirms the importance of technology adoption in education. This is essential to limit the effects of coronavirus and other future pandemics or unprecedented situations in education.