An Analysis of Technological Issues Emanating from Faculty Transition to a New Learning Management System

Internet-based learning management systems (LMSs) such as Blackboard, Moodle, WebCT, Canvas, Scholar, and Desire2Learn are some of the popular Internet technologies that support distance, face-to-face, and hybrid/blended teaching-learning processes (Connolly, MacArthur, Stansfield, & McLellan, 2007; Dahlstrom, Brooks, & Bichsel, 2014; DeNeui & Dodge 2006; El Mansour & Mupinga 2007; McGill & Hobbs, 2008). An LMS can be defined as “a self-contained webpage with embedded instructional tools that permit faculty to organize academic content and engage students in their learning” (Gautreau, 2011, p. 2). Again, Alias and Zainuddin (2005) an LMS as “a software application or web-based technology used to plan, implement, and assess a specific learning process” (p. 28). Another definition still, looks at LMSs as web-based technologies that provide instructors with a way to create and deliver content, to monitor student participation and engagement, and to assess student performance online (Venter, van Rensburg, & Davis, 2012). What is common in all these definitions is that an LMS is a web-based application that supports teaching and learning by enabling instructors to create and organize content for learners.

LMSs are a technology that enables the communication of course expectations through various resources such as a syllabus, as well as of assignment instructions, grades, and instructional materials (Rubin, Fernandes, Avgerinou, & Moore, 2010). As Bonk and Reynolds (1997) observed, the paradigm shift from traditional educational environments to online educational environments in higher education can also be seen as a challenge to create an active and interactive learning environment, one that gives the learner opportunity to engage and think in multiple ways. In a study that investigated technology adoption into teaching and learning by university faculty, for example, Nicolle (2005) found the link between effective teaching and the use of technology to be critical in helping faculty through the process of integration. As Baia (2009) observed, university faculty members are concerned with effective teaching; hence, if they perceive technology as having a positive impact toward this effort, they are likely to get motivated to integrate it in their teaching.

Several scholars have investigated how faculty and students value and use an LMS in teaching and learning. Pajo and Wallace (2001) stressed that successful integration of technology in teaching depends not only on availability of technology but also on how instructors embrace and use it. In a survey on faculty attitudes on technology, most faculty reported using an LMS, but using limited features as follows: posting course syllabus (78%), recording grades (58%), and communicating with students (52%). Only 20% of faculty reported using the LMS to record lecture content (Jaschik & Lederman, 2014). Recent LMS studies suggest that a variety of system issues, such as suitability of design in screen and system, easiness of course procedure, interoperability of system, easiness of instruction management and appropriateness of multimedia use, flexibility of interaction and test, learner control, variety of communication and test types and user accessibility are important LMS features that directly or indirectly benefit LMSs or e-learning users and influence their attitudes toward LMSs (Fathema & Sutton, 2013; Kim & Leet, 2008; Panda & Mishra, 2007; Pituch & Lee, 2006; Russell, Bebell, O’Dwyer, & O’Connor, 2003; Weaver, Spratt, & Nair, 2008). The literature further indicates that other studies on LMS have focused on how faculty and students value and use an LMS in teaching and learning. Yet more studies have focused on faculty perceptions on the whole transition process to a new LMS. Against this background, it was found necessary to also identify specific technological issues faculty members encounter while transitioning to a new LMS and how such issues can best be mitigated.

LMSs enable the communication of course expectations through various resources such as a syllabus, as well as of assignment instructions, grades, and instructional materials (Rubin et al., 2010). The present study investigated the process 119 faculty members at a state university in the southern part of the United States went through as they transitioned from using Desire to Learn (D2L) learning management system to using Canvas. The study did not only analyze issues faculty encountered while navigating aspects of the Canvas interface per se, but also issues they came across while integrating other technological applications that work in Canvas. So, apart from general Canvas interface challenges, intricate issues emanated from using four applications leading to the subsequent revision of workshops for the future. The study presents implications to instructional designers, administrators and faculty members on the intricate process of implementing new educational technologies and the best way to manage learning management system transition all together. Specifically, the study sought to answer the following questions:

• What general Canvas interface issues did faculty members face in transition from D2L?

• What issues did faculty members encounter while integrating various software applications in Canvas?

• What implications did these issues have on preparing future faculty development workshops?

A state university in the southern part of United States had been using Desire to Learn LMS in the past. An administrative decision led to a switch. While the University administration made the decision to switch, campuswide consultations with faculty members were made for the selection of a new LMS from a shortlist. Canvas was, in the process, selected to be the new LMS the university would be adopting. The Center for Excellence in Teaching and Learning was charged with the task of training faculty members to using Canvas. Between August 2013 and January 2014, 119 faculty members attended four different workshops that primarily focused on training them on how to use and navigate various course-related components in Canvas. Four workshops covering different aspects of Canvas LMS were designed and taught repetitively for a week in October 2013. Workshops were taught again in January 2014, again, repetitively for a week. The majority of the faculty members who took training were those who taught online. However, some faculty members who never taught online also attended the workshops. And so from a total of 119 faculty members, about 95% of these taught online and only about 5% did not.

The four workshops were divided based on sections in the Canvas interface. Again, inevitably, faculty members had to integrate other software applications that work in an LMS. Due to limited time allocated to running the workshops, these applications were not covered deeply in the four workshops since priority was placed on training faculty members on using features of the new LMS rather than addons. To that end, it would be found that mastering the latter would pose more challenges to faculty members than general learning of the Canvas interface. A faculty member who is also the teaching and learning coordinator prepared the workshops and ran them repeatedly in collaboration with the director of excellence in teaching and learning. As expected, after scheduled workshops were done, faculty members encountered contextual issues while using Canvas. They would call the teaching and learning coordinator for assistance with various issues while those who needed more specialized assistance would make one-on-one appointments. This was found to be a helpful strategy since it was naturally found that there were varying degrees of technology proficiency among users.

Data for this qualitative study were collected by compiling case study reports on a day-to-day basis. A case study is an empirical inquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident. It is also concerned with studying the phenomenon in context, so that the findings generate insight into how the phenomenon actually occurs within a given situation (Creswell, 2009; Yin, 2009). A Google document (Google doc) was created and all issues faculty raised during workshops, on the telephone, and in one-on-one meetings were systematically recorded on it. After formal workshops, faculty members continued to make calls and hold one-on-one meetings with the teaching and learning coordinator. During these calls and one-on-one meetings, faculty members would raise different issues with Canvas that were addressed by the teaching and learning coordinator. These issues were all recorded on the Google doc. The recorded notes focused on technological issues users encountered while learning how to navigate the general Canvas interface. The issues ranged from general Canvas interface queries to more complex issues having to do with integrating four software applications: Panopto, Voice Thread, Respondus Lockdown Browser, and Turnitin. While some of these applications, such as Respondus Lockdown Browser. had been used by faculty members in D2L LMS, they still posed a variety of integration issues in Canvas. Notes relating to issues faculty members encountered while navigating the general Canvas interface and while integrating the four external applications were later complied into a detailed case study report from which this study was developed.

The teaching and learning coordinator, a faculty member who performs instructional design duties, was the researcher in this study. The researcher, therefore, had the insider’s perspective. His initial role began with synthesizing content to be used to facilitate the workshops. He went on to facilitate the workshops by collaborating with the director of Center of Teaching and Learning. All content questions during workshops were addressed by the researcher (the teaching and learning coordinator). The researcher further recorded all the issues raised during workshops on the Google doc that was created for data collection purposes. After formal workshops, users either made calls to the teaching and learning coordinator (the researcher) or made one-on-one appointments. Issues that users inquired about during these calls and meetings were recorded on the Google doc by the researcher. The researcher analyzed the data from which the present artifact was created.

Data collected from arising issues were analyzed based on two categories. Category One was comprised of general Canvas interface issues, while Category Two was comprised of issues deriving from integration of four the software applications. While there was an enormous amount of data collected over a period of Canvas delving, whole text analysis was used to examine the notes recorded on the Google doc. This technique requires the researcher to fully understand the purpose of the study to enable continuous study of the data in order to identify specific codes. This procedure for analysis was developed by Glaser and Strauss (1967) and Strauss and Corbin (1998). From the theme “technological issues arising from faculty use of a new learning management system,” the two categories are presented in Tables 1 and 2.

Data mostly collected from faculty calls and one-on-one meetings with the teaching and learning coordinator (refer to Table 1) indicated that, as faculty members went to apply the skills they had learned in the workshops, contextual issues arose as they navigated various aspects of the Canvas interface. An instructional design role is about problem solving, it can be argued. Some of the Canvas issues faculty members encountered were resolved within minutes simply by showing an instructor how it is done. These were categorized as Level One issues. Other issues, however, took a reasonable amount of effort to be resolved. These were categorized as Level Two issues and were mostly resolved face to face with a faculty member. Yet other issues took a substantial amount of research, including involving contacting Canvas and other relevant software providers on behalf of faculty members or with them. The latter were categorized as Level Three issues. Level One issues like how to change course dates or deciphering meaning of the various quiz icons were resolved with a simple explanation or demonstration. Level Two issues like how to give true extra credit or how to import question banks form an external source or indeed how to moderate a quiz in order to allocate more time to specific students required a well prepared tutorial to faculty members. Depending on how quickly a faculty member would master the action, steps would be repeated mostly in a one-on-one meeting until the user was able to perform the required action on his or her own. Even more interesting were Level Three issues such as importing quiz banks from an external source, reinstating a deleted quiz, reinstating grades for a student who had been dropped from a course, and others. While the first task, importing quiz banks from an external source, would be done with the faculty member, the other two issues, reinstating grades and deleted quizzes, could not be handled at the level of Canvas access of a college administrator. These would therefore require the teaching and learning Coordinator contacting Canvas by either creating a ticket or using the chat feature. Later on, faculty members were walked through creating tickets of their own and being able to use the chat feature to have such type of Level Three issues resolved. This strategy allied well the philosophy of doing it together in instructional design (Dick, Carey, & Carey, 2009).

All issues that arose from integrating the four external applications into Canvas were categorized as Level Three issues due to the depth of their complexity. From a general perspective, the integration of these applications posed more challenging issues to faculty members than general Canvas interface use.

A Voice Thread is a dynamic, living conversation space that can be altered anytime. Basically, the application is about creating collaborative space with video, voice and text commenting (Voice Thread, 2015).

Issues with Voice Thread, such as students not being able to see one another’s projects when the instructor felt they had done all the settings correctly, posed an intricate challenge to solve. The same applied to issues like a Voice Thread project not available on on-campus computers. These issues required very systemic diagnosis, which began by asking the instructor for very specific details of what exactly was happening. The diagnosis would take place through e-mail or telephone. Once details of the problem were made clear, the teaching and learning coordinator would identify solutions that would essentially solve the problem. Again, the whole process would be done while doing it together with the faculty member rather than doing it on their behalf. Interestingly, though, some faculty members would ask that the problem be solved for them rather than work through it together. This simply highlighted varying faculty preferences when it comes to solving technological issues. Procedurally, issues that could not be solved at the level of access of on-campus Voice Thread administrators would require creating a ticket with Voice Thread engineers who would in turn examine the problem and offer solutions.

Panopto is a software package for businesses and universities that makes it easy for anyone to record, live stream, and share video (Panopto, 2015). In education institutions, Panopto is mostly used for lecture capture. Faculty members would normally record a lecture with video and/or PowerPoint and in turn post it in a module for students to view at any time. For example, an instructor wanted to find out how best to use Panopto to run a face-toface flipped classroom. This scenario actually justified the application’s valuable use. While working with the instructor, the Teaching and Learning Coordinator suggested that the best way to do it would be to make Panopto recordings of all lessons and post them in a module, for example, and set each lesson recording to open a few days prior to when they would be taught. That way, students would watch the stream prior and in turn discuss it when they came to class. Another interesting issue involved students not being able to make Panopto recordings themselves as the instructor had asked them to. While running a diagnosis of this issue, it was found that the instructor had actually made the settings in such a way that only teachers could create projects. This made sense in the context, considering that the University had purchased licensing for the application primarily for lecture capture and for student view. Finally, some issues that faculty members encountered with Panopto made the Teaching and Learning Coordinator offer the advice that sometimes it was a browser issue as the application did not normally work well in Internet Explorer. To that end, Google Chrome and Firefox were made recommended browsers.

Turnitin is an Internet-based plagiarismprevention service that enables submitted essays to be checked for unoriginal content. It normally integrates with Grademark, which enables instructors to grade students’ written work online by providing them the ability to add comments within the body of a paper, point out grammar and punctuation mistakes or works of art (Turnitin, 2015).

One of the pertinent issues that faculty members encountered while using Turnitin and Grademark was a scenario where they would give feedback through commenting and pointing but students could not see the comments. As seemingly complicated as it looked, sometimes the issue had to do with the browser students were using as Turnitin, we learned, works best in Firefox and Google Chrome. Yet interestingly, in certain instances, it still posed varying issues with the two browsers. Apart from browser issues, sometimes the application would just malfunction without any reasonable diagnosable cause. These latter issues would occur from time to time in some sort of a flare. When such was the case, the teaching and learning coordinator would create a ticket with Turnitin engineers who would then look into the issues and mostly elucidated them by pointing out the complexity of having to integrate an LMS and two external applications that have to complement one another. In certain instances, faculty members would have issues with Turnitin failing to process essays that were submitted before the application was enabled in a particular assignment. In such a scenario, instructors would be advised to have students resubmit the assignments after it (the application) had been turned on. Also, sometimes server issues would bring a glitch, which resulted in submissions not generating an originality score due to a “class does not exist” error. Being server issues, Turnitin engineers would be asked to look into the problem and do necessary maintenance work. Other issues faculty encountered would, for example, be students’ inability to print out feedback provided through Grademark. To address an issue like this one, the teaching and learning coordinator would devise a series of steps and walk the concerned faculty member through them so they would offer proper assistance to students.

Respondus is a tool for creating and managing exams that can be printed to paper or published directly to an LMS. It is a custom browser that locks down the testing environment within an LMS such as Canvas. When students use Respondus LockDown Browser, they are unable to print, copy, go to another URL, or access other applications. When an assessment is started, students are locked into it until they submit it for grading. It is complemented by Respondus Monitor, a companion application that integrates webcam technology with LockDown Browser (Respondus, 2015).

One of the outstanding issues that faculty would seek assistance for was how to print Canvas tests using Respondus. This would require the teaching and learning coordinator to walk them through steps for performing the action. In certain instances, users would receive an error message while trying to connect to a test bank server after Respondus had been enabled. Such an issue would require figuring out whether it was a Respondus issue or an issue to do with the owner of the test bank they were drawing questions from. Sometimes students would simply report to their instructor that Respondus would not work. Such a very general issue required systematic diagnosis by advising the instructor to ask for more details on what was exactly happening when students attempted to take a test. From that information provided by students, a determination would be made, mostly engaging Respondus engineers to locate the root of the problem, which would be as complex as a server malfunction case. Yet other times, while an instructor had made all the settings correctly enabling Respondus Monitor, students would take a test without being prompted to use a webcam. Again, Respondus engineers would be engaged to look into such a complex glitch.

From a general perspective, it was noted that while faculty members mostly asked general Canvas interface questions during workshops, more complex, context-specific issues arose when they went to use Canvas on a daily basis. Also, it was interesting to note that most faculty members who had reservations about Canvas at the beginning of the whole process, preferring to continue with D2L that they had used for a long time, ended up liking it (Canvas) later after learning how to use it. Again, with time, more and more faculty who did not teach online ended up wanting to learn Canvas. This group of faculty members found features like the gradebook and announcements very helpful in managing classes and student work. Overall, the most challenging issues arose from faculty integration of the four applications in Canvas. Naturally, there was variation in speed of mastery of features in the new LMS among faculty members. For example, some users were still learning the basics on a one-onone basis 6 months after the initial workshops were run. Others, however, only took as less as a couple of weeks after the workshops to get familiar with the whole interface. The variation in speed of mastery, it was learned, was due to different levels of proficiency with technology and also personal enthusiasm and motivation.

In the long run, issues faculty members encountered helped us to go back and formatively evaluate and revise the workshops. Again, the whole philosophy of doing it together in instructional design (Dick et al., 2009) became reminiscent in this study as it was clearly shown that those faculty members who preferred to learn by doing it together with the teaching and learning coordinator easily overcame issues they were encountering with the technology than those who preferred for the teaching and learning coordinator to solve problems for them. Findings of this study indicate that the whole process of implementing faculty development workshops require proper planning between administrators and instructional designers in order to ensure proper allocation of time and other resources, thereby facilitating a more efficient transition from one LMS to another.

The study presents several implications to instructional designers, faculty members, administrators, and educational software developers on the intricate process of implementing new educational technologies and the best way to manage learning management system transition as a whole. First, while time is always a constraint, it is important to include all aspects of an LMS in a workshop in order to mitigate arising issues when faculty members get down to use it. It is likely that if more workshop time had been allocated to covering integration of the four applications to a deeper level during planning, some of the issues that arose while faculty members used them would have been mitigated. Also, the study does show that faculty members tend to master technological skills faster and more efficiently when they get involved in solving issues with instructional designers rather than having the latter do it for them. Again, while educational software developers do produce applications that integrate well in an LMS such as Canvas, the study showed that there are issues that do arise when the applications are actually used in context. Server issues and sudden malfunction of applications as evidenced in this study provide useful feedback to the administrators of the four applications discussed in this study.

In the end, the whole process of transitioning to a new LMS became an innovation being adopted. To be called an innovation, an idea does not have to be necessarily newly invented (Rogers, 1995; Van de Ven, 1986). As Rogers (2003) contended, one of the distinct innovation attributes is complexity, the extent to which an innovation is considered difficult to learn and utilize. Issues that arose while faculty members used the new LMS prompted designers to go back and revise workshops in order to make future adoption processes of this type less difficult for users. Finally, the fact that this was a case study of one institution of higher learning is a limitation to this study. Similar studies in other institutions undergoing LMS transition including integrated applications would extend the scope of this study and probably corroborate the present findings.