Using Addie And Systems Thinking As The Framework For Developing A Mooc Free

In their brief existence, MOOCs have received a great deal of attention in both higher education and in mainstream news. The term massive open online course was coined in 2008 by Stephen Downes and George Siemens and came into broad use in 2012 (Johnson et al., 2014). The first MOOC, an online course in “Connectivism and Connective Knowledge,” was offered by the University of Manitoba in 2008. The course enrolled a small number of paying students and was also offered to a group of online “auditors” who participated in the course free of charge. Unexpectedly, enrollment exceeded 2000 auditors (Sandeen, 2013). The first U.S.-based MOOC was offered in 2011 by the University of Illinois, Springfield, and enrollment exceeded 2,500 (Defining a MOOC, 2013).

Since the first MOOCs were launched, prestigious public and private universities as well as for-profit providers (Heller, 2013; Lewin, 2012) have enrolled millions of students around the world in these courses. Rapid growth in enrollment has occurred, in part, by the emergence of three leading platforms in the MOOC market—Coursera (founded by Stanford University professors), Udacity (progeny of Palo Alto), and edX (jointly founded by Harvard University and MIT). Illustrative of this trend, when a Harvard University professor offered his popular course, “Concepts of the Hero in Classical Greek Civilization” via HarvardX (Harvard’s proprietary line of edX) in Spring 2013, enrollment exceeded 31,000 (Heller, 2013).

While the number of MOOCs and learner enrollment is on the rise, many higher education institutions (39.9%) report they are undecided about MOOCs, while the single largest group (46.5%) have no plans for a MOOC (Allen & Seaman, 2015). In their 2015 report on the state of online learning in the United States, Allen and Seaman noted that the percent of higher education institutions that currently have a MOOC increased from 2.6% in 2012 to 5.0% in 2013, and to 8.0% in 2014. In this same report, Allen and Seaman (2015) noted that only 16.3% of academic leaders believe that MOOCs represent a sustainable method for offering online courses, down from 28.3% in 2012.

While MOOCs were widely embraced in 2012, the low completion rates (5–16% overall) reported in 2013 brought about a major change in attitude about the quality of education MOOCs can offer (Johnson et al., 2014). MOOC skeptics questioned how engaging such learning environments actually are (Fischer, 2014; Johnson et al., 2014). As leading MOOC providers analyzed the reasons for these low completion rates, they began to realize that a “more expansive, human-centered support structure is key to helping students retain information, stick with the course—and finish” (Westervelt, 2013). John Dewey (1938), early reformer in education, explained that learning is based on discovery guided by mentoring, rather than transmission of information. Thus, learning approaches must move beyond traditional lecture style pedagogies typically found in MOOCs and instead focus on providing students with opportunities to engage with their teachers to feel comfortable, confident, and successful.

Analyzing the movement of a million MOOC users through 16 Coursera courses offered by the University of Pennsylvania from June 2012 to June 2013, researchers noted that only about 50% of those who registered for a course ever viewed lecture materials and only about 4% completed the courses, with user engagement falling off dramatically in the first 2 weeks of enrollment (Stein, 2013). While many MOOCs are heavy on course content, the engagement piece for learners may be lacking. Ray Schroeder, director of the Center for Online Learning, Research and Service at the University of Illinois, Springfield, suggested three things matter most in online learning: quality of material covered, engagement of the teacher, and interaction among students (Pappano, 2012). While quality content does not appear to be an issue, creating opportunities for instructor connection and student interaction creates a significant challenge when there may be thousands of students enrolled in a single course.

Creating a MOOC that is satisfying, engaging, and meets the needs of learners calls for a user/learner-centered design approach. Usercentered design is the process of designing a tool (e.g., course) from the perspective of how it will be understood and used by the user (Nielsen & Loranger, 2006; Usability First, n.d.). By employing a user-centered design approach to MOOC design, a course can be offered that is more efficient, satisfying, and user-friendly for the learner.

MOOCs, designed largely to be media driven and asynchronous in nature, provide a teaching/learning environment that fits nicely with the application of user-centered instructional design principles. Asynchronous communication does not require all parties involved in communication to be present and available at the same time (e.g., online discussion boards). With a learner-centered rather than teaching-centered approach, the ADDIE model is an ideal framework that can be utilized in MOOC design that can help to make courses more applicable and meaningful for learners (Culatta, 2013; Peterson, 2013). While successful use of the ADDIE model in online course design has been documented in the literature (Evans & Lockee, 2008; Summerville, 2003; Wang & Hsu, 2009), there is currently no reference that has been identified by this study’s authors with regard to the use of the ADDIE model in MOOC design.

The ADDIE framework (Figure 1) is a cyclical, iterative process consisting of five stages: analysis, design, development, implementation, and evaluation. In the analysis phase, the main consideration is the target audience (MOOC students) and the determination of what they know by the end of the course (Peterson, 2013). In addition, available resources, timeline, and overall course goals from both learner and organizational perspectives must be understood. During the design stage, course developers must determine the objectives of the course, how objectives will be met, how they will be assessed, and the methods that will be most effective in delivery of the objectives (Seels & Glasgow, 1988). If the goals, objectives, and assessments of a course do not align, learners may find themselves losing interest in the course and developing adverse perceptions of instructional quality (Peterson, 2013). Students’ participation during the design phase can contribute to their overall course satisfaction and plays an important role in determining whether students continue in a course (Murphy, 1999). The availability and time constraints of the faculty member and/or instructor also must be taken into account. It is essential that learner pilot testing and feedback are gathered before development begins so that the appropriate adjustments can be made before resources toward development are fully committed and expended.

Utilizing the results from the analysis and design phases, course designers in the development phase construct the course for delivery of information while, at the same time, determining if students will learn from the course and how it can be refined before implementation (Peterson, 2013). Again, it is highly recommended that several rapid learner feedback and course redesign iterative phases occur before rolling it live. In the implementation phase, course developers, learners, and the instructor actively work together to analyze, redesign, and evaluate the course and make instantaneous modifications to ensure effectiveness. Finally, during the evaluation phase, the course designer must determine if the objectives have been met, evaluate the impact of the course through summative and formative evaluations, and make changes necessary for future delivery of the course.

In a time when MOOCs and systemic change are being called for in higher education, this case study explores how systems thinking and the ADDIE framework were utilized to develop one of the first MOOCs at a mid-sized university in the southeastern United States. The study’s three guiding research questions were:

• Research Question 1: What is the process for designing and developing a MOOC?

• Research Question 2: What impact did using the ADDIE model have on MOOC design?

• Research Question 3: What is the relationship between instructional design and MOOCs?

The MOOC instructor and coauthor of this article was closely followed throughout the course design and development process. Through a lens of instructional systems design and systems thinking, a case study of the development of a MOOC designed to teach web design and usability principles and skills at a public, midsized regional university in the southeastern United States was undertaken. Throughout MOOC development, interviews were conducted with administrators, the design team, a user group that provided input throughout the process, and the first student that completed the MOOC. The instructor also added his own perspectives throughout the project, detailed the goals and purposes of the MOOC, the process of design and development, and the challenges and implications faced throughout the project.

The university had been following the evolution of the MOOC and wanted to test out the space. Initially, two pilot courses were identified for development, to be offered in spring 2014. This study reports the conceptualization, development, and implementation of one of the courses, “Web Design and Usability” (moocs.uncg.edu/wdu). The results of the case are presented according to the three primary research questions for the study, using the key elements of the ADDIE model throughout.

To develop this course, a full design team was assembled at the onset of the project and included a project manager, instructional designer, graphic designer, video production team, and the lead faculty member. The lead faculty member made sure the ADDIE process was followed throughout the project. The total cost of production for the MOOC was estimated to be $13,000 for the development of this MOOC, which included $3,500 for faculty compensation, with the remaining funds allocated for production costs.

At the onset of the analysis phase, university leaders, the chancellor, production team, and lead faculty member met and decided upon the goals of the MOOC. Ultimately, three goals were identified that aligned closely to the university’s larger mission: (1) teach, free of charge, how to design a website and basic usability concepts and methodology, (2) publicize and brand the university, and (3) serve as test case for different instructional strategies. Because the MOOC was designed to be a free course and statistics continue to show that retention among students who enroll in MOOCs is low, it was decided that the MOOC would be used as a way to brand the university as a place people might consider attending as a campus-based or online student. With this in mind, the production team tried to ensure the university was branded throughout the course using strategic logo placement, video clips recorded throughout the campus with the lead faculty member wearing university branded clothing, and a university mascot presence throughout the course to “keep it light.” As a free, open course, the MOOC provided an opportunity to test the efficacy of different instructional strategies with a large number of learners and to measure the effectiveness of these strategies. Early market analysis identified young, male, tech-savvy professionals as the primary target for this MOOC, thus the course was developed primarily with this user group in mind. With the goals of the course and the primary target population identified, the design phase of the course was ready to begin.

With the analysis phase underway, the production team met on two occasions during which time they identified course objectives, created a project timeline, and established the basic architecture of the course using an outcomes map (Figure 2). The production team decided that the course would be comprised of five asynchronous units (Usability, User-Centered Design, ADDIE Process, Evaluation and Testing, and Web Design) that included a variety of ways of presenting information. On average, each unit was to have between 2-5 sections (15 altogether for the course) that would also undergo the ADDIE process and emphasize hands-on application throughout.

All main units and sections were designed to include a video introduction with the lead faculty member as the presenter (closed-captions and written transcript provided for all videos), unit review sessions comprised of five multiple-choice questions, online discussion, hands on activities, and readings. Finally, being under the assumption that learners would want to know how the MOOC was designed and developed using the very usercentered approach taught in the course that illustrated “we practiced what we preached,” video case studies were designed for inclusion as final components of each unit.

An important element of instructional design is deciding what constitutes acceptable performance. Because the design team wanted to give all individuals who successfully completed the MOOC a certificate of completion (no university credit awarded), a basic concept of mastery learning (90% or above) was utilized. Each unit section had five questions and each unit had a test that included the section questions. In order to earn the certificate of completion, students had to earn at least 90% on all unit tests as well as on a final MOOC exam.

A key element of the development phase of the MOOC was creating introductory videos for all unit and section introductions as well as lectures. Because the production team staff had expertise in video production, all videos were filmed and edited by expert videographers using high-end video equipment as well as a teleprompter from which the faculty/presenter read preprepared transcripts. The lead faculty member noted, “one of the hardest things developing the MOOC was the time it took to write out the introductions and lectures and reading these off the teleprompter” (A.S. Chow, personal communication, October 14, 2014). While this was the process for recording lectures at the onset of online learning (for him in 2000) online teaching had become more synchronous and spontaneous and “precanned” lectures were difficult for him to go back to. WordPress was used as the course platform because of its flexibility as well as the widgets that were available such as online review sessions with automatic grading and discussion forums.

An essential component of the development phase of any instructional project is to determine if students will learn from the course and how it can be improved before implementation (Peterson, 2013). With this in mind, a group of usability reviewers analyzed the course after the first functional unit that included videos, content, review questions, and a discussion forum was complete. While the usability group should ideally have included young, techsavvy male professionals (target population identified in the market analysis), the production team did not have ready access to these types of individuals, as most of the usability reviewers were female.

As a result of the usability reviewer feedback, numerous modifications were made in the MOOC’s design. The most significant changes to the course related to the videos recorded for the course. One of the struggles when designing online content is the ideal length of videos. Initially, videos were shortened to be 15 to 20 second clips, with numerous clips following one after another in a single introduction. According to user feedback, the videos were, ironically, too short. As a result, the production team moved away from short clips to longer instructional videos that were kept relatively short (average 3–4 minutes, longest was 10 minutes). Because some individuals in the user group indicated they would prefer to read a video transcript instead of watching a video, transcripts as well as closed-captioning were created and made available for all videos.

During the implementation phase of the MOOC, the first version of the first section of the course was reviewed by over 60 graduate students enrolled in two courses within the university’s master of library and information studies program, “Instructional Design” and “Library Management.” Overall, user feedback was good, indicating to the production team members that they had heard the user group and had responded positively. The positive feedback received from the reviewers, while not reflecting true representation of the desired young male demographic, still gave the production team confidence that they were on the right track and to move forward with development of the remaining four units and 14 sections.

The MOOC was released with a quiet rollout to ensure any remaining problems were ironed out which included the entire registration to course completion process. The MOOC begins with an introduction page where prospective learners can gain a clear overview of what the course has to offer. Each unit begins with a brief introductory video along with a full transcript. Usability reviewer feedback suggested that some learners would prefer to read rather than watch a video while other preferred the video over text. In addition, the MOOC navigation was designed to clearly orient the learner to where they were. A right side bar also provided learners with easy access to all other units. For each of the five unit introduction videos, an attractive area of campus or the local area was chosen and also the university mascot was introduced as both a way to brand the university as well to bring a bit of fun into what otherwise was more academic and intellectual content. Each section or module within a Unit was presented in the right side bar for easy access and orientation.

Each section is comprised of a brief instructional video, a five-question review session or assessment, threaded discussion, applied activity (posted on the discussion board), and readings and supplemental resources. Review sessions involve mostly multiple-choice questions (Figure 3). While more interactive questions were intended for the unit reviews, most ended up as simply multiple-choice due to time and resource limitations. Despite being basic, the questions were created to promote reflection and cognitive processing of the main points of each section. The MOOC’s final exam involves a combination of all section review sessions. A learner has to pass this exam at 90% or above in order to receive a Certificate of Completion.

Within each section, learners are encouraged to apply learning concepts to their daily lives or something applied. This helps create a learning community around each section that allows each learner to contribute their individual perspectives and worldviews to the instructional material and content. Student-to-student interaction in an asynchronous learning environment is an essential success factor of any online course. While everything is optional, learners are presented with hands-on activities at the end of each section, working toward the ultimate goal of creating a website at the end of the course (Figure 4). In addition, learners are presented with additional resources that are open access. Because learners are not formally enrolled in the university as students, they are not allowed access to library resources.

Each unit includes between two and four sections. At the end of each unit, an illustrative case study is provided that details how each unit was designed and developed following the user-design process. Another goal of the MOOC was to facilitate the creation of longterm learning communities and social networks for all learners. Facebook, Linked-in, and Twitter accounts were created to help support the formation of such communities.

Throughout the MOOC design and development, the usability process reflected the “evaluation” in the ADDIE model, which required that the production team constantly evaluated how they were doing. The MOOC was launched in July 2014 with a soft roll out, with official launch in August 2014.

Anecdotal evidence from a learner who is the first person to earn the certificate of completion suggests the course has been an initial success. In a recent interview, this student indicated she appreciated the flexibility the MOOC offered, noting, “The freedom to pick up and plow through parts of the MOOC at moments where I had free time/during my lunch break at work made it easy to get through the process” (A. G. Bradley, personal communication, October 20, 2014). She further stated that the MOOC has helped her gain skills and knowledge that have been applicable to her work, stating, “Thanks to the MOOC (and working with [the lead faculty member]), I am undertaking a user-centered study with the State Library … I have also implemented the Aunt-ADDIE based user-design principles into my own work.” Reflecting on opportunities for engagement with other students enrolled in the MOOC, this same student noted, “I admittedly rushed through this [the discussion portion of the course] because of time constrains my summer course load placed on me.” Finally, when asked about her impressions of MOOCs in general, this student explained,

End-of-MOOC evaluation results suggest learners are largely satisfied, giving the MOOC an overall satisfaction rating of 6.5 (n = 36) on a 7-point scale (Figure 5). Some user comments include:

The ADDIE process, which required that the production team constantly evaluate how they were doing (the “E” in the ADDIE) throughout the course development process, helped to keep the team focused. The ADDIE model was useful for the lead faculty member, who also served as leader of the production team, as it allowed him to keep the entire project moving throughout an 8-month design and production process. Essentially, the ADDIE model helped to:

• D: With clear course and organizational goals in mind, create a timeline and course outline that laid out the needs of the project so the production staff knew what resources (human and financial) they needed to encumber.

• D: With a course plan and outline in hand, curriculum development involving heavy multi-media and multiple user/redesign iterative cycles took place within the identified timeline that allowed the team to stay on track (although delayed at times) and produce something utilizing user-centered, instructional design principles.

• I: Soft rollout that allowed invariable process issues to be identified and refined before public rollout. This is also the formal stage in which feedback is encouraged and received.

• E: While formative evaluation is part of the life cycle process of any course or website design, creating an environment for continuous feedback that allows for continuous refinement and also opportunities for summative evaluation is the most critical aspect of this process.

One of the key factors in MOOC design is to accept the fact that a MOOC is intended to have such a large number of students that it eliminates the instructor’s ability to provide feedback to most students. This “makes the need for careful instructional design very important” (A.S. Chow, personal communication, October 14, 2014). In the lead faculty member’s opinion, courses that are primarily lecture based, as is typical in many MOOCs, depart from sound instructional design principles. He explains that, while there is something to be said to listening to a lecture from a wellknown, captivating speaker,

The key element of instructional design is placing the learner at the center of what is trying to be accomplished and designing the learning activity in such a way that is value added and allows the student to learn the material in a way that it is actually paid attention to, retained, and can be recalled in a performance environment where it will be used. Instructional design principles require that courses are designed with clear learning objectives and are implemented with activities that include learner engagement, cognitive processing, social interaction, opportunities to review and react, and include a diverse set of learning experiences. In the online context, these learning experiences often include multimedia, traditional reading, some basic assessment, and provide a social context of providing input and sharing ones experiences with others. According to instructional design principles, these elements together create a strong learning environment.

In the MOOC discussed in this case study, all of the instructional design elements noted above were included—video introductions and lectures, readings, assessments, hands-on activities, and social interaction via online forums. Because it is essential that students have the opportunity to practice what they have learned, hands-on activities are included throughout each section of the course. Learners are asked to apply some hands on skills and post their results to the discussion board to encourage peer-to-peer feedback, reflection, and interaction. To encourage social connections, Facebook, LinkedIn, and Twitter groups were created to encourage networking among students. In addition, because literature has found that student retention in courses tends to be higher when people have a start/end date that is formal, a cohort model was initially planned that would require students to have a particular start and end date. This plan, however, fell through in actual implementation due to limited resources. However, there are recommended guided experiences that begin every six weeks, though adherence to this guideline is not required and students may enroll and begin the course anytime they wish.

Systems thinking and systemic change were used as the guiding frameworks for understanding both the macro organizational goals and micro issues dealing with design and development involved with creating and offering a MOOC. A critical facet of systems thinking is to ensure there is alignment between institutional goals and organizational elements (Kaufman, 1988). Additionally, it is highly recommended that a well-qualified team of both faculty and production specialists is assembled, properly compensated, and works together throughout a carefully implemented instructional design process such as the ADDIE model that allows for a constant feedback cycle between users and the production team. This is not always an easy process, as the lead faculty member and study coauthor explained in a recent interview:

There is an inherent conflict between how a faculty member intends to deliver instruction and how the production process works. This is very important for those considering developing MOOCs. The production team must work closely with the faculty member in such a way that the faculty member does not feel directed so that it shifts the paradigm of what he is used to—a great deal of autonomy, creativity, and spontaneity. (A.S. Chow, personal communication, October 14, 2014)

A self-contained, self-paced, and asynchronous MOOC where learner and instructor will never meet one another is a new, still largely uncomfortable way of providing online instruction. A MOOC is indeed a performance information system that allows the instructor to share knowledge and skill in the hopes that it will prove interesting and useful to learners from all around the world. The ADDIE model provides structure and comfort to instructors and designers by providing a systematic framework to work through in developing a learning experience and environment that supports and guides learning. Furthermore, it helps incorporate macro university goals as part of the analysis and design process to ensure that alignment occurs between instructor, learner, and the sponsoring organization.

In our strange, new world of online learning and formal higher education, systems thinking allows us to turn chaos into organization and develop a plan for success regardless of paradigm shifts and rapid changes in technology, learning environments, learner expectations, university goals, and organizational infrastructure.