Using a Multiperspective Design Team to Develop and Manage Multilayered Online Courses

Through group discussions, critical thinking papers, multiple-choice quizzes, or group assignments, students who opt to take online classes are offered a myriad of methods to learn. But are they really learning? Is online education leading to student achievement? The method outlined in this article presents the course development model being used successfully at Colorado State University-Global Campus (CSU-Global) and illustrates the critical actions this institution has taken that have led to a multilevel method for designing courses that ensure continued student achievement and satisfaction, as well as instructor involvement and pedagogy.

While other researchers have discussed CSU-Global’s collaborative course development model (Puzziferro & Shelton, 2008), this article will examine CSU-Global’s online course development procedures from three key dimensions and campuswide standards. These areas were directly identified and directed the multidimensional course development approach that took place during the course revision initiative of the master of science teaching and learning (MSTL) program in 2013. We argue that it is the combination of the multidimensional, collaborative method with the quality of the culture evidenced in the institutionwide participatory course components outlined in this article that has led to the designing of effective online courses that ensure continued student achievement and satisfaction, as well as instructor involvement and pedagogy.

One of our most popular programs at CSU-Global is the master of science in teaching and learning (MSTL) program. This teaching degree program is designed to elevate leadership, intellect, and professional knowledge of educational theory and techniques for the promotion of lifelong learning and academic excellence. Designed for educators of all types, earning a master’s degree in teaching and learning is ideally suited for teachers, school administrators, adult educators, special education instructors, corporate trainers, instructional designers, and other educational leaders.

There are more than 200 active students in the MSTL program at CSU-Global. More than 75% of students are female, about 21% are male, and the average age of the students is 35. The most popular degree concentrations in this online program include teacher leadership, English language learning, and K–12 instructional technology and instructional design. The degree coursework emphasizes issues of instruction, program development, and program management in pre-K–12 educational learning environments and nontraditional settings. Each course was originally designed to conclude with a portfolio-quality, research driven project with close supervision and collaboration by the instructor, requiring students to apply skills to specific scenarios. The final course in the core was designed as a capstone course, in which students design an innovative and original approach to address a current problem within the field of education. Utilizing knowledge, theories, and research-study understanding gained throughout the program, students analyze and evaluate teaching and learning principles in a customized professional project.

Applying all these items to online learning, however, is different from traditional classroom settings, and simply transferring a lecture-based course to an online environment is not effective. Online learning requires specific pedagogical techniques through instructional design that ensures students have the right information, in the correct order available, when they need it. Online learning may have advantages to traditional classroom learning, including:

• Students are more willing to participate in the classroom discussion and activities. There can be any number of reasons that students do not participate equally in a traditional classroom, including the simple reason of shyness.

• Students can work at their own pace, on their own schedule.

• Students may be able to concentrate on material better when in control of the environment where their learning occurs.

• Courses accommodate various learning styles.

CSU-Global is well aware of the advantages of online learning, but not all higher learning institutions are in agreement as to the best ways to address online course development. A review of the history of online learning, studies pertaining to online course development and research about the online teaching environment will now be addressed.

Online learning and class delivery have basically taken three different approaches in higher education (Sloan, 2013):

1. New institutions have formed to offer online classes;

2. Existing institutions have morphed currently offered face-to-face classes into an online format; or

3. Existing institutions have expanded their footprint by making an online-only, often separate, entity that offers the online classes.

CSU-Global falls into the third category. Our degree programs promote academic excellence, offer innovative technology and delivery of material, and take into consideration the engagement of our stakeholders (CSU-Global, 2014). These three guiding statements are used both in course development and in several research studies regarding online learning.

Academic excellence should be the driving force in education no matter if a class is offered face to face, in a hybrid/blended format, online, or in any other manner. Online classes should always promote pedagogy and support educational theories (Sims, 2008). Some have argued for disruptive pedagogy strategies in delivering classes online (Hedberg, 2006). This can be done in a three-stage process of transferring instruction online, translating delivery to online, and transcending teaching and learning to the online environment (Sims, 2008). As almost any research study shows, at the heart of any type of class academic excellence should always be considered.

Past research on how to best educate others held to the idea to “learn first and to innovate later” (Hong & Sullivan, 2009, p. 614). However, technology can also be used to expand both the depth and breadth of ideas (Hong & Sullivan, 2009). Recent studies on the use of technology in online classrooms has shown positive student feedback, particularly in being able to have the flexibility and choice to review concepts and ideas presented through innovative technology methods (Preston et al., 2010). In addition to offering flexibility, providing technology in courses also offers varied ways to learn and master the content (Dixson, 2010; Snyder, 2009). Asynchronous technology in the classroom should be relevant and use other resources as an extension to expand the learning boundaries beyond the classroom (Snyder, 2009; Tunks, 2012). If the research tells us anything, the use of technology in any classroom setting requires a concise strategy and method for faculty to follow and apply in online courses (Laaser, 2011).

Throughout the several decades of online learning growth, several themes, strategies and principles of online learning or e-learning have emerged (Mayes, Ku, Akarasriworn, Luebeck, & Korkmaz, 2011). Clark and Mayer (2002) presented their principles of effective e-learning, which included six guidelines to follow when developing online courses. These six principles are multimedia, contiguity, modality, redundancy, coherence, and personalization. Falling in line with CSU-Global’s mission of taking stakeholders into consideration, these researchers took a consumer-based approach in developing a structured model for online learning (Clark, 1999; Clark & Mayer, 2002). Still other research has delved deeper into specific course design principles for online classes and programs. One study offered a multidimensional approach with six elements: technology affordances; alignment of objectives, instructional methods and assessments; disciplined-specific practices and approaches; meaningful and timely feedback; authenticity and transferability; and transparency of assessment criteria (McCracken, Cho, Sharif, Wilson, & Miller, 2012). These and other principles were also taken into consideration and applied to the CSU-Global course development process, which will be discussed throughout the rest of this article.

Although Dewey (1916) wrote a century ago about motivation and meaningful education and alternatives to standardization and mechanization of course development, his work has never been more relevant than today, as traditional schools turn more and more to the online environment to deliver instruction. More than half a century later, Keller (1979) introduced principles of motivation in course design, which led educators to take a more holistic approach to learning with the learner placed at center stage. CSU-Global has crafted an instructional model that transformed with Dewey’s (1916) classical mantra of education as life itself with its dedication to practical, applied learning, but one that also supports Darling-Hammond’s (2001) belief that good instruction is connected not only to practice but also to theory. CSU-Global’s multidimensional approach to learning, its strategies for implementing the approach, and the process by which it achieves the institutional mission drive the instructional model, which is inherently progressive and undeniably effective (see Appendix A).

CSU-Global understands and even embraces that the life of the working adult student is often filled with complex demands ranging from job and family responsibilities to other life issues, all of which can impede the learning process. CSU-Global’s course schedule is crafted around a dedicated principle: any course, any semester, any term. CSU-Global’s purposeful approach to learning includes a curriculum with a streamlined format; all courses are eight weeks in length, with eight learning modules, eight discussion board assignments, six to seven critical thinking assignments, and one final portfolio project. This repetitive design helps familiarize students with the format of every course so that time can be spent, not on platform navigation and style acclimation, but on concept attainment and mastery. The purposeful learner with steady course and swift degree completion in mind finds compatibility and satisfaction with CSU-Global, an institution of higher learning committed to the process of continually improving student learning and the teaching and learning process.

Perhaps one of the biggest opportunities for CSU-Global was the realization that course offerings needed significant revision to remain relevant, adjust to ever-changing technological advancements, and keep with industry standards. Within in the MSTL program, through focus groups consisting of faculty and students, student semester course evaluations, and efforts of the MSTL teaching team, significant improvements have been and continue to be made to the MSTL courses and projects. This ongoing review of culminating course and associated project parameters helped CSU-Global develop a comprehensive instructional design process resulting in an instructional model steeped in participatory tenets (Clark, 1999; Clark & Mayer, 2002; Dixon, 2010; Mayes et al., 2011; Snyder, 2009). This revision process led to the creation of new and revised courses in both the MSTL program and campuswide that are now more participatory in nature and industry relevant across three dimensions: students, courses, and revisions.

The curriculum at CSU-Global includes both problem-based and project-based learning, making it student centered. Each course at CSU-Global offers opportunities for learning that are relevant to students, where students engage in finding solutions to complex and challenging problems. CSU-Global students connect disciplinary knowledge to real-world problems, thus increasing students’ motivation to learn.

The online classes at CSU-Global present students with similar challenges they might encounter as a practitioner in the discipline. Discussion board questions offer a twopronged approach to the course content. The first component of each question asks for content knowledge; the second asks for practical application and allows students to share either hypothetical or actual personal, business, and/ or community-related experiences. Critical thinking assignments ask students to apply their theoretical and working knowledge of the course content to specific industry-relevant scenarios in order to improve processes and increase efficiency for said organizations. Final portfolio projects require students to weave all course concepts into one comprehensive presentation to demonstrate their competence in core learning outcomes.

Knowing that student success is heavily related to building community in an online program, Spinks (2007) recommended that designers build in opportunities to create community and academic self-efficacy. The instructional design team at CSU-Global considers compatibility with all student learning styles, and seeks ways to increase interactions between students and instructors, students and their classmates, and students and technology. This participatory model results in a cross-cultural, media-infused, yet human-driven, positive sense of community.

Recognizing that students need to be prepared for success in a global society, the incorporation of advanced interactive modules, infused multimedia, content with international relevance, including current industry and problem-based case studies, and reduced students’ costs through text book-free courses have resulted in courses that are intellectually stimulating and participative in nature (see Appendix A). Students are expected to use problem solving, critical thinking, and questioning skills during course interaction with colleagues and instructors, with content designed to make learning rigorous and relevant. Consequently, student and faculty engagement, overall satisfaction, and success rates remain high.

In the MSTL program, students are challenged in classes through assignments that are project-based. Students must apply the skills they are acquiring through course content and peer learning interactions into their daily practices. The goal of the MSTL program is to produce teacher leaders who have not only the skills, but also the tools to implement meaningful teaching practices and affect change in their schools. By the time students graduate from the MSTL program at CSU-Global, they will have had multiple opportunities to practice using these skills and tools, collect real data, receive meaningful and just-in-time feedback from instructors, and grow their professional competencies.

All CSU-Global courses undergo a full review and revision cycle every 12 to 18 months to determine content relevance to student and industry needs as well as to update content to meet new technology innovations. Quality Matters Program (2013) play a significant role in the development and revision process at CSU-Global. Each course is offered for 8 weeks, with a new section beginning every 4 weeks. The courses are designed around one learning module per week. Within each learning module are an overview that includes the learning outcomes for that module, required readings, module content, and critical thinking discussion question. The number of assignments within a course is dependent on the course content, but all courses are required to culminate in a portfolio assignment.

In the summer of 2013, CSU-Global instituted a program revision initiative to ensure that all courses are developed to include internationalization, interactivity, multimedia, manageable module content length, course readings and materials that are current—no more than 3–5 years old, and rubrics that are directly aligned to assignment requirements. In the MSTL program specifically, each course under this revision initiative integrated projectbased and practical assignments. Whatever the students are learning in the course, they do or implement in their actual teaching and learning environments. Each program within the University will undergo a comprehensive revision assessment, in addition to ongoing course assessments, that includes multiple stakeholders (e.g., program coordinator, program faculty, program students, program dean, university provost, current industry professionals, current industry professional organizations, and businesses). The MSTL program was the first program to go through this comprehensive revision process, including all required core courses and all specialization options (32 courses). This comprehensive program revision process, versus individual course revisions conducted in isolation, allowed for more cohesion among courses in the core and specializations.

To begin the comprehensive revision process in the MSTL program, a variety of stakeholders engaged in conversations around the skills needed to be effective educators in 21st century learning environments. The stakeholders involved in these conversations were the program coordinator, program faculty, program students, program dean, university provost, current industry professionals, and current industry professional organizations and businesses. In addition, the program coordinator conducted ongoing program and course assessments and analyzed the results for patterns or trends around the revised universitywide course development goals and educator effectiveness standards. The program coordinator and content experts read and analyzed literature relevant to K–12 educator effectiveness in their daily practices to assess current curriculum and learning outcomes. Some examples of the literature used in the MSTL course revision process include the work of John Hattie, Robert Marzano, the Framework for Teaching by The Danielson Group, the InTASK Standards, and Goodwin/ Hubbell. These seminal research initiatives in education reform, teacher leadership, education technology, and teacher best practices provided a foundation for the MSTL core and specialization course revisions.

Table 1 outlines the course design process in terms of steps and each practitioner’s role and responsibilities within those steps on the team. During the revision process, the program coordinator and instructional designer remained in consistent lead roles across all 32 program course revisions. This consistency allowed for a unified message to be communicated among the instructional designers and content experts to ensure that the course revisions resulted in purposeful, participatory, and project-based materials, content, and assignments throughout every course in the MSTL program. The program coordinator, in collaboration with the dean of professional studies, provost, lead faculty members, and the assigned instructional designer, developed a program analysis report (PAR) and program design document (PDD). The purpose of these documents was to gain a variety of perspectives that would guide the current program design and proposed program scope and redirection. These documents also assisted the marketing team in disseminating more effective marketing of the new program scope and sequence so students are aware of the updates and changes in the MSTL program. The goal was also to bolster enrollment in the MSTL program, targeting educators who truly want to become leaders and effect change in their educational environments.

The PAR and PDD also allowed for an identification of courses and/or specializations that needed to be substantially revised and thereby reapproved by the governance council and the university’s curriculum committee in order to purposefully meet the new course development standards and align with the scope and sequence of the revised MSTL program. Courses were further evaluated and revised based on their ability (or inability) to meet the needs of current and future CSU-Global students who are at different points in their careers as educators and how to best differentiate the learning experience to make the learning relevant to each student. For example, the previous MSTL program had a common capstone course experience that is typical of many education master’s programs. In making the curriculum more project based and relevant to educators, the capstone course approach was reinvented, dividing the previously single-focused course into two courses: one with a more traditional research-based paper approach and the other with a hands-on research-based professional project approach. Both of these culminating course offerings are preceded by a new research-preparatory course devoted to education-relevant action research that focuses on teaching educators how to make evidence-based decisions in their processes, programs, and daily practices. The content of these culminating courses is based on the work of Sagor (2010) and The Action Research Guidebook: A Four-Stage Process for Educators and School Teams. Also, in the revised program, students are given the choice for a culminating experience, allowing for more differentiated and personalized learning options. In the revised program, students have an opportunity to integrate and synthesize their learning across the program curriculum in various ways that best match their career goals (e.g., research-based professional project or researching effective educational programs).

Based on the level of revisions identified in each course, content experts were hired to develop the course content based on the program scope and direction. During the course design, there were multiple levels of expertise involved. In addition to the program coordinator, instructional designer, and content expert, there was a finisher, a multimedia designer, and a quality assurance reviewer, all working together to edit, build, and ensure functional and instructional quality the course in Black-board, the university’s learning management system.

Hattie (2012) found that determining if students have attained success on the identified program outcomes requires using evidence of student learning from the student’s perspective, instructor’s perspective, curricular perspective, and a formative and summative perspective.

The data reveal that students’ performance on program outcomes 1–5 has remained consistent (see Table 2). The students were able to meet learning expectations across three semesters, fall 2012, spring 2013, and fall 2013, with none of the outcomes being less than 91%. Outcome 6 has had a steady decline in students’ ability to meet expectations, with a three-semester low of 70% in fall 2013. Further analysis of program data aligned with Outcome 6 reveal 12 areas where the ratings fell between 80–89%.

When analyzing the program assessment data across all program courses, the students were able to demonstrate an ability to analyze content, critically think, demonstrate collegelevel proficiency in organization, grammar, and style, and use research to support their thinking. The skills were demonstrated above 90% in all program courses, with the exception of OTL568 Action Research and OTL599 Capstone. Based on the assessment data it appears that the content of these two courses needs to be analyzed and revised.

While overall our MSTL program was effective, students also provided comments about what they liked in the courses, as well as how the course could be improved from their perspective. Some examples of comments included the following:

• “Would love to have a choice for final assessments as students have diverse levels of experiences.”

• “The course could be improved with more interactive or applicable assignments.”

• “Provide examples of what is expected.”

• “Videos or real world examples.”

• “This courses was well designed and very relevant to my practice. I appreciate the real-world application of the critical thinking assignments.”

• “I noticed an improvement in my comprehension level of the material when the weekly critical thinking assignment was altered from a typical paper to a more interactive assessment.”

• “Choice in assignments.”

• “I think that the assignments were beneficial and applicable to my work. I never felt like I was wasting my time … each assignment was thought provoking in a different way.”

The instructors’ perspectives were also assessed through an online survey they completed for all of the courses in the MSTL Program. This allowed instructors an opportunity to provide input regarding revisions based on observations they had of student learning. Instructor feedback consisted of content, research-based effective teaching practices, ordering of instruction of concepts, projectbased ideas, collaboration approaches that would facilitate learning, and more relevant educational practices and processes to address issues that pre-K–12 schools are currently facing.

Based on the feedback and analysis of the curriculum from these various perspectives, all MSTL program curriculum, including course descriptions, outcomes, and content were revised to increase expectations of students’ ability to write professionally, think critically, and implement evidence-based practices regarding what they are learning throughout all program courses. The courses were updated to include:

• Internationalization. Internationalization was integrated into every course through assignments, discussions, multimedia, and/ or content. Specifically, the core courses addressed successful educational programs, practices, standards, and progress in other countries or cultures.

• Interactivity. All modules, in all courses, now contain multiple interactive opportunities to engage with course content that are developed in our new HTML5 interactive templates, making the content both engaging and mobile-friendly.

• Multimedia. Relevant, market-driven multimedia was used in all courses in the majority of modules. When possible, these media provided classroom examples of teachers applying the concepts being taught in the module. Also added were opportunities for teachers to differentiate their learning using multimedia. This is important because students may be preservice teachers, experienced teachers, information technology/technology coordinators, administrators, or some other education professional. Because each student has very different learning needs and educational goals, each course had to provide students with a variety of web-based resources and multimedia to help ensure their success in the course.

• Module Length. The module lecture content, in most courses, falls within the suggested limits of 800–1,500 words, depending on the nature of the course textbook (some courses that are textbook-free required more robust lectures). The university has determined this to be a manageable length for text-based content when supplemented by additional articles, interactive exercises, and multimedia.

• Currency of Case Studies/Materials. Materials in the MSTL courses are current (fewer than 5 years old) where appropriate, realizing that some education-based theories and resources that were developed previously are still relevant in the industry today. Great care was given to each course so that students are utilizing relevant materials in their coursework for future use.

• Reduce Student Costs. Courses were designed to be textbook-free when possible, especially if the multimedia in the course was of high quality. Videos of teachers modeling the strategies that were being taught in the module in K–12 classrooms helped to replace textbooks and still provide students with a content-rich resource. If textbooks were used, they were current, industry-relevant texts or guidebooks that teachers will be able to reference and use again in their specific educational environments.

• Rubrics. Rubrics are directly aligned with assignment requirements and every assignment has a rubric. Assessments are also aligned to course outcomes and each course outcome is assessed in at least one course assessment.

• Assignments. Practice-based assignments that are market relevant are now the norm for all courses. Every course provides students with an opportunity to implement and practice the skills and competencies they are developing in that course. Students who are not presently teaching or who are in an educational role outside of the classroom are given support in finding a network in which to complete their practice-based work. This work will hold students to a high standard to truly develop the skills they need in order to become master educators.

• Certification and/or Industry Alignment. All course outcomes are aligned with national standards for teaching and corporate training (e.g., educator effectiveness rubrics, research-based effective instructional practices, standards). For a culminating program experience students are provided with two final course options: (1) a research-based professional project or (2) researching effective educational programming. Throughout all courses, students learn to analyze student-learning data in order to make evidence-based program, process, or instructional decisions.

Research on effective instructional practices, relevant standards, and educator effectiveness criteria were also used to assess the program from the curricular perspective. Examples of this included analyzing the work of Hattie (2012). He synthesized more than 800 education articles that included 200+ million students and identified various influences on student achievement. The 12 Touchstones of Good Teaching by Goodwin and Hubbell (2013) was also reviewed, and the program coordinator conducted a gap analysis of the curriculum using the components of The Framework for Teaching by the Danielson Group, which is a research-based set of components of instruction aligned to the InTASC standards. In addition to evaluating content for industry relevance, the curricular perspective also assessed courses for internationalization, interactivity, multimedia, student costs, module length, currency of case studies/materials, integration of practice-based assessments, rubric alignment, and certification and/or industry alignment.

Because the number of students taking at least one online course totaled 6.7 million in 2013 (Allen & Seaman, 2013), CSU-Global understands the importance of providing the highest level of instruction possible to students. Committing to a solid instructional model steeped in best practices research and utilizing historical data on the university’s student success rates have proven invaluable.

CSU-Global is committed to the development of quality online courses and processes. All courses are designed and implemented to ensure that CSU-Global promotes student engagement and provides learners with the tools and information they need to be successful. Courses are based on the best practices identified in the research literature as well as sound instructional design principles and standards. The process of maintaining and upholding courses to these standards involves continuous quality improvement that is faculty driven and implemented via a peer-review process. Alignment of learning objectives, assessments, resources, student engagement, and technology all contribute to the success of CSU-Global students.