Learning Through Web-Based Multistoryline Case Studies: A Design-Based Research

Case studies have been widely used as a teaching tool in teacher education. Well-designed cases are more vivid and contextual than book discussions, yet more disciplined and manageable than observing or doing work in the real world (Shulman, 1992). To illustrate the ambiguity and complexity of teaching in real classrooms, teacher educators use cases in teacher preparation and in-service teacher training programs to help students gain a deeper understanding of specific principles and learn how to apply theories to situations they may encounter when they are in the workforce (Koh & Branch, 2004). Cases exemplify a great variety of teaching issues and present an alternative to learning in the field (Richert, 1991).

Effectively using case studies, students can make analogical inferences which include identifying important issues, forming ideas about how to make progress, and analyzing the effects of solutions they have come up with (Kolodner, 1997). In the process, students not only learn new cases but also learn new concepts and new content knowledge.

Traditionally, cases, mostly text based, are used widely in classrooms for various purposes including knowledge acquisition, knowledge transfer, situation confrontation, problem exploration, and analysis and synthesis skills development. With the development of information technology and boost of e-learning programs, cases are portrayed in a more realistic way (Bronack & Kilbane, 1998) and become more and more popular in online learning environments to support a wide variety of teaching functions (Smith, Smith, & Booane, 2000). Online learning environments combined with various media provide rich representations of case situations and flexibility for students to work on cases at their own convenience.

McLelian (2004) argues that online case studies can be used to engage students, enhance their interest, improve retention, and promote students’ problem-solving skills. Research by Nathoo, Goldhoff, and Quattrochi (2005) showed that using online cases, students identified real-time engagement, stronger relationships with faculty, increased accountability to the material; cases also reflected more realistic student experiences.

Moreover, several studies suggested that teacher preparation programs can be enhanced by case studies. For example, Russel, Bebell, O'Dwyer, and O'Connor (2003) claim that case studies have the potential to influence preservice teachers’ beliefs about how to use computer technology and, ultimately, have impact on their practices in real classrooms. In Andrews’ (2002) study, teaching cases were integrated into a web-enhanced instruction for preservice teachers to adapt instruction for limited English proficient students with disabilities. Most teachers thought that analyzing teaching cases and developing adapted lesson plans for the case characters increased their ability and confidence to make curricular and instructional adaptations for different students’ needs. Teaching cases have been shown to increase transfer of learning from theory to practice and improve novice teachers’ classroom problem-solving skills.

Bronack, Kilbane, Herbert, and McNergney (1999) found the combination of cases and technology provided a valuable opportunity to engage developing preservice and in-service teachers’ professional behaviors. By analyzing over 40 participants’ written comments, these authors concluded that preservice and in-service teachers developed and refined their ability to recognize, analyze, and address professional problems via exploring authentic teaching tasks in a web-based, case-based learning environment. By analyzing various data sources including questionnaires, reflection papers, course evaluations, and focus interviews, Angeli (2004) also speculated that case-based learning affected preservice teachers’ beliefs and conceptions about the use of information and computer technology.

Using design-based research, we investigated the effects of case studies, specifically designed under the guidance of Spiro's cognitive flexibility theory (Spiro, Vispoel, Schmitz, Samarapungavan, & Boerger, 1987), on preservice teachers’ knowledge acquisition and, at the mean time, documented their reactions to the different case features.

In traditional instructional design study, research is usually conducted after instructional design processes have been completed and is used to test the design's effectiveness rather than to address issues of educational practice (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003). The goals of design-based research projects, however, include research, design, and pedagogical practice (Joseph, 2004). Designing an effective learning environment and developing theories of learning are intertwined in design-based research through continuous cycles of design, enactment, analysis, and redesign (The Design-Based Research Collective, 2003). The research process frequently involves social interactions with participants sharing ideas (Collins, 1999), and multiple methods can be used to analyze the outcomes of an intervention and to refine it (Cobb et al., 2003; Wang & Hannafin, 2005).

Collaborating with participants to manage research processes, we designed and implemented interventions systematically to refine and improve initial designs, seeking to improve educational practices (Wang & Hannafin, 2005). We proceeded through repeated iterative cycles of design and implementation and used each implementation as an opportunity to collect data to support subsequent design (Edelson, 2002).

The design of the case studies was rooted in cognitive flexibility theory, which emphasizes real-world complexity and ill-structuredness of knowledge domains (Spiro, Feltovich, Jacobson, & Coulson, 1992).

The learning objectives addressed by cognitive flexibility theory mainly focus on advanced knowledge acquisition. As suggested by cognitive flexibility theory, multiple case studies should be used to ensure that a variety of possible situations are presented; cross-case differences in how concepts and principles are applied should be focused on; and multiple perspectives should be considered as an aid to understanding the connected nature of domain concepts and promoting flexible knowledge building (Spiro et al., 1987). Through the progress, students can develop a better understanding of the important elements of conceptual complexity, use acquired concepts for reasoning and inference, and flexibly apply conceptual knowledge to new situations (Spiro et al., 1992).

In this study, we designed multistoryline case studies using cognitive flexibility as the framework. Featuring different storylines, cases in this study were designed to impart multiple themes and perspectives, leading to a better understanding of knowledge embodied in the cases. Moreover, we incorporated tools such as animated assistants, content maps, and practice quizzes into the case studies to help addressing students’ various needs.

Using cognitive flexibility theory as the theoretical framework and design-based research as the research method, we designed, developed, and tested multistoryline case studies through repeated design cycles.

The initial case design was drawn from literature review and needs assessment. We looked at literature regarding online case studies and eventually narrowed our focus of attention to literature regarding online case studies as a teaching and learning tool in teacher education. We interviewed an instructor who taught measurement and evaluation courses and analyzed evaluation data (i.e., students’ midterm feedback survey and end-of-semester survey) collected from an undergraduate measurement and evaluation course. Using these resources as jumping-off points, we identified two case themes that students were most interested in—standardized testing and classroom assessment—and chose multistoryline as the format of our online case studies.

We created a variety of storylines around these two themes and asked the instructor and a volunteer in-service teacher to review these storylines for authenticity. After the case studies had been developed, we piloted them to a group of 11 students from the college of education. during the pilot, we observed students’ reactions to the case studies and interviewed them to collect data about their interests and knowledge about the case studies. The pilot was also used to test the validity and reliability of the instruments that we planned to use in a large-scale study.

Analysis of the case studies yielded results that were neutral. Although students showed interest to multiple storylines, they did not like the layout of the case studies. The multistoryline structure gave them the flexibility to view case scenarios from multiple perspectives; however, some students felt they could not catch the knowledge or concepts we wanted to convey in some of the case scenarios and suggested that assistance should be available when needed.

We developed some explanations based upon our analysis to the data collected: (a) PowerPoint might not be a good platform to present the multistoryline structure; (b) students had different levels of subject knowledge, and we needed to provide support to those who were lack of subject knowledge; and (c) the cases we had developed needed more motivational affordances.

In some research paradigms, research would be used to tease out the power and relative weights of these factors in causing the phenomenon of interest; in design-based research, we could address these factors through designing a solution (Joseph, 2004). We redesigned the interface using Adobe Flash, which gave us the flexibility to add interactivity to the case studies. When browsing cases, students interact with case characters and answer the questions asked by case characters. Based on their chosen answers they are led to different storylines. When they finish one storyline, they can continue exploring other storylines.

To help addressing students’ different knowledge levels and various learning styles, we incorporated three learning tools into the case studies: assistants, content maps, and practice quizzes.

We designed two animated assistants, one as a subject matter expert and the other one as a mentor for a new K-12 teacher. The two assistants, showing up in pop up windows when students call on them, give explanations to the concepts and knowledge covered in the cases.

To help students navigate through the various storylines, we designed two content maps as advanced organizers using flowcharts. Same as the assistants, students can open the content maps in pop up windows when clicking on a button on the interface. The two content maps arrange and present concepts and knowledge points covered in the cases in a progressive manner that is easy for students to follow. Important concepts and topics are linked by arrows to present the whole knowledge structure of the cases as well as the progress a certain student has made thus far in his or her case study process.

At the end of each case scenario, we designed practice quizzes so students can selfcheck their understanding of the important concepts and knowledge covered in the cases. When students choose an answer, whether it is right or not, they will be shown the result with instant feedback—detailed explanation about why their answer is right or wrong.

The new design was based on groundwork from the exploratory investigation and was tested through another pilot phase. After minor revisions to the case studies, we tested them to a convenience sample of 138 undergraduate students who took an undergraduate measurement and evaluation course.

During the 2-week study time, students logged in to a course management system, studied the cases at their own pace, and finished all the required tests and surveys. The parallel 25-item pretest and posttest were used to measure students’ knowledge acquisition through the online cases. After finishing the cases, students were requested to complete a 33-item learner feedback questionnaire, which comprises both objective and open-ended questions drawn on students’ overall reactions to the cases and their reactions to different features of the cases.

In design-based research, researchers generally aim at answering questions to the design of the intervention itself, and investigations are centered on the evolution of the designed product (Joseph, 2004). A design-based research might hypothesize that a certain kind of representation will help learners develop certain domain specific knowledge, create artifacts that embody this hypothesis, and place them in the real world for testing.

In this study, we focused on questions that impacted design and hoped to address the key hypotheses embedded in the design. The study addressed the following questions: (a) Do students learn from the designed online case studies? (b) What learning tools (i.e., assistants, content maps, and practice quizzes) do students find beneficial to their learning? (c) What implications do the results have for the refinement of online case studies?

To promote the validity, reliability, and applicability of the research, we collected data from multiple sources (Wang & Reeves, 2006). As described earlier, we used parallel 25-item pretest and posttest to measure students’ performance results. Due to the lack of a comparison group we could not control the variables that might affect students’ score increase. This was one limitation of the study. We thus applied quantitative analysis only as a tool for describing and interpreting data and focused our study on the specific case design.

We used a 33-item learner feedback questionnaire to collect students’ feedback on their experiences of using the web-based multistoryline case studies and the various learning tools incorporated into the cases. Both objective and subjective questions were used in the survey questionnaire. Students were asked to indicate their reactions to the whole multistoryline cases and to the various learning tools incorporated into the cases; they were also asked to specify the reasons for those reactions. We analyzed the data quantitatively by extracting the mean values and percentages of the students’ ratings and qualitatively by analyzing and categorizing the reasons the students gave for the various reactions.

Students showed significantly higher test scores after learning the online cases (t(110) = −14.116, p < 0.1). The effect size equaled to 1.2. They responded that the online cases were easy (or not difficult) for them to use, and they were satisfied with the information that they had received from studying these cases.

The students showed higher preference for using online cases than for reading textbooks (74%). They indicated they prefer to use online cases because the cases are interactive, challenging, practical, interesting, and hands on. They interacted with case characters and answered the questions case characters brought forward. Based on the different answers they chose they were led to various case scenarios following different storylines. The whole learning process was engaging and required more participation than book reading, thus students felt the cases kept their attention. They asserted that the multiple storylines guided them through the cases step-by-step by showing them different perspectives of case scenarios. After finishing studying the cases, their questions were answered clearly. They also enjoyed the layout of the cases and thought it is very interesting.

The two animated assistants, who show up in pop up windows, explain knowledge and concepts conveyed in the case studies. Students showed great interest in this feature. They reported that this feature is very useful for their case studies. The two assistants “explained everything very clearly, clarified any questions they had, and aroused their interest.” They felt it was “very neat” to have them in the cases.

Online learning should accommodate students’ various learning styles. We designed content maps to support students who prefer a well-structured visual guidance during case browsing. Feedback collected from students proved that this feature is “helpful and easy to use.” Students asserted that by viewing the content maps they “clearly understood how ideas fit together in a visual way and thus organized information more effectively.”

Practice quizzes are often used in learning process as formative evaluation. At the end of each case scenario, we designed a practice quiz with feedback. Students were enthusiastic about using these practice quizzes and considered them the most helpful feature. They stated that practice quizzes “summarized what they should master from cases with clear feedback and tested their comprehension.” They indicated that “prompt feedback upon their choices of quiz answers triggered their motivation of finding out why they were right or wrong.”

The main goals of design-based research include solving educational problems and deriving design principles for future innovations (Wang & Reeves, 2006). The result of this study showed that the web-based multistoryline case studies we designed were beneficial for students’ knowledge acquisition; the survey results provided evidence that students displayed interest in learning through these cases. The study also yielded implications for design principles related to future design of online cases, which included the themes discussed below.

A good design is always based on thorough analysis of learner and content. Drawn on needs assessment and evaluation data collected from previous semesters, we picked the themes and case scenarios. After the cases were developed, we conducted formative evaluation through expert review, one-to-one evaluations with learners, small group evaluation, and field trial. We conducted two pilot studies with two groups of preservice and in-service teachers and revised the cases based on their comments and suggestions to ensure the authenticity of the cases. Students who participated in the study described these cases as “very beneficial and have more validity.”

One goal of our study is to support students to acquire as much information from the cases as possible. We designed our cases using Adobe Flash, making the interface interactive and different features animated; these are quite different from any text-based cases students have read before. Instead of using a single scenario, we used multistoryline to cover various themes and perspectives of possible scenarios that might happen in real classroom situations; we also used assistants, content maps, and interactive quizzes with prompt feedback to engage students’ learning. All these efforts helped arouse students’ curiosity and motivation to keep exploring the cases. Through the learning process, students better understood the complexities of topics and deepened their knowledge.

A missed question is a teachable moment. Rather than simply letting students know whether their answers are right or not, we provided “why” or “why not”—a short explanation of why a given response is correct or incorrect—to maximize learning from practice. Students can use the detailed feedback to build the right mental model or correct misconceptions.

Students exhibit significant individual differences in cognitive processing styles that they adopt in learning activities (Robertson, 1985). Thus, we included various features in the cases to adapt to students’ various learning styles. For example, content maps present information in a visual format that helps visual learners. Most students stated that they benefit from content maps, assistants, and practice quizzes. All these features appealed to students’ interests and needs and attracted their attention.

Besides the above design principles, it is very important to include students and teachers in the design-based research process. Most likely, teachers are the ones who will implement the final product, and students along with teachers are the ones who will use and evaluate it. Their input should be integrated into the coordination of objectives, content, pedagogy, technological affordances, and assessment strategies (Wang & Reeves, 2006).

In this study, we worked with teachers to analyze the course evaluation data they provided and used the data to identify students’ interests and the themes of the cases; we discussed case storylines and reworded the wording of the cases with both teachers and students to ensure that these cases represent real school situations and are directly related to participants’ background and experiences. We piloted the cases and used the data collected from the study for case revision. Interacting with practitioners helped us determine the feasible solutions to the instructional problems.

Our next goal is to use online discussion board to bridge the gaps between diverse students and to enhance knowledge sharing among communities of learners, teachers, and experts. In the mean time, we would like to set up a case library collecting high-quality student generated case scenarios.