Web-based multimedia case studies: ahead of our time?

Web-based multimedia case studies: ahead of our time?

Jay LiebowitzDepartment of Information Systems, University of Maryland ­ Baltimore County (UMBC), Baltimore, Maryland, USA.

Gayle Yaverbaum and Lucina SilversInformation Systems Program, Penn State University at Harrisburg, Middletown, Pennsylvania, USA

KeywordsCybernetics, Education, Information systems, Internet, World Wide Web

Abstract The use of case studies is a common part of today's education and business environment. With the World Wide Web, the dissemination of case studies can be greatly enhanced. A number of universities are developing Web-based case studies for their classes. This paper suggests that doing this may not be worth all the effort.

Introduction

Case studies are a popular pedagogy in today's business education. In the information technology field, studies show that only 20-30 percent of information systems are successful. The case study approach is a well-recognized vehicle to show how information technology projects have failed (Liebowitz and Khosrowpour, 1997). Important lessons learned can be gained from these failures via case studies (Liebowitz, 1997).

With the advent and popularity of the World Wide Web (WWW), Web-based case studies can be developed to allow maximum dissemination and application. With the use of the WWW, hyperlinks and multimedia capabilities can be interwoven and embedded within the Web-based case studies for added functionality.

With all these additional,interactive features, one would think that students would prefer to work with Web-based cases versus textbook-based cases (i.e. reading case studies out of the textbook). However, this may not be the case. According to the October 1997 Communications of the ACM article titled "Viewpoint" (pp. 27-8), Richard Hsu and William Mitchell describe a recent experience at the Stanford Graduate School regarding the use of interactive cases:

A marketing professor recently utilized the school's first online business study case. The idea was to allow more efficient learning through the use of pictures, hypertext links and other recent developments in information design. Before beginning the usual course debate on business issues, however, the professor asked a question: "How many of you printed the entire case rather than read it on the screen?" In a class of 70, about 50 hands went up ­ and this is on a campus which does everything on computers and email! Whatever the students may have thought of hypertext, they certainly didn't want to read it on a computer monitor.

This paper will discuss the development of a multimedia Web-based case study and some evaluation results from the graduate students who used it. The underlying theme is that from the student's viewpoint, Web-based cases (versus textbook cases) may be ahead of their time in terms of effectiveness and preference.

The Integrated Advice Management, Inc. (IAM) tax help desk expert system case

For two graduate courses on"Applied expert systems" and "Case studies in information systems", we designed and encoded a multimedia Web-based case study for student use. This case study can be accessed at: http://cac.psu.edu/~gjy1/temp/txhelp.html{another earlier Web-based case study was also built and can be accessed at: http://cac.psu.edu/~gjy1/case(Yaverbaum and Liebowitz, 1998)}. The case involves employees of IAM (Integrated Advice Management, Inc., a company specializing in IT applied to financial/tax services), a fictitious organization that provides help desk services for the US Government. There are approximately 5,000 front-line assistors who help answering questions regarding the instructions for completing the tax forms. The assistors do not provide tax advice. Their primary function is to clarify any procedural/instructional questions.

The case involves the evaluation of Tax Advice System or TAS, an expert system developed to aid the assistors in these tax-related calls. The system development started with preliminary studies and the final proposal was approved in 1991. By 1994, the system was built and transferred from the IAM's Expert System Lab to the evaluation user group. At this point, the system was being used by 400 front-line assistors and the accuracy rates were in the mid-90 percent range. The IAM Expert System Lab provided one year of full support in testing, enhancing, and maintaining the expert system and TAS implementation seemed to be achieving all its objectives. After the first year, the system's usefulness and accuracy greatly diminished and managers were faced with discontinuing TAS, investing in TAS and upgrading its accuracy, or start everything again.

Features of the IAM case

This Web-based case can be accessed by any Web browser and contains hyperlinks, graphics, animation, and text. The system is frame-based and the navigation template/frame contains the main menu. The menu includes:

• Scenario I: 1995

• Scenario II: 1996

• November 1996 meeting

• Your feedback

• Organization chart

• Timeline of events.

The case first describes some broadcast news in April 1995 where a TV reporter is interviewing the President of IAM at one of the IAM facilities to discuss the use of the Tax Aid System,TAS. A conversation in 1995 between a front-line assistor (i.e. an IAM employee)and a taxpayer takes place over the phone as TAS is being used. Then, about a year later, a scenario in 1996 occurs involving a phone conversation between the same IAM employee and a worried taxpayer, a memo from a worried front-line assistor, and an invitation to attend a meeting. The memo highlights that several issues concerning TAS performance have been brought to the attention to the President of IAM by concerned supervisors. They believe the system has not been adequately updated relative to the new tax legislation and misadvice is being given to the taxpayers. A meeting is called with the MIS Department and selected others to perform group brainstorming in order to determine the next steps. In this November 1996 meeting, the IAM President makes some opening statements, and then graphics showing the TAS history, TAS statistics, and IAM organization chart are presented, along with alternatives on how next to proceed. A feedback form with multiple choice questions is shown (which needs to be completed by the student), along with an open section for asking the students to provide their comments and suggestions and e-mail these forms (via clicking on the "submit form" button) to their professors.

The case was developed in Java with graphics from the Net-User's Graphics Archive, OurDomain.Com, and Clip Art Universe. The development took about four months with one student developer, a professor serving as the multimedia/Web technology specialist, and another professor serving as the content specialist. About 40 information system graduate students at George Washington University used this case study and evaluated its effectiveness and usefulness via a questionnaire and case critique.

Preliminary evaluation results

The questionnaire consisted of ten questions with an open section for general comments. Each question and recommendation(s) will be discussed in turn.

Question 1: I learned from the case

This question had mixed responses. Part of the problem was due to not specifically stating the case objectives in the beginning of the case. In this manner, students were not exactly sure as to what the learning objectives should be when reviewing the case. However, as the students worked through the case, it quickly became apparent that they needed to decide on the future of TAS. Recommendation: State the learning objectives up front in the case.

Question 2: I preferred using the Web-based version versus reading a case from the book

Most of the students surveyed still preferred reading a case from a textbook instead of working through the Web-based version of the case. This was an interesting result because the students surveyed were primarily information systems students who should enjoy taking advantage of technology. Of course, studies have indicated that it is about 20 percent slower reading off the computer screen versus reading from a book. The students mentioned that they liked to be able to read the cases in bed, on a couch, and in some relaxed position instead of having to sit in front of the computer reading through the case and associated hyperlinks. This finding seemed to echo the Stanford experience cited earlier in this paper. Recommendation: Do not clutter the Web-based case with too much verbiage and expect the students to want to be able to print out the case.

Question 3: The interactivity of the case enhanced the presentation and my understanding of the case

Most students indicated that they liked the interactivity of the case, but suggested that overkill should be minimized with the use of hyperlinks, video clips, graphics, animation, and sound. Recommendation: In other words, be careful not to dress the case too fancy in terms of its appearance.

Question 4: I enjoyed being able to e-mail my responses to you directly from thecase

The students greatly enjoyed this feature of sending the responses directly by e-mail to the professor. However,they wanted a way to get instant feedback to their inputted responses to questions in the case without having to wait for the professor to e-mail the responses and comments back to the student. Recommendation: Allow the students to be able to both receive instant feedback as well as e-mailing the professor from within the case.

Question 5: The case was presented in a logical manner

The students, for the most part,believed the case was structured in a logical, easy-flowing style. Most of the students knew to follow the menu in the navigation frame item by item. Other students skipped around and felt slightly confused. Recommendation: Include a timeline of events within the case and also state up-front that the student should follow the items as ordered in the menu.

Question 6: The graphics, animation, colors, and text were well done

The students generally agreed that the presentation of the case material was well done; however, they felt there might have been too many graphics presented. Recommendation: Allow for a modicum of multimedia features to minimize overkill.

Question 7: The case needs to contain more content to minimize assumptions

The students felt the content was sufficient in the case. Of course, it is difficult to judge if the student goes through "each part" of the case and associated hyperlinks. Without making the way through each part of the case, the student will not be obtaining all the necessary content on which to base his/her decisions. Recommendation: Indicate a specific warning up-front to the student indicating that each part of the case needs to be reviewed before reaching a decision.

Question 8: I would like to review more cases over the Web

Again, as with question two, most students surveyed were not extremely excited about reviewing more cases over the Web. They preferred to read them in a book as opposed to working through them over the computer. Recommendation: Educators should think carefully about developing Web-based case studies, as they take a fair amount of time to develop and may not be justified in the student's mind.

Question 9: Student feedback on the answers to the issues part of the case is needed

The students wanted to be able to see if their responses were correct to the multiple choice questions in the case immediately upon submitting the form. They did not want to have to wait to hear back from the professor on the correctness of these choices. Recommendation: Allow a mechanism for immediate student feedback regarding the correctness of their answers to questions in the case.

Question 10: All sections were generally well done in the case

For the most part, the students felt the Web-based case was done well. Recommendation: Improvement could be made in stating learning objectives, providing more content and assumptions, reducing some graphics, and allowing for immediate student feedback.

Summary

Designing and building a Web-based case study is not an easy task. It requires programming skills, instructional technology skills, visual and artistic flair, and sound pedagogy. From this case, preliminary findings suggest that it may not be worth the effort. Students seem to enjoy reading cases from the textbook over working through Web-based cases. Hsu and Mitchell's article in the Communications of the ACM may be correct in stating, "After 400 years, print is still superior!" Perhaps, from the student's perspective, Web-based case studies are still ahead of our time!