Effectiveness of Combined Delivery Modalities for Distance Learning and Resident Learning

Technology in the information-based sector of the US economy has linked learners at a distance in a variety of industries and professions. Physicians attempting to reclaim the control of their destiny under managed care have emerged on the distance education scene. The purpose of this paper is to explore the effectiveness of such a distance education program on learning for physicians and what progressive administrators can do to remove the traditional obstacles to unpacking a distance education program. This exploration will include:

1. a commentary on the effects of global change and technology on the economy;

2. discussion of the University of Tennessee’s Physician Executive MBA (PEMBA) Program using a mixed-mode of delivery: residence periods (RP), synchronous cyber classes (CC), and asynchronous Web works (WW) and;

3. description of the methodology, results, implementations, and discussion of the evaluation of learning using the different modes of delivery.

The challenges of change in business are global in scale. They range from sophisticated global competition and demands for improved quality, to the pressure to reduce the time it takes to communicate, produce, deliver, and respond to cost-cutting and downsizing agendas. These and other socio-political and economic forces drive the need for change in organizations. Among the realities of today’s business, swift and demanding currents of change exist in health care as well. Healthcare organizations and hospitals are constantly negotiated and renegotiated in response to immediate crisis and the practicalities of change (Stevens, 1991). The focus for this paper about distance education and its value to healthcare uses two vantage points that set up the context of this study:

1. the need of physicians for basic business knowledge and;

2. newer technologies enabling improved quality, better communication and faster delivery of information.

The information-based sector of the U.S. economy is about 15 percent of the gross domestic product. That sector includes computers, entertainment, and telecommunications. When distance education is factored in, the percentage increases. This information-based sector will influence learning in the global economy. Are we ready? Kelly (1998) in New Rules for the New Economy suggested restructuring our strategies for a connected world.

In New Rules for the New Economy he suggested that human expression, thought, communication, and even human life itself have been touched by technology. This is the result of a new economy created by shrinking computers and expanding communications, of which distance education is a major player. In fact, Kelly said that communication is the economy. With computers as an ally, communication impacts the cultural, technological and conceptual foundations of our lives. This leads to a networked society. The issues of distance education beg for attention with respect to both the content and the process of passing on knowledge and skills. Other voices joined in the chorus championing this network of capitalism and learning.

Shapiro and Varian (1998) gave an overview of the network economy in their book entitled, Information Rules: A Strategic Guide to the Network Economy, where they spoke about online environments and their effect on the economy. Hagel and Armstrong (1997) revealed the new economy as seen through the vantage point of commercial communities and emerging networks in their book entitled, Net Gain: Expanding Markets through Virtual Communities. The Rise of the Network Society by Castells (2000) unpacked the consequences of the ongoing transformation of society by network technologies. Webonomics by Schwartz (1997) discussed the practical problems of using Web sites to create commerce. Martin (1996) in his book entitled, The Digital Estate: Strategies for Competing, Surviving and Thriving in an Internetworked World, revealed the business models at work on the Web that are not visible with online businesses. The classic textbook for e-commerce is a book by Whinston, et al., (1997) entitled The Economics of Electronic Commerce. The Death of Distance: How the Communication Revolution Will Change Our Lives by Cairncross (1996) is a factual treatise of the effect of global communications. Digital Money: The New Era of Internet Commerce is by Lynch and Lundquist (1995) who discussed the nature of money in the emerging global economy.

The point of all these references is to demonstrate the amount of confidence professionals in business have in this network of technology. The reality of the reach and range of this networked society unfolds every day, and every day more links are connected. This allows the network to expand, confident that the technology will allow successful exchange of information, money and ideas. One could even say that the global society has already taken hold in the business world, with networked learning exchanges becoming an increasingly commonplace reality for employees. Yet, there still may be doubt about the effectiveness of networked distance learning.

There are some signs of insipient life in education when it comes to adapting to the notion of a networked society, as evidenced recently at Colorado State University (CSU). CSU (Carnevale, 2000) revealed a plan to create a virtual college that relies heavily on distance education. It is truly a virtual college, as there will be no new physical facilities created for it. It is designed to teach computer savvy so students can be marketable in the workplace. Physicians need this kind of computer savvy too, as well as knowledge of business basics in order to respond to the changes in health care.

Table 1 lists programs targeting health-care executives and doctors in the southeast US. The five programs differ in approach, yet the region’s third physicians-only program at the University of Tennessee, opened their doors up with a comprehensive and networked strategy. Students meet on the Knoxville campus for four weeks during the year-long program, taking the rest of their classes on-line. The remaining sections of this paper focus on the effectiveness of the Physicians-only Executive Master’s in Business Administration (PEMBA) program at the University of Tennessee.

The University of Tennessee, Knoxville College of Business Administration is in the third year of its groundbreaking Physicians-only Executive MBA (PEMBA) program. The program is delivered to a national audience with only four total weeks of on-campus residence during the year-long compressed program. Making use of physician feedback, PEMBA’s faculty designed, implemented, and evaluated a tightly integrated MBA program, leveraging Internet synchronous and asynchronous distance learning technology. PEMBA’s alumni are now actively involved in new leadership roles ranging from inner city health clinics to offshore pharmaceutical e-commerce.

PEMBA equips physicians with sound business fundamentals and the knowledge to help them understand what impact each decision they make will have on the financial stability of their organizations in the ever-changing healthcare environment. Changes in the government’s role, organizational structures, customers, disease management, financing, outcomes measurement, economics, operations, leadership, law, ethics, communication and information systems impact the healthcare market’s shift from a fee for service payment system to a capitated system. Physicians need to understand the impact of these changes on their practice and their organization. The physicians’ knowledge of business will allow them to strategically position their practices and organizations for future success. Subsequently, by equipping physicians with business fundamentals, this program will further service patient care by insuring high quality physician attention to the needs of individual patients.

The features of the software used in synchronous PEMBA classes include audio over the Internet, where all in attendance can hear each other upon request. This increased control over audio avoids the disadvantages of poor audio quality as found in telemedicine attempts as cited in Byers, et al., (1999). Forty weekly Internet-synchronous “Cyber Classes” deliver real-time, instructor-led classes to the physician’s work or home personal computer via state-of-the-art audio and data conferencing, replicating a face-to-face classroom without physician travel. The professor can control any of the features below: audio-over- the-Internet for real-time interactive communication, synchronized Web browsers for interactive use of the World Wide Web as on-line references pulled up at will during class, shared electronic whiteboard allowing real-time group collaboration, synchronized playback of PowerPoint presentations, and anonymous on-line pop quizzes allowing instructors to adjust content pacing. The value of these software features allow PEMBA to bridge across the silos of the medical specialty as well as between and across states in the USA and different countries.

PEMBA’S design uses the three elements of distance education as cited by Moore (1973; 1993):

1. autonomy or learner independence, which is indicated mostly by asynchronous Web Works (WW);

2. dialogue where there is interaction between learner and instructor, used by PEMBA in Residence Periods (RP), synchronous Cyber Classes (CC) and WW, and;

3. structure, as indicated by the complete year-long syllabus on line detailing both the CC and WW assignments.

Moore later named his theory Transactional Distance Theory (Moore & Kearsley, 1996) where he repeatedly called for empirical research to identify the right combination of dimensions constituting a successful learning environment for a distance education program. This study adds to that empirical research.

PEMBA’s path-breaking Executive MBA program merges fourteen separate core courses normally taught in residential mode over two years into a single, year-long, integrated curriculum delivered through a multiple technology-mode learning delivery system. Four, week-long “Residential Periods”—including the program’s launch—are held on Knoxville’s campus in January, April, August and December of each calendar year quarter.

Between the residential periods, weekly asynchronous assignments are undertaken through the Internet’s World Wide Web. Called “Web Works”, this delivery mode allows physicians to study when and where they wish—24 hours a day, 7 days a week. Web Works accesses hyper-linked multimedia pages through an integrated program syllabus, topical discussion groups who practice cohort learning, automatically downloadable spreadsheets and word processing documents, VCR-like video-on-demand for short audio/ video playback over the internet and browser-integrated e-mail allowing hyperlinks and attachments within messages to instructors and cohort class-mates.

Other key program ingredients include:

1. the creation of an integrated curriculum and consensus grading;

2. on-line post-assessments of learning for each CC to insure the transfer of knowledge;

3. continuous evaluation of the process of learning for RP and WW has likewise been designed;

4. cohort learning groups who engage in group learning over the internet and;

5. experiential learning of this interdisciplinary program is used through-out the program.

The administrators of PEMBA designed and implemented a quantitative pre-post data-based assessment of knowledge and skills expected of physician MBA students. The purpose is to create an on-going, statistically valid database to develop answers to the following questions: How much progress in learning is occurring during this program? Are there statistically valid differences in quality of learning by delivery modality? Such a scholarly analysis, if statistically valid, would not only assist with planning PEMBA’s future program but may assist other educators in taking the bold leap of quantifying and tracking academic outcomes of higher education distance learning programs.

The methodology evaluates the different delivery modes in PEMBA with gain scores in learning from the physicians. Data from the 1999 class was used.

Participants in this study were 26 physicians from the Physician Executive MBA (PEMBA) program in the College of Business Administration at The University of Tennessee. The physicians were from 16 states and Puerto Rico and were enrolled in the second class of the PEMBA program. On average, the physicians were 47 years old and had 18 years of experience in the health care industry.

This study used a pre-test—post-test design. The PEMBA program merged fourteen courses normally taught in residence over two years into a single, year-long, integrated curriculum delivered through multiple systems: Residential Periods (RP), Cyber Classes (CC), and Web Works (WW). With physician work styles in mind, PEMBA’s faculty and administrators designed, implemented, and evaluated a tightly integrated MBA program leveraging Internet synchronous and asynchronous distance learning technology. A one week-long “Residential Period” was held on Knoxville’s campus each quarter. Residential Periods accounted for about 50 percent of the contact hours and involved professor-lead seminar-type sessions. Internet synchronous and asynchronous distance learning technology accounted for about 50 percent of the 600 accreditation-mandated contact hours.

Between residential periods, weekly synchronous classes were delivered through the Internet. These “Cyber Classes” were delivered real-time and were instructor-led via state-of-the-art audio and data conferencing, replicating a face-to-face classroom without physician travel.

The third mode of delivery was asynchronous “Web Works”. It allowed physicians to study when and where they wished—24 hours a day, 7 days a week.

Each faculty member identified between five and fifteen learning objectives for his/her portion of the course. These items were then edited for clarity. The PEMBA Executive Self-Assessment Instrument consisted of the 108 objectives that related to the 11 subject areas created by the faculty. The instrument included a Likert-scale ranging from 0 (no past experience) to 10 (extensive experience). The instrument was administered on the first day of the first program RP and again 50 weeks later on the last day of the last RP.

All physicians attended one week-long RP each quarter, participated in CC on Saturday mornings and were given assignments using WW. The coursework covered MBA subject material including but was not limited to, business ethics, leadership styles, statistics and organizational change (Stahl and Dean, 1999). Instruction was delivered using a single mode of delivery (RP, CC, or WW) or a combination of two of these (CC + WW, RP + CC, or RP + WW) or all three of these (RP, CC + WW). Each professor chose the delivery mode(s) that would be used for each of her/his objectives.

For each of the 108 objectives, a student’s gain score was calculated by subtracting the score on the pre-test from the score on thepost-test. Summary statistics indicated that students differed from each other in regard to their typical gain score. Accordingly, the gain scores were standardized by dividing each gain score for a student by that student’s gain score standard deviation. Descriptive statistics (e.g., means; standard deviations) were calculated to understand the patterns in the gain scores and the standardized gain scores.

Using the professor-specified choices, the 108 objectives were partitioned into seven instructional mode groups:

1. WW only,

2. CC only,

3. RP only,

4. CC + WW only,

5. RP + CC only,

6. RP + WW only, and

7. RP + CC + WW.

Analysis of variance was used to compare the mean standardized gain scores for the different learning modes: RP, CC, WW, and all possible combinations of the three learning modes. The combinations include: CC + WW; RP + CC; RP + WW, and RP + CC + WW. In this study, we considered results significant if the alpha-level was less than .05.

According to the 248 studies that were compiled by Russell (1997), there is no significant difference between distance learning and traditional classroom learning. In other words, distance learning is as effective as face-to-face learning. Our results are congruent with those published by Russell (See Table 2). There is not a significant difference between the mean gain scores of the subjects taught in RP and those taught in a combination of CC and WW (F = 2.52, p = .11). However, our hypotheses do not target the difference between distance learning and face-to-face learning. Instead, we focus on the manner by which a mixed mode of delivery of distance learning (CC and/or WW) may enhance the typical face-to-face method of learning (RP).

Our results suggest that multiple modes of learning do increase the amount learned. A mixed mode of delivery does enhance the quality of learning. More specifically, the data suggest that physicians learn significantly more when CC is used in addition to RP than when RP is used alone (F = 6.58, p < 0.01). Similarly, the physicians learned more when WW and RP were combined than with RP alone (F = 14.30, p < 0.01). Finally, physicians learned more when the three delivery modes were used than with RP alone (F = 9.07, p < 0.01).

Distance education is undergoing a public relations backlash from its early years of implementation. We believe that distance education is critically important today even though California Virtual University has effectively shut down eight months after launch.

Traditional academic institutions are predisposed to dismiss anything not exhaustively validated by decades of critical scholarly review. Due to the recent and rapid emergence of technology-delivery options, very little scholarly study exists to repudiate such claims. Further, the studies that exist generally focus on a single modality within a single course. PEMBA’s multi-mode delivery system stands as a rare proven and user accepted model from which data can be drawn across multiple content areas and multiple delivery modes.

Based on three years delivering the PEMBA program, the authors agree that the power of on-line learning appears to rest in an appropriate combination of both asynchronous and synchronous learning opportunities. Most of Web-based learning today fails to fully integrate the power of multiple delivery modes, generally emphasizing only the value of asynchronous delivery. Having experience in both asynchronous and synchronous systems for PEMBA’s formal in-class learning, it appears that the power of virtual presence/instant mes- saging/peer collaboration extends learning from faculty.

This study demonstrates that a mixed mode of delivery enhances the quality of learning. This is evidenced by the three significant findings in the combinations of RP and CC; RP and WW; and RP, CC and WW. The implication is that the effort to research, design and develop distance learning programs should become a part of the future agendas in education.

One recommendation is that in order to produce more generalizable results, more participants are needed. In addition, these physicians averaged age 47 and were highly motivated to reclaim their rightful place in leading health care decisions. Thus, they do not represent the general population. In addition, we did not have data for objectives that were taught solely in CC or WW. Future research should examine the Web-based learning further to discover the advantages of distance learning, especially when combined with resident learning.