An Analysis of Distance Education Research: Implications for the Instructional Technologist Free

Learning has always been a significant component of individual growth and development. Recently, public- and private-sector organizations have also recognized that learning is a critical component for organizational success (Senge, 1990). In the past, many of the learning events that students experienced required that they a meet at the same time and place. Through the use of instructional technologies, many individuals can now be provided with learning experiences anywhere and anytime.

The history of distance education can be traced back to the mid-1800s in countries including France, Germany, the United Kingdom, and the United States. Moore and Kearsley (1996) identify three generations in the evolution of distance education. The first generation is linked to correspondence study. The beginning of the second generation is marked by the emergence of the open universities. The use of recorded media (e.g., broadcast television) was part of the transition to the third generation in the evolution of distance education. The third generation is characterized by teleconferencing, networking, and multimedia. Today, distance education is becoming a significant “mainstream” activity for many organizations (Simonson, Smaldino, Albright, & Zvacek, 2000).

With the proliferation of distance education programs and their associated costs, it is important to develop theory, describe practice, and evaluate the effectiveness of distance education programs. Significant work has already been completed in the area of distance education theory. Theories based on the autonomy and independence of the learner, learning, teaching, industrialization, interaction and communication, and independent learning have been proposed (Bååth, 1982; Holmberg, 1989; Keegan, 1996; Moore, 1973; Peters, 1971; Wedemeyer, 1977). We believe that professionals in instructional design and technology should play an active role in the development of the knowledge-base for distance education. However, we are concerned with the minimal contributions that instructional designers and technologists have made up to this point. For example, in a premier research journal in educational technology, Educational Technology Research and Development, we identified all articles that concerned distance education over a ten-year period from 1990-1999. We identified three primary research studies, two conceptual/theoretical articles, and two “how-to” and “lessons learned” articles. There are many unanswered questions about the design, development, and effectiveness of distance education programs that are appropriate for professionals in instructional technology to address.

One way for us to begin thinking about how to expand our knowledge of distance education is to examine the existing research. In this paper, we will attempt to describe and analyze, at a general level, the focus of distance education research as reflected in the articles published in two well-known distance education research journals, The American Journal of Distance Education, published in the United States, and Distance Education, published in Australia.

There are a number of definitions of distance education that present various views of the field (Keegan, 1996). Keegan’s definition represents the European view that emphasizes the separation of the instructor and student in both time and geographical location, and the use of an industrialized form (e.g., instructional design team’s effort) of instructional materials. Keegan’s definition also includes provisions for two-way communication, using media for delivery of instruction, and the “possibility of occasional meetings” (p. 44). Simonson, Schlosser, and Hanson (1999) present a definition that reflects the practice of distance education in the United States. Their definition emphasizes the separation of the instructor and student and the use of “two-way interactive telecommunication systems to synchronously and asynchronously connect them for the sharing of video, voice, and data-based instruction” (p. 70). Simonson et al. do not require separation of the instructor and student in both time and geographic location, as does Keegan. Simonson et al view courses taught primarily by compressed video as a form of distance education, while Keegan refers to such courses as virtual classrooms. For the purpose of this article, we will use the Simonson et al. definition when selecting articles for review as it includes a broader range of delivery methods.

Similarly, we will make a distinction between distance learning and distance education. Distance learning describes the learning half of the distance education process (Keegan, 1996); that is, distance learning describes the change in behavior in the leaner. Distance education includes the distance teaching and distance learning processes.

The two distance education journals selected for inclusion in our analysis were The American Journal of Distance Education (AJDE) and Distance Education (DE). For The American Journal of Distance Education, all volumes from 1987-1999 were included in our classification and analysis. For Distance Education, all volumes from 1991-1999 were included. For each article, the author(s) name, publication date, article type (primary research, conceptual/theoretical, literature review, “lessons learned,” “how to,” evaluation, other), topic, audience described, and type of data collected (if appropriate) were identified and recorded in a database. When it was difficult to determine the article type, the investigators’ consensus was used to classify the article. We attempted to classify such studies according to their primary purpose. The number of articles analyzed for AJDE was 222, and for DE the total number of articles included was 161.

A descriptive summary of the results of the classification and analysis for the two journals is presented in Table 1. Approximately 38% (146) of the studies analyzed were classified as primary research studies. The primary research studies were further classified as comparative-experimental, descriptive, or qualitative. Of the 35 comparative-experimental studies, more than 25 reported the results of survey research. A vast majority (over 70) of the descriptive and qualitative studies also used surveys as their primary method of collecting data. Structured interviews and analysis of videotaped sessions were other methods used to collect data. Only a handful of studies assessed student learning. The primary population represented in the studies was students in higher education. Other significant populations included military personnel and private sector employees. Very few studies included public school students.

The conceptual and theoretical papers represented 30% (115) of the 383 studies analyzed. As could be expected, the topics discussed in the 115 papers were varied. Topics included approaches to defining distance education, distance education theories, costbenefit analysis, specific emerging delivery technologies, programs in specific universities, learner independence, distance education policies, and teacher education, to name just a few. It is not our purpose here to discuss, review or integrate the conceptual/theoretical papers, but just to make a modest attempt to describe “the lay of the land.” Other writers have presented the various perspectives and theories of distance education (Keegan, 1996; Moore, 1973; Moore & Kearsley, 1996; Simonson et al., 2000).

Fourteen reviews of the literature reviewed and integrated distance education research. Research areas reviewed included educational television, leadership, and professional development in distance education, management, personalized system of instruction (Keller Plan), computer-assisted instruction, computer conferencing, distance education in the United Kingdom, faculty resistance to instructional technology, and distance education in the military.

The topics discussed in the evaluation articles (14 total) included distance education cost, evaluation of teletraining, use of electronic bulletin boards, evaluation of audio teleconferencing, and evaluation of models for designing online education.

The 52 “lessons learned” articles as well as the nine “how-to” articles discussed specific lessons learned from reflective practice or described how specific distance education programs were developed in a particular context. For example, selected specific topics discussed include descriptions of specific instructional television programs and various distance education projects completed by state, school, national and international organizations.

The 33 articles classified as other consisted of texts of interviews with various people and numerous responses to other articles appearing in the two journals.

The researchers who have contributed to the American Journal of Distance Education and Distance Education have done significant work. A number of variables have already been identified that will facilitate the development of effective distance education courses. However, we concur with Moore’s (1996) conclusion that:

Summarizing across the articles analyzed-even within a particular category (i.e., primary research)-was very difficult. Summarizing across categories was even more troublesome. This difficulty can be attributed to several causes. First, many of the articles were anecdotal in nature and described a particular distance education project for a particular audience in a particular setting. This anecdotal information may be useful to individuals involved in similar projects, but it does not help us identify general trends and variables that help us predict success across distance education settings and contexts. Second, many of the articles summarized did not base their studies on distance education, learning, or instructional theories. This weakness makes it very difficult to evaluate the appropriateness and significance of the questions addressed in the studies and relate and integrate the work of one researcher with another. Third, in many cases a distinction was not made between delivery technologies and instructional technologies. Clark (2000) suggests that:

When researchers do not make the distinction between instructional technologies and delivery technologies, it is difficult to determine causal relationships between variables included in the research studies.

We were particularly interested in the content and methodologies used in the primary research studies included in the two journals. As discussed earlier, a vast majority of the studies included survey methodologies. Many of the surveys collected data on participant reactions to a particular distance education program but few directly assessed students’ learning, a primary goal for distance-education programs. Assessing participant reactions is useful for assessing student attitudes and motivation or to discover unanticipated benefits of a particular program (Clark, 2000), but numerous other research questions need to be addressed. Also, the return rate for the articles analyzed was not impressive, threatening the reliability and validity of the survey research results.

Another criticism of the distance education research is the abundance of evaluation and comparative studies that have the weaknesses identified by Clark (2000). Lockee, Burton, and Cross (1999) suggest a variety of different strategies researchers can use to contribute to the distance education knowledge base. Ross and Morrison (1989) proposed the use of replication studies, which are also applicable to distance education. Their methodology compares instructional strategies across delivery technologies to determine both effectiveness and efficiency.

The use of one way of knowing (as defined by research method) to do inquiry in any field is limiting. There are many methods that can help us address educational questions including arts-based, historical, philosophic inquiry, ethnographic research, case study methods, survey methods, comparative-experimental methods, and quasi-experimental methods (Jaeger, 1997). The use of multiple methods in distance education research would help us develop a better understanding of how to design, develop, deliver, analyze the cost and benefits, and determine the effectiveness of distance education programs as each method of research will allow us to answer different questions. Shulman (1997), when referring to research in education, states: “the best research programs will reflect intelligent deployment of a diversity of research methods applied to their appropriate research questions” (p. 25). The use of a “diversity” of research methods is needed in future distance education research. A variety of research designs are needed to answer questions on distance education. Ross and Morrison (1996) provide a description of a number of designs including aptitude-treatment interactions and longitudinal studies. Other methodologies include developmental studies (Richey & Nelson, 1996) and media replication studies (Ross & Morrison, 1989).

We conclude that the research published in the American Journal of Distance Education and Distance Education has contributed to the development of a significant knowledge base and helped us to identify variables that will help researchers, designers, developers, and managers ascertain the effectiveness of distance education programs. For a discussion of these variables, see Moore (1996) and Simonson et al. (2000). However, we think that in the future, additional ways of knowing as defined by research methodology should be used. Much of the research is not theory-based and many of the results of the reports are not generalizable. Also, there are few programs of research reflected in the content of the articles. Distance education is a multifaceted endeavor. There are many issues to consider, including theory, planning, instructional design, development, delivery, evaluation, and policies. Other related components of distance education systems include learners, content, organizational context or setting, process staff, and communications technology (Chute, Thompson, & Hancock, 1999). The entire distance education system should be considered when planning and conducting research on distance education.

In order to go beyond developing “pockets” of knowledge concerning distance education, significant additional theory-building must occur. Otherwise, it will be difficult for researchers to develop research programs to test the theory and it will also continue to be difficult to integrate the research that is produced. Theories of distance education need to incorporate and relate the facets of distance education that we have discussed. For example, a theory of distance education that only focuses on students and learning that does not incorporate issues of process staff, and costs and benefits will not be a useful theory in the current “lean” economic environment. As researchers add to the knowledge base, others need to provide conceptual and theoretical articles that attempt to integrate the research findings to support, extend, and develop new theories and distance education methodologies. These conceptual and theoretical articles can serve as a barometer of the cohesiveness of the knowledge base developed. We envision that future theory-based studies should incorporate multiple methodologies given the many facets of a distance education system. If there was ever a situation “begging” for multiple research methodologies within single studies we believe distance education research is the prototype.

Learning at a distance is rapidly becoming a way of life. There is a significant need for the development of distance education theories and programs of research based on the theories. It is important that instructional technology researchers participate in the development of the knowledge base for distance education.