The development of technologies in distance education continues to grow at accelerating rates. From discussion boards and instant messengers to iPods and PDAs, new technologies offer a number of new conveniences and options for students and instructors alike. These innovations are, nonetheless, accompanied by a growing requirement for valid research findings to guide their selection, design, development, and implementation in distance education programs. Empirical research in the field of distance education seems to, however, continually lag behind advances in technology. As soon as reliable evidence is available supporting the application of either electronic technologies (e.g., Blackboard, podcasts, wikis, virtual learning environments) or conceptual technologies (e.g., theories, procedures, frameworks, models), new technologies change the landscape and send researchers back to the basics. As a result of this dynamic horizon, research is often outdated before results of scientific studies can be calculated, let alone published.
Continually chasing after the newest technologies has, for some time, left research in distance education on a merry-go-round of sorts, with researchers always reaching for a golden ring, but continually going around in circles. This has not only slowed the progress of research in distance education, but also put the discipline in a position of always following technological advances rather than providing research findings that can guide the development of new technologies. Blackboard, WebCT, and other learning management systems are valuable examples of how research in distance education has missed significant opportunities. Instead of providing research to lead the development of these systems, most research in distance education is limited to application studies when these products have already been put into service (and then they will often be updated to a new version before any research is concluded).
This cycle must stop. Distance education researchers must find a way to get off the merry-go-round and start leading the field of distance education with grounded research. Research that is focused on development and assessment of foundational theories, principles, and models has the opportunity to guide advances in distance education rather than merely testing new products.
The scope of distance education research is broad (and continually expanding), drawing on fields including psychology, education, information technology, communications, and business, as well as a diverse host of specific curriculum areas. As a consequence, research questions related to distance education are often a corollary to primary research interest in sister-disciplines or focused on discipline-specific impacts alone. For example, a student in economics who is conducting research for a dissertation may include online courses as a variable in his or her study, but the focus remains on the research questions of economics. Likewise, a science educator may conduct research comparing specific online and on-campus science courses, leaving few generalizations available to researchers in other disciplines.
While these challenges are neither new to distance education nor unique to this discipline, they do present significant difficulties to a field of study that has recently seen tremendous growth in its applications in both education and training. The “demand” for research findings today greatly outpaces our ability to produce quality evidence through empirical research. This isn’t to say that there is not an overabundance of articles being published on distance education; there is. But what is missing from the discipline is a bedrock of foundational research that develops and tests grounded theories, principles, and models.
Without this foundation, research is without a unifying direction and is susceptible to the trends created by new technologies. All you have to do is visit the No Significant Difference Phenomenon Website to realize that a research discipline without a theoretical foundation and comprehensive models will continue to ask similar questions of new technologies without ever making progress on the fundamental questions.
To guide the future of distance education and meet these challenges, the foundations, scope, and rigor of distance education research should be examined by researchers on a regular basis, and new researchers must look beyond product testing. Based in part on the work of Driscoll (1995) and Briggs (1982), in 2003 a colleague and I offered a matrix for conceptualizing pragmatic research on distance education. The matrix is based on the both the research paradigm of the researcher and the sub-systems of distance education development (see Table 1). Each cell of the matrix then provides an example of the type of research that may be a candidate for the particular research paradigm when applied to the specific component of the distance education program.
A Matrix for Conceptualizing Research on Distance Education
| Needs Assessment | Analysis | Solutions and Alternatives | Instructional Design | Instructional Development | Media and Delivery | Evaluation and Continuous Improvement | Diffusion and Adoption | |
|---|---|---|---|---|---|---|---|---|
| Experiment and Quasi-Experiment | Effect of alternative planning tactics in aligning goals and objectives within a strategic planning and needs assessment process | Comparison of alternative task analysis methods for identified performance discrepancies | Effect of changes in specified variable of a reward/incentive program in improving performance | Which of specified variables in materials increase student motivation/mastery/satisfaction. | Comparison of two feedback systems for improving learner performance | Effect of alternative timing of feedback on learner performance | What factors influence a learners level one (or two, three, four, or five) evaluation of an online course | Effects of specific variables in adoption of distance education in a professional school |
| Meta-Analysis | Systematic review of needs assessment research on similar variables in terms of performance accomplishment | Systematic review of task analysis research on similar variables in terms of performance accomplishment | Systematic review of performance system design research on similar variables in terms of performance accomplishment | Systematic review of instructional strategy research on similar variables in terms of performance accomplishment | Systematic review of visual literacy research on similar variables in terms of performance accomplishment | Systematic review of instructional video research on similar variables in terms of performance accomplishment | Systematic review of goal-free evaluation research on similar variables in terms of performance accomplishment | Systematic review of diffusion research on similar variables in terms of performance accomplishment |
| Case Study/Ethnography | Case study of alterative tactics in conducting needs assessments | Case study of teams or organizations conducting needs analysis | Case study of learners using EPSS systems | Case study of projects converting courses for online delivery | Case study of development projects of CD-Rom media | Case study of an institution utilizing a new educational technology | Case study of programs’ improvement processes for training units | Case study of a new technology being integrated into schools |
| Technology Development and Evaluation using a Novel Technique | Development and evaluation of a tool for conducting aligning strategic planning and needs assessment using a novel technique | Development and evaluation of a tool for conducting a context analysis using a novel technique | Development and evaluation of job aid systems using a novel technique | Development and evaluation of tools for increasing creativity problem solving tactics in design using a novel technique | Development and evaluation of tools for developing online activities using a novel technique | Development and evaluation of a tool for integrating mixed media using a novel technique | Development and evaluation of a tool for synthesizing evaluation data using a novel technique | Development and evaluation of tool for generating adoption of a new technology using a novel technique |
| Cost-Effectiveness and Costs-Consequences Analysis | Cost-effectiveness/efficiency of conducting a societal-focused needs assessment | Cost-effectiveness/efficiency of conducting a performance analysis | Cost-effectiveness/efficiency of instructional and/or course delivery alternatives | Cost-effectiveness/efficiency of conducting formative evaluations; summative evaluations; goal-free evaluations. | Cost-effectiveness of the application of computer design tools | Cost-effectiveness/efficiency of compressed video delivery | Cost-effectiveness/efficiency of collecting useful evaluation data online | Cost-effectiveness/ of slow adoption models |
| Model Development and Evaluation | A model for prioritizing needs (gaps in results) in a unique context | A model for conducting context analysis in a unique context | A model for selecting effective and efficient non-training solutions | A model for selecting appropriate/effective learner to learner interactions at a distance | A model for the rapid prototype development of online instruction | A model for effectively using a new media for the delivery of instruction | A model for assessing learner performance on the job after instruction | A model for spreading the use of a technology in a unique population |
| Novel Technique Development and Evaluation | Method for using online technologies in the collection of needs assessment data | Method for conducting a task or performance analysis at a distance | Method for the application and impact of alternative performance technologies | Method for selecting the appropriate media for instructional objectives | Method for using storyboards to develop instruction | Method for motivating learners in distance education | Method for improving low quality distance education courses | Method for involving leaders in the diffusion of a technology |
| Needs Assessment | Analysis | Solutions and Alternatives | Instructional Design | Instructional Development | Media and Delivery | Evaluation and Continuous Improvement | Diffusion and Adoption | |
|---|---|---|---|---|---|---|---|---|
| Experiment and Quasi-Experiment | Effect of alternative planning tactics in aligning goals and objectives within a strategic planning and needs assessment process | Comparison of alternative task analysis methods for identified performance discrepancies | Effect of changes in specified variable of a reward/incentive program in improving performance | Which of specified variables in materials increase student motivation/mastery/satisfaction. | Comparison of two feedback systems for improving learner performance | Effect of alternative timing of feedback on learner performance | What factors influence a learners level one (or two, three, four, or five) evaluation of an online course | Effects of specific variables in adoption of distance education in a professional school |
| Meta-Analysis | Systematic review of needs assessment research on similar variables in terms of performance accomplishment | Systematic review of task analysis research on similar variables in terms of performance accomplishment | Systematic review of performance system design research on similar variables in terms of performance accomplishment | Systematic review of instructional strategy research on similar variables in terms of performance accomplishment | Systematic review of visual literacy research on similar variables in terms of performance accomplishment | Systematic review of instructional video research on similar variables in terms of performance accomplishment | Systematic review of goal-free evaluation research on similar variables in terms of performance accomplishment | Systematic review of diffusion research on similar variables in terms of performance accomplishment |
| Case Study/Ethnography | Case study of alterative tactics in conducting needs assessments | Case study of teams or organizations conducting needs analysis | Case study of learners using EPSS systems | Case study of projects converting courses for online delivery | Case study of development projects of CD-Rom media | Case study of an institution utilizing a new educational technology | Case study of programs’ improvement processes for training units | Case study of a new technology being integrated into schools |
| Technology Development and Evaluation using a Novel Technique | Development and evaluation of a tool for conducting aligning strategic planning and needs assessment using a novel technique | Development and evaluation of a tool for conducting a context analysis using a novel technique | Development and evaluation of job aid systems using a novel technique | Development and evaluation of tools for increasing creativity problem solving tactics in design using a novel technique | Development and evaluation of tools for developing online activities using a novel technique | Development and evaluation of a tool for integrating mixed media using a novel technique | Development and evaluation of a tool for synthesizing evaluation data using a novel technique | Development and evaluation of tool for generating adoption of a new technology using a novel technique |
| Cost-Effectiveness and Costs-Consequences Analysis | Cost-effectiveness/efficiency of conducting a societal-focused needs assessment | Cost-effectiveness/efficiency of conducting a performance analysis | Cost-effectiveness/efficiency of instructional and/or course delivery alternatives | Cost-effectiveness/efficiency of conducting formative evaluations; summative evaluations; goal-free evaluations. | Cost-effectiveness of the application of computer design tools | Cost-effectiveness/efficiency of compressed video delivery | Cost-effectiveness/efficiency of collecting useful evaluation data online | Cost-effectiveness/ of slow adoption models |
| Model Development and Evaluation | A model for prioritizing needs (gaps in results) in a unique context | A model for conducting context analysis in a unique context | A model for selecting effective and efficient non-training solutions | A model for selecting appropriate/effective learner to learner interactions at a distance | A model for the rapid prototype development of online instruction | A model for effectively using a new media for the delivery of instruction | A model for assessing learner performance on the job after instruction | A model for spreading the use of a technology in a unique population |
| Novel Technique Development and Evaluation | Method for using online technologies in the collection of needs assessment data | Method for conducting a task or performance analysis at a distance | Method for the application and impact of alternative performance technologies | Method for selecting the appropriate media for instructional objectives | Method for using storyboards to develop instruction | Method for motivating learners in distance education | Method for improving low quality distance education courses | Method for involving leaders in the diffusion of a technology |
Source: From Watkins and Schlosser (2003), based in part on Driscoll (1995) and Briggs (1982).
Note: The elements of educational research on distance education (along the top) and research paradigms (down the side) are not intended to represent all possible research in the field. Rather, they offer a reasonable sample and a starting place for those looking to develop a research agenda.
Though the matrix does not provide a conclusive array of all possible research on distance education, it can provide initial guidance for researchers (Watkins & Schlosser, 2003). The matrix offers a map (or frame) for viewing the many types of viable and valuable research questions that can be asked within distance education, which can be especially helpful to novice researchers. Nevertheless, research must be directly linked to a theoretical foundation that can provide a basis for both research hypotheses and findings. It is, therefore, the challenge of seasoned researchers in distance education to provide the theories, principles, and models that can be tested through these various research methods.
Guiding the future of distance education can (and should) be a partnership of technology developers and researchers. As researchers in distance education, we can begin to provide leadership in this endeavor by producing a significant base of grounded research now and in the future.

