Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies is must reading for anyone involved in education generally, and distance education specifically. This report is a comprehensive review of 51 studies that:
“contrasted an online to a face-to-face condition,
measured student learning outcomes,
used a rigorous research design, and
provided adequate information to calculate an effect size.” (p. ix)
The report’s most quoted conclusion is printed in italics in its abstract and states, “The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction” (p. ix).
The 70-page report is well-written, informative, and scholarly. It is an important document that attempts to provide a state-of-the-research report on the effectiveness of online/distance education. Unfortunately, unless carefully read, the report can be misleading.
On page 51, the report’s authors, staffers from SRI International’s Center for Technology in Learning under contract to the U.S. Department of Education, clearly state what should be the most quoted outcome of this meta-analysis:
Clark (1983) has cautioned against interpreting studies of instruction in different media as demonstrating an effect for a given medium inasmuch as conditions may vary with respect to a whole set of instructor and content variables. That caution applies well to the findings of this meta-analysis, which should not be construed as demonstrating that online learning is superior as a medium. Rather, it is the combination of elements in the treatment conditions, which are likely to include additional learning time and materials as well as additional opportunities for collaboration that has proven effective. (p. 51)
Learning time, materials and collaboration—the big 3. Apparently online students spent more time, had access to more materials, and collaborated differently than did the traditionally taught comparison students. No wonder online students tended to achieve better.
What we do not know from this report is why some students spent more time, accessed different materials, and had more collaboration opportunities. It is somewhat unfortunate that these important outcomes were not stressed instead of the misleading conclusion that “students in online learning conditions performed better.”
Many will remember the meta-analyses of the 1980s that also misled a generation of educators into thinking that computer-based instruction was superior to classroom instruction (Kulik, Bangert, & Williams, 1983; Kulik, Kulik, & Cohen, 1979, 1980). The “Kulik” studies, as they were called, concluded that students using com-puter-based-instruction achieved better than students who were traditionally taught. More critical analyses revealed that most of the studies included in the “Kulik” studies were methodologically flawed (Clark, 1983). Unfortunately, a whole generation of educators implemented computer-based instruction, and then waited for positive effects that never materialized.
Certainly, the USDE Report is important. It represents a review of the best studies available. The Study’s authors made every attempt to be methodologically and conceptually rigorous. Perhaps the author of the abstract was a marketing adviser rather than a researcher. At any rate, this report should be read and analyzed by all distance educators.
And finally, as George Washington said over 230 years ago, “facts are stubborn things: and whatever may be our wishes, our inclinations, or the dictates of our passions, they cannot alter the state of facts and evidence.”
About the author
