CHAPTER 2: Questioning the Meta-Analyses of Computer-Based Instruction Research

A 30 percent sample of the computer-based instruction (CBI) studies metaanalyzed by Kulik et al. was examined for evidence of confounding. The purpose of the analysis was to explore the validity of competing claims about the contribution of the computer to measured achievement gains found in CBI studies. Some of these claims propose that CBI effects are overestimated and others argue that CBI effects are underestimated. The result of the analysis strongly suggests that achievement gains found in these CBI studies are overestimated and are actually due to the uncontrolled but robust instructional methods embedded in CBI treatments. It is argued that these methods may be delivered by other media with achievement gains comparable to those reported for computers. Construct validity cautions are offered for those who wish to use meta-analytic results as evidence for implementing CBI in schools and for the design of future CBI research.