Chapter 11: Using Latent Trait Models for Program Evaluations: An Example Using a Preservice Teachers’ Instructional Media Course

This paper outlines limitations of True Score Theory, describes advantages of Rasch measurement, compares the results of a pre-post evaluation using both methods, and discusses the importance of including measures of self-efficacy for program evaluations. Forty students enrolled in a preservice teachers’ instructional media course participated in this study. A measure of self-efficacy for implementing various types of instructional media, the Classroom Technology Questionnaire, was given prior to—and following—instruction. A dependent t-test demonstrated statistically significant gains in the group means measuring self-efficacy (p < .01). In addition to using ordinal data, where interval measures are required, this type of analysis uses aggregate data, which lacks diagnostic power at the individual level. These limitations may be overcome using individually-based statistics with the interval measures that result from the proper use of Rasch models. Results of the Rasch analysis in this study demonstrated that 20 of the 40 participants had statistically significant gains in self-efficacy (p < .01). Using these individually-based analyses, researchers are able to target individuals without significant gains in order to offer further self-efficacy enhancing activities and to conduct followup interviews to determine why the program was seemingly unsuccessful for them. The use of Rasch models for establishing evidence of construct validity is also discussed.