Chapter 7: The Roots of Score Inflation: An Examination of Opportunities in Two States’ Tests

Since the 1970s, policymakers have relied on test-based accountability (TBA) as a primary tool for improving student achievement and reducing racial and socioeconomic achievement gaps. These policies have produced striking gains in scores on some accountability tests and, in some cases, seeming evidence of narrowing achievement gaps. As a result, support for test-based accountability has been widespread. Most policymakers are confident that score gains signify commensurate increases in achievement and will translate into improvements in children’s long-term life chances, particularly for poor and minority children.

However, more than 2 decades of research indicate that TBA policies have not been the unqualified success that gains on high-stakes tests might suggest and have led to a variety of undesirable side effects. A recent National Research Council review concluded that the effects of TBA programs have ranged from zero to small (National Research Council, 2011). In addition, studies have identified distortions of practice, such as reducing instructional time allocated to material not emphasized on state tests (e.g., Koretz, Barron, Mitchell, & Stecher, 1996; Koretz, Mitchell, Barron, & Keith, 1996a; Stecher, 2002; Stecher, Chun, Barron, & Ross, 2000) in order to focus instruction on material that is predictably emphasized on the test. These practices can lead to score inflation—that is, gains in scores on the tests used for accountability that are markedly larger than the actual gains in student learning they are intended to signal (Koretz & Hamilton, 2006). Numerous studies have found that score gains on high-stakes tests often do not generalize to lower stakes assessments used as audit tests, such as the National Assessment of Educational Progress (NAEP) (e.g., Center on Education Policy, 2008; Fuller, Gesicki, Kang, & Wright, 2006; Jacob, 2007; Klein, Hamilton, McCaffrey, & Stecher, 2000; Koretz & Barron, 1998; Koretz, Linn, Dunbar, & Shepard, 1991; Lee, 2007). Although score inflation is highly variable across states (Ho, 2007), it is often severe. Some studies also have found that TBA programs can generate illusory improvements in equity (Klein, Hamilton, McCaffrey, & Stecher, 2000).