Chapter 7: Validity Issues For Technology-Enhanced Innovative Assessments

There can be little question that technology has opened a great many possibilities for assessment, some of which have already become realities and others which are still in the early stages of development. Technologies such as adaptive item selection and adaptive stopping rules are well known and used in many assessment contexts (Wainer, 1990). Development of automated scoring approaches for essays and other text-based constructed-response formats has been ongoing for nearly half a century; these approaches are currently in use in admissions and placement tests and are being considered by multistate consortia for widespread use in K-12 assessment (Shermis & Hamner, 2012). Interactive computer simulations supported by automated scoring algorithms have been used for more than a decade for credentialing purposes (e.g., Bejar & Braun, 1999; Dillon & Clauser, 2009), and these technologies are now making important inroads into K-12 assessment. In addition, computer technology makes it possible to analyze test data using cognitive diagnostic models and other theorybased approaches that would have been beyond the scope of operational computing power. These advances are exciting and herald a new era in assessment. New technologies typically come with promises of better things to come: sometimes those promises are kept, and sometimes they are not. The atomic age, ushered in after the Second World War, promised safe, clean electricity too cheap to meter. Our electric bills and the Fukushima disaster remind us that the promise of technology and what is delivered may not always be identical.