Chapter 10: Differences in the Amount of Adaptation Exhibited by Various Computerized Adaptive Testing Designs

Computerized adaptive testing (CAT)¹ is a widely used approach to customize the items administered to each examinee during a test. The item selection is made to optimize a criterion such as maximizing the amount of Fisher information at the most recent estimate of proficiency or maximizing decision accuracy for placement above or below a cut score (e.g., see Thompson, 2009). CAT designs have been successfully used in a variety of settings from military testing (ASVAB; Pommerich, Segall, & Moreno, 2009), placement in 2-year college courses (COMPASS; ACT, 2006), licensure and certification (NCLEX; NCSBN, 2016), and so forth. In recent years, there has been interest in using CAT designs for evaluating student achievement within the context of state testing programs (e.g., Minnesota; Minnesota Department of Education, 2017). Like any other approach to test design, there is not a single approach to CAT. There are many alternative designs using different options related to item selection, proficiency estimation, exposure control, item pool composition, content constraints, item types, and so forth. Our own previous work (Reckase, Ju, & Kim, 2018, 2019), plus observations of operational CATs, have shown that all designs are not equally adaptive. That is, two test designs for the same purpose may both be labeled as a CAT, but the amount of customization of the test for each examinee for the two designs might not be the same. The purposes of this chapter are to suggest some descriptive measures of adaptation, summarize previous research on amount of adaptation for some CAT designs, and show the differences in the amount of adaptation that result from the use of some common CAT designs.