Chapter 7: Automated Test Assembly: Case Studies in Classical Test Theory and Item Response Theory

Selecting items for an item pool to constitute a test form can be a relatively simple task. However, complex test specifications, item enemies, strict statistical requirements, and other constraints can make it challenging to manually assemble test forms. There could be multiple moving components to consider and making one change can impact other components.

One way to illustrate test constraints is as a series of layers. Let us say that we are selecting 20 items to build a math test form. The hypothetical specifications for the test form are as follows (see Figure 7.1 for illustration):

Each layer constitutes specifications that have items that are mutually exclusive. In the example, an algebra item cannot be a statistics item. On the other hand, an algebra item can be an MCQ item or an SPR item. In this case the constraints are not mutually exclusive.