Chapter 4: Validity as Contextualized and Pragmatic Explanation, and its Implications for Validation Practice

This chapter has two aims: provide an overview of what I consider to be the concept of validity and then discuss its implications for the process of validation. I articulate an explanation focused view of validity that centers on a contextualized and pragmatic view of explanation—in essence, a contextualized and pragmatic view of validity. In the closing section of the chapter I describe the methodological implications of this view in terms of not assuming homogeneity of populations (from the Draper-Lindley-de Finetti framework) and allowing for multilevel construct validation, as well as the overlap between test validity and program evaluation.

We are as sailors who are forced to rebuild their ship on the open sea, without ever being able to start fresh from the bottom up. Wherever a beam is taken away, immediately a new one must take its place, and while this is done, the rest of the ship is used as support. In this way, the ship may be completely rebuilt like new with the help of the old beams and driftwood—but only through gradual rebuilding. Otto Neurath (1921, pp. 75–76)