Chapter 11: Controlling Experiment-Wise Type I Errors: Good Advice for Simultaneous and Sequential Hypothesis Testing

The flurry of research on controlling Type I errors in multiple testing situations in the latter half of the twentieth century produced a plethora of choices, many of which are quite powerful, as solutions to this real data analysis problem. It is surprising, therefore, that many authors (e.g., M. C. W. Braver & Braver, 1988; Braver & W. Braver, 1990a, 1990b, 1995), restrict the concern for this problem to tests conducted in parallel, thereby exempting or ignoring sequential procedures.

Two vignettes that are often excluded are (1) controlling experimentwise Type I errors when conducting preliminary tests of underlying assumptions prior to a test of experimental outcomes, and (2) buildingblock tests, such as all interactions and main effects in a three-factor ANOVA. Therefore, the purpose of this chapter is to demonstrate the necessity of guarding against false-positives in these two situations, in addition to the typical simultaneous hypothesis testing application.