Chapter 12: Robustness and Power of Ordinal d for Paired Data

Research questions concerning whether scores from one group or on one occasion tend to be higher than those from the other are usually answered by comparing mean scores of the two groups/occasions. For example, paired t test is the most frequently used test for comparing means of two matched groups, and for pre- and posttest comparisons. Parametric tests, such as paired t test, assume that variables are measured at least at the interval level, as well as assuming normality of the test variable and homogeneity of variance. However, most behavioral and social variables have only ordinal justification, and the parametric assumptions are always violated to a certain degree. It has been shown that the bulk of psychological and educational data are at least moderately and sometimes strikingly nonnormal (e.g., highly skewed, polymodal, or heavy-tailed) (O’Brien, 1988; Micceri, 1989; Wilcox, 1991). Nonnormality and heterogeneity of variance are known to inflate the actual Type I error rate and severely reduce the power of normal-based mean comparison procedures (e.g., Pearson & Please, 1975; Tan, 1982; Wilcox, 1991, 1992; Zumbo & Jennings, 2002).