Chapter 4: Residual Diagnostics and Model Assessment in a Multilevel Framework: Recommendations Toward Best Practice

Diagnostic approaches and assessment methods helpful in evaluating the quality of single-level regression models are widely used, but corresponding residual techniques are not as well developed or consistently applied to multilevel regression models. An informal review of recently published multilevel research articles in substantive education research journals revealed few instances of use or discussion of residual diagnostics for the models presented. This is problematic, since the quality of research conclusions rests with the quality of the models used to generate those conclusions, including the reasonableness of model assumptions. Our goal in this chapter is to contribute to researcher understanding of residual strategies for the multilevel case. We begin with a review of basic multilevel model assumptions, including the interpretation and estimation of residuals. Next, we summarize software-specific differences in available residual diagnostic methods. We demonstrate applications of diagnostic methods useful for improving the derived model and for identifying unusual cases within multilevel studies. We then propose a collection of recommendations for researchers who are using multilevel models to address their substantive research questions.