Chapter 3: Evaluation of Model Fit and Adequacy

How do researchers evaluate multilevel models (MLMs)? How should they choose among competing models? The utility of any model depends upon its ability to explain the phenomenon under investigation. Therefore, model evaluation should consider two aspects of the model: (a) model fit, or the use of model selection criteria to choose among competing models; and (b) the adequacy/explanatory power of the model, or the ability of the predictors to explain variability in the outcome variable. As such, model selection processes should assess both model fit and predictive utility.

In this chapter, we describe and demonstrate the model evaluation process, considering both model fit and model adequacy. After providing a brief conceptual overview of MLM estimation, we identify common measures of model fit and adequacy within the MLM literature; highlight several areas of controversy or confusion; and provide general recommendations for evaluating MLM fit and adequacy. We review the concept of deviance and explain how to use the chi-square difference test to compare the deviances of two nested models. We also describe index comparison approaches (e.g., the Akaike information criterion [AIC] and Bayesian information criterion [BIC]) to model evaluation. Then, we consider model adequacy and predictive utility, explaining use of proportion of variance explained-type measures to determine the predictive power of MLMs. Using the example data from Chapter 2 in this volume, “Introduction to Multilevel Models for Organizational Data,” we compare the model fit and predictive utility of two competing models. Finally, we provide guidance for assessing model fit and model adequacy in MLM.