Effect displays

Article Type:

Tutorial

From:

Journal of Modelling in Management, Volume 8, Issue 3

Keywords: Effect displays

Effect displays are tables or graphs which represent terms in a statistical model and are designed to make the task of interpreting them much simpler. They provide predictions for the response variable over the range of each explanatory variable in the model, whilst holding other variables at typical values. A particular feature of these displays is that they are easy to produce and manipulate and circumvent many of the problems analysts typically have with interpreting statistical models. Effect displays can be used to effectively interpret models and can also help in the process of model-selection, model-checking and with model presentation.

Effect displays are sophisticated analytical tools that can be applied to many different models and are particular useful for displaying interaction terms (Fox, 1987, 2003; Fox and Hong, 2009). The focus of this tutorial is, however, to introduce them to a wider audience using basic models and demonstrate how they can be used and manipulated simply using readily available software.

This tutorial aims to highlight the use of effect plots as a basic analytical tool and also as an effective means of communicating results. The following analyses will demonstrate how effect displays can be produced for a generalized linear logit model, and how these displays can be used to effectively describe the statistical model. We also look at ways in which the displays can be manipulated to provide detailed illustrations of the model.

A major aim of this tutorial is to encourage analysts who are not familiar with effect displays to try them out and investigate how they may be used in model-building as well as model communication. The simple, intuitive way that these displays illustrate models also makes them ideal tools for teaching; the main advantage being that models do not have to be interpreted using complex numerical outputs that often prove to be problematic for many audiences.

The software

This tutorial will demonstrate effect displays through the use of the effects package (Fox et al., 2013), which is written for the R statistical computing environment (R Core Team, 2013). In order to illustrate these displays as simply as possible, the R-commander interface (Rcmdr, Fox, 2013) is used where possible as it allows some of the functions of the effects package to be accessed automatically using pull-down menus. In order to reproduce the analyses shown in this tutorial, R and the Rcmdr must be loaded, along with the effects library (a tutorial on installing R and the Rcmdr is available from the Journal of Modelling in Management, 5:3, 2010, web site: http://www.emeraldinsight.comjournals.htm?issn=1746-5664). Simply load R, and then run the following commands:

> library(Rcmdr) load the Rcmdr>library(effects) load the effects library

An example dataset

The data used in this tutorial is the “Arrests” data set, which is provided as part of the effects package (Fox et al., 2013). These data are easily accessed once the effects library has been loaded into R (see above) either directly using the R console […]

> data(Arrests) load dataset “Arrests” > head(Arrests) display the first few lines (Table I).

[…] or by using the pull-down menu system in the Rcmdr (Figure 1).

Table I

Figure 1 Loading the “Arrests” dataset

Figure 2 The Arrests data set: a data frame with 5,226 observations on eight variables

A full description of the data can be obtained by requesting Help on selected data set in the Rcmdr, or by entering the command help(Arrests) directly into the R-console. For convenience, the description provided for these data are reproduced in Figure 2.

Recoding year to a factor

We can see from the help file for the Arrests data set in Figure 2, that year is coded as a numeric vector. In the following analyses, however, year is regarded as a categorical variable and needs to be recoded; for this demonstration, a new categorical variable is constructed and named yearCAT. This is easily achieved using the Data, Manage variables in active dataset, Convert numeric variables to factors […] menu options in the Rcmdr, using the original numbers as category identifiers (Figure 3), or by running the following command in the R-console:

> Arrests yearCAT <- as.factor(Arrests year)

Figure 3 Recoding the numeric variable “year” into a factor, identifying the factor levels using the existing numbers and saving the new variable as yearCAT

A logistic regression model of variable “released”

In this demonstration, the binary outcome variable “released” is modelled using the explanatory variables checks, colour, sex and yearCAT; a binary logit model that can be easily run directly from the Rcmdr using the menu commands shown in Figure 4, or directly from an R-console using the commands:

> GLM.1<- glm(released check s+ colour + sex + yearCAT, + family=binomial(logit), data=Arrests) > summary(GLM.1)

□

Coefficients are in Table II.

Table II

Interpreting the model using tabular output

The table of numbers shown in Figure 4 is typical of the output provided by a number of software packages for regression models. This output provides estimates of the relationships between the explanatory variables and the response and also provides estimates of how significant these relationships are. For example, Figure 4 shows that the sex of the individual is not a significant predictor of whether someone is released, whilst checks and colour are (as evidenced by the z-values and the stars in the final column). The output also suggests that some of the categories for year are significantly different to the reference category (the major difference being between the reference and year 2000). These results, as presented here are, however, not particularly easy to interpret, especially for those not used to dealing with raw statistical output or logit regression models. Some of the common difficulties with interpreting the output from this model are:

* It does not explicitly identify the response variable or the response reference category. Looking at the output, it is not immediately obvious what the model actually predicts, or whether predictions are being made about being “released” or are being made about “not being released”.

* The “Estimates” are provided in logits, which can be difficult to interpret. For example, for each unit increase in checks, the log-odds of the probability of being in one of the response categories decreases by 0.40367; a description that is not particularly easy to interpret.

* Although treatment contrasts have been used for the categorical explanatory variables (as evidenced by the “T” in the output), it is not immediately clear which category is being used as a reference for the variable yearCAT.

* The relationship between yearCAT and released is not immediately obvious across the range of years. It is also not obvious how significant yearCAT is overall, or which individual years are significantly different (for example, years 1999 and 2000).

* It does not show the differences in the standard errors of the individual groups (for example, there is no indication of the standard errors for males and females separately).

* There is no indication of whether checks has a non-linear component to it’s relationship with released.

Interpreting the model using effect displays

Many of the difficulties associated with interpreting the regression output shown in Figure 4 can be addressed by using effect displays. Effect displays for the model above provide predictions for the response variable across the range of values of each explanatory variable, whilst holding values of the other explanatory variables constant (usually the average value of the variable, or the average across all categories; the values used can, however, be easily defined by the analyst – see the section on “setting values of the X variables” below). Effect plots for all of the X-variables in the model above (model GLM.1) can be obtained using the Rcmdr pull-down menus (Figure 5) or obtained directly from the R-console using the command:

> plot(allEffects(GLM.1))

The effect displays provide much the same information as the tabular output, but are presented in a form that is more comprehensive and much easier to understand. First the predictions from the model are easier to interpret, as they are represented as probabilities, which are more intuitive than logits. For our model, the Y-axes show the probability of being released. The plots also indicate which variables and categories are significant. The effect plots for our model show that the variables checks and colour are significant as the predicted probability of being released change according to the value of the explanatory variables. For example, values of released are lower for the black category than they are for the white, and also change considerably as the number of checks change. Predicted probabilities of being released are also different for some of the categories of year (particularly 1997 and 2000). Gender is not significant, as the effect plot for sex shows that the predicted probability of being released for males and females overlap.

Figure 4 Running a logistic regression model of released from the Rcmdr

The effect plots provide a number of other pieces of information useful for understanding the model that are either not immediately apparent or are completely missing from the tabular results display in Figure 4:

* Information about the response variable is provided on the Y-axis and does not need to be interpreted with respect to a reference category.

* Estimates are provided as probabilities, which are easier to understand than the logits used in the tabular output. For example, the difference in the probability of release for black and white people is now quite obvious (0.79 for black compared to 0.87 for white).

* All categories of the explanatory variables are shown, making it easier to compare and contrast predictions for each category.

* Information about standard errors is provided. For example, the effects plot of “sex” shows that the standard errors for females are much larger than they are for males.

* The progression of “year” from 1997 to 2002, is much more obvious with comparisons between all years now possible.

* The variable scale of the Y-axis on the “checks effect plot” (the gap between 0.8 and 0.9 is much larger than that between 0.5 and 0.6) suggests that there may be a non-linear component to the relationship between released and the number of checks. Non-linearity can be investigated using an appropriate contrast coding scheme (see the tutorial on polynomial coding from the Journal of Modelling in Management, 2011, 6:2, web site: http://www.emeraldinsight.comjournals.htm?issn=1746-5664)), or more easily by transforming the Y-axis of the effect plot (see “changing the Y-axis” section below).

Editing and manipulating effect displays

Effect displays can be used to illustrate many of the model properties shown in standard regression outputs and the plot(allEffects(model)) command provides a simple method of producing these plots. Effect displays can, however, be employed much more widely than this as they are easily manipulated and edited to illustrate particular aspects of models in great detail and also help with model selection and exploration. The following sections describe just a few of the options that are available to manipulate effect displays to effectively illustrate models. A much more detailed and comprehensive description of available options can be viewed in the manual for the effects package which is available for download from the web site – http://cran.r-project.orgwebpackageseffectseffects.pdf

Rescaling the Y-axis

The effect displays in Figure 5 show estimates of the probability of being released plotted on the Y-axes which is indicated using a logit scale (the model is, after all, a linear logit model). Interpreting relationships using this scale can be confusing, particularly for those not accustomed to using logit models. Effect plots can, however, be easily transformed into a more conventional scale of probability. This is most simply carried out using the Rcmdr to change the plotting command. When the Rcmdr is used to draw the effect plots, the command it uses is shown in the Rcmdr Script window:

Figure 5 Effect plots for a logistic regression model of released (model GLM.1)

□

plot(allEffects(GLM.1), ask=FALSE)

In order to change the Y-axis to a conventional scale of probability, simply add rescale.axis=FALSE to the command:

□

plot(allEffects(GLM.1), ask=FALSE,

rescale.axis=FALSE)

[…] and then submit this using the Rcmdr “Submit” button. A screenshot of this procedure and the resulting plot is shown in Figure 6.

Figure 6 Rescaling the effects plots

Y

-axes to a conventional probability scale

The re-drawn effect displays provide a similar picture of the analysis, but are particularly useful in the checks effect plot, which now clearly demonstrates a potentially interesting non-linear component to the relationship between the number of checks and probability of being released. Although the form of this relationship can be determined from the original effect display (i.e. the plots provided using the allEffects default settings), the rescaled image is, perhaps, easier to interpret.

Defining the Y-axes

The default settings for the allEffects command optimise the Y-axes, which results in different ranges of probability for each plot. For example, in Figure 6 predictions for the checks effect plot range from 0.5 to 0.9, whereas the yearCAT effect plot only range from 0.76 to 0.88. The effect displays can be easily changed to represent each plot on the same scale by adding in a command to define the limit of the Y-axes; the command ylim=c(low,high). Using the Rcmdr, to define the Y-axes to range between 0.4 and 1, simply add ylim=c(0.4,1) to the allEffects command […]

□

plot(allEffects(GLM.1), ask=FALSE,

rescale.axis=FALSE,

ylim=c(0.4,1))

[…] and then “Submit” this command. A screenshot of this procedure and the resulting plot is shown in Figure 7. This analysis simply recreates the graphs in Figure 6 using different Y-axes to emphasise the relative effects of the explanatory variables on the response.

Figure 7 Effect displays with identical

Y

-axes

The ability to define the scale of the Y-axis is particularly useful as it enables easy comparisons to be made between graphics. Comparisons can be made not only within an individual set of plots (i.e. comparing different x-variables, as shown in Figure 7), but also for individual effect plots produced in successive calls to the allEffects command. For example, yearCAT effect plots can be computed using a range of values of the other explanatory variables in the model (see the section below on setting x-values) and then combined in multiple or overlay scatterplots (or in an animation) to obtain a picture of how the probability of being released each year changes as a function of colour or the number of checks.

Selecting individual effect plots

It is often useful to select individual plots from the allEffects function for the purposes of specific illustration or to enable the plots to be saved and edited, or used in an animated sequence. Individual plots can be identified by adding selection=c(x) to the plotting command, where the value of x indicates the effect plot of interest. This is most simply achieved by editing the allEffects command in the Rcmdr script window. Below shows the commands required to obtain the effect plots for the variables colour and checks.

□

plot(allEffects(GLM.1), ask=FALSE,

selection=c(2)) colour effect plot

plot(allEffects(GLM.1), ask=FALSE,

selection=c(1)) checks effect plot

Setting values of the explanatory variables

The effect plots predict the response variable while holding values of the explanatories constant. For example, the effect plot for the variable colour in the plots shown previously shows the probabilities of being released for black and white people who have an average number of checks (the mean value of the variable checks) and a mixture of sex and yearCAT proportional to their distribution in the data (the data is 91.52 percent male, which means that the predictions are for a sample which is 91.52 percent male). Whilst the default settings are usually adequate, it is sometimes useful to over-ride these settings to make specific predictions (e.g. what is the predicted probability of release for males and females in 2002?). This is easy to achieve using the typical=x command to set values for continuous covariates (such as the mean and median) and the given.values=c() command to set values for categorical variables.

Important Note:

Defining values of the explanatory variables within the Rcmdr script window can result in errors. It is advised, therefore, to run the following examples in a newly opened R-console.

To illustrate the given.values function, predictions from the GLM.1 model are restricted to specific years. For example, to assign the year 2000, the category yearCAT2000 (this is the variable name “yearCAT” followed immediately by the category label “2000”) is given the proportion 1 (this sets the proportion of all other years to 0). It should be noted that when defining given.values, the reference category from the model cannot be used (it’s use results in an error). 1997, which is the reference category for the yearCAT variable in the GLM.1 model (the “missing” year from the output in Figure 4), is therefore defined as 0 for all other categories, which is technically the same things as the code yearCAT1997=1. Figure 8 shows the R commands and the resulting effect plots for 1997 and 2000 for the GLM.1 model.

Figure 8 Effect displays for years 1997 and 2000

Figure 9 Re-coding checks as a categorical variable

The following is the code used to produce the two colour effect plots for the years 1997 and 2000. They are commented here so the code can be easily copied to an R terminal.

library(effects) load effects library data(Arrests) load Arrests data change year to categorical Arrests yearCAT <- as.factor(Arrests year) run the GLM.1 model GLM.1<- glm(released checks+colour+sex+yearCAT, family=binomial(logit), data=Arrests) produce the colour effect plot for 1997 plot(allEffects(GLM.1, given.values=c(yearCAT1998=0, yearCAT1999=0, yearCAT2000=0, yearCAT2001=0, yearCAT2002=0)), rescale.axis=FALSE, selection=c(2), ylim=c(0.65,0.95), main=“year 1997”) produce the colour effect plot for 2000 plot(allEffects(GLM.1, given.values=c(yearCAT2000=1)), rescale.axis=FALSE, selection=c(2), ylim=c(0.65,0.95), main=“year 2000”)

The effect plot in Figure 8 shows that the probability of being released for blacks is lower than that for whites in both years. In addition to this, the probability of being released is greater in 2000, where the difference between between black and white is less pronounced. These graphics just show two of the years available; it is, however, easy to produce effect plots for all years and combine these into one graphic, or use an animation to show the changes over the complete range of dates available in the data.

Conclusion

Effect displays provide a very simple and effective way to illustrate and investigate statistical models. They can be applied simply using readily available software and used as an integral part of the investigation and analysis process and are useful for inclusion in academic reports to illustrate complex models more effectively (Hutcheson et al., 2011).

A major advantage of effect displays is the ease with which they can be applied and manipulated. For example, further exploratory analysis of the Arrests data identifies potentially interesting information about the relationship between the number of checks and the probability of being released. The effect plot in Figure 9 clearly shows that this relationship is not “smooth” and may be better described considering checks as a categorical variable. This analysis is very easy to do using R and the Rcmdr, as the code provided in the top panel of Figure 9 shows (this analysis essentially just re-codes checks as categorical and then re-runs the model and plots the effect displays).

This tutorial has just provided a basic introduction to effect displays and how they may be used to illustrate models. They provide a versatile and valuable tool for analysing data, are very useful for teaching purposes and deserve to used and reported much more widely in the academic literature.

Graeme D. HutchesonUniversity of Manchester, Manchester, UK

References

Fox, J. (1987), “Effect displays for generalized linear models”, in Clogg, C.C. (Ed.), Sociological Methodology, Vol. 17, American Sociological Association, Washington, DC, pp. 347–361Fox, J. (2003), “Effect displays in R got generalized linear models”, , Vol. 8 No. 15Fox, J. (2013), “R package ‘Rcmdr’: a platform-independent basic-statistics GUI (graphical user interface) for R, based on the tcltk package (version 1.9-6)”, available at: Fox, J., Weisberg, S. and Hong, J. (2013), “R package ‘effects’: effect displays for linear, generalized linear, multinomial-logit, proportional-odds logit models and mixed-effects models”, available at: Hutcheson, G.D., Pampaka, M. and Williams, J. (2011), “Enrolment, achievement and retention on traditional and use of mathematics pre-university courses”, , Vol. 13 No. 2, pp. 147–168R Core Team (2013), , R Foundation for Statistical Computing, Vienna, available at:

Journal of Statistical Software

http://http:cran.r-project.orgwebpackagesRcmdrhttp://http:cran.r-project.orgwebpackageseffectsResearch in Mathematics Education

R: A Language and Environment for Statistical Computinghttp://www.R-project.org