Chapter 6: The Hazards of Subgroup Analysis in Randomized Business Experiments and How to Avoid Them

Experiments are ever more widely used in business. Frequently the experimental units are represented in databases with available covariates, raising the opportunity for subgroup analysis. There are two types of subgroups analyses: hypothesis generation and hypothesis testing. The former is very easy to do, but the latter is difficult. We offer an example of subgroup analysis that shows the pitfalls, and gives rules for avoiding the pitfalls. We also provide a dataset that can be used for classroom exercises.