Tuning parameters for XGBoost
| Tuning parameter | Description | Default value | Optimal value |
|---|---|---|---|
| Number of trees B | B is also known as the number of estimators. Unlike random forests, XGBoost can overfit if B is too large | A relatively small number of trees (e.g. 100 trees) | 100 |
| Learning rate λ | λ is a small positive number that controls the rate at which boosting learns. Unlike fitting a single large decision tree to the data, the boosting approach instead learns slowly | Typical values are 0.01 or 0.001 | 0.1 |
| Max depth | The max depth is the maximum number of nodes allowed from the root to the farthest leaf of a tree. Deeper trees can model more complex relationships by adding more nodes, but sometimes end up following noise, causing the model to overfit | The default number of the max depth is 6 | 3 |
| Min child weight | The min child weight is the minimum weight (or number of samples if all samples have a weight of 1) required in order to create a new node in the tree. A smaller min child weight allows the algorithm to create children that correspond to fewer samples, thus allowing for more complex trees, but again, more likely to overfit | The default number of the min child weight is 1 | 7 |
| Tuning parameter | Description | Default value | Optimal value |
|---|---|---|---|
| Number of trees B | B is also known as the number of estimators. Unlike random forests, XGBoost can overfit if B is too large | A relatively small number of trees (e.g. 100 trees) | 100 |
| Learning rate | λ is a small positive number that controls the rate at which boosting learns. Unlike fitting a single large decision tree to the data, the boosting approach instead learns slowly | Typical values are 0.01 or 0.001 | 0.1 |
| Max depth | The max depth is the maximum number of nodes allowed from the root to the farthest leaf of a tree. Deeper trees can model more complex relationships by adding more nodes, but sometimes end up following noise, causing the model to overfit | The default number of the max depth is 6 | 3 |
| Min child weight | The min child weight is the minimum weight (or number of samples if all samples have a weight of 1) required in order to create a new node in the tree. A smaller min child weight allows the algorithm to create children that correspond to fewer samples, thus allowing for more complex trees, but again, more likely to overfit | The default number of the min child weight is 1 | 7 |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.