Photo: Kaiser-Heinzmann Susanne, CC BY-SA 4.0, Wikimedia Commons
Peter Bühlmann
ETH Zurich
- Research field
- High-dimensional statistics, boosting, causal inference
Peter Bühlmann and Torsten Hothorn presented boosting to statisticians as a way of fitting a statistical model by functional gradient descent, with any base learner in place of the tree. Their central observation is that the number of boosting steps acts as a regularization parameter, like the penalty of ridge regression and the lasso: each step adds a little to the fit, so stopping early shrinks the model, and the stopping point can be chosen by an information criterion or by cross-validation. With one predictor per step, boosting becomes a variable selection method for high-dimensional data. The unit uses this view in its comparison section, where the number of trees of the boosted model is chosen by cross-validation.
In the masterclass
Section 5, after the cross-validated comparison of a tree, a random forest, and a boosted model on the heart disease data.
The research is presented in Unit 15: Trees, Random Forests, and Boosting, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.