← All research spotlights

Peter Bühlmann

Photo: Kaiser-Heinzmann Susanne, CC BY-SA 4.0, Wikimedia Commons

Research spotlight · Unit 15

Peter Bühlmann

ETH Zurich

Research field
High-dimensional statistics, boosting, causal inference

Peter Bühlmann and Torsten Hothorn presented boosting to statisticians as a way of fitting a statistical model by functional gradient descent, with any base learner in place of the tree. Their central observation is that the number of boosting steps acts as a regularization parameter, like the penalty of ridge regression and the lasso: each step adds a little to the fit, so stopping early shrinks the model, and the stopping point can be chosen by an information criterion or by cross-validation. With one predictor per step, boosting becomes a variable selection method for high-dimensional data. The unit uses this view in its comparison section, where the number of trees of the boosted model is chosen by cross-validation.

In the masterclass

Section 5, after the cross-validated comparison of a tree, a random forest, and a boosted model on the heart disease data.

The research is presented in Unit 15: Trees, Random Forests, and Boosting, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

Bühlmann, P., Hothorn, T. (2007). Boosting Algorithms: Regularization, Prediction and Model Fitting. Statistical Science, 22(4), 477‑505. https://doi.org/10.1214/07-STS242

The scientists named here are not contributors to the Statistics Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.