← All research spotlights

Research spotlight · Unit 15

Jerome Friedman

Stanford University

Research field
Statistical learning, data mining, tree-based methods

Jerome Friedman, one of the four authors of the CART book of 1984, gave boosting its general form in 2001. He regarded the fitting of a model as a numerical optimization in which the unknowns are the values of the function itself, so that any loss has a gradient and a step along the negative gradient lowers the loss; a regression tree fitted to that gradient at the training cases, added to the model in a shrunken multiple, is one step of gradient boosting. The paper works this out for squared error, absolute error, outlier-resistant losses, and classification, and introduces the partial dependence plot. The unit follows the paper in its boosting section and uses its method on the heart disease data.

In the masterclass

Section 4, after the derivation that shows the residuals to be the negative gradient of the squared loss and the history of AdaBoost.

The research is presented in Unit 15: Trees, Random Forests, and Boosting, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.

The paper

Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189‑1232. https://doi.org/10.1214/aos/1013203451

The scientists named here are not contributors to the Statistics Masterclass, are not affiliated with it, and have not endorsed it. The program presents and explains their published research, with references to the original papers.