Jerome Friedman
Stanford University
- Research field
- Statistical learning, data mining, tree-based methods
Jerome Friedman, one of the four authors of the CART book of 1984, gave boosting its general form in 2001. He regarded the fitting of a model as a numerical optimization in which the unknowns are the values of the function itself, so that any loss has a gradient and a step along the negative gradient lowers the loss; a regression tree fitted to that gradient at the training cases, added to the model in a shrunken multiple, is one step of gradient boosting. The paper works this out for squared error, absolute error, outlier-resistant losses, and classification, and introduces the partial dependence plot. The unit follows the paper in its boosting section and uses its method on the heart disease data.
In the masterclass
Section 4, after the derivation that shows the residuals to be the negative gradient of the squared loss and the history of AdaBoost.
The research is presented in Unit 15: Trees, Random Forests, and Boosting, in a research spotlight box that explains the result at the level of the unit and links it to what you have just learned.