Bagging and Boosting Decision Trees for Trading Forecasts
Summary
The article explains two ways to combine decision trees to address weaknesses such as noise, underfitting, and overfitting. Bagging trains trees independently on bootstrap samples drawn with replacement, then averages regression outputs or uses the majority classification. It is presented as a way to reduce forecast variance, especially for complex trees whose predictions vary substantially. Boosting instead builds learners sequentially, giving greater weight or selection priority to examples earlier models classified incorrectly, with the aim of improving weak learners and reducing bias. The article introduces AdaBoost, gradient boosting, and XGBoost, describing their basic distinctions.
For a trading application, it outlines a general workflow of cleaning instrument data, selecting predictors and a target, splitting training and test data, fitting a model, and evaluating it. It offers no trading dataset, measured performance, or comparison showing that either ensemble is profitable or universally better. The descriptions are introductory, and model choice depends on the problem and validation; the article also notes that bagging may not help when tree variance is not high.
Key ideas
- Bagging trains trees independently on bootstrap samples and combines their predictions to reduce variance.
- Boosting trains learners sequentially, emphasizing observations that earlier learners handled poorly.
- AdaBoost focuses on misclassified examples, while gradient boosting seeks to reduce a chosen loss function.
- A trading model workflow includes data preparation, predictor and target definition, a train-test split, fitting, and evaluation.
- Ensemble methods do not guarantee profitable forecasts, and bagging may not help when tree variance is low.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.