Decision Trees: Training, Pruning, and Feature Importance
Summary
This overview explains decision trees as nonparametric models that divide observations through sequential feature rules. Each path defines a group, and a leaf assigns a prediction based on the labels in that group. The same feature can be used at multiple points in a tree.
Training repeatedly chooses a feature and split threshold intended to improve model performance, then stops when further splits do not help or a size limit is reached. The document describes controlling overfitting by limiting depth, requiring minimum sample counts for splits and leaves, or capping the number of leaves. It also outlines assessing feature importance by the deterioration in Gini-based performance or classification error when a feature is removed and the tree is retrained. These are conceptual explanations rather than a trading application or empirical evaluation; the text does not specify split criteria, validation procedures, or implementation details.
Key ideas
- A decision tree partitions observations through a sequence of feature-based rules.
- Leaf predictions can reflect the majority label among observations that follow the same path.
- Training selects features and thresholds that improve model performance and stops when growth limits are reached.
- Pruning controls tree complexity through depth, sample-count, and leaf-count limits.
- Feature importance can be assessed by how much performance worsens when a feature is removed and the tree is retrained.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.