Adaptive Mutual Information for Stepwise Feature Selection
Summary
This article explains a stepwise feature-selection method based on mutual information, designed to identify predictors that relate to a target while avoiding redundant features. For continuous data, it contrasts density estimation with Parzen windows, whose results depend on bandwidth choice, and adaptive partitioning, which recursively divides data regions where dependence appears meaningful. Chi-square tests guide subdivision, with additional checks and stopping rules intended to balance missed structure against overfitting sparse regions.
The feature-selection procedure applies maximum relevance and minimum redundancy criteria, adding candidate variables iteratively and using Monte Carlo permutation tests to assess significance. The article describes demonstrations on synthetic and more realistic data, but the supplied text does not include the datasets’ detailed results or quantitative comparisons. The method’s estimates depend on partitioning thresholds and adequate sample sizes; overly coarse partitions can miss relationships, while overly fine ones can capture noise. It is a general modeling technique rather than a trading strategy, and selected features still require validation in the intended market and model.
Key ideas
- Mutual information can detect nonlinear dependence between continuous variables and a target.
- Adaptive partitioning uses chi-square tests to focus subdivisions on regions with evidence of dependence.
- Stepwise selection balances target relevance against redundancy with features already chosen.
- Monte Carlo permutation tests are used to assess the significance of candidate features.
- Partition thresholds and sample size affect whether the method misses structure or overfits noise.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.