Feature Selection Methods

Article BigQuant

Summary

The article explains why feature selection can reduce irrelevant or redundant inputs while preserving a model’s predictive ability. It illustrates the idea with a water park ticket-sales example, using weather, ice cream consumption, coffee consumption, and month as candidate features. The example suggests ice cream consumption may be more informative than temperature or coffee consumption, though it is only an illustration.

Three approaches are outlined: exhaustive search over feature subsets, random removal followed by model validation, and minimum redundancy maximum relevance (mRMR), which aims to favor relevant features while limiting overlap among them. Exhaustive search becomes impractical as the feature count grows, while random selection remains a trial-and-error process. The article presents mRMR as a way to address larger feature sets, but its description of the implementation is incomplete. It provides no detailed dataset, model specification, or comparative performance measurements, so the example does not establish that any method will work best for a particular trading or prediction task.

Key ideas

Feature selection can reduce input count and redundancy while retaining predictive information.
Exhaustive subset search evaluates every possible feature combination but scales poorly.
Random feature removal requires repeated model training and validation.
mRMR seeks features that are relevant to the target and less redundant with one another.
The water park example is illustrative and does not establish general predictive performance.

Three Feature Selection Methods for Predictive Models

Summary

Key ideas

Tags