Lookahead Bias from Full-Sample Min-Max Normalization

Summary

This note warns that several strategies in a folder contain lookahead bias and presents them as exercises for identifying the problem. It points to normalization procedures that calculate minimum and maximum values using an entire dataset. When those statistics include future observations, values used in earlier trading decisions depend on information that would not yet have been available.

The examples name several strategies and identify full-data min-max scaling, including fitting a MinMaxScaler to a series, as the source of leakage. This is a useful backtesting lesson: preprocessing must be fitted only on information available at each decision point, such as a training window or rolling historical window. The document gives no strategy returns or empirical comparison, and its examples are brief; it explains the bias mechanism rather than quantifying its effect or providing corrected implementations.

Key ideas

• Calculating normalization bounds over a full dataset can introduce future information into historical signals.
• Min-max scaling is vulnerable when fitted using observations beyond the trading decision date.
• The note identifies this issue in several named strategy examples.
• Preprocessing in a backtest should use only data available at the time of each decision.
• No performance impact or corrected code is reported.

Tags

# readme


Warning, Strategies in this folder do have a lookahead bias.

Please see these as practice to see if you can spot the lookahead bias.


<details>
<summary>Expand for spoilers / solution</summary>

Please Click on each strategy to see details of the mistakes.

<details>
<summary>DevilStra</summary>

`normalize()` uses `.min()` and `.max()`. This uses the full dataframe, not just past data.

</details>

<details>
<summary>GodStraNew</summary>

`normalize()` uses `.min()` and `.max()`. This uses the full dataframe, not just past data.
</details>
<details>
<summary>Zeus</summary>

uses `.min()` and `.max()` to normalize `trend_ichimoku_base` as well as `trend_kst_diff`.

</details>

<details>
<summary>wtc</summary>

``` python
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
```

Using a MinMaxScaler will automatically take the absolute maximum and minimum of a series.

</details>
</details>

</details>

Shown in full with attribution under the source's licence. Licence: GPL-3.0

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.