Robust Sequential Methods for Alerting on Non-Stationary Risk Scores
Summary
The document considers how to set alerts for a market-derived risk score that rises with structural instability, despite changing conditions, heavy-tailed noise, and potentially long calm periods. It outlines three statistical approaches: sequential Wilcoxon–Mann–Whitney procedures for detecting shifts in location, extreme value theory for modeling tail behavior, and robust sequential detectors built from score signs or ranks.
The proposed sign- or rank-based preprocessing can be paired with EWMA or CUSUM to retain sequential information while reducing sensitivity to extreme observations. The response presents these as alternatives to fixed thresholds and simple rolling averages, but gives no derivation, calibration procedure, empirical comparison, or citation details beyond naming the methods. In particular, it does not specify how to control false alarms under non-stationarity or how to validate an extreme-value model when the underlying tail distribution changes. The suggestions are therefore a starting point for method selection, not a complete thresholding recipe.
Key ideas
- Sequential Wilcoxon–Mann–Whitney procedures can detect location shifts without relying on Gaussian observations.
- Extreme value theory can help characterize rare, high-impact score observations.
- Applying EWMA or CUSUM to score signs or ranks can make sequential monitoring less sensitive to outliers.
- The document gives candidate approaches but no calibration details or empirical evidence about their false alarm rates.
Tags
Full text
# How to define robust alert thresholds for non-stationary risk scores? # How to define robust alert thresholds for non-stationary risk scores? Suppose we have a univariate risk score derived from market data that increases under structural instability. The score is not a volatility estimate, is not predictive, and is intended only for regime classification (e.g. stable / elevated risk). In a non-stationary environment, what are theoretically sound approaches to define alert thresholds that: minimize false positives tolerate long quiet periods remain robust under heavy-tailed noise I am not looking for ML-based calibration or rolling window heuristics, but for statistical or sequential testing perspectives. Any references or guidance would be appreciated. ## Answer by QuantCalc.net (score 1, accepted) https://quant.stackexchange.com/a/85302 Here are three theoretically sound approaches: - Wilcoxon-Mann-Whitney Statistic: This is a popular choice for detecting a shift in location (mean/median) for non-Gaussian data. A sequential procedure based on this is often referred to as a Robust Sequential Wilcoxon Change Point Procedure. - Non-Stationary Extreme Value Theory (EVT): Your score is non-predictive but increases under structural instability, suggesting that it measures the magnitude of rare, high-impact events. Since the environment is non-stationary and the noise is heavy-tailed, classic models will underestimate risk. EVT provides a robust framework for modeling the tail behavior. - Robust Exponentially Weighted Methods: While you dismissed rolling windows, Exponentially Weighted Moving Average (EWMA) is a sequential testing method, not just a heuristic, and it can be generalized to be robust. Use the time series of the sign or the rank of your score as the input for your sequential detector and then apply a standard EWMA or CUSUM chart to this robust input series. The combination of a robust pre-processor and a sequential memory statistic (EWMA) achieves all your criteria without resorting to fixed thresholds or simple rolling averages.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.