Winsorization and Truncation for Financial Time-Series Outliers
Summary
The document explains winsorization as replacing observations in the upper or lower tails of a variable with the corresponding percentile cutoff. It contrasts truncation, which marks tail observations as missing, and frames both as possible treatments for influential extremes in financial data. The discussion cautions that outliers can reflect either errors or genuine market events, so their source should be investigated before changing the data.
The choice between the methods depends on what the extreme values are believed to represent: truncation may suit observations thought to come from a different distribution, while winsorization can be more appropriate when values are plausible but overly extreme. The cited empirical asset-pricing practice often favors winsorization, but sensitivity of results to the choice is a reason for skepticism. The original question proposes expanding rolling lookbacks for a time series, but the answer does not validate that specific rolling design or prescribe a universal threshold; it emphasizes careful data review and robustness analysis.
Key ideas
- Winsorization replaces tail observations with percentile cutoff values, while truncation removes them from analysis.
- Financial extremes may be valid events or data errors, so inspect them before treatment.
- Truncation may fit cases where extremes appear to come from another distribution.
- Winsorization may fit plausible observations whose magnitudes are considered unreliable.
- Results that change materially between treatments should be viewed cautiously.
Tags
Full text
# Rolling Winsorization for Time-Series # Rolling Winsorization for Time-Series I'm running a multivariate time series analysis and need to deal with some outliers. I'm thinking about using a rolling winsorization (e.g., pull anything above 99.5 percentile and replace with the 99.5 percentile value). The look back period would start at 2-years and continue to expand as I moved forward in the time series. Is this an appropriate way to deal with outliers? I have a few six sigma moves in a dataset of >3,000. ## Answer by skoestlmeier (score 2) https://quant.stackexchange.com/a/43511 #### Winsorization is fine, but carefully analyze your data and think about the nature of your outliers! I assume you are aware of this answer here: > [...] So you have to be extremely careful and double check all the data points you remove by whatever available method out there! [...] Financial data are notoriously subject to outliers. In many statistical analyses, such data points may exert an undue influence on the results, making the results unreliable. Commonly, empirical asset pricing researchers usually take an ad hoc approach to dealing with outliers (instead of several statistical methods that are designed to assess the effect of outliers). Winsorization is performed by setting the values of a variable $X_n$ with $n$ observations, that are in the top $h$ percent of all values of $X$, to the $(100-h)$th percentile of $X$. Similarly, values of $X$ in the bottom $l$ percent of $X$ values are set to the $l$-th percentile of $X$. Truncation is very similar to winsorization, except instead of setting the values of $X$ above $Pctl_h(X)$ to $Pctl_h(X)$, we set them to missing or unavailable. Similarly, values of $X$ that are less than $Pctl_l(X)$ are taken to be missing. Bali/Engle/Murray (2016) state (p. 6): > When to use winsorization or truncation is a difficult question to answer because some outliers are legitimate while others are data errors. In a statistical sense, one might argue that truncation should be used when the data points to be truncated are believed to be generated by a different distribution than the data points that are not to be truncated. Winsorization is perhaps preferable when the extreme data points are believed to indicate that the true values of the given variable for the entities whose values are to be winsorized are very high or very low, but perhaps not quite as extreme as is indicated by the calculated values. Most empirical asset pricing researchers choose to use winsorization instead of truncation. However, if the results of an analysis are substantially impacted by this choice, they should be viewed with skepticism.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.