Least Median Squares Regression: Robustness, Complexity, and L1 Alternatives

Summary

The document considers whether Least Median Squares (LMS) regression is useful for financial estimation, such as estimating the beta between two securities. LMS minimizes the median of squared residuals rather than their mean, with the intended benefit of reducing the influence of outliers. The question raises this as a possible reason to prefer LMS over ordinary least squares, but provides no empirical comparison or finance application results.

The answer suggests two practical reasons LMS may see limited use: the algorithm the respondent found has cubic computational complexity, and L1 regression offers a comparatively accessible robust alternative that can be solved with linear programming. The answer conjectures that for typical financial outliers, L1 and LMS may produce similar estimates. These are claims and expectations in a short discussion, not a broad benchmark; the document does not assess particular algorithms, data sizes, or outlier patterns in detail.

Key ideas

• LMS minimizes the median of squared residuals to limit the effect of outliers.
• The discussion frames beta estimation as one possible financial application of robust regression.
• The cited LMS algorithm is described as having cubic computational complexity.
• L1 regression is presented as a faster, practical robust alternative solvable with linear programming.
• The claim that L1 and LMS estimates would be similar for financial outliers is a conjecture, not demonstrated evidence.

Tags

# Is Least Median Squares (LMS) regression commonly used in Finance?


# Is Least Median Squares (LMS) regression commonly used in Finance?












Least Median Squares is often argued to give more stable results than does OLS. Whereas in OLS one minimises the mean of squared residuals, in LMS, one instead minimises the median of squared residuals. Intuitively, should give estimators that are largely (completely?) invariant to outliers. As such, i would have thought this approach would find a natural home in finance applications, however, I haven't come across it being used (a search on this site, for example gives zero citations).

Does any one recommend or discourage a LMS approach - say, for a simple case of estimating a beta coefficient between two securities?

## Answer by Marc Shivers (score 3, accepted)

https://quant.stackexchange.com/a/33620

Interesting idea. I'm guessing this isn't used for two reasons:

First, the only algorithm I could find is $O(n^3)$, which is horrible if you're using a moderately-sized high-frequency dataset. Least squares is $O(nk^2)$ (n is the number of rows, and k is the number of predictors; typically $k<<n$).

More relevantly, L1 regression is almost as outlier-insensitive as LMS, and that can be solved easily and quickly with any LP-solver. For the kinds of outliers we see in finance, I doubt there would be a material difference in the L1 and LMS solutions.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.