Skip to content
All library documents

Handling Erroneous Prices in Historical Equity Data

Article Quant Q&A · Author: Gregor Best
Read the original

Summary

The document describes a daily stock-screening workflow that downloads OHLC prices, calculates moving-average signals, and alerts the user. Its practical issue is that a few records from a convenient public data source contain prices far outside the stock’s usual range. Such outliers may have limited impact on averaged signals, yet make it difficult to judge whether a strategy is sensible for that security.

The response characterizes data quality as a known concern with the source and suggests considering specialist historical-data vendors, with different providers named for global and US equities. It does not explain the cause of the anomalous observations, show validation methods, compare providers, or establish that any vendor is error-free. The takeaway is to assess source reliability as part of strategy research; the document provides a recommendation rather than a detailed data-cleaning procedure.

Key ideas

  • A few extreme price records can distort evaluation of a trading signal even when averages soften their effect.
  • Historical data quality should be checked before drawing conclusions about a strategy.
  • The response recommends considering specialist data vendors for research-ready historical prices.
  • No cause for the anomalous records or systematic validation procedure is provided.

Tags

EquitiesTechnical indicatorsBacktestingUS markets
Full text
# How to deal with unreliable data sources?


# How to deal with unreliable data sources?












I'm building a little system for myself that helps me make trade decisions, essentially downloading daily OHLC prices for a few hundred stocks once a day, running various sizes of crossing averages on them and sending me a push notification when something interesting pops up.

Right now, I'm using stock data from Yahoo finance, mostly because their API is really easy to use (no need to parse HTML tables and such).

This seems to work quite nicely, except for one thing:

For a few stocks, the price data contains really weird entries. Take ISIN IE00B8FHGS14 for example. On most days, the prices returned by Yahoo's API are reasonable:

```
Apr 21, 2020    47.54       47.54       46.35       46.64       46.64
```

But on some days, the prices are wildly out of any reasonable range for that stock:

```
Feb 17, 2020    4,204.00    4,219.00    4,198.36    4,219.00    4,219.00
```

While this doesn't throw off my trade signal calculations too much (they're based on averaged prices after all), it makes actually evaluating stocks (i.e. "Does it make sense to trade this stock using crossing averages as the signal?") essentially impossible.

Is there some reason the prices are sometimes way out of range? How do you guys deal with unreliable data? Am I completely on the wrong track crawling Yahoo and should I use something else?

## Answer by Mike Williams (score 1)

https://quant.stackexchange.com/a/53596

Data quality and reliability have been notoriously an issue for most people using Yahoo Finance. I suggest looking into other market data vendors like TickDatafor global stocks or AlgoSeek for US stocks. Both are known for high-quality and research-ready historical market data.

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.