Skip to content
All library documents

Finding Nonfinancial Time Series for Backtesting

Article Quant Q&A · Author: Dave
Read the original

Summary

The document addresses how to find long historical series of nonfinancial measurements for testing correlations with a financial analysis or trading algorithm. It suggests looking at unusual public datasets and gives examples of possible sources, including weather observations, economic and energy data, drought records, NASA catalogues, and a machine-learning repository.

One reply points to daily weather data extending back to the 1940s, while cautioning that the source serves data one year at a time, which means assembling a longer history requires multiple downloads. The thread is a collection of suggestions rather than a vetted dataset guide: it does not document data quality, licensing, exact coverage for each source, or whether the series are suitable as predictive features. A user should check availability, definitions, missing observations, and publication timing before using any candidate series in a backtest, since these details can affect correlation estimates and introduce look-ahead bias.

Key ideas

  • Nonfinancial historical series can serve as comparison data when testing financial algorithms.
  • Potential sources include weather, energy, drought, economic, and scientific data collections.
  • The suggested weather source offers daily observations but requires separate downloads for different years.
  • The thread does not verify the quality, coverage, or licensing of the listed datasets.
  • Check observation timing and missing data before using a series in backtesting.

Tags

BacktestingStatisticsMachine learning
Full text
# Seeking Historical Non-Finance Datapoints for Backtesting


# Seeking Historical Non-Finance Datapoints for Backtesting












I'm working on some financial analysis code which I'd like to test against a historical dataseries to analyze the correlations to my algorithm to some non-finance related data. Ideally, I'd like to test against a dataseries which provides a monthly float value as far back as 1997. Maybe it's the temperature somewhere on the 1st of the month. Maybe its the number of traffic accidents on the 1st day of each month in a city. Basically, just something somewhat random, but with the monthly value to test against.

Can anybody think of a publicly-available data-series which might have a monthly point of data as far back as 1997. A link to the data-source would be even better.

## Answer by jayers (score 8)

https://quant.stackexchange.com/a/694

The website www.infochimps.com has a lot of unusual datasets, many of which are free.

## Answer by joshayers (score 7)

https://quant.stackexchange.com/a/718

Here's a link to daily weather data. It looks like it goes as far back as the 1940s. There's a link to a CSV file at the bottom of the page. It will only give you one year's worth of data at a time, so you'll have to manually download several files.

http://www.wunderground.com/history/airport/KNYC/2011/3/13/CustomHistory.html

## Answer by bill_080 (score 6)

https://quant.stackexchange.com/a/693

Here are a few links:

http://www.economagic.com/

http://www.eia.doe.gov/electricity/data.cfm

http://www.drought.unl.edu/dm/source.html

http://gcmd.nasa.gov/KeywordSearch/Home.do?Portal=GCMD&MetadataType=0

## Answer by Ralph Winters (score 4)

https://quant.stackexchange.com/a/761

Here is another link to some non-financial data sets from the University of California Machine Learning Repository:

http://www.ics.uci.edu/~mlearn/MLRepository.html

Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.