Choosing Long-History ETFs and Index Proxies for Portfolio Backtests
Summary
The discussion considers how to select liquid, low-cost ETFs spanning broad asset classes and regions for testing a portfolio selection algorithm. It emphasizes choosing funds that closely track indexes with long histories, so older index data can extend a backtest beyond an ETF’s own inception. Examples include country and regional equity funds, bond funds, and individual metals, oil, and currency exposures. Historical benchmarks such as MSCI country indexes and national equity indexes are suggested as possible backfill sources.
The replies also flag important limits. Index histories and ETF histories differ, and currency data before the euro may require a predecessor currency series. Commodity ETFs can diverge from their underlying markets because futures-curve contango creates roll costs; therefore, a popular fund may be a poor proxy for spot commodity performance. The suggestions are illustrative rather than a complete asset universe, and the document does not provide a systematic liquidity, fee, or tracking-error comparison. Researchers should verify availability, index definitions, and survivorship and look-ahead effects before using proxies in a backtest.
Key ideas
- Select ETFs across broad markets and asset classes using liquidity, cost, and index-tracking history as criteria.
- Use long-running regional and national indexes to extend data before an ETF began trading.
- Bond, equity, currency, and individual commodity series can help broaden a test universe.
- Commodity futures ETFs may diverge from spot prices because contango can erode returns.
- Proxy histories require checks for benchmark comparability and backtest biases.
Tags
Full text
# Which ETFs should I use to test my portfolio selection algorithm? # Which ETFs should I use to test my portfolio selection algorithm? I have a portfolio selection algorithm I want to backtest, but I don't want to limit the inputs at any point in time. For example, I don't want to exclude the Japanese stock market, just because it hasn't done well in recent history. Ideally, I'll have a pool of ETFs that: 1. Cover broad market segments (i.e. "US Stocks," "Commodities," "Precious Metals," "Japanese stocks," "US Real Estate." etc.) 2. Are fairly liquid 3. Have low expense ratios 4. (Most important) Closely track an index that has a long history (This allows for a longer backtest). For example, SPY is perfect. It's large, liquid, covers a major segment of the market, it has a long history, and I can substitute the S&P500 index if I need to go back further than the 1990's. CEF might also be a good option, as it's large, liquid, covers gold and silver, and has a pretty long history (back to about 1986 it seems). Any other obvious choices ## Answer by Tal Fishman (score 3) https://quant.stackexchange.com/a/1815 MSCI has country indices for developed markets going back to 1970 in many cases and a decent history for emerging markets (starting 1988). iShares has pretty liquid ETFs for many of the most popular countries and regions, such as EAFE (EFA), Emerging Markets (EEM), Japan (EWJ), Germany (EWG), Canada (EWC), etc. Other major indices with very long histories include the German DAX, Japanese Nikkei 225, UK FT-30 (predecessor to FTSE), and Canada's TSX. These indices may also serve as substitutes for the corresponding MSCI indices to backfill even more data. The most popular bond index with a long history is the Barclays Aggregate (1976), which iShares also covers (AGG). Intermediate Treasuries (IEF) and US Corporate Investment Grade (LQD) go back even further (1973). I don't know much about other major asset classes, such as REITs and metals, as you've mentioned, but I believe they have very long histories as well. I'd track the main metals, such as gold and silver, directly rather than through a mixed commodity index. I'd also add crude oil to the mix, since that, too, has a pretty long history. The only "alternative" asset class you haven't mentioned is currencies. Each major currency has its own ETF (FXE, FXY, FXB, etc.), and the currencies themselves have been freely floating since the end of Bretton Woods in 1971. Here you would probably want to substitute DEM for EUR before 1999. I worked on a similar project myself recently, so I believe that is all. ## Answer by B Seven (score 2) https://quant.stackexchange.com/a/2012 I found this searchable/sortable list: http://www.etftrends.com/etf-analyzer.php . Premium membership is required to sort by inception date, $15/month. ## Answer by Valtinho (score 0) https://quant.stackexchange.com/a/24804 Be very careful when selecting commodity ETFs because many of the most popular and well-established ETFs, such as USO and UNG, suffer from a terrible contango problem, which make these ETFs poor trackers of the indexes they are supposed to represent. So, when the markets go down they lose, and when the markets go up they don't go up as much as they should. https://www.google.com/search?q=uso+vs+spot+crude+oil&source=lnms&tbm=isch&sa=X&ved=0ahUKEwiq1f3Tv7fLAhXBlB4KHURJBGkQ_AUICCgC&biw=1568&bih=760
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.