Backtesting Trading Strategies with Out-of-Sample and Point-in-Time Data
Summary
The document outlines a basic workflow for testing an equity strategy: encode it in a platform or record discretionary trades manually, optimize parameters on one portion of historical data, and evaluate the chosen approach on separate out-of-sample data. Walk-forward testing can help reveal whether performance is spread across history or depends on isolated periods. These steps are intended as a starting outline rather than a detailed implementation guide.
It emphasizes that credible results depend on data that reflects what was knowable when each decision was made. A historical universe should retain firms that later left an index or ceased trading, and fundamental data should preserve original release dates and values alongside later restatements. Otherwise, survivorship or look-ahead bias can distort results. The discussion also warns that trying many hypotheses can produce apparently significant results through overfitting. It gives no strategy-specific returns, transaction-cost model, or complete validation protocol, so these checks reduce common biases but do not guarantee live performance.
Key ideas
- Tune strategy parameters on one segment of historical data and assess them on separate data.
- Walk-forward checks can show whether results rely on isolated historical periods.
- Historical datasets should include securities that later disappeared from the investment universe.
- Fundamental records need point-in-time release dates and values to avoid look-ahead bias.
- Searching many strategy variations increases the risk of overfitting.
Tags
Full text
# How to setup a back test step by step?
# How to setup a back test step by step?
I would like to setup a back test for Indian equities, Kindly help out with step by step procedure, No need to go into details, outlining of procedure are enough in bullet point, I will research further, In case of further queries, I'll raise my question in the same forum.
## Answer by Valtinho (score 1)
https://quant.stackexchange.com/a/26402
There are many ways of doing this, but what most traders do is program their strategies into their trading platform and use the backtest feature to make the program give you the set of indicator parameters that will give you the best returns.
You ideally perform the backtest optimizations on only a portion of your historical prices. Once you are content with the results, run backtests on out of sample data to see if they come close to the optimized results. Also, perform walk-forward tests to make sure the returns (or losses) are distributed throughout history and not in isolated instances.
If you cannot program your strategy due to it being discretionary in nature, you will have to go through your charts and log the buys and sells manually in a spreadsheet.
Try out the Ninjatrader platform. It is free and really good for simple individual tests. You might even be able to make your strategies with a visual wizard without having to program much if your strategy is simple enough.
## Answer by nbbo2 (score 1)
https://quant.stackexchange.com/a/26405
The most difficult aspect of a backtest is making sure you are working with all stocks that were available at the time each decision is made and using only information that would have been available at the time each decision is made.
This means (1) the database must include stocks that were once in the index but now are not ("dead stocks" like Enron or Northern Rock Bank), (2) the database must include "point in time" figures that capture the date that earnings were released as well as the amount announced, if the earnings are restated there should be a second date and figure with the restated earnings and restatement date, and so on. Many databases include only the currently accepted earnings figure and are not clear on the date they were released.
Also, you have to be careful about overfitting, which is the testing of a large number of hypotheses at random in the attempt to generate a statistically significant one.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.