Testing Whether a Delayed-Data Equity System Has a Real After-Cost Edge
Summary
The document describes a research system for Borsa Istanbul built from delayed market data and intended to rank candidates for manual daily decisions. Across multiple architectures, it combines price behavior, intraday moves, market and sector context, candidate screening, risk checks, allocation, holding periods, and exit decisions. The authors report extensive historical research but say they have not established repeatable, executable performance after costs.
The account identifies several reasons historical results may fail to generalize: transaction costs, liquidity and slippage, allocation and replacement choices, changing market periods, and repeated experimentation that increases overfitting and false discovery risk. It also notes that incomplete historical records prevent reconstructing some results as real capital performance. The proposed research priority is an immutable, preregistered test with genuinely unused or prospective data, realistic execution assumptions, and independent validation. The included response adds that weak out-of-sample results may reflect overfitting or look-ahead bias, and cautions that delayed intraday data may offer little edge. The material is a problem statement, not evidence that the system does or does not work.
Key ideas
- Historical candidate-selection results do not establish a repeatable trading edge after costs.
- Repeated experimentation on the same history raises the risk of overfitting and false discoveries.
- Execution details, including liquidity, slippage, costs, and exit timing, can separate paper results from realized capital growth.
- Incomplete decision and trade records limit reconstruction of historical executable performance.
- A preregistered, immutable test using unused or prospective data and realistic assumptions can help evaluate persistence.
- Weak out-of-sample results may indicate overfitting or overlooked look-ahead bias.
Tags
Full text
# We built 19 Borsa Istanbul research systems but still cannot prove a real trading edge — what are we missing? # We built 19 Borsa Istanbul research systems but still cannot prove a real trading edge — what are we missing? We have been developing an independent decision-support system for Borsa Istanbul for approximately 22 months. The system uses only delayed market information that was available at the exact decision time. It does not use future data, real-time private feeds, other stock markets, or automated order execution. It does not manage client money. Its purpose is to examine the market, identify candidates, compare their relative strength and risk, and produce a manual daily decision. #### How the system works Over time, we built and tested 19 different research architectures. These were not simple parameter changes; they examined different combinations of price behaviour, intraday movement, market and sector conditions, historical similarities, candidate ranking, downside protection, capital allocation, holding periods, and exit decisions. The current AVCI architecture contains: - 20 main modules - 23 supporting files - Data collection and validation - Decision-time locking - Market and sector analysis - Proprietary pattern and similarity layers - Candidate-pool creation - Candidate filtering and ranking - Risk and uncertainty checks - Decision recording - Next-session result and cost measurement - Final consistency and evidence checks Historical daily and real one-minute BIST price and trading data have also been examined. Information created after the decision time is not supposed to enter the decision process. #### The unresolved problem Despite thousands of hypotheses, simulations, tests, and several complete architecture changes, we have not been able to prove a repeatable and executable after-cost edge. Promising historical results often weaken or disappear when: - The market period changes - Transaction costs are included - Capital is distributed among candidates - Holding and replacement rules change - Large winning days or stocks are removed - Previously unused periods are tested - Realistic execution assumptions are applied A further problem is that much of the historical period has already been examined during research. After thousands of experiments, even an apparently excellent historical result may simply be a false discovery caused by overfitting and repeated testing. There are also unresolved differences between a paper result and actual capital growth. A correct candidate does not automatically mean a profitable trade. Entry price, liquidity, slippage, tradable quantity, transaction costs, corporate actions, holding time, and exit timing can all change the result. Our historical records also do not provide a complete real-money ledger containing every decision, executed quantity, entry, exit, cost, and daily capital change. For this reason, some old results cannot be reconstructed as genuine executable performance. At present, we cannot confidently distinguish between three possibilities: - Delayed BIST information may contain no persistent edge strong enough to survive all costs. - A real edge may exist, but it may be lost during ranking, allocation, holding, replacement, or risk control. - The apparent edge may exist only on paper because execution and historical validation assumptions are unrealistic. #### What we are looking for We are not looking for: - Stock tips - A ready-made strategy - Another generic machine-learning model - Hundreds of new indicators - A twentieth architecture built without first identifying the real problem We are looking for experienced researchers, graduate students, quantitative developers, market-microstructure specialists, or independent practitioners who are willing to examine this problem carefully and patiently. The central question is: > What immutable and pre-registered test could determine whether AVCI contains a real, executable and persistent after-cost edge—or whether the apparent historical advantage is only the result of overfitting, repeated testing, or unrealistic execution assumptions? We are especially interested in people with experience in: - Backtest overfitting and false discovery - Delayed financial data - Emerging or relatively illiquid equity markets - Borsa Istanbul - Transaction costs, liquidity and slippage - Portfolio allocation and risk - Holding and replacement decisions - Walk-forward or prospective testing - Pre-registration and independent validation A negative conclusion is acceptable. The objective is not to make an unsuccessful system appear successful. The objective is to determine, with defensible evidence, whether a real edge exists, where it disappears, or why it cannot be extracted under the current constraints. This is not a quick question that can be solved with one indicator or a few comments. We are looking for serious contributors who are willing to understand the architecture and help define a small number of decisive experiments. A concise anonymized technical summary can first be shared with serious contributors. Proprietary selection rules, the complete source code, and raw data that we do not have the right to redistribute will not be posted publicly. Our core question is: Why, although this large research infrastructure appears able to identify strong candidates, can we not convert that ability into repeatable, executable, after-cost capital growth across different market periods? ## Answer by del_nabla (score 4) https://quant.stackexchange.com/a/85736 If your IS performance is stellar but your OOS performance is sub-par or negative than you can conclude that you've built a model that is complex enough to have overfit, or you're possibly missing something when looking for potential look-ahead biases. If by delayed you mean the 15m broadcast delay compulsory for data vendors streaming BIST data for free than you are not likely to find any edge there (at least intraday though it would be quite problematic for any horizon). P.S. You are unlikely to achieve your targets with free consulting. As someone already working in the industry, I assume this approach might have been caused by your lack of experience.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.