Small Samples, Statistical Significance, and Trading Research
Summary
The document discusses how small samples can mislead readers evaluating research claims, using a paper associated with AQR and the 1987 crash as context. It emphasizes that an observed result may come from a deliberately unrepresentative sample, an accidentally biased sample, or a sample that is representative but too small to support a firm conclusion. A valid and significant sample is also possible, though a reader may lack enough information to judge which case applies.
The central lesson for traders and researchers is to separate the apparent story from the strength and representativeness of the evidence. A striking historical episode or a sample of one cannot by itself establish a general effect. The excerpt provides no details of the paper’s methods, data, or statistical calculations, and it ends before developing its promised discussion of small-sample statistics or a conclusion. It is therefore a brief caution about inference, rather than a complete evaluation of a specific trading strategy or study.
Key ideas
- A small or unrepresentative sample can make an apparent research result misleading.
- A representative sample may still be too small to establish a conclusion.
- Readers may not have enough information to determine whether a study’s evidence is reliable.
- A vivid historical episode alone does not demonstrate a general trading effect.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.