Detecting Price Inputs Mistaken for Returns
Summary
The discussion considers how an allocation algorithm can flag input data that may be prices rather than returns. A simple check for negative observations can identify some return series, since prices are generally nonnegative, but it misses price paths whose changes are always nonnegative. The question frames the challenge as imperfect classification: prices and returns can overlap in their observed values, so no universal test can reliably distinguish them from values alone.
One proposed diagnostic is to regress the series on its lag and test the lag coefficient. The answer suggests that prices often behave like an integrated series with a coefficient near one, while returns may be stationary with a coefficient nearer zero. This is a statistical heuristic, not a definitive classification rule, and the discussion does not specify a full testing procedure or address all data-generating processes. A further caveat is that recorded price datasets can contain negative values for conventions unrelated to an economically negative price, so sign checks can create false warnings.
Key ideas
- A negative observation can suggest returns, but its absence does not establish that a series contains prices.
- Testing whether a series is integrated or stationary can help distinguish price levels from returns.
- A lag regression may show a coefficient near one for prices and nearer zero for returns.
- These diagnostics are probabilistic warnings rather than necessary and sufficient classification tests.
- Some data vendors encode missing trades with negative reported prices, complicating sign-based checks.
Tags
Full text
# How do I statistically differentiate a series of prices from a series of returns?
# How do I statistically differentiate a series of prices from a series of returns?
I developed an optimization algorithm that uses returns (among other parameters) as input and basically output an allocation.
As I'm pretty happy with the results, I am in the process of putting the algorithm in production. As a matter of fact, I have to embed some checks into the algorithm to make sure that the input was right and that the output makes sense. For example, I check that the allocation sums to 1, I check that the time series provided are all of the same length and so on...
I would like to add another test which allows me to display a warning if the input time series appear to be prices instead of returns (and people will do this mistake one day I'm sure). So, I would like to setup a statistical test on my set of points $x=x_1, ... , x_n$ to determine whether they are likely to be prices.
Formally the statistical test if a function defined as follows:
$$A(x) \rightarrow \{0,1\}$$
Ideally we would like to find a test that is sufficient and necessary for a time series to be returns (as compared to prices, not to "a random time series").
There are three types of interesting statistical tests:
Type I (necessary and sufficient)
$$A(x)=1 \iff x ~ \text{are returns}$$
Type II (sufficient)
$$A(x)=1 \Longrightarrow x ~ \text{are returns}$$
Type III (necessary)
$$A(x) \neq 1 \Longrightarrow x ~ \text{are prices}$$
The following test is a dummy one:
$$A_{\text{dummy}}(x)=\frac{1}{n} \sum_{i=1}^n x_i > 1$$
This is not good because a Forex time series are prices of a currency expressed in the base currency of the portfolio, and such a series would produce a result of 1.
I came up with a Type II test:
$$A_\text{Type II}(x) = \exists i ~ x_i<0$$
Note that I assume that the input could be either prices or returns.
A series of prices with non-negative returns would fool this test so it is not necessary.
I believe it is impossible to come up with a Type I test, which implies that I can't come up with a Type III test either (otherwise I could construct a Type I test easily).
I would be looking for extra Type II tests to improve the probability of wrong input detection.
Have you ever had to do such test? What method would you recommend?
## Answer by Marc Shivers (score 4)
https://quant.stackexchange.com/a/3676
You could test for whether the input series is I(0) vs the alternative of I(1). Specifically, regress the input series on its own lag, and test whether the coefficient on the lag is significantly different from zero. Price series should have a coefficient close to 1, while return series should have a coefficient close to 0.
## Answer by Akavall (score 3)
https://quant.stackexchange.com/a/3668
I think that can never be 100% sure, and the most you could do is raise a warning, and your approach makes perfect sense to me.
I want to point out one thing though. While prices cannot be negative, they are sometimes recorded with a negative sign, where negative sign conveys some other information. For example in CRSP:
> Price Usually, the CRSP price is the closing price, the price of the last reported trade on any given day. However, if no trade is recorded on a given trading day, the reported price will be the negative of the average of the bid and asked prices for that day. The price reported in CRSP monthly files is the price of the last trading day of the month.
linkShown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.