Dynamic Programming for Optimal Trading in a Known Price Model

# Answer by SBF (score 6, accepted)


# Mathematical theories of (sub)-optimal trading strategies under "idealized" assumption - price is random process known to trader












The question is NOT about real trading, but about simplified mathematical models for trading.

One of the main problems in trading is that asset prices are not correctly described by the some random processes.

Let us consider idealized situation - assume that asset's price $p(t)$ is given by some random process, which is known to trader. It roughly speaking means that probabilities of all events $p(t_1) = p_1, p(t_2) = p_2, ..., p(t_k) = p_k$ are known to trader. Both for time $t$ in future or in the past.

What are some theoretical results on optimal or (sub)-optimal trading strategies in this idealized setup? What are some theoretical results on profit estimates ? (I.e. some bounds - we cannot earn more than ...)

If nothing like that is known - what is the reason - is it difficult or "no one needs"?

By "optimal" I mean the following. Of course, intuitively it means that trader will gain the most profit, but here is subtlety - our price $p(t)$ is a random variable, so profit is also a random variable, so it should be specified what means "the most".

It can be $E(p(T))$ (mean value for some $t=T$), or can be $\frac{E(p(t))}{std(p(t))}$ - or what whatever, any mathematically correct result is very welcome.

Let me emphasize that from my point of view this is question mathematically well-defined, and I would expect that many mathematically rigorous theorems (and/or conjectures) should be known on this for experts. If someone doubts that question is mathematically rigorous, please let us discuss in comments.

It might be that such kind of results known only for some special kind of random processes - e.g. Brownian motion or whatever - any information is welcome, I am novice in the field.

For example assume that our random process is actually deterministic process - i.e. only one trajectory $p(t) = p_0(t)$ has probability $1$, all other trajectories has probability $0$. Then the optimal trading strategy is to buy at local minimums and sell at local maximums; profit is variation of $p_0(t)$.

Well, this is of course over-simplified situation, but I just put it to demonstrate that there exist rigorous mathematical results.

Also this example gives some bound on maximal possible mean profit for general random process - we should sum over all possible trajectories their variations with the weight - probability of the trajectory (a'la Feynmann's path-integral).

This bound should not be sharp - it seems it is impossible to achieve it in general - is it correct? What are the sharper bounds?

## Answer by SBF (score 6, accepted)

https://quant.stackexchange.com/a/8134

On the request, here are my two cents. Suppose that the price follows the dynamics $$ \begin{cases} \mathbf z_{k+1} &= F(\mathbf z_k,\mathbf i_k,\mathbf w_k), \\ \mathbf i_{k+1} &= G(\mathbf i_k, \mathbf w_k) \end{cases} $$ where $\mathbf z_k$ is a price of a traded assets at the time $k$, $\mathbf i_k$ is the value of parameters of the model (drift, volatility, state of economy etc.) and $\mathbf w_k$ is a noise process. Parameters and noise can be infinite-dimensional (e.g. parameters can be the whole history), the only restriction we put here is that they belong to some Borel spaces. The price process $\mathbf z_k$ however has to be 1-dimensional, with values in $\Bbb R$.

Now, we assume that at any moment of time the trader

- either can close position if the one is already opened

- if no position is opened, he can open it on the amount of money he has

Let $\mathbf x_k$ be the trader's capital, and let $\mathbf u_k$ be the volume of the currently open position (zero if the position is closed). As a result, the dynamics of the capital is $$ \mathbf x_{k+1} = \mathbf x_k + \rho\mathbf u_k(\mathbf z_{k+1} - \mathbf z_k), $$ where $\rho$ is a leverage. The state space for the trader is hence $$ \underbrace{\Bbb R}_{\text{for }\mathbf x_k}\times \underbrace{\Bbb R}_{\text{for }\mathbf u_k} $$ and the control space is given by $U = [0,\infty)$ with admissible control structure being state dependent: $$ \mathbf u_{k+1} \in \begin{cases} [0,x],&\text{ if } \mathbf u_k = 0, \\ \{0\},&\text{ if }\mathbf u_k > 0. \end{cases} $$ Your performance criterion is a final capital for some fixed time horizon $T$, that is $\mathbf x_T$.

Stated in terms of classical stochastic dynamic programming, you have the following Markov control process $(S,U,\{U(s)\}_{s\in S},f,c)$ where the ultimate state space is $$ S = \underbrace{\Bbb R}_{\text{for }\mathbf x_k}\times \underbrace{\Bbb R}_{\text{for }\mathbf u_k} \times \underbrace{\Bbb R}_{\text{for }\mathbf z_k} \times \underbrace{\Bbb I}_{\text{for }\mathbf i_k}, $$ the control space is $U = [0,\infty)$, the control structure is $$ U(x,u,z,i) = \begin{cases} [0,x],&\text{ if } u = 0, \\ \{0\},&\text{ if } u > 0. \end{cases} $$ and the dynamics $\mathbf s_{k+1} = f(\mathbf s_k,\mathbf w_k)$ in details look as $$ \begin{cases} \mathbf z_{k+1} &= F(\mathbf z_k,\mathbf i_k,\mathbf w_k), \\ \mathbf x_{k+1} &= \mathbf x_k + \rho\mathbf u_k(\mathbf z_{k+1} - \mathbf z_k), \\ \mathbf i_{k+1} &= G(\mathbf i_k, \mathbf w_k). \end{cases} $$ Finally, the running cost is $c_t(s) = 0$ and the final cost is $c_T(s) = x$, where $s = (x,u,z,i)$ is a state vector. The optimal solution to such problems is given by the dynamic programming, and can be consulted in the book of H.-Lerma and Lassaire on Control Markov Processes.

P.S. This is a model, where you can only trade 1 asset, only long positions are allowed, and you can close the whole position only (rather than a part of it). However, extensions are rather trivial and concern mostly the admissible control structure (which controls are available at which states).