Using Quantized Order Imbalance Entropy to Track Distribution Changes

# Lopez de Prado Advances in Financial Machine Learning- entropy for adverse selection


# Lopez de Prado Advances in Financial Machine Learning- entropy for adverse selection












In chapter 18: Entropy Features, Lopez de Prado discusses how entropy can be used to estimate adverse selection. He suggests a method where order imbalance is mapped to quantiles and entropy is calculated for this time series, he then says we can estimate a distribution F(H(X)), but H(X) is a single value calculated across the time series, so how can we find a distribution for this, then proceed to calculate values of F(H(Xt)) for each time point (what even is the notion of F(H(Xt)) when Xt is a single value, is the author defining Xt=X{t’<=t}, if so this isn’t very clear, and we certainly wouldn’t be able to calculate the entropy for the first few values in the time series as there would be too few points)

## Answer by lehalle (score 1)

https://quant.stackexchange.com/a/80879

Here is the structure of the methodology described in the paragraph you mention

- "given a sequence of volume bars indexed by $\tau = 1,\ldots , N$, each bar of size $V$, we determine the portion of volume classified as buy, $\nu^B_\tau\in [0, 1]$." `=>` it says that (1) you choose a unit of traded volume $V$ (like around 100 Average Trade Size) and you bin your tick by tick data by grouping successive transactions summing to $V$, inside each bin, $\nu^B_\tau$ is the proposition of shares that have been bought by a liquidity consumer.

- "compute the $q$-quantiles on $\nu^B_\tau$ that define a set $K$ of $q$ disjoint subsets, ${\cal K}= \{K_1, \ldots , K_q\}$." `=>` it says you group historical values of the proportions $\nu^B_\tau$ in $q$ consecutive intervals of equal size.

- "produce a mapping from each $\nu^B_\tau$ to one of the disjoint subsets, $f : \nu^B → \{1,\ldots,q\}$, where $f[\nu^B] = i \Leftrightarrow \nu^B\in K_i ,\forall i \in [1,q]$" `=>` it means that once you have the set of intervals ${\cal K}$, you make your best to associate a proportion to the corresponding intervals. It is true that if the distribution shifts, the two extreme intevals $K_1$ and $K_q$ can change so even if it is not mentioned, you should consider them as open-ended (i.e. all that is at the left of $K_1$ is considered as being inside it, and similarly at the right of $K_q$)

- "quantize $\{\nu^B_\tau\}$ by assigning to each value $\nu^B_\tau$ the index of the subset $K$ it belongs to, $f [\nu^B_\tau ]$." `=>` Now you replace any sequence of proportions $(\nu^B_1, \ldots, \nu^B_T)$, by a sequence of quantiles $X=(i_1,\ldots, i_T)$

- you estimate the entropy $H(X)$ using the approximation algorithms of Kontoyiannis, Ioannis. "An implementable lossy version of the Lempel-Ziv algorithm" IEEE Transactions on Information Theory 45, no. 7 (1999): 2293-2305.

The overall procedure corresponds to

- take your indicator (the proportion of shares bought a liquidity consuming manner) and freeze quantile intervals before time $t^*$

- after this time, take sequences of $\ell$ consecutive values of the indicator and rank it according to the frozen intervals.

What does it means on an example? Say that your reference period if of length $t^*=100$ and you take $q=100$, then it means that each element is related by its rank. Then say you take $\ell=100$ too (to make it simple): it means that if the distribution the same after $t^*$ as before $t^*$, then $X$ is always $X_0=(1,2,\ldots,99,100)$, hence its entropy is $$H(X_0)=\sum_{n=1}^{100} p_n\log(p_n)=\sum_n 1/100 \log(1/100)=-\log(100).$$ It never changes.

If the sequence of indicators changes, for instance let's say all the values increase and now $X_+=(50,50,51,51,\ldots,99,99,100,100)$ meaning that the new values are all on the half upper interval, uniformly. Then $$H(X_+)=\sum_n 2/100\log(2/100)=2\log(2)-2\log(100)<-\log(100)=H(X_0).$$

I hope it is clear for you now: this will measure if the distribution changed. Unfortunatley, it will not tell you how it will change; for instance if the indictors change exactly the opposite way, i.e. their values all decreases uniformly: $X_-=(1,1,2,2,\ldots,49,49,50,50)$, then $$H(X_-)=\sum_n 2/100\log(2/100)=H(X_+).$$ There is not difference. I would probably recommend another indicator that does not destroy the information on a direction, especially if (like in the example), you are considering a directional indicator (i.e. the proportion of buy).

An entropy-driven indicator is more naturally adapted to a non-directional indicator, like the distribution of the bid-ask spread, or of the volatility.