Visualizing Changes in Covariance and Correlation Matrices

# How do I graphically represent the evolution of a covariance matrix over time?


# How do I graphically represent the evolution of a covariance matrix over time?












I am working with a set of covariance matrices evaluated at various points in time over some history. Each covariance matrix is $N\times N$ for $N$ financial time-series over $T$ periods. I would like to explore some of the properties of this matrix's evolution over time, particularly whether correlation as a whole is increasing or decreasing, and whether certain series become more or less correlated with the whole. I am looking for suggestions as to the kinds of analysis to perform on this data-set, and particularly graphical/pictorial analysis. Ideally, I would like to avoid having to look in depth into each series as $N$ is rather large.

Update

The following graphs were generated based on the accepted answer from @Quant-Guy. PC = principal component = eigenvector. The analysis was done on correlations rather than covariances in order to account for vastly different variances of the $N$ series.

## Answer by Ram Ahluwalia (score 25, accepted)

https://quant.stackexchange.com/a/1566

I would consider a motion chart that plots the eigenvalues of the covariance matrix over time.

For a static view you can create a table: rows represent dates, and columns represent eigenvectors. The entries of the table represent changes in the angle of the eigenvector from the previous row. This will show how stable your covariance structure is.

You can also create a second table this time with eigenvalues as the columns sorted from high to low (and the corresponding values below for each date). This shows the variance described by each eigenvector so you can see whether correlation as whole is increasing or decreasing

Update: You can also measure the distance between the two covariance matrices via some distance measure metric such as Kullback-Leibler divergence, euclidean distance, Mahalanabois, etc.

## Answer by 楊祝昇 (score 10)

https://quant.stackexchange.com/a/1584

Here's an interesting possibility: correlation network analysis + motion chart.

Thanks to the hot research efforts in social network analysis (SNA), network analysis and graphics libraries such as R and Gephis are now easily accessible. I am well-versed in correlation analysis, and have a feeling that SNA can be effectively adapted for it. After all, the 'linear' correlation is just a special relationship among many others. Since SNA is hot area now. Many efforts are devoted and many resources are available. Can't help imagining that we can leverage it in finance.

Then the motion chart concept can further leverage the power of network representation. I would imagine that when a regime shifts toward high correlation, we should see the dynamics of 'clustering' effects, whereas low correlation regimes would show nodes scattered around. For example, if there is a sector-level event, we would see nodes in that sector start to cluster more tightly around few benchmark nodes. In a macro event, we would see all benchmarks or eigenvectors cluster.

In any cases, I think the motion network graph will be a much richer representation than the motion matrix, where the dynamics are usually represented in numbers, colors, or angles. If one still prefers to visualize 'covariance', then the node size will be a natural place for volatility (though I still prefer to separate correlation and variance visualization). The line width / color / distance can represent something else intuitively.

Last, I guess the reason evolving network graphs are not often used for financial correlations is that a lot of the traditional financial applications are static, in which case the advantages of network graphs over matrix representation are limited.

## Answer by ZAxisMapping (score 4)

https://quant.stackexchange.com/a/1580

I'd look at the evolution of a heat map based on the correlation structure (literally the lower triangle). I'd probably write a script in R or python that writes out the heat map per t to disk, then use a command line program like imagemagick to stitch images together into an animated gif, for example. I'm sure you could do it entirely in Processing too, and there you'd be able to mouse over pairs, etc. Perhaps seriation techniques could group sets ex post within the triangle.

## Answer by Qbik (score 2)

https://quant.stackexchange.com/a/3355

On conceptual level making cluster analysis or kmeans, for arbitrarily chosen k (in cluster analysis with hierarchical methods we simple would cut dendrogram into k pieces/subtrees) and then copmuting average correlations is much simpler then PCA.

But there are some problems with cluster analysis on correlation matrix of time series. If ,for each day, we have one correlation matrix of price changes with 15 minutes time resolution and then clusterize it into K pieces, then for each day we could have different sets of stocks in each cluster, we can fight or use it.

Fight it: if data are from the period of 6 months then we find clusters by merging daily time series of each stock into one time series and then clusterize and compute averages or some different measure. For Future data we can hold stocks memberships to clusters or merge new observations with previous set or make analysis in moving window.

Use it : compute total number of changes in membership of stocks - first day we have clusters A,B `A={stock1,stock2} B={stock3,stock4,stock5}` next day : `A={stock1,stock3,stock4} B={stock2,stock5}` so 3 stocks have changed its clusters. The more stocks have changed clusters the more market behaviour is unusual that day. This could be done easy with kmeans - we take means from day `T`, and attribute memberships for stocks `T` and `T+1`day, then count differences.

and distance matrix `=(2*(1-corelationMatrix))^0.5`

Some cool visualizations of correlation matrix by networks analysis : http://www.maths.tcd.ie/~coelhor/Palermo_Presentation_v1.0.pdf

## Answer by michaelv2 (score 1)

https://quant.stackexchange.com/a/1573

Minimum spanning trees are another option, with edges between nodes based on either Euclidean distances of the matrix or another distance measure of your choosing. They can be more effective at illustrating the underlying structure of the matrix than some other methods (eg heatmaps, eigenvalue ratio plots), but this may not be practical if T is large.

You didn't specify what environment you're working in, but R has several packages (see vegan:spantree, ape:mst, igraph:minimum.spanning.tree, ade4:mstree and fAssets:assetsDendrogramPlot or fAssets:assetsCorEigenPlot) that support plotting both minimum spanning trees (with various layout types) and dendrograms. Gephi also produces plots that may be more aesthetically pleasing (and it's open source), although it does require that the data processing be done elsewhere and there is a bit of a learning curve.