How Time-Series Similarity Methods Relate to Dependence and Factors
Summary
The document surveys ways to study relationships among time series, including covariance, correlation, principal component analysis, factor analysis, cointegration, information measures, graph theory, and clustering. Its central question is whether these methods can be understood within a broader framework, and it points to entropy-based interpretations of some PCA problems as a possible connection.
The answer groups covariance, correlation, PCA, and factor analysis as methods that, under Gaussian assumptions, use shared factors to explain observed relationships. It distinguishes cointegration by its focus on combinations with stationary residuals, and describes mutual information as a way to detect nonlinear dependence without identifying a model that can capture it. Independent component analysis is presented as a search for independent rather than merely orthogonal factors. Graph and cluster methods are characterized as useful for grouping series by discrete behavior, complementing factor methods. These are conceptual comparisons rather than a unified theory or empirical test; the document offers no common scale that makes all methods directly comparable.
Key ideas
- Covariance, correlation, PCA, and factor analysis can be viewed as related tools for finding shared factors under Gaussian assumptions.
- Cointegration focuses on relationships whose residuals are stationary.
- Mutual information can indicate nonlinear dependence but does not specify a model for exploiting it.
- Independent component analysis seeks independent factors rather than only orthogonal ones.
- Graph and cluster methods can group series by discrete features and complement factor analysis.
Tags
Full text
# Meta-view of different time-series similarity measures? # Meta-view of different time-series similarity measures? While I spend most of my StackExchange time on MathematicaSE, I'm in the business and follow the questions and answers on this site with great interest. Recently questions like the following (and some others): - Time-series similarity measures - How are correlation and cointegration related? - Can the concept of entropy be applied to financial time series? - What is the intuition behind cointegration? And even something like this from the Wolfram blog: - Graph Theory and Finance in Mathematica have me thinking about the different approaches available to analyze time-series similarity. These approaches could include: - covariance, - correlation, - co-integration, - PCA, - factor analysis, - entropy, - graph theory, - cluster analysis, and - probably others They certainly have different uses and apply to different aspects of time-series. They all seem to have something to contribute to understanding time-series similarity. So this leads me to a number of questions (or maybe just restatements of the same question): - Can anyone provide a good intuition describing how they all relate to one another? I hope here for something beyond things like some of the excellent discussion of the differences between correlation and co-integration and the kinds of series to which they apply that have appeared elsewhere on this site. I hope for something more along the line of... - Does some point of view or insight exist that provides a better idea of all of these measures? - From some meta-perspective could one view these approaches as aspects of some broader idea? - Could some meta-perspective recast all of these approaches so one could view them in similar units of measure? Not even certain this is possible. Even as I write out this post it seems like asking someone to deliver something like a unified field theory of time-series similarity. Still, if possible it might prove useful so, maybe the questions will spark an interesting answer or two. Of course, any recommendations of papers or other resources that explore any of this appreciated. ... Updated 15 Aug 2012 2:00 PM EDT The following paper gives an example of the kind of thinking that moves in the direction of a broader answer to this question: Empirical Entropy Manipulation and Analysis. The section on casting (at least some) PCA problems as entropy maximization problems follows: And yes PCA can be done by eigenvalue decomposition of a data covariance (or correlation) matrix. So, clearly some of these kinds of analysis have some kind of relationship. Perhaps some view can include more of them. Maybe an entropy or information description of correlation or cointegration. I'm not sure, but to me it seems a interesting question. ## Answer by lehalle (score 11, accepted) https://quant.stackexchange.com/a/3953 Here is a structured list of your bullet points: - covariance, - correlation, - PCA, - factor analysis, Are similar. They are based on Gaussian assumptions (i.e. correlations means dependencies) and try to identify common factors (i.e. a variable in small dimension) explaining the observed relationships. - co-integration is more specific in the sense that you focus on capturing relationships which residuals are stationnary (say i.i.d. to make it simpler). - entropy, is the non linear version of the first list. The only bad point is that when a mutual information (it is the correct term) analysis is positive, you have no guarantee that you will be able to build the adequate model to capture the identified relationship. Entropy gives you the existence of a non linear model, but no clue to find it. Moreover, there is a possible mix between entropy and PCA: the ICA (Independent Component Analysis). It focuses on finding not few orthogonal factors but few independent factors. - graph theory, - cluster analysis, are more adequate for discrete relationships: they will given you homogenous groups of stocks with respect to binary variables (increasing vs decreasing for instance and not take into account the intensity of the vatiations). It is a good complement to a factor or PCA analysis, to help you to understand the meaning of the factors.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.