Using Correlation Clusters to Define Statistical Equity Sectors
Summary
The document considers whether correlation-based clustering can create statistical groupings of tradable instruments to supplement standard classifications such as GICS. The questioner converts correlations into distances and asks how hierarchical clustering should update distances when previously grouped instruments become new nodes in a dendrogram. The proposed use is portfolio organization, including comparison with existing classifications and resolution of overlap between managers’ chosen universes.
The response cautions that clusters built from security returns may be unstable and may reflect many influences beyond sector similarity. It questions whether the resulting groupings would be meaningful or persuasive enough to replace established industry classifications. No clustering algorithm, distance-update rule, code, or empirical result is supplied, so the document is better read as guidance on limitations and implementation difficulty than as a complete method. Any practical use would require careful feature selection, robustness checks, and evidence that the groups improve portfolio decisions.
Key ideas
- Correlation distances can be used as inputs to hierarchical clustering of tradable instruments.
- A dendrogram can provide a data-driven comparison with conventional sector classifications.
- Return-based clusters may be weak or unstable because many forces affect individual security returns.
- A statistical grouping needs clear evidence of practical value before it can support portfolio assignments.
Tags
Full text
# Dendrogram/cluster analysis of correlation matrix # Dendrogram/cluster analysis of correlation matrix First post, hope I'm explaining things sufficiently clearly. I want to take a universe of potential, trade-able instruments and allocate them to portfolio managers. Traditionally, this is done using a sector classification, such as GICS. When PMs submitted their owned proposed universes they obviously had their own sector classifications which differed from GICS which made me think: 1. Is GICS the best way of organizing things (which I've always concluded that it's not ideal but better than all the other methods)? 2. Is there a statistical way to take all the instruments within our universe and do a cluster analysis and make a dendrogram and then compare the clustering to both GICS and the PM's personally chosen subsets? This would also help for instances of overlap of chosen subsets, as it would help to make the decision as to who should get the overlapping instruments. 3. I created a correlation matrix, and converted it to distances using d^2 = 2(1-|r|) I'm trying to figure out the next step and I'm having a few issues. Euclidean distances are comparable for each node calculation, but it seems like each subsequent node needs a new matrix and new distances as the combination of a prior node represents an additional row/column to add to the original matrix. Since distances are not additive due to lack of orthogonality. Any ideas and code suggestions for doing something in python would be greatly appreciated. ## Answer by Chris (score 1) https://quant.stackexchange.com/a/46830 To summarize, you're attempting to create statistical 'sectors' in lieu of more standard equity classifications (eg, GICS, ICB)? It's something you can do, though to be frank, it's likely a fools-errand. For one, the robustness of your clusters is likely to be pretty weak even if you're thoughtful about the variable(s) you're clustering on and your approach. I'm guessing you're using monthly returns at this point? There's a LOT that drives individual security returns aside from sector-similarity, hence it'll be difficult to get any meaningful results. Not to mention, actually using the clusters will be a difficult sell. 'Fancy' only really works (unless you're the boss) when its benefits are clearly and unambiguously demonstrated, particularly in the face of industry standards. IMO, I don't think that's possible doing what you're trying to do. The rest are implementation issues. I'd suggest getting a text that covers clustering or googling. It's not really something someone can 'tell' you how to do absent sitting together and working through it. HTH
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.