Testing Feature Dependence with the Hilbert-Schmidt Independence Criterion
Summary
The article presents the Hilbert-Schmidt Independence Criterion (HSIC) as a non-parametric test for dependence between data features and a target. Using kernel matrices, HSIC can detect nonlinear as well as linear relationships and can handle scalar or multidimensional observations. The implementation discussed uses a Gaussian radial basis kernel, with its width estimated from pairwise distances, and centers the kernel matrices to calculate the sample statistic.
Because a positive sample statistic alone does not establish significance, the article describes permutation testing and a faster gamma approximation. Permutations estimate the null distribution without assuming a particular data shape, but require repeated calculations; the gamma approach is quicker and may be less accurate with small samples. A synthetic nonlinear example is reported to show dependence that Pearson correlation can miss. HSIC indicates whether dependence is present, not its strength or causal meaning, and results depend on kernel width, sample size, and computational cost.
Key ideas
- HSIC uses kernel functions to test dependence, including nonlinear dependence that linear correlation can miss.
- The sample statistic is calculated from centered kernel matrices for the two variables.
- Permutation testing estimates significance without assuming a specific data distribution but can be computationally expensive.
- A gamma approximation is faster but may be less accurate on small samples.
- HSIC detects evidence of dependence but does not quantify its strength or establish causation.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.