Source-linked AI summary
Spectral community detection in sparse networks
M. E. J. Newman
TL;DR
Conventional spectral community detection can fail in sparse networks, motivating alternatives such as the non-backtracking matrix. This paper develops and studies the flow matrix, a variant enabling relaxed spectral modularity optimization, and finds better-behaved spectra and eigenvectors for broad-degree networks.
Problem
Conventional spectral methods can become virtually useless in sparse networks because hubs create localized outlying eigenvectors that interfere with community information.
Source: p. 1Method
The paper studies the flow matrix, a variant of the non-backtracking matrix, and derives a spectral algorithm using the leading eigenvector of F − 11T to approximate modularity optimization.
Source: pp. 1–3Results
The flow matrix has a clearer spectral band edge and more tightly peaked leading-eigenvector elements than the original non-backtracking matrix, with 93% versus 90% correct classification in the tested network.
Source: pp. 2, 4Takeaways & Limitations
The flow matrix provides a practical spectral community-detection approach for sparse networks with broad degree distributions, including realistic networks.
Source: pp. 1–2The relaxation gives only an approximation to the objective optimum, and the dangling-tree problem is not completely solved.
Source: p. 5
Abstract
from arXiv · showhide
Spectral methods based on the eigenvectors of matrices are widely used in the analysis of network data, particularly for community detection and graph partitioning. Standard methods based on the adjacency matrix and related matrices, however, break down for very sparse networks, which includes many networks of practical interest. As a solution to this problem it has been recently proposed that we focus instead on the spectrum of the non-backtracking matrix, an alternative matrix representation of a network that shows better behavior in the sparse limit. Inspired by this suggestion, we here make use of a relaxation method to derive a spectral community detection algorithm that works well even in the sparse regime where other methods break down. Interestingly, however, the matrix at the heart of the method, it turns out, is not exactly the non-backtracking matrix, but a variant of it with a somewhat different definition. We study the behavior of this variant matrix for both artificial and real-world networks and find it to have desirable properties, especially in the common case of networks with broad degree distributions, for which it appears to have a better behaved spectrum and eigenvectors than the original non-backtracking matrix.