Source-linked AI summary
Deep Learning with Topological Signatures
Christoph Hofer, Roland Kwitt, Marc Niethammer, Andreas Uhl
TL;DR
Topological signatures provide useful information but are awkward for standard learning methods and existing mappings are fixed rather than task-specific. The paper introduces a trainable, stable neural-network input layer for persistence diagrams and evaluates it on shapes and social-network graphs, considerably outperforming the state of the art on the latter.
Problem
Topological signatures such as persistence diagrams have unusual multiset structure, while existing machine-learning representations are predefined and agnostic to the target task.
Method
The paper introduces a neural-network input layer that learns a parametrized, task-optimal projection of persistence diagrams during training.
Results
Experiments on 2D object shapes and social-network graphs demonstrate versatility, with considerable state-of-the-art improvement on both social-network datasets.
Takeaways & Limitations
Topological signatures can be learned directly by deep networks and used as additional inputs to existing deep architectures.
Takeaways & Limitations
Kernel-based vectorization was computationally impractical on the evaluated shape and graph data.
Abstract
from arXiv · showhide
Inferring topological and geometrical information from data can offer an alternative perspective on machine learning problems. Methods from topological data analysis, e.g., persistent homology, enable us to obtain such information, typically in the form of summary representations of topological features. However, such topological signatures often come with an unusual structure (e.g., multisets of intervals) that is highly impractical for most machine learning techniques. While many strategies have been proposed to map these topological signatures into machine learning compatible representations, they suffer from being agnostic to the target learning task. In contrast, we propose a technique that enables us to input topological signatures to deep neural networks and learn a task-optimal representation during training. Our approach is realized as a novel input layer with favorable theoretical properties. Classification experiments on 2D object shapes and social network graphs demonstrate the versatility of the approach and, in case of the latter, we even outperform the state-of-the-art by a large margin.
1 Introduction
Topological signatures capture useful geometric and topological information but have structures that are difficult for standard machine-learning methods. The paper proposes learning task-optimal representations with a trainable neural-network input layer.
- Persistent homology represents features by birth–death pairs collected as persistence diagrams or barcodes.
- Multiset-valued signatures are impractical for learning because metric comparisons such as Wasserstein distance require optimal matching problems.
- Existing vectorization strategies use predefined mappings, making their representations fixed and agnostic to the target learning task.
- The proposed input layer computes a parametrized projection of persistence diagrams whose representation can be learned during network training.
- Experiments cover 2D object-shape classification and social-network graph classification, with a large state-of-the-art improvement reported for the latter.
2 Background
The background develops homology, filtrations, persistent homology, and persistence diagrams as the mathematical basis for topological signatures. It also introduces distances and notes the use of simplicial homology over Z/2Z.
- Homology: Homology studies an object through algebraic structures formed from chain groups, boundary operators, and homology groups.
- Homology: Simplicial homology is the homology theory used throughout the work, with chains generated by simplices over Z/2Z.
- Persistent homology: A filtration is a nested sequence of simplicial complexes, and persistent homology tracks homological information across that sequence.
- Persistent homology: Persistent Betti numbers count features of dimension n that persist from index i to at least index j, and these quantities encode the persistent homology information.
- Topological signatures: Sublevel-set filtrations of functions on vertices produce multisets of birth–death points that encode persistent homology and form persistence diagrams.
- Distances: Bottleneck and Wasserstein distances compare persistence diagrams through bijections, supporting stability analysis of topological signatures.
3 A network layer for topological signatures
The proposed layer projects persistence diagrams onto a fixed collection of structure elements whose locations and scales are learned during training. Its output concatenates the resulting projections into a neural-network-compatible representation.
- The layer takes a persistence diagram and projects it with respect to a fixed-size collection of structure elements.
- A rotation places the diagonal on the x-axis, so the y-axis represents feature persistence.
- Unlike earlier constructions with fixed element locations, this layer learns structure-element locations and scales during training.
- The layer combines the structure-element mappings by concatenation to produce its output.
- Backpropagation can train the layer because each element is differentiable, each projection is a finite sum, and the final operation is concatenation.
4 Theoretical properties
The layer is designed to remain stable under the 1-Wasserstein distance while learning a representation tailored to the downstream task. The paper illustrates this robustness using noisy and clean shape filtrations.
- The proposed layer is stable with respect to the 1-Wasserstein distance.
- The stability construction relies on sufficient conditions for the underlying functions, including bounded first-order partial derivatives and Lipschitz continuity.
- The chosen structure element is one member of a broader family of input layers satisfying the stability requirements.
- After training, the input layer can be retained as a persistence-diagram mapping specifically tailored to the learning task.
- In the noisy-shape example, added noise mainly shifts dominant points slightly and creates near-diagonal points with little effect on Wasserstein distance or layer output.
5 Experiments
Experiments evaluate the proposed topological-signature approach on 2D object shapes and social network graphs using persistent-homology representations and task-specific neural architectures. It outperforms tailored shape methods and state-of-the-art graph classifiers, while direct vectorization performs worse and kernel comparisons are computationally impractical.
- Experimental setup: Experiments classify binary 2D object shapes and social network graphs using balanced labels, 90/10 train/test splits, five-run averages, and fixed ν = 0.1.The shape datasets are Animal and MPEG-7; the graph datasets are reddit-5k and reddit-12k.
- 2D object shapes: For 2D shapes, the method constructs simplicial complexes directly from raw pixels and uses height-function filtrations across multiple directions.Pixels inside each object form vertices, 4-neighbor pixel pairs form edges, and filtration values use the object barycenter and bounding-circle radius.
- 2D object shapes: The shape network uses 32 independent direction branches, with each branch receiving persistence diagrams from neighboring filtration directions to capture directional dependencies.Training uses cross-entropy with SGD for 400 epochs and mini-batches of 128.
- 2D object shapes: The proposed architecture beats other problem-specific shape approaches despite remaining below the reported state of the art, without requiring contour-extraction preprocessing.The architecture also generalizes to 3D by replacing directions in S1 with directions in S2.
- Social network graphs: On both reddit graph datasets, topological signatures considerably outperform GK, DGK, PSCN, and feature-based random-forest baselines.The authors report that including essential features is key to these improvements; graph filtration uses normalized vertex degree.
- Reference comparison: Direct persistence-diagram vectorization performs poorly on MPEG-7 and Animal, can be competitive on reddit-5k and reddit-12k, but remains inferior to the proposed approach.The reference vectors retain the first N persistence values after sorting and pad with zeros when necessary.
6 Discussion
The approach learns task-optimal stable representations of persistence diagrams through a deep-network input layer, while acknowledging a logarithmic bending drawback and broader multiset applicability.
- 6 Discussion: The proposed input layer learns task-optimal stable representations of persistence diagrams for deep neural networks.It also permits topological signatures to serve as additional inputs to existing deep architectures.
- 6 Discussion: The layer’s structure elements are not restricted to exponential functions when the conditions of Lemma 1 are satisfied.
- 6 Discussion: A drawback is the artificial bending of the persistence axis caused by a logarithmic transformation.The paper leaves investigation of potentially better strategies for future work.
- 6 Discussion: In principle, the proposed layer can handle multisets of R^n, whereas previous works handle fixed-size sets.
A Technical results
The technical results omit a proof, referring readers to supplementary material and noting that only one of two statements requires demonstration.
- A Technical results: The proof is omitted for brevity and deferred to the supplementary material.
- A Technical results: Only statement (i) needs to be shown because statement (ii) follows immediately.
- A Technical results: The section presents this proof-status distinction as part of the technical result.
B Proofs
The proofs establish the Wasserstein-based result by decomposing the relevant sets and analyze derivative behavior separately across the definition’s two domains.
- B Proofs: A bijection realizing the Wasserstein distance is used to define reduced sets after removing the diagonal.
- B Proofs: Because points in the removed diagonal contribute zero, the summation can be restricted to the union E = A ∪ B ∪ C.
- B Proofs: The proof of Lemma 2 distinguishes x1 ∈ [ν, ∞) from x1 ∈ (0, ν) because s_μ,σ,ν is defined differently on those domains.
- B Proofs: For x1 ∈ [ν, ∞), the partial derivative is analyzed and Eq. (7) tends to zero as x0 or x1 diverges.
- B Proofs: For x1 ∈ (0, ν), the derivative analysis uses the same asymptotic behavior in x0 and separately examines the x1 derivative.
C Additional proofs
The additional proof establishes the limiting behavior of a logarithm-exponential expression using a decomposition and l’Hôpital’s rule.
- C Additional proofs: Lemma 4 assumes α ∈ R+, β ∈ R, and γ ∈ R+ and contains two statements.
- C Additional proofs: Only the first statement requires proof because the second follows immediately.
- C Additional proofs: The expression x · e^−α(ln(x)^γ+β)^2 is rewritten as a product involving ln(x) and e^−ln(x).
- C Additional proofs: The proof applies l’Hôpital’s rule to the resulting limit.
D Network architectures
The 2D shape and graph classifiers use distinct network architectures tailored to their respective persistence-diagram inputs. Both are trained with stochastic gradient descent, but differ in branch structure and training schedules.
- 2D object shape classification: The 2D shape network uses 32 input branches, one for each filtration direction.Each branch receives persistence-diagram vectors from three consecutive directions, capturing dependencies among neighboring filtration directions.
- Graph classification: The graph network separates homological features into input branches for essential and non-essential components.The architecture uses branches associated with dimensions 0 and 1, with dimension-1 features containing only essential components.
D.1 Technical handling of essential features
Essential features are handled differently for 2D shapes and social-network graphs. Shape diagrams retain essential-feature death times at the maximum filtration value, while graph essential features are mapped to the real line and separated into branches.
- 2D object shape classification: For 2D object shapes, essential-feature death times are mapped to the maximum filtration value and retained in the original persistence diagrams.Animal and MPEG-7 each have one connected component, so dimension-0 contains only one essential feature.
- Graph classification: For social-network graphs, essential features are mapped to the real line using their birth time.These features are handled in separate input branches with one-dimensional structure elements.
- Architectural consequence: The graph architecture separates essential and non-essential features across homological dimensions, unlike the single-essential-feature shape setting.The graph design accommodates potentially many essential features in dimensions 0 and 1.