Source-linked AI summary
A Topology Layer for Machine Learning
Rickard Brüel-Gabrielsson, Bradley J. Nelson, Anjan Dwaraknath, Primoz Skraba, Leonidas J. Guibas, Gunnar Carlsson
TL;DR
Persistent-homology methods need differentiable machinery for gradient-based machine learning applications. This paper introduces a differentiable topology layer and demonstrates regularization, topological priors for generative models, and persistence-based adversarial attacks, with improvements across reported generative-model metrics.
Problem
Machine learning lacks an accessible differentiable topology layer for using persistent-homology features in gradient-based models and attacks.
Method
The paper computes persistent homology through level-set and edge-based filtrations in a differentiable layer used for regularization, generative losses, and adversarial attacks.
Results
The Topology-Generator improves all reported L2, Inception, and Wasserstein metrics, with results averaged over five computations per metric.
Takeaways & Limitations
Differentiable persistence can promote topological structure in data, images, model weights, generative outputs, and adversarial comparisons.
Takeaways & Limitations
The resulting complexes can grow substantially with sample size and dimension, so the approach targets rapid computation on small- to medium-sized spaces.
Abstract
from arXiv · showhide
Topology applied to real world data using persistent homology has started to find applications within machine learning, including deep learning. We present a differentiable topology layer that computes persistent homology based on level set filtrations and edge-based filtrations. We present three novel applications: the topological layer can (i) regularize data reconstruction or the weights of machine learning models, (ii) construct a loss on the output of a deep generative network to incorporate topological priors, and (iii) perform topological adversarial attacks on deep networks trained with persistence features. The code (www.github.com/bruel-gabrielsson/TopologyLayer) is publicly available and we hope its availability will facilitate the use of persistent homology in deep learning and other gradient based applications.
1 Introduction
The paper introduces a differentiable persistence layer for deep learning, addressing prior work’s focus on preprocessing and specialized applications. It applies the layer to model regularization, topological priors in generative networks, and adversarial attacks.
- Motivation: Prior deep-learning uses of persistence mainly provided topological features as preprocessing or targeted specialized functions.The paper positions its layer as a broader differentiable framework for gradient-based applications.
- Applications: The paper incorporates topological priors into deep generative networks for image and 3D data settings.The motivation includes topological failure cases in generative models and the use of topology to improve such models.
- Applications: The layer enables topological adversarial attacks, which the paper identifies as a first use of persistence features for such attacks.The introduction presents these attacks as an additional application of the differentiable layer.
- Contributions: The layer supports level set and edge-based filtrations while enabling direct regularization of machine-learning model weights.This extends persistence from feature construction to optimization of model parameters.
2 Topological Preliminaries
Persistent homology tracks how homology changes across filtrations and represents feature lifetimes in persistence diagrams. The paper develops differentiable constructions for point-, edge-, and image-based filtrations, while noting computational scaling constraints.
- Persistent homology: Homology dimension counts topological features: H0 counts connected components, H1 counts holes, and higher dimensions capture analogous structures.Homology is invariant under continuous deformations of the space.
- Persistent homology: Persistent homology studies how homology changes across an increasing sequence of complexes called a filtration.For sublevel sets, X_α consists of points where f is at most α, subject to the cell-complex condition.
- Persistence diagrams: Each filtration cell either creates or destroys a homology class, producing birth-death pairs that form persistence diagrams.The paper maps each birth-death pair back to the cells that created and destroyed the corresponding class, enabling differentiation of diagram functionals.
- Persistence losses: The paper uses polynomial diagram losses and Wasserstein distance, with parameters controlling which persistent features are exempt from penalties and how features are weighted.For example, i0 = 2 with PD0 promotes one connected component, while i0 = 2 with PD1 promotes one hole.
- Filtrations and computation: Lower-star filtrations extend values from vertices to simplices, supporting image constructions, while flag filtrations extend edge values and include Rips filtrations based on pairwise distances.The paper also considers weak Alpha filtrations and implements differentiation through persistence diagrams in PyTorch with C++ algorithms.
- Filtrations and computation: Complexity depends strongly on the filtration: Alpha complexes can have O(n^(d/2)) simplices, whereas Rips complexes can reach O(n^(k+1)) simplices.The work targets rapid computation on small- to medium-sized spaces rather than the largest possible complexes.
3 Applications
The applications demonstrate that differentiable persistence can regularize models, encode topological priors in generative networks, and support adversarial attacks using persistence features.
- Topological Noise Reduction and Regularization: Superlevel-set persistence can regularize model parameters by penalizing the number of local maxima.The penalties are imposed on superlevel-set diagrams to target maxima in the parameter topology.
- Topological Noise Reduction and Regularization: Topological penalties improve parameter estimation while directly encoding structures such as clusters, local maxima, and holes.In regression examples, penalties based on persistence outperform ordinary least squares across specified value patterns and can promote one maximum and one hole in an image.
- Incorporating Topological Priors in Generative Models: A topology loss improves generated MNIST images after only 50 batch iterations, producing images with a single connected component.The result is visually significant and similar to using a persistence-aware discriminator.
- Incorporating Topological Priors in Generative Models: The Topology-Generator improves all reported L2, Inception, and Wasserstein-based metrics relative to the baseline.The metrics average five computations, using test-set sizes of 1,000 for L2 and Inception and 100 for Wasserstein distance.
4 Discussion
The paper presents three applications of a differentiable topology layer and identifies several directions requiring further investigation. It positions the layer as a tool for promoting topological structure across data, model weights, and adversarial comparisons.
- The authors identify future work on intermediate activations, persistence-feature extraction within deep networks, and theoretical properties of topological regularization.
- Training autoencoders with distances between persistence features may produce stronger results than the functions considered in this paper.
- Using topological features to train networks more robust to adversarial attacks will require additional work.
A.1 Topological Definitions
This section defines simplicial complexes, homology, filtrations, persistence diagrams, diagram distances, and the inverse maps used for differentiation. The framework connects filtration events to critical simplices or vertices so gradients can be computed through the chain rule.
- Simplicial complexes: A simplicial complex is a collection of simplices containing every face and requiring intersections of simplices to be common faces.
- Filtrations and homology: Persistent homology studies homology across an increasing filtration of complexes, with sublevel sets Xα = f^-1(-∞, α] required to remain valid cell complexes.
- Persistence diagrams: A persistence diagram records birth-death pairs from the interval decomposition of a filtration, together with the diagonal.
- Diagram distances: The p-Wasserstein distance matches diagram points to points in the other diagram or to the diagonal, forming an optimal transport problem.
- Differentiation: Gradients are computed by mapping each birth-death pair to the simplices that create and destroy its homology class, then applying the chain rule through an additional simplex-to-vertex inverse map.
- Differentiation: The inverse map is well defined after extending the filtration's total order to a strict order, which may be done deterministically or randomly.
A.2 Comparison
The paper compares super-level set and Vietoris-Rips filtrations using a pixel representation of the digit “9.” The filtrations capture different geometric and topological features because they use different notions of structure.
- Super-level set versus Rips: Super-level set filtrations capture connected components and holes from pixel values but do not account for grid distance.
- Super-level set versus Rips: Vietoris-Rips filtration embeds pixel values in 3D and uses Euclidean distance, producing more connected components and rings at small to medium filtration values.
- Super-level set versus Rips: The Rips filtration detects not only the obvious ring in the digit “9” but also rings formed relative to the vertical axis.
A.3 Explore latent space
The authors use topological backpropagation to query the latent space of generative models and explore the structure of generated images. An InfoGAN example illustrates searching for a digit type near a specified topological or visual target.
- Latent-space exploration: Topological backpropagation queries a generative model's latent space to improve the topological desirability of its outputs.
- Latent-space exploration: The same latent-space query can explore the nature of the latent space, illustrated with images generated by a trained InfoGAN-generator.
A.4 Topological Mapping
A learned mapping is placed before the Topology Layer to transform image features for topological classification. The mapping appears to thicken digits and remove topological noise, improving accuracy from 84% to 86%.
- A.4 Topological Mapping: The setup inserts a mapping before the Topology Layer while maintaining the input dimensionality.The mapping consists of two convolutional layers.
- A.4 Topological Mapping: 84% accuracy is obtained without the convolutional mapping on naive PD1-feature classification of MNIST.The classes assign digits 1, 2, 3, 4, 5, and 7 to zero; 0, 6, and 9 to one; and 8 to two.
- A.4 Topological Mapping: 86% accuracy is obtained after adding two convolutional mapping layers before the Topology Layer.The mapping preserves the input dimensionality.
- A.4 Topological Mapping: The mapping appears to thicken the digits in the images to remove topological noise.
A.5.1 Features for classification model
The classification model uses directional filtrations to add orientation information to persistence features. Persistence diagrams from eight directions are converted into 400 features for training.
- A.5.1 Features for classification model: Classification using only topological features requires orientation and directional information in the persistence homology features.
- A.5.1 Features for classification model: Eight directional functions are defined using gθ(x, y) = cos(θ)x+sin(θ)y for θ from 0 through 7π/4.
- A.5.1 Features for classification model: The directional functions are shifted and scaled to range from 0 to 1 over the image domain before defining the filtrations.
- A.5.1 Features for classification model: 400 features are produced by computing dimensions 0 and 1 persistence diagrams for each filtration and evaluating 25 features per diagram.
A.6 Regularization Sparsity Visualization
The Rips PD0 diagram compares logistic-regression weights without and with L1 regularization. The diagram changes as test accuracy improves.
- A.6 Regularization Sparsity Visualization: Figure 19 shows Rips PD0 diagrams for logistic-regression weights without regularization and with L1 regularization.
- A.6 Regularization Sparsity Visualization: The persistence diagram changes alongside improved test accuracy under L1 regularization.