Source-linked AI summary
SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels
Matthias Fey, Jan Eric Lenssen, Frank Weichert, Heinrich Müller
TL;DR
Deep convolutional networks are difficult to transfer from grids to irregular graphs and meshes. SplineCNN introduces trainable continuous B-spline kernels that aggregate features spatially, and the paper reports improved or state-of-the-art results across several tasks with fast computation.
Problem
Traditional CNN convolutions rely on grid-like structure, creating a need for methods that handle irregular graphs, meshes, and other geometric data.
Method
SplineCNN uses trainable continuous B-spline kernels to aggregate local features in the spatial domain from geometric relations, without requiring handcrafted descriptors.
Results
SplineCNN improves state-of-the-art results across image graph classification, graph node classification, and mesh shape correspondence while enabling fast training and inference.
Takeaways & Limitations
The architecture supports end-to-end learning directly from geometric data and applies to irregular graphs and meshes.
Takeaways & Limitations
Spectral approaches may omit relative positions in embedded graphs or meshes and can be restricted by domain-dependent Fourier bases, motivating SplineCNN’s spatial formulation.
Abstract
from arXiv · showhide
We present Spline-based Convolutional Neural Networks (SplineCNNs), a variant of deep neural networks for irregular structured and geometric input, e.g., graphs or meshes. Our main contribution is a novel convolution operator based on B-splines, that makes the computation time independent from the kernel size due to the local support property of the B-spline basis functions. As a result, we obtain a generalization of the traditional CNN convolution operator by using continuous kernel functions parametrized by a fixed number of trainable weights. In contrast to related approaches that filter in the spectral domain, the proposed method aggregates features purely in the spatial domain. In addition, SplineCNN allows entire end-to-end training of deep architectures, using only the geometric structure as input, instead of handcrafted feature descriptors. For validation, we apply our method on tasks from the fields of image graph classification, shape correspondence and graph node classification, and show that it outperforms or pars state-of-the-art approaches while being significantly faster and having favorable properties like domain-independence.
1. Introduction
Traditional CNN convolutions depend on grid structure, limiting their direct use on graphs and meshes. SplineCNNs address this with trainable continuous spatial kernels and support irregular data, end-to-end learning, and strong task performance.
- Motivation: CNN convolution properties are difficult to transfer directly from grid-like inputs to non-Euclidean graphs and meshes.Irregular structures nevertheless arise naturally in practical graph and mesh tasks.
- Approach: SplineCNNs introduce a trainable, spatial, continuous convolution kernel based on B-spline bases for irregular structured data.The approach uses geometric relations in the input rather than spectral filtering.
- Scope: SplineCNNs apply to arbitrary embedded graphs and meshes while using spatial geometric relations of the input.Figure 1 illustrates spatial aggregation for image graph representations and meshes.
- Training: SplineCNNs enable end-to-end training without handcrafted feature descriptors.The network learns directly from the geometric structure supplied by the input.
- Results: SplineCNNs improve or match state-of-the-art performance in geometric learning tasks and support fast training and inference.The paper also provides an efficient GPGPU implementation for computation.
2. Related work
Geometric deep learning includes spectral and spatial approaches for extending neural convolutions to irregular data. SplineCNN builds on spatial continuous kernels while addressing positional information and domain-generalization issues associated with related methods.
- Spectral approaches: Spectral graph methods interpret Laplacian eigenvalues as signal frequencies and filter node signals in the spectral domain.Related work also includes spatial filtering approaches for irregular data.
- Geometric information: Spectral approaches may omit relative node positions that provide information in embedded graphs and meshes.SplineCNN incorporates these spatial relations.
- Generalization: Domain-dependent Fourier bases can restrict spectral methods to inputs with identical graph connectivity.SplineCNN works directly in the spatial domain and is not prone to this specific restriction.
- Local descriptors: Local-descriptor methods extract and align Euclidean neighborhoods from meshes before applying convolution operations.Examples use intrinsic shape descriptors, two-dimensional patches, and patch rotation based on local principal curvatures.
- Spatial continuous kernels: SplineCNN can use local representations but does not require them, making it orthogonal to improvements in descriptor extraction.Its spatial continuous kernels build on related neural message-passing and self-attention approaches.
3. SplineCNN
SplineCNN defines spatial convolution on irregular structures by weighting local feature aggregation with trainable continuous B-spline kernels over pseudo-coordinates. Its local support enables efficient computation, while coordinate choices and closed splines extend the operator across graphs, meshes, and periodic dimensions.
- 3.1. Preliminaries: SplineCNN maps irregular structured data to directed graphs whose edges carry normalized pseudo-coordinates and whose nodes carry input feature vectors.The pseudo-coordinates determine how neighboring features are weighted, while node features determine what is aggregated.
- 3.3. Convolution operator: The convolution aggregates neighboring node features using a trainable continuous kernel evaluated at each edge’s pseudo-coordinate.The kernel converts each spatial relation u(i, j) into a scalar aggregation weight.
- 3.3. Convolution operator: B-spline kernels use trainable control values whose tensor-product basis functions define continuous weights over the coordinate domain.The control values can be interpreted as heights of a B-spline surface sampled according to each neighbor’s pseudo-coordinate.
- 3.3. Convolution operator: Only s = (m + 1)^d basis products are nonzero for each edge, so evaluating a kernel depends on a small local subset rather than all K parameters.The active parameter indices can be found in constant time when spline degree m and coordinate dimensionality d are fixed.
- 3.3. Convolution operator: Closed B-spline approximation supports periodic coordinate dimensions by identifying boundary control values and reducing parameters and basis functions.For angular coordinates, this makes angle 0 and 2π receive the same weight and can ensure continuous differentiability for higher degrees.
- 3.3. Convolution operator: Except for normalization, the spline-based operator generalizes traditional CNN convolution for suitable grid graphs, while supporting arbitrary coordinate types and irregular structures.The paper also describes applying separate trainable parameters across input-output feature-map combinations in deep architectures.
4. GPGPU algorithm
The GPGPU implementation parallelizes SplineCNN computation over graph edges, gathering node features, evaluating active B-spline terms, and scatter-adding edge outputs back to nodes. Its parallel time complexity depends on the local support size rather than the total number of kernel weights.
- 4. GPGPU algorithm: The implementation provides efficient training and inference for the spline-based convolution operator, with a PyTorch implementation available on GitHub.Algorithm 1 specifies inputs including node counts, feature dimensions, active basis terms, weights, and node features.
- 4. GPGPU algorithm: The GPU algorithm gathers edge-wise input features from target nodes, computes edge-wise outputs, and scatter-adds them into node-wise output features.This realizes neighborhood aggregation through edge-parallel processing.
- 4. GPGPU algorithm: The algorithm has parallel time complexity O(s · M_in) using O(E · M_out) processors, assuming constant-time parallel scatter-add.Here s is the number of nonzero B-spline products per edge.
- 4. GPGPU algorithm: Matrices P and B store, respectively, the indices of active parameters and their B-spline basis products for each edge.These arrays can be preprocessed for a graph or computed directly inside the kernel.
- 4. GPGPU algorithm: Mini-batches are parallelized by forming sparse block-diagonal coordinate matrices and concatenating node and edge representations without redundant overhead for differing graph sizes.Examples in one batch may have different numbers of nodes and edges.
5. Results
SplineCNN is evaluated on image graphs, citation graphs, and 3D meshes, matching or improving prior results across these irregular-structure tasks. The experiments also examine coordinate choices, B-spline degrees, learned kernels, and network depth.
- Experimental scope: SplineCNN experiments cover image graph classification, graph node classification, and shape correspondence on meshes.The paper denotes spline-based layers as SConv(k, Min, Mout) and evaluates architectures across three task families.
- Image graph classification: 4.11 percentage points: SplineCNN improves previous MNIST superpixel classification results, while grid accuracy is approximately comparable to LeNet5 and MoNet.The superpixel comparison uses similar architecture and input data to MoNet.
- Image graph classification: Lower B-spline degrees and Cartesian pseudo-coordinates perform slightly better on MNIST superpixels, although configuration differences are small.Results for varying m and pseudo-coordinates show only small accuracy differences overall.
- Image graph classification: Learned first-layer kernels exhibit edge-detecting patterns on both regular MNIST grids and irregular superpixel graphs.The visualization uses 32 kernels with kernel size (5, 5) and B-spline degree m = 1.
- Graph node classification: 1.58 percentage points: SplineCNN improves the state of the art on Cora node classification averaged over 100 experiments.The authors associate the improvement with filtering based on normalized target-node degrees.
- Shape correspondence: 99.20%: SplineCNN predictions on FAUST have zero geodesic error, outperforming the other approaches on that measure without SHOT input descriptors.FMNet performs slightly better over larger geodesic error bounds, which the authors relate to its specialized soft error loss.
6. Conclusion
SplineCNN learns directly from irregular geometric data using a trainable spatial convolution operator, improving benchmark results while enabling fast training and inference. The authors identify recurrent neurons and un-pooling layers as future extensions.
- SplineCNN applies a trainable spatial convolution operator to irregular geometric data, including graphs and meshes.The filter aggregates local features through a continuous kernel parametrized by trainable B-spline control values.
- SplineCNN improves state-of-the-art results across image graph classification, graph node classification, and mesh shape correspondence.
- The architecture supports fast training and inference computation by avoiding preprocessing.
- Future work includes recurrent neurons for geometric spatio-temporal data or dynamic graphs and un-pooling layers for encoder-decoder or generative architectures.