Source-linked AI summary
Topological Deep Learning: Going Beyond Graph Data
Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K. Dey, Soham Mukherjee, Shreyas N. Samaga, Neal Livesay, Robin Walters, Paul Rosen, Michael T. Schaub
TL;DR
Graph-centered views can fall short of capturing non-local properties and dependencies in topological data. The paper introduces combinatorial complexes and CCNNs as a framework for higher-order learning, and reports competitive performance against task-specific state-of-the-art models. It also identifies scaling CCNNs further as an area for improvement.
Problem
Graph-centered geometric views can fall short of capturing non-local properties and dependencies in data.
Source: §1 · p. 5Method
The paper introduces combinatorial complexes, which combine set-type and hierarchical higher-order relations, and builds CCNNs with operators for higher-order learning.
Source: §1 · p. 6Results
CCNNs attain competitive predictive performance against task-specific state-of-the-art neural networks in applications including shape analysis and graph learning.
Source: §1 · p. 7Takeaways & Limitations
The framework unifies concepts in topological deep learning, with most deep-learning architectures reducible to CCNN instances based on operations defined on CCs.
Source: §11 · p. 59Further scaling CCNNs requires exploring sparsification, randomization, and other algorithmic improvements.
Source: §11 · p. 60
Abstract
from arXiv · showhide
Topological deep learning is a rapidly growing field that pertains to the development of deep learning models for data supported on topological domains such as simplicial complexes, cell complexes, and hypergraphs, which generalize many domains encountered in scientific computations. In this paper, we present a unifying deep learning framework built upon a richer data structure that includes widely adopted topological domains. Specifically, we first introduce combinatorial complexes, a novel type of topological domain. Combinatorial complexes can be seen as generalizations of graphs that maintain certain desirable properties. Similar to hypergraphs, combinatorial complexes impose no constraints on the set of relations. In addition, combinatorial complexes permit the construction of hierarchical higher-order relations, analogous to those found in simplicial and cell complexes. Thus, combinatorial complexes generalize and combine useful traits of both hypergraphs and cell complexes, which have emerged as two promising abstractions that facilitate the generalization of graph neural networks to topological spaces. Second, building upon combinatorial complexes and their rich combinatorial and algebraic structure, we develop a general class of message-passing combinatorial complex neural networks (CCNNs), focusing primarily on attention-based CCNNs. We characterize permutation and orientation equivariances of CCNNs, and discuss pooling and unpooling operations within CCNNs in detail. Third, we evaluate the performance of CCNNs on tasks related to mesh shape analysis and graph learning. Our experiments demonstrate that CCNNs have competitive performance as compared to state-of-the-art deep learning models specifically tailored to the same tasks. Our findings demonstrate the advantages of incorporating higher-order relations into deep learning models in different applications.
1 Introduction
The introduction argues for a shared foundation for topological deep learning and presents combinatorial complexes as a domain combining set-type relations with hierarchy, with CCNNs as a learning framework.
- The growing field of topological deep learning lacks a broad synthesis of foundational principles, making connections, comparisons, and entry points more difficult.Source: §1 · p. 6
- Combinatorial complexes combine arbitrary set-type relations with hierarchical higher-order relations, unifying traits of hypergraphs and cell complexes.They generalize graphs, simplicial and cell complexes, and hypergraphs as special cases.Source: §1 · p. 6
- CCNNs use a push-forward operator to support convolution, attention, pooling, and unpooling, and the paper characterizes their permutation and orientation equivariance.The push-forward operation moves data across dimensions and underlies higher-order message passing.Source: §1 · p. 6
- CCNNs attain competitive predictive performance against task-specific state-of-the-art neural networks in shape analysis and graph learning.The introduction reports evaluation in practical scenarios.Source: §1 · p. 7
- The paper connects the mapper construction to topological deep learning and demonstrates its use in higher-order pooling and unpooling on CCs.Source: §1 · p. 7
- Any CC can be reduced to a Hasse graph, enabling aspects of CCNNs and higher-order representation learning to be characterized through graph-based models.The reduction uses an enlarged computational graph.Source: §1 · p. 7
2 Motivation: what does topology offer to machine learning?
Topology offers ways to model data and relations beyond graphs, account for higher-order structure, and unify deep learning across structured domains. CCNNs generalize graph neural networks through topological constructions.
2 Motivation: what does topology offer to machine learning?
- CCNNs generalize graph neural networks and higher-order analogues through topological constructions.Source: §2 · p. 8
2.1 Modeling and learning from data on topological spaces
- Data and its supporting domain are distinct; higher-order domains can support features on entities such as mesh edges and faces, not only vertices.Examples include vector fields on meshes and class-labeled data on mesh edges and faces.Source: §2.1 · pp. 8–9
- Higher-order relations represent multi-party interactions that pairwise graphs cannot directly capture, including groups, co-authors, and interacting genes or proteins.Such relations can be modeled as hypergraphs, cell complexes, or combinatorial complexes.Source: §2.1 · p. 9
2.2 The utility of topology
- Hierarchical higher-order relations enable abstractions among relations, a capacity graph-based relational reasoning models are limited in building.Higher-order network models are presented as a promising way to address this need.Source: §2.2 · p. 10
- Adding higher-order cells to graphs supplies an inductive bias used to improve graph-based learning performance.Examples include adding cells to missing faces for molecular classification [46] and lifting graphs to one-hop neighborhood hypergraphs for vertex tasks [102].Source: §2.2 · p. 11, Figure 5 · p. 11
- Hierarchical pooling and unpooling can propagate vertex signals across distant dependencies connected by long edge paths.The cited approach uses these operations to capture distant dependencies and learn local and global structure.Source: §2.2 · p. 11, Figure 6 · p. 11
- Higher-order networks can capture dependencies that traditional graph-based models miss and may improve expressive power and performance.Source: §2.2 · p. 11
2.3 A unifying perspective on deep learning and structured computations
- A unifying topological framework can compare and design models across discrete domains, while CCs may also help investigate links between network topology and generalization.The relationship between topology, computational function, and generalization on higher-order domains remains an open question.Source: §2.3 · p. 12
3 Preliminaries: from graphs to higher-order networks
The section develops proximity from neighborhood choices on sets into a framework for higher-order relations. It contrasts existing higher-order domains and introduces hierarchies and set-type relations as features that combinatorial complexes combine.
3 Preliminaries: from graphs to higher-order networks
- Neighborhoods on a set have no inherent definition; auxiliary structures such as edges, distance, triangles, or trapezoids provide alternative notions of proximity.The application typically determines which neighborhood is appropriate.Source: §3 · p. 13
3.1 Neighborhood functions and topological spaces
- A neighborhood topology equips a set with a neighborhood function satisfying axioms, a choice that is compatible with graph message passing and generalizes to higher-order networks.Source: §3.1 · pp. 13–14
3.2 Bridging the gap among higher-order networks
- Higher-order relations let neighborhood functions capture multi-way interactions, while simplicial complexes, cell complexes, and hypergraphs differ in their relation constraints and structure.Simplicial complexes require faces of each relation, and their required relations can increase memory use or fail to match data such as traffic flows on rectangular street networks.Source: §3.2 · pp. 14–15
- Simplicial and cell complexes provide hierarchies of relations, whereas hypergraphs permit arbitrary set-type relations without directly encoding relation dimensions.Source: §3.2 · pp. 15–16
3.3 Hierarchical structure and set-type relations
- A rank function preserves set inclusion and induces a hierarchy; set-type relations are not implied by other relations in the domain.Source: §3.3 · p. 16
- Combinatorial complexes combine hierarchies of relations with arbitrary set-type relations, bridging features of cell complexes and hypergraphs.Source: §3.2 · pp. 14, 16, §3.3 · p. 16
4 Combinatorial complexes (CCs)
This section develops combinatorial complexes as domains combining hierarchical and unconstrained set-type relations, defining their structural properties, homomorphisms, and rank-based adjacency and coadjacency. It also introduces cochains and cochain maps for higher-order message passing and pooling or unpooling.
4.1 CC definition
- CCs combine hierarchical and unconstrained set-type relations, viewing them as relaxed simplicial or cell complexes and as ranked hypergraphs.Their rank function organizes relations into a hierarchy, while the definition imposes no constraints among relations.Source: §4.1 · p. 17
4.2 CC-homomorphisms and sub-CCs
- CC-homomorphisms relate CCs and support lifting graphs or other domains into CCs, while sub-CCs include skeletons and complexes induced by vertex subsets.CC-embeddings can augment a domain with higher-order cells while preserving cell ranks.Source: §4.2 · pp. 18–19
4.3 Motivation for CCs
- The section motivates CCs through pooling operations and their structural advantages for higher-order modeling.These motivations include expressing graph pooling and representing relations among relations.Source: §4.3 · p. 19
4.3.1 Pooling operations on CCs
- CCs express graph pooling by representing super-vertices as higher-ranked cells, enabling recursive coarsening and a general pooling framework that includes graphs and images.Set-type cells allow flexible, task-specific cluster shapes, while higher-ranked cells represent coarser versions of the underlying space—capabilities not jointly accommodated by hypergraphs or cell complexes.Source: §4.3.1 · pp. 19–20
4.3.2 Structural advantages of CCs
- CC rank functions support flexible higher-order structures and fine-grained message passing; CCs also permit arbitrary relations while retaining hierarchy for modeling relations among relations.Hypergraphs allow arbitrary set-type relations but lack hierarchy, while simplicial and cell complexes constrain which relations can be represented.Source: §4.3.2 · pp. 20–21
4.4 Neighborhood functions on CCs
- CC-neighborhood functions describe local relations around sub-CCs and provide a foundation for generalized message passing that subsumes several GNNs [104] [188] [28] [160].The section focuses on adjacency and incidence neighborhoods and represents neighborhood functions computationally with matrices.Source: §4.4 · pp. 21–22
4.4.1 Incidence in a CC
- Incidence neighborhoods relate cells of different ranks, with definitions for arbitrary-rank and rank-specific incidence and matrices for cells of chosen ranks.Incidence matrices induce neighborhood functions used to construct higher-order message passing.Source: §4.4.1 · p. 22
4.4.2 Adjacency in a CC
- CC adjacency and coadjacency relate cells of the same rank through a shared higher-rank or lower-rank bridge cell, respectively.These neighborhood functions extend incidence-style descriptions to relations among cells of similar rank.Source: §4.4.2 · p. 23
- CCs define k-adjacency through shared higher-rank cells and k-coadjacency through shared lower-rank cells; binary matrices encode both relations among same-rank cells.Source: §4.4.2 · pp. 23–24
4.5 Data on CCs: cochain spaces and maps
- A k-cochain is a function assigning d-dimensional data to each rank-k cell, so cochains represent signals on cells of a CC.Source: §4.5 · p. 24
- Incidence matrices map cochains across ranks, while adjacency and coadjacency matrices map between cochain spaces of equal dimension, supporting higher-order message passing and pooling or unpooling.Source: §4.5 · p. 24
5 Combinatorial complex neural networks (CCNNs)
This section presents CCNNs as a tensor-based framework for topological learning on combinatorial complexes, supporting convolutional and attention-based architectures. It describes how CC-attention learns weighted neighborhood operators and supports push-forwards across cells of equal or unequal ranks, including incidence relations.
5 Combinatorial complex neural networks (CCNNs)
- Topological learning tasks include cell classification, complex classification, and cell prediction, covering per-cell labels, whole-complex labels, and cell-interaction properties or existence.Examples include mesh segmentation, mesh class prediction, and predicting linkages among entities in hyperedges.Source: §5 · pp. 25–26
- CCNNs form an abstract template for neural networks on combinatorial complexes, generalizing convolutional and attention architectures and supporting results applicable across CCNN instances.The framework selects neighborhood functions for a learning task and uses them to build a topological neural network.Source: §5 · pp. 25–26
5.1 Building CCNNs: tensor diagrams
- Tensor diagrams represent CCNNs as directed graphs whose labeled cochain-map edges specify signal flow, with map choices depending on the learning task.Higher-order networks can involve many interacting subnetworks because they entail more signals than vertex- or edge-based graph models.Source: §5.1 · pp. 26–27
5.2 Push-forward operator and merge node
- The push-forward sends an i-cochain to j-cells by aggregating neighboring features and applying a differentiable function; it underlies merge nodes, message passing, and pooling or unpooling.The aggregation function is permutation-invariant.Source: §5.2 · p. 28
5.3 Three tensor operations for the construction of CCNNs
- Push-forwards, merge nodes, and split nodes are elementary operations; compositions and concatenations of them can define arbitrary parameterized topological neural-network architectures.A split node is determined by push-forwards, so parameterizing push-forwards and merge operators suffices to define a parameterized CCNN class.Source: §5.3 · pp. 30–31
5.4 Combinatorial complex convolutional networks (CCCNNs)
- CCCNNs equip CCNNs with CC-convolutional push-forward operators and convolutional merge nodes; the push-forward generalizes a convolutional graph neural network [160].The convolutional merge node is built from the abstract merge operation.Source: §5.4 · p. 31
5.5 Combinatorial complex attention neural networks (CCANNs)
- CCANNs equip CCNNs with attention-based push-forwards, extending attention to assign weights to neighboring sub-complexes around a target sub-complex.Higher-order attention weights are defined over the neighbors specified by a neighborhood function.Source: §5.5 · p. 32
- CC-attention push-forwards support cochains on cells of equal rank and attention blocks connecting cells of unequal ranks.The unequal-rank formulation applies to neighborhood matrices such as incidence matrices.Source: §5.5 · pp. 32–33
- For incidence neighborhoods, CC-attention learns distributions from each 2-cell to its incident 1-cells and from 1-cells to 2-cells.The learned incidence-attention matrices have the same shapes and nonzero locations as the incidence matrix and its transpose.Source: §5.5 · p. 33, Figure 24 · p. 33
- At inference, a CC-attention push-forward requires only the source cochain, while the target cochain is needed to compute attention during training.The target cochain influences the push-forward indirectly through the learned attention weights.Source: §5.5 · p. 33
- CCANNs can learn arbitrary types of discrete exterior calculus operators by using attention-weighted versions of neighborhood operators.The operators G ⊙ att and G ⊙ att_s→t are learned attention versions of G.Source: §5.5 · p. 33
6 Higher-order message passing
The section formalizes higher-order message passing on combinatorial complexes and shows how it relates to CCNN operations. It also compares merge nodes with message passing and introduces an attention-based version.
6 Higher-order message passing
- The section frames higher-order message passing as a generalization of notions introduced in and relates it to CCANNs and tensor operations.It establishes the connection between merge nodes and higher-order message passing, including an attention version.Source: §6 · p. 34, §6.4 · p. 38
6.1 Definition of higher-order message passing
- The framework exchanges messages between cells using selected neighborhood functions, then aggregates messages within and across neighborhoods to update cell features.The message may depend on the cells themselves as well as their data, and the framework uses two aggregation functions rather than the single aggregation function common in graph-based message passing.Source: §6.1 · pp. 34–35
6.2 Higher-order message-passing neural networks are CCNNs
- Higher-order message-passing computations can be realized by merge nodes, making higher-order message-passing networks special cases of CCNNs.This unifies message passing on simplicial complexes, cell complexes, and hypergraphs within the CCNN framework.Source: §6.2 · pp. 35–36
6.3 Merge nodes and higher-order message passing: a qualitative comparison
- Merge nodes can compute cochains on selected ranks from available input cochains, whereas message passing assumes inputs and updates across all ranks at every iteration.The section notes that message-passing difficulties can be addressed with ad hoc engineering, while merge nodes are designed to avoid these constraints.Source: §6.3 · p. 37
6.4 Attention higher-order message passing and CCANNs
- Attention higher-order message passing weights both individual neighbors and the information contributed by each neighborhood function.The neighborhood-level weights are specific to higher-order domains and do not arise in graph-based attention models.Source: §6.4 · p. 38
7 Push-forward and higher-order (un)pooling
This section builds CC pooling and unpooling from push-forward operations, connecting them to common graph and image pooling. It also describes pooling CCNNs and a mapper-based way to represent graph shape for pooling.
7 Push-forward and higher-order (un)pooling
- Push-forward operations provide a shared building block for higher-order message passing and pooling or unpooling operations.The section describes this as a foundation for a unifying API across topological domains.Source: §7 · p. 38
7.1 CC-(un)pooling
- CC-pooling pushes a cochain from lower-rank to higher-rank cells, while CC-unpooling pushes it to lower-rank cells.The pooling operation uses the hierarchical structure of a CC and can be applied successively to coarsen signals.Source: §7.1 · p. 39, Figure 27 · p. 39
7.2 Formulating common pooling operations as CC-pooling
- CC-based pooling provides a common formulation for graph and image pooling through push-forward operations.The section presents these common pooling operations as instances of CC-pooling.Source: §7.2 · p. 39, §7.2.1 · p. 40, §7.2.2 · p. 40
7.2.1 Graph pooling as CC-pooling
- Graph pooling can be realized as CC-pooling by augmenting the graph with 2-cells that represent clusters of original vertices.The resulting 2-cochain represents the coarsened vertex signal.Source: §7.2.1 · p. 40
7.2.2 Image pooling as CC-pooling
- Image pooling can be realized as CC-pooling by augmenting the image lattice graph with 2-cells corresponding to pooling windows.The illustrated setup uses a 2 × 2 window and stride one.Source: §7.2.2 · p. 40, Figure 28 · p. 40
7.3 (Un)pooling CCNNs
- A pooling CCNN pushes its lowest-rank signals to higher-rank cells, and a readout operation can be realized as a pooling CCNN.The section defines pooling and unpooling CCNNs from layers that pool or preserve the lowest rank, with unpooling reversing the rank direction.Source: §7.3 · pp. 41–42
7.4 Mapper and the CC-pooling operation
- On a graph, CC-pooling may only move a node signal to an edge signal, which may not provide a compact summary.The section proposes augmenting the graph with higher-dimensional cells that approximate its shape; the resulting cells summarize graph shape features.Source: §7.4 · p. 42
8 Hasse graph interpretation and equivariances of CCNNs
This section represents combinatorial complexes with Hasse graphs, expresses CCNN computations on augmented Hasse graphs, and examines CCNN permutation and orientation equivariances in relation to conventional GNN equivariance. It characterizes these equivariances through merge nodes and cautions that standard GNNs on augmented Hasse graphs do not generally preserve CCNN structure.
8.1.1 CCs as Hasse graphs
- Every CC has a unique Hasse graph whose vertices are cells and whose edges encode immediate incidence, enabling CC structure to be characterized by incidence, adjacency, or coadjacency matrices.The Hasse graph supports interpreting higher-order signals and message passing as graph-based computations.Source: §8.1 · p. 43, §8.1.1 · p. 44
8.1.2 Augmented Hasse graphs
- Augmented Hasse graphs add edges induced by neighborhood functions, allowing message-passing computations that are not represented by the Hasse graph’s immediate-incidence edges.The added edges correspond to nonzero entries in matrices associated with the neighborhood functions.Source: §8.1.2 · p. 45
8.1.3 Reducibility of CCNNs to graph-based models
- Any computation in a tensor-diagram CCNN can be reduced to a computation on the underlying CC’s augmented Hasse graph, but this does not make a standard GNN on that graph equivalent to a CCNN.The reduction encompasses push-forward and merge-node operations; the augmented Hasse graph provides an alternative representation of the tensor diagram’s computations.Source: §8.1.3 · p. 46, §8.1.6 · p. 48
8.1.4 Augmented Hasse graphs and CC-pooling
- CC-pooling pushes signals from lower-rank cells to higher-rank cells, while CC-unpooling maps them in the opposite direction.These operations can be viewed on the poset structure represented by the augmented Hasse graph.Source: §8.1.4 · p. 46
8.1.5 Augmented Hasse diagrams, message passing, and merge nodes
- Higher-order message passing updates cochains across all dimensions with a uniform topological structure, whereas tensor operations allow a more flexible augmented Hasse graph structure.The flexibility of tensor operations also applies to the network input.Source: §8.1.5 · pp. 46–47
8.1.6 Higher-order representation learning
- Higher-order representation learning can be reduced to graph representation learning, and TopoEmbedX uses this principle to compute embeddings for cell, simplicial, and combinatorial complexes.TopoEmbedX converts a higher-order domain into a subgraph of its augmented Hasse graph and applies existing graph representation-learning algorithms.Source: §8.1.6 · p. 48
8.2 On the equivariance of CCNNs
- The section introduces permutation and orientation equivariance definitions for CCNNs and relates them to conventional GNN equivariance notions.It presents these properties as considerations for higher-order deep-learning models, including CCNNs.Source: §8 · p. 43, §8.2 · p. 48
8.2.1 Permutation equivariance of CCNNs
- CCNN permutation equivariance is defined using rank-preserving cell permutations acting on cochain maps and corresponding transformations of network inputs and outputs.The definition generalizes the conventional permutation-equivariance notion for GNNs.Source: §8.2.1 · pp. 48–49
- A CCNN is permutation equivariant if and only if every merge node in it is permutation equivariant.This characterization applies across network depths: deeper CCNNs compose height-one CCNNs.Source: §8.2.1 · p. 50
- Standard GNNs on augmented Hasse graphs are not generally equivalent to CCNNs because their weight-sharing and permutation-equivariance structures differ.To preserve CC structure, consider only augmented-Hasse-graph vertex permutations induced by permutations of the CC's 0-cells.Source: §8.2.1 · p. 50
8.2.2 Orientation equivariance of CCNNs
- For CCs reduced to regular cell complexes, a CCNN is orientation equivariant if and only if every merge node is orientation equivariant.The orientation action uses diagonal sign matrices on cells in each dimension, with D0 = I.Source: §8.2.2 · pp. 50–51
9 Implementation and numerical experiments
The experiments evaluate CCNNs on mesh segmentation and classification, point-cloud classification, and graph classification, using varied higher-order architectures and pooling strategies. Results are competitive with task-specific mesh models and GNN baselines, while ablations highlight the performance relevance of pooling and higher-order structures.
9 Implementation and numerical experiments
- The evaluation spans four datasets and mesh, point-cloud, and graph tasks; the authors implement topological structures, embeddings, and models in three Python packages.The packages are TopoNetX, TopoEmbedX, and TopoModelX.Source: §9.1 · p. 51, §9 · p. 51, §9.2 · p. 52
9.3.1 Mesh segmentation
- CCNNs outperform task-tailored mesh networks on two of four segmentation datasets and rank among the top two on all four.The comparisons are against HodgeNet, PD-MeshNet, and MeshCCN.Source: §9.3.1 · p. 53, Table 2 · p. 53
9.3.2 Mesh and point cloud classification
- 99.17% accuracy with tanh and 96.67% with identity activations; CCNNs outperform HodgeNet and MeshCCN and rank second behind PD-MeshNet.The mesh-classification model trains for 40 epochs, compared with 600 for the segmentation CCNNs.Source: §9.3.2 · p. 54, Table 3 · p. 54
9.3.3 Graph classification
- CCNNs outperform six GNNs on the hard graph dataset and five on the easy dataset, matching MinCutPool on easy and exceeding it on hard.Table 4 reports the graph-classification comparison.Source: Table 4 · p. 54, §9.3.3 · p. 55
9.4.1 Mesh classification: CC-pooling with input vertex and edge features
- 98.1% accuracy with CCNNMOG1, below CCNNSHREC's 99.17%, using 10 rather than 30 mesh-augmentation replications.CCNNMOG1 uses vertex and edge features and MOG-generated 3-cells.Source: §9.3.2 · p. 54, §9.4.1 · p. 56
9.4.3 Point cloud classification: CC-pooling with input vertex features only
- CCNNMOG2 reaches 95.2% on point-cloud classification using sampled SHREC11 surfaces, k = 7 neighborhood graphs, and MOG-generated 2-cells.The model uses vertex features and rotation augmentation.Source: §9.4.3 · p. 56
9.5 Ablation studies
- 97.1% with MOG-based CCNNMOG2 versus 89.2% with a neighborhood-based pooling alternative suggests pooling choice materially affects predictive performance.Both results use the SHREC11 classification dataset.Source: §9.5 · p. 57
- Replacing CCNNMOG2 pooling with a graph cochain operator yields 84.56% accuracy, below the 97.1% CCNNMOG2 result.The authors attribute CCNN advantages to higher-order input features or pooling that augments higher-order cells.Source: §9.5 · p. 57
10 Related work
The related work situates topological deep learning among graph, higher-order, attention, pooling, and applied-topology approaches. It identifies limits in pairwise graph relations and existing higher-order attention and pooling, while presenting CCs and CCNNs as a unifying framework.
10.1 Graph-based models
- Graph and multiplex models represent pairwise relations, motivating higher-order networks for multi-way interactions.The section surveys graph applications and notes that multiplex layers still contain vertices and edges.Source: §10.1 · pp. 57–58
10.2 Higher-order deep learning models
- Higher-order learning spans Hodge-based, convolutional, and message-passing methods across simplicial complexes, hypergraphs, and cell complexes.Prior work also addresses hierarchical coupling and applications including link prediction, classification, and trajectory prediction.Source: §10.2 · p. 58
10.3 Attention-based models
- Most surveyed attention mechanisms target graphs, and existing higher-order attention models do not combine blocks across entity dimensions.The stated limitation constrains the architecture space and application scope.Source: §10.3 · pp. 58–59
10.4 Graph-based pooling
- Graph pooling ranges from clustering and coarsening to learned spectral, top-K, and hierarchical methods, while higher-order pooling has received little study.The section identifies a general simplicial-complex pooling strategy as an exception.Source: §10.4 · p. 59
10.5 Applied algebraic topology
- Applied algebraic topology includes persistent-homology applications and TDA methods for machine learning, including persistence-diagram vectorizations.Examples of vectorizations include Betti curves, persistence landscapes, and persistence images.Source: §10.5 · p. 59
- The paper introduces CCs and CCNNs as a framework that formalizes deep learning in topological terms and incorporates constructions such as mapper.The framework draws on algebraic topology, topological quantum field theory, and TDA.Source: §10.5 · p. 59
11 Conclusions
The conclusions present CCs and CCNNs as a unifying framework for learning on topological domains and relating existing architectures. They also identify computational and methodological limitations and outline extensions for future work.
11 Conclusions
- CCs provide a topological domain for learning representations, and many existing deep-learning architectures can be reduced to CCNN instances.The framework is presented as a platform for systematic exploration and comparison of topological learning protocols.Source: §11 · pp. 59–60
- Known limitations include computational complexity, architecture selection, interpretability, and limited availability of topological datasets.Ad hoc conversions of existing data to add higher-order relations may not always be ideal.Source: §11 · p. 60
- Further scaling may require sparsification, randomization, or other algorithmic improvements; directed, weighted, multilayer, and time-varying domains remain future directions.The paper also calls for further study of domain selection and CCNN theory.Source: §11 · p. 60
- Topological models can encode properties invariant to discretization, including the Euler characteristic, and homeomorphism-related invariants may matter when discretization is an artifact.The conclusions contrast this setting with graph isomorphism and point to mesh-similarity applications.Source: §11 · p. 60
A Glossary
The glossary tabulates the paper’s notation and acronyms. Its entries define matrix notation, feature spaces, cochain maps, attention, and related terms.
A Glossary
- B_{r,k} denotes incidence matrices between r-cells and k-cells; A_{r,k} and coA_{r,k} denote adjacency and coadjacency matrices among cells of X^r with respect to cells of X^k.The table also lists these matrix notations under separate matrix and coadjacency entries.Source: A Glossary · p. 74
- C_k(X, R^d) is the k-cochain space with features in R^d, while C_k denotes a k-cochain space with features in a Euclidean space.Source: A Glossary · p. 74
- The glossary defines G as a set of cochain maps, G as a cochain map, a higher-order attention matrix and function, and CCNN as a CCNN or its tensor-diagram representation.It also lists W as trainable parameters and defines the Hasse graph notation.Source: A Glossary · p. 74
B Lifting maps
Lifting maps featured domains to other featured domains, enabling CCNNs to be applied to graphs and cell or simplicial complexes. The section describes n-hop and path-based constructions for graphs.
B Lifting maps
- Lifting maps a featured domain to another featured domain by a well-defined procedure, allowing CCNNs to apply to graphs and cell or simplicial complexes.The section notes that its examples only scratch the surface of possible lifting constructions.Source: B Lifting maps · p. 75
B.1 n-hop CC of a graph
- An n-hop CC uses graph nodes, edges, and sets of nodes in n-hop neighborhoods as its 0-cells, 1-cells, and n-cells, respectively.For n ≥ 2, this construction has dimension n.Source: §B.1 · p. 75
B.2 Path-based and subgraph-based CC of a graph
- A path-based CC uses graph nodes and edges as 0-cells and 1-cells, and sets of path vertices as 2-cells for paths with at least two edges.A tree or subgraph can replace the path to induce a similar CC structure.Source: §B.2 · p. 75
B.3 Loop-based CC of a graph
The section constructs loop-based CCs by adding 2-cells associated with graph loops, and coface CCs by augmenting a 2-dimensional simplicial complex with 3-cells. The coface construction generalizes to augmenting an n-dimensional CC with (n + 1)-cells.
B.3 Loop-based CC of a graph
- A loop-based CC keeps the graph nodes and edges as 0-cells and 1-cells, and adds 2-cells formed from the nodes of a loop.The loop is a closed path with no repeating edges; every 2-dimensional regular cell complex can be constructed in this manner.Source: §B.3 · pp. 75–76, Figure 37 · p. 76
B.4 Coface CC of a simplicial complex/CC
- A coface CC of a 2-dimensional simplicial complex adds a 3-cell for each triangle, comprising that triangle and the 0-cells in its 1-coface.The construction yields a dimension-three CC and can be generalized to augment any n-dimensional CC with (n + 1)-cells.Source: §B.4 · p. 76, Figure 37 · p. 76
B.5 Augmentation of CCs by higher-rank cells
A general lifting construction augments a CC with new cells of rank one greater than its dimension. The proposition establishes that this yields a CC of the next dimension.
B.5 Augmentation of CCs by higher-rank cells
- Adding new cells of rank dim(X) + 1 captures the shared structure of the lifting methods in Sections B.2, B.3, and B.4.Source: §B.5 · p. 76
- If each added cell strictly contains any existing cell it contains, assigning the added cells rank n + 1 produces a CC of dimension n + 1.The original ranks are retained, and the construction can be iterated to build CCs of arbitrary dimension from a nonempty set of abstract points.Source: §B.5 · pp. 76–77
C CCNN architecture search and topological quantum field theories
The section recasts CCNN architecture search as a search over marked trivalent graphs, using a TQFT-inspired functor to connect those graphs to tensor diagrams. This gives a graph-based route to CCNN architectures represented by tensor diagrams.
C CCNN architecture search and topological quantum field theories
- Searching marked trivalent graphs and mapping the result to a CCNN offers an alternative to computationally expensive architecture search.The search space is bounded by fixed Cartesian products of cochain spaces.Source: C CCNN architecture search and… · p. 77
- TQFT constructions break topological spaces into simpler subspaces and use maps between them to build maps between the original spaces.The passage highlights knot theory and three-dimensional manifolds as applications where map topology is central.Source: C CCNN architecture search and… · p. 77
- Marked trivalent graphs connect bottom and top marked points inside a disk, with exactly three edges meeting at each vertex; composition uses vertical concatenation.Such graphs can be built by horizontal and vertical concatenation of elementary trivalent graphs.Source: C CCNN architecture search and… · pp. 77–78
- TQFT assigns vector spaces to manifolds and linear maps to cobordisms, extending assignments from elementary morphisms while preserving their structure.Source: C CCNN architecture search and… · p. 78
- The functor Ftri labels marked trivalent graphs with cochain maps, yielding tensor diagrams and an equivalent search space because CCNNs are represented by tensor diagrams.Source: C CCNN architecture search and… · p. 78
D Learning discrete exterior calculus operators with CCANNs
The attention framework treats classical discrete exterior calculus operators as special cases and learns task-adapted operators on a domain. Its stated scope includes domains such as triangular and polygonal meshes.
D Learning discrete exterior calculus operators with CCANNs
- Gtr retains G's shape while acting as its learned version, so an attention-based k-Hodge Laplacian can adapt to the domain and learning task.Source: D Learning discrete exterior calculus… · p. 78
- DEC operators act on cochains, and transposed signed incidence matrices represent discrete exterior derivatives analogous to gradient, curl, and divergence.Source: D Learning discrete exterior calculus… · p. 78
- The attention framework is a nonlinear version of DEC in which a classical linear operator A is a special case, and it extends operator learning to domains such as triangular and polygonal meshes.The passage contrasts this scope with existing methods limited to particular DEC operators.Source: D Learning discrete exterior calculus… · p. 78
E A mapper-induced topology-preserving CC-pooling operation
The MOG construction coarsens a graph while preserving its shape and uses connected components to augment it with higher-rank cells. The resulting CC supports pooling vertex signals onto 2-cells, whose adjacency corresponds to the coarsened graph.
E A mapper-induced topology-preserving CC-pooling operation
- The MOG algorithm uses a scalar function and a cover of [0, 1] to form pullback sets, takes their connected components as vertices, and connects components that intersect.Source: E A mapper-induced topology-preserving… · p. 79
- The connected components augment the input graph with 2-cells, producing the CC Xg,U.Source: E A mapper-induced topology-preserving… · p. 79, Figure 40 · p. 80
- MOG coarsens an input graph while preserving its shape and topological features, providing a skeleton for feature-preserving pooling.The section demonstrates this construction on graphs and says it can be extended to CCs.Source: E A mapper-induced topology-preserving… · pp. 79–80, Figure 41 · p. 80
- The MOG graph adjacency matrix is equivalent to the CC's 2-cell adjacency matrix, A2,2.Figure 42 illustrates the correspondence: intersecting 2-cells map to connected vertices in the MOG graph.Source: E A mapper-induced topology-preserving… · p. 80, Figure 42 · p. 81
- The map B^T_0,2 pushes a vertex-supported signal H0 forward to a signal H2 supported on the CC's 2-cells, defining a shape-preserving pooling operation.Source: E A mapper-induced topology-preserving… · p. 81
Related papers
This paper cites
- Attention Is All You Need2017
- Very Deep Convolutional Networks for Large-Scale Image Recognition2014
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Neural Machine Translation by Jointly Learning to Align and Translate2014
- Semi-Supervised Classification with Graph Convolutional Networks2016
- Sequence to Sequence Learning with Neural Networks2014
Cited by
- Beyond Homology: Non-Abelian and Orientability Invariants Sensitivity in Cellular Networks2026
- Batch Before You Lift: Scalable Topological Deep Learning on Large Graphs2026
- Higher-Order Morphology Priors for Quadruped Reinforcement Learning Under Actuator Degradation2026
- Mathematical Invariant-Enabled Topological Neural Networks for Molecular and Materials Property Prediction2026
- From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning2026
- Look Before You Lift: Visual and Quantitative Diagnostics for Topological Deep Learning2026