Position: Topological Deep Learning is the New Frontier for Relational Learning

1. Introduction

• The paper focuses on TDL methods for understanding existing models and designing architectures, and identifies open problems and future research opportunities.It asks what conditions would make TDL a go-to tool for practitioners and outlines research directions addressing that question.Source: §1 · pp. 2–3
• Topological domains can represent multi-way interactions, supporting higher-order message-passing schemes and varied relational structures.The paper names simplicial and cellular complexes as domains for modeling different types of multi-way interactions.Source: §1 · p. 3
• Topological views may capture manifold regularities such as remeshing symmetry, which can be challenging to express with purely combinatorial graph constructs.Examples include invariance to different triangulations of a sphere and similar behavior across meshing resolutions.Source: §1 · p. 3
• TDL can capture topological equivariances through complex homeomorphism groups while preserving some embedding information.The paper gives knot classification and the arrangement of critical points in Morse-Smale complexes as examples.Source: §1 · p. 3
• The paper presents TDL as a relational-learning frontier and describes its broader roles in architecture choice, manifold regularities, and topological equivariances.It also states that relational data encode features of binary or higher-order interconnections.Source: §1 · p. 3
• TDL provides a framework for learning from relational data whose objects are connected by binary or higher-order relations.Traditional machine learning often assumes data lie in a linear vector space and can be represented by feature vectors, an assumption that may not fit these examples.Source: §1 · pp. 1, 3

2.1. Examples

• Attributed graphs combine structural and feature information, with applications spanning biological systems, drug design, molecules, and materials.Reported examples include TDL work on D3R Grand Challenges and SARS-CoV-2 evolution and variant forecasting.Source: §2.1 · pp. 3–4
• Despite compelling applications and competitive empirical results, broader real-world adaptation of TDL has not yet taken place.Source: §2.1 · p. 4
• Research directions include developing TDL applications across scientific disciplines to produce impactful real-world uses.Source: §2.1 · p. 4

2.2. Datasets

• Higher-order datasets are scarce; they may come from applications or synthetic data generated by lifting graphs and generalizing rewiring algorithms.Source: §2.2 · p. 4

2.3. Benchmarks

• Benchmark suites are needed for efficient, objective evaluation and should combine higher-order datasets, lifting algorithms, and predictive and stability metrics.The section also recommends open, reproducible suites and a taxonomy organizing datasets by size and learning task.Source: §2.3 · p. 4

• Higher-order software includes HyperNetX, XGI, DHG, and TopoX, while other packages support persistent-homology computations and topology-informed learning.Source: §3 · pp. 4–5
• TopoX provides computation, embedding, and model packages for learning with multiple higher-order domains and topological neural networks.Source: §3 · p. 5
• Despite theoretical advances, scarce easy-to-use software limits TDL experimentation and deployment, and available tools need broader domain and algorithm coverage.Source: §3 · p. 5
• More human and financial resources are needed for software development; demonstrating TDL's value may help attract investment.The section links compelling applications with conditions for addressing software-development resources.Source: §3 · p. 5
• Hardware friendliness and computational trade-offs matter because users may choose transformers with topological encodings over harder-to-implement TDL methods.Graph rewiring and positional encoding are noted as possible routes toward more hardware-friendly implementations.Source: §3 · p. 5

4.1. Complexity

• TDL's added higher-order structure increases space and time complexity relative to GNNs, motivating a cost-benefit framework for choosing among TDL, GDL, and GRL.Source: §4.1 · p. 5
• Complexity could be studied through performance-cost trade-offs or by tracking topological information as it propagates through network layers.Prior work cited in the section examines transformations in data topology, decision-boundary topology, and persistent homology across layers.Source: §4.1 · pp. 5–6

4.2. Scalability

• Scalable TDL models that work across multiple higher-order domains remain an open problem; a model for simplicial complexes may not apply to cell complexes, hypergraphs, or path complexes.Source: §4.2 · p. 6
• Proposed scalability directions include using fewer data points, faster-converging optimization, subsampling, and extending distributed GNN training to TDL.Source: §4.2 · p. 6

5.1. Explainability and Generalization

• Topology's potential contributions to understanding deep learning include explaining network weights, decisions, and training mechanisms and characterizing generalization error.Source: §5.1 · p. 6
• Research directions link generalization to data, training-trajectory, weight, and representation topology, including training TDL on computational graphs or controlling topological properties.The section notes prior reported relationships between these topologies and generalization error.Source: §5.1 · p. 6

5.2. Fairness

• TDL fairness remains inadequately studied, with open issues including valid fairness metrics and graph sampling or rewiring algorithms.Source: §5.2 · p. 6
• Proposed fairness directions generalize graph measures such as homophily, assortative mixing, disparate impact, and statistical parity to higher-order structures.Source: §5.2 · p. 7

6.1. Advantages of Topological Deep Learning

• Theory has not yet consolidated TDL’s relative advantages, including when higher-order domains offer richer representations or improve downstream performance.The section calls for evidence about when higher-order relations are useful and whether they are semantically meaningful.Source: §6.1 · p. 7
• Generative TDL remains open: current methods lack a unifying representation across 3D forms, while assigning meaning to generated higher-order structure remains a challenge.The section also identifies topological diffusion and flow-matching models as unexplored directions.Source: §6.1 · pp. 7–8
• Expressivity proofs and studies of oversmoothing, oversquashing, Hodge Laplacian spectra, and priors could clarify TDL’s properties and its comparison with GNNs.The proposed questions include which topological invariants neural networks can learn and whether higher-order priors improve reliability or uncertainty quantification.Source: §6.1 · pp. 7–8

6.2. Topological Representation Learning

• TRL concerns learning and exploiting multiscale topological descriptions of data and internal embeddings, with efficiency and transparency benefits compared with graph-based representations.Open questions include automating semantically meaningful representations and choosing properties to promote or penalize, especially for complex, high-dimensional inputs.Source: §6.2 · pp. 8–9
• Higher-order cells or hyperedges, latent topological inference, and architecture search are proposed as routes toward effective TRL.Research also considers shaping representations using prior or input-domain properties, generalization aims, and comparisons between neural representations.Source: §6.2 · pp. 8–9

6.3. Transformers in Topological Deep Learning

• A TDL transformer architecture could unify models across higher-order domains, with domain-specific designs such as diffusion-inspired hypergraph or simplicial transformers as a first milestone.The section also raises whether such architectures could apply to text, audio, or imaging data.Source: §6.3 · p. 9

7. Final Remarks

• Moving TDL from theoretical constructs toward real-world applications calls for collaboration across mathematics, computer science, and machine learning.The remarks frame innovation, collaboration, and open science as foundations for this multidisciplinary approach.Source: §7 · p. 9
• TDL workshops and conferences are proposed as platforms for knowledge sharing, discussion, and collaborative ventures, with open science principles.Source: §7 · p. 9
• The paper aims to guide collaborative research by highlighting TDL challenges and outlining research strategies.It presents openness and transparency as ways to support participation across fields.Source: §7 · p. 9

A. Literature Review

• The review’s first theme is higher-order deep learning models.Source: A. Literature Review · p. 22
• A second theme concerns research connecting the topology of data with neural networks.Source: A. Literature Review · p. 22
• A third theme covers research linking topological data analysis with neural networks.Source: A. Literature Review · p. 22

A.1. Higher-Order Deep Learning Models

• Higher-order models use Hodge-theoretic methods, message-passing schemes, and skip connections.The review also describes Hodge Laplacians for complex data analysis and edge-centric convolution using the 1-Hodge Laplacian.Source: §A.1 · p. 22
• Convolutional operators and message-passing approaches have been developed for hypergraphs and topological spaces, including unified graph and hypergraph learning frameworks.Source: §A.1 · p. 22
• Some higher-order models avoid traditional message passing, including neural k-forms and an MLP-based simplicial network.The MLP-based approach is described as offering fast inference and robustness to missing connectivity information at inference.Source: §A.1 · p. 22
• Training-related approaches include skip connections generalized to simplicial complexes and higher-order graph structures represented at multiple scales.Source: §A.1 · p. 22
• Attention mechanisms have been extended from graphs to higher-order domains, including simplicial, cell, combinatorial, and hypergraph structures.The described methods use mechanisms such as Hodge decomposition, masked self-attention, and interactions across simplex orders.Source: §A.1 · pp. 22–23
• Other reviewed approaches include tangent neural networks on Riemann manifolds and task-tailored extraction of latent topology information.Source: §A.1 · p. 23

A.2. Topology of Data and Neural Networks

• Olah (2014) used topological experiments to illustrate that underlying data topology matters when choosing a neural network.Source: §A.2 · p. 23
• Persistent homology has been used to study neural-network activations as point clouds and to analyze individual MLP layers.Naitzat et al. (2020) studied activations in binary-classification networks; Rieck et al. (2019b) developed neural persistence to describe deep-learning phenomena.Source: §A.2 · p. 23
• Bianchini & Scarselli (2014), later extended by Sun et al. (2016), studied the expressivity of deep neural networks.Source: §A.2 · p. 23
• Caveats to neural persistence were observed and rectified, leaving open how topology can effectively describe neural-network training.Hajij et al. (2021a) used topological notions to provide insights into supervised classification, while Oballe et al. (2022a) and Love et al. (2023) used data topology to infer architectures.Source: §A.2 · p. 23

A.3. Topological Data Analysis and Neural Networks

• Persistent homology has been applied to complex data problems, and topology-rooted deep-learning work spans domains including neuroscience, bioscience, forecasting, Trojan detection, segmentation, and 3D reconstruction.The section also notes topological data signatures and time-varying setups.Source: §A.3 · p. 23
• Network analyses have used persistent homology for observational insights, topology-based readouts for representations, and topological information for graph and simplicial-complex learning.The review also describes efforts to improve simplicial-complex efficiency and a 1-skeleton approach used for performance reasons.Source: §A.3 · p. 23
• Other work links local persistent-homology singularity information with generalization performance and uses topological information in image reconstruction and generative-graph evaluation.The section describes quantized and approximated Euler characteristic transforms, a differentiable Euler characteristic transform, and curvature-based topological information.Source: §A.3 · pp. 23–24
• Probabilistic approaches have developed sampling methods for distributions of topological summaries, while decision-boundary topology has been used to quantify DNN complexity and match datasets to pretrained models.Source: §A.3 · p. 24

Data compression

• TDL-based graph-signal compression outperformed GNNs and feed-forward architectures on two real-world internet datasets, while topological dictionary learning showed advantages in edge-flow compression.Source: B. Application Areas for TDL · p. 24

Biological imaging

• For biological imaging, topological or geometric principles can guide noisy cryo-EM 2D-to-3D reconstruction, and topological priors have contributed to improved inference in segmentation and shape reconstruction.A cellular-complex reference can encode protein residues as nodes, covalent bonds as edges, and rings as faces.Source: B. Application Areas for TDL · p. 25

Virus evolution

• TDL has been used to study SARS-CoV-2 mutation and selection, with dominant variants BA.2, BA.4, and BA.5 forecast approximately two months in advance.Source: B. Application Areas for TDL · p. 25

Drug design; Neuroscience

• In drug design, TDL won the D3R Grand Challenges and benchmark studies report enhanced predictive performance from dynamic TDL methods and static topological features.The section also describes static topological features in fMRI analysis and prediction, and higher-order models for representing and decoding neural structures.Source: B. Application Areas for TDL · p. 26

Protein engineering

• Persistent-homology TDL performed strongly across 34 protein-engineering datasets, though persistent Laplacian outperformed it in protein-engineering applications.Source: B. Application Areas for TDL · p. 26

Chip design

• For chip design, directed-hypergraph models combining persistent homology and spectral features have been used to predict netlist properties amid scale, long-range interaction, path-information, and symmetry challenges.Source: B. Application Areas for TDL · p. 26

Semantic communications; Satellite imagery

• TDL could extract relational semantic features for communications, where transmitted information must meet task and performance requirements; topological invariants are also proposed for robust multiscale satellite-image analysis.Source: B. Application Areas for TDL · p. 27

Materials science; Other application areas

• Materials applications include persistent-homology models that improved crystalline-compound formation-energy prediction, while the section also identifies potential in data compression, NLP, vision, graphics, and quantum TDL.The quantum discussion describes persistent-Betti-number computation and quantum Takens delay embedding, while noting that quantum topological methods remain in their infancy.Source: B. Application Areas for TDL · pp. 24, 25, 27