Source-linked AI summary
giotto-tda: A Topological Data Analysis Toolkit for Machine Learning and Data Exploration
Guillaume Tauzin, Umberto Lupo, Lewis Tunstall, Julian Burella Pérez, Matteo Caorsi, Wojciech Reise, Anibal Medina-Mardones, Alberto Dassatti, Kathryn Hess
TL;DR
Topological data analysis has remained difficult for many machine-learning practitioners to access. giotto-tda addresses this by integrating diverse TDA algorithms and preprocessing into scikit-learn-compatible Python pipelines with interactive plotting and high-performance implementations, enabling TDA workflows across multiple data types.
Problem
TDA’s power and versatility have not translated into broad machine-learning use because many implementations target research purposes rather than high-level-language practitioners.
Method
giotto-tda combines scikit-learn-compatible preprocessing, persistent-homology and Mapper components, modular pipelines, interactive plotting, and C++-backed computation.
Results
The library supports end-to-end topological feature generation and exploration for time series, images, graphs, simplicial complexes, and other supported inputs.
Takeaways & Limitations
giotto-tda makes TDA available as a component of larger-scale machine-learning and data-exploration workflows for research and data-science users.
Abstract
from arXiv · showhide
We introduce giotto-tda, a Python library that integrates high-performance topological data analysis with machine learning via a scikit-learn-compatible API and state-of-the-art C++ implementations. The library's ability to handle various types of data is rooted in a wide range of preprocessing techniques, and its strong focus on data exploration and interpretability is aided by an intuitive plotting API. Source code, binaries, examples, and documentation can be found at https://github.com/giotto-ai/giotto-tda.
1 Introduction
giotto-tda addresses the limited accessibility of topological data analysis for machine-learning practitioners by bringing TDA algorithms and broad data support into Python.
- Motivation: TDA extracts shape-related features using algebraic and combinatorial topology, including persistent homology and Mapper.These methods have been applied to porous materials, CNN weight spaces, brain structure-function links, and breast-cancer subgroups.
- Motivation: giotto-tda targets the gap between TDA’s power and its limited adoption by machine-learning practitioners using accessible Python implementations.The paper attributes this gap largely to research-oriented implementations that are not written in high-level languages.
- Publication context: The work was presented at the Topological Data Analysis and Beyond Workshop at NeurIPS 2020.
- Contribution: The library combines scikit-learn compatibility with persistent-homology and Mapper-type algorithms for univariate and multivariate time series, images, graphs, and simplicial complexes.This scope is presented as making giotto-tda a comprehensive Python library for topological machine learning and data exploration.
2 Architecture
giotto-tda extends scikit-learn-compatible workflows with modular TDA estimators, visualization tools, and transformer behavior suited to time-delay embeddings.
- Pipeline design: Modular estimators support flexible TDA pipelines, while a Plotly-based API visualizes and interacts with intermediate NumPy-array results.The scikit-learn interface also supports large-scale hyperparameter search and feature selection.
- Time-series integration: TransformerResamplerMixin and an extended Pipeline accommodate TDA methods combined with time-delay embedding techniques for time-series prediction.These extensions address transformer flexibility beyond scikit-learn’s basic architecture.
- Model integration: Scikit-learn wrappers for PyTorch and TensorFlow allow deep-learning models to serve as final estimators in giotto-tda pipelines.
3 Persistent homology
giotto-tda provides an end-to-end, scikit-learn-compatible framework for turning diverse raw data into persistent-homology features through preprocessing, diagram computation, and feature extraction.
- Persistent-homology workflow: Persistent homology summarizes multi-scale relational information in persistence diagrams while capturing higher-order connectivity.giotto-tda supports transformations from diverse data types, multiple diagram algorithms, and rich diagram-feature extraction methods.
- Persistent-homology workflow: The resulting pipeline can generate crafted topological features from each sample in a raw data collection.Diagram representations include curves, images, and kernels, each typically requiring problem-specific hyperparameter tuning.
- Library support: Directed persistent homology is a highlighted feature distinguishing giotto-tda’s support from other Python persistent-homology libraries.It emphasizes the non-symmetric nature of many real-world interactions and is supported through preprocessing transformers for varied inputs.
- Implementation: The library uses state-of-the-art external C++ libraries with pybind11-based bindings and combines edge collapse with ripser to improve Vietoris–Rips barcode runtimes.Joblib parallelizes computations across data batches.
- Library comparison: Table 1 provides a snapshot of feature support across major Python open-source libraries with persistent-homology capabilities.
4 Mapper
giotto-tda implements Mapper as interoperable scikit-learn pipeline components with parallelization, caching, and interactive visualization for exploring high-dimensional data.
- Mapper representation: Mapper combines filter functions and partial clustering to represent high-dimensional data as a topologically meaningful graph or simplicial complex.It is primarily used to visualize and explore substructures of interest.
- Pipeline integration: The Mapper workflow can use scikit-learn estimators, integrate into larger machine-learning workflows, and parallelize its clustering step.
- Related work: The cited related-work list includes GUDHI, scikit-tda, and Dionysus 2.
- Pipeline overview: Figure 1 depicts available transformer operations and potential pipelines, but only non-exhaustively.
- Interactive exploration: Memory caching avoids unnecessary recomputation and supports real-time Mapper hyperparameter tuning during interactive visualization.Users can observe how the resulting graph changes as hyperparameters are adjusted.
- Comparison: Compared with KeplerMapper, giotto-tda implements Mapper steps in a scikit-learn pipeline and adds caching and real-time visualization interactivity.KeplerMapper is described as only partially compatible with scikit-learn pipelines and lacking those two features.
5 Project management
giotto-tda provides accessible distribution, quality assurance, and project infrastructure for using topological methods in Python.
- Binary packages for major operating systems are available through PyPI for straightforward installation.Users can install giotto-tda with python -m pip install -U giotto-tda.
- A Mapper graph generated from a 3D model’s height is provided as an example of the project’s visual outputs.
- The codebase is extensively tested, with 98% test coverage as of version 0.3.1.Testing uses pytest and hypothesis, with continuous integration and delivery via Azure Pipelines.
6 Concluding remarks
giotto-tda makes topological data analysis available to research and data-science communities as part of large-scale machine-learning workflows. Future development targets performance optimization and new topological algorithms.
- giotto-tda aims to support the use of TDA algorithms at any step of a machine-learning pipeline.
- The library is designed for both research and data-science communities, including large-scale machine-learning tasks.
- Future developments include further C++ performance optimization and an implementation of persistence Steenrod diagrams.