Source-linked AI summary
TensorLy: Tensor Learning in Python
Jean Kossaifi, Yannis Panagakis, Anima Anandkumar, Maja Pantic
TL;DR
Tensor methods are increasingly relevant for multidimensional data, but software support has lagged behind matrix methods and limited adoption. TensorLy addresses this gap with an open Python API offering tensor methods, optimized operations, and interchangeable numerical and deep-learning backends. The paper presents a tested, documented library designed to make tensor learning accessible and straightforward.
Problem
Tensor methods have not been widely adopted because accessible Python libraries for tensor operations, decompositions, and learning are lacking or limited.
Method
TensorLy provides tensor algebra, decompositions, regression, and optimized operations through a Python API with interchangeable NumPy, MXNet, PyTorch, TensorFlow, and CuPy backends.
Results
TensorLy offers competitive decomposition speed through optimized formulations and implementation across CPU and GPU backends.
Takeaways & Limitations
TensorLy makes tensor learning accessible through simple interfaces, a permissive license, systematic testing, documentation, and transparent backend switching.
Abstract
from arXiv · showhide
Tensors are higher-order extensions of matrices. While matrix methods form the cornerstone of machine learning and data analysis, tensor methods have been gaining increasing traction. However, software support for tensor operations is not on the same footing. In order to bridge this gap, we have developed \emph{TensorLy}, a high-level API for tensor methods and deep tensorized neural networks in Python. TensorLy aims to follow the same standards adopted by the main projects of the Python scientific community, and seamlessly integrates with them. Its BSD license makes it suitable for both academic and commercial applications. TensorLy's backend system allows users to perform computations with NumPy, MXNet, PyTorch, TensorFlow and CuPy. They can be scaled on multiple CPU or GPU machines. In addition, using the deep-learning frameworks as backend allows users to easily design and train deep tensorized neural networks. TensorLy is available at https://github.com/tensorly/tensorly
1. Introduction
Tensor methods suit increasingly common multidimensional data, but their adoption has been limited by inadequate, accessible software libraries. TensorLy is introduced to address this gap with open, documented Python support for tensor learning.
- Tensors generalize matrices to arbitrary numbers of indices and naturally represent multidimensional and multimodal data.
- Tensor methods extend dimensionality reduction and support higher-order moment analysis for applications including neural networks and probabilistic latent-variable models.
- Despite theoretical, algorithmic, and practical advantages, tensor methods have not been widely adopted because accessible software libraries are lacking.
- Existing libraries are limited by non-free platforms, low-level implementations, restricted algorithms or licenses, and missing advanced operations in deep-learning frameworks.
- TensorLy provides state-of-the-art tensor operations, decompositions, regression, flexible backends, BSD licensing, NumPy/SciPy defaults, tests, and documentation.
2. TensorLy: functionalities and implementation
TensorLy provides a simple, tested, documented tensor-learning API integrated with the Python ecosystem. Its backend system supports computation across multiple numerical and deep-learning platforms.
- TensorLy’s flexible backend system runs tensor operations with NumPy, MXNet, PyTorch, TensorFlow, or CuPy.
- The library is available as open source with documentation and is designed to integrate with the Python scientific environment.
- TensorLy builds tensor algebra operations on the Python ecosystem and uses them for higher-level methods such as regression and decomposition.
- Its backends provide CPU, GPU, multi-machine, NumPy-like, automatic-differentiation, or eager-execution capabilities through their respective frameworks.
- Implemented methods include CP and Tucker decompositions, non-negative variants, Robust Tensor PCA, and low-rank tensor regression.
- All functions include documentation and unit tests, with 97 % coverage at the time of writing.
3. Performance
TensorLy improves tensor-operation performance through Python-oriented implementations, including an efficient unfolding that reorders fibers for C-ordering. Experiments report competitive decomposition speed across CPU and GPU backends.
- TensorLy targets competitive performance by manipulating multidimensional arrays directly and optimizing tensor operations for Python’s high-level environment.
- The proposed unfolding differs from the traditional formulation by changing the ordering of tensor fibers.
- The unfolding achieves better performance with C-ordering, the default element ordering in NumPy and most Python libraries.
- Tests compare rank–50 CP and rank (50, 50, 50) Tucker decomposition on 500×500×500 tensors across CPU and GPU backends.The experiment used 100 iterations and was repeated 10 times.
- Figure 2 reports speed comparisons for Tucker and CANDECOMP-PARAFAC decomposition.
4. Conclusion
TensorLy makes tensor learning accessible through state-of-the-art methods, consistent interfaces, testing, documentation, and a permissive license. Its backend flexibility supports switching platforms and combining tensor methods with deep learning.
- TensorLy offers state-of-the-art tensor methods through simple, consistent interfaces under a permissive license.
- The library is optimized for speed and robustness and includes systematic unit tests and documentation.
- Its speed and ease of use support comparison of existing methods and implementation of new ones.
- The flexible backend system enables transparent switching between libraries and platforms and combination of tensor methods with deep learning.