PaperlayerView exact revision on arXiv

Source-linked AI summary

The ITensor Software Library for Tensor Network Calculations

Matthew Fishman, Steven R. White, E. Miles Stoudenmire

arXiv:2007.14822v2cs.MScond-mat.str-elphysics.comp-ph

TL;DR

Tensor-network programming requires managing connectivity and implementation details that are difficult to express with traditional notation. ITensor addresses this with a diagram-inspired, index-aware library and reviews its interfaces, algorithms, storage, applications, and performance. The paper presents ITensor as a system for rapid prototyping, with Julia performance reported as ahead of its already optimized C++ implementation in the discussed benchmarks.

  • Problem

    Traditional tensor-network notation makes connectivity and index bookkeeping difficult, motivating an interface that hides implementation details while preserving the diagram's conceptual structure.

  • Method

    The paper gives a high-level overview of ITensor's diagram-inspired design and demonstrates its index objects, tensor operations, decompositions, network algorithms, storage types, and implementations.

  • Results

    Julia performance was reported as ahead of ITensor's already highly optimized C++ implementation in the discussed tensor-network benchmarks.

  • Takeaways & Limitations

    ITensor provides a library interface for expressing tensor-network algorithms at high and low levels while reducing manual index-order bookkeeping.

  • Takeaways & Limitations

    The intelligent-index approach can require extra code for manipulating index properties and can reduce control over low-level details when high-level features are used.

Abstract

from arXiv · show

ITensor is a system for programming tensor network calculations with an interface modeled on tensor diagram notation, which allows users to focus on the connectivity of a tensor network without manually bookkeeping tensor indices. The ITensor interface rules out common programming errors and enables rapid prototyping of tensor network algorithms. After discussing the philosophy behind the ITensor approach, we show examples of each part of the interface including Index objects, the ITensor product operator, tensor factorizations, tensor storage types, algorithms for matrix product state (MPS) and matrix product operator (MPO) tensor networks, quantum number conserving block-sparse tensors, and the NDTensors library. We also review publications that have used ITensor for quantum many-body physics and for other areas where tensor networks are increasingly applied. To conclude we discuss promising features and optimizations to be added in the future.

5 Tensor Decompositions

ITensor presents tensor networks through persistent, information-rich indices and a diagram-inspired interface that hides bookkeeping while supporting multiple programming levels. The library is implemented in C++ and Julia, with documentation and examples intended to support practical use.

  • Tensor networks: Tensor networks represent high-order tensors as contracted products of low-order tensors, avoiding exponential costs for important operations.The representation can remain efficient even when the implicit tensor has hundreds, thousands, or infinitely many indices.
  • Design philosophy: ITensor lets users translate tensor diagrams into code while automatically handling index matching, ordering, and related implementation details.Summing tensors requires the same set of indices in any order, and the system handles the remaining details.
  • Programming model: ITensor supports multiple abstraction levels, from high-level algorithms such as DMRG to intermediate MPS routines and low-level individual ITensors.This multilevel access was a reason for designing ITensor as a library rather than a single executable.
  • Index system: Persistent, intelligent indices let collections of ITensors specify network connectivity and store information such as conserved quantum-number subspaces.This design also supports generic code across tensor types and helps coordinate storage ordering for quantum-number-conserving tensors.
  • Trade-offs: The intelligent-index approach simplifies tensor operations but can require extra code for index-property manipulation and reduce low-level control in high-level features.The library also provides advanced features for users who need complete control over operation details.

3 Index Objects

ITensor Index objects encode vector-space identity through unique ids, while tags and prime levels provide additional ways to distinguish indices and control contractions.

  • Index identity: Index ids encode vector-space identity, so separately constructed indices remain unequal even when their dimensions match.Each construction receives a new id, and equality requires matching ids.
  • Tags: Tags are limited to eight characters per tag and four tags per index in the current library implementation.These hard-coded limits may be increased for future use cases.
  • Tags: Tags help identify indices, collect subsets, and prevent selected index pairs from contracting.Tags are application-specific metadata in addition to their role in equality and contraction rules.
  • Implementation: Index ids are randomly generated 64-bit values, making collisions highly improbable and supporting parallel algorithms and file interoperability.The random-id design also allows ITensors read from files to mix with newly generated ones.
  • Prime levels: Prime levels distinguish temporary Index copies, while tags are preferable when the distinction reflects application-specific meaning.Calling noprime(T) resets all index prime levels on an ITensor.

4 The ITensor Product Operator (∗)

The ITensor product operator uses shared-index structure to unify outer products, inner products, and tensor contractions without requiring users to track index ordering manually. Special tensors extend the same operator to replace, duplicate, combine, and uncombine indices.

  • Product rules: The * operator computes an outer product when two ITensors share no indices, an inner product when all indices match, and a contraction otherwise.An inner product returns a scalar ITensor, while partial index overlap produces a tensor contraction.
  • Product rules: Users need only know which indices tensors share; ITensor determines the uncontracted-index ordering in the result.This removes manual bookkeeping of index order in typical tensor products.
  • Special tensors: Delta tensors can replace an index with an equal-dimension index or duplicate an index through multiplication by a specially stored copy tensor.Delta tensors use diagonal-sparse storage to reduce memory use.
  • Special tensors: Combiners merge multiple indices into one, and multiplying by their conjugates reverses the operation.Combiner storage and contraction algorithms are optimized for combining and uncombining indices.
  • Special tensors: For symmetric tensors, combiners can group quantum numbers and manage anticommutation properties in the planned fermion system.For dense tensors, the operation corresponds conceptually to permuting and reshaping a multidimensional array.

5 Tensor Decompositions

ITensor defines tensor factorizations by grouping indices into rows and columns, allowing familiar matrix decompositions such as QR and SVD to operate directly on tensors. The resulting factors share generated indices, and SVD supports controlled truncation.

  • Factorization setup: Tensor QR and SVD decompositions map selected row and column index groups to matrix dimensions before applying standard linear-algebra factorizations.This lets tensor-network decompositions use matrix methods without manually reshaping tensors.
  • QR decomposition: QR generates a new linking Index between Q and R, so their product reconstructs the original tensor.Users need not know the generated Index’s position or dimension to compute Q*R.
  • QR decomposition: The commonind function retrieves the Index shared by two factor tensors when direct access to the generated linking Index is needed.It returns the first shared Index found between the tensors.
  • SVD: SVD is implemented by specifying row indices, with the remaining indices treated as columns.In the example, j,i are rows and m,k are columns.
  • SVD: 10^-8 truncation error and a maximum of 10 retained singular values can be specified for a controlled decomposition.ITensor does not truncate by default, although it computes thin decompositions when available.

6 Tensor Storage Layer

ITensor’s storage layer presents diverse tensor representations through one interface, automatically selecting formats and specialized algorithms while supporting sparse, dense, GPU, and future storage extensions.

  • ITensors can mix sparse and dense tensors while using identical high-level code.
  • Storage types are often selected automatically, including diagonal-sparse storage after SVD and block-sparse storage for quantum-number subspaces.
  • Distinct storage types enable optimized memory layouts and specialized contraction and factorization pathways through Julia multiple dispatch.
  • Common formats include dense, diagonal, combiner, block-sparse, and empty storage, which can represent zeros without allocating memory.
  • GPU storage uses specialized routines and can provide speedups ranging from two to a hundred times CPU calculations.
  • Planned work includes identity and unitary storage, delayed operations, broader sparsity, additional numerical types, and documented custom storage types.

7 High Level Features: MPS and MPO Algorithms

ITensor provides high-level MPS and MPO algorithms alongside OpSum/AutoMPO tools for constructing and manipulating tensor-network representations. These features support customizable DMRG workflows, operator compilation, and direct access to network factors and observables.

  • ITensor includes mature MPS and MPO formats and algorithms for rapid tensor-network prototyping.
  • Core algorithms cover MPS and MPO summation, truncation, DMRG optimization, and MPS–MPO multiplication with configurable tradeoffs.
  • 7.1 OpSum and AutoMPO: OpSum expresses sums of local operator products, while AutoMPO compiles them into MPO tensor networks for Hamiltonian calculations.
  • 7.1 OpSum and AutoMPO: AutoMPO handles multi-operator and arbitrarily separated terms, then uses SVD-based compression to obtain a nearly optimal MPO bond dimension.
  • 7.2 DMRG Algorithm: DMRG computes low-energy states or dominant eigenvectors of large Hermitian operators and can operate on Hamiltonians represented as MPOs.
  • 7.2 DMRG Algorithm: DMRG supports separate MPO terms, excited-state constraints through orthogonality penalties, sweep schedules, and tunable accuracy parameters.
  • 7.3 MPS and MPO Operations: Users can retrieve and update individual MPS factor tensors, compute local expectations and correlations, and orthogonalize an MPS.

8 Quantum Number Block Sparse ITensors

ITensor represents conserved-quantity structure through quantum-number-aware Index objects, producing block-sparse tensors that preserve symmetry constraints while retaining nearly the same interface as dense tensors. This structure can reduce memory and computation, with practical contraction speedups reported up to 5×.

  • Quantum number conservation reflects physical symmetries such as particle number and spin, while naturally imposing a block-sparse tensor structure.The block-sparse representation can reduce memory use and accelerate calculations.
  • Quantum-number-conserving ITensors offer nearly the same interface as dense ITensors, allowing generic algorithms to work for both cases.The passage specifically includes algorithms written for dense tensors that automatically work for symmetric tensors when conjugation is handled correctly.
  • QN information stored in Index objects determines block layouts, allocated blocks, and propagated quantum-number structure through tensor operations.The system uses this information when tensors are constructed, summed, contracted, or factorized.
  • QN Index objects use named quantum numbers, optional modular addition rules, subspaces, and arrow directions to enforce valid contractions and calculate tensor flux.Opposite arrow directions are required for contraction; Out and In quantum numbers contribute with opposite signs to flux.
  • General block-sparse operations can run up to 5× faster than dense equivalents in practical physics applications through skipped zero blocks and multi-core parallelism.The reported speedup varies with the application.
  • ITensor supports dense-tensor operations, including combiners, SVD and QR factorizations, and MPS/MPO algorithms, for quantum-number-conserving tensors with the same interface.

9 NDTensors Library

NDTensors separates low-level tensor algorithms and storage from ITensor’s high-level intelligent-index interface. It provides generic dense and sparse tensor operations, interoperates with ITensor indices, and uses matrix-multiplication and parallel block-contraction strategies for performance.

  • NDTensors modularizes low-level contraction routines and sparse storage separately from high-level ITensor logic.This separation is intended to simplify code and prevent bugs.
  • NDTensors is a standalone, generic high-performance library supporting multiple sparse tensor types, but users must track index ordering explicitly.
  • Generic index objects allow NDTensors to interoperate with ITensor and support features such as tensor slicing.
  • Dense tensor contraction permutes and reshapes tensors into matrices so the operation can use high-performance BLAS matrix multiplication.The strategy accepts permutation overhead because matrix multiplication dominates asymptotically and benefits from optimized BLAS implementations.
  • Block-sparse contraction decomposes into smaller dense contractions that can run independently, enabling multi-core parallelism.
  • Multi-core block-sparse contractions produced 2–3× speedups for DMRG and related MPS calculations and up to 5× for tree tensor networks.The speedups depend on sparsity and block sizes.
  • TBLIS is an experimental optional backend that reduces permutation overhead and has shown speedups over current contraction code, particularly for larger tensors.The paper identifies broader benchmarking with DMRG as future work.

10 Other Features of ITensor

ITensor extends its core tensor interface with portable data storage, customizable local Hilbert spaces, mixed site types, and observer-driven DMRG customization. These features target reproducible workflows and flexible implementations of physical models and algorithms.

  • ITensor can read and write nearly every tensor-network type using HDF5 files.HDF5 provides portability, metadata organization, memory efficiency, and compression for large heterogeneous datasets.
  • Users can define custom local Hilbert spaces and operators for physical systems beyond ITensor’s built-in degree-of-freedom definitions.Examples include bosons, higher spins, and ZN parafermions.
  • Julia streamlines custom Hilbert-space definitions through tagged Index objects and associated SiteType methods.A tag such as S=3/2 can determine the corresponding local operators and site behavior.
  • SiteType operator definitions are reused by OpSum and MPS/MPO constructors, allowing a Heisenberg MPO for S = 3/2 spins to work after the definitions are added.
  • Different SiteType tags can be mixed in Index arrays to construct systems containing alternating spin or boson sites.
  • The DMRG observer interface lets users customize measurements and stopping criteria by defining measure! and checkdone! methods.These methods are called at each step of the DMRG algorithm and can inspect the current sweep, site location, and MPS.

11 Applications of ITensor

ITensor has been used broadly in equilibrium and dynamical quantum many-body studies, as well as quantum chemistry, tensor-network, quantum-computing, applied-mathematics, and machine-learning applications. The examples illustrate its use across ground-state, finite-temperature, out-of-equilibrium, and non-physics problems.

  • ITensor had been cited in approximately 450 research articles from 2009 to 2021, spanning quantum many-body physics and other fields.
  • Equilibrium applications include ground-state DMRG, finite-temperature methods such as METTS, quantum chemistry, and PEPS calculations.
  • ITensor DMRG has been used for two-dimensional systems despite exponential scaling in transverse system size.
  • A study of the hole-doped triangular-lattice Hubbard model found a transition from p-wave to d-wave superconductivity as on-site interactions increased.
  • Dynamical applications include predicting entanglement entropy in integrable systems, simulating random circuits, and matching cold-atom experimental observations.
  • Other applications include DMFT solver subroutines, quantum-device simulation, quantum-circuit optimization, tomography, tensor decomposition, and machine learning.

12 Benchmarks of ITensor Performance

The benchmarks compare ITensor’s Julia and C++ implementations across tensor-network algorithms, threading settings, and other software. Julia is highly competitive overall and often faster, especially for block-sparse operations, while performance depends on algorithm and configuration.

  • High-level conclusions: Julia is very competitive with C++ for tensor-network algorithms and is currently fastest for some block-sparse ITensor algorithms.The reported advantage is attributed to recent optimization efforts and Julia’s productive development environment.
  • Julia versus C++: TRG benchmarks show similar C++ and Julia performance, with C++ slightly better at bond dimension 40 and Julia better at bond dimension 50.BLAS and LAPACK threading speeds up these contraction-dominated calculations.
  • Julia versus C++: Julia is consistently faster than C++ for CTMRG across a wide range of larger boundary-tensor bond dimensions.Four BLAS threads provide a speedup, while eight threads add little additional benefit.
  • Block-sparse performance: Julia handles block-sparse tensors more efficiently than C++, following optimizations for nonzero-block storage and contractions of small blocks.The relative benefit of threading differs between dense and block-sparse calculations.
  • Overall assessment: The C++ implementation already approaches state-of-the-art performance, making Julia’s stronger results a non-trivial outcome.The Julia implementation was originally modeled on C++ but has moved ahead after recent optimization work.
  • Comparison with other software: Once algorithmic details and external dependencies are aligned, ITensor and TeNPy show comparable DMRG performance.TeNPy’s sparse Hamiltonian representation often speeds up DMRG, motivating ITensor’s splitblocks capability, whose benefit depends on the system.

13 Future Directions

ITensor’s future development prioritizes automatic differentiation, broader fermion and symmetry support, optimized contraction workflows, and expanded tensor-network algorithms. Planned infrastructure also targets infinite MPS/MPO methods, PEPS, and GPU execution with minimal user effort.

  • Core capabilities: Automatic differentiation is a high-priority feature for differentiating through ITensor operations in tensor-network applications.The planned system is intended to work with Julia differentiation rules and frameworks such as ChainRules.jl and Zygote.jl.
  • Symmetries and fermions: ITensor plans automatic support for fermionic Hilbert spaces to reduce situations where users must manually introduce Jordan–Wigner strings.Current automatic fermion support is limited to the OpSum/AutoMPO system.
  • Symmetries and fermions: Support for non-Abelian symmetries such as SU(2) is planned for Heisenberg, Hubbard, and electronic-structure applications.This follows planned completion of the fermion system and expansion to other non-trivial vector spaces.
  • Tensor-network tooling: Future tensor-network tooling includes contraction-sequence optimization and visualization for inspecting and debugging tensor contractions.Planned visualization backends include text and interactive output, alongside additional optimization backends.
  • Algorithm support: First-class support is planned for infinite MPS and MPO algorithms, including canonical forms, eigenvalue methods, and applying infinite MPOs to infinite MPS.These methods are intended to offer the convenience of existing finite MPS and MPO functionality.
  • Hardware support: GPU support is being expanded from dense tensor operations toward block-sparse operations and direct high-performance reuse of CPU code.The eventual goal includes automatic differentiation with minimal user effort.
  • Algorithm support: ITensor plans more high-level PEPS features using general tensor-network interfaces, automatic differentiation, and contraction-sequence optimization.Targeted methods include variational iPEPS and fixed-point approaches for PEPS environments.

A Full Code Examples

The examples demonstrate ITensor from basic tensor contraction and index inspection to a complete DMRG calculation. A Heisenberg-chain example constructs an MPO and MPS, configures sweeps, and returns a ground-state energy and wavefunction.

  • Basic tensor operations: ITensor code can express complicated tensor contractions and represent the resulting contraction with tensor diagrams.The example uses code to contract two ITensors and a diagram to describe that contraction.
  • Basic tensor operations: Index tags are optional in code but make indices easier to identify when printed.Collections of indices can also be created with compact Julia broadcast syntax.
  • Basic tensor operations: The hasinds function checks that an ITensor contains the specified indices.The example checks that tensor C contains a, b, c, i, and j.
  • DMRG example: The DMRG example constructs an MPO Hamiltonian from OpSum and a random MPS with bond dimension 10.It then configures five sweeps with specified maximum bond dimensions and a truncation cutoff of 10^-11.
  • DMRG example: The five-sweep example reaches a ground-state energy of -44.127739882501665.The maximum link dimension grows from 10 after sweep 1 to 139 after sweep 5; the first sweep includes compilation time.
  • DMRG example: The DMRG run returns the ground-state energy and ground-state wavefunction MPS.The example calls dmrg with the Hamiltonian, initial MPS, and sweep schedule.

B ITensor Implementation and Interface in the C++ Language

The C++ ITensor interface closely parallels the Julia version while adapting syntax, headers, variable declarations, and output controls to C++. A DMRG example also illustrates configurable sweeps, cutoffs, quantum-number handling, and named arguments.

  • Overview: C++ ITensor examples demonstrate similarities and differences from the Julia interface through tensor contraction and DMRG code.The appendix uses corresponding examples to compare both language versions.
  • Interface differences: C++ syntax adds library includes, auto variable declarations, semicolons, and the ITensor Print macro.Print provides behavior similar to Julia’s @show macro.
  • DMRG example: The C++ DMRG example constructs an MPO from spin interactions and configures five sweeps with increasing maximum dimensions and a 1E-11 cutoff.The displayed Hamiltonian includes SzSz and spin-raising/lowering terms.
  • DMRG example: C++ site Index arrays include quantum-number information by default, but the example disables it, while dmrg output is silenced with {"Quiet=",true}.These choices illustrate the C++ version’s custom named-argument system.
Loading 2007.14822v2…