PaperlayerView exact revision on arXiv

Source-linked AI summary

Tensor Logic: The Language of AI

Pedro Domingos

arXiv:2510.12269v3cs.AIcs.LGcs.NEcs.PLstat.ML

TL;DR

AI lacks a language combining neural and symbolic capabilities: Python libraries omit automated reasoning and knowledge acquisition, while LISP and Prolog lacked scalability and learning support. Tensor logic unifies neural and symbolic AI mathematically, using tensor equations as its program construct. At T = 0, the Gram matrix effectively reduces to the identity, making the program’s reasoning purely deductive; unlike LLMs, which may hallucinate even at T = 0.

  • Problem

    AI lacks a language combining neural and symbolic capabilities: Python libraries omit automated reasoning and knowledge acquisition, while LISP and Prolog lacked scalability and learning support.

    Source: §1 · p. 1

  • Method

    Tensor logic unifies neural and symbolic AI mathematically, using tensor equations as its program construct.

    Source: §1 · p. 2, §3.1 · p. 6

  • Results

    At T = 0, the Gram matrix effectively reduces to the identity, making the program’s reasoning purely deductive; unlike LLMs, which may hallucinate even at T = 0.

    Source: §5 · p. 15

  • Takeaways & Limitations

    Tensor logic can make scientific-computing control structures learnable by relaxing Boolean tensors to numeric ones and optionally thresholding them back into logic.

    Source: §7 · p. 16

    A join tree may be exponentially larger than the original model, necessitating approximate inference.

    Source: §4.4 · p. 13

Abstract

from arXiv · show

Progress in AI is hindered by the lack of a programming language with all the requisite features. Libraries like PyTorch and TensorFlow provide automatic differentiation and efficient GPU implementation, but are additions to Python, which was never intended for AI. Their lack of support for automated reasoning and knowledge acquisition has led to a long and costly series of hacky attempts to tack them on. On the other hand, AI languages like LISP and Prolog lack scalability and support for learning. This paper proposes tensor logic, a language that solves these problems by unifying neural and symbolic AI at a fundamental level. The sole construct in tensor logic is the tensor equation, based on the observation that logical rules and Einstein summation are essentially the same operation, and all else can be reduced to them. I show how to elegantly implement key forms of neural, symbolic and statistical AI in tensor logic, including transformers, formal reasoning, kernel machines and graphical models. Most importantly, tensor logic makes new directions possible, such as sound reasoning in embedding space. This combines the scalability and learnability of neural networks with the reliability and transparency of symbolic reasoning, and is potentially a basis for the wider adoption of AI.

1. Introduction

The paper argues that fields advance through languages that focus thought and reduce repeated work, while AI still lacks a language that combines the strengths of neural and symbolic approaches. Tensor logic addresses this by unifying their mathematical foundations; the paper then develops its engines, implementations, scaling approaches, and applications.

1. Introduction

  • A field’s language can save practitioners time, focus their attention, shape how they think, and reduce repeated work.The introduction uses examples from mathematics, engineering, and computing to motivate the role of specialized languages and notations.Source: §1 · p. 1
  • AI lacks a purpose-built language: Python’s AI libraries do not support automated reasoning or knowledge acquisition, while LISP and Prolog lacked scalability and learning support.Graphical models and Markov logic also face inference costs, and neurosymbolic systems can inherit shortcomings of both deep learning and symbolic AI.Source: §1 · pp. 1–2
  • A suitable AI language should support prior knowledge, automated reasoning and learning, transparency, reliability, and effortless scaling while hiding non-AI details.The paper argues that symbolic AI and deep learning each have some of these properties, but neither has all.Source: §1 · p. 2
  • Tensor logic unifies neural and symbolic AI by treating tensor algebra and logic programming as fundamentally equivalent apart from their atomic data types.The paper’s premise is that essentially all neural networks can be built with tensor algebra and all symbolic AI with logic programming.Source: §1 · p. 2
  • The paper defines tensor logic and its inference and learning engines, then presents implementations, scaling approaches, and potential applications.Its roadmap includes neural networks, symbolic AI, kernel machines, graphical models, and reasoning in embedding space.Source: §1 · p. 2

2. Background

The background introduces logic programs as rules and facts that support sound inference and learning from data, then reviews tensors and their algebraic operations. It connects rule evaluation to joins and projections, and tensor computation to Einstein summation and tensor decomposition.

2.1 Logic Programming

  • A Datalog program consists of facts and rules, with a rule deriving its head when the relations in its body are simultaneously true.In the example, the rules and facts imply Ancestor(Alice, Charlie).Source: §2.1 · pp. 2–3
  • A Datalog rule corresponds to joins followed by projection, combining tuples that agree on shared arguments and retaining selected arguments.The example joins Ancestor(x, y) with Parent(y, z) on y and projects onto x and z.Source: §2.1 · p. 3
  • Forward chaining repeatedly derives facts to a fixpoint, while backward chaining recursively matches a query against facts and rule bodies; both procedures are sound.The resulting deductive closure answers queries by examining the derived facts.Source: §2.1 · p. 3
  • Inductive logic programming learns logic programs from data and can discover relations absent from the data, analogous to neural-network hidden variables.The example describes inducing ancestry rules from a small database of parent and ancestor relations.Source: §2.1 · p. 3

2.2 Tensor Algebra

  • A tensor is characterized by its type and shape, with shape specifying its rank and the size of each index dimension.Matrices, vectors, and scalars are rank-2, rank-1, and rank-0 tensors, respectively.Source: §2.2 · pp. 3–4
  • Einstein summation expresses tensor operations by summing over repeated indices, and neural-network linear and multilinear operations can be written as einsums.Tensor decomposition can also represent a tensor through a compact core and factor matrices.Source: §2.2 · p. 4

3. Tensor Logic

Tensor logic represents relations and computations with tensor equations, linking symbolic rules to tensor operations. The section develops this representation for inference and learning.

3.1 Representation

  • Sparse Boolean tensors can be represented compactly as relations, with each tuple recording a nonzero element.The passage says the efficiency gain typically increases exponentially with tensor rank.Source: §3.1 · p. 5
  • Datalog rules correspond to Boolean einsums followed by step functions, while tensor joins and projections correspond to database join and projection.Shared-index joins multiply matching entries; projection sums over omitted indices.Source: §3.1 · pp. 5–6
  • A tensor logic program consists of tensor equations whose right-hand sides join tensors, project onto left-hand-side indices, and may apply an elementwise nonlinearity.Tensor elements default to zero, and equations with the same left-hand side are implicitly summed.Source: §3.1 · p. 6

3.2 Inference

  • Forward chaining executes equations repeatedly until no new elements can be computed or a stopping criterion is met; backward chaining recursively resolves queried right-hand-side tensors.Unresolved subquery elements default to zero, and the choice of chaining method depends on the application.Source: §3.2 · p. 7

3.3 Learning

  • Differentiation produces a tensor logic program, with gradient equations for the source equations and tensors on their right-hand sides.The paper attributes this simplicity to tensor equations being the language's only statement type.Source: §3.3 · pp. 7–8
  • Training specifies losses with tensor equations, and backpropagation through structure updates equation parameters at each occurrence in an example's derivation.With a data tensor as the sole data input, learned Tucker components can be thresholded into invented predicates; fixed equation sets can represent rules sharing a join structure.Source: §3.3 · p. 8

4. Implementing AI Paradigms

This section demonstrates tensor logic across neural, symbolic, kernel, and probabilistic graphical-model paradigms. It expresses their implementations and inference procedures through tensor equations, while graphical-model inference may require approximation when join trees grow exponentially.

4.1 Neural Networks

  • Tensor equations implement CNN convolution and pooling, attention and transformers, and GNN message passing over graph neighbors.A transformer can be implemented with a dozen tensor equations; GNN aggregation sums neighboring nodes’ features before combining them with each node’s own features.Source: §4.1 · pp. 9–10

4.2 Symbolic AI

  • Any Datalog program is a valid tensor logic program, covering reasoning and planning in function-free domains; Prolog-style functions require unification.The paper identifies unification as the additional requirement for accommodating functions.Source: §4.2 · p. 11

4.3 Kernel Machines

  • Kernel machines use support vectors and kernels, including polynomial and Gaussian kernels; structured prediction adds equations for interactions among outputs and inputs.The output can be represented as a vector whose elements interact with one another and with the inputs.Source: §4.3 · p. 11

4.4 Probabilistic Graphical Models

  • Graphical models map factors and Bayesian-network CPTs to tensors; marginalization and pointwise products become projection and joins, enabling backward-chaining inference over join trees.Inference is linear in join-tree size, but a join tree may be exponentially larger than the original model, necessitating approximate inference such as loopy belief propagation or Monte Carlo sampling.Source: §4.4 · pp. 12–13

5. Reasoning in Embedding Space

Tensor logic represents objects, relations, and rules in embedding space, enabling approximate forward or backward chaining and analogical reasoning. Temperature controls the balance between deductive and analogical inference, with extracted intermediate tensors supporting transparency.

5. Reasoning in Embedding Space

  • A set can be represented by superposed random unit-vector embeddings; dot products and thresholding estimate membership, with error probability decreasing as embedding dimension grows.The dot product has standard deviation N/D, where N is set size and D is embedding dimension.Source: §5 · p. 13
  • Relations can be represented as superpositions of tuple tensor products, computed in time linear in |R|, and queried by retrieving the embedding of a tuple.The relation representation corresponds to a Tucker decomposition; relations of arbitrary arity can be reduced to (relation, argument, value) triples.Source: §5 · pp. 13–14
  • Embedded Datalog rules support forward or backward chaining, with query answers extracted approximately and error decreasing as embedding dimension grows.Extracting, thresholding, and re-embedding inferred tensors at intervals can further reduce error.Source: §5 · p. 14
  • With learned object embeddings, the Gram matrix measures pairwise similarity, so similar objects borrow inferences and enable analogical reasoning that combines similarity with compositionality.The weight of borrowed inferences is proportional to object similarity.Source: §5 · p. 15
  • Setting temperature T to 0 makes reasoning purely deductive, while increasing T makes it more analogical; inferred tensors can be inspected during inference.The paper describes reasoning as highly reliable and immune to hallucinations at sufficiently low temperature.Source: §5 · p. 15

6. Scaling Up

Tensor logic can scale through GPU execution of dense tensors and two approaches for sparse or mixed tensors: split execution between GPUs and a database engine, or densify via Tucker decomposition.

6. Scaling Up

  • Dense-tensor equations can be implemented directly on GPUs for large-scale learning and inference.Source: §6 · p. 15
  • A separation-of-concerns approach sends dense subtensors to GPUs and sparse subtensors to a database query engine, which can apply query optimization when combining them.The database engine can treat a dense subtensor as a single tuple, with an argument pointing to it.Source: §6 · p. 15
  • Alternatively, Tucker decomposition converts sparse tensors to dense ones for GPU operations, which the paper describes as exponentially more efficient than direct sparse operations.This approach allows a small, controllable error probability through embedding dimension and denoising, and combines with the paper’s learning and reasoning algorithms.Source: §6 · pp. 15–16

7. Discussion

The discussion extends tensor logic beyond AI to scientific computing and argues that backward compatibility with Python could support adoption. It identifies potential applications and outlines implementation and community-building next steps.

7. Discussion

  • In scientific computing, tensor logic maps equations to code more directly and makes logical control structure learnable by relaxing Boolean tensors to numeric ones, optionally thresholding them back to logic.The paper says the same approach is applicable in principle to making any program learnable.Source: §7 · p. 16
  • The proposed adoption path starts with Python-compatible einsum and reasoning extensions, then could absorb NumPy and PyTorch features over time.Source: §7 · p. 16
  • The paper presents potential cures for LLM hallucinations and opacity, and positions tensor logic for reasoning, mathematical, and coding models.Source: §7 · p. 16
  • Next steps include direct CUDA implementation, broader applications, libraries and extensions, and research into new directions.The discussion also proposes an open-source community, integrated IDEs, and teaching as possible routes to adoption.Source: §7 · p. 16
Loading 2510.12269v3…