Source-linked AI summary
From Statistical Relational to Neuro-Symbolic Artificial Intelligence
Luc De Raedt, Sebastijan Dumančić, Robin Manhaeve, Giuseppe Marra
TL;DR
NeSy and StarAI both integrate learning with logical reasoning, but use neural networks and probabilistic graphical models, respectively. This survey aligns their systems across seven dimensions, identifies shared issues and differences, and points to research directions.
Problem
NeSy and StarAI share an interest in combining learning with symbolic reasoning, yet the survey identifies limited interaction between the fields.
Source: §1 · p. 1Method
The survey positions representative StarAI and NeSy systems along seven shared dimensions and compares their analogies.
Source: §1 · p. 1Results
The survey identifies shared issues and techniques across the fields while distinguishing StarAI’s symbolic emphasis from NeSy’s sub-symbolic emphasis.
Source: §1 · p. 1Takeaways & Limitations
The seven dimensions help characterize and compare approaches and identify areas for further research.
Source: §1 · p. 1The survey adopts a logical and probabilistic perspective, covers only consistent neuro-symbolic developments, and focuses on representative rather than exhaustive systems.
Source: §1 · p. 1
Abstract
from arXiv · showhide
Neuro-symbolic and statistical relational artificial intelligence both integrate frameworks for learning with logical reasoning. This survey identifies several parallels across seven different dimensions between these two fields. These cannot only be used to characterize and position neuro-symbolic artificial intelligence approaches but also to identify a number of directions for further research.
1 Introduction
The survey addresses the limited interaction between NeSy and StarAI, two fields that integrate symbolic reasoning with different learning paradigms. It compares their approaches along seven shared dimensions to characterize systems and identify research opportunities.
1 Introduction
- NeSy incorporates symbolic reasoning into neural networks, while StarAI integrates logic with probabilistic graphical models.The survey is motivated by the limited interaction between fields that share an interest in combining learning and reasoning.Source: §1 · p. 1
- The survey identifies seven shared dimensions and uses them to position a wide range of StarAI and NeSy systems and their analogies.The dimensions span model direction, inference, integration, semantics, learning, entity representation, and logic type.Source: §1 · p. 1
- StarAI operates more at the symbolic level, while NeSy operates more at the sub-symbolic level.The survey associates StarAI more naturally with explainable AI and NeSy with computer vision and natural language processing.Source: §1 · p. 1
- The survey uses a logical and probabilistic perspective, focuses on representative systems, and includes approaches used for learning.It does not aim for completeness and limits its coverage to neuro-symbolic developments consistent with that perspective.Source: §1 · p. 1
- Table 1 summarizes neuro-symbolic approaches across the dimensions, listing each system’s key reference.The following sections each describe one dimension.Source: §1 · p. 1
2 Directed vs undirected
The first dimension distinguishes systems by whether their models preserve a direction among variables or instead express constraints among them. The survey applies this distinction to both StarAI and NeSy approaches.
2 Directed vs undirected
- Directed models indicate a natural direction between random variables, whereas undirected models do not.The distinction yields two types of StarAI systems, generalizing directed models such as Bayesian networks and undirected models such as Markov networks or random fields.Source: §2 · p. 2
- StarAI’s directed category includes probabilistic logic programs, while its undirected category includes MLNs and PSL, which specify weighted constraints.The survey also lists plate notation, PRMs, and BLPs among directed representations.Source: §2 · p. 2
- Definite-clause rules express what can be inferred from what, while disjunctive-headed rules allow multiple conclusions and function more as constraints.This contrast reflects whether the knowledge expresses direct influence or a soft constraint among variables.Source: §2 · p. 2
- NeSy directed approaches include backward-chaining Prolog or Datalog systems and forward-chaining systems that compile definite clauses into neural architectures.Examples include NTPs, DeepProbLog, LRNNs, and ∂ILP.Source: §2 · p. 2
- Undirected NeSy methods use logical knowledge as soft constraints or extend MLN-style approaches with neural predicates or factors.SBR and SL use logical knowledge to regularize solutions; LTN, NMLN, and RNM implement predicates or factors as neural architectures.Source: §2 · p. 2
3 Grounding vs proofs
The survey distinguishes inference that grounds rules or formulas from inference that follows proofs or traces. It describes both strategies in StarAI and shows how they also appear in NeSy systems.
3 Grounding vs proofs
- Grounding replaces variables in a clause or formula with constants from the domain to produce ground instances, often retaining only those relevant to inference.A substitution assigns constants or other ground terms to the variables.Source: §3 · p. 2
- StarAI grounding instantiates a relational model for inference, while proof- or trace-based methods use proofs as the basis for probabilistic inference.Grounded models may use graphical-model inference or methods such as belief propagation and weighted model counting; proof-based approaches include PLPs and probabilistic databases.Source: §3 · pp. 2–3
- In NeSy, NTPs, DeepProbLog, ∂ILP, and DiffLog are proof-based, whereas LRNNs, LTNs, RNM, NMLN, NLM, and NeuralLP use grounding.Proof-based systems derive outputs from enumerated proofs; grounding-based systems learn shared parameters over the ground model and infer from possible groundings.Source: §3 · p. 3
4 Logic vs Probability vs Neural
The survey compares where integrated systems place logic, probability, and neural computation, noting that this position affects their properties. It describes neural-focused and logic-focused approaches, including differentiable intermediate representations.
4 Logic vs Probability vs Neural
- StarAI systems that compile logic into graphical models can make logical operations harder to reason about, while logic-centered systems can make probabilistic independencies harder to exploit.The survey presents these as trade-offs associated with the position on the logic–probability spectrum.Source: §4 · p. 3
- Most current NeSy approaches focus on neural computation, either compiling logic into network architectures or turning it into a regularization function.The latter penalizes violations of the desired logical theory or constraints.Source: §4 · p. 3
- A smaller set of NeSy methods retains a focus on logic by extending logic frameworks for differentiable operations or by making inference concepts differentiable.Examples include DeepProbLog and DiffLog, and ∂ILP, ∂4, and NTPs, respectively.Source: §4 · p. 3
- Weighted model counting can be compiled into a differentiable structure, such as an arithmetic circuit, supporting integration between logic-based systems and neural networks.DeepProbLog uses this approach, and the cited work describes the intermediate representation as an AI assembly language.Source: §4 · p. 3
5 Semantics
StarAI combines logical and probabilistic semantics, while statistical relational approaches also use fuzzy semantics. Neuro-symbolic systems can be characterized by how neural computation augments logical, probabilistic, fuzzy, or numerically relaxed reasoning.
5 Semantics
- StarAI combines truth-valued logical semantics with probability distributions over possible worlds, while fuzzy semantics relaxes truth values to [0,1] and logical operators to real-valued functions.Fuzzy semantics can use differentiable t-norms to reduce complexity, but the chosen connectives can alter logical properties such as satisfiability.Source: §5 · pp. 3–4
- Neural enhancements of logical semantics either map perceptive inputs to logical atoms or relax reasoning into tensor operations.The survey names ABL as an example of the first approach and NLM and NeuralLP as tensor-calculus examples.Source: §5 · p. 4
- Neural enhancements of probabilistic semantics reparameterize distributions with neural components, including neural predicates for facts and neural potentials for factors.DeepProbLog, NMLN, and RNM are examples described in this passage.Source: §5 · p. 4
- Neural enhancements of fuzzy semantics use differentiable t-norms and can produce continuous truth values through neural processes.SBR and LTN map constant features to atom truth values, while LRNN and Wang and Pan (2019) interpret logical-network neuron outputs as fuzzy truth values.Source: §5 · p. 4
- Other approaches assign atoms neural scores and numerically approximate formula combination or proof aggregation, yielding differentiable architectures trained end-to-end.These approaches do not impose a specific probabilistic or fuzzy semantics.Source: §5 · p. 4
6 Learning parameters or structure
StarAI distinguishes learning logical structure from estimating clause probabilities or weights. Neuro-symbolic approaches generally search clause parameters within templates or sketches rather than searching the discrete space of possible clauses, and some combine neural guidance with symbolic search or execution.
6 Learning parameters or structure
- StarAI separates structure learning—the learning of logical clauses—from parameter learning, which estimates clause probabilities or weights.The passages cite Kok and Domingos (2005) for structure learning and Gutmann et al. (2008) and Lowd and Domingos (2007) for parameter learning.Source: §6 · p. 4
- Neuro-symbolic approaches typically search clause parameters within enumerated templates rather than searching the discrete space of possible clauses.Alternatively, a program sketch can leave decisions blank for a system such as DeepProbLog or ∂4 to fill.Source: §6 · p. 4
- Hybrid approaches use neural models either to guide symbolic search or to produce programs that are then executed symbolically.The passage gives separate examples for neural guidance and neural program production.Source: §6 · p. 4
7 Symbols vs Sub-symbols
StarAI commonly represents entities as symbols, while neural methods also use vectorized sub-symbols, with implications for generalization to unseen entities. Reasoning can operate symbolically through unification, numerically through embeddings, or through methods that combine both.
7 Symbols vs Sub-symbols
- Neural methods may replace entity symbols with vectorized sub-symbols, and unseen entities can require new learned embeddings, affecting generalization.Sub-symbols can be inherent features such as image pixels, one-hot encodings, or learned embeddings.Source: §7 · p. 4
- Unification supports symbolic equality, while embedding-based reasoning uses algebraic operations and distance-based similarity between entities.The survey identifies learned embeddings as another representation approach [Minervini et al., 2017; Demeester et al., 2016].Source: §7 · p. 5
- NTPs and NLProlog use soft unification, allowing similar rather than only identical entities to unify and interweaving symbols with sub-symbols during inference.Similarity is computed from embedding distance using a radial basis function; the additional proof paths can create computational issues for larger programs, addressed in later iterations [Minervini et al., 2020].Source: §7 · p. 5
8 Type of logic
Neuro-symbolic representations extend propositional, relational, first-order, or logic-programming forms of logic. The choice of logic affects semantics and the kinds of structured representations available, and the survey locates representative systems across these types.
8 Type of logic
- Logical representations range from propositional to relational logic, first-order logic, and logic programs.Relational logic has no structured terms, while logic programs are usually restricted to definite clauses.Source: §8 · p. 5
- Logic programs use the least Herbrand model of entailed ground facts, whereas first-order logic allows other models; Markov Logic uses maximum-entropy semantics when a theory has multiple models [De Raedt and Kimmig, 2015; De Raedt et al., 2016].Logic programs also support structured terms such as lists, which is relevant to program synthesis from examples.Source: §8 · p. 5
- The survey places SL in propositional logic; ∂ILP, NTPs, and DiffLog in Datalog; LTNs and SBR in fuzzy first-order logic; and DeepProbLog, NLProlog, and LRNN in logic programming.RNM and NMLN use first-order logic to generate a random field.Source: §8 · p. 5
9 Open challenges
The survey identifies open challenges for neuro-symbolic AI in probabilistic integration, structure learning, scalable inference, data efficiency, and symbolic representation learning. It highlights complementary strengths of StarAI and neural methods as a direction for further research.
Probabilistic reasoning
- The authors consider probabilistic reasoning a promising way to integrate logical and neural methods and call for further investigation of its applicability to NeSy.They note that relatively few methods explore this integration through probabilities [De Raedt et al., 2019].Source: §9 · p. 5
Structure learning
- Learning StarAI model structure remains challenging because inference is complex and the problem combinatorial, while adding neural aspects increases the difficulty.Existing NeSy methods show potential but remain limited and mostly domain-specific, impeding wide application.Source: §9 · p. 5
Scaling inference
- Scalable inference remains a major challenge for StarAI and NeSy with explicit logical or probabilistic reasoning, with lifted or approximate inference and intermediate-representation reasoning proposed for investigation.The survey also proposes investigating whether neural methods can help address this challenge.Source: §9 · p. 5
Data efficiency
- StarAI methods learn efficiently from small datasets but do not scale to large ones, whereas neural methods are data-hungry but can handle large datasets.Understanding how the methods can help overcome these complementary weaknesses is a promising research direction.Source: §9 · pp. 5–6
Symbolic representation learning
- Changing data representations helps deep learning solve target tasks, and changing representations at the symbolic level could significantly increase NeSy system capabilities.Neurally inspired methods could help make progress on this major open challenge [Cropper, 2019; Dumanˇci´c et al., 2019].Source: §9 · p. 6