PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Probabilistic Logic Programs with Functional Gradient Guided Language Models

Saurabh Mathur, Sahil Sidheekh, Bhavan Vasu, Farbod Tavakkoli, Prasad Tadepalli, Kristian Kersting, Sriraam Natarajan

arXiv:2610.12303v1cs.AI

TL;DR

Inducing probabilistic logic programs from relational data remains difficult because symbolic search grows exponentially and standalone LLM proposals lack training-data grounding. GRASP uses functional gradient boosting to guide LLM-proposed rules and symbolically evaluate them. On Tox21, mutagenesis, and Cora, it outperforms symbolic, neural, and LLM-based baselines while producing interpretable weighted rule ensembles.

  • Problem

    Inducing probabilistic logic programs from data remains difficult because rule search grows exponentially, while standalone LLM proposals lack grounding in the training distribution.

    Source: §1 · p. 2

  • Method

    GRASP casts relational structure learning as functional gradient boosting, using an LLM to propose first-order rules that are symbolically evaluated and added when they reduce residual loss.

    Source: §1 · p. 2

  • Results

    On Tox21, mutagenesis, and Cora, GRASP outperforms symbolic, neural, and LLM-based baselines while producing interpretable weighted rule ensembles.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Analysis of the induced rules finds relational patterns consistent with established domain knowledge.

    Source: §5 · p. 14

    GRASP currently operates over symbolic predicates on noisy structured relational data; extending it to neural predicates for images and text remains future work.

    Source: §5 · p. 14

Abstract

from arXiv · show

Declarative logic programs offer a powerful and interpretable abstraction for encoding relational structure and neurosymbolic reasoning, by expressing dependencies as weighted compositional rules. However, inducing them from data remains fundamentally hard, bottlenecked by the combinatorial explosion of symbolic search spaces. LLMs have recently emerged as powerful hypothesis generators, but when used in isolation, they lack the capacity to do systematic inductive reasoning needed to reliably synthesize valid programs that fit complex relational distributions. We introduce grasp (Gradient-boosted Synthesis of Probabilistic logic programs), a neurosymbolic framework that casts relational structure learning as functional gradient boosting in which the weak learner is a first-order rule and the intractable inner search is delegated to an LLM proposal oracle. We evaluate grasp on four relational benchmarks spanning molecular toxicity prediction (Tox21), mutagenesis, and citation matching (Cora), and show that it improves over purely symbolic, neural, and LLM-based baselines, while producing interpretable weighted rule ensembles. By replacing combinatorial search with gradient-guided LLM hypothesis generation, grasp retains boosting guarantees without sacrificing the transparency of symbolic outputs.

1 Introduction

Inducing probabilistic logic programs from relational data is difficult because symbolic rule search grows rapidly and standalone LLM proposals lack grounding in the training distribution. GRASP uses functional gradient boosting to guide and verify LLM-proposed rules, and is reported to outperform symbolic, neural, and LLM-based baselines on the evaluated datasets.

1 Introduction

  • Inducing program structure requires searching a hypothesis space that grows exponentially with rule length, predicate arity, and relational depth.Functional gradient boosting decomposes the task into successive regression problems, but symbolic weak-learner search remains a bottleneck.Source: §1 · p. 2
  • Standalone LLMs can propose semantically plausible but inaccurate rules because they lack a mechanism to ground generation in the training distribution.Generated clauses also lack guarantees of validity, executability, or consistency with observed data.Source: §1 · p. 2
  • GRASP frames relational structure learning as functional gradient boosting, using an LLM to propose first-order rules and symbolic evaluation to select and weight them.A proposed rule is admitted to the ensemble only if it reduces residual loss.Source: §1 · p. 2
  • Pseudo-residual-guided example selection steers rule proposals toward relational variance that the current ensemble has not explained.The framework selects examples carrying the largest functional gradient and conditions proposals on their local relational descriptions.Source: §1 · p. 2
  • GRASP outperforms symbolic, neural, and LLM-based baselines on Tox21, mutagenesis, and Cora while producing interpretable weighted rule ensembles.The introduction reports evaluation across three relational datasets.Source: §1 · p. 3

2 Background and Related Work

The background frames probabilistic relational learning as clause-structure induction under uncertainty, where gradient boosting reduces global search but relational weak-learner induction remains a bottleneck. Neurosymbolic and LLM approaches address parts of the problem, while the paper treats explainability as more than simply using rules.

2.1 Statistical Relational Learning

  • Classical ILP accepts clauses only when they cover all positive examples and no negatives, a brittle assumption for noisy or incomplete data; SRL adds probabilistic reasoning.SRL represents clauses with real-valued weights and uses a sigmoid transformation to produce graded prediction probabilities.Source: §2.1 · p. 3
  • Structure learning induces clause structure from data, but its hypothesis space grows exponentially; boosting fits weak learners to residuals while relational regression-tree induction retains symbolic search.Boosting adds a learner at each step to address poorly explained examples, but inducing that relational learner remains a computational bottleneck as complexity grows.Source: §2.1 · p. 4

2.2 Neurosymbolic Learning

  • Neural predicates let neural networks annotate logical facts with learned probabilities, but they do not solve program-structure induction; Neural LP also restricts rules to predefined templates and binary predicates.Higher-arity relations in Neural LP require decomposition into binary predicates using composite entities.Source: §2.2 · p. 4
  • ChatRule proposes and ranks rules in one shot, whereas grasp couples LLM generation with functional-gradient feedback grounded in the training distribution.The related-work passage describes prior LLM-generated rules as potentially statistically uninformative without that grounding.Source: §2.2 · p. 5

2.3 Explainability

  • The paper's explainability criteria include succinctness, broad coverage, simulatability, decomposability, and weights that meaningfully map to confidence or probability.Rules should let a human trace their conditions and weights to verify an output without external computation.Source: §2.3 · p. 5

3 grasp: Gradient-Boosted Synthesis of Probabilistic Logic Programs

grasp frames relational structure learning as an additive boosting process in which an LLM proposes first-order clauses. It enriches and constrains the relational vocabulary, supplies compact descriptions of high-residual examples, then scores candidate rules against residuals before updating the ensemble.

3 grasp: Gradient-Boosted Synthesis of Probabilistic Logic Programs

  • grasp learns an additive ensemble of first-order-clause scoring functions to minimize empirical log-loss, using LLM-proposed clauses within a functional-gradient boosting loop.Each iteration evaluates a proposed clause against current pseudo-residuals and adds it with a learned weight.Source: §3 · p. 6

3.1 Relational Abstraction and Aggregation

  • A one-time abstraction phase generates aggregate predicates and a typed clause template, giving the LLM compact relational patterns and constraining proposals to semantically admissible clauses.Aggregates capture higher-order structure such as shared intermediate entities, relational attributes, and neighborhood cardinalities; derived predicates are checked for syntax and execution.Source: §3.1 · pp. 6–7

3.2 Relational Context Construction

  • At each iteration, grasp selects the k examples with largest pseudo-residual magnitude and presents their compact local relational descriptions as the gradient signal to the LLM.The selected examples are those whose prediction error would benefit most from a new rule.Source: §3.2 · p. 7

3.3 Rule Proposal and Evaluation

  • The LLM proposes clauses from the schema, template, domain, ensemble, and selected examples; each rule scores covered and uncovered examples, and the lowest-squared-residual proposal updates the ensemble.The optimal leaf weights are the mean residuals for covered and uncovered examples, respectively, so every rule contributes a prediction for every example.Source: §3.3 · pp. 7–8

4 Empirical Evaluation

The empirical section evaluates GRASP's predictive performance and gradient-guided steering, then examines rule importance, compactness, and domain alignment across three relational benchmarks.

  • Evaluation covers Mutagenesis, Cora, and 12 Tox21 targets, reports three-trial AUC-ROC and AUC-PR, and caps GRASP and BoostedRDN ensembles at 20 rules.Source: §4 · pp. 8–10
  • GRASP variants jointly recorded the highest AUC-ROC on 10 and AUC-PR on 12 of 14 datasets.Source: Table 1 · p. 9, Table 2 · p. 10, §4 · p. 10
  • Top rules recover domain-aligned patterns: mutagenicity tracks large molecules with low electron affinity, while Tox21 rules reflect endpoint-associated structures and.Source: Table 3 · p. 11, §4 · pp. 13–14
  • One-shot LLM was consistently weakest across both metrics, sometimes yielded no valid rules, and reached standard deviations of 21.6 AUC-ROC and 17.8 AUC-PR on Cora, versus below 3 for GRASP variants.Source: §4 · p. 11
  • Performance peaked within the first 20% of ensembles, with 2 to 4 high-weight rules; Codex's initial deletion robustness reflects redundant structural variants.The insertion and deletion curves assess rules ordered by absolute discriminative weight.Source: §4 · pp. 11, 13, Figure 2 · p. 12

5 Conclusion

Across citation matching, mutagenesis, and molecular toxicity, GRASP outperforms classical symbolic and neural baselines under a shared interpretability constraint, with gradient-guided steering essential to reliable rule synthesis. Future work includes neural predicates for unstructured inputs and fine-tuning the black-box LLM proposer.

5 Conclusion

  • Across citation matching, mutagenesis, and molecular toxicity, GRASP outperforms classical symbolic and neural baselines under a shared interpretability constraint.Source: §5 · p. 14
  • Gradient-guided steering is essential for reliable rule synthesis, and induced rules recover relational patterns consistent with established domain knowledge.Source: §5 · p. 14
  • GRASP currently operates over symbolic predicates defined over noisy structured relational data; neural predicates could extend it to images and text.Source: §5 · p. 14
  • The interpretable ensembles offer a path to human-allied learning, where experts can inspect and refine rules, aggregation predicates, and entity representations.Source: §5 · p. 14
  • GRASP currently treats the LLM as a black-box proposer, and fine-tuning it could improve proposed-rule quality and validity.Source: §5 · p. 14

Acknlowedgements

The acknowledgements recognize research funding, internship support, and experimental infrastructure.

Acknowledgements

  • Sidheekh and Natarajan acknowledge AFOSR, ARO, and DARPA support; Mathur and Kersting acknowledge the DFG-funded Reasonable AI Cluster of Excellence.Source: Acknlowedgements · p. 14
  • Vasu and Tadepalli acknowledge Army Research Office support under grant W911NF2210251.Source: Acknlowedgements · p. 14
  • The work was partly conducted during Sidheekh’s AT&T summer internship funded through Tech Mahindra, with H100 GPU infrastructure and API credits provided by AT&T CDO.Source: Acknlowedgements · p. 14
Loading 2610.12303v1…