PaperlayerView exact revision on arXiv

Source-linked AI summary

A Foundation Model for Zero-Shot Logical Rule Induction

Yin Jun Phua

arXiv:2605.04916v2cs.AIcs.LGcs.SC

TL;DR

Inductive Logic Programming learns interpretable rules, but existing differentiable methods remain tied to specific predicates and require retraining for new tasks. NRI encodes literals through statistical properties and induces rules zero-shot on diverse real-world benchmarks after training on synthetic Boolean formulas.

  • Problem

    Existing differentiable ILP methods tie learned weights to specific predicates, so transferring across domains requires retraining.

    Source: §1 · p. 1

  • Method

    NRI encodes literals through statistical properties rather than identities and uses a parallel slot-based decoder to induce rules from synthetic Boolean formulas.

    Source: §4.2 · p. 3

  • Results

    A model trained entirely on synthetic Boolean formulas performs zero-shot rule induction on diverse real-world benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors view NRI as opening the possibility of foundation models for symbolic reasoning, while this work focuses on Boolean variables.

    Source: §1 · p. 2

    Cross-task transfer assumes binarized tasks admit sparse bounded-DNF descriptions, and the paper provides no formal cross-task guarantees.

    Source: §4.1 · p. 3

Abstract

from arXiv · show

Inductive Logic Programming (ILP) learns interpretable logical rules from data. Existing methods are transductive: their learned parameters are bound to specific predicates and require retraining for each new task. We introduce Neural Rule Inducer (NRI), a pretrained model for zero-shot rule induction. Rather than encoding literal identities, NRI represents literals using domain-agnostic statistical properties such as class-conditional rates, entropy, and co-occurrence, which generalize across variable identities and counts without retraining. The model consists of a statistical encoder and a parallel slot-based decoder. Parallel decoding preserves the permutation invariance of logical disjunction; an autoregressive decoder would instead impose an arbitrary clause order. Product T-norm relaxation makes rule execution differentiable, allowing end-to-end training on prediction accuracy alone. We evaluate NRI on rule recovery, robustness to label noise and spurious correlations, and zero-shot transfer to real-world benchmarks, and we believe this work opens up the possibility of foundation models for symbolic reasoning. Code and the reference checkpoint are available at https://github.com/phuayj/neural-rule-inducer.

1 Introduction

NRI uses domain-agnostic statistics to induce logical rules across unfamiliar predicates, and the paper reports zero-shot rule induction on real-world benchmarks after synthetic-only training. Its design targets variable-size problems, inter-variable dependencies, and transfer from synthetic to real data.

Introduction

  • NRI is motivated by the value of interpretable logical rules in high-stakes domains, where black-box predictions are unacceptable.Traditional symbolic ILP methods are described as noise-sensitive and often forced to compromise between precision and coverage.Source: §1 · p. 1
  • NRI represents literals by statistical signatures rather than names, enabling transfer across renamed or reordered variables and held-out real-world tasks without retraining.A high class-conditional rate can indicate that a variable belongs in a rule, regardless of what it represents.Source: §1 · p. 1
  • The introduction identifies variable-count generalization, inter-variable redundancy or complementarity, and synthetic-to-real transfer as three challenges for the approach.The cited examples include XOR patterns as a case where individual statistics can miss dependencies.Source: §1 · p. 1
  • NRI combines a statistics encoder, a parallel slot-based clause decoder, and product t-norm execution for differentiable end-to-end training without clause supervision.Parallel decoding preserves the permutation invariance of logical disjunction, unlike autoregressive clause generation.Source: §1 · p. 1
  • Training on randomly generated Boolean formulas aims to teach the induction procedure rather than domain-specific patterns and removes the limitation on training-data size.The paper focuses on Boolean variables while noting possible extension to multi-valued and continuous domains.Source: §1 · pp. 1–2
  • A model trained entirely on synthetic Boolean formulas performs zero-shot rule induction on diverse real-world benchmarks.The statistical encoding framework is also described as extendable to multi-valued and continuous domains through discretization or fuzzy predicates.Source: §1 · p. 2

2 Related Works

Prior work spans differentiable logic systems and language-mediated hypothesis generation; this work instead uses statistical properties for transfer and generates interpretable rules directly from observed data.

Differentiable ILP

Generative Neuro-Symbolic AI

  • LLM-based ILP-CoT [Peng et al., 2025] and DeepSeek-Prover-V2 [Ren et al., 2025] draw on language knowledge, while LINC [Olausson et al., 2023] translates natural language into logic for external verification.The passage cautions that LLM concepts may not correspond to measurable real-world quantities; this approach instead generates hypotheses from observed data without an explicit semantic model.Source: §2 · p. 2
  • TabPFN [Hollmann et al., 2023] uses transformers trained on synthetic tabular data for in-context classification, but produces black-box predictions rather than interpretable logical rules.Source: §2 · p. 2

3 Background

The background frames the task as mapping Boolean examples and labels to DNF rules that work across variable counts and identities without retraining.

3 Background

  • The target is a function from Boolean examples and labels to a logical DNF hypothesis that works across different variable counts and identities.The aim is to avoid retraining for each new problem.Source: §3 · p. 2

3.1 Disjunctive Normal Form (DNF)

  • DNF expresses any Boolean function as a disjunction of conjunctions, with each literal being a variable or its negation.Learning DNF rules can therefore represent any propositional rule.Source: §3.1 · p. 2

3.2 T-Norms

  • Product t-norm relaxes logical conjunction to continuous truth values, enabling differentiable evaluation of DNF formulas.The passages describe clause truth from its literals and formula truth from its clauses; t-norms are cited as.Source: §3.2 · p. 2

3.3 Inductive Logic Programming

  • ILP seeks hypotheses that entail positive examples while remaining consistent with negative examples; this work treats examples as complete Boolean assignments classified by DNF rules.This setting follows learning from interpretations, rather than learning from entailment over ground facts.Source: §3.3 · p. 2

3.4 Foundation Models

  • The foundation-model approach trains on diverse synthetic formulas and learns literal statistics rather than predicate-specific weights, enabling rule induction in new domains without retraining.The described architecture encodes class-conditional rates, entropy, and co-occurrence, then uses cross-attention and a parallel slot-based decoder to produce candidate clauses for differentiable rule evaluation.Source: §3.4 · pp. 2–3, Figure 1 · p. 3

4 Neural Rule Inducer (NRI)

NRI induces bounded DNF rules using identity-free literal statistics, example-conditioned information, and parallel clause slots. Differentiable T-norm execution enables end-to-end training, while transfer across tasks relies on the assumption that they admit sparse bounded-DNF descriptions.

4 Neural Rule Inducer (NRI)

  • NRI replaces predicate-specific weights with domain-agnostic literal statistics, then combines statistical encoding, example attention, slot-based decoding, and differentiable T-norm evaluation.The statistical features include class-conditional truth rates, entropy, polarity, and co-occurrence; the full vector has 18 components.Source: §4 · p. 3, §4.1 · p. 3, §4.2 · p. 3

4.1 Problem Formulation

  • NRI treats training as meta-learning over episodes and predicts bounded DNFs; its cross-task transfer assumption is sparse bounded-DNF structure, without formal guarantees.Evaluation uses a fixed collection of UCI tasks, with within-task support-to-held-out generalization in the usual i.i.d. setting.Source: §4.1 · p. 3

4.2 Literal Statistics Encoder

  • Encoding literal statistics across examples supports zero-shot generalization under missing or noisy data, though weaker observation rates reduce the signal available to estimate those statistics.Missingness is reflected in observation-rate features, while noise moves truth rates toward 0.5 and increases entropy.Source: §4.2 · pp. 3–4

4.3 Example-Conditioned Encoding

  • Because aggregate statistics can miss which positive examples literals cover, an example-conditioned branch restores support-pattern information; its dimension adaptation is a pragmatic approximation, not an invariance guarantee.The branch zero-pads when the target has fewer variables and expands its first layer with randomly initialized weights when it has more.Source: §4.3 · p. 4

4.5 Slot-Based Decoder

  • FiLM gives clause slots distinct views to prevent identical patterns, while parallel slot decoding preserves permutation invariance of disjunctions.For example, A ∨ B and B ∨ A should represent the same rule.Source: §4.4 · p. 4, §4.5 · p. 4

Clause Gates (“OR” Level)

4.6 Neuro-Symbolic Execution

  • Product T-norm execution combines literal and clause gates into predictions, and the complete computation is differentiable for end-to-end training.A prediction is high when at least one active clause is satisfied.Source: §4.6 · pp. 4–5

4.8 Training Objective

  • Training combines synthetic pretraining, opposing spurious-feature environments, and clause dropout; slot balancing reduced utilization variance from 0.35 to 0.003, while max-margin stabilized entropy near 0.26 and counterfactual losses penalized literal inconsistency.The counterfactual objective tests whether selected literals are necessary and whether ignored literals can change predictions.Source: §4.7 · p. 5, §4.8 · p. 5

5 Experiments

NRI is trained only on synthetic Boolean expressions, then tested zero-shot on UCI datasets and benchmarks of rule complexity, noise robustness, and spurious-feature robustness.

5.2 Rule Complexity Scaling

  • At N=12, logical match falls from 99.5% for K=1, L=1 to 24.0% for K=4, L=3, while prediction accuracy remains 85–100%.Increasing clause count has a larger effect than increasing literals per clause: K=4, L=1 yields 34.2% recovery, while K=1, L=3 yields 81.7%.Source: §5.2 · p. 6

5.3 Label Noise Robustness

  • At 30% label noise, NRI retains 87.4% accuracy versus 70.3% for RIPPER and 69.9% for DT, outperforming both baselines by 17 percentage points.NRI accuracy changes from 92.3% to 87.4%; the symbolic baselines are more accurate at low noise, but degrade sharply beyond 15% noise.Source: §5.3 · p. 6, Figure 3 · p. 7

5.4 Spurious Variable Robustness

  • NRI accuracy stays above 92% across distractor settings, including 97.6% at D=32, ρ=0.9 and 96.7% at D=32, ρ=0.1.The distractors are statistically predictive but causally irrelevant to the true rule.Source: §5.4 · p. 6, Figure 4 · p. 7

5.1 Baseline Context

  • NRI’s zero-shot UCI accuracy is 69.7%, 13 points below dataset-trained EBM; it reaches 68.0% on diabetes and 88.3% on breast cancer.The evaluation covers 14 UCI datasets, 12 of which have more features than the training range; each method is evaluated with 5-fold stratified cross-validation.Source: §5.1 · p. 6, Table 1 · p. 7

5.5 Computational Scaling

  • At N=512, inference finishes in under 12ms with 593MB peak memory; latency grows sub-linearly with N and stays near 7.5ms as M increases from 32 to 512.Memory scales as O(N^2) due to attention.Source: §5.5 · p. 7

5.6 Loss Ablation Study

  • Removing CF Necessity, Max-Margin Coverage, or Slot Balance lowers UCI accuracy by 0.7%, 2.8%, and 1.5%, respectively; the full model reaches 74.8%.The losses affect literal selection and characteristic failure modes, while rule complexity remains similar across ablations; the full model maintains interpretable rule sizes.Source: §5.6 · p. 7, Table 2 · p. 7

6 Conclusion

Neural Rule Inducer (NRI) is a foundation model trained on synthetic Boolean formulas that induces interpretable DNF rules on new domains without retraining. Its identity-free statistical encoding and parallel slot-based decoder enable end-to-end training on prediction accuracy alone; future work will extend it to richer variable types and logic.

6 Conclusion

  • NRI induces interpretable DNF rules on new domains without retraining after training entirely on synthetic Boolean formulas.The work presents this as a foundation model for zero-shot rule induction.Source: §6 · p. 7
  • Identity-free literal statistics and a parallel slot-based decoder under product T-norm semantics let NRI train end-to-end on prediction accuracy alone.The encoded properties include class-conditional rates, entropy, and co-occurrence.Source: §6 · p. 7
  • Future work will extend NRI to multi-valued and continuous variables and to first-order logic with relational predicates.The authors say this work opens up the possibility of foundation models for symbolic reasoning.Source: §6 · p. 7

A Synthetic Data Generation … A.4 Spurious Environment Features

Because real-world datasets are limited in quantity and coverage, the paper trains NRI on synthetically generated episodes spanning possible logical rules.

A Synthetic Data Generation

  • Training uses exclusively synthetic episodes to provide diverse examples across the space of possible logical rules.The paper motivates this choice by the limited quantity and coverage of real-world datasets.Source: A Synthetic Data Generation · p. 9

A.1 Episode Structure

  • Each episode contains Boolean features, labels, and a ground-truth DNF rule; experiments sample 6–12 causal variables and 24–48 examples.The feature matrix includes N causal and S spurious variables, while N and M are sampled uniformly from their stated ranges.Source: §A.1 · p. 9

A.2 DNF Rule Sampling

  • Rules sample 1–6 clauses of up to four distinct variables each, assign each literal polarity with probability 0.5, and retain possible redundancy without canonicalization.Clause count and length are sampled uniformly, with length capped by the number of causal variables.Source: §A.2 · p. 9

A.3 Example Generation

  • Causal features are sampled independently as Bernoulli(0.5), and labels are computed from the rule, producing class ratios that vary with rule structure.A single clause of length L has positive-label probability 2^-L; additional clauses raise the positive rate through their union, partly offset by overlap.Source: §A.3 · p. 9

A.4 Spurious Environment Features

  • Spurious features have opposite label correlations across two example halves: in environment 1, P(s=1|Y=1)=ρ and P(s=1|Y=0)=1−ρ; in environment 2, these probabilities reverse (experiments use S=3 and ρ=0.3).Spurious variables are concatenated with causal features, then all examples are randomly permuted to hide environment boundaries.Source: §A.4 · pp. 9–10
  • Across the full dataset, each spurious feature is marginally independent of the label, so detection requires identifying its inconsistent performance across environments.The paper says the model detects this pattern through encoder attention over examples and a counterfactual necessity loss penalizing features whose removal does not change predictions.Source: §A.4 · p. 10

A.5 Data Generation Parameters

The data generator targets sparse, interpretable DNF rules with bounded complexity and moderate-sized variable sets, rather than uniformly sampling all Boolean functions.

A.5 Data Generation Parameters

  • The generator covers sparse DNF rules with K ≤6, L ≤4, and N ≤12, targeting interpretable rules rather than the full Boolean-function space.Many Boolean functions require exponentially many DNF clauses.Source: §A.5 · p. 10

B Literal Feature Vector

Each literal receives an 18-dimensional feature vector computed from the episode, combining truth-rate, co-occurrence, and observation statistics. The features capture class-specific literal relationships and account for missing values when estimating truth rates.

B Literal Feature Vector

  • For each positive or negated literal, NRI computes an 18-dimensional feature vector from the episode (X, Y).The literals comprise both xi and ¬xi for each variable xi.Source: B Literal Feature Vector · p. 10
  • Centered-covariance co-occurrence features include class-specific statistics that help identify literals participating in conjunctive patterns within a class.Table 4 describes co-occurrence as centered covariance between literal truth values.Source: B Literal Feature Vector · p. 10, Table 4 · p. 10
  • When literal values are unknown, observation-rate features quantify the effective sample size for truth-rate estimates, letting downstream layers discount statistically thin literals.Unknown values are excluded from truth-rate calculations but contribute to observation rates.Source: B Literal Feature Vector · p. 10

C Auxiliary Terms in Lcf … E.3 Rule-Based Classifiers

The appendices describe counterfactual-objective regularizers, computational scaling, and the per-dataset baselines used to compare NRI, including non-interpretable accuracy ceilings, additive models, and rule-based classifiers.

C Auxiliary Terms in Lcf

  • Clause coverage overlap (λo = 0.1) penalizes clauses firing on the same positive, while counterfactual load balance (λc = 0.01) encourages multiple clauses to explain each positive.Load balance is the negative mean per-positive responsibility entropy within the counterfactual objective.Source: C Auxiliary Terms in Lcf · p. 10

D Computational Scaling

  • ~7.5 ms inference latency remains nearly constant as examples increase from 32 to 512; memory scales as O(N^2) with schema size.For N=16 to N=512, latency increases from 4.2 ms to 11.8 ms; at N=512, inference takes under 12 ms with 593 MB peak memory. Figure 5 attributes quadratic memory scaling to attention over 2N literals.Source: D Computational Scaling · p. 11, Figure 5 · p. 11

E.1 Non-Interpretable Ceilings

  • XGBoost [Chen, 2016] and LightGBM serve as non-interpretable accuracy ceilings rather than human-readable rule models.XGBoost is described as a highly optimized boosted-tree implementation; LightGBM uses histogram-based algorithms and leaf-wise tree growth.Source: §E.1 · p. 11

E.2 Generalized Additive Models

  • Generalized additive models use feature-specific shape functions, while EBM [Nori et al., 2019] provides feature-effect graphs but not explicit logical rules.EBM may include pairwise interaction terms that reduce transparency.Source: §E.2 · p. 11

E.3 Rule-Based Classifiers

  • RIPPER [Cohen and others, 1995] greedily grows and prunes an ordered list of if-then rules; RuleFit [Friedman and Popescu, 2008] weights extracted rules in a sparse linear model.RuleFit’s weighted combination of many rules can reduce transparency compared with pure rule lists.Source: §E.3 · p. 11
  • FIGS sums shallow decision trees, while CART-style decision trees provide hierarchical rules and can be converted into DNF path clauses.The DT baseline uses scikit-learn defaults; DT-DNF extracts the conjunction along each path to a positive leaf, making each path a clause.Source: §E.3 · p. 11

E.4 Neural DNF Methods · E.5 Summary

Neural DNF methods use differentiable approximations of logical operations for gradient-based rule learning. The section also identifies a scratch-trained neural DNF baseline and summarizes the experiment’s baselines by output type and interpretability.

E.4 Neural DNF Methods

  • Neural DNF learning uses differentiable approximations of logical operations to optimize rule structures with gradients.Source: §E.4 · p. 11
  • Neural DNF (Scratch) learns DNF rules with sigmoid gates approximating AND/OR and is trained from scratch on each dataset, without synthetic-data pretraining.It serves as a baseline for comparing per-dataset neural DNF learning with zero-shot transfer.Source: §E.4 · p. 12

E.5 Summary

F NRI Predicted Rules

The section lists NRI-predicted rules for UCI datasets using 5% training data and auto-tuned clause selection. Duplicate clauses were removed from the displayed rules.

Predicted rules

  • The examples include single-clause rules for adult, breast-cancer-wisconsin, and diabetes, alongside multi-clause rules for car, credit-approval, and german-credit.For example, the adult rule conjoins negated marital-status and relationship conditions with ¬sex Female; car rules are shown for four classes.Source: F NRI Predicted Rules · p. 12
  • The section also shows multi-clause rules for hepatitis, ionosphere, kr-vs-kp, and mushroom, plus nursery clause counts across five classes.Examples include a two-clause ionosphere rule and a six-clause kr-vs-kp rule.Source: F NRI Predicted Rules · p. 12
  • For nursery, NRI predicts rules for five classes, including an empty rule for recommend.The listed clause counts are 1+2+0+1+1; other displayed class rules use conjunctions of conditions, with priority given two disjunctive clauses.Source: F NRI Predicted Rules · p. 12
  • Spambase, tic-tac-toe, and vote each have a displayed one-clause rule.The spambase rule negates four word-frequency conditions; the tic-tac-toe and vote rules also combine feature conditions in a conjunction.Source: F NRI Predicted Rules · p. 12

Related papers

Loading 2605.04916v2…