A Foundation Model for Zero-Shot Logical Rule Induction

Yin Jun Phua

Source-linked AI summary

A Foundation Model for Zero-Shot Logical Rule Induction

Yin Jun Phua

arXiv:2605.04916v2May 6, 2026cs.AIcs.LGcs.SC

TL;DR

Existing ILP methods are tied to specific predicates and require retraining for each new task. NRI uses identity-free statistical encoding and parallel rule decoding to induce rules zero-shot, including on diverse real-world benchmarks.

Problem
Existing differentiable ILP methods remain transductive, tying learned weights to specific predicates and requiring retraining for every new dataset.
Method
NRI encodes literals with identity-free statistics and synthesizes differentiable DNF rules through a parallel slot-based decoder.
Results
A model trained entirely on synthetic Boolean formulas performs zero-shot rule induction on diverse real-world benchmarks.
Takeaways & Limitations
NRI demonstrates an empirical route toward foundation models for symbolic reasoning across new domains without retraining.
Takeaways & Limitations
Evaluation uses a fixed collection of UCI tasks and assumes many binarized tasks admit sparse bounded-DNF descriptions, without formal cross-task guarantees.

Abstract

from arXiv · show

Inductive Logic Programming (ILP) learns interpretable logical rules from data. Existing methods are transductive: their learned parameters are bound to specific predicates and require retraining for each new task. We introduce Neural Rule Inducer (NRI), a pretrained model for zero-shot rule induction. Rather than encoding literal identities, NRI represents literals using domain-agnostic statistical properties such as class-conditional rates, entropy, and co-occurrence, which generalize across variable identities and counts without retraining. The model consists of a statistical encoder and a parallel slot-based decoder. Parallel decoding preserves the permutation invariance of logical disjunction; an autoregressive decoder would instead impose an arbitrary clause order. Product T-norm relaxation makes rule execution differentiable, allowing end-to-end training on prediction accuracy alone. We evaluate NRI on rule recovery, robustness to label noise and spurious correlations, and zero-shot transfer to real-world benchmarks, and we believe this work opens up the possibility of foundation models for symbolic reasoning. Code and the reference checkpoint are available at https://github.com/phuayj/neural-rule-inducer.

1 Introduction

The introduction motivates Neural Rule Inducer (NRI) as a foundation model for interpretable, zero-shot logical rule induction that avoids predicate-specific retraining. NRI uses domain-agnostic statistics, permutation-invariant parallel decoding, differentiable rule execution, and synthetic training to transfer from Boolean formulas to real-world benchmarks.

Motivation: ILP learns interpretable logical rules from examples, which are valuable when black-box predictions are unacceptable in high-stakes domains.The passage illustrates this with a symptom-based rule predicting flu.
Problem: Differentiable ILP methods are more noise-robust but remain transductive because learned weights are tied to specific predicates, requiring retraining for each dataset.A model trained on family relationships cannot directly transfer to biology.
Approach: NRI represents literals with statistical signatures that are invariant to renaming and reordering and use fixed-dimensional vectors independent of the number of variables.Examples include class-conditional rates and co-occurrences; high class-conditional rates can identify rule-relevant variables regardless of their meaning.
Challenges and training: NRI addresses variable-sized problems, inter-variable dependencies, and synthetic-to-real transfer while learning from randomly generated Boolean formulas rather than domain-specific patterns.These challenges include generalizing across variable counts, capturing redundancy and complementarity such as XOR, and learning induction procedures.
Architecture: NRI synthesizes multiple clauses in parallel with learned slot queries, preserving the permutation invariance of logical disjunction.Parallel slot-based decoding avoids imposing an arbitrary clause order as autoregressive decoding would.
Training and transfer: Product t-norm relaxation executes rules differentiably, enabling end-to-end training without explicit clause supervision; synthetic-only training transfers zero-shot to diverse real-world benchmarks.The statistical encoding framework can also extend beyond Boolean variables through discretization or fuzzy predicates.

2 Related Works

Prior differentiable and classical ILP methods are generally tied to fixed predicates, schemas, or tasks, while recent work explores zero-shot transfer and generative neuro-symbolic reasoning. NRI differs by using statistical properties for noisy or missing data and producing interpretable logical rules.

Differentiable ILP: Differentiable ILP methods learn rules through gradient descent or differentiable reasoning, but typically require fixed rule templates, background predicates, or predicate inventories.NeuralLP and DRUM extend differentiable reasoning to knowledge bases, while Neural Theorem Provers introduce differentiable unification.
Differentiable ILP: Classical symbolic systems such as FOLD-R++ also remain task-specific and must be rerun from scratch on each new task.FOLD-R++ learns answer-set rules from mixed numerical and categorical data through top-down heuristic search.
Zero-Shot Transfer: Phua and Inoue proposed zero-shot transfer by exploiting variable permutation symmetries, whereas this approach uses statistical properties rather than raw examples to handle missing and noisy data.The supplied passage identifies this difference in mechanism directly.
Generative Neuro-Symbolic AI: Generative neuro-symbolic methods use language-model knowledge and human concepts to generate hypotheses or translate natural language into formal logic for verification.The passage names ILP-CoT, DeepSeek-Prover-V2, and LINC as examples.
Generative Neuro-Symbolic AI: Unlike black-box prediction systems, this approach outputs interpretable logical rules.The supplied passage contrasts the approach with TabPFN.

3 Background

This section frames rule induction as learning DNF hypotheses from boolean assignments and labels, with a function that generalizes across variable counts and identities without retraining. It also situates the approach in ILP and foundation-model paradigms, using differentiable product t-norm execution.

Problem formulation: The goal is to learn a function that maps boolean examples and labels to DNF hypotheses across varying numbers and meanings of variables without retraining.The desired function is f : (X, Y) → R, where X contains boolean variables and Y contains labels.
Problem formulation: DNF expresses rules as disjunctions of conjunctions, with each literal being a variable or its negation, and can represent any propositional rule.A DNF rule has the form R = C1 ∨ C2 ∨ · · · ∨ CK, with each clause formed from literals.
Differentiable rule execution: Product t-norm relaxes logical conjunction to continuous truth values, enabling differentiable evaluation of clauses and DNF formulas.The relaxed execution supports end-to-end training through differentiable rule evaluation.
Inductive Logic Programming: ILP learns hypotheses from positive examples, negative examples, and background knowledge subject to completeness and consistency.The hypothesis must entail positive examples while not entailing negative examples when combined with the background knowledge.
Inductive Logic Programming: The setting mainly follows learning from interpretations, where each example is a complete boolean assignment and the output DNF classifies positive examples.Learning from interpretations treats examples as complete states rather than isolated ground facts.
Foundation-model framing: NRI trains on millions of diverse synthetic boolean formulas and learns statistical patterns rather than predicate-specific weights, enabling rule induction in new domains without retraining.This applies the foundation-model paradigm of learning at scale and generalizing beyond the training distribution.

4 Neural Rule Inducer (NRI)

NRI is an end-to-end differentiable, domain-agnostic framework that learns logical rules from literal statistics rather than predicate identities. It combines statistical and example-conditioned encoders with permutation-invariant slot-based DNF decoding and differentiable product T-norm execution for zero-shot transfer.

Overview: NRI learns to select variables from domain-agnostic statistical properties instead of weights tied to specific predicates or domains.The framework is designed for zero-shot generalization and robustness to missing and noisy data by encoding each literal through statistics over all examples.
Architecture: The four-stage pipeline computes per-literal statistics, cross-attends them over examples, synthesizes a slot-based DNF, and evaluates it with differentiable T-norms.The statistical features have 18 components, including truth rates, entropy, polarity, and mean absolute co-occurrence.
Encoders: The example-conditioned encoder restores support-pattern information that aggregate literal statistics cannot guarantee, while dynamic dimension adaptation handles variable feature counts at inference.Inputs smaller than the training dimension are zero-padded, whereas larger inputs expand the first layer using copied trained weights.
Decoder: Parallel clause slots assemble DNF rules without imposing an arbitrary disjunction order, and FiLM differentiates slots so they can specialize.The decoder treats synthesis as slot filling, selecting literals and activating clauses while preserving A ∨B ≡B ∨A.
Rule Execution: The differentiable soft DNF computes literal and clause gates once per episode, combines clauses with probabilistic OR, and supports end-to-end training from prediction accuracy.At inference, clauses with gates wk ≥0.5 are retained; top-K′ filtering improved synthetic exact-match recovery but not UCI accuracy and is therefore disabled by default.
Training Objectives: CV2 reduced clause-slot utilization variance from 0.35 to 0.003, while max-margin coverage stabilized training at low entropy of ∼0.26.These objectives counteract task-loss gradients, encourage clause specialization, and allow multiple slots to cover the same pattern.

5 Experiments

NRI is evaluated zero-shot on out-of-distribution UCI schemas and synthetic rule-recovery, robustness, efficiency, and ablation benchmarks. It achieves strong robustness and interpretable predictions, while rule recovery declines with complexity and inference remains fast at large feature counts.

UCI evaluation: NRI is evaluated zero-shot on 14 UCI datasets, including 12 with more features than its N ∈[6, 12] training range, using 5-fold stratified cross-validation.Each fold uses approximately 20% of data as a support set and evaluates accuracy on the remaining approximately 80%.
UCI evaluation: 69.7% accuracy is achieved zero-shot, 13 points below EBM; performance is strongest on diabetes and breast-cancer, at 68.0% and 88.3% respectively.EBM is trained separately on each dataset, unlike NRI.
Rule recovery: 99.5% logical match for K=1, L=1 falls to 24.0% for K=4, L=3, while prediction accuracy remains 85–100%.Recovery degrades with both clause count and literals per clause, with clause count having the larger effect.
Robustness: At 30% label noise, NRI accuracy declines from 92.3% to 87.4%, while RIPPER declines from 98.4% to 70.3% and DT from 100% to 69.9%.Beyond 15% noise, NRI outperforms both baselines by 17 percentage points at 30% noise.
Robustness: Accuracy remains above 92% across distractor settings, reaching 97.6% at D=32, ρ=0.9 and 96.7% at D=32, ρ=0.1.The distractors are statistically predictive but causally irrelevant features.
Efficiency: Inference latency is under 12ms with 593MB peak memory at N=512, while latency grows from 4.2ms to 11.8ms as N increases from 16 to 512.Latency stays approximately 7.5ms as M grows from 32 to 512, and memory scales O(N^2).

6 Conclusion

The paper presents Neural Rule Inducer (NRI), a foundation model trained on synthetic Boolean formulas that performs zero-shot, interpretable DNF rule induction on new domains without retraining.

6 Conclusion: NRI encodes literals using identity-free statistical properties and synthesizes clauses with a parallel slot-based decoder under product T-norm semantics.Its statistical features include class-conditional rates, entropy, and co-occurrence; training is end-to-end on prediction accuracy alone.
6 Conclusion: NRI induces interpretable DNF rules on new domains without retraining.The model is trained entirely on synthetic Boolean formulas.

A Synthetic Data Generation … A.4 Spurious Environment Features

NRI is trained exclusively on diverse synthetic episodes containing randomly generated DNF rules, causal examples, labels, and environment-dependent spurious features. The generator varies episode dimensions, rule structure, class ratios, and spurious correlations while masking environment boundaries and preserving marginal label independence by design.

A Synthetic Data Generation: Training uses exclusively synthetic episodes because real-world datasets have limited quantity and coverage, enabling diverse examples across possible logical rules.The synthetic generation procedure is described in detail to support foundation-model training.
A.1 Episode Structure: Each episode contains Boolean features X, labels Y, and a ground-truth DNF rule R∗ over N causal variables, with S additional spurious variables.X has M examples and N+S variables.
A.1 Episode Structure: The generator samples N uniformly from {6, . . . , 12} and M uniformly from {24, . . . , 48} across episodes.In general, N and M are sampled from discrete uniform ranges.
A.2 DNF Rule Sampling: Each ground-truth rule is a random DNF with K uniformly sampled clauses, clause lengths up to min(Lmax, N), distinct variables, and Bernoulli(0.5) polarities.Experiments use Kmax = 6 and Lmax = 4; rules are not canonicalized, allowing redundancy.
A.3 Example Generation: Causal features are sampled independently as Bernoulli(0.5), then labels are computed by evaluating the sampled DNF rule.The resulting class imbalance depends on rule structure: a single clause of length L gives P(Y =1) = 2−L, while additional clauses raise positives through union.
A.4 Spurious Environment Features: The generator adds spurious variables whose correlations with labels reverse between two environments, testing separation of causal and noncausal features.Environment 1 contains examples 1 to ⌊M/2⌋, and Environment 2 contains the remainder.
A.4 Spurious Environment Features: For flip rate ρ ∈(0, 0.5), Environment 1 uses P(s=1|Y =1) = ρ and P(s=1|Y =0) = 1 −ρ, with these probabilities swapped in Environment 2.This construction creates opposite spurious-label correlations across environments.
A.4 Spurious Environment Features: After concatenating causal and spurious features, the generator randomly permutes examples to hide environment boundaries while making each spurious feature marginally independent of the label.The full feature matrix is X = [Xcausal | Xspurious] ∈{0, 1}M×(N+S).

A.5 Data Generation Parameters

The data generator targets sparse, interpretable DNF rules with bounded complexity over moderately sized variable sets, rather than uniformly sampling all Boolean functions.

Scope of Coverage: The generator covers sparse DNF rules with K ≤6 clauses, L ≤4 literals, and N ≤12 variables.It targets sparse, interpretable rules relevant to rule induction.
Scope of Coverage: The generator does not uniformly sample Boolean functions, many of which require exponentially many DNF clauses.Its scope is intentionally restricted to the sparse, interpretable rules that are the focus of rule induction research.

B Literal Feature Vector

Each literal is represented by an 18-dimensional feature vector of domain-agnostic statistics, including class-conditional truth, observation, entropy, polarity, and co-occurrence features. Observation rates explicitly capture effective sample size under missing data, enabling downstream layers to discount statistically thin literals.

Feature construction: Each literal, including positive literals and negations, receives an 18-dimensional feature vector computed from the episode.The vector is defined for 2N literals indexed across variables and their negations.
Feature construction: The vector includes class-conditional truth and observation rates, marginal truth and observation rates, binary entropy, polarity, a reserved zero-padded slot, and co-occurrence summaries.Co-occurrence summaries include mean absolute and signed strengths overall and within positive and negative examples.
Co-occurrence features: Co-occurrence is computed as centered covariance between literal truth values, with aggregate and class-specific statistics measuring relationships overall or within each class.Class-restricted co-occurrence features help identify literals participating in conjunctive patterns within a class.
Missing-data handling: Missing values are excluded from truth-rate calculations but contribute to observation rates, which quantify the effective sample size behind each estimate.Downstream layers can therefore discount literals whose rates are statistically thin because of limited observed data.

C Auxiliary Terms in Lcf … E.3 Rule-Based Classifiers

The paper adds counterfactual regularizers, characterizes inference scaling, and compares NRI with non-interpretable, partially interpretable, and explicit rule-based baselines. These baselines differ in transparency, rule representation, and whether rules are evaluated sequentially, weighted, hierarchical, or converted to DNF.

C Auxiliary Terms in Lcf: Clause coverage overlap (λo = 0.1) penalizes clauses that fire simultaneously on positive examples, discouraging redundant coverage.This auxiliary term is added to the counterfactual objective.
C Auxiliary Terms in Lcf: Counterfactual load balance (λc = 0.01) encourages each positive example to be explained by more than one clause.It is defined as the negative mean per-positive responsibility entropy within the counterfactual objective.
D Computational Scaling: At N=512, inference completes in under 12ms with 593MB peak memory, while memory scales as O(N^2) because attention covers 2N literals.Latency is nearly constant at ∼7.5ms as M increases from 32 to 512 and grows from 4.2ms to 11.8ms as N increases from 16 to 512.
E Baseline Descriptions: NRI is compared with gradient boosting, generalized additive models, and interpretable rule-based classifiers trained per dataset with 5-fold cross-validation.The baseline groups represent accuracy ceilings, partial interpretability, and explicit logical rules.
E.1 Non-Interpretable Ceilings: XGBoost and LightGBM provide strong non-interpretable accuracy ceilings through gradient-boosted tree ensembles.XGBoost uses regularized objectives and efficient tree construction, whereas LightGBM uses histogram-based algorithms and leaf-wise growth.
E.2 Generalized Additive Models: Generalized additive models provide partial interpretability through separate feature shape functions rather than explicit logical rules.EBM combines gradient boosting with additive models, visualizes feature contributions, and may include pairwise interactions that reduce transparency.
E.3 Rule-Based Classifiers: Rule-based baselines include sequential rule lists, weighted rule ensembles, shallow tree sums, and hierarchical decision trees.RIPPER greedily grows and prunes ordered if-then rules; RuleFit weights extracted rules; FIGS sums small trees; DT recursively partitions features.
E.3 Rule-Based Classifiers: DT-DNF converts each decision-tree path to a positive leaf into a conjunction, producing a DNF clause for direct comparison with NRI.The resulting clauses are extracted from root-to-positive-leaf paths.

E.4 Neural DNF Methods · E.5 Summary

The paper situates Neural DNF learning as differentiable rule-structure optimization and compares it with a from-scratch neural baseline. Its summary table organizes competing methods by type, interpretability, and training regime.

E.4 Neural DNF Methods: Differentiable approximations of logical operations enable gradient-based optimization of DNF rule structures.Neural DNF methods use differentiable logic operations to learn rule structures with gradient-based optimization.
E.4 Neural DNF Methods: Neural DNF (Scratch) (N-DNF) learns DNF rules separately on each dataset using sigmoid gates that approximate AND/OR operations.Its architecture is similar to the paper’s approach but lacks pre-training on synthetic data.
E.4 Neural DNF Methods: The N-DNF baseline tests whether per-dataset neural DNF learning outperforms the paper’s zero-shot transfer approach.The comparison isolates the contribution of pre-training and transfer rather than changing the broad differentiable-logic paradigm.
E.5 Summary: Table 5 summarizes the compared methods according to their method type, output types, and interpretability characteristics.The table is presented as a summary of baseline methods used in the experiments.
E.5 Summary: The summary distinguishes gradient boosting, GAM, rule-list, rule-ensemble, tree, DNF-via-tree, and neural-DNF method types.Listed methods include XGB, LGBM, EBM, RIPPER, RuleFit, FIGS, DT, DT-DNF, and N-DNF.
E.5 Summary: All listed methods train per-dataset, while interpretability ranges from No through Partial to Yes across the compared approaches.XGB and LGBM are marked No; EBM and RuleFit Partial; RIPPER, FIGS, DT, DT-DNF, and N-DNF Yes.

F NRI Predicted Rules

NRI predicts interpretable logical rules on UCI datasets from 5% training data with auto-tuned clause selection. The examples span multiple datasets and use conjunction, disjunction, and negation over dataset-specific literals.

Example predicted rules: NRI predicts a one-clause adult rule combining negated marital, relations, purpose, employment, and property conditions with checking-status alternatives.Duplicate clauses were removed before displaying the rules.
Example predicted rules: The hepatitis example contains three disjunctive clauses, while ionosphere contains multiple disjunctive clauses combining positive and negated feature conditions.The displayed rules use conjunction within clauses and disjunction between clauses.
Example predicted rules: Spambase, tic-tac-toe, and vote each have one predicted clause defined by conjunctions of negated or positive feature-value conditions.The listed conditions include word-frequency thresholds, board-square values, and voting attributes.