What Comes Next and Why: Interpretable Next-Event Prediction with Neuro-Symbolic Rules

1 Introduction

• Neural sequence models predict accurately across domains, but opaque internal representations make their predictions difficult to interpret or justify.The problem is especially salient in safety- and regulation-sensitive contexts.Source: §1 · p. 1
• Temporal dependencies make next-event prediction difficult to address with neuro-symbolic methods designed for tabular inputs; flattening sequences creates sparse, high-dimensional representations.The cited approaches rely on binarized tabular inputs, limiting their scalability and expressiveness for sequential data.Source: §1 · p. 2
• RULES learns temporal sequential rules from data, and RUNES adds a residual network trained to correct predictions while retaining rule-level interpretability.Training uses two phases: RULES is learned first, then frozen while residuals are trained using additive logits.Source: §1 · p. 2
• On synthetic data, RULES recovers ground-truth rules; on real data, it learns compact rule sets, while RUNES achieves competitive accuracy and exposes rule and residual contributions.The residual network mostly corrects RULES errors when learned rules are reliable.Source: §1 · p. 2

2.1 Temporal Conjunctive Prediction Rules

• Temporal Conjunctive Prediction Rules combine event-time conditions in an antecedent with a consequent that specifies likely next-event outcomes.Each antecedent event is paired with a delay interval, and the antecedent activates when every condition is satisfied.Source: §2.1 · p. 2

2.2 The RUNES Architecture

• RUNES trains RULES first and then trains a residual network with RULES frozen, so residual logits correct predictions made by the rule network.The residual network can be instantiated by a sequence model; this work uses a transformer.Source: §2.2 · p. 3, §2.6 · p. 6

2.3 RULES: Predicate Layer

• The predicate layer learns event identities and delay intervals, approximating event-time predicates with differentiable functions that approach logical conditions as temperature decreases.The method uses soft binning for delay intervals and can mask events beyond a specified lookback window.Source: §2.3 · pp. 4–5

2.4 RULES: Antecedent Layer

• The antecedent layer combines selected predicates through a soft conjunction that approaches binary activation when all selected predicates hold.Nonnegative predicate-selection weights determine which conditions belong to each antecedent.Source: §2.4 · p. 5

2.5 RULES: Consequent Layer

• The consequent layer sums logits from active rules, with a bias that can model independent noise and encourage learning meaningful rules.After annealing, rule activations are effectively binary, and rule logits are unweighted in the symbolic regime.Source: §2.5 · p. 5

2.6 RUNES: Training

• RUNES is trained with cross-entropy over valid next-event prediction steps, excluding each sequence’s first event because no rule can yet be active.Predictions condition on the target time, supporting sequences with variable inter-event offsets; irregularly sampled sequences require a supplied or estimated prediction time.Source: §2.6 · pp. 5–6
• RULES uses a self-supervised symbolic phase with sequential temperature annealing; then its parameters are frozen and a residual network is trained to correct its errors.The predicate-layer temperature is annealed before the antecedent-layer temperature for more stable optimization.Source: §2.6 · p. 6

3 Related Work

• Most sequential rule-mining approaches rely on thresholds or combinatorial search, and many rules lack timing or a principled way to combine them for next-event prediction.The cited approaches mine ordered subsequences or unordered event sets; minimum-description-length methods address rule selection but search a super-exponential space.Source: §3 · p. 6
• Temporal point-process models predict event timing, while RUNES targets interpretable next-event type prediction conditioned on the next event time.Prior interpretable temporal-point-process approaches focus on explaining when events occur.Source: §3 · p. 6
• RUNES differs from prior neuro-symbolic rule learners by targeting categorical sequences, which tabular methods and several temporal alternatives do not support in this setting.The paper states that, to its knowledge, RUNES is the first neuro-symbolic approach for next-event prediction on categorical sequences.Source: §3 · p. 6
• Figure 2 evaluates synthetic-data test accuracy against baselines and compares discovered-rule similarity with ground truth across signal-to-noise settings.Results average 10 rule sets and datasets per setting, with error bars showing one empirical standard deviation.Source: Figure 2 · p. 7

4.1 Synthetic Data

• Synthetic datasets used ground-truth rule sets and varied signal-to-noise through consequent reliability and the share of target events that were consequents rather than noise.The predictive evaluation averaged held-out accuracy across datasets generated from 10 rule sets per setting.Source: §4.1 · p. 7
• RULES achieved strong synthetic accuracy, often near tree ensembles and the transformer, and surpassed CATBOOST and the transformer in noisy settings.The performance gap narrowed with more training samples.Source: §4.1 · p. 8
• RULES returned no rules on random-event sanity checks and mapped learned rules faithfully to ground truth, though it often shortened large antecedents and sometimes missed rules.Ground-truth recovery was driven mainly by reliable consequents and remained robust to random noise events.Source: §4.1 · pp. 7–8

4.2 Real-World Data

• On real-world datasets, RULES is competitive on Holidays and Commits, while RUNES is competitive with the transformer on the remaining datasets, outperforming other baselines and ranking second overall.The reported test-accuracy comparison is in Table 1.Source: §4.2 · p. 8, Table 1 · p. 8
• Across real-world datasets, RULES typically has tens of rules with antecedents of size 1–5; tree ensembles often induce thousands of distinct conditions.PPM can be simple when short contexts suffice, but its rule set grows with vocabulary size and context length.Source: §4.2 · p. 9
• At least one rule is active for 64% of test predictions on average; RUNES rarely overturns correct rule-based predictions (3%) and corrects many errors (17%).These counts are averaged across test splits of real datasets and normalized within rule-activity groups.Source: §4.2 · p. 9, Table 2 · p. 9
• The additive rule and residual logits expose how predictions change; on Rolling Mill, incorrect updates are rare for KL > 3 but common for KL < 0.7.Figure 3 groups test predictions by KL and shows RULES correctness alongside RUNES prediction changes.Source: §4.2 · pp. 9–10, Figure 3 · p. 9

5 Conclusion

• RULES learns event- and delay-based rule antecedents through continuous optimization, while RUNES adds a residual network whose contributions remain visible through additive logits.The residual network is intended to capture structure not explained by rules.Source: §5 · p. 10
• On synthetic data, RULES recovers ground-truth rules and produces strong predictions; on real-world data, its compact rule sets combined with RUNES achieve competitive performance.Source: §5 · p. 10
• RULES introduces randomness and tunable hyperparameters, models one categorical event per time step, and currently forms rules through conjunctions over event-time predicates.The authors note that learned models and extracted rule sets can vary across runs.Source: §5 · p. 10

A.1 Basic Functions

• As τ → 0+, the sigmoid, softplus, LogSumExp, and softmax converge respectively to the Heaviside step, ReLU, maximum, and argmax-based selection; the softmax-weighted average converges to the maximum.With tied maxima, softmax distributes its limit evenly across maximal entries; with a unique maximum, it becomes one-hot.Source: §A.1 · pp. 16–17, Figure 4 · p. 16

A.2 RULES: Predicate Layer

• The soft event match converges to 1 when the selected weight is maximal and exceeds log(n), to 0 when the maximum is negative or the selected weight is nonmaximal, with a fractional limit at equality.Here n is the number of maximal weights; the text describes other boundary cases as extremely unlikely in practice.Source: §A.2 · pp. 17–18
• Soft interval membership converges to the hard interval indicator in the interior and exterior, with limits 1/2 at either boundary and 1/3 when a = b = x.The proof represents the membership value as the second coordinate of a temperature-scaled softmax over three logits.Source: §A.2 · p. 19

A.3 RULES: Antecedent Layer

• Under annealing, positive antecedent weights correspond to predicates with p_i = 1, yielding the conjunction condition described for the antecedent layer.The proof uses softplus convergence to ReLU and analyzes the limit of the antecedent activation.Source: §A.3 · p. 20

B.1 Layer Initialization

• Predicate intervals are initialized using a known or data-derived lookback window and minimum offset, with widths sampled from a scaled Beta(2.5, 5.0) distribution favoring moderate-to-narrow intervals.Antecedent incoming weights are initialized small to avoid activation saturation and facilitate training.Source: §B.1 · p. 21

B.2 Interpreting RULES as a Symbolic Rule Set

• Softmax translation invariance makes rule influence depend on relative event logits, so mean-centered logits indicate which events a rule promotes or suppresses.Large positive mean-centered logits favor an event, while large negative ones oppose it.Source: §B.2 · pp. 21–22
• RULES weights yield symbolic predicates and rules: positive conjunction weights identify antecedent predicates, while discretization thresholds determine retained rules and consequents.The illustrated thresholds are θvar = 0.9, θquant = 0.875, and θinc = 0.9; a second consequent can be included by the inclusion threshold even when it does not meet the quantile criterion.Source: Figure 5 · p. 23, §B.2 · p. 23

B.3 Antecedent Pruning for Improved Rule Extraction

• Preprocessing removes redundant neurons and subsumed predicates, then unifies equivalent antecedents by combining their logits without changing model behavior.Equivalent antecedents are perfectly co-active, so their logits contribute additively; a wider-interval predicate is redundant when a stricter predicate for the same event is also present.Source: §B.3 · p. 24
• Always-off neurons and predicates can be pruned, and always-on antecedents can be removed by adding their logits to the bias vector.Frequency thresholds can also remove rare or overly specific neurons, but unlike strict always-on/off definitions, this may cause minor behavioral changes.Source: §B.3 · p. 24

B.4 Symbolic Refinement of Learned Rules

• Post-processing reduces antecedent complexity and spurious activations while preserving correct rule support, and refined rules can be re-embedded by retraining only the output layer.Refining a small set of learned rules is described as more efficient than selecting rules from a large candidate pool.Source: §B.4 · p. 24
• Predicates with identical truth values across all events in each training sequence can be merged without changing when their rule activates.The method checks each pair of predicates within the same rule on the training data and removes one of an identical pair.Source: §B.4 · pp. 24–25
• Overlapping delay intervals for the same event can be replaced by their intersection when the joint predicate preserves the original rule's supported activations on the dataset.The original predicates need not be logically equivalent to their intersection, even when their rule activations are equivalent on a particular dataset.Source: §B.4 · p. 25
• Testing all subsets of same-event predicates can grow exponentially, but many overlapping predicates are rarely encountered and candidate counts can be capped.Limiting the number of predicates considered when forming joint predicates controls computational cost if needed.Source: §B.4 · p. 25
• Delay tightening searches observed occurrence bounds, prunes subsumed intervals, and selects the smallest remaining interval, which cannot add incorrect activations.Only events supporting the rule's antecedent and consequent need to be considered when tightening its delay constraints.Source: §B.4 · p. 25

C Connections to Temporal Logic

• Rule antecedents form bounded past-time MTL formulas, while additive-logit consequents depart from hard implication; discretizing consequents yields fully symbolic MTL rules.The antecedent corresponds to a conjunction of bounded “ONCE” operators, expressible through MTL's metric “SINCE” operator.Source: C Connections to Temporal Logic · p. 26

D Scaling Behavior of RULES

• For short-range rules, shorter training sequences reduce computation without impairing performance or rule learning; increasing L fourfold raises epoch time about 3.5-fold.With L fixed, processing Ltot events per epoch scales linearly in the number of subsequences.Source: D Scaling Behavior of RULES · p. 26
• For both RULES and the transformer, runtime is largely independent of vocabulary size; RULES shows quadratic sequence-length scaling at larger L, while RULES (S) substantially reduces inference time.At L = 256, RULES runtimes remain around 0.26s for training and 0.13s for inference.Source: D Scaling Behavior of RULES · p. 27, Figure 6 · p. 27

E.1 PPM-C

• PPM-C predicts from up to C previous events, falling back to shorter contexts when needed; it ignores sampling times and uses the last C ordered events.Its context probabilities are based on empirical event counts.Source: §E.1 · pp. 27–28

E.2 Gradient-Boosted Decision Trees

• XGBOOST and CATBOOST use one tabular sample per predicted event, with categorical features for positional offsets and numerical time-offset features for irregularly sampled data.Padding ensures samples have the same number of features.Source: §E.2 · p. 28

E.3 RoPE Transformer

• The decoder-only transformer uses causal attention and RoPE based on continuous temporal offsets, masking future events and events beyond RULES's lookback window.Attention is restricted to the same fixed context window used by RULES.Source: §E.3 · p. 28

F Sequential Rule Interpretations and Limitations of Markov Models and Trees

• Markov models map fixed-length event contexts to empirical next-event frequencies, but cannot represent time beyond event order and grow exponentially with context length.Their growth restricts them to short-range dependencies.Source: F Sequential Rule Interpretations and… · p. 28
• A decision-tree root-to-leaf path forms a rule antecedent and its leaf logits define the consequent, but time features are not tied to specific events.Subset partitions for high-cardinality categories can compact tree representations while reducing interpretability.Source: F Sequential Rule Interpretations and… · p. 28
• RULES directly associates specific events with delay conditions, unlike Markov event-count contexts or tree splits that do not explicitly tie time to events.The figure illustrates predictions of the next event x4.Source: F Sequential Rule Interpretations and… · p. 28, Figure 7 · p. 29

G Real-World Datasets

• The datasets cover public holidays, Bach chorales, Samba repository commits, steel-manufacturing logs, and Amazon product-category purchases.The collection ranges from musical events and production logs to purchases and ordered commits.Source: G Real-World Datasets · p. 29
• Holidays and Commits are split by cutoff dates to simulate forecasting, while other datasets randomly assign sequences; long sequences are handled using subsequences for training and testing.All models are tested on the same subsequences.Source: G Real-World Datasets · pp. 29–30
• Amazon preprocessing removes customers with fewer than 10 purchases, groups rare categories as OTHER, orders same-day purchases alphabetically with small time offsets, and scales timestamps by five.Categories occurring fewer than 200 times are grouped.Source: G Real-World Datasets · p. 30

H.1 Hardware

• Experiments used NVIDIA A100 or L40S GPUs and AMD EPYC CPUs; models were trained on a single GPU except PPM-C, which used a CPU.Most experiments used 32GB of RAM, while XGBOOST rule extraction on Amazon required up to 400GB.Source: §H.1 · p. 30

H.2 Compute Time

• Total compute was not tracked precisely; synthetic runs are estimated at roughly 3–4 weeks of GPU-equivalent compute across 28 settings, with additional tuning and preliminary compute estimated at 1–2× reported runs.On real data, run times vary substantially by dataset and model; exact tree-ensemble rule extraction is especially costly on Rolling Mill and Amazon.Source: §H.2 · p. 30

I.1 Synthetic Datasets

• Synthetic experiments use a maximum lookback window ω = 64; RULES and the RoPE Transformer share manually tuned hyperparameters, while PPM and tree baselines use dataset-specific grid searches.The neural-model settings are fixed across synthetic settings.Source: §I.1 · p. 30
• RULES uses 300 predicate neurons and 64 antecedent neurons, masks events outside the lookback window, and anneals predicate temperature before antecedent temperature.Training uses Adam at learning rate 0.03; annealing proceeds in stages after warm-up.Source: §I.1 · p. 31
• For synthetic rule evaluation, RULES models are selected by training loss, and unreliable or extremely rare extracted rules are filtered before rule-set metrics are computed.The filtering thresholds are confidence below 0.05 or frequency below 5 · 10^-4 on training data.Source: §I.1 · p. 31
• RULES∗ optimizes only its softmax output layer, while the RoPE Transformer uses four decoder layers, four attention heads per layer, and validation-loss-based training controls.The transformer uses 128-dimensional event, query, key, and value vectors and dropout of 0.1.Source: §I.1 · p. 31
• PPM-C tests maximum context lengths from 0 to 5, while XGBOOST and CATBOOST use GPU grid searches with early stopping or validation-loss selection.The boosting searches vary tree depth, leaf regularization, and learning rate; CATBOOST applies subsampling per split.Source: §I.1 · p. 32

I.2 Real-World Datasets

• The lookback window is tuned for RULES and then held fixed across models, while several real datasets use randomly sampled subsequences.Chorales uses batch size 30 because it has 60 training sequences; Holidays and Commits use 1,024 subsequences per epoch, and Amazon uses 2,048.Source: §I.2 · p. 32
• RULES training mostly reuses synthetic-experiment schedules, with dataset-specific warm-up or annealing adjustments for Holidays and Amazon.On Amazon, an additional regularizer counters excessive antecedent-neuron deactivation during annealing.Source: §I.2 · p. 32
• After selecting the best RULES model, the pipeline embeds and symbolically processes its antecedents, then retrains only the output layer.The processed-rule model uses Adam at learning rate 0.03, staged learning-rate decay, early stopping, and selection by validation loss across four seeds.Source: §I.2 · pp. 32–33
• The RoPE transformer and tree-ensemble baselines use dataset-matched settings, with truncated-history augmentation for tree ensembles on sampled-subsequence datasets.The PPM-C baseline selects context length from 0 to 5 using validation cross-entropy.Source: §I.2 · p. 33
• RUNES freezes the best processed RULES model and adds a transformer residual network, summing the two models’ softmax logits.The residual transformer follows the standalone RoPE transformer’s architecture and optimization protocol.Source: §I.2 · p. 33

J Additional Results on Synthetic Data

• More training sequences improve RULES’ ground-truth reconstruction and prediction accuracy, while accuracy remains in the same region across vocabulary sizes.RULES works well for D ∈ [16, 1024]; at D ∈ {4, 8}, it is hard to find structure beyond base frequencies.Source: J Additional Results on Synthetic Data · p. 34, Figure 8 · p. 34, Figure 9 · p. 34
• At higher vocabulary sizes, sim(R, R*) increases while sim(R*, R) decreases, as a larger rule-set search space also permits sparser rules with smaller antecedents.Figure 8 varies training-set size, Figure 9 varies vocabulary size, and Figure 10 varies signal-to-noise parameters; each reports accuracy and rule-set similarity.Source: J Additional Results on Synthetic Data · p. 34, Figure 8 · p. 34, Figure 9 · p. 34

K.1 Test Loss and Model Sizes on Real Datasets

• Real-world test cross-entropy trends mirror the accuracy results reported in Table 1.Table 5 reports test cross-entropy; Table 6 reports rule count, average antecedent size, and accuracy for models decomposable into symbolic rules.Source: §K.1 · p. 36, Table 5 · p. 36, Table 6 · p. 36

K.2 Influence of the Residual Network for Individual Datasets

• The individual-dataset analysis excludes Commits because RULES performs well there, leaving the residual effectively untrained under early stopping.The reported table normalizes counts within rule-activity groups and includes each group’s data fraction and total test-event count.Source: §K.2 · p. 37, Table 7 · p. 37

K.3 Influence of the Residual Network for Individual Predictions

• When RULES is uncertain, the residual usually causes larger distribution shifts; high entropy and divergence with model disagreement can accompany correct adjustments or noisy cases where both are wrong.On Rolling Mill, symmetric KL divergence above 3 strongly correlates with correct prediction changes during disagreement; on Chorales, below entropy 1.5, correct changes do not occur while incorrect ones are frequent.Source: §K.3 · p. 37, Figure 11 · p. 38

K.4 Effect of Symbolic Rule Processing

• Symbolic post-processing reduces antecedent predicate counts and delay-interval widths, and reduces the number of active rules per prediction, while accuracy changes vary by dataset.The authors hypothesize that refinement increases sparsity and may promote more self-contained rules, sometimes at a predictive-performance cost.Source: §K.4 · p. 39, Table 8 · p. 39, Figure 12 · p. 40

K.5 Variability of RULES, RUNES, and the Transformer

• Across RUNES training stages, variability is moderate during rule learning and small in later stages.Tables 9 and 10 report means and standard deviations across four seeds for the stages and standalone transformer.Source: §K.5 · p. 41, Table 9 · p. 41, Table 10 · p. 41

K.6 Externally Mined Rules vs. End-to-End Learning

• RULES with end-to-end learning and symbolic refinement outperforms the SQS-based pipeline on three of four comparable datasets, while SQS helps rule discovery on Rolling Mill.SQS does not finish on Amazon within 24 hours, whereas RULES trains in approximately 30 minutes.Source: §K.6 · p. 41, Table 11 · p. 42

K.7 Independent Training of Rule and Residual Networks

• Adding RULES logits during residual-network training improves performance by helping the residual learn corrective logits relative to rule-based predictions.The comparison averages residual-network training results over four random seeds.Source: §K.7 · p. 42, Table 12 · p. 42

K.8 Ablation on the Lookback Window

• On Commits, at ω = 12, RULES outperforms every configuration of symbolic-decomposition baselines; performance drops at larger windows, likely because hyperparameters tuned for ω = 24 transfer poorly where useful delay intervals are harder to learn.Source: §K.8 · pp. 42–43

K.9.1 Comparison across Models

• On Amazon, a RULES movie-purchase rule uses a 68-day interval, while first-order PPM lacks detailed timing and fails to use the dependency in 38% of test cases where RULES activates it correctly.The example contrasts temporally specified rules with a time-agnostic Markov model.Source: §K.9.1 · p. 43
• A RULES computer-purchase rule occurs 114 times in training, covers 13% of events when active, and generalizes to 30 test cases; order-3 PPM misses the temporal-gap pattern and the power-supply consequent.Among 27 contiguous training occurrences of the three antecedent events, PPM observes the listed consequents only twice, despite 39 power-supply purchases when the RULES antecedent is active.Source: §K.9.1 · p. 43
• RULES’ extracted rules can be fully listed for human inspection, whereas enumerating PPM’s empirical counts is infeasible without substantial pruning or summarization.XGBOOST paths can also be difficult to interpret because categorical subset splits create complex conjunctions; CATBOOST’s continuous encodings further obscure explicit rules.Source: §K.9.1 · pp. 43, 45, 46

K.9.2 Analysis of RULES across Datasets

• On Holidays, every event is covered by at least one rule, and most extracted rules are exact with perfect confidence; lower-confidence rules remain correct when activation windows admit alternatives.Examples include Good Friday three days before Easter Monday and Ascension Day eleven days before Whit Monday; New Year’s Day can predict Good Friday with lower confidence because Easter Monday may also fall in the window.Source: §K.9.2 · pp. 46–47
• On Commits, RULES captures author-succession patterns with non-trivial temporal relations, but the authors lack domain expertise to verify whether these rules reflect true workflows.The examples include rules with training supports of 539, 110, and 27 and confidences of 34%, 10%, and 59%, respectively.Source: §K.9.2 · p. 47
• On Chorales, high-support rules capture expected melodic motion, while a two-antecedent rule with distinct delay intervals expresses a descending line.The examples include whole-tone and semitone movement, note repetition, and a descending line from C5 and A4 to G4 and F4.Source: §K.9.2 · p. 47
• RULES also produces unreliable rules, including high-confidence, low-support rules that overfit individual occurrences and low-confidence rules; users can filter them by support or confidence.An Amazon rule has 100% confidence but support of only 2 on the training set.Source: §K.9.2 · p. 47

NeurIPS Paper Checklist

• The checklist reports that limitations are discussed, including runtime and scalability and the possibility that interpretable rule structures miss dependencies; source code and reproduction instructions are provided.It also reports standard deviations across four random seeds for real data results.Source: NeurIPS Paper Checklist · pp. 48–50

• The paper states that its core method development does not involve LLMs as an important, original, or non-standard component.Source: NeurIPS Paper Checklist · p. 54