Candidate Retention for Abductive Learning

• Single-candidate selection can reinforce a mistaken assignment, while retaining more explanations can spread supervision across competing labels.Source: §1 · p. 1
• Equal-weight candidates can affect supervision differently because their contribution depends on the labels already supported by the retained set.Source: §1 · p. 1
• An upper bound on coordinate supervision error separates retained uncertainty and discarded model mass from model mismatch; only the first two depend on the retained set for a fixed model and pair.Source: §1 · p. 2
• ACR greedily adds the candidate that most decreases its criterion, stopping when no addition improves it.The retained candidates' weighted labels are then marginalized to supply concept targets for the neural update.Source: §1 · p. 2
• The paper analyzes coordinate-level supervision error and develops a greedy rule that balances added candidate mass with changes in uncertainty.Source: §1 · p. 2
• ACR improves concept accuracy over single-candidate baselines and A3BL in most evaluated aggregated mod-addition settings.Source: §1 · p. 2

2 Problem Setup

• Training reveals the target but not which valid candidate matches the inputs, leaving the hidden concepts unidentified by the target alone.For example, a digit-sum target can admit several candidate label assignments, though only one describes the images.Source: §2 · p. 3

2.1 Existing ABL Supervision

• Classical ABL selects one candidate for hard-label updates, whereas A3BL weights all valid candidates and marginalizes them into soft concept targets.Source: §2.1 · p. 3

3.1 A Retention Criterion

• The retention criterion combines retained uncertainty and discarded model mass with model mismatch in an upper bound on coordinate supervision error; only the first two vary with the retained set.The full valid set minimizes the criterion, so the benefit of an intermediate subset along the greedy path is empirical, not guaranteed by the bound.Source: §3.1 · pp. 4–5

3.2 Retained Concept Supervision

• Retained conditional marginals form soft concept targets by pooling label support across joint explanations; singleton targets are one-hot, while full-set targets recover full-distribution marginals.During the neural update, the retained targets and mass are held fixed, and retained mass weights the training example.Source: §3.2 · p. 5

3.3 Greedy Candidate Retention

• Greedy retention starts from the highest-weight candidate and adds the candidate with the smallest criterion change only when that change is negative, stopping otherwise.Equal-weight candidates can affect uncertainty differently depending on how their label support relates to the retained set; the algorithm terminates at a one-addition local optimum over the candidate pool.Source: §3.3 · pp. 5–6

4.1 Learning from Ambiguous Explanations

• 0.81–1.38 percentage points: on KMNIST, ACR exceeds A3BL for n ≥2, while A3BL performs better at n = 1.Across operand lengths, both ACR and A3BL maintain high character accuracy, while hard-selection methods have lower means or greater run-to-run variation.Source: Table 1 · p. 7, §4.1 · p. 8
• Across 19 evaluated modulus pairs, using 30,000 examples and three runs per pair, ACR improves concept accuracy over A3BL in most settings, though gains vary, including small or negative gains.Source: §4.1 · p. 8

4.2 Understanding Candidate Retention

• 94.98% concept accuracy: ACR exceeds ACR w/o mass at 20.01% and ACR w/o uncertainty at 65.22% across aggregated mod-addition settings.The ablation averages 19 modulus pairs across two datasets; removing either term also lowers retained true-label weight.Source: Table 2 · p. 9, §4.2 · p. 9
• Without mass, the initial singleton has zero uncertainty and accepts no additions; without uncertainty, positive-weight additions lead to retaining the full search pool.Figure 3 presents task-wise ablation distributions for character accuracy, TLM, and retained set size.Source: Figure 3 · p. 8, §4.2 · p. 9
• On MNIST mod addition (3, 4), retained uncertainty rises early and then falls as discarded mass approaches zero.The later trace combines sharper coordinate targets with coverage of nearly all model-posterior mass; Figure 4 reports medians and interquartile ranges.Source: §4.2 · p. 9, Figure 4 · p. 10

4.3 Beyond Arithmetic Reasoning

• On chess attack and judicial sentencing, both marginalization-based methods outperform hard selection; the judicial task shows less separation between ACR and A3BL.Warm-started ACR improves more on piece identification than attack prediction.Source: §4.3 · p. 9, Table 3 · p. 10

5.1 Neuro-Symbolic Learning

• Neuro-symbolic systems connect perception and reasoning through architectures, differentiable losses, or explicit inference, with hybrid systems using different symbolic reasoners and information-exchange mechanisms.Examples include DeepProbLog, NeurASP, and DeepStochLog.Source: §5.1 · p. 10

5.2 Abductive Learning

• ABL makes abduction a learning mechanism: revised candidate explanations supervise neural updates, but classical selection can reinforce poor pseudo-labels when early predictions are unreliable.ABLSim instead uses inter-sample similarity to guide consistency optimization before the classifier becomes reliable.Source: §5.2 · p. 11
• Prior ABL extensions address knowledge scope, identifiability, and reasoning control, whereas ACR studies how much of the candidate posterior returns as supervision.Different candidates may explain one target while assigning different concept labels, producing reasoning shortcuts.Source: §5.2 · p. 11
• ACR addresses the distinct choice of which abductive explanations should contribute to supervision, between classical ABL’s single-candidate selection and A3BL’s marginalization over all valid candidates.Different valid candidates can assign different concept labels while explaining the same target.Source: §5.2 · p. 11

6 Limitations and Future Directions

• ACR operates on abduction’s candidate set, so combinatorial growth in that space can make enumeration a bottleneck.Candidate retention alone does not establish identifiability of intended concept semantics, which is central to reasoning shortcuts.Source: §6 · p. 11
• The criterion uses unit coefficients, and task-dependent weighting remains unexplored.Jointly generating and retaining explanations and extending retention to inaccurate rules or partial labels are proposed future directions.Source: §6 · p. 11
• Comparisons use fixed K and temperature; validation-selected settings and retained-size-matched controls would further characterize the selection rule’s benefit.The chess result also warrants repeated runs with matched warm-up schedules.Source: §6 · p. 11

7 Conclusion

• ACR improves concept accuracy over single-candidate baselines and A3BL in most evaluated aggregated mod-addition settings.Objective ablations support complementary roles for posterior mass and uncertainty in retaining alternatives while keeping concept targets informative.Source: §7 · p. 12

A.1 Fixed-Support EM Connection

• This likelihood connection applies to fixed support with normalized product-posterior targets; the task-specific aggregated-mod-addition score and raw-weight convention are specified separately.Source: §A.1 · p. 15
• With product-posterior weights and a fixed retained set during the update, setting q_t to the retained posterior gives the exact E-step for log Zθ,R(x, y).Holding q_t fixed, reducing the retained loss performs a generalized M-step.Source: §A.1 · p. 15
• The variational identity makes the bound tight when q equals the retained posterior, because the KL term is then zero.For fixed q, entropy is constant in θ, so reducing the retained loss raises the variational objective and likelihood bound.Source: §A.1 · p. 15

B Experimental Protocol

• Experiments ran on two NVIDIA A100-PCIE GPUs with 40 GB each and two Intel Xeon Platinum 8358 CPUs.Source: B Experimental Protocol · p. 15
• Aggregated mod-addition experiments and ablations use three runs, while digit addition and judicial sentencing average five runs.Chess attack uses one run with a 20-epoch schedule; ACR uses five warm-up epochs of A3BL supervision.Source: B Experimental Protocol · p. 15
• Comparison methods follow official implementations with task-specific splits and hyperparameters; for aggregated mod addition, ACR uses K = 32 and temperature 0.2, matching A3BL.Source: B Experimental Protocol · p. 15
• Supplementary materials include preprocessing, training, and analysis code, task configurations, and instructions to reproduce reported results.Source: B Experimental Protocol · p. 16

C Full Aggregated Mod-Addition Results

• The evaluation follows He and Li (2025) and covers all 19 modulus pairs classified as learnable.Source: C Full Aggregated Mod-Addition Results · p. 16
• Across both datasets, ACR achieves higher accuracy than A3BL on most pairs; the largest gap favoring A3BL occurs at (2, 7), where it leads by about three points.Source: C Full Aggregated Mod-Addition Results · p. 16
• Figures 5 and 6 show character-accuracy trajectories for all 19 MNIST and KMNIST modulus pairs, respectively.Figure 5’s bottom-right panel summarizes ACR’s test gain over the strongest baseline.Source: Figure 5 · p. 16, Figure 6 · p. 17