PaperlayerView exact revision on arXiv

Source-linked AI summary

Pairwise Logical Selection of Enthymeme Completions under Semantic-Link Uncertainty

Xuyao Feng, Antonis Bikakis

arXiv:2608.18820v1cs.AI

TL;DR

Enthymeme completion requires selecting between candidate premises or claims, but existing methods often lack transparent logical inference or assume formal resources are available. PWAL averages logical resistance across alternative semantic-link configurations and outperforms Top-Link across five tasks, with higher strict accuracy and lower tie rates.

  • Problem

    Existing methods for enthymeme completion either do not expose the completed logical inference or assume formulae, candidates, or background knowledge are available.

  • Method

    PWAL averages each candidate’s logical-resistance score over alternative cross-formula atom-link configurations instead of selecting one highest-confidence configuration.

  • Results

    2.95–30.86 percentage points: PWAL raises mean strict accuracy and lowers mean tie rates by 4.57–58.00 percentage points versus Top-Link across five tasks.

  • Takeaways & Limitations

    PWAL provides higher selection accuracy with fewer ties while exposing the formulae, link configurations, and resistance components underlying each decision.

  • Takeaways & Limitations

    The framework is limited to two candidates, one fixed AMR-derived propositional representation per statement, and independent uniform distributions over local link choices.

Abstract

from arXiv · show

Arguments often omit premises or claims, forming enthymemes. We study pairwise logical selection between two candidates for the omitted component. Existing natural language methods can identify or generate candidates but often do not expose how the selected candidate completes the inference, while logic-based approaches usually assume that the required formulae and background knowledge are available. We extend a prior neuro-symbolic pipeline from missing-premise to missing-claim selection and replace binary entailment outcomes with logical-resistance scores. Top-Link uses weighted Partial MaxSAT under a single configuration of highest-confidence semantic links. We then introduce Possible-World Atom-Link Formalization (PWAL), which keeps translated formulae fixed and marginalizes logical resistance over alternative cross-formula semantic-link configurations. We evaluate PWAL on five tasks: ARCT and a CDED-derived task for missing-premise selection, iDebate- and AAE2-derived tasks for missing-claim selection, and alphaNLI for abductive hypothesis selection. Relative to Top-Link, PWAL raises strict accuracy by 2.95-30.86 percentage points and reduces tie rates by 4.57-58.00 percentage points on all five tasks. When ties receive half credit, accuracy still increases by 0.45-6.04 percentage points. PWAL also records the translated formulae, sampled link configurations, and resistance components for every comparison, providing a transparent trace of each score.

Introduction

The paper studies pairwise logical selection between two candidate completions for an enthymeme’s omitted premise or claim. It replaces binary entailment checks with transparent logical-resistance comparisons, including PWAL’s marginalization over alternative semantic-link configurations.

  • Problem: Enthymeme completion makes an unstated premise or claim explicit so the inference can be logically evaluated.The study compares two supplied candidate completions, covering either omitted premises or omitted claims.
  • Motivation: Natural-language methods often select omitted components without exposing the logical inference they complete.Symbolic approaches make the inference explicit through formal reasoning methods.
  • Motivation: The prior neuro-symbolic pipeline independently tested candidate completions with a SAT solver, so equal entailment outcomes could leave pairs unresolved.It combined candidate generation with logical verification but returned only entailment or nonentailment for each candidate.
  • Method: The proposed method assigns each candidate a logical-resistance score and compares candidates through source and target statement collections.Missing-premise tasks vary the source while holding the target fixed; missing-claim tasks do the reverse, with separately translated formulae preserving distinct Boolean atoms.
  • Method: Top-Link evaluates one highest-confidence semantic-link configuration, whereas PWAL averages resistance over independent alternative entailment, contradiction, and unlinked choices.PWAL selects the candidate with lower expected logical resistance and records formulae, link configurations, and resistance components for each comparison.
  • Evaluation: The evaluation covers five pairwise tasks spanning missing-premise, missing-claim, and abductive hypothesis selection.ARCT and CDED-derived data test missing premises; iDebate- and AAE2-derived data test missing claims; αNLI tests abductive selection.

Background and Related Work

The paper situates PWAL within neuro-symbolic reasoning, enthymeme completion, and weighted or probabilistic logic, while using a fixed AMR-to-propositional compiler for solver-facing representations. Its representations preserve semantic atoms, formula structure, and polarity in a truth-preserving formalization.

  • Related Work: Neuro-symbolic systems translate natural-language problems into formal representations and use symbolic solvers, sometimes refining or aggregating multiple formalizations.The related-work discussion also notes natural-language multi-agent methods for comparing supplied premise candidates.
  • Related Work: PWAL combines soft-clause weights for within-world violation costs with world probabilities representing uncertainty over active semantic links.This connects the method to weighted and probabilistic logic frameworks that attach weights to formulae or rules.
  • AMR Formalization: AMR represents sentence semantics as a rooted, labeled, directed graph whose nodes denote concepts or PropBank frames and whose edges encode semantic roles and relations.Frame labels include a predicate lemma and PropBank sense identifier, while edges include roles such as :ARG0 and :ARG1.
  • AMR Formalization: A fixed rule-based compiler maps AMR graphs to structured semantic atoms and propositional formulae, using conjunction, disjunction, implication, and negation before Boolean simplification.The resulting representation Rep(x) = ⟨Ax, Φx, Vx⟩ contains active atoms, the compiled formula, and deterministic surface verbalizations; polarity is encoded in Φx rather than Vx.

Pipeline

The pipeline represents each candidate as a source–target pair, translates and assembles its statements, and links source and target atoms explicitly. It scores candidates by logical resistance under Top-Link or PWAL, preferring the lower score and returning a tie when scores are equal.

  • Candidate representation: Each pairwise instance contains two candidate-specific source–target pairs, with statements conjoined within each source and target collection.Common-source tasks share the source collection, while common-target tasks share the target collection.
  • Decision rule: The pipeline compares candidate-level logical resistance scores, selects the candidate with the lower score, and returns Tie when σA = σB.This is a pairwise comparison of candidate-specific evaluations rather than an absolute validity judgment.
  • Formula construction: Statements are translated independently, renamed into disjoint atom namespaces, and assembled into source and target formulae.The same renaming is applied to each statement’s formula and verbalization map; fixed CNF conversion introduces no auxiliary Boolean atoms.
  • Semantic linking: Semantic relations between distinct source and target atoms are encoded as weighted clauses derived from exact matches and non-neutral NLI links.Exact matches are retained, while unmatched target atoms receive alternative entailment or contradiction links with confidence-based weights.
  • Scoring methods: PWAL treats every non-neutral link and the no-link outcome as local alternatives for each unmatched target atom.This contrasts with selecting a single highest-confidence semantic-link configuration in Top-Link.

Experimental Setup

The experiments construct pairwise candidate-selection tasks from five datasets and compare PWAL with Top-Link under controlled logical and NLI-based baselines. They report accuracy, tie rate, and half-credit accuracy using fixed evaluation settings and uncertainty estimates.

  • Tasks and pair construction: Five datasets support pairwise selection: original ARCT and αNLI, plus derived tasks from CDED, iDebate, and AAE2.Each instance contains two candidate-specific source–target pairs and one designated gold candidate.
  • Compared methods: PWAL and Top-Link share source and target formulae, exact links, NLI-derived alternatives, clause weights, and the Partial MaxSAT evaluator.FH-HardSAT and Direct NLI are additionally reported as comparison methods.
  • Implementation: K = 100 is the main PWAL sampling budget, with sampled results reported as means ± sample standard deviations across seeds 2026–2035.Top-Link, FH-HardSAT, Direct NLI, and fixed exact/105-capped references are reported as single values.
  • Metrics: Accuracy is primary; TieRate counts valid ties, while EAcc gives valid ties half credit and errors none.The study uses pointwise 95% paired two-way bootstrap intervals over dataset-specific source clusters and seed runs, with 10,000 replicates.

Results

PWAL improves accuracy, tie handling, and expected accuracy over Top-Link across five tasks, while ablations and comparisons clarify the contribution of its logical design. Sampling is stable at K = 100, though exact marginalization is substantially more expensive.

  • Main results: 2.95–30.86 percentage points: PWAL raises mean Acc relative to Top-Link across all five tasks.PWAL also lowers mean Tie by 4.57–58.00 points and raises mean EAcc by 0.45–6.04 points.
  • Main results: Wrong-to-correct repairs exceed correct-to-wrong damages for PWAL on all five tasks, while Acc gains are supported on αNLI, ARCT, and CDED but uncertain on iDebate and AAE2.The transition matrices are reported in the supplementary material.
  • Main results: 5.21–15.71 percentage points: Direct NLI exceeds PWAL in EAcc on every task.Direct NLI is a predictive reference, whereas PWAL and Top-Link share the formalization and scoring pipeline.
  • Ablations: Removing the no-link outcome lowers mean Acc and raises mean Tie on every task; removing the negative guard lowers mean Acc on every task.Full-minus-ablation Acc intervals exclude zero for the no-link ablation on αNLI, ARCT, and CDED, and for the negative-guard ablation on ARCT and CDED.
  • Sampling analysis: 1.07 points: K = 100 mean EAcc is within this distance of the exact/105-capped reference on every task.Increasing K from 100 to 200 changes mean Acc by at most 1.10 points, mean Tie by at most 0.68 points, and mean EAcc by at most 1.02 points.
  • Runtime: 84×, 172×, and 5.8 × 10^3: mean task-level runtime multipliers relative to Top-Link for K = 100, K = 200, and exact enumeration, respectively.On the controlled fully enumerable runtime subset, Top-Link takes 1.10–1.68 ms/example.

Conclusion · A Dataset Construction and Validity

PWAL averages logical resistance across alternative atom-link configurations and outperforms Top-Link on mean strict accuracy, tie rates, and mean EAcc across five selection tasks. The paper also specifies the evaluation interface and identifies representation, link-distribution, error-propagation, and sampling limitations.

  • Conclusion: PWAL averages logical resistance over alternative cross-formula atom-link configurations instead of selecting one highest-confidence configuration.This defines the method’s central departure from Top-Link.
  • Conclusion: Across five missing-premise, missing-claim, and abductive selection tasks, PWAL achieves higher mean strict accuracy and lower mean tie rates than Top-Link.The tasks cover three forms of selection.
  • Conclusion: PWAL achieves higher mean EAcc on all five tasks, while Direct NLI remains the stronger predictive reference.PWAL nevertheless exposes the reasoning information underlying its decisions.
  • Conclusion: PWAL exposes the formulae, sampled link configurations, and resistance components underlying each decision.This provides a transparent trace of the comparison process.
  • Conclusion: The framework is limited to two candidates, one fixed AMR-derived propositional representation per statement, and independent uniform distributions over local link choices.Future work will consider alternative structured representations and joint uncertainty over representations and links.
  • Conclusion: PWAL inherits errors from AMR parsing, AMR-to-logic compilation, and NLI-based atom-link construction, while Monte Carlo estimates vary with sampling budget and seed.These are methodological sources of uncertainty in the framework.
  • A Dataset Construction and Validity: Each evaluation instance contains two candidate-specific source–target pairs, (SA, TA) and (SB, TB), plus a gold candidate y ∈{A, B}.This defines the released dataset interface.
  • A Dataset Construction and Validity: Dataset identifiers construct examples, prevent split leakage, and define bootstrap clusters, but are excluded from Sc, Tc, Direct NLI input, and system outputs.The identifiers include topic, debate, prompt, stance, and essay fields.

A.1 Pair Mappings and Dataset Splits … Decisions

The paper constructs and audits pairwise datasets through deterministic mappings, graph-based source extraction, split controls, and author-verified validity checks. These procedures preserve source structure, prevent leakage, and quantify remaining ambiguities without modifying evaluation sets.

  • A.1 Pair Mappings and Dataset Splits: Complete CDED evidence passages remain intact source units because supervision is passage-level.Sentence segmentation would introduce an unannotated aggregation choice.
  • A.1 Pair Mappings and Dataset Splits: αNLI and ARCT use seeded, duplicate-filtered splits with released candidate order and no complete-example or candidate-pair overlap between development and test.αNLI samples 400 development and 400 test examples; ARCT development combines 205 development and 195 training rows, while test retains 444 official-test rows.
  • A.1 Pair Mappings and Dataset Splits: Derived-task distractors are selected deterministically using evidence type, token similarity, debate constraints, and token-length differences.The iDebate same-debate distractor is topic-controlled but may receive partial support because unsupportedness is not annotated.
  • A.2 AAE2 Graph Extraction: AAE2 sources retain only Premises directly supporting the gold Claim, excluding indirect supports, attacks, and the opposite-stance support subtree.The opposite-stance Claim is not assumed to be the logical negation of the gold Claim.
  • A.2 AAE2 Graph Extraction: AAE2 sources average 2.59 Premises in development and 2.57 in test, with median 2, maximum 7, disjoint essays, and no redistributed full derived text.These statistics characterize the extracted source sizes under the archive’s redistribution constraints.
  • A.3 Automatic Construction Audit: The automatic audit finds no duplicate complete examples, identical within-example candidates, development–test cluster overlap, or AAE2 distractor-support path overlap.For AAE2, all 550 development and test examples were revalidated against the released relation graphs.
  • A.3 Automatic Construction Audit: Excluding one iDebate test item with gold text embedded in the normalized source changes every reported test accuracy by at most 0.17 percentage points and preserves comparisons.The item is retained rather than replaced after observing results.
  • A.4 AI-Assisted Validity Audit with Author Verification: A fixed 200-item validity audit uses seeded, stratified samples across CDED, iDebate, and AAE2, with cluster-aware ordering and author confirmation.The AAE2 audit found 92 of 100 distractors were meaningful opposite-stance contrasts; the remaining 8 were topic-related but not meaningful contrasts, and labels did not alter test sets or method settings.

B.1 Evaluation Settings and Compute Environment … C.1 Top-Link-to-PWAL Transitions

The evaluation uses a fixed neuro-symbolic pipeline and explicit decision diagnostics, with PWAL transitions computed across multiple seeds and compared against deterministic Top-Link outcomes. Threshold selection is development-based, while transition analysis shows that repairs exceed damages across all datasets.

  • B.1 Evaluation Settings and Compute Environment: Experiments used a 64-bit Windows system with an i5-13600KF, 32 GB RAM, RTX 4090, Python 3.8.18, and specified neural, CUDA, transformer, and PySAT libraries.The pipeline combines Structured-BART for AMR parsing, mDeBERTa-v3 for NLI-based atom links, and RC2 in PySAT for Partial MaxSAT inference.
  • B.1 Evaluation Settings and Compute Environment: Absolute score differences of at most τtie = 10−12 count as valid ties, while invalid outputs remain errors and receive zero credit.Invalid outputs are never relabeled as ties.
  • B.2 FH-HardSAT Threshold Selection: FH-HardSAT selects its threshold grid on the development split and applies the chosen threshold pair once to the held-out test split.Table B.1 reports selected thresholds and test counts rather than repeating the main-paper metrics.
  • B.2 FH-HardSAT Threshold Selection: Test W/L/T counts in Table B.1 represent correct unique decisions, errors, and valid ties, with unsuccessful evaluations included among errors.τm denotes the BGE cosine-similarity threshold, and τc denotes the NLI contradiction-confidence threshold.
  • C.1 Top-Link-to-PWAL Transitions: Top-Link is deterministic, whereas PWAL transitions are computed separately for each of ten K = 100 seeds before aggregation.Table C.1 evaluates PWAL expected accuracy over Top-Link ties and full-test unique-to-unique repairs and damages.
  • C.1 Top-Link-to-PWAL Transitions: 57.13% is CDED’s highest tied-subset EAcc, with 50% serving as the uniform tie-breaking reference rather than a significance threshold.This comparison concerns tied subsets only.
  • C.1 Top-Link-to-PWAL Transitions: Repairs exceed damages on every dataset, although the net difference is small on αNLI, iDebate, and AAE2.The result summarizes PWAL’s transition balance relative to Top-Link decisions.

C.2 Score-Component Error Signatures … D.2 Reference Coverage and Approximation

The paper diagnoses PWAL’s score-component errors, sensitivity, and stance behavior, then evaluates sampling stability and approximation against exact or capped references. PWAL remains stable at K = 100, while its sampled scores closely track reference evaluations but can alter unique decisions through exact marginalization.

  • C.2 Score-Component Error Signatures: Both score components favor incorrect selections on a majority of errors for αNLI, CDED, iDebate, and AAE2, while witness-only attribution dominates ARCT.These are score-component signatures rather than linguistic causes.
  • C.4 Component Contrasts: AAE2 PWAL EAcc is 56.41% for For claims and 54.29% for Against claims, averaged over seeds 2026–2035.This stance diagnostic was not used to select any method setting.
  • D.1 Sampling Stability: PWAL evaluates K ∈ {10, 20, 50, 100, 200} using seeds 2026–2035, with smaller budgets as prefixes of the same K = 200 world stream.The corresponding summaries include accuracy, tie rate, EAcc, and the fixed exact/105-capped reference.
  • D.1 Sampling Stability: At K = 100, across-seed accuracy standard deviation is at most 1.12 percentage points, while increasing K to 200 changes mean accuracy by at most 1.10 and mean tie rate by 0.68 percentage points.These results quantify sampling stability across the evaluated budgets.
  • D.2 Reference Coverage and Approximation: Each candidate is enumerated when its world count is at most 105 and otherwise sampled uniformly without replacement using seed 2026; a pair is fully exact only when both candidates are enumerated.The resulting reference combines exact marginalization for fully enumerated candidates with a deterministic 105-world estimate for capped candidates.
  • D.2 Reference Coverage and Approximation: Candidate-score MAE is 0.0066–0.0089, margin MAE is 0.0094–0.0122, and mean-score decision agreement is 88.51%–99.71%.These metrics compare K = 100 ten-seed mean scores and margins with the corresponding reference evaluations.
  • D.2 Reference Coverage and Approximation: Within fully exact subsets, 78.72%–98.90% of sampled incorrect unique decisions remain wrong under exact marginalization, 1.10%–10.95% become correct, and the remainder become exact ties.For capped candidates, the reference is a deterministic uniform 105-world estimate rather than an exact expectation.

D.3 Runtime

Runtime is measured on a fixed, fully enumerable set of 20 test pairs per dataset, using medians of three timed runs after warm-up and excluding preprocessing. PWAL is substantially slower than Top-Link, with runtime increasing with K and becoming much higher under exact enumeration.

  • Runtime: 20 fully enumerable test pairs per dataset are used for every configuration, with timing starting from prepared logical states and excluding parsing, translation, NLI, and cache construction.Each per-example value is the median of three timed runs after one warm-up; finite-K configurations use seed 2026.
  • Runtime: 94.04–142.47 ms per example is the mean K = 100 PWAL runtime, compared with 1.10–1.68 ms for Top-Link.These measurements use the same fixed pairs and report controlled logical-scoring runtime.
  • Runtime: 526.31–14,764.91 ms per example is required by exact enumeration, while K = 200 approximately doubles the K = 100 runtime.The comparison is reported on the same fully enumerable test pairs.

E Paired and Clustered Statistical Inference … Example

The paper combines paired clustered bootstrap inference with a deterministic AMR-to-logic translator and an explicit PWAL calculation example. Together, these sections define the statistical protocol, translation trace, logical representations, polarity handling, validation rules, and world-level resistance aggregation.

  • E Paired and Clustered Statistical Inference: 10,000 bootstrap replicates resample task-specific clusters and complete seed runs with replacement while preserving method pairing.Point estimates average per-example utility over ten complete K = 100 seed runs; intervals are pointwise two-sided 95% bootstrap percentile intervals.
  • E Paired and Clustered Statistical Inference: PWAL’s strict-accuracy interval relative to Top-Link excludes zero on αNLI, ARCT, and CDED but includes zero onThe supplied passage truncates the remaining task names, so no further interval claim is added.
  • F.1 AMR-to-Logic Translation: The fixed AMR-to-logic pipeline returns a Boolean formula, active atoms, and base verbalizations, or fails without emitting a translator frame.It uses AMR, PropBank numbered semantic roles, and the Penman library for PENMAN decoding.
  • F.2 Records, Atoms, and Verbalizations: Normalization partitions graph records into semantic, structural, and metadata roles, while endpoint identity retains concept, normalized name, and descriptor metadata.Semantic records can produce dyads or triples; structural records determine Boolean structure; metadata may refine identity or lexical realization.
  • Active Atoms: Same-event composition forms separate anchor-based triples for eligible core-role and adjunct-role pairs, while unconsumed semantic records produce active dyads.Component dyads absorbed into successful triples remain provenance records but are not additional active formula atoms.
  • Atom Identity, Provenance, and Verbalization: Structured atom identity, canonical keys, ownership, provenance, and Boolean formulas remain distinct from deterministic surface verbalizations.Surface realization can refine lexical forms without changing structured identity or the formula; negation is carried by Φx rather than verbalization.
  • F.3 Formula Construction, Polarity, and Validation: Formula construction preserves coordination, conditions, polarity, coverage, and branch structure, rejecting invalid cycles, missing coverage, or empty condition antecedents.Participant-local polarity negates only relation atoms containing the explicitly negative participant and does not apply De Morgan’s law or negate the predicate.
  • Example: (0, 100, 0, 0), (0, 98, 1, −1), and (66, 66, 0, 1) are the reported world score tuples for the constructed example.Each world has probability 1/3; candidate-specific links activate a1 →b, a2 →b, d1 →b, or d2 →b, enabling Top-Link and PWAL aggregation.

H Detailed ARCT Success Trace

This fully enumerable ARCT trace shows PWAL selecting the gold Candidate B across sampled and exact marginalization, while Top-Link ties. The trace also exposes the fixed translations, semantic links, sampled worlds, and resistance components behind the comparison.

  • Outcome: PWAL selects gold Candidate B for every seed 2026–2035 and under exact marginalization, whereas Top-Link ties.The instance is illustrative only and was not used for method or setting selection.
  • Fixed translations: The compiler translates the candidate formulas as ΦSA = a1 ∧a2 ∧a3 ∧a4 ∧a5 ∧a6 and ΦSB = b1 ∧b2 ∧b3 ∧¬b4.The negative literal ¬b4 compiles “learning never hurts.”
  • Semantic links: Exact surface matching fixes a1 ↔c and b1 ↔c at weight 100, leaving only unmatched target atom f to be sampled.The atom inventory includes f for “section fail” and c for “comment section.”
  • Resistance trace: ρA4 = 42/142 −0 = 0.295775, while ρA2 = 0/198 −1 = −1 because A2 supports ¬f without semantic penalty.The final B-link uses unsigned b4 while the source formula enforces ¬b4, so its implication is satisfied through ¬b4.

I Detailed Stable αNLI Failure Trace

A fully enumerable αNLI case exposes a stable PWAL failure: Top-Link selects gold Candidate A, but PWAL selects distractor Candidate B across seeds 2026–2035 and exact marginalization. The trace attributes the decision to lower sampled semantic tension and a higher target-clause witness ratio for B, without identifying an upstream cause.

  • Failure outcome: Top-Link selects gold Candidate A, whereas PWAL selects distractor Candidate B for every seed 2026–2035 and under exact marginalization.Candidate A is gold; the instance was not used for method or setting selection.
  • World construction: 450 Candidate-A worlds and 400 Candidate-B worlds are marginalized separately, with 100 worlds sampled per candidate in the displayed seed-2026 run.All A worlds have probability 1/450 and all B worlds probability 1/400; candidate-specific spaces are not crossed.
  • Failure mechanism: B20 illustrates the mechanism: four positive target occurrences are entailed, producing zero semantic penalty and a witness ratio of four sevenths.Its resistance is ρB20 = 0/322 −4/7 = −0.571429, illustrating why many B worlds receive lower resistance than A worlds.
  • Failure mechanism: In the seed-2026 sample, B has lower normalized semantic tension (0.3738 versus 0.6673) and a higher target-clause witness ratio (0.2843 versus 0.2043).The trace does not assign the upstream cause to AMR parsing, formula compilation, atom-level NLI, or the resistance function.
Loading 2608.18820v1…