Can We Do Interpretable NLI with Graphs Based on Atomic Propositions?

1 Introduction

• SNLI and MultiNLI contain class-correlated artifacts: a hypothesis-only classifier achieves 67% accuracy on SNLI and 53% on MultiNLI.ANLI’s iterative human-in-the-loop rounds create increasingly difficult adversarial examples, making it a testbed for whether graph-based NLI relies on structured knowledge rather than superficial patterns.Source: §1 · p. 1
• The paper evaluates whether interpretable intermediate structures can enable competitive NLI, contrasting them with text models that receive no explicit reasoning cues.Its central question is whether structured representations alone can achieve competitive performance or whether interpretability entails an accuracy trade-off.Source: §1 · pp. 1–2
• The pipeline converts sentence propositions into ConceptNet triples, retrieves a ConceptNet subgraph, and classifies serialized graphs without sending raw text to the classifier.It uses a 28-relation extraction vocabulary and constrained JSON schema; the classifier receives the graphs in K, P, H order.Source: §1 · p. 2, Figure 1 · p. 2
• −1.9 points on SNLI and −9 to −14 points on ANLI quantify the accuracy drop between graph-based and text-based models.The paper identifies representational limitations, rather than data scarcity, as the source of the gap; adding 600,000 MNLI and FEVER-NLI training pairs improves ANLI by only +0.6 points.Source: §1 · p. 2
• The planned ablations isolate ConceptNet’s contribution and test combining graph and text models.The introduction identifies external knowledge and graph–text combination as factors for analysis.Source: §1 · p. 2

2 Related Work

• QA-GNN (Yasunaga et al., 2021) and KagNet (Lin et al., 2019) use ConceptNet in QA; KIM (Chen et al., 2018) and KGNLI incorporate lexical or graph knowledge in NLI.KIM uses WordNet relations, while KGNLI builds a subgraph linking concepts extracted from premises and hypotheses.Source: §2 · pp. 2–3
• Atomic-proposition decomposition aids interpretability and diagnosis of logical flaws but does not inherently improve accuracy without fine-tuning; this pipeline feeds those propositions into triplet extraction.The passage cites Srikanth and Rudinger (2025) and Huang (2026) for the accuracy qualification.Source: §2 · p. 3
• ProoFVer (Krishna et al., 2022) reads verdicts from natural-logic proof operators, while Yuan and Vlachos (2024) use semantic triples, an external graph, and an off-the-shelf NLI model zero-shot.ProoFVer’s proof consists of lexical mutations between claim and evidence spans, each marked with a natural-logic operator.Source: §2 · p. 3
• Unlike these systems, our pipeline extracts triples from atomic propositions, constrains decoding to 28 ConceptNet relations, and fine-tunes the classifier on serialized graphs.This enables a ceteris paribus comparison with a text model using the same backbone.Source: §2 · p. 3
• The classifier serializes graphs rather than using a dedicated graph network, preserving pretrained relation-name semantics and making each triplet individually accessible to attention.A systematic comparison with graph encoders remains future work.Source: §2 · p. 3

Proposed Approach

• The pipeline maps each premise–hypothesis pair to an NLI label through four stages, with no original text used in the final decision.The stages are atomisation, triplet extraction, knowledge enhancement, and classification.Source: §3 · p. 3
• Atomic propositions are extracted first, simplifying triplet extraction by processing one minimal semantic unit at a time and reducing omissions.MPropositionneur-V2-large performs multilingual atomisation with coreference resolution (Pommeret et al., 2026).Source: §3 · p. 3
• Qwen3.5-9B3 converts each proposition into subject–relation–object triplets under a constrained schema enumerating 28 ConceptNet relations.The union of proposition triplets forms the premise graph P and hypothesis graph H.Source: §3 · p. 3
• The retrieved ConceptNet graph K adds one- and two-hop bridge edges between entities unique to P and H, plus each entity’s one-hop neighbourhood.Retrieval draws from ConceptNet’s 3.3M edges.Source: §3 · p. 3
• The serialized graphs, ordered K, P, H, are fed to fine-tuned Qwen3.5-0.8B-Base; excluding text makes graph errors decisive and predictions traceable to triplets.The model is fine-tuned on the training sets of different NLI benchmarks; including raw text would make graph use post-hoc.Source: §3 · pp. 3–4

4.1.1 Propositionneur

• MPropositionneur-V2-large produces JSON proposition lists; because it repeats in 7% of ANLI sentences, outputs are cleaned, capped at 16 propositions, or replaced by the source sentence if emptied.Cleaning deduplicates case- and whitespace-insensitively and removes propositions exceeding 1.6 times the source length plus twenty characters.Source: §4.1.1 · p. 4

4.1.2 Information Extraction

• Qwen3.5-9B extracts 1–8 triplets under a JSON schema restricted to 28 ConceptNet relations, aligning the extracted graphs with the retrieval base.The prompt requests at least two triplets, while the schema permits at least one and caps output at eight.Source: §4.1.2 · p. 4
• A fixed normal form canonicalizes lemmas, argument roles, cardinality, and negation, making contradictions explicit through graph relations.For example, active and passive versions of an event share a representation, while negated facts use Not* relations with the full verb phrase as tail.Source: §4.1.2 · pp. 4–5
• Every content word must yield a triple, and two LLM-judge-driven prompt revisions addressed systematic extraction failures.The revisions targeted errors including bare-plural cardinality hallucinations, agent/patient inversion, and reversed CreatedBy direction.Source: §4.1.2 · p. 5

4.2 Knowledge Enhancement

• The retrieved graph uses English ConceptNet 5.7 assertions, truncates hubs, limits informative neighbourhood edges, and caps the full subgraph at 80 edges.Lexical relations may serve as informative bridges but are excluded from background neighbourhoods; hubs retain their 64 best-ranked neighbours.Source: §4.2 · p. 5
• ConceptNet retrieval anchors premise and hypothesis entities by exact match or lexical head, then ranks direct and two-hop bridges between nonshared entities.Two-hop paths use the weight of their weakest edge, enabling transitive evidence not present in either sentence graph.Source: §4.2 · p. 5

4.3 Serialisation and Fine-tuning of Qwen

• The classifier receives three graph sections—knowledge, premise, then hypothesis—so an auditor can trace predictions to individual propositions, triplets, or retrieved bridges.The input can be checked without rerunning the model: each triplet must follow from an atomic proposition, each proposition from its sentence, and each decision-carrying bridge must be licit.Source: §4.3 · p. 5, Figure 2 · p. 6
• The classifier is fine-tuned with a three-way head using effective batch size 64, learning rate 2e-5, and validation-selected checkpoints.Training runs for two epochs with a 1024-token maximum sequence length, increased to 1536 for ANLI.Source: §4.3 · p. 6

5 Datasets and Benchmarks

• SNLI (Bowman et al., 2015) and ANLI (Nie et al., 2020) bracket structural difficulty, from concrete single-event captions to adversarial multi-sentence passages that stress triplet compression.With the pipeline and backbone fixed, the gap rises from 1.9 to 14 points as inputs move from captions to adversarial passages, isolating representational cost.Source: §5 · p. 6
• MNLI (Williams et al., 2018) and FEVER-NLI (Thorne et al., 2018; Nie et al., 2019) add about 600,000 training pairs, never test sets.MNLI adds ten written and spoken genres; FEVER-NLI adds retrieved encyclopaedic evidence used as the premise.Source: §5 · p. 6

5.1 Metrics

• The study reports accuracy, and ANLI test sets contain 1,000 to 1,200 examples depending on the round.Source: §5.1 · p. 6

5.2 Defining the Price of Interpretability

• The price of interpretability is the accuracy drop from raw-text to graph-only Qwen3.5-0.8B fine-tuning, not a state-of-the-art comparison.The comparison holds the backbone, optimiser, and nominal epochs constant; the SNLI price uses the graph model's three-seed mean against a single-run text baseline.Source: §5.2 · p. 6

6.1 Main Results

• 0.897 ± 0.006 accuracy on SNLI; on ANLI, the graph model scores 0.480 vs. 0.489 for RoBERTa-large on R2 and 0.449 vs. 0.444 on R3, but trails 0.576 vs. 0.738 on R1.The model is graph-only; its ANLI results follow a second training phase on ANLI R1–R3 starting from the SNLI checkpoint.Source: §6.1 · p. 7, Table 1 · p. 7, Table 2 · p. 7
• 0.343 on SNLI and 0.334 to 0.335 on ANLI: the untuned base model is at chance on every split, with the same zero-shot score for text and graphs.The passage attributes the reported competence to fine-tuning, not the backbone; SNLI-to-ANLI zero-shot transfer scores 0.285, 0.330, and 0.328 on R1–R3, and sequential fine-tuning lowers SNLI retention from 0.892 to 0.782.Source: §6.1 · p. 7

6.2 The Price of Interpretability

• 97.9% of the text model’s accuracy is retained on SNLI, but only 80% on ANLI R1; the passage locates the adversarial signal in information compressed away by triplet extraction.Disagreement analysis identifies lost numeral, temporal, coreference, and fine-grained premise information, including modifiers stripped by normalisation or clauses dropped by the 8-triple cap.Source: §6.2 · p. 7, Table 3 · p. 8

6.3 Ablations

• 0.895 against 0.892 on SNLI: removing the retrieved ConceptNet graph costs nothing, while adding graphs to text raises accuracy from 0.916 to 0.921.The authors interpret this as evidence that premise and hypothesis graphs suffice for SNLI and that graphs contribute complementary information when combined with text.Source: §6.3 · pp. 7–8

6.4 Is the Gap a Data Problem?

• 0.568, 0.479, and 0.461 on ANLI R1–R3 with additional training data and SNLI replay, versus 0.565, 0.462, and 0.463 without it; the authors conclude the adversarial gap is representational, not data-limited.The tested recipe trained on SNLI + MNLI + FEVER, then ANLI with SNLI replay; phase 1 raised SNLI accuracy from 0.892 to 0.896.Source: §6.4 · p. 8

• On SNLI, graph-only NLI costs 1.9 points; on ANLI, 9 to 14 depending on round, though it matches published RoBERTa-large on R2 and R3.Source: §7 · p. 8
• The pipeline carries atomic propositions through decisions and combines ConceptNet vocabulary, constrained extraction, and serialized three-graph input so every prediction traces to finite triplets.Source: §7 · p. 8
• The ConceptNet triplet vocabulary loses numeral mentions, temporal granularity, and fine details of long premises, so the target structure must be enriched rather than the classifier scaled.Source: §7 · p. 8
• A human study in which annotators repair wrong predictions by editing triplets would directly test whether the intermediate structures are auditable.Source: §7 · p. 8

• Interpretability is qualitative rather than measured for human readability, and graph quality is evaluated only indirectly through NLI benchmarks.The authors leave explicit interpretability measurement and direct graph-quality evaluation, including in other domains, for future work.Source: Limitations · p. 8
• Upstream errors propagate unchecked: atomiser loops affect 7% of ANLI sentences, cleaning can drop facts, and incorrect coreference is silently frozen into premise triplets.The authors propose scoring propositions and triplets against a reference to distinguish representation limits from extractor errors.Source: Limitations · pp. 8–9
• The experiments use English only, and ConceptNet’s coverage is culturally biased despite the propositionneur being multilingual.Source: Limitations · p. 9
• The SNLI interpretability cost is an upper bound of roughly two points given single-run baselines and interrupted graph seeds, while ANLI’s 9-to-14-point gap is unaffected.Source: Limitations · p. 9
• The usefulness of multi-hop ConceptNet retrieval for adversarial inference remains open because the no-K ablation was run only on SNLI.SNLI’s high vocabulary overlap leaves little for a bridge to connect, unlike ANLI’s low-overlap hypotheses; whether K helps there or adds noise remains to be tested.Source: Limitations · p. 9
• The closed 28-relation vocabulary cannot faithfully encode temporal and spatial granularity, scalar and comparative meaning, or propositional attitudes and modality.The first two meaning families, along with coreference errors and truncation of long premises, account for most of the residual ANLI gap and are rare in SNLI captions.Source: Limitations · p. 9

A Extraction Prompt

• The prompt comprises a task statement with 28 relations, a normal form for nodes and argument structure, nine few-shot pairs, and a JSON schema for the constrained decoder.These components are shown in Figures 3–6, respectively.Source: A Extraction Prompt · p. 10

B Reproduction Code

• The reproduction code is available in the l-pommeret/graph-nli repository, and the end-to-end commands assume ConceptNet 5.7 English assertions in data/conceptnet/.Figure 7 presents the commands and pipeline from atomic propositions to the fine-tuned classifier.Source: B Reproduction Code · pp. 10–11, Figure 7 · p. 13
• The extraction prompt specifies a 28-relation ConceptNet vocabulary and a normal form for nodes and argument structure.These prompt components are shown in Figures 3 and 4.Source: Figure 3 · p. 11, Figure 4 · p. 12
• Sequential fine-tuning starts from the SNLI checkpoint selected on the validation split, and the atomic proposition datasets are released on the HuggingFace Hub.The listed datasets are atomic-snli and atomic-anli.Source: B Reproduction Code · p. 11
• The few-shot prompt contains nine pairs in compact triple notation, with JSON arrays of {h, r, t} objects in alternating user and assistant turns.Figure 5 shows the pairs and their formatting.Source: Figure 5 · p. 12
• The constrained decoder receives a JSON schema whose enum lists the 28 extraction relations.Figure 6 shows the schema and refers to the vocabulary in Figure 3.Source: Figure 3 · p. 11, Figure 6 · p. 13