Source-linked AI summary
LLM-based Atomic Propositions help weak extractors: Evaluation of a Propositioner for triplet extraction
Luc Pommeret, Thomas Gerald, Patrick Paroubek, Sahar Ghannay, Christophe Servan, Sophie Rosset
TL;DR
Extracting knowledge-graph triplets from complex, information-dense text is difficult, and the paper asks whether atomic propositions can help while keeping the intermediate structure auditable. It introduces a multilingual propositioner and evaluates proposition-based extraction. Atomic propositions improve relation retrieval on FewRel and SMiLER, while combining original and atomized text yields better performance across metrics on CaRB and DocRED.
Problem
The paper asks whether splitting information-dense text into atomic propositions can improve triplet extraction while retaining an auditable intermediate structure.
Source: §1 · p. 1Method
The paper introduces a multilingual propositioner and evaluates atomic propositions as an intermediate representation with entity-centric and generative extraction approaches.
Source: §1 · p. 1, §2 · p. 3, §5.5 · p. 5Results
Atomic propositions improve relation retrieval on FewRel and SMiLER, while combining original and atomized text yields better performance across metrics on CaRB and DocRED.
Source: §7 · p. 7Takeaways & Limitations
Atomic propositions provide an interpretable, auditable intermediate structure that complements triplet extractors.
Source: §1 · p. 1The study does not evaluate LLMs larger than Qwen3-4B, citing fair comparison and computational efficiency.
Source: §8 · p. 7
Abstract
from arXiv · showhide
Knowledge Graph construction from natural language requires extracting structured triplets from complex, information-dense sentences. In this paper, we investigate if the decomposition of text into atomic propositions (minimal, semantically autonomous units of information) can improve the triplet extraction. We introduce MPropositionneur-V2, a small multilingual model covering six European languages trained by knowledge distillation from Qwen3-32B into a Qwen3-0.6B architecture, and we evaluate its integration into two extraction paradigms: entity-centric (GLiREL) and generative (Qwen3). Experiments on SMiLER, FewRel, DocRED and CaRB show that atomic propositions benefit weaker extractors (GLiREL, CoreNLP, 0.6B models), improving relation recall and, in the multilingual setting, overall accuracy. For stronger LLMs, a fallback combination strategy recovers entity recall losses while preserving the gains in relation extraction. These results show that atomic propositions are an interpretable intermediate data structure that complements extractors without replacing them.
1. Introduction
The paper asks whether atomic propositions can improve triplet extraction while making the intermediate data structure auditable. It introduces a multilingual propositioner and evaluates a pipeline that extracts propositions before triplets.
1. Introduction
- The study tests whether atomic propositions make triplet extraction easier by splitting information-dense text into less dense chunks.The authors hypothesize this may help extract entities and relations while preserving interpretability through an auditable proposition structure.Source: §1 · p. 1
- Interpretability is framed as auditability of the atomic propositions used to map sentences to subject–relation–object triplets.The paper describes an atomic proposition as a minimal, semantically autonomous information unit.Source: §1 · p. 1
- The paper contributes MPropositionneur-V2, a multilingual propositioner, and a pipeline using atomic propositions to enhance entity-relation extraction.The model produces propositions through limited-depth recursive prompting and can perform coreference resolution.Source: §1 · p. 1
- The pipeline extracts propositions first, then triplets, and builds a knowledge graph from retrieved entities and relations.For evaluation, the study limits its scope to the triplet entity-relation benchmark.Source: Figure 1 · p. 2
- The evaluation compares the distilled propositioner with a dependency-parser baseline across benchmarks spanning open and closed, sentence- and document-based extraction.The benchmarks also include tasks based solely on relations and triplets.Source: §1 · p. 2
2. Related Works
Related work covers entity-centric knowledge-graph extraction, OpenIE, and text simplification. The paper builds on atomic-proposition approaches to propose a multilingual propositioner for triplet extraction.
2. Related Works
- Knowledge-graph extraction approaches include entity recognition and linking followed by relation extraction, as well as OpenIE, which extracts triplets without predefined entity or relation sets.The former can lack flexibility with natural-language complexity.Source: §2 · p. 2
- Generative OpenIE models can struggle with long-range dependencies, nested clauses, and coordination, while GLiREL can be sensitive to initial entity recognition quality.GLiREL performs zero-shot relation classification by jointly encoding entity pairs and candidate relation labels.Source: §2 · p. 2
- Earlier sentence-simplification approaches rely on syntactic rules or dependency parsing, are limited to one language, and lack coreference resolution.Source: §2 · pp. 2–3
- Atomic propositions have been applied to retrieval, inference, fact-checking, and summary evaluation (Stacey et al., 2023; Min et al., 2023).Their fixed structure is described as more easily parseable for information retrieval and summary evaluation.Source: §2 · p. 3
- The paper applies LLM-based atomic propositions to knowledge-graph construction using a distilled multilingual model for simplification and coreference resolution across six languages.After proposition extraction, the text is flattened before entity-relation triplets are extracted.Source: §2 · p. 3
3. Formal Framework
The framework treats atomic propositions as information-preserving units, with safe decomposition governed by information content. It relates atomicity to conjunctive normal form and identifies limits to the implementation’s validation.
3. Formal Framework
- Semantic Information Theory supplies a criterion for cutting propositions while preserving information.Source: §3 · p. 3
3.1. Information Content
- A cut from ϕ to its sub-formula ψ is safe when I(ϕ) > I(ψ), and bad when I(ϕ) ≤ I(ψ).Source: §3.1 · p. 3
3.2. The CNF Condition
- A proposition is atomic if and only if it is a clause in Conjunctive Normal Form (CNF).Source: §3.2 · p. 3
- Splitting a conjunction into one of its components is safe under the framework, whereas splitting a disjunction is classified as bad.Source: §3.2 · p. 3
- The propositioner is designed to split conjunctions while preserving the integrity of disjunctive facts.Source: §3.2 · p. 3
3.3. BCP Paradox
- The BCP paradox assigns infinite information to contradictions, but CNF clauses cannot be contradictions, so atomic propositions cannot have infinite information.Source: §3.3 · p. 3
- The implementation’s adherence to the framework has only been evaluated manually and partially, and positive findings cannot guarantee all extracted propositions follow it.Further experiments are left outside the scope of this work.Source: §3.3 · p. 3
4. Proposed Approach
The proposed approach first atomizes complex text, then extracts triplets from the resulting propositions and aggregates them into a knowledge graph. A parser-based alternative replaces the generative extraction stage with parsing and triplet extraction.
4. Proposed Approach
- The pipeline has three stages: atomization, triplet extraction, and knowledge-graph construction.The paper’s stated aim is to test whether triplet extraction benefits from atomic propositions.Source: §4 · p. 3
- MPropositionneur-V2 is distilled from Qwen3-32B into a Qwen3-0.6B architecture and recursively splits text until propositions are stable and autonomous.The atomization uses the prompt proposed for Figure 2.Source: §4 · p. 3
- The generative extraction stage uses Qwen3-4B to generate triplets from an atomized chunk.Source: §4 · p. 4
- Extracted triplets are aggregated into a knowledge graph whose nodes are entities and whose edges are relations.Source: §4 · p. 4
- The baseline replaces generative extraction with parsing propositions for POS tags, named entities, and dependency trees, then extracting triplet candidates with an LLM or pattern matcher such as GLiREL.Source: §4 · p. 4
5. Experimental Protocol
The experimental protocol trains a multilingual propositioner to recursively produce atomic propositions, then evaluates triplet extraction across four benchmarks, metrics, and direct, proposition-based, and combined configurations.
5.1. Propositioner
- The propositioner is distilled from Qwen3-32B into a Qwen3-0.6B student using Wikipedia chunks in six European languages.Training used two epochs on an NVIDIA A6000 GPU.Source: §5.1 · p. 4
5.2. Nat. Lang. Recursive Propositioner
- Recursive propositioning applies MPropositionneur-V2 until propositions reach a fixed point or depth N = 5, returning only proved atomic propositions.The method recursively processes generated propositions and retains those judged stable.Source: §5.2 · p. 4
5.3. Datasets and Benchmark
- Evaluation covers SMiLER, FewRel, DocRED, and CaRB, spanning relation extraction, document-based triplet extraction, and OpenIE.SMiLER covers 14 languages and varied relation types, with entities already known.Source: §5.3 · p. 5
5.4. Metrics
- Triplet extraction is evaluated with precision, recall, and F1, with AUC additionally used for CaRB.Entity recall and relation recall are also specified as metrics.Source: §5.4 · p. 5
5.5. Baselines & Configurations
- The study compares direct extraction from source text with extraction from propositions, using GLiREL and zero-shot Qwen3-0.6B and Qwen3-4B instruction models.For SMiLER and FewRel, Comb falls back to proposition-based extraction when entities are not found; Union merges direct and proposition-derived triplets.Source: §5.5 · p. 5
6. Results and Analysis
Atomic propositions generally help extractors recover relations, while direct text can retain entity recall and accuracy. Results vary by extractor, and the qualitative example shows how transitivity can recover a relation absent from the extracted triplets.
6.1. Evaluation on Multilingual Triplet Extraction
- In almost all macro-average cases, atomic propositions benefit correctly extracted relations; direct pipelines retain better accuracy and entity recall, while combinations improve performance for small models.The table caption reports significantly improved relation recall, decreased entity recall, and increased accuracy for Qwen3-0.6B; flattening makes relations explicit so extractors can capture otherwise missed facts.Source: §6.1 · p. 5, §6 · p. 5, Table 1 · p. 6
- Qwen3-4B is more powerful across all metrics, but costs more computationally and in memory; the evaluated pipeline assumes a known relation and is therefore relation classification, not triplet extraction.Source: §6.1 · p. 5
- On DocRED, proposition granularity raises GLiREL’s F1, but Qwen3-4B scores higher in precision, recall, and F1.The table caption reports improved precision and F1 with atomization for weaker extractors, but decreased performance with LLM extraction.Source: §6.1 · p. 6, Table 2 · p. 7
- On CaRB, propositioner input improves CoreNLP recall and AUC but not precision; combining propositions with original text yields the highest recall and AUC across pipelines.The table caption reports improved recall and F1 with weaker extractors, but decreased performance with LLM extraction.Source: §6.1 · p. 6, Table 3 · p. 7
6.2. Qualitative Results
- Even error-free pipeline components may leave transitive relations out of the graph because atomic splitting can make them latent; the paper says inference can recover them.Source: §6.2 · p. 6
- The example’s proposition chain runs from Šafov through Znojmo District and the South Moravian Region to the Czech Republic, allowing a transitive hasLocation edge.That direct Šafov–Czech Republic triplet is not extracted from the propositions; Figure 5 shows the resulting graph.Source: §6.2 · pp. 6–7, Figure 5 · p. 7
7. Conclusion
Decomposing paragraphs or documents into atomic propositions improves relation retrieval on FewRel and SMiLER, while combining original and atomized text improves recall on CaRB and DocRED. The authors call for stronger atomicity evaluation and leave information-retrieval applications and rule-based graph traversal for future work.
7. Conclusion
- Atomic propositions improve relation retrieval on FewRel and SMiLER.The conclusion attributes the improvement to decomposing documents or paragraphs into atoms.Source: §7 · p. 7
- Combining original input text with atomized text improves performance across all metrics and yields better recall on CaRB and DocRED.The conclusion presents these benchmarks as validating the combination hypothesis.Source: §7 · p. 7
- Future work should include human evaluation of the propositioner to check whether text units are atomic under the paper’s definition.The authors say this evaluation is needed to ensure the units are indeed atomic.Source: §7 · p. 7
- The authors suggest information-retrieval systems as a possible application for triplet extraction methods and constructed graphs.Source: §7 · p. 7
- The authors propose that interpretable atoms could provide a statistical basis for creating or deducing logical rules that may benefit knowledge-graph traversal.They leave these research directions for future work.Source: §7 · p. 7
8. Limitations
The study leaves two comparisons unresolved: recursive versus non-recursive propositioning, and evaluation with LLMs larger than Qwen3-4B.
8. Limitations
- The recursive propositioner was not compared with a non-recursive version.The authors say the choice to refine recursively followed preliminary observations that some propositions were not atomic.Source: §8 · p. 7
- The authors plan a future comparison of recursive and non-recursive propositioners.This comparison is intended to address the unresolved relevance of recursive refinement.Source: §8 · p. 7
- The study did not evaluate LLMs larger than Qwen3-4B, citing fair-comparison and computational-efficiency considerations.Source: §8 · p. 7
A. Prompts
The appendix specifies prompts for closed- and open-domain triplet extraction and for training MPropositionneur-V2 to produce autonomous atomic propositions. The prompts constrain both the requested content and its output format.
A.1. Evaluation Prompts
- For SMiLER, FewRel, and DocRED, the closed-IE prompt classifies the relation between two identified entities using a supplied text and entity pair.It requires exactly one relation from the provided labels and an answer containing only the relation name.Source: §A.1 · p. 9
- For CaRB, the open-IE prompt requests all factual subject-predicate-object triples, one per line, and specifies no output when none can be extracted.Each line uses the format subject | predicate | object and excludes explanations.Source: §A.1 · p. 9
A.2. Propositioner Training Prompt
- During distillation, Qwen3-32B is prompted to decompose Wikipedia passages into fully autonomous atomic propositions used as targets for the Qwen3-0.6B student.The prompt frames the task as producing single facts that are fully autonomous.Source: §A.2 · p. 9
- The training prompt forbids pronouns, requires each sentence to stand alone, and repeats the subject in every sentence.It instructs the model to replace pronouns with the entity’s full name.Source: §A.2 · p. 9
- The training prompt requires the propositions to be returned only as a JSON array of strings.Source: §A.2 · p. 9
- The training prompt supplies a title and content as input, followed by an output field.Source: §A.2 · p. 9
B. Proofs for the Formal Grounding
The proofs formalize when extracting a subformula is safe or bad in terms of information content. They use these criteria to characterize atomic formulas as formulas equivalent to a clause.
Lemma 1: Divisibility of Conjunction
- For a conjunction of logically independent A and B, extracting A is strictly safe because I(A ∧B) > I(A).The proof derives the inequality from the conjunction’s smaller content set and the decreasing −log2 function.Source: B. Proofs for the Formal Grounding · pp. 9–10
Lemma 2: Indivisibility of Disjunction
- For a disjunction of logically independent A and B, extracting A is bad because I(A) > I(A ∨B).The proof characterizes this decomposition as hallucinating a more specific fact than the original disjunction states.Source: B. Proofs for the Formal Grounding · p. 10
Case of implication
- Cutting A →B into A is bad because A →B is equivalent to ¬A ∨B, so the disjunction result applies.Source: B. Proofs for the Formal Grounding · p. 10
Theorem 1: Structural characterisation
- A formula is atomic if and only if it is logically equivalent to a clause, a finite disjunction of literals.The proof uses conjunctive normal form: an atomic formula cannot contain two or more clauses, while a single clause has only bad cuts to strict subformulas.Source: B. Proofs for the Formal Grounding · p. 10