PaperlayerView exact revision on arXiv

Source-linked AI summary

Logical Embeddings for Argument Analysis

Leander Heldring, Santiago Torres

arXiv:2608.15325v1cs.CLcs.AI

TL;DR

Contextualized embeddings can mistake linguistic similarity for similarity in logical reasoning, limiting argument representation. The paper introduces logical embeddings grounded in mathematical-logic similarity and reports that they outperform standard semantic baselines on a classification task.

  • Problem

    Contextualized embeddings can confuse linguistic similarity with similarity in argumentation, limiting reliable representation of logical reasoning.

  • Method

    The paper defines a mathematical-logic similarity over premise–conclusion arguments, proves it induces an injective positive semi-definite kernel, and derives logical embeddings through RKHS theory.

  • Results

    Logical embeddings outperform semantic embedding baselines across logistic and linear regressions, a small neural network, and a random forest on IBM-ArgQ-6.3kArgs.

  • Takeaways & Limitations

    The framework provides argument representations that preserve logical information while offering a theoretically grounded similarity measure for downstream tasks.

  • Takeaways & Limitations

    The empirical approach approximates ideal logical embeddings because natural-language arguments lack explicit premises, formal conclusions, and canonical consequence sets.

Abstract

from arXiv · show

We propose a new framework for machine-learning-oriented argument analysis tasks. Our proposal involves replacing traditional contextualized word embeddings used in most NLP tasks with logical embeddings, an alternative encoding that directly exploits argumentation structures. In essence, logical embeddings encapsulate the logical semantics of an argument, allowing for a better representation of its meaning. Supporting these embeddings is a mathematical logic-based similarity measure that offers a transparent notion of proximity and is guaranteed to satisfy several desirable theoretical properties that current cosine similarity-based contextualized word embeddings cannot assure. This similarity measure induces a positive semi-definite kernel on the set of arguments, enabling us to uniquely define logical embeddings using the theory of Reproducing Kernel Hilbert Spaces (RKHS). Moreover, we prove that this encoding is optimal, in the sense that no logical information is lost in the process. As with other RKHS applications, logical embeddings can be used in numerous supervised and unsupervised tasks. We provide an implementation of the method and aim to test it against literature benchmarks. Additionally, we demonstrate that logical embeddings outperform most standard embedding methods on a classification task.

1 Introduction

The introduction argues that contextualized word embeddings can confuse surface linguistic similarity with logical argument equivalence, causing downstream errors. It proposes logical embeddings based on mathematical-logic similarity and premise–conclusion structure, with theoretical guarantees, an implementation procedure, and classification evaluation.

  • Motivation: Contextualized word embeddings can assign higher similarity to arguments with overlapping language than to arguments with closer logical equivalence, causing classification errors.The introduction contrasts surface-level semantic similarity with argument-level equivalence and notes violations of monotonicity in embedding-derived similarity metrics.
  • Proposed framework: Logical embeddings encode the logical content of arguments to address shortcomings of contextualized word embeddings and support analysis of complex argumentation schemes.The proposed framework is presented as an alternative representation for argument similarity and learning tasks.
  • Method: The framework defines argument similarity through a mathematical-logic metric over premise–conclusion structures, treating arguments as logical objects.The metric builds on arguments composed of premises and a conclusion linked by a logical association that permits the conclusion to follow from the premises.
  • Theoretical guarantees: The Syntactic-Semantic Jaccard similarity is proved to be a positive semi-definite kernel, enabling unique RKHS embeddings that preserve logical information.The introduction also describes the resulting similarity matrix as an injective positive semi-definite kernel.
  • Implementation: The practical procedure decomposes arguments into premises and conclusions, maps conclusions to logical consequences, and trains an entailment model for pairwise premise and sub-conclusion entailment.The first two steps are described as retrieval tasks.
  • Evaluation: The method is evaluated on IBM-ArgQ-6.3kArgs using a Gram matrix of pairwise argument-similarity scores and compared with averaged GloVe, BERT-base, RoBERTa-base, SBERT, and OpenAI text-embedding-3-small embeddings.The reduced-dimensional representation of the Gram matrix is used as the logical embedding for each argument.

2 Logical embeddings

The framework represents propositional arguments structurally, defines a logic-aware similarity over premises and conclusions, and uses its kernel-induced RKHS feature map as a logical embedding. The similarity is logically faithful, positive semidefinite, and preserves argument information up to logical equivalence.

  • Argument representation: Arguments are finite, consistent, non-redundant premise sets paired with entailed conclusions, enabling comparison by logical structure rather than wording.The representation maps natural-language arguments into a common premise–conclusion structure.
  • Argument equivalence: Arguments are equivalent when their premises match up to logical equivalence and their conclusions are logically equivalent.This equivalence identifies arguments by mutual entailment rather than surface wording.
  • Similarity measure: The syntactic–semantic similarity combines Jaccard overlap of premise equivalence classes with Jaccard overlap of CNF consequence sets, weighted by σ and 1 − σ.The parameter σ controls the relative emphasis on shared premises versus shared conclusions.
  • Similarity measure: simσ(a, b) = 1 if and only if a ≈b, so maximal similarity exactly corresponds to logical argument equivalence.The measure also assigns graded values to partial overlap and is determined by logical equivalence rather than wording or topic overlap.
  • Kernel and embedding: For any 0 < σ < 1, simσ is a positive semidefinite kernel on Arg(L), inducing logical embeddings in an RKHS.The embedding preserves all formal argument information up to logical equivalence.

3 NLP implementation of logical embeddings

The implementation approximates logical embeddings from natural-language arguments through a three-step NLP pipeline: entailment modeling, premise–conclusion extraction, and CNF conversion. Argument similarity is then computed from logical overlap and evaluated on downstream classification.

  • Implementation pipeline: The proposed pipeline trains an entailment model, decomposes arguments into premises and conclusions, and maps conclusions into CNF.These steps operationalize Syntactic-Semantic Jaccard similarity measures using existing NLP and machine-learning tools.
  • Entailment modeling: Logical equivalence is tested by checking entailment in both directions, making Recognizing Textual Entailment suitable for estimating logical overlap.The approach leverages existing RTE resources, including the 570,000-pair SNLI dataset and effective machine-learning tools.
  • Argument decomposition: LLMs extract conclusions and premises from text, approximating the ideal logical embedding when natural-language arguments lack explicit formal structure.Premise and conclusion extraction is established in argumentation NLP, while chained prompting provides a practical LLM-based approach.
  • CNF conversion: The final step converts each conclusion into Conjunctive Normal Form, the hardest subtask but one modern LLMs can perform with high precision.The prompt identifies propositions and combines them into disjunctions of unique, meaningful propositions containing at least a noun and a verb.
  • Similarity computation: For two arguments, the method extracts premises and conclusions, converts conclusions to CNF, measures entailment-based syntactic and semantic overlap, and aggregates simσ(a1, a2).The resulting similarity supports production of logical embeddings through the similarity measure’s reproducibility property.
  • Evaluation: The embeddings are evaluated on IBM-ArgQ-6.3kArgs using a fixed 70/30 train-test split and binary F1, focusing on doping, vaccins, and cryptocurrencies.The neural-network classifier uses five-fold cross-validation within the training set for hyperparameter selection.

4 Conclusion … A.2 Logical arguments

The paper introduces and tests optimal logical embeddings, with implementation details and an F1-score evaluation, while the appendix formalizes the propositional logic and logical-argument notions underlying the method.

  • 4 Conclusion: The paper introduces and tests logical embeddings, characterizes them as optimal, and provides implementation code.The code is available at the cited GitHub repository.
  • 4 Conclusion: F1 scores are reported on 1137 arguments using a 70/30 train-test split and 100-dimensional clause-level logical embeddings.The logical embeddings use the top 100 eigenvectors of the pairwise similarity matrix.
  • A Appendix / supplemental material: The appendix states the propositional language, consequence relation, CNF convention, and canonical consequence sets used to support the main text.Its presentation follows Amgoud and David [2021], David [2021], and Simpson [2013].
  • A.1.1 Sentential logic: Sentential logic builds formulas from atoms using Boolean connectives, with valuations assigning truth values according to truth-functional rules.The connectives include negation, conjunction, disjunction, implication, and biconditionality.
  • A.1.1 Sentential logic: Logical equivalence identifies formulas with mutual entailment and identical truth conditions, allowing formulas to be grouped into equivalence classes.CNF representatives provide a unique representative for each relevant equivalence class while preserving the original literals.
  • A.2 Logical arguments: A logical argument is a finite pair of a consistent, non-redundant premise set and a conclusion entailed by those premises.The premise set and conclusion are denoted Prem(a) and Conc(a), respectively.
  • A.2 Logical arguments: Arguments are equivalent when their premise sets are logically equivalent and their conclusions are logically equivalent.This comparison defines equivalence over the argument space Arg(L).

A.3 Measuring similarity

Logical argument similarity compares premises modulo logical equivalence and conclusions through canonical logical consequences. The syntactic–semantic Jaccard measure combines both dimensions, with semantic similarity invariant to the valid finite CNF representative language.

  • Similarity dimensions: Arguments are compared separately through premise overlap and conclusion overlap, and these measures satisfy the stated rationality properties.This two-dimensional comparison is enabled by the logical representation of arguments.
  • Premise comparison: The common-formulas operator selects formulas in one premise set that occur in the other up to logical equivalence.It is defined as Co(Φ, Ψ) = {ϕ ∈ Φ : there exists ψ ∈ Ψ such that ϕ ≡ ψ}.
  • Conclusion comparison: Finite CNF consequence sets retain representative logical consequences using only literals on which the conclusion depends, removing equivalent reformulations and irrelevant literals.Because the atom set is finite, the resulting CNF(ϕ) is finite.
  • Syntactic–semantic Jaccard: The syntactic–semantic Jaccard measure combines premise similarity and conclusion similarity as a convex combination controlled by σ.Premise similarity counts shared premises modulo logical equivalence, while conclusion similarity counts shared canonical consequences.
  • Invariance: The semantic similarity value is invariant to the choice of valid finite CNF representative language.Different valid languages only relabel logically equivalent representatives, leaving the relevant intersection and union cardinalities unchanged.

A.4 Example of computation · A.5 Formal definitions of the properties satisfied by similarity measures · A.6 More information on Reproducing Kernel Hilbert Spaces

The appendices illustrate logical-argument similarity on two examples, formalize the properties expected of similarity measures, and explain how positive semi-definite kernels induce unique RKHS embeddings.

  • A.4 Example of computation: The example compares two arguments whose conclusions share only “Susan is the shortest sibling,” while their premises share three statements modulo logical equivalence.The shared premises are p1 ≡ q1, p2 = q2, and p3 = q4.
  • A.4 Example of computation: The premises are encoded as logical formulas, with the first argument using three premises and the second using five.The first argument’s premises are p1–p3, whereas the second’s are q1–q5.
  • A.4 Example of computation: The syntactic similarity is computed as 3/5 from three shared premises across premise sets of sizes three and five.The appendix gives ssyn(Φ1, Φ2) = 3/(3 + 5 − 3) = 3/5.
  • A.5 Formal definitions of the properties satisfied by similarity measures: The similarity axioms require maximality, symmetry, triangle inequality, substitution, minimality, non-zero similarity, strict monotony, and strict dominance.These properties respectively constrain self-similarity, argument order, distance-like behavior, interchangeability, shared content, premise overlap, premise additions, and conclusion consequences.
  • A.5 Formal definitions of the properties satisfied by similarity measures: Minimality assigns zero similarity when arguments share no premise or conclusion content, whereas non-zero similarity requires shared premise content.The non-zero condition is Co(Φ, Ψ) ≠ ∅ ⇒ S(a, b) > 0.
  • A.5 Formal definitions of the properties satisfied by similarity measures: Strict monotony favors candidates sharing more reference premises with fewer unrelated additions, while strict dominance favors equivalent-premise candidates whose conclusions share more finite logical consequences.Both properties compare S(a, b) and S(a, c), with strict cases requiring S(a, b) > S(a, c).
  • A.6 More information on Reproducing Kernel Hilbert Spaces: A positive semi-definite kernel uniquely determines an RKHS, whose canonical feature map represents each object by its kernel similarities and satisfies ⟨ΘK(x), ΘK(y)⟩HK = K(x, y).The reproducing property is h(x) = ⟨h, K(·, x)⟩HK, and the feature representations may be infinite-dimensional.

A.7 Finite-dimensional approximation by kernel PCA

Kernel PCA converts potentially infinite-dimensional logical embeddings into finite coordinates using the corpus Gram matrix, with rank-r projection providing the best linear approximation on observed arguments. When the estimated similarity matrix approximates the ideal kernel, these coordinates approximate the corresponding ideal finite-sample embeddings.

  • Finite-dimensional approximation: Logical embeddings may be infinite-dimensional, preventing direct use as finite vector representations.Each embedding is a function recording similarity to every possible argument.
  • Finite-dimensional approximation: On a finite corpus, restricted feature functions form Gram-matrix columns that approximate each argument’s logical embedding.Each column contains similarities to the observed arguments.
  • Kernel PCA construction: Kernel PCA projects centered RKHS embeddings onto leading empirical principal directions, yielding the best rank-r linear approximation on the observed corpus.The method compresses the high-dimensional similarity representation into finite coordinates.
  • Approximation sources: Using the estimated similarity matrix introduces two approximation sources: empirical similarity estimation and rank-r PCA truncation.If the estimated matrix is close to the ideal kernel, the resulting coordinates approximate the ideal finite-sample kernel PCA coordinates.
  • Kernel PCA construction: The rank-r coordinates are computed from the first r eigenvectors and corresponding eigenvalues of the centered Gram matrix.Kernel PCA therefore reduces ordinary PCA in the RKHS to a finite matrix eigenproblem.

A.8 Omitted proofs · A.8.1 Auxiliary Lemmas

The omitted proofs establish three auxiliary lemmas: non-equivalent premises are distinct, a correspondence between common premise classes is bijective, and quotient projection is bijective.

  • A.8.1 Auxiliary Lemmas: Lemma 1 shows that equivalent formulas in an argument’s premise set must be identical.The proof removes one premise and derives the conclusion from the remaining premises, contradicting non-redundancy.
  • A.8.1 Auxiliary Lemmas: Removing one of two distinct equivalent premises preserves entailment because every valuation satisfying the remaining premises also satisfies the removed premise.This makes the original premise set redundant, yielding the contradiction used in Lemma 1.
  • A.8.1 Auxiliary Lemmas: Lemma 2 defines a map from common premises to shared equivalence classes and proves that it is well-defined.A premise in the common set belongs to both quotient premise sets through its equivalence class.
  • A.8.1 Auxiliary Lemmas: The Lemma 2 map is surjective because every class shared by the two quotient premise sets has representatives in both premise sets.Such representatives are equivalent, placing one in the common premise set and mapping it to the shared class.
  • A.8.1 Auxiliary Lemmas: The Lemma 2 map is injective because equal equivalence classes imply equivalent premises, which Lemma 1 forces to be identical.Together with surjectivity, this proves the map is bijective.
  • A.8.1 Auxiliary Lemmas: Lemma 3 proves that the quotient projection on an argument’s premise set is injective and therefore bijective.Equal projected classes imply equivalent premises, and Lemma 1 then gives equality of the premises.

A.8.2 Proof of Theorem 2

The proof extends the Tanimoto kernel to finite sets including the empty set while preserving positive semi-definiteness. It then transfers this property to the argument-level components and their weighted similarity kernel.

  • Tanimoto kernel extension: The extended Tanimoto kernel kT is positive semi-definite on finite subsets of any ambient set, including cases involving empty sets.The proof handles all-empty, all-nonempty, and mixed cases using block-diagonal Gram matrices.
  • Argument-level kernels: Because kT is positive semi-definite and pullbacks preserve this property, KΠ and Kρ are positive semi-definite kernels on Arg(L).The pullbacks use the syntactic and semantic maps defined for arguments.
  • Argument-level kernels: The map ρ : Arg(L) → Pfin(F), defined by ρ(Φ, ϕ) = CNF(ϕ), is well-defined because premise sets are finite and F is finite.This map represents the semantic component of each argument through its consequence set.
  • Final kernel construction: Since σ > 0 and 1 − σ > 0, nonnegative scaling and finite summation imply that simσ is a positive semi-definite kernel on Arg(L).The result follows by combining the syntactic and semantic positive semi-definite kernels.

A.8.3 Proof of Theorem 3 · B Exhibits

The proof establishes that the canonical logical embedding identifies exactly the arguments equivalent under ≈ and that the induced quotient map is well-defined. The exhibits visualize pairwise argument connections for doping-related arguments, distinguishing pro-doping from con-doping nodes.

  • A.8.3 Proof of Theorem 3: For arguments a, b ∈ Arg(L), the canonical feature map Θσ associated with simσ provides the starting point for the proof.The argument compares the feature-map representations for arbitrary arguments and elements x, y ∈ Arg(L).
  • A.8.3 Proof of Theorem 3: Expanding the squared norm and using maximal self-similarity shows that equality of the embeddings implies a ≈ b.The proof derives the equivalence from the norm expansion and the fact that each argument is maximally similar to itself.
  • A.8.3 Proof of Theorem 3: The central result is Θσ(a) = Θσ(b) if and only if a ≈ b.Thus, the embedding equality exactly matches the argument-equivalence relation.
  • A.8.3 Proof of Theorem 3: The proof then checks that the quotient map is well-defined when equivalent quotient classes are represented by a and b.If [a]≈ = [b]≈, then a ≈ b, so the previously established result supplies the needed equality for the quotient construction.
  • B Exhibits: Figure 2 presents logical embeddings alongside a full-argument LLM representation.The exhibit is titled “Logical Embeddings and Full-argument LLM.”
  • B Exhibits: For doping arguments, the figure reports each method’s network of pairwise connections, with nodes as arguments and edges as non-zero connections.Blue nodes denote pro-doping arguments, whereas red nodes denote con-doping arguments.

B.1 Features of the Logical Embedding · B.2 F1 Scores

Logical Embeddings enforce sparse, clause-level logical similarity through mutual entailment, yielding clearer argumentative separation and traceable scores. In evaluation, retaining only the top 100 Gram-matrix eigenvectors avoids the severe overfitting associated with the full embedding dimensionality.

  • B.1 Features of the Logical Embedding: Logical Embeddings use clause-level decomposition and mutual entailment to enforce stricter logical similarity than LLM-based comparisons.LLMs rarely assign zero logical similarity and often detect vague thematic or rhetorical overlap.
  • B.1 Features of the Logical Embedding: 89598 edges cover 98.5% of possible pairs in the full-argument graph, compared with 6408 edges covering 7.0% for Logical Embeddings.Both graphs contain the same nodes from the doping subset of IBM-ArgQ-6.3kArgs.
  • B.1 Features of the Logical Embedding: Logical Embeddings produce clearer separation between pro and con arguments because they capture logical rather than rhetorical or thematic proximity.The full-argument graph mixes pro and con arguments, whereas the Logical Embeddings graph is sparse and more selective.
  • B.1 Features of the Logical Embedding: The score is traceable because it records which clauses are connected through mutual entailment and why those clauses were judged entailed.This traceability follows from computing scores from the number of clauses covered by mutual entailment.
  • B.1 Features of the Logical Embedding: Premises and subconclusions are treated separately with equal weight, and each component divides shared clauses by the distinct clauses in that family.In the illustrated case, an entailed sole subconclusion contributes 0.5 × 1 = 0.5 to the final score.
  • B.1 Features of the Logical Embedding: 0.75 is obtained when mutually entailed subconclusions contribute 0.5 and entailed premises contribute an additional 0.25.The example concerns arguments decomposed into premises and subconclusions.
  • B.2 F1 Scores: Top 100 eigenvectors were retained in the reported evaluation, while retaining all eigenvectors caused severe overfitting in the neural-network model.The overfitting is attributed to the high effective dimensionality of the full embedding space.

B.3 Practical implementation · B.3.1 LLM prompts

The implementation contrasts pairwise logical similarity, which is locally interpretable but not guaranteed to yield a positive semi-definite kernel, with a globally partitioned alternative that restores positive semi-definiteness. The analysis also uses separate LLM prompts for full-argument similarity and bidirectional clause-level entailment.

  • B.3 Practical implementation: The kernel guarantee requires fixed reduction maps applied independently to every argument, whereas pairwise reductions use only clauses from the compared argument pair.Under pairwise implementation, different Gram-matrix entries use different equivalence relations, so positive semi-definiteness is not guaranteed.
  • B.3 Practical implementation: Pairwise estimation remains the preferred way to compute true similarity because it captures local logical relations and makes matched premises and subconclusions directly inspectable.The passage also characterizes this implementation as substantively best and easier to trace.
  • B.3 Practical implementation: A naive global entailment relation creates oversized connected components, so the robustness implementation partitions the clause graph recursively instead of collapsing such components wholesale.The graph procedure leaves components below M = 20 unchanged and recursively splits oversized components.
  • B.3 Practical implementation: Edges are weighted by shared neighborhood support, and oversized components are bipartitioned recursively using the Laplacian Fiedler vector and a balanced median split.Edges with stronger internal support are harder to cut, while weakly supported edges are easier to separate.
  • B.3 Practical implementation: The final bounded components define fixed global reduction maps before comparison, making the resulting Gram matrix positive semi-definite by construction.This global method sacrifices local flexibility but prevents giant transitive components by recursively producing bounded-size subcomponents.
  • B.3.1 LLM prompts: Two LLM prompts support the analysis: one directly scores complete-argument similarity, while the other evaluates directed entailment between clause-level statements for Logical Embeddings.The full-argument prompt constructs the LLM comparison Gram matrix and generates Figure 1; the clause-level prompt is the main Logical Embeddings prompt.
  • B.3.1 LLM prompts: The full-argument prompt returns a continuous 0-to-1 score and brief explanation, emphasizing logical structure and inferential content rather than topic or surface wording.The response is requested in strictly valid JSON format.
  • B.3.1 LLM prompts: For each ordered clause pair, the entailment prompt requests a YES/NO judgment, reasoning, and confidence score from 1 to 10; logical equivalence requires positive judgments in both directions.The prompt also requires structured JSON output containing the statements, answer, reasoning, score, and comment.
Loading 2608.15325v1…