PaperlayerView exact revision on arXiv

Source-linked AI summary

An Explainable Coherence Score for Detecting Temporal Inconsistencies in Political News

Marius Nicusor Pantea, Adrian Groza

arXiv:2608.29175v1cs.AI

TL;DR

Political news can contain temporal inconsistencies that stylistic fake-news detectors miss, motivating an article-level, explainable coherence measure. The paper introduces TCS, computed through temporal fact extraction, graph construction, hierarchical verification, and score aggregation with explanations. At the selected operating point, the system reaches precision 0.909 with a single false positive, supporting human-in-the-loop fact-checking assistance within temporally grounded political narratives.

  • Problem

    Existing temporal verification focuses on isolated claims, while article-level temporal coherence metrics and verifiable explanations remain limited.

  • Method

    TCS combines temporal knowledge graphs, eight interval-algebra-based internal checkers, hierarchical external verification, and interpretable score aggregation with explanations.

  • Results

    Precision is 0.909 at the selected operating point, with 10 of 11 flagged articles truly fake and a single false positive.

  • Takeaways & Limitations

    Every verdict includes a verifiable justification, and the reported precision-oriented profile is suited to human-in-the-loop fact-checking assistance.

Abstract

from arXiv · show

Temporal inconsistencies, such as mandates attributed outside their real interval, events presented as past before they occurred, or inverted causal sequences, are a form of political disinformation that evades style-based fake news detectors: a well-written article with a single wrong date carries no lexical signal of falsehood. This paper introduces the Temporal Coherence Score (TCS), a continuous, intrinsically interpretable metric that quantifies the temporal coherence of a news article, computed by a four-stage pipeline: extraction of temporal facts, construction of a temporal knowledge graph, hierarchical verification against internal consistency rules and external reference sources, and score aggregation with automatically generated explanations. Verification combines eight internal checkers derived from Allen's interval algebra with a five-level external hierarchy ranging from a locally stored reference knowledge base of 1{,}256 curated political facts to live Wikidata SPARQL queries. On a benchmark of 100 political news articles with injected temporal errors, the system reaches a precision of 0.909 at the selected operating threshold, with a single residual false positive, a profile deliberately tuned for human-in-the-loop fact-checking assistance, where false alarms are costlier than missed detections. Unlike lexical baselines that output only a binary label, every flagged article is accompanied by the inconsistency type, the entities involved, and the reference source that contradicts the claim.

I. INTRODUCTION

The paper targets temporal errors that stylistic detectors can miss and addresses gaps in article-level temporal coherence scoring and explainable verification. It introduces TCS and an end-to-end pipeline evaluated on political news with injected temporal errors.

  • Motivation: Temporal errors can make credible, stylistically neutral political articles factually wrong while remaining difficult for style-based detectors to identify.Examples include attributing laws outside a politician’s mandate and reversing the order of treaties and the events they addressed.
  • Research gaps: Existing temporal fact verification handles isolated claims, leaving article-level coherence measurement and explainable verdicts underdeveloped.The paper identifies the absence of a dedicated interpretable metric for interdependent claims and the limited use of verifiable justifications in editorial workflows.
  • Contributions: The Temporal Coherence Score is a continuous value in [0, 1] that aggregates severity-weighted inconsistencies, graph coherence, and external verification coverage.Its factors correspond to defined semantic concepts, making the score interpretable by construction.
  • Contributions: The end-to-end system performs extraction, temporal knowledge graph construction, hierarchical verification, scoring, and explanation on-premise.A compact local language model generates natural-language explanations.
  • Evaluation: The evaluation uses 100 political news articles containing five categories of injected temporal errors and includes threshold analysis, lexical-baseline comparison, and external-dataset scope testing.The benchmark is purpose-built to evaluate temporal inconsistency detection and applicability boundaries.

II. RELATED WORK

Prior work covers fake-news detection, temporal claim verification, and explainability, but the paper positions TCS as their article-level temporal combination. Its novelty is the joint use of temporal graphs, explanations, and an interpretable aggregate score.

  • Fake-news detection: Fake-news detection includes knowledge-, style-, propagation-, and source-based methods, but style-based classifiers miss factual errors in stylistically neutral text.Structured-source verification addresses this vulnerability, while LIAR and FEVER operate on short isolated claims without article-level temporal context.
  • Temporal verification: TemporalFC, TSVer, and ChronoFact address temporal verification but operate primarily at claim level rather than producing an article-level score with explanations.Their tasks include validity intervals, time-series evidence, and event timelines.
  • Temporal reasoning: Prior studies show that implicit temporal constraints and interval-algebra validation matter for reliable temporal reasoning and can support justified decisions.The present work adopts a fixed-checker strategy as a complementary approach to automatically discovered constraints.
  • Explainability: Explainability research identifies human-in-the-loop integration as underexplored, while existing explainable article-level systems do not address temporal coherence.Other structured-justification approaches likewise lack an article-level temporal coherence metric.
  • Novelty: The paper claims no existing system combines article-level input, temporal-knowledge-graph verification, explanation generation, and an aggregated interpretable score.This combination is presented as the contribution of the work.

III. METHOD

The method represents article-level temporal facts in a structured graph, verifies their consistency and external support, and aggregates the findings into the TCS. It targets five temporal inconsistency categories spanning mandates, events, causes, precedence, and simultaneous roles.

  • Problem formalization: The system extracts temporal facts as entity–relation–value tuples with point or interval anchors, using five relation types selected for political-news frequency.The relation types are HOLDS_POSITION, OCCURRED_ON, MEMBER_OF, FOLLOWED, and GENERIC.
  • Problem formalization: Facts are checked against a reference knowledge base and against one another, then aggregated into TCS ∈ [0, 1] for threshold-based suspicion classification.An article is classified as temporally suspect when TCS < θ, with θ determined empirically.
  • Target inconsistencies: The system targets mandate errors, anachronisms, causal violations, temporal cycles in precedence chains, and impossible simultaneous roles.These categories cover incorrect validity intervals, premature events or policies, inverted causality, cyclic ordering, and mutually exclusive concurrent positions.
  • Pipeline: The four-stage TCS pipeline comprises extraction, knowledge graph construction, hierarchical verification, and scoring with explanation.The stages transform temporal facts into an interpretable article-level result.

B. Pipeline Architecture

The pipeline separates extraction, graph construction, internal checking, external verification, and score generation. Its verification architecture combines graph-only interval reasoning with escalating reference-source checks.

  • Architecture: The pipeline exposes four sequential components through a REST API and web interface, transforming raw article text into a score and explanation.Figure 1 summarizes the flow and the components maintain strict separation of responsibilities.
  • C1 — Extraction: Extraction derives temporally anchored subject–predicate–object triples through four strategies with decreasing confidence.The strategies are dependency-based extraction, reflexive event facts, nominal association, and entity–date fallback.
  • C2 — Temporal knowledge graph: Graph construction filters facts by confidence and valid temporal anchors, deduplicates exact tuple signatures, and inserts them into a directed temporal multigraph.An optional graph database provides cross-article persistence and caching.
  • External verification: Unresolved facts are escalated through local and cached knowledge sources, Wikidata SPARQL, and later recent-fact verification before severity aggregation.The pseudocode shows internal checking first, followed by successive external verification levels and LOW/MEDIUM/HIGH/CRITICAL severity aggregation.
  • C3 — Hierarchical verification: Eight internal checkers operate without network access to detect cycles, causal and ordering errors, incompatible roles, premature positions or actions, and future events presented as past.The checkers are labeled V1–V8 and run on the graph alone.
  • C3 — Hierarchical verification: The checkers instantiate Allen’s interval algebra, whose thirteen mutually exclusive relations support mandate, role, and precedence-cycle checks.Explicit tolerances include 30 days for inverted intervals and 180 days between election and investiture.

C. Temporal Coherence Score

The Temporal Coherence Score aggregates verification outcomes into an interpretable score whose components reflect detected inconsistencies, temporal coherence, and external coverage. The system also generates explanations and supports human review while offering alternative extraction paths with different latency and coverage trade-offs.

  • Score aggregation: The penalty ratio is a severity-weighted sum of detected inconsistencies, normalized by the maximum possible penalty, while coherence measures unaffected temporal relations.Severity weights range from 0.0 to 1.5, and the coherence factor counts relations unaffected by inconsistencies of MEDIUM severity or higher.
  • Score aggregation: External verification coverage contributes multiplicatively but is bounded to [0.85, 1.00], limiting its influence to at most 15%.Penalty and internal coherence therefore dominate the score by design.
  • Explanation generation: The explanation flow uses the inconsistency list for narrative generation, while detected severity drives both the TCS penalty and the user-interface highlight.If generation fails or times out, static per-type templates preserve a justification for every result.
  • Interpretability: Articles with no extractable temporal facts receive the neutral score 0.5 and are labeled as having insufficient temporal data rather than given a truth verdict.The score is intrinsically interpretable because its named factors and detected inconsistencies directly justify classification.
  • System operation: Pipeline A favors high-throughput batch processing, whereas Pipeline B favors coverage of implicitly phrased claims but adds model-inference latency.The two extraction strategies are therefore available as user-selectable alternatives rather than a single fixed extractor.

A. Benchmark and Protocol

Evaluation uses a purpose-built benchmark of 100 synthetic political news articles, balancing temporally coherent examples with articles containing one controlled temporal error across five error categories.

  • Dataset: The benchmark contains 100 articles: 56 temporally coherent TRUE articles and 44 FAKE articles with exactly one injected temporal error.TRUE chronologies were validated against Wikidata, while FAKE articles were derived through controlled error injection.
  • Error categories: The injected errors cover date mismatch, ordering error, future-as-past, entity inconsistency, and implicit contradiction.Each FAKE article is annotated with its injected inconsistency type.
  • Decision rule: An article is predicted FAKE when it contains at least one temporal fact and its TCS falls below the decision threshold.Articles without temporal facts follow the benchmark protocol’s separate handling for insufficient temporal evidence.

B. Results

At the selected operating point, the system prioritizes precision: 10 of 11 flagged articles are truly fake, while recall remains low. Iterative corrections improved precision over the baseline, and case studies illustrate both accurate scoring and severe temporal-error detection.

  • Operating-point results: Precision is 0.909 at the operating point: 10 of 11 flagged articles are truly fake, leaving one false alarm.The estimate is based on only 11 positive predictions, with a 95% Wilson score interval of [0.62, 0.98].
  • Operating-point results: Recall is deliberately low, detecting 10 of 44 fake articles at the precision-oriented operating point.The mean TCS is 0.880 for true articles versus 0.747 for fake articles, a separation of +0.133 with partial overlap.
  • Iterative refinement: Targeted corrections raised precision from 0.714 with four false positives to 0.909 with one false positive.The corrections included strict object matching, canonical-event verification, reduced date tolerance, role-category grouping, and an entity identity guard.
  • Case studies: The Obama case yields TCS = 0.925 with zero inconsistencies, whereas the false Clinton date claim yields TCS = 0.0 after six HIGH-severity date mismatches.The contrasting cases demonstrate how mandate intervals support both a coherent score and an explicit contradiction explanation.

C. Comparison with Lexical Baselines

The proposed system is less effective than lexical classifiers on this benchmark’s lexical-patterned false class, but uniquely provides actionable explanations and is specialized for temporal inconsistencies.

  • F1 = 0.572 for random forest and F1 = 0.535 for SVM exceed the proposed system’s F1 of 0.364.The lexical classifiers used TF–IDF features with 5,000 unigram and bigram terms.
  • Every TCS verdict identifies the inconsistency type, involved entities, and contradicting reference source, unlike baselines that emit only binary labels.This explanation format is presented as the functional requirement for editorial integration.
  • F1 falls to 0.000 on ISOT and 0.039 on RAGuard, delimiting TCS’s applicability to temporal inconsistencies in political narratives.ISOT emphasizes emotional style, while RAGuard contains short claims with limited temporal context and often receives the neutral score 0.5.
  • Failure analysis: The single residual false positive arises when V4 matches objects linked to distinct entities legitimately co-involved in the same events.The checker lacks sufficient narrative context to distinguish co-involvement from contradiction.
  • Failure analysis: The primary limitations are low recall, a synthetic benchmark, dependence on extraction and reference coverage, and restriction to English-language text.An extended 150-article benchmark is identified for future validation.

V. CONCLUSIONS

The paper concludes that TCS provides an interpretable article-level signal for temporal inconsistencies, combining temporal graphs, internal checks, and hierarchical external verification. Its precision-oriented profile supports human-assisted fact-checking, while future work targets recall and broader applicability.

  • 0.909 precision with a single false positive supports TCS as a precision-oriented aid for human-in-the-loop fact-checking.The selected operating point prioritizes avoiding false alarms.
  • TCS combines temporal knowledge graphs, eight interval-algebra checkers, and five-level external verification.The metric is computed through these complementary verification components.
  • Future work targets recall through broader reference coverage, improved extraction of implicit temporal facts, and context-sensitive contradiction verification.The stated goal includes eliminating the residual false positive.
  • The system is presented as a viable explainable signal for a specific class of disinformation, complementing rather than replacing lexical detectors.

ETHICAL CONSIDERATIONS

The system is intended to assist human fact-checkers rather than autonomously determine article veracity, because its precision-oriented design has low recall and limited generalization.

  • The system should assist, not replace, human fact-checkers because its low recall and limited domain generalization constrain autonomous use.The stated boundary is especially important given the reputational risk of erroneous flags about public figures.
Loading 2608.29175v1…