Beyond the Evidence Coherence Trap: Agent-Driven Evidence Selection for Temporally Disordered Fake News Detection

1 Introduction

• Independent event checks can confirm every event yet miss a false temporal relation that makes the article misleading.The cited impeachment example involves two events that occurred, but the memorandum release preceded Schiff’s account by one day.Source: §1 · p. 2
• Prior temporal retrieval and timeline methods do not explicitly use article-level chronological assessment to revise selected evidence.When event mentions lack explicit dates, evidence choice and inferred event position depend on one another.Source: §1 · p. 3
• ChronoFake isolates temporal manipulation using factual event units, contrasting manipulated false articles with sequential and flashback true articles.Labels depend on whether expressed temporal relations match the evidence-grounded chronology, so surface order alone does not determine the label.Source: §1 · p. 3
• DelSA iteratively reselects evidence during training using article-implied event context and reusable rules updated from downstream verification feedback.It combines LLM-inferred article chronology with a verifier-trained reranker balancing local relevance and timeline consistency.Source: §1 · p. 3
• DelSA outperforms strong baselines on ChronoFake while remaining competitive on conventional fake news benchmarks.The conclusion names content-based, LLM-enhanced, and retrieval-augmented baselines, and reports competitive performance on PolitiFact and GossipCop.Source: §1 · p. 3

2.1.1 Content-Based Fake News Detection.

• Content-based detectors model textual, stylistic, semantic, or multimodal signals, whereas this task concerns factual events organized into a misleading chronology.The related work includes content and multimodal approaches such as HAN, BERT, dEFEND, BREAK, DRES, CAFE, HMCAN, CSFND, ALGM, and MIMoE-FND.Source: §2.1.1 · p. 4

2.1.2 LLM-Based Fake News Detection.

• LLM-based fake-news methods use rationales, generated evidence, explanations, or robustness signals, while DelSA accumulates reusable document-selection rules from temporal evidence-selection failures.FactLLaMA [10] grounds LLM verification in external knowledge; the described LLM detectors otherwise typically reason sample-independently or generate one-shot auxiliary signals.Source: §2.1.2 · p. 4

2.1.3 Evidence-based Fake News Detection.

• Evidence-based detectors improve grounding through retrieval and evidence interaction, but most prioritize semantic relevance, sufficiency, or context rather than temporally ambiguous evidence selection.Examples include multi-step retrieval, suspicious-fragment retrieval, multi-source evidence conversion, and multi-agent debate [17] [36].Source: §2.1.3 · pp. 4–5

2.2 Temporal Fact Verification

• Temporal verification work models time-aware evidence, temporal claims, or event timelines, but primarily targets claim-level verification or structured temporal evidence.This paper instead studies article-level chronological misinformation requiring iterative evidence repair and rule-guided selection under temporal ambiguity.Source: §2.2 · p. 5

3.1 Evidence Initialization

• DelSA extracts check-worthy events and article-implied temporal ranks from dates and discourse cues, so legitimate flashbacks need not follow mention order.It retrieves Wikipedia evidence using semantic relevance and entity alignment, then initializes each event with a top-ranked sentence and its timestamp.Source: §3.1 · pp. 7–8

3.2.1 Rule-Guided Evidence Scoring.

• An LLM scores evidence across six dimensions, while a learnable MLP reranks candidates using verifier-derived weak supervision from classification-loss changes.The dimensions include action and role consistency, semantic relevance, context quality, temporal alignment, and evidence role.Source: §3.2.1 · p. 9

3.2.2 Iterative Evidence Update.

• DelSA synchronously reconsiders all event evidence against the evolving evidence-grounded timeline, using classifier transitions and prior update feedback to guide revisions.Wrong-to-right and right-to-wrong trajectories become positive and negative memories; unresolved wrong-to-wrong cases continue with feedback, while validation and testing use a frozen rulebook for one-pass selection.Source: §3.2.2 · pp. 10–11

3.2.3 Rulebook Updating.

• During training, DelSA updates its rulebook from verifier-guided positive and negative evidence-selection memories, then freezes it for validation and testing.The LLM abstracts reusable selection principles from these cases without copying raw articles or gold labels into the rulebook.Source: §3.2.3 · p. 11

3.3 Evidence-Grounded Classification

• DelSA encodes the displacement between the article-implied and evidence-grounded event orders as an explicit conflict signal, without treating legitimate flashbacks as chronology errors.The order representation compares evidence-derived chronological position with the LLM-inferred rank asserted by the article.Source: §3.3 · p. 12
• DelSA contextualizes the order-aware event–evidence sequence with a bidirectional LSTM to capture temporal dependencies and assess consistency with the article-level narrative.The sequence retains article-implied temporal positions through order embeddings after evidence update.Source: §3.3 · p. 13
• Attention pooling aggregates the sequence into an article-level representation, weighting pairs that indicate temporal inconsistency or misinformation before predicting real or fake labels.Source: §3.3 · p. 13
• The full training objective combines fake-news classification with reranker supervision, whose contribution is controlled by η.Source: §3.3 · p. 13

4 ChronoFake Benchmark Construction

• ChronoFake keeps event contents grounded in real-world evidence and introduces falsehood by manipulating temporal relations among events.It targets misleading chronology even when individual events are factually correct.Source: §4 · p. 13

4.1 Temporally Anchored Event Unit Construction

• Source documents are chunked to extract informative event statements, normalize temporal expressions, and produce coherent, temporally localized event units.Absolute dates are normalized directly; relative expressions are resolved using the closest valid local temporal reference.Source: §4.1 · pp. 13–14
• Entity-centered graphs link event units sharing entities within temporal windows, while source-disjoint splits keep each source document in only one partition.Graph neighborhoods support chronology perturbations that preserve entity continuity and discourse plausibility.Source: §4.1 · pp. 14–15

4.2 Temporal Narrative Generation

• ChronoFake includes sequential true articles, flashback true articles, and fake articles created by perturbing graph-connected events without changing their content.Perturbations include local reversal, cross-time insertion, and temporally inconsistent reordering, constrained to keep narratives semantically coherent and entity-consistent.Source: §4.2 · p. 15

4.3 Label Definition and Quality Control

• Labels depend on whether expressed temporal relations match the gold chronology, so valid flashbacks remain true; quality control filters ambiguous or incoherent instances.The benchmark thereby isolates temporal veracity from factual veracity.Source: §4.3 · pp. 15–16

5.1.1 Datasets.

• PolitiFact contains 741 articles, GossipCop 12,252, and ChronoFake 30,706 instances, including fake, sequential-true, and flashback-true articles.The general datasets assess performance beyond temporal manipulation; ChronoFake targets article-level temporal inconsistency.Source: §5.1.1 · p. 16, Table 1 · p. 16

5.1.2 Implementation Details.

• DelSA is trained for 10 epochs with a maximum sequence length of 256 and batch size 8, using at most five temporal update rounds.The reported metrics are accuracy, weighted precision, weighted recall, and weighted F1.Source: §5.1.2 · p. 16

5.1.3 Baselines.

• Baselines span content-based, LLM-enhanced, and retrieval-augmented approaches, including BERT, ARG, MUSER, and IMRRF.The retrieval-augmented group is included because the task centers on evidence-grounded temporal verification.Source: §5.1.3 · pp. 16–17

5.2 Performance Comparison

• DelSA outperforms traditional content-based baselines, which rely on surface features rather than temporal reasoning and LLM-guided evidence selection.The authors highlight resolving multi-time-point ambiguities through temporal reasoning and evidence selection.Source: §5.2 · pp. 17–18
• DelSA achieves the highest accuracy, precision, and recall and ties for best F1 on PolitiFact, while remaining highly competitive with the strongest content-based baseline on GossipCop.The performance comparison covers both datasets and reports only marginal differences across GossipCop’s main metrics.Source: §5.2 · p. 17, Table 2 · p. 18
• DelSA surpasses LLM-enhanced approaches by combining LLM reasoning with self-updating rules and iterative evidence alignment.The comparison indicates stronger results than relying on LLMs for auxiliary reasoning or semantic understanding alone.Source: §5.2 · p. 18

5.3 Temporally Disordered Fake News Detection

• Content-based, LLM-enhanced, and retrieval-augmented methods lack explicit temporal-order control or evidence consistency, leaving chronology conflicts insufficiently addressed.The authors note that retrieved evidence may be noisy or temporally mismatched.Source: §5.3 · pp. 18–19
• DelSA improves accuracy by 8.91%, precision by 7.58%, recall by 8.91%, and F1 by 9.17% over the best-performing baseline for each metric on ChronoFake.It outperforms all baselines across accuracy, precision, recall, and F1.Source: §5.3 · p. 18, Table 3 · p. 19
• DelSA combines temporal repair with rule-guided evidence selection to identify temporal conflicts and reduce the influence of irrelevant evidence.The authors report consistent improvements across all four metrics.Source: §5.3 · p. 19

5.4 Ablation Study

• DelSA scores weighted F1 of 0.924 on PolitiFact, 0.863 on GossipCop, and 0.893 on ChronoFake; all four ablations underperform the full model.Figure 3 summarizes component-wise ablation trends.Source: Figure 3 · p. 19, §5.4 · p. 20
• Removing the time-aware agent causes a clear performance drop, underscoring its role in repairing temporal event–evidence misalignment.The w/o agent variant uses initialized event–evidence pairs without the agent.Source: §5.4 · pp. 19–20
• Removing rulebook guidance degrades performance, as reusable selection rules help resolve ambiguity among evidence tied to multiple time points.The w/o rule variant removes rulebook guidance from evidence selection.Source: §5.4 · pp. 19–20
• Disabling the MLP reranker makes evidence selection less stable by removing learned calibration of LLM-generated quality scores.The w/o reranker variant lets the LLM select evidence directly.Source: §5.4 · p. 20
• Replacing the supervised classifier with direct LLM verification weakens integration of event-level evidence and article-level temporal structure.This ablation produces less reliable predictions.Source: §5.4 · p. 20

5.5 Sensitivity Analysis

• DelSA generally maintains stable performance across update-round settings on all three datasets, indicating limited sensitivity to this hyperparameter.Figure 4 plots F1-score against the maximum number of training-time evidence update rounds.Source: §5.5 · p. 20, Figure 4 · p. 20
• On PolitiFact, performance stabilizes after a small number of rounds, recovering from slight drops at shallow update depths.The authors suggest limited evidence correction is sufficient for reliable event–evidence alignment on this dataset.Source: §5.5 · p. 20
• GossipCop fluctuates more across update depths, but overall performance improves when more rounds are allowed to address complex evidence ambiguity.The authors suggest its more diverse topics and noisier evidence space may make reselection more sensitive to update depth.Source: §5.5 · p. 20
• On ChronoFake, DelSA performs strongly across most settings, especially with moderately large update depths.The authors associate iterative updating with repeatedly adjusting event–evidence assignments to reconstruct a more reliable evidence-grounded timeline.Source: §5.5 · p. 21

5.6 Out-of-Distribution Testing

• DelSA achieves the best out-of-distribution performance on GossipCop and ChronoFake in leave-one-dataset-out testing.Each run trains on two datasets and tests on the third unseen dataset.Source: §5.6 · p. 21
• On ChronoFake, DelSA obtains the highest accuracy and F1-score, including when temporally disordered samples are absent from training.Source: §5.6 · p. 21
• On PolitiFact, DelSA is slightly behind DRES but outperforms the LLM-based and retrieval-augmented baselines under domain shift.Source: §5.6 · p. 21

5.7 Case Study

• Initially, semantically coherent but temporally insufficient evidence fails to expose the article’s reversed chronology, and DelSA predicts it as real.The article places the 1868 opening after the 1869 closure and 1910 reopening.Source: §5.7 · p. 22
• Iterative updates align evidence with the 1869 closure, 1910 reopening, and 2 March 1868 opening, revealing the article’s chronological conflict and changing its prediction from real to fake.The reconstructed timeline places the 1868 opening and service access before the later closure and reopening.Source: §5.7 · p. 22

• DelSA outperforms evaluated content-based, LLM-enhanced, and retrieval-augmented baselines on ChronoFake while remaining competitive on PolitiFact and GossipCop.Source: §6 · pp. 22–23
• DelSA refines event-level evidence during training, reconstructs an evidence-grounded timeline, and uses a frozen rulebook for one-pass evidence selection at inference.Source: §6 · p. 22
• ChronoFake is a large-scale benchmark designed to evaluate models on temporally manipulated narratives.Source: §6 · p. 22