Collection-Date Assignment Redirects Answers in Temporal QA

1 Introduction

• The central question is whether changing only collection dates redirects an answer about t1 toward the earlier t0 reference despite both revisions being supplied.The motivating example contrasts an athlete’s former and newer teams.Source: §1 · p. 1
• Collection dates record when revisions were acquired, not when their contents changed, so collection order need not follow revision order.The study examines answer selection after both revisions enter the context, not retrieval improvement.Source: §1 · p. 1
• Swapping collection dates changes later-reference EM in all five LLMs, with large drops when dates match or lie near the question date.The result replicates on nonoverlapping page sets; it does not establish a general nearest-date rule.Source: §1 · p. 1
• The contribution combines fixed-evidence date interventions, replication, and controls for ordering, instructions, wording, and archive provenance.Answer-content measures distinguish answer selection from wording changes, while reasoning and closed-book comparisons characterize sensitivity.Source: §1 · p. 2

2 Task, Evidence and Measurements

• DynaQuest supplies dated questions and reference answers, while BM25 selects up to four 1,000-character passages per revision using the question alone.Answer presence is not required in the selected evidence.Source: §2 · p. 2
• The 595-page cohort supports the main date-assignment experiment, with 497 pages eligible for question-day/previous-day swaps; separate 596- and 599-page cohorts support controls and comparisons.The 128-page swap replication uses the separate 596-page cohort, and studies use raw Wikipedia markup.Source: §2 · p. 2
• Within-study date contrasts fix the text, while cross-study accuracy also reflects cohort and rendering differences.Inputs preserve words and numbers in plain text.Source: §2 · p. 2
• Five open-weight LLMs generate short answers, scored by normalized exact match and additional measures including earlier-reference inclusion and token F1.Alias inclusion captures answer content beyond exact wording but does not establish entailment.Source: §2 · p. 2

3 Controlled Date Assignments

• For t1 questions, the matching swap exchanges January 10 and 11 between revisions while keeping date values, question, texts, and revision timestamps fixed.Both older-first and newer-first input orders are tested and averaged per page.Source: §3 · p. 3
• The 497-page past-side comparison contrasts (0, −1) with (−1, 0), alongside a rerun of (0, +1) versus (+1, 0), separating question-date matching from earliest collection.Both revisions must be timestamped by midnight on t1 −1.Source: §3 · p. 3
• Seven prompt variants test equivalent timestamp explanations, ignore-collection instructions, archive provenance, field renaming, and field placement against original-prompt controls.The variants retain both input orders and fixed evidence.Source: §3 · p. 3
• A 128-record synthetic test varies four relations and two formats, while a separate test changes Wikipedia question dates from ISO to English month names.Both tests use all five LLMs and both assignments and orders.Source: §3 · p. 3
• Shared-date, unknown, definition-only, omission, and nondate conditions distinguish date values from field presence and prompt format; only (c, c) uses observed acquisition dates in the 128-page replication.That replication also tests assigned c and c + 7, plus shared c + 30.Source: §3 · p. 3
• Reasoning comparisons test Qwen3.5 in matched thinking and nonthinking modes, native reasoning models, and API models across matching swaps and related answer-selection tasks.The A/B answer-selection task tests shared-date conditions, not the matching swap.Source: §3 · pp. 3–4
• Paired effects use predicate-cluster bootstrap intervals after averaging source orders within page, with adjusted decisions reported separately from nominal intervals.Secondary reanalyses are descriptive.Source: §3 · p. 4

4 Date Assignment Redirects Answers

• 37.6–51.4 percentage-point drops in later-reference EM occur across five LLMs when the older revision receives the question-day collection date on 595 Wikipedia pages.Earlier-reference EM rises 37.6–49.8 points under the same swap, with revision texts and timestamps unchanged.Source: §4 · p. 4
• Near-question proximity swaps lower EM by 28.2–38.9 points for four LLMs, while Qwen2.5-7B changes by +1.2 points; the effect is not symmetric across models.Qwen2.5-7B loses 40.1 points in the question-date matching swap, and distant swaps are much smaller.Source: §4 · p. 4
• 38.3–53.1-point EM losses recur across all five LLMs on 128 disjoint Wikipedia pages under question-date matching swaps.The pages do not overlap with the 595-page set, though they were previously used for other conditions.Source: §4 · p. 4, Table 2 · p. 5
• 29.0–39.6-point losses persist on 497 pages when collection dates fall on or before the question date, despite the question-day collection going to the newer revision.Earlier-reference EM rises 22.5–37.1 points; the result rules out earliest collection alone as an explanation.Source: §4 · p. 4
• All seven prompt variants retain large swap losses, including 37.9–50.8 points with ignore-collection instructions and 42.2–57.8 points with archived-revision provenance.Fictional records also show 92.2–95.7-point losses, while English month-name question dates retain 31.6–53.1-point losses, so remembered answers and identical date strings are unnecessary.Source: §4 · pp. 4–5

5 Date Roles and Answer Selection

• Displaying one observed acquisition date on both revisions yields 3.5–15.2-point lower later-reference EM than omission on 128 pages.After expanded correction, omission contrasts remain in three LLMs, but the additional date-value effect relative to unknown is unresolved in all five.Source: §5 · p. 5
• Publication, last-modified, and explicitly irrelevant administrative dates also change answers in the 595-page plain-display study.These use shared-minus-question-matched comparisons rather than the main swap.Source: §5 · p. 5
• On 411 pages containing both reference answers, later-reference EM falls 44.6–59.4 points while earlier-reference EM rises 44.4–57.5 points.In A/B answer selection, only Phi-4 shows corrected evidence of an earlier-answer shift at both question times.Source: §5 · p. 6
• The nondate placebo changes EM by −1.5 to +0.6 points, while direct shared-minus-channel-cache effects remain negative in four LLMs, with the 72B interval including zero.Missing-value baselines also matter: Phi-4’s later EM is 4.4 points lower with unknown than definition-only.Source: §5 · p. 6

Matched thinking

• Thinking reduces Qwen3.5’s EM swap penalty from −30.1 to −11.1 points, but residual losses remain and the comparison does not isolate thinking from greedy outputs.The matched modes share sampling settings and output caps; their interaction is +18.9 points, with interval [11.1, 26.5].Source: §6 · p. 6, Table 6 · p. 7

API tests

• Wikipedia API swap effects are small and intervals establish neither a loss nor an effort interaction; the tested subsets are conditioned on closed-book misses.The comparison includes 2,560 responses across high- and low-effort settings.Source: §6 · p. 7

Synthetic answer content

• On fictional records, Claude’s low-effort loss mainly reflects longer answers: 99.6% include the later answer, while 0.4% contain only the earlier answer.Including an answer does not establish unambiguous selection.Source: §6 · p. 7

Distant dates

• Distant-date swaps yield only 0.0–6.2 EM points for t0 and 0.0–2.7 for t1; Temporal Wiki’s year-level questions leave the chronology-valid exact-match contrast untested.Source: §6 · p. 7

Revision selection

• 15.0–30.9 points: selecting the latest eligible revision before generation improves t0 EM over two-version omission, but assumes known source identity and can discard later corrections.Source: §6 · p. 7

Audit

• A practical audit keeps revision evidence fixed, swaps chronology-valid collection dates in both orders, and scores agreement and earlier-answer content.Source: §6 · p. 7

Evidence conflicts and temporal validity

• Conflict and temporal-validity studies examine conflicting sources, evidence receptiveness, changing facts, and retrieval conflicts; this work changes a separate collection-date field while fixing both revision texts and timestamps.The cited lines of work include ClashEval and ConflictBank, among others.Source: §7 · p. 7

Preselection and source credibility

• Prior work studies stale-retrieval selection, version lineage, publication-date injection, and temporal ordering; this study tests answering with fixed evidence and isolates date assignment between two revisions.Date injection itself is not new; the paper distinguishes the collection-date field’s role from revision timestamps.Source: §7 · p. 8

Surface cues and controlled diagnostics

• Heuristic, date-format, temporal-reasoning, option-order, and context-position studies motivate controlled cue tests; this paper varies timestamp definitions and instructions while keeping conflicting revision evidence fixed.Source: §7 · p. 8

Temporal benchmarks and scoring

• Temporal QA benchmarks study changing answers and updates; this diagnostic uses Özer and Yıldız’s distinct dataset rather than Jang et al.’s TemporalWiki and separates reference agreement from factual error.Source: §7 · p. 8

• Collection-date swaps redirect answers toward earlier answers on disjoint Wikipedia pages and fictional records despite fixed revision texts and timestamps.Source: §8 · p. 8
• Losses persist across question-date directions, seven prompt variants, and English question dates; neither earliest collection nor identical date strings is required.Earlier-answer content increases.Source: §8 · p. 8
• Matched thinking reduces Qwen3.5’s penalty to 11.1 points, native reasoners differ, and API Wikipedia effects are small; Claude’s synthetic low-effort loss mainly reflects longer answers.Source: §8 · p. 8
• Large losses concern matching and nearby collection dates, while distant swaps are much smaller; displaying one shared observed acquisition date lowers agreement relative to omission in three of five LLMs after correction.The date-display comparison combines a field and its value rather than isolating field presence.Source: §8 · p. 8
• The conclusion recommends assessing evidence, date assignment, and answer form separately rather than assuming metadata is neutral.Source: §8 · p. 8

Limitations

• EM measures agreement with dated benchmark references, not independently adjudicated historical truth.The study does not establish whether edits describe events or correct earlier statements; alias inclusion checks lexical availability, not entailment.Source: Limitations · p. 9
• Assigned-date losses and mirror/cache scenarios do not estimate live-system error rates, and only one acquisition day is observed.Distant-date and external proximity swaps do not reproduce the large near-question losses, but that does not establish distant metadata is neutral.Source: Limitations · p. 9
• Seven prompts do not represent a prompt population, and the matching swap has not been crossed with removing revision timestamps.Thus, cue competition is a behavioral description rather than an identified internal weighting rule; tested prompt controls do not rule out other remedies.Source: Limitations · p. 9
• The three principal cohorts are page-disjoint, but follow-ups reuse pages and were informed by earlier results.Local plans precede their own outputs, not all preceding research, and are not public preregistration.Source: Limitations · p. 9
• The 32-page reasoning study is nested within the 128-page expansion, and matched thinking tests one Qwen family with two seeds.Cross-family differences do not isolate reasoning; incomplete final answers remain in the denominator, with completion bounds distinct from confidence intervals.Source: Limitations · p. 9
• API comparisons cannot separate model identity from reasoning presence, while revision selection assumes known lineage and question time.It can discard later corrections and is not a general solution to temporal factuality or a measured latency or energy reduction.Source: Limitations · p. 9

Ethical Considerations

• The paper reports no new human participants, labels assigned dates as interventions, and retains source attribution and applicable data, repository, and code licenses.Supplements exclude credentials, accounts, and model weights.Source: Ethical Considerations · p. 9

A Samples, Rendering and Measurement

• 595 pages remain from the main 600-page sample, and the 128-page date-swapping replication is a fixed subset of a disjoint 596-page cohort.The three principal cohorts are disjoint by canonical page ID; exclusions reflect recorded prior use, not pretraining.Source: A Samples, Rendering and Measurement · p. 13
• BM25 selects up to four 1,000-character evidence blocks per revision using only the question, while raw and plain displays preserve the same ordered words and numbers.The cohorts use different rendering settings, so their absolute accuracy levels are not estimates of relative cohort difficulty.Source: A Samples, Rendering and Measurement · p. 13, Table 8 · p. 14

B Exact Inputs and Execution

• The input variants distinguish question-matched, shared-date, unknown, definition-only, and omission conditions, with a channel placebo describing how rather than when a revision was fetched.The system prompt states that fetched and revision timestamps do not guarantee that every statement is true at the requested time.Source: B Exact Inputs and Execution · pp. 13–15

C Assignment, Replication and Content

• The past-side test covers 497 pages whose revision timestamps meet its chronology constraints; the separate 411-page alias subset is defined by reference answers appearing in evidence, not by response behavior.The alias subset does not label the remaining pages unanswerable.Source: D Date Direction and Prompt Robustness · p. 15, C Assignment, Replication and Content · p. 15

D Date Direction and Prompt Robustness

• Only three of six prompt-variant interactions survive the broad follow-up correction, and each increases rather than removes the swap penalty.Unresolved interactions do not establish equivalence.Source: D Date Direction and Prompt Robustness · p. 17

E Baselines and Constrained Choice

• For shared-date comparisons, a lower exact-match score is not itself an earlier-answer error rate, because nonexact responses may still contain the later alias.The A/B task removes refusal and answer-form variation but changes the task, so opposite answer-selection shifts do not identify the free-answer mechanism.Source: E Baselines and Constrained Choice · pp. 18–19

Matched reasoning and completion

• The reasoning comparison changes only Qwen3.5 thinking, while native reasoners use separate settings; equal token caps do not imply equal token consumption.The initial 32-page mode study remains inconclusive under its original correction, as does the separate shared-date interaction.Source: F Reasoning, Distance and API Controls · pp. 19–20

Distant and external proximity

• No distant-date EM effect survives the complete 72-test follow-up adjustment on either the 128-page DynaQuest set or the 262-page Temporal Wiki set.These results are not evidence of equivalence, and the external diagnostic is not entirely unseen-data evaluation.Source: F Reasoning, Distance and API Controls · p. 20, Table 29 · p. 21, Table 30 · p. 21

Independent API calls

• The API study uses independent, tool-free requests, but provider instructions and immutable backend versions are not observable.The closed-book questions precede documented knowledge cutoffs, so prior exposure remains possible.Source: F Reasoning, Distance and API Controls · p. 21
• Closed-book miss and no-later-alias subsets are fixed from complete high-effort responses, but neither subset proves that the model lacked the relevant knowledge.Conditional intervals fix observed subset membership; all 128 pages remain the primary analysis.Source: F Reasoning, Distance and API Controls · p. 21, Table 32 · p. 22

G Synthetic Records and Question-Date Spelling

• The synthetic test fixes 128 fictional records across four relations and two formats, with deterministic nonce names and known state changes.Record-bootstrap intervals condition on eight templates and do not estimate generalization to natural QA or a template population.Source: G Synthetic Records and Question-Date… · p. 22
• All five synthetic and all five English-format EM swap effects survive the Holm correction across 60 follow-up tests.Only Phi-4’s spelling interaction survives; the other spelling interactions remain unresolved.Source: G Synthetic Records and Question-Date… · p. 23
• Claude low-effort API responses on fictional records contain 36 nonexact older-matching outputs: 35 retain the later alias in longer answers, while one contains only the earlier alias.No post hoc extraction replaces the frozen whole-response EM score.Source: G Synthetic Records and Question-Date… · p. 24

H Selection and Reproduction

• Selection of the latest revision yields 15.0–30.9-point earlier-question EM gains, but selection can discard later corrections.The comparison uses paired controls from the same 595-page plain-display execution.Source: H Selection and Reproduction · p. 24
• The reproduction package covers ten principal experiment blocks and includes exact messages, answers, outputs, scoring code, and 5,060 numerical checks.Public release of code, prompts, and outputs is planned on acceptance.Source: H Selection and Reproduction · p. 24