Source-linked AI summary

STAIR: Semantic-Temporal Automaton for Interpretable Reasoning in Temporal Question Answering

Xinlong Dai, Jinchuan Zhang, Lei Gao, Xinzhe Hu, Yuefeng He, Hui Gao

arXiv:2608.16224v1cs.CLcs.AI

TL;DR

Existing temporal QA systems struggle to combine flexible semantic interpretation with exact, verifiable temporal execution. STAIR separates these functions through an answer-free semantic adapter and guarded deterministic automaton, consistently improving over NeSTR across four TQA benchmarks.

  • Problem

    Temporal QA requires flexible interpretation of diverse formulations alongside unambiguous execution for interval and ordered-state decisions.

  • Method

    STAIR uses a rule-first architecture that maps questions to executable temporal intents and applies guarded deterministic policies, with semantic adaptation for failed rule paths.

  • Results

    STAIR consistently improves over the NeSTR baseline across four TQA benchmarks and supported CronQuestions subsets across open-source and proprietary models.

  • Takeaways & Limitations

    STAIR provides procedural traces reflecting the computation used for temporal decisions, including normalized intents, policies, guards, selected evidence, and provenance.

  • Takeaways & Limitations

    STAIR does not support highly implicit ordering, duration comparisons, negation, nested constraints, or multi-hop temporal composition within its predefined policy scope.

Abstract

from arXiv · show

By leveraging large-scale pretraining, LLMs can interpret diverse temporal expressions and question formulations without task-specific training. However, existing prompt-based neuro-symbolic systems continue to rely on LLMs for both semantic interpretation and exact temporal inference. Consequently, discrete decisions regarding intervals, time anchors, and ordered states remain vulnerable to probabilistic errors and difficult to verify. We present STAIR, a \textbf{S}emantic-\textbf{T}emporal \textbf{A}utomaton for \textbf{I}nterpretable \textbf{R}easoning. STAIR separates semantic interpretation from precise temporal inference: an answer-free LLM adapter maps complex question formulations to normalized temporal intents, while a deterministic temporal automaton with finite control and guarded transitions executes the corresponding policies over canonicalized evidence. Following a rule-first design, STAIR resolves standard questions without invoking an LLM and applies semantic adaptation only when the rule path fails to produce an executable intent. This approach reduces free-form reasoning, making temporal decisions verifiable and interpretable. Specifically, guarded execution supports precise point-time containment and before/after selection, while semantic adaptation handles non-exact intervals and time-anchored queries. Across the TimeQA-Easy, TimeQA-Hard, TempReason-L2, and TempReason-L3 datasets, STAIR consistently outperforms strong baselines in the TQA task using matched model settings, achieving average F1 improvements of 16.57\% and 3.10\% when utilizing the Qwen2.5-7B and GPT-4o-mini models, respectively. Furthermore, ablations and diagnostic analyses demonstrate that STAIR excels at handling both boundary-sensitive and order-sensitive queries, while its guarded execution and semantic adaptation ensure precise point-time reasoning and inexact intervals, respectively.

Introduction

STAIR addresses temporal question answering by separating flexible semantic interpretation from precise, deterministic temporal execution. Its rule-first automaton resolves canonicalized temporal relations with guarded, interpretable policies, while an answer-free semantic adapter handles difficult expressions when rule parsing fails.

  • Motivation: Temporal question answering requires boundary-sensitive interval and point-containment reasoning alongside order-sensitive predecessor, successor, and anchored before/after selection.These constraints arise from multiple time-indexed facts and varied question-specific temporal requirements.
  • Motivation: Flexible LLM interpretation can select incorrect adjacent states, whereas strict rule matching fails when query boundaries do not exactly align with supporting facts.The challenge is the mismatch between diverse surface forms and exact temporal execution.
  • Motivation: 76.91% of TimeQA-Easy, 76.91% of TempReason-L2, and 91.46% of TempReason-L3 are resolved by STAIR’s rule-only path, compared with 27.65% of TimeQA-Hard.Rule-only execution mainly handles explicit intervals, point-time containment, and entity-anchored before/after relations; TimeQA-Hard contains more non-exact intervals.
  • STAIR: STAIR uses a rule-first architecture in which canonicalized facts and recognizable questions become finite executable intents for a temporal automaton with explicit policies and guarded transitions.The system separates semantic interpretation from deterministic temporal execution.
  • STAIR: When rule-based parsing fails, an answer-free semantic adapter maps difficult temporal expressions into executable intents, while validation and constrained repair ensure executability.The adapter targets non-exact intervals and time-anchored before/after questions without selecting the final answer.
  • Interpretability: STAIR exposes canonical facts, normalized intents, activated policies, guard outcomes, selected evidence, answer provenance, repairs, and fallback invocations as procedural computation traces.These records reflect the actual computation performed rather than post-hoc explanations.

Related Work

Prior temporal QA research spans benchmarks, stronger models, temporal representations, prompting, and inference-time reasoning. STAIR builds on programmatic and cognitive perspectives by emphasizing zero-shot temporal QA with explicit execution.

  • TQA Benchmarks and Prior Systems: Temporal QA benchmarks extend relation extraction to questions over time-indexed facts, evolving entities, and multi-level temporal relations.The cited resources include TempQuestions, TimeQA, and TempReason, alongside TempEval for temporal relation identification.
  • TQA Benchmarks and Prior Systems: Prior systems enhance temporal modeling through stronger reading-comprehension models, temporal pretraining, structured knowledge representations, prompting, and programmatic reasoning.These approaches are associated with supervised adaptation, task-specific graph construction, or LLM-mediated inference.
  • Inference-Time and Neuro-Symbolic Reasoning: Inference-time and neuro-symbolic methods use rationales, sampled paths, search, actions, feedback, self-reflection, or symbolic representations to improve reasoning.In temporal QA, TISER revises constructed timelines through self-reflection, while NeSTR combines symbolic representations with abductive reasoning.
  • Programmatic Execution and Cognitive Perspectives: Program-aided modeling externalizes exact operations, aligning with cognitive offloading, dual-process reasoning, and classical interval formalisms for precise temporal comparison.These perspectives motivate separating flexible interpretation from deliberate symbolic manipulation.

Method

STAIR separates semantic interpretation from deterministic temporal inference by canonicalizing evidence, validating executable intents, and applying guarded automaton policies. It follows a rule-first route, uses answer-free semantic repair for difficult cases, and invokes a four-stage LLM fallback only after deterministic execution fails.

  • Rule-First Routing: Rule-first execution handles recognizable operators without an LLM, while semantic adaptation is restricted to difficult cases such as non-exact intervals, anchored queries, and unresolved dates.The hard-structure detector routes supported failures to an answer-free adapter, reducing unnecessary LLM calls and semantic drift.
  • Canonical Representation: STAIR canonicalizes temporal facts as tuples (s, r, o, ts, te), normalizes values, rejects incomplete records, merges duplicate keys, and preserves provenance for auditable evidence tracing.Fact construction combines deterministic rule parsing with constrained LLM-based symbolic parsing; subsequent reasoning operates on inspectable structured records.
  • Semantic Adaptation: The answer-free semantic adapter produces a structured intent specifying the operator, arguments, and execution policy, but cannot select evidence or generate an answer.Supported intent types include interval, point, before_anchor, after_anchor, first, and last.
  • Semantic Adaptation: Validation canonicalizes aliases and dates while rejecting unsupported intents, missing arguments, invalid dates, and answer-like content before deterministic reselection.Repaired facts and intents must pass the same validation procedure.
  • Guarded Execution: The guarded finite-state automaton verifies facts, intents, fact-group selection, policy output, and answer aggregation, returning typed failure reasons when guards fail.A successful trace proceeds from S0 through S5; failed execution enters S⊥, and the first failed guard identifies the stage requiring repair.
  • Temporal Policies: For before/after queries, TAS selects the nearest predecessor or successor in the relevant temporal state chain, while point-time queries use containment and first/last queries select temporal extremes.Policies route instances to S⊥ when they yield no admissible evidence or multiple distinct answers.

Experiments

Across matched evaluations, STAIR achieves the strongest overall EM and F1, with especially large gains on boundary- and order-sensitive TempReason queries. Ablations, transfer results, and efficiency analyses show that semantic adaptation expands coverage while deterministic selection preserves interpretable execution at a structure-dependent computational cost.

  • Overall performance: STAIR records the highest average EM and F1 across models, improving EM in all 16 model-dataset configurations and F1 in 15.Relative to NeSTR, average F1 gains range from 2.78 points with GPT-4o-mini to 12.71 points with Qwen2.5-7B, while EM gains range from 5.96 to 17.08 points.
  • Dataset analysis: Average F1 gains over NeSTR reach 9.48 points on TempReason-L2 and 9.62 points on TempReason-L3, versus 1.89 on TimeQA-Easy and 3.08 on TimeQA-Hard.These gains align with TempReason’s point-time containment and entity-anchored before/after queries, which directly test boundary- and order-sensitive decisions.
  • Ablation study: The semantic adapter raises F1 from 72.75 to 80.91 while reducing the final fallback rate from 72.35% to 37%.The hard-structure detector alone provides little benefit, whereas validation and repair add modest accuracy while improving reliability.
  • Cross-source transfer: On the converted, operator-supported CronQuestions subset, STAIR surpasses NeSTR by 16.42 EM points and 6.60 F1 points.The subset contains 13,096 instances after excluding unsupported tasks, answer types, and annotations lacking the required head field.
  • Efficiency: On TempReason-L3, average calls fall from 1 to 0.21 and wall-clock time from 3.90s to 0.31s, whereas TimeQA-Hard calls rise from 1 to 2.12.Complex non-exact intervals and time-anchored before/after questions require extra adaptation and repair, reflecting a structure-dependent accuracy-efficiency trade-off.

Diagnostic Analysis

STAIR’s diagnostic analysis shows strong gains across boundary-sensitive and order-sensitive temporal query categories, while residual errors cluster around noncanonical contexts and unsupported operators. Semantic adaptation broadens coverage, but fallback predictions lose the deterministic TAS module’s procedural interpretability.

  • Category-level Question Structure: Diagnostics divide temporal challenges into non-exact interval, point-time, and time-anchored before/after queries, covering boundary-sensitive and order-sensitive reasoning.Non-exact intervals require overlap, point-time queries require containment, and anchored queries combine anchor interpretation with predecessor or successor selection.
  • Category-level Question Structure: 8.19 points: STAIR’s F1 gain on non-exact intervals relies strongly on semantic adaptation.Point-time queries gain 9.28 points and are handled without adapter intervention, indicating deterministic execution benefits both normalized and explicit operators.
  • Category-level Question Structure: 6.92 points and 4.56 points: STAIR’s F1 gains on time-anchored before and after questions, respectively, demonstrate improved order-sensitive selection.The diagnostic reports the before gain as substantially higher than the after gain.
  • Error Analysis: Residual errors arise from noncanonical contexts, ambiguous interval boundaries, invalid adapter outputs, and operators outside STAIR’s predefined policy inventory.Unsupported phenomena include implicit ordering, duration comparisons, negation, nested constraints, and multi-hop temporal composition.
  • Error Analysis: Fallback expands query coverage but sacrifices procedural interpretability because predictions lack the deterministic TAS module’s guard-level execution traces.This limitation follows from STAIR’s deterministic design scope and its reliance on fallback predictions when broader coverage is needed.

Conclusion … D CronQuestions Conversion and Evaluation Protocol

STAIR separates semantic interpretation from deterministic temporal execution: rules handle regular questions, while an answer-free adapter produces validated intents for difficult formulations. Its formal automaton, conservative policies, normalization rules, schema checks, and CronQuestions conversion protocol keep temporal evidence selection executable and auditable.

  • Conclusion: STAIR resolves regular questions with a deterministic temporal automaton and uses an answer-free semantic adapter only for difficult formulations, which return validated intents rather than answers.Both branches execute over canonicalized evidence, while consistent improvements are reported across four TQA benchmarks and an operator-supported CronQuestions subset.
  • A Formal Definition of the Temporal Automaton: The formal TAS contract defines a guarded finite-state controller over canonical temporal facts, validated intents, local fact chains, selected evidence, and an answer buffer.Its successful path proceeds from raw input through canonicalization, intent validation, policy binding, evidence selection, and answer emission; failed guards record typed failure.
  • A.2 Intent Space: The intent space represents interval, point, before-anchor, after-anchor, first, and last queries with typed temporal arguments and deterministic policies.Validation rejects unsupported operators, ill-typed arguments, invalid dates, unresolved target keys, and answer-like adapter outputs.
  • B.2 Policy Variants and Defaults: TAS applies finite policies for exact and overlapping intervals, point containment, before/after selection, and earliest/latest facts within a local subject–relation chain.Ambiguous or empty selections fail the emission guard rather than allowing an LLM to resolve temporal evidence implicitly.
  • B.3 Time Normalization and Closed-Interval Comparison: Temporal comparison uses normalized sortable keys and closed endpoint intervals, with unparseable boundaries filtered or routed to typed failure.Bare years, month-year expressions, and fully specified dates receive progressively more specific normalized representations.
  • B.4 Pre-Transform and Fallback Transform Corrections: For hard temporal structures, pre-transform and fallback modes may rewrite structure or policy into validated intents, but final temporal selection remains a TAS operation.Examples include overlap policies for month-year intervals, latest-start handling for multiple point matches, and time-valued date anchors.
  • C.1 Answer-Free Adapter Contract: The semantic adapter emits answer-free JSON for hard questions, limited to structure normalization, intent generation, policy selection, and schema-compatible fact repair.Answer-like fields are rejected, and the adapter does not instantiate selected evidence or the answer buffer.
  • C.2 JSON Schema: Schema validation requires type-specific temporal fields, parseable time values, resolvable target keys, and canonical facts before execution returns the intent to TAS.This boundary permits schema repair while preserving symbolic temporal comparison, evidence selection, and answer aggregation.

D.1 Source Data and Temporal KG … D.5 Answer Construction

The appendix defines the CronQuestions temporal-KG data, deterministic filtering, timeline construction, specialized inverse-query handling, and answer-label construction. It also documents missing-head exclusions and the limitation that general inverse conversion is not implemented.

  • D.1 Source Data and Temporal KG: The converter uses the official CronQuestions test split and associated Wikidata temporal-KG files to build evaluation contexts.The test split contains 30,000 questions, while the KG includes 328,635 temporal facts, 125,726 entity labels, and 203 relation labels.
  • D.1 Source Data and Temporal KG: Each temporal fact is represented as (s, r, o, [ts, te]) and converted from QIDs and PIDs into natural-language labels.The converted facts are written into the TAS context.
  • D.2 Supported Question Types and Filtering: Under current tas mode, supported items require an entity answer type and a simple entity, first last, or before after question type.The resulting deterministic filtering stages are reported in Table 7.
  • D.2 Supported Question Types and Filtering: Missing head identifies type-supported items without annotation.head, preventing forward temporal-context construction even when a tail is available.This category is not a post-hoc filter for answer availability, missing KG facts, missing labels, or manual data cleaning.
  • D.3 Temporal Context Construction: For standard forward items, the converter retrieves all (head, relation, o, ts, te) facts, maps IDs to labels, sorts them by time, and groups duplicates.It formats the resulting temporal context as timeline lines of the form start - end : subject's relation is ( object ).
  • D.4 Inverse and Before/After Queries: Holder-style P39 before/after questions retrieve facts by (relation, tail), reverse them, and use adjacency-based policies to return the person entity.The question text uses paraphrases[0] when available.
  • D.4 Inverse and Before/After Queries: A general inverse construction for all simple entity and first last questions is not implemented, leaving some inverse items in the missing head bucket.This limitation affects inverse-item conversion rather than the specialized holder-timeline construction.
  • D.5 Answer Construction: Gold Wikidata answers are mapped to labels, and samples remain written even when expected answer IDs cannot be inferred from matched context facts.The converted subset is not selected by checking whether the gold answer appears in the constructed context.

D.6 Shared Evaluation Protocol · D.7 Scope and Limitations

The evaluation compares NeSTR and STAIR under an identical materialized JSONL protocol, with converter-produced filtering counts and no manual annotation. The evaluation is limited to the operator-supported, successfully converted CronQuestions subset rather than the complete test set.

  • D.6 Shared Evaluation Protocol: NeSTR and STAIR use identical JSONL inputs, question IDs, questions, answers, contexts, sample order, prompts, and metrics.This prevents method-specific conversion or sample-ordering differences from affecting the comparison.
  • D.6 Shared Evaluation Protocol: Converter-produced retained and skipped counts are used without manual annotation.The counts reported in Table 7 come directly from the converter.
  • D.6 Shared Evaluation Protocol: Table 7 reports deterministic filtering stages for the CronQuestions test split.
  • D.6 Shared Evaluation Protocol: Table 8 breaks down type-supported questions skipped when annotation.head is absent.
  • D.7 Scope and Limitations: The evaluation covers only the operator-supported and successfully converted CronQuestions subset.It is explicitly not a complete CronQuestions test-set evaluation.
  • D.7 Scope and Limitations: Time-answer questions, time join, unsupported operators, and general inverse timelines remain outside the current scope.

E Additional Main Result Details

Table 9 presents reproduced baseline results alongside three-run STAIR statistics in the main results layout, including NeSTR reproduction as a pipeline consistency check.

  • Additional Main Result Details: Table 9 reorganizes reproduced baseline results and three-run STAIR statistics into the main result table’s layout, with NeSTR reproduced under the pipeline as a consistency check.The passage identifies NeSTR (repr.) rows as the reproduced baseline entries.

F Full Experimental Configuration … H.2 Hard Non-Exact Interval Match

The experiments use shared runtime settings and auditable prompt procedures, while case studies show deterministic exact-interval matching and semantic adaptation for month-level queries over year-level evidence.

  • F.1 Shared Settings: Experiments use a shared runtime configuration documented in Table 10.The supplied passage identifies Table 10 as recording the shared experimental runtime configuration.
  • G Prompt Templates: The prompt-only baseline uses one NeSTR-style prompt for symbolic representation, inference, consistency checking, optional reflection, and answer emission.The same template is filled with the question and temporal context across datasets.
  • G Prompt Templates: STAIR invokes the same prompt family only after TAS and validated semantic repair fail, using four auditable fallback calls.The calls cover symbolic representation, constrained inference, consistency checking, and final reflection or answer emission.
  • G Prompt Templates: All prompts preserve entity spelling, accents, and encoding, while adapter prompts remain answer-free.Only the residual fallback prompt may emit a final answer.
  • H Case Study: The case study presents TAS traces to illustrate its behavior and one boundary limitation rather than new quantitative analysis.The examples focus on trace format and boundary handling.
  • H.1 Regular Interval Match: For the 1985–1989 query, TAS canonicalizes facts, parses an exact-interval intent, selects matching boundaries, and emits the answer without LLM temporal inference.The selected answer is Port F.C Thailand national football team.
  • H.2 Hard Non-Exact Interval Match: For a month-level query over year-level context, the semantic adapter rewrites the interface as an overlap interval, and TAS selects the covering fact.The normalized interval is 1997-07-01 to 1997-09-30, yielding Conservatives.

H.3 Year-Boundary Ambiguity

STAIR’s deterministic selector fails on month-level queries when evidence uses year-level boundaries, producing a boundary-sensitive error rather than an arbitrary LLM-generation error. The case motivates retaining adjacent candidates under shared year boundaries and resolving them with finer-grained rules or validated semantic normalization.

  • The selector returns Great Northern Railway, whereas the gold answer is Westinghouse Electric Corporation.
  • A granularity mismatch causes TAS to map Feb 1908 to the preceding year-bounded interval and exclude the adjacent 1908–1912 interval.The facts provide year-level boundaries, while the question asks for a month-level point.
  • The same boundary failure is observed across multiple model backbones.The supplied results include a GPT-4o-mini TimeQA-Hard TAS run.
  • A boundary-uncertain state could retain both adjacent candidates when a month-level query falls inside a shared year boundary.A finer-grained rule or validated semantic normalizer could then resolve the ambiguity.

I Executor Ablation Details

Executor ablations isolate STAIR-Core’s deterministic design choices, showing that deterministic answer construction and before/after adjacency drive its gains. Fact-source changes matter less once an admissible local chain is available.

  • Ablation setup: STAIR-Core ablations exclude the hard detector and semantic adapter, isolating design choices within the deterministic executor.The analysis is executor-local rather than a second main result table.
  • Deterministic answer construction: 4.47 F1 points for GPT-4o-mini and 12.54 F1 points for Qwen2.5-7B are lost when deterministic answer construction is removed.The system may still identify temporal evidence but must then use a generative step to verbalize the answer, introducing avoidable aggregation errors.
  • Before/after adjacency: Removing before/after adjacency also causes a substantial decline by eliminating the ordered-chain constraint for predecessor and successor questions.Without this constraint, entity-anchored questions can select any temporally compatible fact rather than the relevant predecessor or successor.
  • Fact source: Changing the fact source has a smaller effect, indicating that deterministic policy matters most once an admissible local chain is available.The decisive factor is the deterministic policy applied to that chain.
Loading 2608.16224v1…