Source-linked AI summary
Time as Structure: Temporal Dependency Graphs for Verifiable Deadline Computation over Legal Documents
Maryia Zhyrko, Lifeng Han, Suzan Verberne
TL;DR
Legal deadline computation depends on chains of dated facts and statutory counting rules, raising the question of whether language models should reason directly or delegate arithmetic to code. The paper represents those dependencies explicitly and computes deadlines with a deterministic, calendar-correct engine. Across five settings, the approach is exact on generated reasoning items, reproduces six of seven tribunal verdicts, and reaches 90.2% on 427 perturbed items when it binds an anchor.
Problem
The paper asks whether explicit temporal structure improves legal deadline reasoning and whether extraction from legal prose can recover enough dependencies to support computation.
Method
The approach extracts dated facts and typed dependencies into Temporal Dependency Graphs, then applies deterministic calendar arithmetic, statutory conventions, and conciliation pauses.
Results
Across five settings, the method is exact on generated reasoning items, reproduces six of seven tribunal verdicts, and reaches 90.2% on 427 perturbed items when it binds an anchor.
Takeaways & Limitations
The remaining problem is anchor selection, while self-consistency does not indicate correctness.
Takeaways & Limitations
The evidence covers one jurisdiction and procedural domain, while the controlled benchmark is synthetic and published judgments are not a random sample.
Abstract
from arXiv · showhide
Miss a filing deadline by one day and the claim is barred, however strong the case. Computing that deadline is rarely simple: the period runs from a triggering event, is counted by a statutory convention, and may be suspended by a mandatory conciliation window. We ask whether a language model should answer such questions directly, or read the document and leave the arithmetic to code. We extract dated facts and their dependencies into a temporal dependency graph and compute deadlines from it with a calendar-correct engine. On UK Employment Appeal Tribunal judgments the engine reproduces six of seven timeliness rulings, and matches the judges' own dates to the day. The strongest of four language models, asked the same cases, gets the arithmetic right and the answer wrong: in six of twenty-one responses its stated verdict contradicts its own thinking, and every contradiction runs the same way, calling a late claim timely. To test the systems at scale we move the dismissal date across the statutory boundary, generating 427 cases whose answers are computed rather than annotated. On the cases both systems answer, the pipeline is right 90.2% of the time against 61.2% for direct answering. The limit is extraction: on contracts the errors are almost never in the arithmetic, but in choosing which event the period starts from.
1 Introduction
The paper separates extraction from calendar-correct deadline computation by representing dependent dates in Temporal Dependency Graphs. Across tribunal cases and counterfactual evaluation, the engine outperforms direct answering, while extraction—especially anchor selection—remains the main bottleneck.
- Approach: Temporal Dependency Graphs encode dated facts and typed dependencies for deterministic application of calendar arithmetic, statutory counting, and ACAS conciliation pauses.The engine returns the selected anchor, applied rule, intermediate dates, and confidence; it abstains when the statutory anchor cannot be determined.
- Limitations: An exhaustive per-edge audit located the bottleneck in anchor selection rather than arithmetic.The paper therefore tests whether current extraction recovers enough structure from legal prose to support computation.
- Results: Six of seven tribunal timeliness rulings and all three judge-stated dates were reproduced by the deterministic engine.The statutory rules were recovered from statute text.
- Results: Six of twenty-one GPT-family responses contradicted their own working, with all six reporting a late claim as timely.This working-level audit identified a failure that value-level metrics did not detect.
- Evaluation: 427 counterfactual items were generated by perturbing anchor dates across the statutory boundary and recomputing ground truth through the engine.The labels could not be recovered by memorisation from text alone.
2 Related Work
The work extends temporal information extraction from isolated date normalization and pairwise ordering to dependency-aware anchor identification. It also connects legal formalisation and statutory reasoning with mechanically computed, statute-guided perturbation evaluation.
- Temporal information extraction: Temporal information extraction typically normalises dates, durations, and intervals, but this work additionally represents which event is derived from which and identifies anchors.HeidelTime normalises expressions largely in isolation, while LexTime finds that legal event ordering is reasonable but nested legal language remains difficult.
- Legal formalisation: Legal formalisation includes Catala’s statute-mirroring programming language and mechanised legal date arithmetic that detects counting ambiguities.Recent work also studies translating law into Catala-like representations and documents faithfulness failures in that route.
- Statutory reasoning and evaluation: The study builds on established statutory reasoning, court-outcome prediction, and statute-guided numerical computation, adapting GSM-Symbolic perturbations with statute-computed oracle answers.The perturbation design replaces templated answers with an oracle computed from the statute.
3 Data and Task
The task combines contract extraction over 41 unique documents with deadline evaluation on six post-cutoff EAT judgments, yielding seven statute-case rows. Gold annotations link dated facts and rulings to verbatim, byte-checked evidence, while leakage controls remove explicit deadline information from direct evaluation.
- Contracts: 45 sampled contract documents yielded 41 unique documents, the set used for all extraction figures.The sample came from the en_contracts subset of Multi_Legal_Pile.
- Statutes and cases: Six 2025–2026 EAT judgments produce seven statute-case rows because one judgment is evaluated under both statutes.The statutes are point-in-time versions of the Employment Rights Act 1996 s. 111, Equality Act 2010 s. 123, and corresponding ACAS provisions; all judgments postdate every model’s training cutoff.
- Gold annotation: Each gold row records statutory anchors, presentation and ACAS dates, verdicts, and any judge-stated deadline or boundary date, with every field verbatim-quoted and byte-checked by script.Two of seven rows have judge-stated deadlines, and one has a judge-stated boundary date.
- Leakage control: Direct evaluation removes sentences containing deadlines, day counts, or timeliness conclusions, but appeal direction can still leak the first-instance verdict.Because that leak remains unavoidable, deadline exactness and working-level consistency are treated as more reliable than verdict accuracy alone.
4 Method
The method separates temporal extraction from deadline computation by representing case facts and statutory rules as temporal dependency graphs. A deterministic engine matches statutory anchors to case facts, applies calendar and conciliation rules, and abstains when the graph is unresolved.
- The architecture separates extraction and computation, allowing failures to be attributed to the layer that produced them.
- Temporal dependency graph nodes store entities, semantic roles, and ISO 8601-normalised dates, while typed edges encode offsets or ordering.Only additive edges with resolved source dates support arithmetic.
- The rule-based pipeline extracts temporal expressions, classifies their roles from dependency structure, links dates to entities, and builds the graph.A per-document creation time resolves 43 of 45 incomplete date expressions.
- The deterministic engine binds statutory anchor concepts to case facts and computes deadlines using real month lengths, leap years, statutory counting, and conciliation pauses.For the UK convention, a three-month period is implemented as anchor + 3 months −1 day, with explicit month-end rounding; ACAS suspension and a one-month floor are also implemented.
- When no candidate or conflicting candidates are found, the engine returns INDETERMINATE with a machine-readable cause rather than producing an unresolved deadline.
5 Experiments and Results
The calendar-correct engine is exact when given verified facts, while direct answering suffers from arithmetic, self-reporting, and cascade errors. At scale, the pipeline substantially outperforms direct models when anchors bind, but extraction and semantic binding limit coverage and accuracy.
- Controlled benchmark: 210 generated items separated reasoning from extraction, with calendar-computed ground truth and exact structured-method performance in every category.The benchmark included seven categories of 30 items, including deadline, cascade, and correction-propagation tasks.
- Controlled benchmark: 97%, 96%, and 93% were Gemma’s per-link accuracies across three cascade steps, versus Llama’s decline from 11% at the first link to 3% at the third.The calendar-naive baseline reached 41% overall, exceeding Llama’s 20%.
- Verified legal cases: Six of seven tribunal verdicts, both judge-stated deadlines, and the continuing-act boundary date were reproduced from human-verified facts.The remaining row was document-bound because it omitted the conciliation dates needed to derive its in-time conclusion.
- Verified legal cases: Six of 21 GPT-family responses contradicted their own working, all by reporting in time when the working concluded out of time.The strongest model applied the minus-one-day convention correctly in all seven responses but produced only three of six computable deadlines exactly; the emitted verdict field was unusable without the working.
- Extraction and binding: 14 of 28 cells were answered and 14 abstained, with 12 of 14 answers correct and no fabricated deadlines.Abstentions carried confidence 0.00 and named causes; extraction recall was high, but semantic binding failed to align statutory concepts with extracted labels.
- Generated boundary benchmark: 90.2% versus 61.2% was the pipeline’s accuracy against direct answering on 286 items both Gemma conditions answered.The pipeline reached 98.3% when the extracted anchor equaled the true shifted anchor, but achieved 0.604 overall versus Gemma direct’s 0.646 because extraction limited coverage.
6 Analysis
The structured route is exact by construction for dependent-date arithmetic, while direct accuracy declines as event chains lengthen and the arithmetic becomes harder. Its main limitation is extraction and anchor selection, and it loses when event ordering offers no arithmetic to verify.
- Structure and arithmetic: Explicit dependency graphs with executed computation remain exact by construction, while direct accuracy falls as dependent-event chains lengthen.The effect is largest in cascade categories and perturbed items near the statutory boundary.
- Working-level auditing: Comparing working calculations with emitted fields detects divergences that verdict-level and value-level scoring can miss.Verdict-level scoring passes a wrong computation in four of twelve correct pipeline verdicts, while value-level scoring passes a right number produced by canceling errors once.
- Consistency and correctness: Removing self-contradiction reduced verdict accuracy in both affected models, so self-consistency is not a usable confidence signal.The deterministic layer instead locates whether an error is attributable to an anchor, rule, or input.
- Anchor selection: Competing plausible anchors make extraction the central limitation, especially when selecting among continuing acts, rejected contentions, and adjacent procedural events.A conflict gate presents candidate anchors and their computations rather than selecting silently; better extraction cannot resolve anchors depending on legal conclusions.
- Boundary of the claim: 18% coverage causes the structured route to lose outright on pairwise event ordering, because most event pairs lack extracted relations and ordering has no arithmetic to verify.The public benchmark contains 190 pairs asked in both directions, with exactly one direction able to be true.
7 Conclusion
The paper evaluates a structure-then-compute approach for legal temporal reasoning across five settings, finding exact performance on generated items and strong results on tribunal judgments and perturbed cases. The remaining challenge is binding statutory anchor concepts to the correct factual events with traceable provenance.
- The structure-then-compute method is exact on generated reasoning items, while direct accuracy falls with chain length.
- 6 of 7 tribunal verdicts and all 3 judge-stated dates are reproduced using rules recovered from statute text.
- 90.2% accuracy is achieved on 427 perturbed items when the system binds an anchor, while unbound cases receive abstentions with named causes.
- The open problem is binding statutory anchor concepts to the correct factual events with traceable provenance, rather than extracting more dates.When binding depends on a legal conclusion, the expected behavior is to present candidate anchors and their consequences.
Artifact availability
The research artifacts are released as an installable Apache-2.0 command-line tool with an inspection interface, enabling recomputation and inspection of each reported deadline.
- Artifact availability: The released Timebar tool includes the engine, graph representation, statutory rule packs, and evaluation harnesses under Apache-2.0.It provides an inspection interface and operates from case files and rule packs.
- Artifact availability: Each deadline computation prints its anchor, rule, arithmetic, and confidence, making reported deadlines recomputable from the underlying case file and rule pack.The tool is distributed as an installable command-line application.
Limitations
The evaluation is limited by a seven-row legal gold set, synthetic controlled data, and restricted jurisdictional and procedural coverage. Further limitations concern unvalidated matching, incomplete and variable extraction, single-assessor audits, non-random coverage, and an ineligible llama comparison.
- Evaluation scope: 7 legal gold-set rows support failure-mechanism taxonomy but not error rates, while the 427-case counterfactual sweep retains seven cases as its statistical unit.The statutes cover one jurisdiction and one procedural domain; the controlled benchmark is synthetic and makes extraction trivial.
- Validation: The engine, counting conventions, and rule specifications were fixed before evaluation, but matcher validation beyond the seven studied cases remains required.This limits confidence that the matcher generalizes beyond the tribunal evaluation.
- Extraction and audit: Extraction evaluation reports precision on produced edges but no recall, relies on unannotated contracts, and detects only explicit temporal expressions.Audit verdicts come from one assessor under a prespecified scoring rule and remain under independent review; LLM extraction varies across temperature-0 runs, so counts use one named run.
- Coverage and comparison: All parse failures fall on three cases, including two with difficult anchors, so answered-item accuracy uses an easier subset; llama is excluded from the single-prompt comparison.The llama conditions differ by a lost case and a changed generation task as well as by shared context.
Ethical considerations … C The released tool
The released system represents legal timing as provenance-bearing temporal dependency graphs, applies statute-specific rule packs with calendar-correct computation, and deliberately abstains when selecting the anchor requires legal judgment. It computes deadlines offline without giving legal advice, while exposing source support, uncertainty, and derived results for checking.
- Ethical considerations: The tool computes statutory deadlines rather than giving legal advice, cites source documents for checking, and presents anchor candidates when choosing among them requires legal judgment.Its abstention behavior is deliberate, and the documents are public judgments whose party names are not reprinted.
- A Engine mechanics on one computation: The engine pauses the primary clock between Day A and Day B, then applies the statutory one-month floor when it gives more time than the pause.The pause applies only when Day A falls inside the primary period, and the engine reports which branch governed.
- A Engine mechanics on one computation: 30 days of pause moves a 10 October primary deadline to 9 November when Day A is 1 September and Day B is 1 October.The engine reports the bound anchor, rule pack and provision, primary deadline, pause length, governing branch, and effective deadline.
- A Engine mechanics on one computation: The counting convention can shift a three-month deadline by one day, so the engine records the statute’s prescribed convention in its rule pack.Three months beginning with 8 June ends on 7 September when the anchor day counts, but on 8 September when counting starts the next day.
- B Graph schema and rule packs: Each temporal dependency graph is JSON containing document metadata, facts, and dependencies with normalized values, roles, provenance offsets, confidence, and constraints.Dependencies can be additive, ordering, or interval constraints; additive edges include delta_days offsets.
- B Graph schema and rule packs: A statute is encoded as a graph-based rule pack, allowing a new jurisdiction to be added by supplying a file rather than changing engine code.The ERA 1996 s. 111 pack records the anchor concept, P3M duration, counting convention, and human-readable authority.
- C The released tool: The released command-line and browser tools build document graphs and merged timelines, then recompute deadlines offline without calling a model.Merged timelines preserve document sources, supplied values, extractor confidence, provenance status, and whether values are agreed, disputed, single-source, or derived.
D Constraint-level consistency results · E Formal cross-check with Catala · F Contradiction detection across documents
The consistency analysis localizes gpt-5.4’s incoherence to dates and arithmetic rather than verdicts against its own day counts. Catala provides a complementary formal cross-check with limited field overlap, while contradiction detection is accurate on agreement cases but embedding similarity greatly expands candidate links without improving structural matches.
- D Constraint-level consistency results: 0 of 7 gpt-5.4 cases violate K3 or K5, while all four incoherent cases violate K1 or K2.The verdict never disagrees with the model’s own day count; inconsistencies arise between dates, deadlines, and subtraction.
- E Formal cross-check with Catala: 40 of 45 sampled contracts reach Catala comparison with generated programs that compile and execute.Two programs fail the repair loop and three fail at interpretation.
- E Formal cross-check with Catala: 116 fields appear only in the graph and 23 only in Catala, with only 19 documents yielding a field described by both routes.The small overlap indicates complementary rather than redundant formalisations.
- E Formal cross-check with Catala: 14 agreements, 6 value mismatches, and 5 type mismatches occur over the shared subset.One value mismatch caught the TDG route misreading a duration because of source-document scanning errors.
- F Contradiction detection across documents: 0 contradictions are returned on all three genuinely agreeing document pairs, which also reveal three parallel applications of the same P3M period.The pairs comprise two tribunal judgments applying the same s. 111 rule and each judgment paired with its cited statute.
- F Contradiction detection across documents: 9 of 12 probe cases are scored by the embedder versus 8 of 12 by the lexical setting.The embedder resolves the termination–dismissal synonymy case, with similarity 0.71 versus lexical similarity 0.00; every lexical success is also an embedding success.
- F Contradiction detection across documents: 1183 candidate links are returned lexically versus 11674 with embeddings across 45 contract graphs containing 325 facts.The embedding configuration returns 9587 coreference, 1920 contradiction, and 167 structural-analogy links, compared with 784, 232, and 167 respectively lexically.
G External benchmarks
External benchmarks show that structure does not uniformly improve performance: it leaves numeric tasks near zero, underperforms direct answering on event ordering, but eliminates inconsistent bidirectional affirmatives. The ordering failure arises because extracted relations rarely connect the queried event pairs.
- Statutory computation without structure: Exact match is at or near zero for both local models on SARA and DeonticBench’s airline-fee, housing, and USCIS numeric tasks.On classification tasks, both models remain close to the majority class; the two above-majority cells lead by 6.7 and 3.6 points on 30 and 28 items.
- LexTime: 73.9% and 62.3% direct accuracy beat the 50.4% majority class on LexTime, whereas the TDG route reaches 13.0% and 21.0% over all items.TDG coverage collapses to 18% and 28% because most event pairs lack an extracted connecting relation.
- Consistency: 0% of structured-route answers affirm both directions on 190 public benchmark pairs, compared with 32% and 52% for the direct models.Each pair has exactly one direction that can be true; a single extracted relation projects to both structured answers.