Source-linked AI summary
TEMPS: Temporal Sentence Embeddings for Temporal Information Retrieval
Mourad Hassani, Julien Romero, Amel Bouzeghoub, Christian Jacquelinet
TL;DR
Time-sensitive retrieval needs to distinguish topical relevance from temporal alignment, but common semantic representations and similarity resources do not adequately model temporal constraints. The paper introduces TTS and TEMPS, which use grounded interval supervision in a temporal branch alongside a frozen semantic retriever; TEMPS improves MRR for every tested semantic backbone.
Problem
Method
TEMPS grounds anchored temporal language to intervals, moment-matches intervals to Gaussian targets, and trains an anchor-date-conditioned temporal branch to fuse with a frozen semantic retriever.
Source: §1 · p. 2, §5.1 · p. 4Results
TEMPS improves MRR over existing baselines for every semantic backbone tested; a rule-based grounding control falls below the semantic baseline on TimeQA and TS-Retriever but is stronger on TempReason.
Source: §7 · p. 9Takeaways & Limitations
The reported gains are concentrated where topical and temporal relevance conflict, while the semantic branch carries passages without temporal expressions.
Source: §5.2 · p. 5, Limitations · p. 10Evaluation is English-only and limited to anchored, resolvable expressions, excluding implicit or highly ambiguous references and event–event relations.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
Modern information retrieval (IR) systems rarely represent time, yet many information needs depend on it: in clinical, journalistic, and legal search, when an event occurred can decide whether a document is relevant. Dense retrievers and Retrieval-Augmented Generation (RAG) pipelines match queries to documents well on topic but poorly on time, so they surface content that is on-topic yet temporally wrong. We introduce Temporal Textual Similarity (TTS), a task that measures how well two anchored texts align in time, independent of their topical similarity. We then present TEMPS (Temporal Embedding Model for Precise Search), a modular temporal branch that attaches to a frozen semantic retriever and trains on that signal. It resolves anchored temporal expressions to intervals and moment-matches each one to a Gaussian; the resulting ordering supervises an anchor-date-conditioned encoder, whose score we fuse with the semantic score at inference. Grounding supplies the supervision, so training uses no hand-labeled temporal data. The temporal score itself is the Gaussian-KL inclusion measure from distributional embeddings; what TEMPS adds is the grounding and the moment-matched supervision. On three temporal benchmarks, TEMPS improves MRR for every semantic backbone tested and, on TS- Retriever, lifts R@1 from 19.92 to 25.39 over the prior temporal state of the art.
1 Introduction
Topical matching alone can retrieve passages that conflict with a requested time, while temporal compatibility can be asymmetric. The paper proposes TTS and TEMPS to model temporal alignment and add it to retrieval without retraining the semantic backbone.
1 Introduction
- Time-insensitive retrieval can return topically relevant but temporally mismatched documents, risking contextually misleading answers.The problem is relevant across clinical, journalistic, and legal search.Source: §1 · pp. 1–2
- Directional temporal compatibility distinguishes a specific day contained in a year from a year considered compatible with that day.The paper motivates distributional representations because symmetric similarity or interval overlap cannot express this distinction.Source: §1 · p. 2
- TEMPS adds a temporal branch to frozen semantic retrieval, grounding anchored expressions to intervals and using moment-matched Gaussian targets for supervision.It reuses asymmetric Gaussian-KL scoring rather than introducing a new divergence.Source: §1 · pp. 2–3
- Temporal Textual Similarity (TTS) measures temporal compatibility between anchored texts independently of their topical similarity.It is proposed as a temporal counterpart to semantic textual similarity.Source: §1 · p. 2
- Grounded temporal language supplies training supervision without manual labels, producing 60M training tuples.The introduction also describes a rule-based control that swaps temporal grounding into TEMPS to test where the learned encoder helps.Source: §1 · pp. 2–3
2 Preliminaries
The preliminaries describe temporal expressions by grounding type and define the interval-resolvable temporal language that TEMPS covers. Its scope excludes discourse-level reasoning and depends on the temporal tagger’s detections.
2 Preliminaries
- Temporal expressions include absolute, deictic, anaphoric, and duration forms, distinguished by how they are grounded.Examples include explicit timestamps, speaker-relative expressions, references to prior material, and defined spans.Source: §2 · p. 3
- TimeML annotations represent dates, offsets, references, and intervals in the synthetic dataset.Examples include 2024-09 for September 2024 and PRESENT_REF for now.Source: §2 · p. 3
- TEMPS covers expressions that can be anchored and resolved to intervals, but not unobserved antecedents, unanchored vague ordering, or event–event relations.The pipeline also inherits misses from the temporal tagger.Source: §2 · p. 3
3 Related Work
Related work spans temporal retrieval, semantic similarity, and Gaussian distributional representations, but existing resources and similarity functions leave important temporal-alignment gaps. TEMPS positions its contribution as grounded temporal supervision and a modular retrieval component.
3 Related Work
- Existing STS resources target semantic relatedness rather than temporal alignment, so high STS scores do not ensure compatible temporal information.Cosine similarity also cannot naturally represent asymmetric inclusion, such as a specific date contained in a broader interval.Source: §3 · pp. 3–4
- Gaussian embeddings use means and covariances for location and uncertainty or generality, while prior work applies asymmetric KL to lexical and knowledge-graph relations.GaussCSE uses a bounded inverse-KL score for entailment-oriented sentence similarity.Source: §3 · pp. 3–4
- TEMPS’s novelty is converting grounded intervals into Gaussian supervision by moment matching and training an anchor-date-conditioned branch to reproduce temporal compatibility.It follows prior Gaussian-divergence work rather than introducing a new divergence or similarity head.Source: §3 · p. 4
- Prior temporal retrievers use temporal supervision but remain tied to specific retriever-training setups; TEMPS instead offers a modular temporal component for different semantic retrievers.TS-Retriever and TSContriever train Contriever with negatives formed by perturbing question time specifications while holding passages fixed.Source: §3 · p. 4
4 Problem Definition
TTS measures the temporal alignment of anchored texts independently of semantic or topical similarity. The task uses anchor dates to interpret expressions and applies to passage-sized text at retrieval time.
4 Problem Definition
- TTS measures alignment between temporal information expressed in sentences while disregarding semantic or topical similarity.The task is proposed as a temporal analogue of STS.Source: §4 · p. 4
- TTS compares two texts together with their anchor dates, which provide the context for interpreting their temporal expressions.Its intended output is the degree of temporal similarity, independent of semantic or topical similarity.Source: §4 · p. 4
- Retrieval uses passages of up to 512 tokens rather than single sentences, and a passage with multiple or no temporal expressions is reduced to one Gaussian or handled as lacking an interval.The task is intended as a basis for retrieval systems sensitive to temporal context.Source: §4 · p. 4
5 Methodology
TEMPS combines grounded temporal labels with a lightweight temporal embedding branch that complements semantic retrieval. Its Gaussian-based supervision and inference score share a directional similarity form.
5 Methodology
- TEMPS pairs weakly supervised temporal labels for anchored expressions with a lightweight temporal embedding module that can combine with arbitrary dense retrievers.The labels and inference-time score use the same Gaussian-KL form, making training teach the module to reproduce a closed-form temporal target.Source: §5 · p. 4
5.1 Temporal Supervision
- Grounded intervals become moment-matched Gaussians: their calendar-space means mark temporal location and variances encode granularity, while CoSENT trains latent-space similarities to match their directional ordering.The calendar and learned latent spaces need not coincide; only their rankings must align.Source: §5.1 · pp. 4–5
- Training tuples pair anchored temporal texts with dates and similarity values, drawn from synthetic expressions and English temporal-tagging data for roughly 60M tuples.Expressions span years 1000–2030.Source: §5.1 · pp. 4–5
5.2 Temporal Model
- The temporal branch combines a passage encoding from frozen all-MiniLM-L6-v2 with a sinusoidal anchor-date representation to produce a Gaussian embedding.Its mean captures temporal location, and its diagonal variance captures granularity and uncertainty.Source: §5.2 · p. 5
- At inference, the passage is the unit: multiple dates are pooled into one Gaussian, while passages without temporal expressions have uncalibrated temporal scores bounded by α.Such passages are absent from temporal training, so the semantic branch carries them.Source: §5.2 · p. 5
- TEMPS fuses per-query normalized semantic and directional temporal scores, with α controlling the temporal contribution, and its temporal branch can be added without retraining an external semantic encoder.The semantic branch can be replaced by any external embedding model.Source: §5.2 · p. 6
6 Experiments
The experiments test TEMPS across three temporal retrieval benchmarks and assess its effects on retrieval, scaling, rule-based controls, and downstream RAG. Results show broad MRR gains, with benchmark-specific trade-offs and a strong TS-Retriever comparison.
6.1 Experimental Setup
- Evaluation covers TimeQA, TempReason, and TS-Retriever against lexical, rule-based, dense, LLM-based, and temporal baselines, with α selected per benchmark on validation MRR.The study also evaluates TimeQA RAG with a top-five retrieval stage; its +SUTime control holds fusion, normalization, and α fixed.Source: §6.1 · p. 6
6.2 Results
- TEMPS improves MRR for every tested backbone on every dataset; on TempReason, however, the E5base-v2 gain is only 1.1% and is not significant.MPNetbase-v2 gains 25.1% MRR on TempReason.Source: §6.2 · pp. 6–7, Table 1 · p. 7
- On TimeQA, Mistral + TEMPS has the highest MRR, while E5base-v2 + TEMPS leads NDCG@5, Recall@5, and Precision@5.The reported gains target finer distinctions among otherwise relevant passages.Source: §6.2 · p. 6, Table 1 · p. 7
- TempReason results are mixed: Recall@5 falls for all four backbones, and Mistral + SUTime leads every metric, including 0.6901 MRR and 0.9056 Recall@5.On explicit-year queries, exact-year and query-interval regex oracles each improve MRR by only 0.009 over the semantic baseline, while the temporal head alone reaches 0.8695.Source: §6.2 · pp. 7, 9, Table 1 · p. 7
- R@1 rises from 19.92 to 25.39 and P@1 from 58.63 to 69.14 for E5base-v2 + TEMPS versus TSContriever on TS-Retriever.R@100 reaches 98.95 versus 94.57, and P@20 is 17.83 versus 15.72.Source: §6.2 · pp. 7–8, Table 2 · p. 8
- Every +TEMPS retriever improves EM, F1, and containment in the TimeQA RAG evaluation; E5base-v2 + TEMPS has the strongest overall RAG performance.The evaluation uses the top five retrieved passages and Qwen2.5-7B-Instruct.Source: §6.2 · p. 8, Table 3 · p. 8
- Across three benchmarks, E5base-v2 + TEMPS has higher MRR than unaugmented 7B Mistral despite having around 50 times fewer parameters, though TempReason's margin is small.Mistral + TEMPS achieves the best MRR on all three datasets.Source: §6.2 · p. 8
- Replacing the learned temporal encoder with SUTime lowers MRR below the semantic baseline on TimeQA and TS-Retriever, but SUTime wins on TempReason.The control keeps fusion, normalization, and α fixed, isolating the temporal representation as the changing component.Source: §6.1 · p. 6, §6.2 · p. 9
7 Conclusion
TEMPS combines a temporal embedding model with semantic retrieval, using weak supervision and synthetic data. The conclusion reports MRR improvements across tested semantic backbones, while identifying limits in the scope of temporal references and evaluation.
7 Conclusion
- TEMPS uses weak supervision and synthetic data to train a dedicated temporal embedding model, and improves MRR over existing baselines for every tested semantic backbone.A SUTime-based control falls below the semantic baseline on TimeQA and TS-Retriever, but is stronger on TempReason, where TEMPS trades top-five coverage for better top-one ranking.Source: §7 · p. 9
- The reported benefit is concentrated where topical and temporal relevance conflict, rather than on queries a tagger already resolves.Source: §7 · p. 9
- Future work identifies implicit temporal references, event-based reasoning, broader data generation, multilingual retrieval, and lightweight modules as open directions.The authors also propose tighter semantic-temporal integration through joint training or attention-based fusion.Source: §7 · p. 9
Limitations
The evaluation and method have scope boundaries in language, data, and retrieval setting. Results also depend on one training run and a single end-to-end task.
Limitations
- Evaluation uses English benchmarks with explicit temporal annotations and reliable timestamps, leaving other languages and real-world metadata variability untested.Implicit or highly ambiguous references and event–event relations are outside the current supervision scope, which also inherits noise from the underlying tagger.Source: Limitations · p. 10
- Choosing the temporal–semantic weight by validation MRR accepts a coverage cost on TempReason.At α = 0.8, adding TEMPS lowers Recall@5 for all four backbones and NDCG@5 for two; SUTime outperforms it on every metric.Source: Limitations · p. 10
- Results come from a single training run, and downstream evidence is limited to TimeQA RAG with Qwen2.5-7B-Instruct.The authors report no second benchmark with both genuine temporal retrieval constraints and trustworthy answer-level ground truth under a comparable protocol.Source: Limitations · p. 10
- Passages without temporal expressions are encoded at inference but were excluded from temporal training, leaving their Gaussians uncalibrated.The semantic branch ranks these passages.Source: Limitations · p. 10
- TEMPS inherits biases and factual gaps from its semantic backbone, while its passage-level Gaussian can summarize temporally heterogeneous text lossily.The retrieval unit is a passage of up to 512 tokens, and attention pooling collapses its temporal expressions into one Gaussian.Source: Limitations · p. 10
- The added temporal encoding has small computational overhead, reported as far below scaling to much larger models for comparable gains.Source: Limitations · p. 10
Ethical Considerations
The authors caution that TEMPS inherits backbone biases and could surface misleading evidence in high-stakes search. Its training-data construction combines synthetic patterns with an adapted dataset to produce anchored-text tuples.
Ethical Considerations
- In clinical or legal search, temporally miscalibrated retrieval could surface misleading evidence, so downstream applications should validate temporal alignment.TEMPS inherits any biases or factual gaps of the semantic backbone it augments.Source: Ethical Considerations · p. 10
A.1 Training Dataset Construction
- The dataset-generation pipeline combines synthetic data from predefined patterns with an adapted existing dataset to produce tuples of anchored texts and temporal labels.The tuples contain two texts with temporal expressions, their daily-granularity anchor dates, and a temporal label between them.Source: §A.1 · p. 13
A.1.1 Temporal Similarity Function
The temporal similarity function compares grounded intervals through moment-matched Gaussian distributions and asymmetric query-to-document KL divergence. The appendix then describes data construction, temporal model details, and retrieval baselines.
A.1.1 Temporal Similarity Function
- TSF is an asymmetric query-to-document KL divergence over Gaussian representations of grounded intervals, with asymmetry distinguishing containment from its inverse.The function is non-degenerate on disjoint intervals, continuous and smooth at the overlap boundary, and handles differing granularities without special cases.Source: §A.1.1 · pp. 13–15
- Moment matching projects each interval’s uniform distribution onto a Gaussian, preserving temporal location in the mean and granularity-related uncertainty in the variance.Direct KL between interval uniforms is finite only under full containment and cannot distinguish different alignments of contained intervals.Source: §A.1.1 · pp. 13–14
- Table 4’s canonical pairs show TSF(day → year) = 5.4 and TSF(year → day) = 6.8 × 104, while far-disjoint intervals receive larger labels than near-disjoint ones.A one-day overlap scores 4.86 and a one-day gap scores 6.00, illustrating a smooth transition.Source: Table 4 · p. 15, §A.1.1 · p. 15
A.1.2 Data Sources
- The synthetic dataset spans 1000–2030, converting randomly generated TimeML annotations into natural language and pairing expressions with random daily anchor dates.Source: §A.1.2 · p. 15
Multilingual MLM Temporal Tagging Dataset
- The English portion of the Multilingual MLM Temporal Tagging Resources dataset supplies automatically annotated Wikipedia text as weakly supervised real-world language.It complements the synthetic data.Source: Multilingual MLM Temporal Tagging… · p. 15
A.1.3 Training Instance Generation
- Training-instance generation creates one positive and four negative samples per temporal expression; positives perturb expression boundaries while keeping bounded temporal similarity above 0.9.Random negatives use random anchor dates, while the positive retains the original or logically derived anchor; hard temporal negatives are left for future work.Source: §A.1.3 · p. 16
B Temporal Model Details
- The temporal branch combines sentence representations from a frozen all-MiniLM-L6-v2 backbone with an anchor-date encoding and predicts Gaussian parameters.Attention pooling emphasizes temporal tokens, and the model is trained with a CoSENT loss to enforce the ordering induced by temporal labels.Source: B Temporal Model Details · p. 16
C.1 Baselines
- Baselines span base-scale, large-scale, LLM-based, and temporal-specialist retrievers, with augmented dense models compared against their unaugmented counterparts.The +SUTime control holds downstream scoring and fusion fixed to isolate differences in temporal representation.Source: §C.1 · p. 17
C.2 Benchmarks and Metrics
The section documents the benchmark and evaluation protocol, then reports fusion-weight sensitivity, timestamp-oracle comparisons, computational resources, released artifacts, and statistical-significance patterns.
C.2 Benchmarks and Metrics
- Evaluation covers TimeQA, TempReason, and TS-Retriever, with α selected on a 20% validation split and fixed for the 80% test; retrieval and RAG use separate metrics.TempReason Level 3 is omitted because it focuses on event-event relations.Source: §C.2 · pp. 17–18, Table 6 · p. 18
C.3 Effect of Alpha Values
- The optimal α varies by benchmark: TS-Retriever peaks near 0.3 and falls sharply above it, while TempReason remains within 2% of its maximum across the sweep.The sensitivity curves show benchmark-specific preferences for the temporal-semantic interpolation weight.Source: §C.3 · p. 18, Figure 4 · p. 19
C.4 Explicit-Timestamp Oracle Study
- On 4,306 explicit-year queries, regex oracles raised MRR by 0.009 and Gold@1 by 0.013, while the temporal head reached 0.8695 MRR and the hybrid reached 0.8828.In 48% of the subset, the gold passage contained no year from the query interval; the interval oracle missed 668 of the 724 rank-1 lifts.Source: §C.4 · p. 18, Table 7 · p. 18
C.5 Computational Resources
- Training used one eight-GPU A100 node, and three temporal-module epochs over the full corpus took approximately four wall-clock hours.The node had two AMD EPYC 7543 CPUs and 512 GiB of DDR4 memory.Source: §C.5 · pp. 18–19
C.6 Artifacts, Licenses, and Intended Use
- The datasets and pretrained models are publicly released for research, and the authors release the code and TEMPS temporal module under the MIT license for temporal-retrieval research.The synthetic temporal data uses templates and publicly available Wikipedia-derived temporal-tagging resources.Source: §C.6 · p. 19
C.7 Statistical Significance
- Across 48 comparisons, 36 were significantly positive, 9 significantly negative, and 3 nonsignificant; TempReason had four positive, three nonsignificant, and nine negative comparisons.The four positive, three nonsignificant, and nine negative TempReason comparisons are reported across its benchmark metrics and backbones.Source: §C.7 · p. 20
D Qualitative Analysis: What Moves and Why
On held-out TimeQA queries, adding TEMPS moves gold passages into leading ranks more often than it displaces them. Illustrative cases show temporal-only retrieval recovering dated gold passages missed by topically strong hits.
D Qualitative Analysis: What Moves and Why
- The analysis compares semantic-only, temporal-only, and fused E5base-v2 rankings at α = 0.6 across 710 held-out TimeQA queries, each with one gold paragraph.A promotion means the gold passage enters rank 1 or the top 5 under +TEMPS after being outside that cutoff under the semantic baseline; demotion is the reverse.Source: D Qualitative Analysis: What Moves and… · p. 23
- Top-1 accuracy rose from 0.134 (95/710) to 0.380 (270/710), and Gold@5 rose from 0.603 to 0.723; TEMPS had 31 rank-1 and 49 top-5 regressions.Gold passages were promoted to rank 1 more than six times as often as they were displaced, and into the top 5 nearly three times as often as they dropped out.Source: D Qualitative Analysis: What Moves and… · p. 23
- In three illustrative cases, undated, topically strong hits outranked dated gold passages, while temporal-only retrieval placed each gold passage first; these examples illustrate a mechanism, not an average.The examples were selected as illustrative wins; Table 11 reports the aggregate movement, including regressions.Source: D Qualitative Analysis: What Moves and… · p. 23, Table 12 · p. 24