Source-linked AI summary
TempoQR: Temporal Question Reasoning over Knowledge Graphs
Costas Mavromatis, Prasanna Lakkur Subramanyam, Vassilis N. Ioannidis, Soji Adeshina, Phillip R. Howard, Tetiana Grinberg, Nagib Hakim, George Karypis
TL;DR
Complex temporal questions may require multiple facts, while existing TKG QA methods are mainly designed for simpler questions. TempoQR grounds question representations in context, entities, and temporal information; the paper reports 25–45% improvement over existing methods on complex questions.
Problem
CronKGQA performs poorly on complex questions that combine additional temporal constraints and information from multiple TKG facts.
Source: §1 · p. 1Method
TempoQR combines question text with KG entity representations and question-specific temporal information, then fuses these into a final question representation.
Source: §1 · p. 2Results
25–45%: TempoQR outperforms existing methods on complex questions.
Source: §7 · p. 7Takeaways & Limitations
The framework is designed to handle both simple and complex temporal questions by grounding questions in their entities and time scope.
Source: §1 · p. 1Further research is needed to generalize better to unseen question types.
Source: §7 · p. 8
Abstract
from arXiv · showhide
Knowledge Graph Question Answering (KGQA) involves retrieving facts from a Knowledge Graph (KG) using natural language queries. A KG is a curated set of facts consisting of entities linked by relations. Certain facts include also temporal information forming a Temporal KG (TKG). Although many natural questions involve explicit or implicit time constraints, question answering (QA) over TKGs has been a relatively unexplored area. Existing solutions are mainly designed for simple temporal questions that can be answered directly by a single TKG fact. This paper puts forth a comprehensive embedding-based framework for answering complex questions over TKGs. Our method termed temporal question reasoning (TempoQR) exploits TKG embeddings to ground the question to the specific entities and time scope it refers to. It does so by augmenting the question embeddings with context, entity and time-aware information by employing three specialized modules. The first computes a textual representation of a given question, the second combines it with the entity embeddings for entities involved in the question, and the third generates question-specific time embeddings. Finally, a transformer-based encoder learns to fuse the generated temporal information with the question representation, which is used for answer predictions. Extensive experiments show that TempoQR improves accuracy by 25--45 percentage points on complex temporal questions over state-of-the-art approaches and it generalizes better to unseen question types.
1 Introduction
Temporal KG question answering must handle questions whose time constraints require combining facts, not just answering from a single fact. TempoQR grounds questions in context, entities, and time information, and reports improved accuracy on complex temporal questions.
1 Introduction
- A TKG represents facts with timestamps or time intervals, enabling questions to refer to temporal constraints.Facts can be represented as (subject, relation, object, timestamp) or with a start and end time.Source: §1 · p. 1
- The paper identifies complex temporal questions requiring multiple facts as a weakness for existing TKG question answering.CronKGQA has Hits@1 of 0.392 on complex questions.Source: §1 · p. 1
- The method follows context-, entity-, and time-aware reasoning steps to ground a question such as “Which movie won the Best Picture after The Godfather?”The example’s time scope is after 1972, and its answer is The Sting.Source: §1 · pp. 1–2, Figure 1 · p. 2
- TempoQR improves Hits@1 by 25–45 percentage points over time-unaware methods on complex questions.Its question representation combines context-, entity-, and time-aware information.Source: §1 · p. 2
- TempoQR uses three specialized modules to represent question text, ground it in KG entities, and recover question-specific temporal information.A fusion layer combines these representations for answer prediction.Source: §1 · p. 2
- The authors report state-of-the-art performance on temporal complex questions and provide additional strong baselines.Source: §1 · p. 2
2 Related Work
Prior KGQA methods that enhance question representations address complex questions through logical reasoning or textual side information, but are not suited to temporal constraints. Temporal KGQA methods have used handcrafted rules or embeddings, with CronKGQA challenged by complex temporal questions.
2 Related Work
- KGQA approaches using pretrained KG embeddings perform well on simple questions mapped to incomplete facts but are challenged by complex questions.Source: §2 · p. 2
- Question-enhancement approaches use logical reasoning or text-document side information for complex questions, but are not suited to temporal constraints.Source: §2 · p. 2
- Temporal QA methods have used handcrafted rules, which are not flexible for incomplete KGs.Source: §2 · p. 2
- CronKGQA provides learnable temporal KG reasoning without handcrafted rules, but struggles with complex questions requiring temporal-constraint inference.Source: §2 · p. 3
3 Background
The background defines temporal KGs and their embedding-based fact scoring, then frames question answering as finding an entity or timestamp linked to an annotated question. CronKGQA treats the question as a virtual relation, a setup suited to simple questions but limited for complex temporal constraints.
3 Background
- A TKG consists of entities, relations, timestamps, and facts that associate a subject and object through a relation at a timestamp.Source: §3 · p. 3
3.1 TKG embeddings
- TKG embeddings score valid facts higher than invalid ones; TComplEx uses complex vectors, temporal regularization, and supports inference of missing facts.Its score uses an elementwise relation-time product and the real part of the resulting expression.Source: §3.1 · p. 3
3.2 QA over TKGs
- Table 1 presents different types of temporal questions and marks annotated entities and timestamps with {·}s, {·}o, and {·}τ.Source: Table 1 · p. 3
- TKGQA extracts an entity or timestamp answer, with question entities and timestamps annotated and linked to the TKG.Source: §3.2 · p. 3
- CronKGQA treats the question as a virtual relation in a link-prediction task, using annotated subject and timestamp information to rank answers.Source: §3.2 · p. 3
- CronKGQA is designed for single-link temporal questions, limiting its use on before/after, first/last, and time-join questions.Prior reported results show a 65% improvement over non-TKG methods for simple questions and 15% for complex questions.Source: §3.2 · p. 3
4 Method: TempoQR
TempoQR augments question representations with contextual, entity, and temporal information, using TKG embeddings to ground questions to entities and time scopes. A Transformer fuses these signals into a question representation for answer prediction.
4 Method: TempoQR
- TempoQR uses pre-trained language and TKG embeddings to incorporate question context, linked entities, and question-specific temporal information into question representations.The method’s overall design follows the reasoning steps used to answer temporal questions.Source: §4 · p. 3, §4.1 · p. 3, §4.2 · p. 4
4.1 Context-aware question representation.
- A pre-trained language model encodes the question text into token representations that provide its context.The question includes a [CLS] token, and the language-model representation is projected to the TKG embedding dimension.Source: §4.1 · pp. 3–4
4.2 Entity-aware question representation.
- For entity-aware representations, TempoQR replaces embeddings of tokens linked to entities or timestamps with projected pre-trained TKG entity embeddings.This yields a token embedding matrix that incorporates entity information from the TKG.Source: §4.2 · p. 4
4.3 Time-aware question representation.
- TempoQR-Hard retrieves time scope from TKG facts involving annotated entities, while TempoQR-Soft infers temporal information in the embedding space.The hard approach collects retrieved timestamps and uses the start and end timestamp embeddings; the soft approach treats the question’s time-aware embedding as a relation and uses annotated entities to infer time embeddings.Source: §4.3 · p. 4
4.4 Answer Prediction
- TempoQR adds the two time embeddings to entity-token embeddings, then a Transformer encoder fuses context, entity, and time information into the final question representation.The [CLS] token embedding is used as the final question representation; entity and time scores are trained together with a softmax and cross-entropy loss.Source: §4.3 · p. 4, §4.4 · pp. 4–5
5 Experimental Setting
Experiments use CronQuestions and evaluate TempoQR and baselines under standard and corrupted-TKG settings, including questions with temporal constraints not seen during training. The section also specifies model configuration and the compared methods and variants.
5 Experimental Setting
- CronQuestions contains 410k question-answer pairs over a Wikidata TKG with 125k entities, 203 relations, 1.7k yearly timestamps, and 328k facts.The benchmark includes simple and complex questions with annotated entities and times; the corruption setting removes fact timestamps with probability p, and additional question types combine before/after and first/last constraints.Source: §5 · p. 5
5.1 Model Configuration
- TempoQR uses TComplEx embeddings of dimension 512, a six-layer, eight-head Transformer, and Adam with learning rate 0.0002.Pre-trained language-model parameters and TKG embeddings remain fixed during QA training; training runs for at most 20 epochs with model selection based on validation performance.Source: §5.1 · p. 5
5.2 Baseline methods and TempoQR variations
- The baselines include pre-trained language models, EntityQR, CronKGQA, and EmbedKGQA; CronKGQA and EntityQR also receive hard- and soft-supervision variants using generated time embeddings.EntityQR adds entity awareness to TComplEx scoring without TempoQR’s temporal-information steps; the variants replace a dummy timestamp embedding with the sum of two generated time embeddings.Source: §5.2 · pp. 5–6
6 Results
TempoQR’s results show gains from entity grounding and temporal information on complex questions, with the hard-supervision variant especially strong on some question types. Robustness depends on how TKG corruption is introduced, while neither variant handles unseen question types effectively.
6.1 Main Results
- TempoQR-Hard improves on EntityQR by more than 30% at Hits@1 for complex questions, while TempoQR-Soft improves by 9%; for time answers, TempoQR-Hard beats TempoQR-Soft by 25% at Hits@1.EntityQR also improves over CronKGQA by 17% at Hits@1 and 10% at Hits@10 for complex questions.Source: §6.1 · p. 6
- Methods using the TComplEx scoring function answer 99% of simple questions correctly, while BERT, RoBERTa, EmbedKGQA, and EaE have 35%-65% worse overall Hits@1 and 11%-40% worse Hits@10 than TempoQR, EntityQR, and CronKGQA.The latter methods answer no more than 35% at Hits@1 and 76% at Hits@10 on simple questions.Source: §6.1 · p. 7
6.2 Ablation Study
- Compared with the original CronKGQA, soft and hard supervision improve performance by 2.5% and 6%, respectively, but the same supervision approaches reduce EntityQR’s performance.The authors attribute the EntityQR drop to repeated use of TKG embeddings for answer prediction.Source: §6.2 · p. 7, Table 3 · p. 7
- TempoQR-Hard’s advantage over soft supervision is 27% for ‘first/last’ questions, 3% for ‘before/after’ questions, and 8% for ‘time join’ questions.The paper says retrieving entity start and end timestamps provides more accurate temporal information for ‘first/last’ questions.Source: §6.2 · p. 7
6.3 Robustness over Corrupted TKGs
- With corruption introduced during QA, TempoQR-Hard is greatly affected; at p = 0.8 it performs similarly to EntityQR, but is more robust on ‘first/last’ and ‘time join’ questions with noncorrupted timestamps.For ‘before/after’ questions, it performs worse than TempoQR-Soft even at p = 0.2.Source: §6.3 · p. 7
- When corruption affects both TKG embedding and QA, embedding-reliant TempoQR variants approach EntityQR at large corruption rates such as p = 0.5; CronKGQA is least affected but still performs much lower.TempoQR-Hard has the largest drop for ‘first/last’ questions and does not generalize well to them under a corrupted TKG.Source: §6.3 · p. 7
6.4 Unseen Question Types
- TempoQR-Soft performs best on both generated unseen question types, but Hits@1 remains below 20% for all methods.Figure 3 evaluates unseen complex types during inference, with k of Hits@k on the x-axis.Source: §6.4 · p. 7, Figure 3 · p. 7
7 Conclusion
TempoQR is an embedding framework for complex temporal questions that learns context-, entity-, and time-aware question representations. The paper reports 25–45% improvements over existing methods and identifies generalization to unseen question types as future work.
7 Conclusion
- TempoQR outperforms existing methods for complex questions by 25–45%.The framework learns context-, entity-, and time-aware question representations, using hard or soft supervision for temporal information.Source: §7 · p. 7
- Future research includes extending existing methods to generalize better to unseen question types.Source: §7 · p. 8
8 Appendix
The appendix reports dataset statistics, describes how additional complex questions were generated, and evaluates TempoQR variants, fusion strategies, training sizes, and time embeddings.
8.1 CronQuestions Statistics
- Table 4 counts dataset questions by reasoning type and answer type.The listed reasoning types include simple entity/time, before/after, first/last, and time join; entity-answer and time-answer groups contain different subsets of these types.Source: Table 4 · p. 9, §8.1 · p. 9
8.2 Generation of Additional Complex Questions
- The appendix defines two additional complex question types and reports 261 inference-only QA pairs for them.The types combine before and after constraints, or combine before/after with first/last constraints; the generation process modifies existing templates and filters answers using temporal ordering and proximity.Source: §8.2 · p. 9
8.3 Variants of TempoQR
- TempoQR vastly outperforms TComplEx on complex questions and is competitive across decoders.TComplEx is evaluated with each question’s ground-truth relation; the comparison uses Hits@1.Source: §8.3 · pp. 9–10
- Combining hard and soft supervision improves before/after Hits@1 by 2.1 points, but lowers first/last and time-join performance by 1.9 and 6.7 points, respectively.Source: §8.3 · p. 10
8.4 Fusion of Time Embeddings
- Summing entity and time embeddings performs best among the tested fusion strategies.TempoQR-cat and TempoQR-att perform much worse on time-join and simple questions; the authors attribute this to time information being ignored by their projection and attention layers, respectively.Source: §8.4 · p. 10, Table 5 · p. 10
8.5 Training Size
- As training size decreases, TempoQR-Hard and TempoQR-Soft both perform worse, with a slightly faster drop for TempoQR-Hard.Figure 4 presents performance for complex questions at different training sizes.Source: §8.5 · p. 10, Figure 4 · p. 10
8.6 Is Time All We Need?
- Random time embeddings perform equally with TComplEx embeddings, while positional embeddings perform best for before/after questions.Start and end timestamps are important: TComplEx-Sampled has the lowest performance; random embeddings perform best for first/last questions, while preserving timestamp order benefits before/after reasoning.Source: §8.6 · pp. 10–11, Table 6 · p. 10