Source-linked AI summary
Complex Temporal Question Answering on Knowledge Graphs
Zhen Jia, Soumajit Pramanik, Rishiraj Saha Roy, Gerhard Weikum
TL;DR
Temporal questions over knowledge graphs are important but underexplored, especially when they require multiple entities, predicates, and temporal constraints. Exaqt builds compact temporal answer graphs and applies time-aware R-GCN-based prediction, outperforming three state-of-the-art general-purpose KG-QA systems on TimeQuestions.
Problem
Temporal questions over knowledge graphs have received limited attention despite involving timestamp-constrained answers and complex multi-hop reasoning.
Method
Exaqt uses BERT-based fact selection, compact subgraph construction, temporal-fact augmentation, and time-aware R-GCN answer prediction.
Results
Exaqt outperforms three state-of-the-art complex KG-QA baselines on the TimeQuestions benchmark.
Takeaways & Limitations
The results support specialized treatment of temporal QA rather than relying solely on general-purpose complex KG-QA systems.
Abstract
from arXiv · showhide
Question answering over knowledge graphs (KG-QA) is a vital topic in IR. Questions with temporal intent are a special class of practical importance, but have not received much attention in research. This work presents EXAQT, the first end-to-end system for answering complex temporal questions that have multiple entities and predicates, and associated temporal conditions. EXAQT answers natural language questions over KGs in two stages, one geared towards high recall, the other towards precision at top ranks. The first step computes question-relevant compact subgraphs within the KG, and judiciously enhances them with pertinent temporal facts, using Group Steiner Trees and fine-tuned BERT models. The second step constructs relational graph convolutional networks (R-GCNs) from the first step's output, and enhances the R-GCNs with time-aware entity embeddings and attention over temporal relations. We evaluate EXAQT on TimeQuestions, a large dataset of 16k temporal questions we compiled from a variety of general purpose KG-QA benchmarks. Results show that EXAQT outperforms three state-of-the-art systems for answering complex questions over KGs, thereby justifying specialized treatment of temporal QA.
1 INTRODUCTION
Temporal questions over knowledge graphs require reasoning over entities, predicates, timestamps, and multi-hop constraints, yet have received limited attention. Exaqt addresses this gap with a two-stage pipeline that constructs temporal answer graphs and predicts answers with time-aware graph networks.
- Motivation: Temporal KG questions range from explicit dates to implicit periods and complex multi-hop constraints over intersecting time points and intervals.The motivating example links Obama to his children, their school, and overlapping presidency and study periods.
- Approach: Exaqt is presented as an end-to-end system for answering complex temporal questions over large-scale knowledge graphs.The paper describes it as the first such system and targets questions with multiple entities, predicates, and temporal conditions.
- Approach: Stage 1 constructs a compact answer graph using dense-subgraph algorithms and fine-tuned BERT models to identify relevant KG facts.The system begins from facts associated with detected question entities and distills them into a tractable graph.
- Approach: Stage 2 applies an R-GCN augmented with time-aware entity embeddings, temporal attention, timestamp encodings, and temporal question signals.These additions address temporal constraints that standard R-GCNs do not model.
- Evaluation: About 16k temporal questions are compiled in TimeQuestions, and experiments over Wikidata report superiority over three state-of-the-art complex KG-QA baselines.The benchmark and comparative evaluation are introduced as central project contributions.
2 CONCEPTS AND NOTATION
The paper formalizes knowledge-graph facts, temporal question categories, and answer graphs to represent the structures required for temporal KG question answering.
- Knowledge graphs: A knowledge graph is a collection of subject-predicate-object facts represented as directed graph nodes and labeled edges.Facts may be binary or n-ary, with entities, literals, and types appearing as nodes.
- Facts: N-ary facts combine a main predicate and object with qualifier predicate-object pairs, often storing temporal information such as start and end dates.The paper emphasizes qualifiers because much temporal information in modern knowledge graphs is represented this way.
- Temporal questions: Temporal questions include explicit expressions, implicit expressions, temporal ordinals, and temporal answers, and a question may belong to multiple categories.Examples include “in 2009,” “when Obama became president,” “first,” and questions asking when an event occurred.
- Answers: An answer is a set of entities or literals, while an answer graph is a knowledge-graph subset containing the necessary evidence for answering the question.The formalization supports answer prediction over a question-relevant graph rather than the entire KG.
3 CONSTRUCTING ANSWER GRAPHS
Exaqt constructs answer graphs by linking question entities to relevant facts, connecting them with compact subgraphs, and adding selected temporal evidence for multi-hop reasoning.
- Finding question-relevant KG facts: NERD systems identify and link question mentions to KG entities, whose associated facts provide the initial search zone.Using TagMe and ELQ helps mitigate entity-linking errors and improve answer recall.
- Finding question-relevant KG facts: A BERT classifier scores question-fact pairs, using answer-containing facts as positives and question-specific non-answer facts as negatives.Temporal qualifiers are retained when retrieving facts, increasing coverage for positive examples.
- Computing compact subgraphs: The system connects relevant facts by adding shortest KG paths between question entities before compact-subgraph construction.Candidate paths are verbalized and scored with BERT for question relevance.
- Computing compact subgraphs: An exact Group Steiner Tree algorithm selects minimal connected trees, while retaining multiple trees to reduce risk from spurious connections.The method prioritizes precision but avoids relying on a single potentially irrelevant tree.
- Augmenting subgraphs with temporal facts: The completed graph is augmented with question-relevant temporal facts from entities in the graph, including potentially two-hop evidence.A second fine-tuned BERT model ranks temporal facts, enabling answers that require evidence beyond one hop.
4 PREDICTING ANSWERS WITH R-GCN
EXAQT converts the answer graph into a directed relational graph and extends R-GCN answer prediction with temporal question, entity, timestamp, and relation signals. Its updates combine entity, question, time-aware entity, and temporally attended neighborhood representations.
- R-GCN basics: EXAQT converts the answer graph into a directed relational graph whose nodes are entities, literals, and types, with predicates as edge labels.The model builds on the KG-only setting of GRAFT-Net for answer prediction.
- Question representation: The question representation combines LSTM word encoding with multi-hot temporal category and signal vectors before feed-forward projection.Categories include explicit, implicit, and temporal-answer types, while signals include before, after, start, finish, ordinal, overlap, and no-signal cases.
- Temporal signals: Time encoding represents timestamps with sinusoidal position encodings whose summed components preserve sequential ordering for before-and-after reasoning.The encoding uses positions such as day and month within a timestamp.
- Temporal signals: Time-aware entity embeddings encode associated temporal facts, chronologically process them with an LSTM, and use its final state as each entity’s temporal representation.Each fact combines entity embeddings, averaged relation embeddings, and timestamp encodings.
- Temporal signals: Attention over temporal relations distinguishes edges sharing a relation but having different timestamps by comparing relation-plus-time representations with the question.The attention is normalized over all outgoing edges from an entity.
- Entity updates: Each entity update applies a feed-forward network to four states: prior entity, prior question, prior time-aware entity, and an attention-weighted neighborhood aggregate.Relation-specific transformations depend on edge type and direction.
5 EXPERIMENTAL SETUP
The experiments introduce TimeQuestions, define benchmark evaluation and baselines, and describe the Wikidata and training configuration used for comparison. TimeQuestions expands prior temporal QA collections to 16,181 questions assembled from eight KG-QA datasets.
- Benchmark: 16,181 questions comprise TimeQuestions, compiled by searching eight KG-QA datasets for temporal expressions or signal words and manually filtering and verifying candidates.Questions from Freebase or DBpedia were mapped to Wikidata through Wikipedia links.
- Benchmark: The benchmark is split 60:20:20 into 9,708 training, 3,236 development, and 3,237 test questions.
- Baselines: Uniqorn, GRAFT-Net, and PullNet serve as baselines because they provide Group Steiner Tree, R-GCN, and multi-hop QA components related to EXAQT.All baselines were trained and fine-tuned using the TimeQuestions train and development sets.
- Metrics: Evaluation averages P@1, MRR, and Hit@5 over benchmark questions for ranked answer lists.P@1 checks the top answer, MRR uses the first correct-answer rank, and Hit@5 checks the first five positions.
- Configuration: The processed April 2020 Wikidata dump contains about 2B retained triples after filtering a 12B-triple source dump.The retained dump occupies about 340 GB, compared with 2 TB uncompressed for the original dump.
- Configuration: BERT fine-tuning uses a 1:5 positive-to-negative instance ratio and an 80:20 training-to-hyperparameter-selection split.The model is BERT-base-cased for sequence-pair classification.
- Configuration: R-GCN training uses 100-dimensional Wikipedia2Vec embeddings and 100-dimensional TCE, TSE, TE, and TEE representations.The model was trained on an Nvidia Quadro RTX 8000 GPU server, with hyperparameters tuned on the TimeQuestions development set.
6 KEY FINDINGS
EXAQT outperforms general-purpose KG-QA baselines on temporal questions, while results reveal substantial variation by question type and limitations of fixed-hop reasoning.
- EXAQT achieves across-the-board superiority over three baselines, indicating that general-purpose complex QA systems struggle with temporal QA requirements.The reported comparisons are statistically significant for each category, baseline, and metric.
- GST backbones alone do not suffice for temporal questions, although they remain useful for pruning noisy KG subgraphs.EXAQT’s improvement over Uniqorn shows that GST non-terminals alone cannot answer temporal questions.
- R-GCNs provide the strongest general-purpose baseline, while augmenting them with temporal information performs well.GRAFT-Net is the strongest baseline despite lacking explicit temporal support.
- PullNet’s fixed 2-hop assumption fails when answers occur in qualifiers or when questions are simple rather than multi-hop.Its shortest-path restriction does not generalize beyond the benchmark’s trained and evaluated situations.
- Questions with temporal ordinals are hardest, with P@1 < 0.5, whereas temporal-answer questions are easiest.Explicit temporal expressions are intermediate, and implicit expressions are more challenging.
7 IN-DEPTH ANALYSIS
Detailed analyses examine entity-linking choices, recall-oriented graph construction, statistical testing, and examples where EXAQT succeeds beyond the baselines.
- TagMe plus ELQ is the best NERD configuration because their recall-oriented and precision-biased behaviors complement each other.AIDA plus ELQ detects a similar number of entities but yields slightly lower answer recall.
- Two NERD systems still achieve only 75.8% answer recall before later graph-construction steps.The analysis also tracks how shortest paths affect graph connectivity and answer retention.
- EXAQT’s advantage over GRAFT-Net is tested with a two-tailed paired t-test, marking significance at p < 0.05.The significance marker applies to comparisons against the strongest baseline.
- Table 4 compares EXAQT with three complex QA baselines on the TimeQuestions test set.The cited materials identify the comparison scope but do not provide the table’s individual metric values.
- Table 5 compares alternative NERD methods on the test set, complementing the reported TagMe-plus-ELQ analysis.The table’s caption establishes its comparison target, while the accompanying analysis identifies the best configuration.
- Temporal-category information is especially important for questions with temporal answers, where it provides the greatest assistance.Other factors contribute varying degrees of assistance across categories.
8 RELATED WORK
Prior temporal QA research spans information retrieval and text-based question answering, but often depends on restrictive preprocessing, markup, or question formulations.
- Temporal intent has long been studied in IR, including query and document processing and QA over text.These approaches commonly require substantial document preprocessing and markup.
- Structured KGs do not benefit from corpus markup, and ad hoc Web questions and documents challenge carefully crafted parsers.Such requirements can make text-oriented approaches less aligned with realistic Web settings.
- One notable temporal QA approach decomposes complex questions into simpler components and recomposes answer fragments.This illustrates a prior strategy for handling complex temporal questions over text.
9 CONCLUSIONS
The paper addresses underexplored temporal QA over knowledge graphs with a complete EXAQT pipeline combining BERT classifiers, graph convolutional networks, and temporal signals.
- Temporal questions and temporal KG information remain underexplored despite their importance for analysts, journalists, and advanced information needs.The conclusion frames this gap as the motivation for the work.
- EXAQT combines BERT-based classifiers and graph convolutional networks, augmenting both components with temporal signals.The paper presents this combination as a complete pipeline for temporal KG-QA.
- Experiments on a large collection of complex temporal questions show EXAQT’s superiority over state-of-the-art general-purpose KG-QA methods.The conclusion reports this as the central experimental outcome.