Source-linked AI summary
ChronoSynth: Constraint-Aware Question Synthesis over Temporal Knowledge Graphs
Feng Liang, Weixin Zeng, Runhao Zhao, Mingjun Zhou, Shaoshi Yang, Xiang Zhao
TL;DR
Temporal question generation must preserve the answer field, time condition, reference event, and order that determine the answer, yet fluent questions can alter them. ChronoSynth makes temporal intent explicit and uses compatible patterns with validation and feedback; experiments report gains across CRONQUESTIONS and MultiTQ.
Problem
Temporal KG question generation must preserve answer-determining conditions, but fluent generated questions can alter temporal constraints and change the answer set.
Source: §1 · p. 1Method
ChronoSynth constructs a Temporal Intent State, retrieves and adapts compatible typed patterns from indexed memory, and validates generated questions with feedback-guided refinement.
Source: §1 · p. 2Results
ChronoSynth improves generation quality over compared baselines on CRONQUESTIONS and MultiTQ, including across generator backends.
Source: §1 · p. 2Takeaways & Limitations
Ablation and grouped analyses identify pattern memory as the main source of improvement, and the local efficiency study reports a quality–cost advantage over KQG-CoT.
Source: §6 · p. 8ChronoSynth relies on a predefined temporal-rule inventory and heuristic state construction; verification depends on the interpreter and does not guarantee semantic correctness, and evaluation has not tested other domains or languages.
Source: Limitations · p. 8
Abstract
from arXiv · showhide
Knowledge graph question generation (KGQG) generates natural-language questions from structured graph evidence and a target answer. When the input graph is temporal, a valid question must also preserve the answer field, time condition, reference event, and temporal order that determine the answer set. Existing KGQG methods and prompting baselines can produce fluent questions that alter these conditions, causing temporal semantic drift. This paper studies temporal KGQG as a constraint-preserving graph-to-question problem and proposes ChronoSynth, a memory-augmented and constraint-aware generation framework. ChronoSynth first constructs a Temporal Intent State to make the intended question semantics explicit. It then retrieves compatible typed question patterns from an indexed pattern memory, adapts them to the input graph, and validates the generated question. Validation feedback guides revisions when a supported inconsistency is detected. Experiments on CRONQUESTIONS and MultiTQ demonstrate state-of-the-art generation performance, with ablations supporting the contributions of its components.
1 Introduction
Temporal question generation must preserve the conditions that determine an answer, because fluent questions can still drift from the intended temporal query. ChronoSynth makes that intent explicit and uses it to guide question-pattern retrieval, adaptation, and validation.
1 Introduction
- Changing a time condition, reference event, answer field, or order cue can change the answer set even when the generated question is fluent.The paper identifies preserving these conditions as temporal faithfulness.Source: §1 · pp. 1–2, Figure 1 · p. 2
- TKGQG must identify the intended answer field and temporal rule, reuse only temporally compatible patterns, and validate outputs that may alter requirements or disclose answers.Source: §1 · p. 2
- The paper frames TKGQG as constraint-preserving graph-to-question generation and makes answer field, temporal rule, reference point, and graph context explicit in a Temporal Intent State.Source: §1 · p. 2
- ChronoSynth constructs that state, retrieves and adapts compatible typed patterns from indexed memory, then validates questions and uses feedback to refine them.Source: §1 · p. 2
- ChronoSynth is evaluated on CRONQUESTIONS and MultiTQ, with stronger generation scores than compared baselines and analyses of components, temporal faithfulness, and synthesis efficiency.Source: §1 · p. 2
2 Preliminaries
The preliminaries define temporal graph facts and the generation task, then outline the question-generation and checking procedure. A valid question should be fluent, answerable from the local graph, and consistent with its temporal intent.
2 Preliminaries
- A temporal knowledge graph represents facts as subject–relation–object tuples associated with a timestamp or interval.Its fact set is a subset of E × R × E × T.Source: §2 · p. 2
- Given a local temporal subgraph and target answers, TKGQG generates a question that is fluent, answerable, and consistent with the input’s time condition, reference point, temporal order, and answer field.Source: §2 · pp. 2–3
- ChronoSynth builds a state, retrieves and ranks patterns, adapts and filters them, generates a draft, then checks and repairs it when a supported inconsistency is found.The procedure reinterprets a proposed correction and checks it before returning the revised question.Source: §2 · p. 3
3 Methodology
ChronoSynth makes temporal intent explicit, retrieves and adapts compatible typed patterns, then validates and refines questions against graph evidence and constraints. Its pipeline uses a shared indexed memory and preserves distinctions among answer fields, temporal rules, reference points, and graph context.
3.1 Overview
- ChronoSynth constructs a Temporal Intent State, retrieves compatible typed patterns from indexed memory, adapts them to the input graph, and validates and refines generated questions.Memory construction is offline, and the same index is reused across generation instances and backends.Source: §3.1 · p. 3, Figure 2 · p. 4
3.2 Temporal Intent State Construction
- The Temporal Intent State represents answer field and type, temporal rule, time granularity, focus edge, reference, comparison value, and graph context.It is inferred from normalized graph records and supplied answers; explicit constraint records take precedence when determining the temporal rule.Source: §3.2 · p. 3
3.3 Indexed Pattern Memory Construction
- Training questions are normalized into typed patterns that preserve question structure while separating it from specific entities and dates.Memory records use a primary key combining dataset dialect, temporal rule, answer field and type, granularity, and edge-count bucket; complementary indexes and a bounded fallback pool broaden retrieval.Source: §3.3 · pp. 3–4
3.4 Memory-guided Question Generation
- ChronoSynth retrieves and ranks candidate patterns using temporal, answer, granularity, relation, dialect, and graph-shape compatibility, then adapts roles and realizes a constrained question.The fixed generator uses the state, graph and answers, retrieved records, adapted drafts, and constraints; generation does not update model parameters or pattern memory.Source: §3.4 · p. 4
3.5 Validation and Refinement
- Validation checks format, placeholders, answer leakage, order cues, temporal-rule compatibility, and answer field; supported semantic inconsistencies trigger targeted revisions.A second interpretation verifies that a revision expresses the selected correction and preserves target answers and explicit constraints; otherwise, the base question is retained.Source: §3.5 · pp. 4–5
4 Experimental Evaluation
The evaluation compares ChronoSynth with trained and training-free baselines on two temporal question-generation datasets, then tests its components, human-rated question quality, and downstream QA utility. Across these evaluations, ChronoSynth outperforms the compared baselines, with the largest generation gains on MultiTQ.
4.1 Experimental Setting
- The evaluation uses graph-converted test splits of CRONQUESTIONS and MultiTQ, comparing trained generation with training-free prompting and ChronoSynth.For LLM comparisons, ChronoSynth and its prompting counterparts use aligned backend families; the reported generation metrics include BLEU, ROUGE-L, and CIDEr.Source: §4.1 · p. 5, §4.2 · p. 5
4.2 Main Results
- On MultiTQ, ChronoSynth’s best scores improved BLEU-4 by 46.0% and CIDEr by 76.1% over the strongest baseline for each metric.With Qwen3.5-9B, it reached BLEU-4 0.3117 and CIDEr 2.6509; the dataset contains richer temporal rules and larger local subgraphs.Source: §4.2 · p. 5, Table 1 · p. 6
- On CRONQUESTIONS, ChronoSynth improved BLEU-4 by 15.3% and CIDEr by 34.3% over the strongest baseline for each metric.Across both generator configurations, ChronoSynth ranked first and second across all six metrics, and gains over prompting baselines held for both backends.Source: §4.2 · pp. 5–6, Table 1 · p. 6
4.3 Ablation Study
- Removing pattern memory caused the largest ablation losses, while relation-only retrieval and removing feedback refinement also underperformed the complete method.Without pattern memory, BLEU-4 fell from 0.2956 to 0.1881 on CRONQUESTIONS and from 0.3044 to 0.1543 on MultiTQ; refinement yielded smaller, consistent improvements.Source: §4.3 · p. 6, Figure 3 · p. 7
4.4 Human Evaluation
- ChronoSynth received the highest mean human rating in fluency, temporal faithfulness, and answerability on both datasets.Temporal-faithfulness scores were 2.973 on CRONQUESTIONS and 2.900 on MultiTQ, compared with baseline maxima of 2.360 and 2.420.Source: §4.4 · p. 7, Table 2 · p. 7
4.5 Downstream Question Answering
- ChronoSynth achieved the highest mean EM among synthetic-data conditions on both datasets, exceeding the strongest baseline by 0.34 points on CRONQUESTIONS and 8.66 on MultiTQ.The CRONQUESTIONS difference was small relative to seed variation; the gaps to Gold Reference were 2.84 and 5.65 points, respectively.Source: §4.5 · pp. 7–8, Table 3 · p. 7
5 Related Work
Related work spans graph-to-question generation, temporal question answering over knowledge graphs, and retrieval-guided generation. ChronoSynth’s setting differs by requiring generated questions to preserve temporal conditions and by retrieving typed question patterns rather than evidence passages.
5.1 Knowledge Graph Question Generation
- KGQG progressed from graph-encoded question generation to retrieval, diverse-subgraph modeling, and LLM prompting, while temporal KGQG must preserve temporal rules, reference events or times, and answer fields.The related work also connects KGQG to answer-aware, pretrained, and controlled generation, graph-to-text, and KBQA.Source: §5.1 · p. 8
5.2 Temporal Question Answering over Knowledge Graphs
- Temporal KGQA studies answering questions over temporal graphs, whereas this paper addresses the inverse task of generating questions while preserving temporal rules.CRONQUESTIONS organizes questions by temporal complexity, and MultiTQ stresses multi-granularity temporal questions.Source: §5.2 · p. 8
5.3 Retrieval-guided and Constraint-aware Generation
- Unlike retrieval-augmented generation that retrieves evidence, ChronoSynth retrieves typed question patterns and validates answer leakage, placeholder resolution, order cues, temporal rules, and answer fields.Related work includes retrieval-augmented generation and self-refinement.Source: §5.3 · p. 8
6 Conclusion
ChronoSynth reduces temporal semantic drift through explicit intent construction, indexed pattern retrieval, typed adaptation, and validation with feedback refinement. It achieves consistent gains across benchmarks and backends, with pattern memory identified as the main source of improvement.
6 Conclusion
- ChronoSynth reduces temporal semantic drift through intent-state construction, indexed pattern-memory retrieval, typed adaptation, and validation with feedback refinement.The framework shows consistent gains over trained methods and training-free baselines across CRONQUESTIONS, MultiTQ, and multiple backends.Source: §6 · p. 8
- Ablation and grouped analyses identify pattern memory as the main source of improvement.The local efficiency study also reports a quality–cost advantage over KQG-CoT.Source: §6 · p. 8
- Future work will extend the temporal-rule inventory and strengthen temporal-consistency checking for broader query styles.Source: §6 · p. 8
Limitations
ChronoSynth’s temporal rules, state construction, memory coverage, and verification impose scope boundaries, and evaluation is limited to temporal question-generation benchmarks.
Limitations
- ChronoSynth relies on a predefined temporal-rule inventory and heuristic state construction, which may need adaptation for new graph schemas or temporal expressions.Source: Limitations · p. 8
- Pattern reuse depends on training-memory coverage, while interpreter-based verification does not guarantee semantic correctness.Source: Limitations · p. 8
- Evaluation covers temporal question-generation benchmarks; performance in other domains and languages remains untested.Source: Limitations · p. 8
Ethical Considerations
The appendices describe ethical-use guidance and the temporal-rule, memory, retrieval, validation, and complexity details underlying ChronoSynth.
Ethical Considerations
- Generated questions may reproduce source-graph biases or factual errors, so review is advised before downstream dataset use, especially in sensitive applications.Reuse of source datasets and models should follow their licenses and terms of use.Source: Ethical Considerations · p. 9
A.1 Temporal Rules
- Explicit constraints take precedence; otherwise, focus and neighboring facts with the same relation guide temporal interpretation rather than recovering unique reference wording.Inferred rules use normalized temporal keys and select a nearest applicable reference before checking earliest or latest status.Source: §A.1 · p. 12
A.2 State Fields and Memory Indexes
- The state fields and typed placeholders encode answer, temporal, graph, and mention information used to index and compare question patterns.The memory uses rule-aware, relation-pattern, and graph-shape indexes plus a bounded fallback pool.Source: §A.2 · p. 12
A.3 Offline Memory Construction
- The offline memory stores normalized reference-question patterns and retrieval metadata from training records, then reuses the fixed index during generation and refinement.Source: §A.3 · p. 12
A.4 Retrieval Features and Settings
- Retrieval gathers and deduplicates candidates across indexes, adds fallback candidates when fewer than 24 are available, and ranks them using compatibility features.Each index bucket holds at most 192 records, the fallback pool at most 512, and each prompt retains four memory records and up to four adapted drafts.Source: §A.4 · pp. 12–13
A.5 Validation and Feedback Refinement
- Validation checks format, placeholders, answer disclosure, temporal cues, and answer field, then executes an interpreted program against target answers and explicit constraints.Feedback changes at most two supported temporal fields while keeping entity and relation bindings fixed; a rewrite is accepted only after equivalent-program, execution, and surface checks.Source: §A.5 · p. 13
A.6 Complexity and Reuse
- Index construction is linear in training records, ranking costs O(C log C), and adapting k patterns costs O(kL), excluding model inference and program execution.The memory is built once and reused; verification uses at most two interpretations and one model rewrite per question, excluding retries.Source: §A.6 · p. 13
B.1 Benchmark Preparation
The experiments use graph-converted CRONQUESTIONS and MultiTQ inputs, specified generation and scoring procedures, ablation variants, and grouped comparisons across input types.
B.1 Benchmark Preparation
- Generation uses graph facts, target answers, and explicit constraints, while reference wording and source annotations are withheld from generation and retained for evaluation.Source: §B.1 · p. 14
B.2 Generation and Scoring
- GPT-4o-mini and Qwen3.5-9B generate and rewrite questions; main-table configurations use GPT-4o-mini for interpretation, and metrics follow specified tokenization and scoring procedures.All test records remain in evaluation, including unresolved outputs.Source: §B.2 · p. 14
B.3 Resources and Intended Use
- The resources are English temporal question-generation benchmarks with local graph facts, target answers, and temporal constraints; source data retain their original distribution terms.Model weights are not redistributed, and the supplements preserve source attribution and third-party notices.Source: §B.3 · p. 14, Table A4 · p. 14
C.1 Ablation Configurations
- Ablations omit feedback refinement, replace temporal-compatible retrieval with relation-only retrieval, or remove pattern memory while retaining state construction and default drafts.The latter two variants retain feedback refinement.Source: §C.1 · p. 14
C.2 Performance across Input Groups
- 0.1460 BLEU-4 gain for entity answers and 0.0457 for time answers on CRONQUESTIONS; MultiTQ gains are 0.1382 and 0.1708, respectively.The complete method improves both metrics in every reported group; counts are included because small groups do not support general conclusions on their own.Source: §C.2 · p. 15, Table A5 · p. 15
D Human Evaluation Protocol
The evaluation combines matched human ratings and controlled downstream question-answering tests with a separate model-based comparison of pattern memory. Human ratings show positive paired differences, while the supplementary judge finds dataset-dependent semantic-rating differences.
D Human Evaluation Protocol
- Three blinded raters scored 50 matched inputs per dataset on fluency, temporal faithfulness, and answerability using displayed graph facts, answers, and temporal conditions.Reference questions were withheld, and the displayed conditions were independent of ChronoSynth’s inferred state.Source: D Human Evaluation Protocol · pp. 15–16
- Human-rated mean advantages range from 0.47 to 0.69 points, and all 18 pointwise 95% intervals have positive lower bounds, with a minimum of 0.353.Intervals are unadjusted for multiple comparisons.Source: D Human Evaluation Protocol · p. 16
E Downstream Question Answering Protocol
- Five question-source conditions train the same reader on 1,000 matched inputs, with complete-answer-set EM and F1 measured across 19,122 CRONQUESTIONS and 36,318 MultiTQ test records.The conditions share the reader, training budget, and evaluation protocol.Source: E Downstream Question Answering Protocol · pp. 16–17
F Supplementary Model-Based Assessment
- A separate GPT-4.1-mini judge found memory gains in temporal faithfulness and answerability on CRONQUESTIONS (+0.474; +0.586), but losses on MultiTQ (−0.165; −0.159).This comparison used different samples and an evaluator whose scores are not interchangeable with human ratings.Source: F Supplementary Model-Based Assessment · p. 17
G Local Synthesis Efficiency
In the local Qwen3.5-9B experiment, ChronoSynth leads the reference metrics and is more efficient than KQG-CoT, while direct prompting and Vanilla RAG are faster but have lower BLEU-4.
G Local Synthesis Efficiency
- All four methods used Qwen3.5-9B on one H100 GPU with a shared 900-token output limit; ChronoSynth’s local pipeline also used that model for interpretation and rewriting.The comparison retained all outputs, including requests that reached the output limit.Source: G Local Synthesis Efficiency · pp. 17–18
- Runtime includes prompt construction and inference requests but excludes startup, preprocessing, index preparation, and warmup; token counts include input and output across requests.The eight complete local efficiency runs required approximately 4.68 H100 GPU-hours in total.Source: G Local Synthesis Efficiency · p. 17
- ChronoSynth leads all six reference metrics; BLEU-4 is 0.2993/0.3141 versus KQG-CoT’s 0.2108/0.1811 on CRONQUESTIONS/MultiTQ.Figure A1 compares generation quality and cost across the four methods.Source: G Local Synthesis Efficiency · pp. 17–18, Figure A1 · p. 18
- ChronoSynth’s throughput is 12.17/11.38 questions per second versus KQG-CoT’s 9.64/5.04, with synthesis times of 26.19/53.17 versus 33.05/120.13 minutes.Values are ordered CRONQUESTIONS/MultiTQ.Source: G Local Synthesis Efficiency · p. 18
- ChronoSynth uses 1,316/1,481 tokens per question versus KQG-CoT’s 1,916/3,653, while direct prompting and Vanilla RAG use fewer tokens but have lower BLEU-4.Figure A2 presents the BLEU-4 versus token-use trade-off.Source: G Local Synthesis Efficiency · p. 18
H Additional Evaluation on ChronoQG
On ChronoQG, ChronoSynth leads all six reference-based generation metrics and uses fewer tokens than KQG-CoT, though direct prompting and Vanilla RAG are faster.
H Additional Evaluation on ChronoQG
- ChronoQG-AB85 contains 15,186 records and uses a fixed approximately 80/20 split, yielding 12,159 training and 3,027 test records.Records sharing a source instance or identical task input stay in the same partition.Source: H Additional Evaluation on ChronoQG · p. 18, Table A8 · p. 18
- The ChronoQG adapter uses native constraint codes and conjunctions in the Temporal Intent State and pattern-memory indexes, and validation checks supported answer-role projections and grounded temporal relation pairs.Unresolved validation cases retain their initial questions; the methods use the same graph facts, target answers, and native temporal conditions.Source: H Additional Evaluation on ChronoQG · pp. 18–19
- ChronoSynth leads all six metrics on ChronoQG; BLEU-4 0.4562 exceeds Vanilla RAG’s 0.3834 by 19.0%, with ROUGE-L 0.6472 and CIDEr 4.3373.Table A9 reports the generation scores for all 3,027 test records.Source: Table A9 · p. 19, H Additional Evaluation on ChronoQG · p. 19
- ChronoSynth uses 5,012 tokens per question versus KQG-CoT’s 14,106, a 64.5% reduction, with 2.36 calls per question and a 51.65-minute runtime.Direct prompting and Vanilla RAG are faster, while ChronoSynth obtains stronger reference-based generation scores.Source: Table A10 · p. 19, H Additional Evaluation on ChronoQG · p. 19