TEMA: Evidence-Grounded Temporal Question Answering in Multi-Turn Multi-Audio Dialogs

1. INTRODUCTION

• Existing temporal localization work largely targets single-recording, single-turn queries, while multi-audio dialog benchmarks do not require event boundaries or temporal calculations across turns.Prior settings include timestamped captioning and grounding, dialog follow-ups, and cross-recording attribute selection.Source: §1 · p. 1
• Multi-turn temporal QA must track question and dialog context, identify relevant recordings, and recover every event boundary needed for counts, durations, and comparisons.Missing one occurrence can change a count or cumulative duration.Source: §1 · p. 1
• On TEMA-Bench, TEMA achieves higher overall QA than all compared baselines, including open Qwen2.5-Omni-7B and AF-Next-Instruct and proprietary Qwen3.5-Omni-Plus and Gemini 3.5 Flash.TEMA-Bench jointly evaluates temporal evidence and final answers.Source: §1 · p. 2
• TEMA introduces TEMA-Dialog, with 40,704 dialogs and per-turn evidence, explanations, and answers, plus three-stage training and TEMA-Bench for joint evidence-and-answer evaluation.The benchmark retains model-generated history across 253 dialogs and 1,239 questions.Source: §1 · p. 2

2.1. Task design and temporal evidence

• The task covers event times, durations, counts, ordering, cross-audio comparisons, and historical references, with only uploaded recordings accessible at each turn.Recordings may be uploaded upfront or introduced incrementally.Source: §2.1 · p. 2
• Route identifies recordings to inspect, while Span records every matching event interval or [NONE]; Reason performs selection and temporal calculations before Answer responds.Span retains all instances even when a question uses only one, supporting different temporal operations from the same evidence.Source: §2.1 · p. 2

2.2. Constructing the TEMA-Dialog dataset

• Dataset construction combines normalized event annotations, event-based audio grouping, connected question generation, and dialog auditing against event tables.Similar groups use event-label similarity and CLAP filtering, while random groups diversify acoustic scenes.Source: §2.2 · pp. 2–3
• TEMA-Dialog contains 40,704 dialogs and 198,195 response turns, with 1–6 recordings and 2–13 turns per dialog; TEMA-Bench Test253 has 253 dialogs and 1,239 questions.The training and test resources draw recordings from the source datasets’ respective training and test splits.Source: §2.2 · p. 3

2.3. Three-stage training

• Training proceeds from query-to-interval grounding initialization to full-dialog SFT, then Span-only GRPO that optimizes evidence rather than directly rewarding Reason or Answer.The three stages use AudioGrounding examples, TEMA-Dialog supervision, and verifiable Route and Span evidence, respectively.Source: §2.3 · p. 3
• Open-ended answers make string or value matching insufficient because equivalent facts can be worded differently and identical values can refer to different instances.Route and Span permit direct checks of scope, instance counts, absence states, and endpoints against event tables.Source: §2.3 · p. 3
• The evidence reward gives complete matches at least 0.9, while valid incomplete evidence scores at most 0.1, prioritizing completeness over partial correctness.A complete match requires correct scope, per-recording counts and [NONE] states, and onset and offset errors within 0.1 s.Source: §2.3 · p. 3
• The continuous evidence score combines scope agreement, matched interval quality, and explicit verification of recordings without matching events.Interval matching uses maximum-weight Hungarian matching within each shared recording; the absence score checks inclusion in scope and [NONE] marking.Source: §2.3 · p. 3

3.2. Main results

• 69.81%/58.84% QA on Qwen2.5-Omni/AF-Next; TEMA also outperforms non-TEMA models in QA+T.Source: Table 1 · p. 4, §3.2 · p. 4

3.1. Experimental setup

• Five model families are evaluated on Test253 with upload-order recordings and model-generated history; structured systems are scored on answers and evidence.The first two stages train LoRA adapters in language-model and audio-projection layers while freezing base weights and the audio encoder.Source: §3.1 · pp. 3–4

3.3. Training-stage ablations

• 11.79 and 20.14 percentage-point gains in QA and Span F1, respectively, follow temporal grounding initialization under identical dialog supervision.Adding Span-only GRPO improves Span F1 by 6.35/14.29 percentage points and QA by 0.88/0.81 points on Qwen2.5-Omni/AF-Next, without directly rewarding Reason or Answer.Source: §3.3 · p. 4

3.4. Temporal-evidence analysis

• Reference Route and Span raise Qwen2.5-Omni QA by 27.52 percentage points, but I+S+G lowers cross-audio comparison from 53.33% to 51.11%.Evidence reinforcement learning improves interval recovery and overall answering on both backbones; without explicit Route/Span explanations, the original model reaches 82.81%. The authors recommend turn-level analysis of coverage, boundaries, and cross-turn consistency.Source: §3.4 · p. 4

• TEMA uses complete query-conditioned temporal evidence to connect event perception with multi-turn, multi-audio answering.Source: §4 · p. 4
• TEMA-Dialog supplies per-turn supervision, while TEMA-Bench jointly evaluates evidence and final answers.Source: §4 · p. 4
• Three-stage training improves temporal QA on both backbones, and reinforcement learning on verifiable evidence alone further improves interval recovery and answering.Source: §4 · p. 4