Source-linked AI summary
Query-Conditioned Liquid Temporal Grounding for Long-Form Audio
Yuelan Cheng, Yonggang Zhu, Yongqiang Chen, Peiwei Chang, Yingyi Chen, Wenwu Wang
TL;DR
Long-form temporal audio grounding must locate brief target events in lengthy recordings despite queries with multiple semantic constraints. The paper conditions continuous-time temporal modeling on the query, generates candidates for frozen-model selection, and locally refines boundaries. On English LAT-Bench, it reports consistent improvements with two audio-language backbones, while identifying stronger candidate selection as future work.
Problem
Long-form TAG must locate brief target segments among similar content in lengthy recordings, often using queries with multiple semantic constraints.
Source: §1 · p. 1Method
The framework applies query-conditioned FiLM before CfC temporal modeling, generates multiple candidates for frozen LALM selection, and locally refines the selected boundaries.
Source: §1 · p. 1, §2.1 · p. 2Results
On English LAT-Bench, the framework consistently improves grounding with MOSS-Audio and Qwen3-Omni, although LAT-Audio has the strongest absolute localization performance among reported systems.
Source: §1 · p. 1, §4.1 · p. 4Takeaways & Limitations
With frozen backbones and lightweight trainable modules, the framework supports single-GPU training, and ablations confirm complementary roles for FiLM and CfC.
Source: §5 · p. 4The Top-1–oracle gap motivates future work on stronger candidate selection.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Long-form temporal audio grounding (TAG) is challenging because target events often occupy only a small portion of a recording, while queries may impose multiple semantic constraints over a large temporal search space. We propose a query-conditioned liquid temporal grounding framework that injects query semantics through FiLM before a closed-form continuous-time (CfC) network models long-range temporal dependencies. Multiple query-relevant candidates are generated, from which a frozen large audio-language model selects the most compatible one for lightweight local boundary refinement. The pretrained backbone remains frozen, with only lightweight grounding modules optimized. Experiments on the English TAG subset of LAT-Bench show consistent improvements across MOSS-Audio and Qwen3-Omni, while ablations confirm the complementary roles of FiLM and CfC.
1. INTRODUCTION
Long-form temporal audio grounding searches lengthy recordings for brief events described by queries with multiple semantic constraints. The proposed framework conditions continuous-time temporal modeling on the query, then uses candidate selection and local refinement.
1. INTRODUCTION
- Long-form TAG must find brief target segments within much longer recordings that may contain acoustically or semantically similar content and complex queries.Queries can constrain speakers, events, attributes, and temporal relations.Source: §1 · p. 1
- FiLM injects query semantics before a closed-form continuous-time network models temporal dependencies and generates candidate intervals for frozen LALM selection and local boundary refinement.The pretrained audio-language backbone stays frozen while only lightweight grounding modules are optimized.Source: §1 · p. 1
- Experiments on English LAT-Bench show consistent improvements with MOSS-Audio and Qwen3-Omni, while ablations indicate complementary roles for FiLM and CfC.Lightweight boundary refinement adds localization precision.Source: §1 · p. 1
2. PROPOSED METHOD
The method first preserves multiple query-relevant temporal hypotheses, then selects and refines a candidate. Query-conditioned features feed continuous-time modeling and lightweight proposal-generation modules.
2.1. Framework Overview
- The framework preserves multiple query-relevant temporal hypotheses, selects the candidate most semantically compatible with the query using a frozen LALM, then refines its boundaries.This replaces direct regression of one interval from the full recording.Source: §2.1 · p. 2
2.2. Query-Conditioned Temporal Modeling
- FiLM modulates temporally projected audio features with query semantics before CfC processes them, so query information influences temporal modeling.The modulation parameters are shared across time steps, and CfC's time step represents the recording duration divided by T − 1.Source: §2.2 · p. 2
2.3. Temporal Candidate Generation
- Lightweight heads predict start, end, and relevance distributions from fused FiLM-conditioned features, CfC outputs, and audio–query matching signals.Parameter-free extraction grows regions from high-response seeds, merges nearby regions, and generates multi-scale temporal variants.Source: §2.3 · p. 2
- Candidate extraction is parameter-free, and its losses train temporal evidence rather than candidate-level ranking; the frozen LALM performs semantic candidate selection.The training objective includes boundary, relevance, and matching losses, with coefficients controlling their contributions.Source: §2.3 · pp. 2–3
2.5. Local Boundary Refinement
- The local refiner uses a window extending five seconds beyond each selected boundary and predicts bounded, gated boundary corrections.Training excludes candidates requiring a start or end correction larger than the correction range.Source: §2.5 · p. 3
3. EXPERIMENTS
Experiments train on the English TAG subset of LAT Chronicle and evaluate on English LAT-Bench recordings of up to 30 minutes. They use two frozen audio-language backbones and report the official grounding metrics.
3. EXPERIMENTS
- Training uses 3,737 English TAG query–segment pairs from LAT Chronicle with a 9:1 train–validation split, while evaluation uses English LAT-Bench recordings up to 30 minutes.Both sets are from Listening with Time.Source: §3 · p. 3
- Evaluation reports mIoU and Recall@IoU at thresholds 0.3, 0.5, and 0.7 under the official LAT-Bench protocol.Source: §3 · p. 3
- The experiments use frozen MOSS-Audio-8B-Instruct and Qwen3-Omni-30B-A3B-Instruct backbones, with grounding modules trained for three epochs on one NVIDIA A100 GPU.For Qwen3-Omni, the selected Top-1 candidate receives local refinement using a ±5-second window.Source: §3 · p. 3
4. RESULTS
The framework improves localization across MOSS-Audio and Qwen3-Omni, while candidate-pool analysis and ablations show benefits from diverse proposals, local refinement, and paired FiLM–CfC modeling.
4.1. Main Localization Results
- 38.52 mIoU and 40.61 R@0.5 with MOSS-Audio; Qwen3-Omni reaches 42.72 mIoU at ρg=0.65, with improvements reported across both backbones.The MOSS-Audio results outperform direct localization, sliding-window localization, and QA-SFT; Qwen3-Omni exceeds reported SW and QA-SFT results in mIoU.Source: §4.1 · pp. 3–4
- 43.45 mIoU, 45.54 R@0.5, and 21.13 R@0.7 on Qwen3-Omni with the union pool, versus 42.72, 42.49, and 19.95 for ρg=0.65.The authors suggest the two proposal settings provide complementary hypotheses for frozen LALM selection.Source: §4.1 · p. 4
- With the union pool, refinement raises mIoU from 43.45 to 44.13, R@0.5 from 45.54 to 46.95, and R@0.7 from 21.13 to 22.07.For ρg=0.65, refinement raises mIoU from 42.72 to 43.20 and R@0.7 from 19.95 to 21.13; the refined system remains below QA-SFT at R@0.7 but has higher overall mIoU without end-to-end backbone adaptation.Source: §4.1 · p. 4
4.2. Candidate Pool Analysis
- The oracle outperforms LALM-selected Top-1 predictions, and the remaining gap marks candidate selection as a bottleneck, especially at stricter IoU thresholds.Combining Qwen3-Omni proposal settings improves both Top-1 and oracle performance, suggesting more diverse hypotheses.Source: §4.2 · p. 4
4.3. Ablation Study
- FiLM+CfC achieves the highest mIoU on both backbones; adding FiLM to CfC raises mIoU from 37.21 to 38.52 on MOSS-Audio and 38.37 to 42.72 on Qwen3-Omni.Replacing CfC with a GRU reduces mIoU to 32.00 on MOSS-Audio and 31.29 on Qwen3-Omni, indicating the gains are not explained simply by recurrent modeling.Source: §4.3 · p. 4, Table 3 · p. 4
5. CONCLUSION
The framework combines FiLM, CfC, frozen LALM candidate selection, and lightweight boundary refinement for long-form audio grounding. Experiments show consistent improvements across two backbones, while ablations support complementary FiLM and CfC roles.
5. CONCLUSION
- On LAT-Bench, the framework improves across MOSS-Audio and Qwen3-Omni; its multi-candidate design preserves diverse hypotheses, and local refinement improves boundary precision after selection.Frozen backbones and lightweight trainable modules support single-GPU training, while the Top-1–oracle gap motivates stronger candidate selection.Source: §5 · p. 4