Source-linked AI summary
Coarse Indexing, Fine Evidence: Decoupling Temporal Granularity in Long-Video RAG
Zhe Jin, Zhimin Lin, Bin Zheng, Junhua Fang, Huihua Yang
TL;DR
Existing long-video RAG systems commonly couple graph indexing granularity to fixed video segmentation, although retrieval and downstream reasoning need not require the same temporal resolution. DAGC is a training-free coarse-to-fine design that merges visually redundant neighboring chunks for a compact index while recovering original chunks for fine-grained reasoning. Across benchmarks, it retains about 40–50% of graph nodes, achieves 1.3–1.7× end-to-end acceleration, and preserves approximately 99% of QA performance.
Problem
Existing graph-based video RAG systems typically inherit fixed, uniformly fine temporal granularity from segmentation, despite different resolution requirements for retrieval and downstream evidence reasoning.
Method
DAGC is a training-free approach that merges visually redundant neighboring chunks into a query-independent coarse graph index while preserving mappings to original chunks for fine-grained evidence recovery.
Results
Across three benchmarks, DAGC retains approximately 40–50% of original graph nodes, achieves 1.3–1.7× end-to-end wall-clock acceleration, and preserves about 99% of original QA performance.
Takeaways & Limitations
Long-video RAG can use coarse indexing and fine-grained evidence recovery rather than maintaining one temporal granularity throughout the pipeline.
Takeaways & Limitations
Adjacent visual similarity may miss semantic changes in visually stable regions, and DAGC is designed for QA rather than precise event segmentation.
Abstract
from arXiv · showhide
Graph-based retrieval-augmented generation (RAG) provides a scalable paradigm for long-video understanding, but existing systems typically inherit a fixed temporal granularity from video segmentation when constructing their retrieval index. We argue that this design unnecessarily couples indexing granularity with evidence granularity: coarse representations can often suffice for locating relevant temporal regions, while fine-grained evidence remains important for downstream reasoning. We propose \textbf{Density-Aware Graph Construction (DAGC)}, a training-free approach that decouples a query-independent coarse retrieval index from the original fine-grained evidence space. DAGC constructs a compact, density-adaptive graph index by merging visually redundant neighboring chunks, while preserving mappings to the original temporal units. Retrieved coarse regions are subsequently expanded back to the original chunk granularity for fine-grained evidence refinement and answer generation. Experiments on MLVU, VideoMME, and LongVideoBench show that DAGC retains only about 40--50\% of the original graph nodes and achieves $1.3$--$1.7\times$ end-to-end wall-clock acceleration while preserving approximately 99\% of the original QA performance. The gains transfer across different LVLM backbones and video RAG pipelines, suggesting that long-video RAG need not maintain the same temporal granularity for indexing and evidence reasoning.
1 INTRODUCTION
Long-video RAG must balance computational efficiency against temporal fidelity. DAGC addresses this by using coarse, density-adaptive indexing while preserving fine-grained evidence for reasoning, achieving substantial acceleration with minimal QA loss.
- Long-video understanding requires integrating information across extended temporal contexts, but dense visual representations are costly and aggressive compression can discard evidence.
- Existing graph-based video RAG systems typically inherit uniformly fine indexing resolution from fixed-length segmentation.
- DAGC is a training-free approach that decouples query-independent coarse indexing from the fine-grained evidence space used for downstream reasoning.
- DAGC merges neighboring chunks adaptively for compact indexing while retaining mappings to original chunks for fine-grained refinement and answer generation.
- 40–50% of original indexing units are retained, with 1.3–1.7× end-to-end wall-clock acceleration and about 99% of original QA performance preserved.
2 RELATED WORK
Prior long-video methods reduce visual burden, organize representations hierarchically, or structure video content as graphs. DAGC studies a complementary question: whether graph indexing must use the same temporal resolution as downstream evidence reasoning.
- Long-video VLM approaches reduce computational burden through sparse sampling, visual-token compression, or improved temporal memory.
- Hierarchical methods dynamically allocate representation capacity according to query requirements, but generally optimize information amount or resolution for downstream inference.
- Graph-based video RAG organizes entities, events, and temporal relationships, yet its index granularity is usually inherited from predefined segmentation.
- Video segmentation seeks semantically coherent partitions, whereas DAGC treats temporal granularity as a retrieval-system design variable.
- DAGC complements graph extraction and retrieval research by coarsening indexing while mapping retrieved regions back to fine-grained temporal evidence.
3 DENSITY-AWARE GRAPH CONSTRUCTION WITH GRANULARITY DECOUPLING
DAGC builds a density-adaptive coarse graph for candidate localization, then recovers original chunks for question-specific evidence refinement. It merges redundant neighbors while bounding coarsening and preserving fixed per-node visual input budgets.
- DAGC explicitly separates efficient candidate localization from fine-grained evidence reasoning through a query-independent coarse graph and direct mappings to original chunks.
- 3.1 DENSITY-ADAPTIVE INDEX COARSENING: Adjacent visual similarity serves as a lightweight proxy for local redundancy: high similarity permits merging, while low similarity preserves finer indexing.
- 3.1 DENSITY-ADAPTIVE INDEX COARSENING: The visual-similarity threshold τv and maximum merging span W govern greedy merging, while Im records each super-node’s original chunk indices.
- 3.1 DENSITY-ADAPTIVE INDEX COARSENING: M ≤ N coarse units replace N original chunks, and merged units are uniformly resampled to the original K-frame budget for semantic extraction.
- 3.1 DENSITY-ADAPTIVE INDEX COARSENING: Graph construction runs on the M coarse units, reducing query-independent LVLM extraction calls while retaining original chunks for recovery.
- 3.3 COARSE-TO-FINE EVIDENCE RETRIEVAL: Coarse retrieval ranks and expands candidate regions, projects them through stored mappings, and reranks original-granularity chunks for answer generation.
- 3.3 COARSE-TO-FINE EVIDENCE RETRIEVAL: Unlike conventional representation compression, DAGC compresses search representations without forcing downstream reasoning to use coarse temporal evidence.
4 EXPERIMENTS
DAGC is evaluated across three long-video benchmarks, multiple LVLM backbones, and two video RAG pipelines. It substantially compresses graph indices and accelerates end-to-end inference while largely preserving accuracy.
- Experimental Setup: DAGC is evaluated on MLVU, VideoMME, and LongVideoBench with Qwen2.5-VL-7B, Qwen2.5-VL-3B, and Qwen2-VL-7B backbones.Transferability is additionally tested with InternVL3.5-8B and VideoRAG.
- Main Results: 45%–47% retained graph nodes yield 1.3×–1.7× end-to-end wall-clock acceleration while average accuracy drops only 0.7–0.8 percentage points.This corresponds to approximately 99% performance retention across the three Qwen-family backbones.
- Main Results: DAGC’s compact index preserves largely stable overall performance despite removing more than half of the original indexing units.The results attribute this trade-off to reducing temporal redundancy while recovering fine-grained evidence for downstream reasoning.
- Efficiency Analysis: Offline time decreases from 24.14 to 10.73 seconds per minute of video, while online time increases only slightly from 4.12 to 4.24 seconds.First-query time consequently falls from 28.26 to 14.97 seconds per minute, a 47.0% reduction.
- Generalization: DAGC retains 40.39%–48.34% of VideoRAG indexing units and achieves 1.377×–1.695× offline speedup with over 1.30× wall-clock acceleration.Accuracy improves on LongVideoBench and VideoMME but decreases on MLVU.
- Generalization: The efficiency benefit transfers beyond one LVLM family and one video RAG pipeline.The experiments combine InternVL3.5-8B with Vgent and DAGC with VideoRAG.
5 ANALYSIS AND DISCUSSION
The analyses test whether coarse indexing can coexist with fine-grained evidence recovery, whether content-aware compression matters, and whether semantic event boundaries are a better partitioning basis. Results support coarse-to-fine recovery and density-aware merging, while event-boundary constraints are inconsistent.
- Analysis Scope: The analyses distinguish granularity decoupling, density-aware adaptation, and temporal partition choice as separate design factors in DAGC.These questions examine whether indexing and evidence reasoning need the same temporal resolution and whether local redundancy should guide merging.
- Indexing and Evidence Granularity: 60.6 accuracy from using compressed super-nodes directly shows that coarse representations alone are insufficient for final reasoning.Recovering and reranking original chunks substantially restores performance, and temporal expansion further improves retrieval completeness.
- Indexing and Evidence Granularity: 63.1 accuracy with the complete coarse-to-fine pipeline approaches the 63.3 original fine-grained Vgent baseline while retaining only 47% of graph nodes.This supports separating retrieval indexing granularity from downstream evidence granularity.
- Density-Aware Compression Strategy: At an identical 47% retained-node ratio, DAGC improves over Uniform Merge and Random Merge by 0.79 and 1.54 percentage points, respectively.The comparison evaluates whether compression benefits arise from content-aware density adaptation rather than node reduction alone.
- Temporal Partition Choice: Explicit event-boundary constraints improve temporal ordering but reduce Needle and Count accuracy, producing a small overall decrease of 0.24 percentage points.The result suggests perceptual event transitions do not always match the evidence granularity needed for long-video question answering.
6 CONCLUSION
DAGC decouples coarse graph indexing from fine-grained evidence recovery for efficient long-video graph RAG. Across benchmarks and configurations, it compresses indexing substantially, accelerates execution, and preserves QA performance while event boundaries do not consistently improve results.
- Conclusion: DAGC is a training-free method that merges visually redundant neighboring chunks into a density-adaptive coarse graph index while preserving mappings to original temporal units.Retrieved regions are mapped back to fine-grained evidence for downstream reasoning.
- Conclusion: Across three long-video benchmarks, DAGC retains approximately 40–50% of graph nodes, achieves 1.3–1.7× end-to-end wall-clock acceleration, and preserves about 99% of original QA performance.The efficiency benefit also transfers across LVLM backbones and video RAG pipelines.
- Conclusion: Explicit event boundaries do not consistently improve downstream performance, supporting coarse indexing combined with fine-grained evidence recovery rather than one fixed temporal partition.This design reduces redundant computation without permanently coarsening the evidence available after retrieval.
A.1 DETAILED RESULTS ON MLVU
On MLVU, DAGC largely preserves Vgent’s accuracy after graph compression, with only small category-dependent fluctuations across evaluated backbones. VideoMME and LongVideoBench results further show that compression effects vary with video duration and question category.
- 73.4 versus 73.3 overall accuracy is achieved by DAGC and Vgent with Qwen2.5-VL-7B on MLVU.
- 72.1 versus 71.7 overall accuracy is achieved by DAGC and Vgent with Qwen2-VL-7B on MLVU.
- 0.1 percentage points separate DAGC and Vgent for Qwen2.5-VL-3B on MLVU.
- DAGC retains performance relatively well on short and medium VideoMME videos, while long videos are more sensitive to graph compression.
- DAGC retains approximately half of the original graph nodes on LongVideoBench, with small category-dependent fluctuations including improvements and moderate degradations.
A.4 COMPRESSION STRATEGY ANALYSIS
Under the same 47% retained-node budget, DAGC outperforms content-agnostic merging strategies on LongVideoBench, indicating that local visual similarity contributes beyond generic node reduction.
- 63.07 accuracy is achieved by DAGC under the same retained-node budget used for Uniform Merge and Random Merge.
- 0.79 percentage points separate DAGC from Uniform Merge, while 1.54 percentage points separate DAGC from Random Merge.
- Local visual similarity provides the criterion used by DAGC to identify redundant adjacent regions.
A.5 ADDITIONAL EVENT-BOUNDARY ANALYSIS
Additional analysis finds that richer event-boundary signals do not improve QA consistently or offer a favorable efficiency trade-off. DAGC instead targets redundancy reduction while retaining fine-grained evidence for retrieval and reasoning.
- 82.25% to 81.41% is the accuracy change after incorporating richer boundary signals, with no statistically significant difference.The reported paired test gives p = 0.25.
- Nearly doubled graph-construction time and substantially increased total wall-clock time accompany the richer boundary variant.
- Event segmentation seeks coherent temporal partitions, whereas DAGC seeks to reduce redundant graph-construction units while preserving question-relevant evidence.
- Event-based graph construction adds full-video boundary-inference cost and may create more graph nodes and retrieval candidates.
- DAGC maps retrieved super-nodes back to fine-grained evidence for reranking and reasoning.
A.6 COMPONENT ABLATION
Component ablations show that original-chunk recovery and reranking are central to compressed-pipeline accuracy, while super-node compression supplies the main efficiency benefit. Parameter studies expose a trade-off between retained nodes and preservation of fine-grained temporal evidence.
- Component ablation: 63.1 overall accuracy is achieved by the complete DAGC pipeline among compressed variants.
- Component ablation: 60.6 overall accuracy results when original-chunk recovery and reranking are removed, the largest degradation among compressed variants.
- Component ablation: Super-node compression is primarily responsible for the efficiency gain, while temporal expansion contributes to final performance.
- Parameter sensitivity: 63.1 accuracy with 47% retained nodes is achieved by the default setting τv = 0.95, W = 3, top-ks = 12.
- Parameter sensitivity: 35% to 47% retained nodes and 62.5 to 63.1 accuracy result when τv increases from 0.80 to 0.95.
- Parameter sensitivity: 41%, 36%, and 34% of original nodes are retained for W = 4, 6, and 7, respectively, while accuracy decreases to 62.5 at W = 7.
B LIMITATION
DAGC has several scope and robustness limitations: visual similarity may miss semantic changes, coarsening must match video information density, and super-nodes are not guaranteed to represent complete events.
- Visual similarity may miss semantic changes in visually stable regions, including evolving dialogue and subtle object-state transitions.
- Information-dense videos may require more conservative coarsening thresholds or shorter merging spans.
- DAGC targets efficient long-video QA rather than precise event segmentation, so super-nodes may not correspond to complete semantic events.
- Broader evaluation across additional graph structures and retrieval frameworks remains future work.