PaperlayerView exact revision on arXiv

Source-linked AI summary

VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding

Yongchao Xu, Bowen Ye, Jiefeng Gan, Junkai Ma, Wenzhao Li, Sen Tao, Yi Wei, Jiawei Liu

arXiv:2610.10183v1cs.CVcs.AI

TL;DR

Existing memory-based methods often adapt retrieval while keeping memory construction fixed, although long videos contain sparse task-relevant evidence. VideoEvolve alternates Agentic RL updates to co-evolve memory and retrieval, guided by bottleneck- and capability-aware feedback. It outperforms evaluated open-source baselines across five benchmarks.

  • Problem

    Existing memory-based methods often use fixed construction policies and separately optimize memory construction and retrieval, despite memory quality shaping evidence available for later reasoning.

    Source: §1 · p. 2

  • Method

    VideoEvolve alternates Agentic RL updates to co-evolve memory construction and retrieval, using BEF to target bottlenecks and CEF to prioritize underdeveloped yet learnable capabilities.

    Source: §1 · p. 2

  • Results

    VideoEvolve consistently outperforms evaluated state-of-the-art open-source baselines across five benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    VideoEvolve treats memory as a learnable interface between perception and reasoning, refined together with its use through task experience.

    Source: §5 · p. 10

    Evaluation focuses on offline multiple-choice video understanding and does not establish effectiveness for streaming video, open-ended dialogue, or direct audio-understanding tasks.

    Source: §A.10 · p. 31

Abstract

from arXiv · show

Long video understanding increasingly relies on external memory to organize massive visual streams into compact representations. However, most memory-based methods dynamically adapt how information is retrieved for different questions, while largely fixing what is remembered. This mismatch makes missing details costly to recover, whereas stored information is valuable only when it can be reliably retrieved. To address this issue, we propose VideoEvolve, a novel self-evolving framework that jointly evolves memory and retrieval for long video understanding. Specifically, starting from a coarse low-frame-rate overview, VideoEvolve couples a Memory Evolver for selective memory augmentation with a Retrieval Evolver for adaptive retrieval over the evolving memory. We then co-evolve the two Evolvers through alternating agentic reinforcement learning (Agentic RL), updating one while freezing the other. To steer this alternating evolution, Bottleneck-Aware Evolution Feedback (BEF) identifies whether the current bottleneck lies in memory or retrieval and directs optimization toward the more limiting side. Furthermore, VideoEvolve introduces Capability-Aware Evolution Feedback (CEF) to alleviate downstream feedback from over-specializing memory to a fixed set of training questions, shifting training toward underdeveloped yet learnable video capabilities. By integrating Agentic RL with BEF and CEF, VideoEvolve transforms downstream reasoning experience into transferable capability updates, providing a concrete path from static long-video systems toward experience-driven, self-improving multimodal intelligence. Extensive experiments on multiple long video understanding benchmarks demonstrate the effectiveness of VideoEvolve.

1 INTRODUCTION

Long-video evidence is sparse, and existing memory-based methods often fix memory construction while separately optimizing retrieval. VideoEvolve co-evolves both policies through alternating Agentic RL, guided by bottleneck- and capability-aware feedback; it outperforms evaluated open-source baselines across five benchmarks.

1 INTRODUCTION

  • Existing memory-based methods often use predefined construction policies that do not learn from downstream reasoning and separately optimize construction and retrieval.The paper frames these as limited memory adaptability and memory–retrieval misalignment.Source: §1 · p. 2
  • VideoEvolve alternates Agentic RL updates to co-evolve memory construction and retrieval from downstream reasoning feedback.The Memory Evolver selectively augments stored evidence, while the Retrieval Evolver searches evolving memory and can revisit raw video.Source: §1 · p. 2
  • BEF directs optimization toward the more limiting Evolver, while CEF shifts training toward underdeveloped yet learnable capabilities.CEF also adapts the training curriculum as the Evolvers improve.Source: §1 · p. 2
  • VideoEvolve consistently outperforms evaluated state-of-the-art open-source baselines on Video-MME, LongVideoBench, LVBench, MLVU, and MMVU.The introduction reports results across all five benchmarks.Source: §1 · p. 2

2 RELATED WORKS

Related work spans long-video modeling, selective video access, persistent memory, and self-evolving agents. VideoEvolve focuses on co-evolving memory construction and retrieval, a complementary direction to methods centered on inspecting source video or evolving reasoning strategies.

Long Video Understanding with MLLMs

  • Long-video methods use time-aware modeling, long-context scaling, visual compression, or selective agentic video access to address extended temporal contexts.Selective approaches include retrieval, multi-agent collaboration, and reinforcement-learning-based video inspection.Source: §2 · pp. 2–3
  • Video-revisiting methods focus on where and when to inspect source video, whereas VideoEvolve addresses what should be persistently remembered for retrieval.The paper presents these as complementary questions about selective video access.Source: §2 · p. 3

Memory-Based Long Video Understanding

  • Memory-based methods range from sparse or hierarchical memories to separately trained memorization models and learned tool controllers.VideoEvolve instead alternates optimization of memory construction and retrieval policies while keeping constructed memory reusable across questions.Source: §2 · p. 3

Agentic Reinforcement Learning and Self-Evolving Agents

  • Self-evolving agents improve reasoning strategies, skills, or experience use, while long-video agents also need to evolve what they remember and how they retrieve it.The paper connects these two choices: what is preserved determines what can be retrieved, and retrieval utility informs what is worth preserving.Source: §2 · p. 3

3 METHOD

VideoEvolve combines memory construction, retrieval, alternating Agentic RL, and feedback mechanisms that guide which Evolver and capabilities receive training. Its memory starts from a fixed coarse overview, while retrieval operates over evolving memory and may revisit raw video.

3.1 OVERVIEW

  • VideoEvolve has four components: Memory Evolver, Retrieval Evolver, alternating Agentic RL, and BEF/CEF evolution guidance.The feedback mechanisms diagnose bottlenecks and capability frontiers to guide the next cycle.Source: §3.1 · p. 3

3.2 MEMORY EVOLVER: LEARNING WHAT TO REMEMBER

  • The Memory Evolver selectively augments a fixed hierarchical Base Memory built from a coarse low-frame-rate overview.It observes memory state, construction history, and remaining budget, then chooses a Macro and inspection region, observation amount, and information focus; the Base Memory is rebuilt under each updated policy.Source: §3.2 · pp. 3–4

3.3 RETRIEVAL EVOLVER: LEARNING HOW TO RETRIEVE

  • The Retrieval Evolver searches evolving memory to gather information for answering questions and can selectively revisit raw video.Its actions include navigating memory, searching by meaning or time, and accessing stored visual references.Source: §3.3 · pp. 4–5

3.4 CO-EVOLVING MEMORY AND RETRIEVAL VIA AGENTIC RL

  • Alternating Agentic RL freezes one Evolver while optimizing the other, so retrieval outcomes inform memory construction and updated memory becomes the retrieval learner’s environment.The Memory Evolver’s reward encourages rescuing previously unanswered questions without sacrificing existing memory utility.Source: §3.4 · p. 5

3.5 EVOLUTION GUIDANCE VIA BEF AND CEF

  • BEF uses paired Memory-only and Revisit-enabled probe outcomes, including newly observed frames, to allocate the next evolution share toward the more limiting side.CEF prioritizes underdeveloped yet learnable capabilities and mixes frontier, exploration, and retention samples while maintaining broad coverage.Source: §3.5 · p. 6

3.6 TRAINING AND INFERENCE

  • Training initializes retrieval with two-stage SFT, then alternates memory and retrieval updates while BEF and CEF set evolution shares and capability focus.At inference, evolved memory is built once per unseen video for reuse across questions, retrieval may revisit raw video, and no optimization occurs.Source: §3.6 · p. 7

4 EXPERIMENT

VideoEvolve is evaluated across five long-video benchmarks and against both Video-MLLM and memory-based systems. Ablations examine co-evolution, feedback guidance, video revisiting, and how performance changes over successive cycles.

4.1 EXPERIMENTAL SETTING

  • The experiments report multiple-choice accuracy on five benchmarks, using 514 Video-MME-v2 videos for Agentic RL and a disjoint 128-video set for BEF/CEF diagnosis and scheduling.The training-based variant uses Qwen3.5-4B and Qwen3-VL-8B Evolvers, while the training-free variant uses Qwen3.8-27B for memory writing and retrieval.Source: §4.1 · p. 7

4.2 COMPARISONS WITH STATE-OF-THE-ART METHODS

  • VideoEvolve ranks first among listed open-source methods on all six metrics across five benchmarks, scoring 70.2% on LongVideoBench and 58.9% on LVBench, 9.8 and 17.4 percentage points above the strongest listed open-source baselines.Source: §4.2 · p. 7

4.3 ABLATION STUDIES

  • Co-evolution beats either single-Evolver variant on six of seven metrics, and combined BEF and CEF raise MMVU accuracy from 68.4% to 75.1%; revisiting adds 0.8–1.6 points across all seven metrics.Without revisiting, the system reaches 65.5% on Video-MME (Long) and 57.5% on LVBench.Source: §4.3 · pp. 8–9, Table 3 · p. 9

4.4 ANALYSIS OF CO-EVOLUTION DYNAMICS

  • By Cycle 3, Memory-only accuracy rises from 45.6% to 57.5%, surpassing the initial Revisit-enabled result by 7.2 points, while the gap narrows from 4.7 to 1.4 points.Revisit-enabled accuracy peaks at 58.9%; in Cycle 4, Memory-only improves further while Revisit-enabled falls to 58.6%.Source: Table 4 · p. 9, §4.4 · p. 9

5 CONCLUSION

VideoEvolve closes the learning loop between memory construction and retrieval through alternating Agentic RL. Across five benchmarks, it outperforms evaluated open-source Video-MLLM baselines, with trained and training-free variants leading their respective memory-based comparisons.

5 CONCLUSION

  • Alternating Agentic RL closes the learning loop: reasoning feedback guides what memory preserves, while updated memory shapes how retrieval learns to access evidence.BEF and CEF steer this loop toward current bottlenecks and underdeveloped yet learnable capabilities.Source: §5 · pp. 9–10
  • Across five benchmarks, VideoEvolve outperforms evaluated open-source Video-MLLM baselines, and its trained and training-free variants lead their respective memory-based comparisons.Ablations and cycle-wise analysis further support feedback-guided co-evolution.Source: §5 · p. 10
  • VideoEvolve reframes memory as a learnable interface between perception and reasoning, refined through task experience rather than kept as a fixed record.The paper presents this as a broader framing of memory construction and use.Source: §5 · p. 10

A APPENDIX

The appendix documents VideoEvolve’s implementation, training, evaluation, supplementary experiments, and limitations. It covers memory and retrieval, co-evolution procedures, and extended ablations.

A APPENDIX

  • Sections A.1 and A.2 describe memory construction and versioning, retrieval tools, and the interaction protocol.These sections provide the system and access-protocol details.Source: A APPENDIX · p. 14
  • Sections A.3 through A.7 cover data processing, cold-start training, alternating updates, quality-first optimization, BEF and CEF procedures, hyperparameters, runtime budgets, and prompts.The appendix includes CEF capability definitions, diagnosis, and targeted training.Source: A APPENDIX · p. 14
  • Sections A.8 through A.10 clarify evaluation and memory-based comparison protocols, report extended ablations, and discuss limitations and future work.Source: A APPENDIX · p. 14

A.1.1 FIXED HIERARCHICAL BASE MEMORY

The fixed Base Memory provides a reusable, hierarchical overview and local evidence for retrieval. It is built once per video, while question-agnostic policy-version Deltas supplement it without accumulating historical Deltas.

A.1.1 FIXED HIERARCHICAL BASE MEMORY

  • The Base is built once per video from uniform 0.5-fps sampling in intervals of at most 30 seconds, then remains unchanged during co-evolution.A policy version produces its own Delta from that Base; the published memory combines the Base with that version’s accepted records.Source: §A.1.1 · p. 14
  • Root, Super, and Macro provide video-level, grouped, and local navigation, while detailed local records remain accessible beneath the summaries.The Root is an entry point, not an exhaustive summary; upper-level summaries do not replace underlying Macro records.Source: §A.1.1 · p. 14
  • Macro records retain typed Events, Entities, and OCRText with temporal information, while entity identity is not guaranteed to track consistently across the whole video.Entities without localized visual timestamps are indexed over their containing Macro interval.Source: §A.1.1 · pp. 14–15
  • Hierarchy links connect Macros to Supers and Supers to the Root, but model-generated causal or progression relations are inferences rather than independently verified facts.The local Macro subgraph is structured records, not a fully populated entity-relation knowledge graph.Source: §A.1.1 · p. 15
  • CaptionUnits provide six-second temporal and image anchors, but raw subtitles are not exposed to Retrieval in the evaluated configurations.Subtitle information is available only through Writer-generated memory; “keyframe” refers to a stored uniform sample.Source: §A.1.1 · p. 15
  • Retrieval uses Macro overviews and nonempty local evidence as coarse and fine search entries, while Root and Super summaries provide navigation context rather than ordinary indexed documents.Policies can navigate or search directly without traversing every hierarchy level for each question.Source: §A.1.1 · p. 15

A.1.2 LEARNED AUGMENTATION AND VERSIONING

The learned Controller augments a fixed Base with accepted, source-linked evidence and makes the resulting Delta retrievable. Across training cycles, the Controller evolves from downstream feedback, while each version reconstructs its memory from the same Base.

A.1.2 LEARNED AUGMENTATION AND VERSIONING

  • “Where,” “how much,” and “what” mean a Macro and temporal region, a discrete frame level, and an extraction focus, respectively.Accepted records form the Delta; full records remain searchable while bounded digests expose content at Macro and Super levels.Source: §A.1.2 · p. 15
  • Within a construction trajectory, the Controller accumulates evidence without updating its parameters; across cycles, downstream-question feedback updates its observation decisions.Each updated Controller reconstructs memory from the same Base rather than appending historical Deltas.Source: §A.1.2 · pp. 15–16
  • The Memory Evolver selects where to observe, how many frames to request, and what information to extract; accepted supplemental evidence enriches the retrieval view while preserving the Base.The figure’s timestamps and evidence cards are schematic, not a measured trajectory or guarantee of complete coverage.Source: §A.1.2 · p. 15, Figure 3 · p. 16
  • At inference, the learned policy constructs question-agnostic memory without online policy optimization, and gradients do not pass through memory records or the frozen Writer.Source: §A.1.2 · p. 16
  • The supplemental frame budget is 32, 64, 96, or 128 frames according to video-duration bands, and accepted descriptions are limited to 4,096 tokens per video and 256 per step.Base construction, supplemental observation, and question-time revisiting use separate budgets; these frame counts do not measure all end-to-end compute.Source: §A.1.2 · p. 16
  • Accepted records require source-linked fields and pass structural checks, but there is no learned verifier or semantic audit to establish truth or detect every semantic conflict.Explicitly conflicting records are isolated rather than used to overwrite the Base.Source: §A.1.2 · p. 16

A.2 RETRIEVAL TOOLS AND INTERACTION PROTOCOL

The interaction protocol bounds raw-video access while distinguishing stored-frame inspection from newly acquired frames, and constrains tool actions and stopping decisions through runtime schemas.

A.2 RETRIEVAL TOOLS AND INTERACTION PROTOCOL

  • Revisit-enabled inference permits at most 64 raw-video frames per question and 16 per call; stored-frame inspection does not consume that allowance.Stored-frame inspection still consumes a tool call and contributes multimodal input to the policy.Source: §A.2 · p. 17
  • Tool temporal arguments use absolute video seconds, with availability and parameter bounds enforced by the runtime schema.The supplied table caption identifies the retrieval tool interface.Source: Table 5 · p. 17
  • Search fuses lexical BM25 and normalized dense-vector cosine similarity through weighted reciprocal-rank fusion, with hits treated as navigation cues rather than proof.The embedding service is held fixed during policy learning.Source: §A.2 · p. 17
  • Each interaction allows 12 exploration slots and, if needed, one final-only decision; abstention is available only at that final step.The final-only decision allows no further tool calls, and the same policy acquires evidence, reasons, and answers.Source: §A.2 · p. 17
  • Complete legal JSON actions enforce joint region, frame, and focus constraints; old, updated, and reference log probabilities use identical legal-token masks and sampling temperature.Tool outputs, images, and earlier observations are context rather than newly generated Actor targets.Source: §A.2 · pp. 17–18

A.3 DATA PROCESSING AND TWO-STAGE COLD START

The cold-start and data-processing pipeline trains retrieval with filtered tool-use demonstrations, then separates RL training videos from a diagnostic pool used for scheduling.

A.3 DATA PROCESSING AND TWO-STAGE COLD START

  • 4,317 accepted Stage-I trajectories from 4,205 videos expand into 18,932 decision samples, supervising only the final assistant response and not the Memory Evolver.Stage I establishes the tool-use protocol; 56 validated retry trajectories were added to the initial 4,261.Source: §A.3 · p. 18
  • Stage II retains tool-use demonstrations with correct final answers, schema-conformant original JSON, valid final references, and available image files.Zero-tool answers, repaired trajectories, and cases flagged for review are kept outside the strict pool; the criteria do not forbid zero-tool inference.Source: §A.3 · p. 18
  • Removing 190 near-duplicate trajectories reduces the strict pool from 102,922 to 102,732 trajectories across 9,995 videos.Near-duplicates share a video and complete tool path, with question-word Jaccard similarity of at least 0.8.Source: §A.3 · p. 18
  • All 349,717 Stage-II samples pass the recorded no-truncation check, but video-ID decontamination does not guarantee freedom from content-level duplicates, and neither SFT stage has an internal validation split.Ground-truth labels and filtering metadata are not appended to Actor prompts; prompts and history are masked from supervised loss.Source: §A.3 · p. 18
  • RL uses 514 training videos and 128 disjoint diagnostic videos; the diagnostic pool is reused for BEF/CEF scheduling, and training questions remain fixed.The diagnostic pool is a development probe rather than an independent final test set; the framework does not generate new questions online.Source: §A.3 · p. 18

A.4 ALTERNATING UPDATES AND QUALITY-FIRST OPTIMIZATION

Alternating updates rebuild memory before retrieval training, while quality-based eligibility and bounded resource tie-breaking keep task correctness primary.

A.4 ALTERNATING UPDATES AND QUALITY-FIRST OPTIMIZATION

  • After the Controller update, the Memory Evolver reconstructs a unified memory and index for every training video; retrieval then trains on the fixed rebuilt memory.Construction candidates start from the fixed Base, while the frozen Retrieval Evolver scores candidates and the current Enhanced-memory reference.Source: §A.4 · p. 18
  • With λreg = 1 and λinv = 0.2, valid construction rescue-minus-regression reward equals candidate accuracy minus reference accuracy, without an extra absolute-accuracy term.Resource costs are separate bounded proxies for selected resources, not measured latency or GPU hours.Source: §A.4 · pp. 18–19
  • Resource costs track supplemental Writer frames, accepted description tokens, retrieval tools, and newly acquired raw-video frames, with bounded values in [0, 1].The costs are proxies for selected resources, not measured latency or GPU hours.Source: §A.4 · p. 19
  • Retrieval efficiency preference requires a valid, correct answer; construction eligibility requires validity, utility at least the reference, and nonnegative task reward.Efficiency contributes at most 0.05 and cannot give positive preference to an all-incorrect or all-abstaining Retrieval group merely for saving tools.Source: §A.4 · p. 19
  • The 10^-8 stabilization constant applies to population standard deviation over complete trajectories and identifies numerically equal-quality peers for efficiency tie-breaking.This tolerance is distinct from the policy-clipping parameter and from the efficiency and KL coefficients.Source: §A.4 · p. 19
  • The clipped Actor objective averages tokens within each trajectory before averaging trajectories, and adds KL regularization with coefficient βKL = 0.02.Prompts, tool responses, image positions, and padding are excluded from the output loss mask; all-zero final-advantage groups are skipped.Source: §A.4 · p. 19

A.5.1 PAIRED DIAGNOSIS AND BEF BUDGETS

This appendix details the probes, feedback mechanisms, sampling, and protocols used to train and assess the evolving policies. It also covers evaluation caveats, ablations of memory construction and optimization objectives, Retrieval cold-start, and the framework’s limitations and possible extensions.

A.5.1 PAIRED DIAGNOSIS AND BEF BUDGETS

  • BEF uses paired diagnostics and actual raw-frame use to allocate construction versus retrieval demand, but these operational signals are not causal proof of the underlying failure.The probe evaluates the current memory–retrieval pair on 512 questions under two access modes and four seeds; equal BEF shares correspond to full training passes, not equal compute cost.Source: §A.5.1 · pp. 19–20

A.5.2 CAPABILITY TAXONOMY AND LABEL SEMANTICS

  • CEF uses nine Video-MME-v2 parent categories to label question demands, without exposing labels to the Actors or adding capability-specific prompts, heads, rewards, or questions.The taxonomy spans Frame-Only, Action & Motion, Order, Change, Temporal Reasoning, Complex Plot Comprehension, Video-Based Knowledge Acquisition, Social Behavior Analysis, and Physical World Reasoning.Source: §A.5.2 · pp. 20–21

A.5.3 FROM CAPABILITY DIAGNOSIS TO TARGETED POLICY UPDATES

  • Capability-conditioned BEF signals estimate separate construction and retrieval demands, which CEF smooths over time and converts into bounded sampling distributions with uniform coverage.Demand estimates retain prior values when observations are missing or supported by fewer than ten distinct probe videos; target distributions mix 30% uniform coverage across nine categories with 70% normalized smoothed demand.Source: §A.5.3 · p. 21
  • After an initial full pass, CEF samples capability-specific training instances within BEF’s group budgets, mixing frontier, exploration, and retention examples while regenerating on-policy rollouts.Construction videos are scored across four associated questions, while retrieval questions use the rebuilt fixed memory; probe examples guide sampling but never enter gradient training.Source: §A.5.3 · pp. 21–22

A.6 TRAINING HYPERPARAMETERS AND RUNTIME BUDGETS

  • Experiment settings list supplemental-frame budgets of 32/64/96/128 per video, 4,096 accepted description tokens, 12 exploration slots, and 64 new raw-video frames per question; alternating RL uses verl, FSDP, and vLLM.Sampling and likelihood recomputation retain the same xgrammar constraints, while the Writer and embedding service remain frozen.Source: §A.6 · p. 23

A.7 CONDENSED PROMPT TEMPLATES

  • The condensed prompts constrain the Controller to legal macro and observation choices, require the Writer to produce source-linked memory claims, and direct Retrieval to ground decisions in inspected evidence.The Controller cannot rewrite the fixed interface; the Writer marks conflicts and may return no records, while Retrieval must not infer temporal order from layout or list order.Source: §A.7 · p. 24

A.8 EVALUATION AND MEMORY-BASED COMPARISON PROTOCOLS

  • Final evaluation uses the same paired third-cycle checkpoints across five benchmarks after four co-evolution cycles, with official evaluation splits and benchmark-specific aggregation rules.Training-based results average four seeded inference runs with fixed memory and indexes; MLVU is the unweighted mean of task accuracies, and Video-MME Long and LongVideoBench Overall are distinct from their overall and Long-split scores.Source: §A.8 · p. 25
  • Benchmark inference uses greedy decoding, scores normalized answer labels by exact match, and permits raw-video revisiting while keeping memory-only access to stored keyframes and records.Only technical execution failures are rerun; abstentions and invalid terminations count as incorrect, and BEF/CEF are inactive during inference because their effects are carried by learned policies.Source: §A.8 · pp. 25–26
  • The training-free variant reports 76.7 on LVBench and 78.8 on Video-MME (Long), while the training-based variant reports 58.9 and 67.1, respectively.These are separate configurations, not a matched-backbone training ablation; differences across systems are descriptive because frame budgets, transcript access, and question subsets are not aligned.Source: §A.8 · p. 28

A.9 EXTENDED ABLATION RESULTS

  • The ablations distinguish policy evolution, memory augmentation, and question-time video revisiting: disabling revisiting preserves memory and keyframes, while Base memory only removes Delta but retains revisiting.Construction and objective variants use fixed Retrieval checkpoints or paired co-evolution checkpoints as specified; they should not be conflated with inference-access changes.Source: §A.9 · pp. 28–29
  • On Video-MME and LongVideoBench Long subsets, removing the regression penalty reduces accuracy by 2.2/3.1 points; quality-first beats quality-only by 0.9/1.2 points.The quality-first objective computes task-quality advantages first and applies bounded efficiency preference only among eligible equal-quality trajectories; accuracy results do not establish an efficiency advantage.Source: §A.9 · p. 29, Table 9 · p. 30
  • Learned augmentation achieves the highest accuracy on both Long subsets, while random and uniform augmentation yield smaller gains over Base memory only.All construction variants use the same fixed Retrieval checkpoint and allow raw-video revisiting; the shared augmentation budget is an allowance, not a guarantee of equal frame use or runtime.Source: §A.9 · p. 29, Table 9 · p. 30
  • LVBench accuracy is 58.9% with two-stage SFT, 32.8% without cold-start SFT, 54.2% without Stage I, and 40.6% without Stage II, all with subsequent co-evolution.The no-SFT configuration still undergoes co-evolution; the Memory Evolver and Writer are unchanged across these cold-start variants.Source: Table 11 · p. 31, §A.9 · p. 31

A.10 LIMITATIONS AND FUTURE WORK

  • Evaluation is limited to offline multiple-choice video understanding; finite budgets, frozen-Writer quality, ground-truth supervision, and fixed diagnostics constrain scope, while bottleneck estimates are operational proxies, not causal attributions.The study does not establish effectiveness for streaming video, open-ended dialogue, or direct audio understanding; training and inference also incur repeated rollouts, memory/index reconstruction, and retrieval computation.Source: §A.10 · p. 31
  • Future directions include streaming memory updates, richer audiovisual and open-ended evaluation, evidence verification, adaptive capability discovery, more efficient updates and distillation, and end-to-end cost measurement.The proposed extensions should be assessed across video durations, domains, and backbone models, with preprocessing, training, and question-time costs measured separately.Source: §A.10 · p. 31
Loading 2610.10183v1…