PaperlayerView exact revision on arXiv

Source-linked AI summary

STEER: Structured Event Evidence for Video Reasoning via Multi-Objective Reinforcement Learning

Zinuo Li, Yongxin Guo, Jun Liu, Jiawei Zhan, Xi Jiang, Chengjie Wang, Mohammed Bennamoun, Farid Boussaid, Feng Zheng, Qiuhong Ke

arXiv:2604.04415v3cs.CL

TL;DR

Video-LLMs can obscure temporal cues by applying unstructured CoT directly to redundant visual inputs. The paper introduces structured event evidence, progressive training, and P-FAB for multi-objective RL, with STEER-4B reportedly rivaling 7B-scale baselines using half the input frames.

  • Problem

    Unstructured CoT buries critical temporal cues in verbose narration and can neglect event dependencies, while RL faces conflicting length and accuracy objectives plus sparse hard-sample rewards.

  • Method

    The paper constructs compact time-ordered Structured Event Evidence, trains it through a four-stage curriculum, and uses P-FAB to balance RL objectives along the Pareto frontier.

  • Results

    STEER-4B rivals 7B-scale baselines on temporal grounding and general video understanding benchmarks with half the input frames.

  • Takeaways & Limitations

    Structured evidence anchors inference to explicit, verifiable observations, while P-FAB is presented as task-agnostic and potentially applicable to other multi-objective RL settings.

  • Takeaways & Limitations

    When accuracy rewards are weighted heavily, the model can produce increasingly verbose evidence because the length penalty may provide insufficient corrective signal.

Abstract

from arXiv · show

Human understanding of video dynamics relies on forming structured representations of entities, actions, and temporal relations before engaging in abstract reasoning. In contrast, existing Video-LLMs apply unstructured chain-of-thought directly to raw visual tokens, where critical temporal cues are buried in verbose narration and event-level structure is largely overlooked. We propose Structured Event Evidence, which represents a video as a compact, time-ordered event schema capturing salient events with key attributes and inter-event temporal dependencies, enabling evidence-grounded reasoning through a constrained verification process. This design promotes concise, interpretable reasoning while reducing the drift typical of unconstrained chain-of-thought. To train models under this paradigm, we introduce STEER-60K, a dataset with a four-stage progressive pipeline: evidence training, format warm-start, thinking warm-start, and RL post-training. During RL, CoT length and task accuracy often conflict while rewards for hard samples are too sparse, causing the policy to neglect challenging instances. We formulate this as a multi-objective Pareto optimality problem and propose Pareto-Frontier guided Advantage Balancing (P-FAB), which dynamically resolves reward conflicts and identifies balanced optimization directions along the Pareto frontier. The resulting model STEER-4B rivals 7B-scale baselines on video understanding tasks with half the input frames Code and data will be released.

1 Introduction

The paper argues that unstructured video CoT obscures temporal evidence and proposes structured event representations to ground concise reasoning. It introduces STEER-60K and P-FAB to train and optimize this framework.

  • Motivation: Unstructured CoT can bury pivotal visual cues in verbose narration, drift during reasoning, and reduce videos to isolated frame retrieval.This conflicts with human understanding, which first organizes entities, actions, and event boundaries.
  • Contribution: P-FAB formulates conflicting CoT length, task accuracy, and sparse hard-sample rewards as a Pareto optimality problem during RL.It dynamically resolves conflicts rather than collapsing objectives through fixed scalar weighting.
  • Contribution: Structured Event Evidence represents videos as time-ordered, attribute-rich segments with temporal dependencies for evidence-grounded reasoning.The representation is intended to provide an explicit foundation before abstract reasoning begins.
  • Contribution: STEER-60K contains 60K annotations over 32K videos and supports a progressive curriculum from evidence extraction to structured reasoning.The staged design is intended to reduce hallucinated evidence and degraded reasoning.
  • Results: STEER-4B rivals 7B-scale baselines on temporal grounding and general video understanding benchmarks with half the input frames.This is the paper’s reported model-level comparison.

2 Related Work

Prior video-language models often rely on frame-level processing or unconstrained multimodal CoT, leaving temporal and event-level structure insufficiently modeled. The paper’s approach addresses this gap by structuring evidence before reasoning.

  • Video understanding LVLMs: Early video LLMs treated videos as sequences of static frames and compressed tokens to fit spatio-temporal signals into limited language-model budgets.This frame-level approach overlooks temporal grounding and event-level reasoning challenges.
  • Video understanding LVLMs: Recent video models add time-aware architectures, memory, or tools to capture temporal dynamics beyond frame-level processing.The passage presents these as responses to limitations of earlier approaches.
  • Reasoning and chain-of-thought: Unconstrained vision CoT can become verbose, drift off-topic, or hallucinate unsupported details when visual inputs are redundant and temporally underspecified.Fine-grained temporal and event-level cues can be missed when intermediate reasoning lacks grounding.
  • Reinforcement learning in LVLMs: Existing LVLM reinforcement-learning methods commonly apply GRPO with group-wise relative comparisons and rule-based rewards to video tasks.The cited tasks include reasoning, temporal grounding, and counting.

3 Methodology

The methodology separates evidence construction from structured reasoning through progressive training, then uses P-FAB to balance conflicting RL objectives. The framework grounds search, verification, and answer generation in temporally organized evidence.

  • Data curation: STEER-60K is curated from human-annotated temporal-grounding datasets using event-density filtering, gap filling, and alternating generator-judge LLMs.The generated materials include structured evidence and evidence-grounded thinking traces.
  • Structured evidence: The framework first constructs compact Structured Event Evidence containing salient video cues before applying structured reasoning.Evidence is intended to provide precise, query-relevant clues for later inference.
  • Evidence-grounded reasoning: Structured reasoning proceeds through Global Search & Localization, Evidence Verification, and Final Answering.The process narrows relevant time windows, checks temporal coherence, and consolidates verified evidence.
  • Progressive training: A four-stage pipeline comprises Evidence Training, Format Warm-Start, Thinking Warm-Start, and RL-based Post-training.The stages provide distinct supervision for evidence, reasoning format, structured thinking, and objective balancing.
  • RL challenges: CoT length and task accuracy conflict, while sparse hard-sample rewards can steer RL toward easier objectives.Without length constraints, evidence may become excessively fine-grained and redundant.
  • P-FAB: P-FAB replaces fixed-weight scalar advantages with dynamically balanced Pareto optimization across reward objectives.It uses minimum-norm combinations in standardized reward space to amplify sparse objectives and down-weight saturated ones.
  • P-FAB: Static reward weights can favor verbose evidence when accuracy is heavily weighted, leaving length penalties insufficient as conciseness becomes the bottleneck.This illustrates the trade-off motivating dynamic objective balancing.

4 Experiments

The experiments show that structured evidence, structured thinking, progressive training, and P-FAB each contribute to video understanding, while STEER-4B achieves gains with fewer input frames.

  • Ablation Studies: Removing either evidence extraction or structured thinking severely degrades performance across temporal grounding and general understanding benchmarks.Evidence anchors reasoning, while thinking connects extracted evidence to the final answer.
  • Ablation Studies: Removing Stage 1.5 causes notable drops, while merging all SFT stages into Unified SFT causes severe degradation across metrics.The format warm-start bridges evidence extraction and complex reasoning, and the progressive curriculum supports stable learning.
  • Optimization Strategy: P-FAB generally outperforms standard GRPO on precise temporal localization and general understanding benchmarks.Its advantage generally widens as group size increases from 4 to 8 because larger groups expose multi-objective conflicts more clearly.
  • Comparisons with Existing Video-LLMs: Structured evidence improves temporal-boundary precision across lenient and strict IoU thresholds.Explicit event timestamps provide anchors for aligning predicted activity intervals with ground-truth boundaries.
  • Comparisons with Existing Video-LLMs: Unstructured thinking underperforms the Qwen3-VL-4B-Instruct counterpart across nearly all benchmarks, whereas evidence-first reasoning directs tokens toward grounded temporal inference.The paper attributes this pattern to verbose self-corrections, repetitive narration, and speculative tangents in unstructured CoT.
  • Comparisons with Existing Video-LLMs: STEER-4B uses 1 fps while most competing models use 2 fps with more frames, yet the model achieves stronger temporal signals from fewer inputs.The comparison is intended to show the effectiveness of structured abstraction under reduced frame sampling.

5 Conclusion

The paper presents STEER-4B, which constructs verifiable Structured Event Evidence before reasoning, supported by STEER-60K and P-FAB. It positions structured evidence as a direction for scalable video reasoning and identifies broader applications as future work.

  • STEER-4B constructs Structured Event Evidence before reasoning, anchoring inference to explicit, verifiable observations.
  • STEER-60K enables structured reasoning, while P-FAB dynamically resolves objective conflicts along the Pareto frontier during RL.
  • Structured evidence is presented as extensible to cross-modal evidence fusion, and P-FAB’s broader use in multi-objective RL is left for future work.

A Implementation Details

Implementation uses stage-specific training and evaluation settings for a Qwen3-VL-4B-Instruct base, with full-parameter RL updates and controlled video sampling. The paper also identifies corpus scale and model size as limitations.

  • Training Configuration: Qwen3-VL-4B-Instruct is the base model, with LoRA fine-tuning in Stages 1–2 and full-parameter updates during RL Stage 3.Stage 3 uses four reward objectives: Format, Linear IoU, Multi-choice Accuracy, and Length.
  • Evaluation Configuration: Evaluation re-runs Qwen3-series baselines at 1 fps and 256 maximum frames, while other models typically use official results at 2 fps and 2048 maximum frames.
  • Limitations: The training corpus is limited in scale, and the main experiments use a 4B-parameter model because of computational constraints.Expanding annotation volume and diversity and scaling to larger architectures remain future work.
  • Evaluation Scope: The evaluation covers Video Temporal Grounding and General Video Understanding benchmarks.

B.1 Video Temporal Grounding Benchmarks

The temporal-grounding benchmarks assess localization of queried moments in untrimmed videos, spanning re-annotated datasets with stricter boundaries and a large open-domain benchmark.

  • Temporal grounding benchmarks evaluate precise localization of moments within untrimmed videos from textual queries.
  • Charades-TimeLens contains 1,313 videos and 3,363 annotations focused on dense indoor activities with complex temporal overlaps.It rewrites 66.3% of low-quality Charades-STA annotations.
  • ActivityNet-TimeLens contains 1,455 videos and 4,500 annotations with stricter start and end times for temporal queries.It rewrites 69.7% of low-quality ActivityNet-Captions annotations.
  • ActivityNet-Captions contains 4,885 open-domain YouTube videos and 17,031 annotations covering complex human activities.Performance is evaluated using Recall@1 at IoU thresholds of 0.3, 0.5, and 0.7.

B.2 General Video Understanding Benchmarks

STEER-60K is built from temporally annotated video data through filtering, structured evidence generation, staged reasoning-trace production, and quality control. Its statistics show a diverse, event-rich corpus spanning long-form videos and multiple semantic domains.

  • Evaluation benchmarks: VideoMME, MLVU, ETBench, and NExT-GQA evaluate long-context understanding, temporal reasoning, event-level deductions, and grounded causal or temporal question answering.These benchmarks collectively assess holistic video understanding across diverse tasks and domains.
  • Dataset construction: The curation pipeline filters event-dense videos, fills temporal gaps, generates structured evidence, and produces evidence-grounded thinking traces with automated and human quality checks.Low-quality samples are rejected and iteratively refined before the final dataset is formed.
  • Dataset construction: STEER-60K is curated from high-quality human-annotated temporal grounding datasets using preserved timestamps and regenerated structured descriptions.The corpus contains 32,049 video samples, with original unstructured captions discarded during regeneration.
  • Dataset statistics: Tutorials and Sports dominate the 18 semantic categories, while Vlog, Daily Life, Cooking, and Travel broaden coverage across structured and open-ended activities.The topic mix emphasizes event-rich content with clear temporal dependencies while retaining semantic diversity.
  • Dataset statistics: 109.4s mean and 123.6s median video durations characterize the dataset, whose videos are concentrated around the 1–3-minute range.The duration distribution is intended to provide temporal context for event unfolding while remaining efficient to process.

D.3 RL Training Data Details

The RL training data emphasizes temporal grounding while retaining spatial and reasoning tasks, and the training prompts progressively teach evidence extraction, formatting, and grounded reasoning. P-FAB’s optimization procedure is supported by convergence to Pareto-stationary solutions.

  • RL data composition: 53% of RL training data is Temporal Grounding, while Spatial VQA and Reasoning VQA contribute 21% and 20%, respectively.Temporal grounding supplies objective TIoU-based rewards, while VQA tasks support visual-relationship interpretation and structured deductions.
  • RL data composition: The RL corpus is dominated by action-rich categories, particularly Tutorials with 7,043 samples and Sports with 5,746 samples.These domains provide explicit procedures and goal-oriented behaviors suitable for temporal reasoning optimization.
  • RL data composition: RL videos have a mean duration of 102.1s and median of 98.5s, with most samples concentrated in the 50–250s range.The duration range is selected to provide temporal context while remaining computationally tractable for iterative RL sampling.
  • Progressive training: Three prompt stages teach dense evidence extraction, strict XML-like formatting, and evidence-grounded reasoning before RL post-training.The format warm-up inserts a placeholder thinking block, while the final stage requires global search, evidence verification, and alignment.
  • Reward design: The reward design combines format compliance, temporal IoU, multi-choice accuracy, and length objectives to guide structured reasoning.Format rewards check the evidence-thinking-answering schema and required reasoning steps.
  • Theoretical analysis: P-FAB’s exact line search monotonically decreases the convex objective and converges to a global optimum corresponding to a Pareto-stationary point.Under standardized rewards, the method yields Pareto stationarity in parameter space without further assumptions on policy-gradient structure.

G.2 Pareto Stationarity of Standardized P-FAB

Standardized P-FAB links balanced reward-space optimization to Pareto stationarity in parameter space, while experiments show scalable, efficient, and more concise reasoning.

  • Pareto stationarity: P-FAB reaches a Pareto-stationary point when standardized reward-space updates are perfectly balanced.The theorem requires no assumption on the gradient structure.
  • Objective balancing: P-FAB assigns larger effective weights to lower-variance, harder-to-satisfy objectives.The effective weighting is given as w_m = α*_m/σ_q,m.
  • Convergence: As the standardized update norm decreases, the combined policy gradient approaches zero and optimization becomes scale-invariant.This prevents high-variance objectives from dominating sparse but critical signals.
  • Benchmark evaluation: 70.64% on TempCompass and 63.48% on EventBench demonstrate STEER-4B performance on temporal and event-level reasoning benchmarks.TempCompass measures temporal perception, while EventBench includes temporal and causal reasoning subtasks.
  • Scalability: P-FAB consistently outperforms GRPO at the 8B scale, indicating that the balancing approach extends beyond the 4B model.The comparison uses Qwen3-VL-8B with LoRA fine-tuning.
  • Efficiency: 0.0013% of total step time is added by P-FAB weight computation, while output lengths decrease on temporal tasks.CTL outputs decrease from 4288 to 849 tokens, and TempCompass outputs from 1388 to 436 tokens.
  • Evidence quality: STEER-4B produces higher-quality structured evidence than the base Instruct model across F1, BLEU-4, ROUGE-L, and CIDEr.The evaluation compares generated evidence against gold-standard annotations.

H.7 Training Data Overlap Analysis

Overlap analyses excluding Charades and ActivityNet training videos show only minor performance drops, supporting generalization beyond the training sources.

  • Charades exclusion: Charades-TimeLens drops only slightly after excluding Charades training videos.R1@0.3 changes from 57.1 to 56.7, R1@0.5 from 40.4 to 39.8, and R1@0.7 from 21.6 to 21.1.
  • ActivityNet exclusion: ActivityNet shows a larger performance drop after excluding its training videos because it constitutes the largest portion of the training corpus.The supplied passage states that performance nevertheless remains above all open-source baselines.
  • Generalization: Both exclusion experiments show only minor drops, indicating that temporal grounding generalizes beyond the training data sources.The analysis is designed to test whether gains reflect data memorization.

H.8 Reasoning Quality Evaluation

Reasoning quality is evaluated with Gemini-2.5-Pro across four dimensions, while the curation prompts enforce structured evidence format, timestamp alignment, and logical dependencies.

  • Reasoning quality evaluation: Gemini-2.5-Pro scores Overall quality, Hallucination avoidance, Logicality, and Interpretability on a 1–5 scale.STEER-4B matches or outperforms the Thinking baseline on all dimensions, with a +1.0 gain in Interpretability.
  • Quality judging: The Quality Judge validates model outputs against required formatting, target timestamps, and person-action dependency rules.Outputs must use evidence tags, include every target timestamp exactly once, and preserve logical person-action relationships.
  • Evidence format: Structured event evidence requires Theme and Event Evidence headers plus six tags on every event line.The required tags cover person, human_action, scene, object, camera, and event_caption.
  • Timestamp alignment: Timestamp alignment requires exactly one line for each supplied timestamp, with no missing or hallucinated timestamps.The original timestamp format must be preserved.
  • Logical consistency: Logical consistency checks require human_action to be None whenever person is None.When a person is present, human_action should typically describe an action.
  • Reasoning generation: Thinking Generation uses extracted video evidence to produce training samples demonstrating evidence-based reasoning.The pipeline also includes prompts for evidence generation and progressive training stages.
Loading 2604.04415v3…