STEER: Structured Event Evidence for Video Reasoning via Multi-Objective Reinforcement Learning

• Unstructured CoT can bury pivotal visual cues in verbose narration, drift from evidence, and reduce video temporality to isolated-frame retrieval rather than event dependencies.Source: §1 · p. 2
• P-FAB treats RL post-training as Pareto optimization, dynamically balancing conflicting CoT-length and task-accuracy objectives while addressing sparse rewards for hard samples.Source: §1 · p. 2
• STEER-4B rivals 7B-scale baselines on temporal grounding and general video understanding benchmarks with half the input frames.Source: §1 · p. 2
• Structured Event Evidence represents videos as time-ordered, attribute-rich segments with salient events and temporal dependencies, grounding concise reasoning in evidence.Source: §1 · p. 2
• STEER-60K provides 60K annotations over 32K videos and a staged pipeline that teaches evidence extraction before structured reasoning to reduce hallucinated evidence and degraded reasoning.Source: §1 · p. 2

Video understanding LVLMs

• Video LLMs evolved from static-frame processing toward time-aware architectures, memory, and stronger data and benchmarks, yet often still rely on sparse-frame heuristics rather than event dynamics.STEER instead learns an explicit structured prior summarizing key events and their temporal dependencies before free-form reasoning.Source: §2 · p. 2

Reasoning and chain-of-thought in LVLMs

• Unconstrained vision chain-of-thought can become verbose, drift off-topic, or hallucinate unsupported details; STEER constrains it with a schema of key events, entities, and temporal dependencies.Prior approaches use tool-augmented verification [49] or multiple reasoning paths [41].Source: §2 · p. 3

Reinforcement learning in LVLMs

• Existing LVLM reinforcement-learning methods apply GRPO [12] [43] to video tasks but overlook trade-offs among structural constraints, reasoning, accuracy, and computational budgets; STEER uses MGDA to approximate the Pareto frontier.RLHF typically optimizes scalar rewards with PPO [36], while GRPO [37] uses group-wise relative comparisons and reduces reliance on value-function estimation.Source: §2 · p. 3

3.1 Overview

• STEER addresses an optimization barrier by combining curated evidence and thinking traces, progressive training that separates evidence learning from reasoning, and P-FAB for multi-objective RL.Directly learning accurate evidence and complex reasoning together can overwhelm the model, producing hallucinated evidence or degraded reasoning structures.Source: §3.1 · p. 3

3.2.1 Motivation

• Existing video models often lack structured reasoning [43] [2], so STEER requires event evidence before reasoning to expose clues and temporal structure.Source: §3.2.1 · p. 4

3.2.2 Data Construction Pipeline

• STEER-60K uses event-dense human-annotated videos, temporal gap filling, and alternating generator-judge roles for structured evidence and evidence-grounded thinking traces.Human evaluation sampled 50% of annotations: the three annotators’ average scores were 8.5, 7.9, and 8.2, and 28.3% of batches were rejected and regenerated.Source: §3.2.2 · p. 4
• Structured Event Evidence records major events at roughly 10–30-second intervals with salient people, actions, scene context, and camera position in chronological order.Source: §3.2.2 · p. 4
• Reasoning narrows query-relevant time windows, verifies temporal coherence with surrounding events and mandatory visual evidence, then checks global consistency against observed activity.The process comprises Global Search & Localization, Evidence Verification, and Final Alignment.Source: §3.2.2 · p. 4

3.3.1 Stage Design

• Four stages progress from evidence training through format and thinking warm-starts to RL post-training.Evidence and reasoning instruction tuning use distinct supervision; the intermediate stage pairs evidence output with a required thinking format to reduce malformed reasoning before P-FAB-based RL.Source: §3.3.1 · pp. 4–5

3.3.2 Pareto-Frontier guided Advantage Balancing

• CoT length conflicts with task accuracy, sparse hard-sample rewards favor easier updates, and scalar GRPO advantages [37] obscure objective-specific imbalances.Static reward weights can worsen the trade-off: emphasizing accuracy encourages verbose evidence, while a weak length penalty may not correct it.Source: §3.3.2 · pp. 5–6
• P-FAB finds a minimum-norm compromise among centered per-objective reward signals, standardizes objectives by variance, and amplifies sparse, hard-to-satisfy criteria.It uses a minimum-norm convex combination of objective directions; as the standardized aggregate norm decreases, training moves toward Pareto stationarity, where gains in one objective require trade-offs in another.Source: §3.3.2 · pp. 6–7

4.1 Ablation Studies

• Removing structured thinking causes performance to collapse across benchmarks, while removing evidence significantly impairs performance.The authors say evidence anchors reasoning, while structured thinking helps the model use extracted evidence to reach its answer.Source: §4.1 · p. 8
• Removing Stage 1.5 causes notable drops, and merging all SFT stages into one phase severely degrades performance across metrics.The results support the format warm-start stage and progressive curriculum as bridges from evidence extraction to complex reasoning and as aids to stable learning.Source: §4.1 · p. 8
• P-FAB generally beats GRPO, especially on temporal-localization and general-understanding metrics, with its advantage widening from group size 4 to 8; RL also improves substantially over SFT alone.The authors attribute P-FAB’s advantage to dynamic Pareto weighting, which uses richer samples to resolve multi-objective reward conflicts.Source: §4.1 · p. 8

4.2 Comparisons with Existing Video-LLMs

• Under identical settings, STEER-4B improves over its Qwen3-VL-4B backbone and across lenient and strict IoU thresholds.The authors describe structured evidence as a time-ordered information bottleneck and event timestamps as anchors for precise activity boundaries.Source: §4.2 · p. 9
• Qwen3-VL-4B-Thinking underperforms its Instruct counterpart on nearly all benchmarks, a trend also observed by [2], while STEER-4B gains over Thinking on NExT-GQA.The authors argue that unstructured CoT spends tokens on self-corrections, repetitive narration, and speculation, whereas evidence-first processing directs thinking toward grounded inference.Source: §4.2 · p. 9
• STEER-4B outperforms competing models while using 1 fps, compared with the 2 fps used by most competitors.The comparison spans temporal grounding and general video understanding; the authors attribute the result to extracting stronger temporal signals from fewer frames.Source: Table 2 · p. 8, §4.2 · p. 9, Table 3 · p. 9

5 Conclusion

• STEER-4B constructs Structured Event Evidence before reasoning, anchoring inference to explicit, verifiable observations.The conclusion frames this as the model’s approach to video reasoning.Source: §5 · p. 9
• STEER-60K is curated to enable structured reasoning.Source: §5 · p. 9
• P-FAB dynamically resolves objective conflicts along the Pareto frontier during RL.Source: §5 · p. 9
• Structured evidence is proposed as a general principle for grounding multimodal reasoning, with extension to cross-modal evidence fusion.Source: §5 · p. 9
• Although demonstrated for structured video reasoning, P-FAB is task-agnostic and may scale to other multi-objective RL settings, which the authors leave for future work.Source: §5 · p. 9

A Implementation Details

• Qwen3-VL-4B-Instruct is the base model; Stages 1 and 1.5 use LoRA [17] with rank 64, α = 128, learning rate 1 × 10−5, and weight decay 1 × 10−4.Source: A Implementation Details · p. 13
• Stage 2 retains the LoRA [17] configuration while increasing learning rate to 1 × 10−4 and weight decay to 2 × 10−4.Source: A Implementation Details · p. 13
• RL Stage 3 uses full-parameter updates with learning rate 5 × 10−6, num_generation = 8, and 64 video tokens per frame; video_max_frames is 128 versus 256 in Stages 1–2.All stages use video_fps = 1.0 and epoch = 1.0.Source: A Implementation Details · p. 13
• Evaluation re-runs Qwen3-series models at fps = 1 and max_frames = 256, while other models use official reports, typically at fps = 2 and max_frames = 2048.Source: A Implementation Details · p. 13
• The training corpus is limited in scale, making greater annotation volume and diversity an identified next step.Source: A Implementation Details · p. 13
• The main experiments use a 4B-parameter model because of computational constraints; scaling to larger architectures depends on additional compute.Source: A Implementation Details · p. 13

B Detailed Information of Evaluation Benchmarks

• Evaluation benchmarks are divided into Video Temporal Grounding and General Video Understanding, covering temporal reasoning and open-ended generation.Source: B Detailed Information of Evaluation… · p. 14

B.1 Video Temporal Grounding Benchmarks

• Temporal-grounding benchmarks test whether models can localize specific moments in untrimmed videos from textual queries.Source: §B.1 · p. 14
• Charades-TimeLens provides verified timestamps for dense, overlapping indoor actions across 1,313 videos and 3,363 annotations.It rewrites 66.3% of low-quality Charades-STA annotations and reports Recall@1 at IoU thresholds 0.3, 0.5, and 0.7.Source: §B.1 · p. 14
• ActivityNet-TimeLens tightens boundaries in 1,455 videos and 4,500 annotations, while ActivityNet-Captions covers 4,885 open-domain videos for dense captioning and temporal grounding [22].ActivityNet-TimeLens rewrites 69.7% of low-quality annotations [52]; ActivityNet-Captions reports Recall@1 at IoU thresholds 0.3, 0.5, and 0.7.Source: §B.1 · p. 14

B.2 General Video Understanding Benchmarks

• General video benchmarks assess holistic understanding through temporal logic, structured reasoning, and egocentric perception.Source: §B.2 · p. 14
• VideoMME evaluates long-context understanding without subtitles across domains such as movies, sports, and documentaries, with videos ranging from minutes to hours.Source: §B.2 · p. 14
• Experiments use MLVU’s Temporal Reasoning and Egocentric Video Understanding subtasks to test sequence understanding and first-person reasoning [56].Source: §B.2 · p. 14
• ETBench covers eight time-sensitive tasks, from referred action recognition and event-caption alignment to temporal grounding, episodic memory, and grounded video QA.The benchmark is intended to verify whether model reasoning yields accurate event-level deductions.Source: §B.2 · p. 14
• NExT-GQA tests grounded answers to causal “Why” and temporal “How” questions involving object interactions, requiring answers to be linked to video segments [46].Source: §B.2 · p. 14

C Data Construction Pipeline Details

• Videos with fewer than T/30 events are excluded from evidence training but retained for RL; retained videos are gap-filled to improve temporal coverage.The selection criterion assumes an ideal event segment spans 10–30 seconds; for example, a 150-second video requires at least five events. Unlabeled intervals between annotated events become candidate segments for description.Source: C Data Construction Pipeline Details · p. 15
• Qwen3-VL-235B-A22B-Instruct [2] and Gemini-2.5-Pro [8] alternate between generating and judging structured event evidence.The mutually critiquing models produce data for Stage 1 and Stage 1.5; low-quality samples are rejected and iteratively refined, with random human inspection.Source: C Data Construction Pipeline Details · p. 15, Figure 4 · p. 15
• Gemini-2.5-Pro [8] generates evidence-conditioned thinking traces, while Qwen3-VL judges them for Stage 2.The process first generates challenging questions and conditions explicit thinking traces on each question and its associated evidence.Source: C Data Construction Pipeline Details · p. 15, Figure 4 · p. 15
• 500 of every 1,000 generated annotations were manually inspected, covering 50% of the 60K annotations.The samples were selected randomly from each batch.Source: C Data Construction Pipeline Details · p. 15

D.1 Video Source Distribution

• The corpus contains 32,049 videos curated from diverse high-quality benchmarks to construct a robust training corpus; all videos come only from original-dataset training splits, and original captions are not used.Source: §D.1 · pp. 15–16
• The corpus preserves human-annotated temporal boundaries and replaces original unstructured captions with high-density structured descriptions.This combines precise event timestamps with semantic descriptions for structured reasoning tasks.Source: §D.1 · p. 16

D.2 Video Duration and Topic Distribution

• The 18 semantic categories include action-intensive Tutorials and Sports as well as Vlog, Daily Life, Cooking, and Travel.The distribution combines structured instructional content with open-ended daily scenarios.Source: §D.2 · p. 16, Figure 5 · p. 16
• 109.4 seconds is the average video duration, and 123.6 seconds is the median.Durations are concentrated around one to three minutes, providing temporal context while remaining concise enough for efficient processing.Source: Figure 5 · p. 16, §D.2 · p. 16

D.3 RL Training Data Details

• The RL dataset contains 32,049 videos, with 53% Temporal Grounding and 21% Spatial VQA plus 20% Reasoning VQA.Tutorial (7,043 samples) and Sports (5,746) are the most prevalent topics; other categories include Vlog, Daily Life, Cooking, and Travel.Source: §D.3 · pp. 16–17, Figure 6 · p. 17

D.4 Data Curation Prompts

• Three curation prompts generate structured evidence, judge its format, timestamps, consistency, and content, and guide evidence-grounded question answering.Evidence generation requires six fields and preserves timestamps; thinking generation uses global search and localization followed by antecedent-action-consequence verification.Source: §D.4 · p. 18

E Instruction Tuning Data Format

• Instruction tuning progresses from extracting structured event evidence, through format warm-up with a placeholder thinking block, to evidence-grounded reasoning warm-up.The final stage uses challenging QA pairs and trains global search, evidence verification, and alignment against the extracted evidence.Source: E Instruction Tuning Data Format · pp. 18–19

F.1 Detailed Rewards Design

• The composite reward combines structured-format compliance, task performance, and a soft length constraint to encourage concise, evidence-grounded reasoning.Format scoring distinguishes basic structural adherence from complete reasoning structure, while the length reward decays linearly beyond its target budget rather than imposing abrupt truncation.Source: §F.1 · pp. 19–20
• Temporal grounding uses a hybrid coverage-ratio/Span-IoU reward for one prediction against multiple ground-truth segments; QA receives binary option-match accuracy.Source: §F.1 · p. 19

F.2 Frank-Wolfe Algorithm for Solving the Minimum-Norm Problem

• Frank-Wolfe solves the minimum-norm convex objective over a simplex using a one-hot minimum-gradient update and exact line search, which ensures monotonic decrease and convergence to a Pareto-stationary point.Source: §F.2 · p. 20

G Theoretical Analysis of P-FAB

• Under GRPO with standardized rewards, P-FAB yields Pareto stationarity in parameter space without further assumptions on policy-gradient structure.Source: G Theoretical Analysis of P-FAB · p. 21

G.1 Setup and Notation

• The setup centers each response’s reward by objective, standardizes the reward matrix, and defines per-response log-probability gradients for GRPO policy gradients.P-FAB operates in standardized reward space.Source: §G.1 · p. 21

G.2 Pareto Stationarity of Standardized P-FAB

• Perfect balance in standardized reward space implies Pareto stationarity without assumptions on gradient structure; unlike MGDA, P-FAB optimizes in reward space with O(1) overhead.The stated guarantee connects the reward-space proxy to a Pareto-stationary point in parameter space.Source: §G.2 · pp. 21–22
• Standardization gives smaller-variance objectives larger effective weights; as the standardized imbalance norm approaches zero, the combined policy gradient approaches zero and high-variance objectives cannot dominate.The effective weight is wm = α∗m/σq,m, amplifying under-satisfied objectives.Source: §G.2 · p. 21

H.1 TempCompass and EventBench Evaluation

• 70.64% on TempCompass and 63.48% on EventBench, STEER-4B outperforms both Qwen3-VL-4B-Instruct and Qwen3-VL-4B-Thinking across sub-tasks, with gains of +3.45% in attribute change detection and +2.60% in causal reasoning.The benchmarks assess temporal perception and event-level understanding, respectively.Source: Table 4 · p. 22, §H.1 · p. 22

H.2 Scalability to 8B Model

• At 8B scale, P-FAB consistently outperforms standard GRPO across all benchmarks, showing the balancing approach generalizes beyond the 4B model.The comparison uses Qwen3-VL-8B with LoRA fine-tuning.Source: Table 5 · p. 22, §H.2 · p. 22

H.3 P-FAB Computational Overhead

• P-FAB’s weight computation adds 0.0013% of total training step time across all group sizes; it uses a small quadratic program over M = 4 objectives, independent of model size.Source: §H.3 · p. 22

H.4 Training Dynamics of P-FAB

• Over 949 training steps, P-FAB reduces the Format weight as its reward saturates, increases IoU weight for the hardest objective, improves all four rewards, and decreases mean output token length.Source: §H.4 · p. 23, Figure 8 · p. 23

H.5 Token Length and Inference Efficiency

• On temporal grounding tasks, STEER-4B with P-FAB outputs 849 vs. 4288 tokens on CTL and 436 vs. 1388 tokens on TempCompass, shorter than the Thinking baseline and GRPO variant.The passage attributes the shorter reasoning chains to structured evidence and says P-FAB balances length and accuracy.Source: §H.5 · pp. 23–24, Table 7 · p. 24

H.6 Evidence Quality Evaluation

• STEER-4B substantially outperforms the base Instruct model on F1, BLEU-4, ROUGE-L, and CIDEr for generated structured evidence, indicating higher-quality evidence generation.The evaluation compares generated evidence against gold-standard evidence annotations.Source: §H.6 · p. 24, Table 8 · p. 24

H.7 Training Data Overlap Analysis

• Excluding Charades and ActivityNet training videos causes only minor temporal-grounding performance drops on their corresponding test benchmarks.Charades-TimeLens R1@0.3: 57.1 →56.7, R1@0.5: 40.4 →39.8, and R1@0.7: 21.6 →21.1; ActivityNet-Captions R1@0.3: 61.7 →59.3, R1@0.5: 41.2 →38.9, and R1@0.7: 28.1 →26.4.Source: §H.7 · p. 24, Table 9 · p. 24
• The ActivityNet drop is larger because it is the largest training-corpus source, yet the model still outperforms all open-source baselines after exclusion.The results support gains from structured reasoning rather than data memorization.Source: §H.7 · p. 24

H.8 Reasoning Quality Evaluation

• The evaluation rates outputs on Overall quality, Hallucination avoidance, Logicality of the reasoning chain, and Interpretability of the structured output.Gemini-2.5-Pro serves as the automatic judge, and each dimension is scored on a 1–5 scale.Source: §H.8 · p. 25
• The authors conclude that structured event evidence enhances the quality of the reasoning process.This evaluation considers intermediate reasoning quality beyond final-answer accuracy.Source: §H.8 · p. 25
• STEER-4B matches or outperforms the Thinking baseline on all four reasoning-quality dimensions, with gains of +1.0 in Interpretability and +0.25 in Logicality.The evaluation results are reported in Table 10.Source: Table 10 · p. 25, §H.8 · p. 25