FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting

1 INTRODUCTION

• Existing methods rely on large, uniformly sampled frame sets and text-only reasoning, while video agents often use predefined workflows and are not end-to-end learnable.The paper says irrelevant frames make long contexts inefficient and degrade reasoning performance.Source: §1 · pp. 1–2
• FrameThinker first sparsely scans a video, then uses its reasoning to select promising segments for targeted frame retrieval, interleaving visual frames with textual thought.This iterative process can be repeated as needed while processing a fraction of the video’s frames.Source: §1 · p. 2, Figure 1 · p. 2
• Training uses supervised fine-tuning to teach action execution, then reinforcement learning to develop strategic action policies and explore reward designs.The authors report that unconditional action rewards can cause mode collapse, while conditional rewards alone may still reinforce illogical thought-action pairs; CCV is proposed to address consistency.Source: §1 · p. 2
• Across tested benchmarks, FrameThinker averages +10.4% accuracy over baselines; on LongVideo-Reason it reaches 76.1% with 20.6 frames versus LongVILA-R1’s 72.0% with 512.The paper also reports 46.8% accuracy on Video-Holmes, establishing a new state-of-the-art there.Source: §1 · p. 2

Video Understanding

• RLVR-based video models including Video-R1, VideoChat-R1, and LongVILA-R1 report substantial gains, but predominantly rely on passive uniform frame sampling [Feng et al., 2025] [Li et al., 2025] [Chen et al., 2025b].The paper frames FrameThinker as challenging that sampling paradigm.Source: §2 · p. 3

Video Agents

• Video agents often use external tools and fixed workflows, whereas FrameThinker learns an autonomous end-to-end policy to decide when and how to interact with video [Wang et al., 2024b] [Fan et al., 2024] [Wang et al., 2025] [Zhang et al., 2025b] [Liu et al., 2025].The paper identifies limited policy learnability as a constraint on flexibility.Source: §2 · p. 3

Thinking with Image

• “Thinking with Images” enables iterative consultation of visual data, and FrameThinker adapts this idea by querying video frames as its cognitive state evolves [Su et al., 2025a;b;c] [Zheng et al., 2025].The queried frames are selected according to the model’s evolving reasoning.Source: §2 · p. 3

3.1 FRAMEWORK OVERVIEW

• The model sparsely scans a video, then selects frames from promising segments for targeted inspection, using thought-action-observation steps until it outputs an answer.Its action space also includes converting timestamps to frame numbers for later frame selection.Source: §3.1 · pp. 3–4

3.2 SUPERVISED FINE-TUNING

• SFT teaches action syntax from 2,392 curated examples; RL then uses a more diverse 28k-example dataset and GRPO to optimize policy.SFT predicts tokens in the thought and action sequences, excluding the query and action observations from the loss.Source: §3.2 · p. 4, §3.3 · p. 5

3.3.1 SHOULD WE USE A FORMAT REWARD?

• A format reward can favor answer-only shortcuts over exploration, so the authors omit it; malformed actions already forfeit the final accuracy reward.They report that a large format reward can make a random final guess a low-risk way to earn reward, while an attempted action risks a format error.Source: §3.3.1 · p. 5

3.3.2 SHOULD ACTION REWARDS BE UNCONDITIONAL OR CONDITIONAL?

• Unconditional action bonuses can cause repetitive action loops, while conditional bonuses reward actions only in trajectories with correct answers; CCV filters inconsistent trajectories.At inference, CCV can terminate an illogical attempt and allow a retry or fallback.Source: §3.3.2 · pp. 5–6

3.3.3 WHICH ACTION SHOULD THE REWARD PRIORITIZE?

• The authors set λgfn ≫ λcf because frame-number retrieval is objectively verifiable and can indirectly supervise meaningful frame selection.The bonus requires the retrieved frame number to be used correctly in the selected interval, a condition checked by CCV.Source: §3.3.3 · p. 6

3.3.4 SHOULD WE ENCOURAGE THE MODEL TO PERFORM AS MANY TURNS AS POSSIBLE?

• A reward for additional turns initially increased turn counts but destabilized training and degraded reasoning, so the authors excluded it from the final design.The reward counts analysis turns, excluding the final answer action.Source: §3.3.4 · p. 6

4 EXPERIMENTS

• Small-scale and large-scale experiments share SFT but use different RL phases; resource constraints make small-scale runs the platform for ablations, while large-scale training uses the complete RL dataset.Implementation uses Qwen2.5-VL-7B-Instruct and samples 8 frames per turn for most videos, or 12 for videos longer than 300 seconds.Source: §4 · p. 7, §4.2 · pp. 7–8

4.1 BENCHMARKS AND BASELINES

• The six benchmarks cover reasoning and long-video comprehension, including exceptionally long scenarios; Qwen2.5-VL-7B is the primary baseline, with related Qwen-based models also compared.Video-Holmes and LongVideo-Reason assess reasoning; four other benchmarks assess comprehension across increasing durations.Source: §4.1 · p. 7

4.3 SMALL-SCALE EXPERIMENTS

• 56.1% accuracy on Video-Holmes establishes a new state of the art, while using significantly fewer frames on average than existing models.The small-scale model is trained on Video-Holmes; no specific frame count is given here.Source: §4.3 · p. 8
• A direct comparison with the fine-tuned Qwen2.5-VL-7B baseline under identical SFT or RL setups found the full approach achieved a clear performance advantage; format reward rapidly reduced early actions, while CCV improved performance.For action rewards, λgfn = 0.2 and λcf = 0 yielded the most significant performance benefit; Figure 4 presents these ablations.Source: §4.3 · p. 8

4.4 LARGE-SCALE EXPERIMENTS

• 76.1% accuracy with 20.6 frames on average sets a LongVideo-Reason state of the art, surpassing LongVILA-R1, which requires 512 frames.Table 2 presents the reasoning-benchmark results.Source: §4.4 · p. 8, Table 2 · p. 8
• 53.2% average accuracy, a +10.4% improvement over Qwen2.5-VL-7B, is reported across six benchmarks, alongside high performance and frame efficiency on long-video comprehension tasks.Figure 5 summarizes the six-benchmark comparison; Table 3 covers long-video benchmarks.Source: §4.4 · p. 9, Figure 5 · p. 9, Table 3 · p. 9

5 CONCLUSION

• FrameThinker shifts video understanding from passive, uniform sampling to active, iterative analysis of video content.The model dynamically selects and reasons over a minimal number of frames.Source: §5 · pp. 9–10
• Its training pipeline combines Supervised Fine-Tuning and Reinforcement Learning with Cognitive Consistency Verification for logically grounded, interpretable actions.The paper explores reward design for multi-turn video analysis.Source: §5 · p. 9
• FrameThinker achieves state-of-the-art performance on challenging reasoning benchmarks while drastically reducing the required visual context.Source: §5 · pp. 9–10

A MORE IMPLEMENTATION DETAILS

• SFT used rank-8 LoRA on 8 H800 GPUs, while RL used 8 H800 GPUs with batch size 32 and 8 rollouts.SFT used per-device batch size 1, four gradient-accumulation steps, effective batch size 32, learning rate 1.0e-4, and cosine scheduling with 0.1 warmup.Source: A MORE IMPLEMENTATION DETAILS · p. 13

B.1 SYSTEM PROMPT FOR FRAMETHINKER

• FrameThinker’s system prompt defines its available actions and the required thought-and-action output format.Figure 6 outlines the get frame number and choose frames action syntax.Source: §B.1 · p. 13, Figure 6 · p. 14

B.2 FALLBACK PROMPT FOR CCV FAILURES

• When CCV detects an illogical action, inference terminates and a simplified prompt directs the model to answer directly.The fallback is intended to prevent the model from getting stuck or failing the task.Source: §B.2 · p. 13, Figure 7 · p. 14

B.3 CHAIN-OF-THOUGHT PROMPT FOR BASELINES

• Baseline evaluation uses a Chain-of-Thought prompt matching the original Video-Holmes benchmark evaluation prompt.The appendix illustrates this prompt for Qwen2.5-VL-7B and derivatives.Source: §B.3 · p. 14, Figure 8 · p. 14

C ADDITIONAL EXAMPLES

• For a timestamp-specific query, the model locates the event with get frame number, then selects frames around it to inspect the light-switch reflection.Source: C ADDITIONAL EXAMPLES · p. 14, Figure 9 · p. 15
• For an ambiguous query, the model zooms into a hypothesized interval, where denser frames reveal a chase and attack supporting the “Hunter and Fugitive” answer.Source: Figure 10 · p. 16, C ADDITIONAL EXAMPLES · p. 16
• In a hierarchical search, the model first searches a broad interval after noticing a mask, then narrows the interval to confirm its function.Source: Figure 11 · p. 17, C ADDITIONAL EXAMPLES · p. 17
• When Frame 0 already answers a question about the video’s beginning, the model outputs an answer without taking further actions.Source: Figure 12 · p. 18, C ADDITIONAL EXAMPLES · p. 18

D.1 MODE COLLAPSE FROM UNCONDITIONAL REWARDS

• A 0.2 unconditional get frame number bonus caused incoherent, circular reasoning and repeated calls at 00:22 without using the returned information.Source: §D.1 · p. 18
• Unconditional action rewards led to degenerate policies that exploited the bonus instead of solving the task, motivating conditional action rewards.The model could earn a high-certainty immediate bonus more easily than an uncertain reward for a correct answer requiring multi-step reasoning.Source: §D.1 · pp. 18–19
• An unconditional 0.2 choose frames bonus produced a single elongated turn with repeated, often illogical frame-selection actions.The model’s reasoning became tautological, and it could stuff a turn with rewarded actions before making a likely random guess.Source: §D.1 · pp. 18–19

D.2 MODE COLLAPSE FROM MULTI-TURN REWARDS

• An alternative step-based reward designed to encourage more extensive reasoning awarded 0.2 per turn (k = 0.2), with total bonus capped at 0.6; both unconditional and conditional versions were explored.Source: §D.2 · pp. 19–20
• Unconditional turn rewards collapsed around step 270, while conditional rewards collapsed around step 170 after turns first increased.In the unconditional case, thought was eliminated and the model produced a random answer; the conditional case showed a similar rise-then-drop pattern.Source: §D.2 · p. 20, Figure 15 · p. 21, Figure 16 · p. 21
• Both reward variants encouraged extending trajectories over solving tasks, demonstrating that interaction quality, not turn count, must be rewarded.The authors attribute collapse to long-horizon RL instability and incentives to produce valid syntax that prompts another turn instead of coherent reasoning.Source: §D.2 · p. 20

D.3 TRAJECTORIES FAILING COGNITIVE CONSISTENCY VERIFICATION (CCV)

• The CCV redundancy check penalizes repeating an action with identical parameters, a behavior observed mainly early in training or during mode collapse.Repeatedly requesting the same timestamp or frame interval are examples; such repetition becomes rare after training stabilizes.Source: §D.3 · pp. 21–22
• The logical-flow check flags a choose-frames action that ignores a frame number retrieved by get-frame-number.In the example, retrieval returned frame 815, but the next action selected interval 565–645, so CCV terminated the trajectory.Source: §D.3 · p. 22, Figure 17 · p. 23
• The fidelity check compares numerical arguments in thought and action, terminating trajectories when the executed action contradicts the preceding reasoning.One example identifies a relevant area near frame 4974 in thought but selects frames 1400–1500, earning zero reward.Source: §D.3 · p. 24, Figure 18 · p. 24

E TRAINING DYNAMICS

• Accuracy rose consistently and began converging around step 1100 as performance on the task distribution approached its peak.Figure 19a tracks average accuracy during RL training.Source: E TRAINING DYNAMICS · p. 24, Figure 19 · p. 25
• After stabilizing initially, average action reward rose significantly after about 850 steps, when the model began its second pass through the training data.The authors interpret the later rise as more precise action use on harder instances after accuracy gains had begun to plateau.Source: E TRAINING DYNAMICS · pp. 24–25, Figure 19 · p. 25
• Action counts first fell as RL penalized erroneous or superfluous actions, then recovered and settled into a stable range as beneficial action use improved.The authors attribute the initial decline to incomplete action mastery after training on a small SFT dataset.Source: E TRAINING DYNAMICS · p. 25, Figure 19 · p. 25
• Average response length declined during RL, with the model shedding verbosity without an explicit reward for length.The authors report that the model retained core task reasoning while moving away from the longer style of its SFT teacher data.Source: E TRAINING DYNAMICS · p. 25, Figure 19 · p. 25

• Video-Holmes and LongVideo-Reason target complex narrative reasoning, and their training sets are used for training while evaluation uses their test sets.Video-Holmes emphasizes scattered visual and audio clues; LongVideo-Reason includes temporal, object-related, spatial, and plot-based questions.Source: §F.1 · p. 26
• LSDBench tests locating brief, action-dense segments in long videos, while CG-Bench focuses on answering questions from specific video clues.Source: §F.1 · p. 26
• NExT-QA targets causal and temporal action relations; STAR tests four types of situated reasoning, and PerceptionTest diagnoses perceptual and reasoning abilities.STAR’s reasoning types are interaction, sequence, prediction, and feasibility.Source: §F.1 · p. 26
• LongVideoBench covers diverse videos up to one hour, while MLVU spans 3 minutes to 2 hours across genres including movies, surveillance, and gameplay.Source: §F.1 · p. 26
• VideoMME covers six domains and 30 sub-categories across videos from 11 seconds to one hour, while LVBench videos average over one hour.Source: §F.1 · p. 26
• The six primary benchmarks span increasing video durations and include reasoning-focused Video-Holmes and LongVideo-Reason alongside long-video tests VideoMME-Long and LVBench.LongVideoBench and MLVU complete the primary suite; Figure 20 summarizes average video length across the six benchmarks.Source: §F.1 · p. 26, Figure 20 · p. 27

• The SFT dataset contains process-supervised and outcome-supervised subsets, with trajectories synthesized by Gemini-2.5-Pro and filtered for correct answers and illogical reasoning.The curation is intended to focus SFT on meaningful and effective action execution strategies.Source: §F.2 · p. 27
• Process-supervised templates include direct answering, single-pass frame selection, and timestamp-grounded selection using get-frame-number before choosing frames.Source: §F.2 · p. 27
• Other process-supervised templates refine a previously selected interval or select frames from two distinct intervals.Source: §F.2 · p. 27
• Outcome supervision applies only to final-answer correctness, and Figure 21 shows the reasoning-template and action-count distributions.Source: §F.2 · p. 27, Figure 21 · p. 27