PaperlayerView exact revision on arXiv

Source-linked AI summary

Tempo-Zero: Recursive Self-Play in Executable Worlds for Video Reasoning

Jingmin Zhu, Yanyan Zou, Xiuyu Li, James Bailey, Dinh Phung, Tao Yu, Yunyun Hou, Jun Liu, Jun Zhou, Sirui Wang, Nan Duan, Jiachen Liu, Qiuhong Ke

arXiv:2610.32433v1cs.CV

TL;DR

Video self-play on recorded collections relies on model-derived labels and cannot change event dynamics, despite the cost of temporal annotation. Tempo-Zero uses executable environments for evolving, verifiable self-play and outperforms base models and recent self-play baselines across twelve real-world benchmarks, especially in temporal grounding.

  • Problem

    Video self-play on fixed recorded collections relies on model-derived labels that risk reinforcing shared errors and cannot change event dynamics.

    Source: §1 · p. 2

  • Method

    Tempo-Zero uses executable programs to render videos with verified answer options and intervals, then trains a shared Questioner-Solver policy with execution-grounded rewards and evolving curricula and rules.

    Source: §1 · p. 2

  • Results

    Tempo-Zero outperforms base models and recent self-play baselines on average across twelve real-world benchmarks at different model scales, especially in temporal grounding.

    Source: §5 · p. 10

  • Takeaways & Limitations

    Training solely on simple programs without external video data or supervised fine-tuning is accompanied by gains on real-world benchmarks, particularly temporal grounding.

    Source: §5 · p. 10

    The environments cover twelve hand-designed physical-behavior families with fixed task schemas and rule menus, while natural video has greater perceptual variability and commonsense breadth.

    Source: G Limitations and Broader Impact · p. 26

Abstract

from arXiv · show

Video reasoning requires understanding how events unfold over time, yet videos with accurate temporal annotations are expensive to collect. Self-play offers an alternative by letting a model generate and solve its own tasks without manual annotation. However, existing video self-play methods train on fixed sets of recorded videos, so they cannot alter the underlying event dynamics and rely on noisy pseudo-labels derived from the model's own outputs, such as majority voting, that risk reinforcing shared model errors. To overcome these limitations, we introduce Tempo-Zero, a self-improvement framework that closes a loop of executable environments, execution-verified self-play, and environment evolution, where executing a program renders a video with the ground-truth option and interval of every question. A single policy plays both Questioner and Solver, proposing questions and answering them with timestamped evidence lines, and is trained on execution-grounded rewards instead of model consensus. The Solver's performance steers a curriculum over tasks and difficulty levels, and the policy selects rule changes that enrich the event dynamics of later rounds. Despite being trained solely on simple programs, without supervised fine-tuning or external video data which prior methods typically require, Tempo-Zero outperforms the base models and recent self-play baselines on average across twelve real-world video benchmarks at different model scales, with particularly strong gains on temporal grounding tasks. Project page: https://alchemist-jo.github.io/tempo-zero/

1 Introduction

Tempo-Zero addresses fixed video dynamics and unreliable pseudo-labels by using executable environments for evolving, execution-verified self-play. It improves over base models and recent self-play baselines on average across twelve real-world benchmarks, especially in temporal grounding.

1 Introduction

  • Collecting diverse videos, designing multi-step tasks, and verifying temporal annotations are costly, motivating self-improvement approaches that generate training tasks and feedback.Source: §1 · p. 1
  • Existing video self-play uses recorded videos with labels derived from model outputs, risking reinforcement of shared errors while leaving event dynamics fixed.Source: §1 · p. 2
  • Executable programs generate both video event traces and verifiable answer options and evidence intervals, while rule changes can alter the rendered events.Source: §1 · p. 2
  • Tempo-Zero combines executable environments, a shared Questioner-Solver policy trained with execution-grounded rewards, a pass-rate curriculum, and policy-selected rule changes.Source: §1 · p. 2
  • Tempo-Zero improves over base models and recent self-play baselines on average across twelve real-world benchmarks, with particularly strong gains in temporal grounding.Source: §1 · p. 3

2 Related Work

Prior video self-play uses fixed recorded collections, while programmatic video methods provide deterministic supervision but often use fixed rules and templates. Tempo-Zero adapts the training distribution through pass-rate curricula and policy-selected rule changes.

Self-play

  • Video-Zero and EvoVid train on fixed recorded videos, using annotations, model consensus, or sampled-clip positions for labels and intervals; Tempo-Zero computes them by program execution.Its temporal checks screen labels before rollouts.Source: §2 · p. 3

Programmatic environments

  • SynRL and SIMS-V use rules and question templates fixed before training, whereas Tempo-Zero steers its curriculum by pass rates and has the policy select validated rule changes.Re-execution recomputes videos and labels as the training distribution evolves.Source: §2 · p. 3

3 Method

Tempo-Zero couples executable video generation to one policy that asks and solves criterion-grounded questions. Solver performance adjusts training instances, while the policy selects validated rule changes.

3 Method

  • Tempo-Zero trains one shared vision-language policy as Questioner and Solver, using execution criteria to score questions, answers, and cited evidence.Source: §3 · pp. 3–4

3.1 Executable Environments

  • Programs render videos and criteria from event traces; labels are recomputed on sampled frames, and criteria insensitive to event order are excluded.Source: §3.1 · pp. 4–5

3.2 Execution-Verified Self-Play

  • A gate filters proposed questions, while Solver rollouts measure pass rate and timestamped evidence earns interval credit only when its event matches the criterion.Source: §3.2 · pp. 5–6

3.3 Environment Evolution

  • Pass-rate statistics reallocate tasks and difficulty, while policy-selected rule changes introduce new event dynamics after executor validation.Source: §3.3 · pp. 6–7

4 Experiments

Tempo-Zero is evaluated on real-world video benchmarks after training only on rendered videos, with ablations and analyses examining its components and execution-grounded supervision. The reported results span temporal grounding, long-video understanding, video reasoning, and multiple model backbones.

4.1 Experimental Setup

  • Qwen3-VL-4B-Instruct and Qwen3-VL-8B-Instruct each serve as Questioner and Solver, trained with LoRA adapters for five rounds on videos rendered by executable environments alone, without external videos, human annotations, or supervised fine-tuning [1].Source: §4.1 · p. 7

4.2 Main Results

  • Tempo-Zero improves every grounding metric over the base model at 4B and 8B, with the best result in 26 of 28 metrics.On Charades-STA, its mIoU gain over the base model is 9.40, compared with Video-Zero’s 0.64 gain.Source: §4.2 · p. 8
  • It improves all eight long-video and reasoning benchmarks on every Qwen3-VL backbone, leading in 13 of 16 metrics; gains also extend to 21 of 22 metrics on each of two other backbones.The additional backbones are Qwen3-VL-30B-A3B and Gemma 4 E4B.Source: §4.2 · p. 8

4.3 Ablations

  • Removing Questioner proposals causes the largest Overall ablation drop, from 50.81 to 47.71; removing Solver evidence rewards lowers Temporal Grounding by 5.70 points.The learnability reward matters more than novelty, while rule selection adds event dynamics beyond curriculum reallocation and its removal lowers Temporal Grounding by 1.77 points.Source: §4.3 · pp. 8–9, Table 3 · p. 9

4.4 Analysis

  • At 8B, held-out Solver accuracy rises from 25.9% to 42.8%; at 4B, evidence score rises from 24.3% to 47.0%, while 4B gate-passing proposals increase from 80.7% to 94.9%.Rule selection occurs in 56 of 96 family decisions, mostly in rounds 2 and 3, and every change remains verifiable.Source: §4.4 · p. 9, Figure 6 · p. 9
  • Majority voting would mislabel 52.5% of 7,420 Solver groups; execution checks ensure labels match sampled frames, while rgrd withholds evidence credit for lines naming another event.The wrong-label share is 56.8% at 4B and 48.2% at 8B.Source: §4.4 · p. 10, Figure 7 · p. 10
  • Both training banks improve over the base model in every column, and fine-tuning on evolved-rule data scores higher in every column than on data without rule selection.The comparison fine-tunes the 4B base model on each run’s bank, apart from the self-play loop.Source: §4.4 · p. 10, Table 4 · p. 10

5 Conclusion

Tempo-Zero uses executable programs to provide verifiable supervision while one policy plays both Questioner and Solver and evolves the environments. Trained without supervised fine-tuning or external video data, it outperforms base models and recent self-play baselines on average across twelve benchmarks.

  • Across twelve real-world benchmarks and model scales, Tempo-Zero outperforms base models and recent self-play baselines on average, especially in temporal grounding.Source: §5 · p. 10
  • Executable programs render synthetic videos with ground-truth answer options and time intervals, keeping supervision verifiable without external video corpora or majority voting.Source: §5 · p. 10
  • A single policy acts as Questioner and Solver, learns from execution-grounded rewards, follows a pass-rate curriculum, and selects rule changes.Source: §5 · p. 10
  • The framework is trained only on simple programs, without external video data or supervised fine-tuning.Source: §5 · p. 10

Ethics Statement

Training videos are program-rendered, and evaluation uses public benchmarks under their released protocols.

AI Use Statement

The authors report generative-AI assistance in drafting and editing the paper and software, with author review and responsibility. The appendix describes the self-play loop, shared training configuration, and compute setup.

AI Use Statement

  • Generative AI assisted with paper text, references, tables, figures, and software; the authors reviewed the outputs and take responsibility for the work.Source: AI Use Statement · p. 11

A.1 Self-Play Algorithm

  • Each round renders allocated batches, samples Questioner proposals and Solver rollouts, computes rewards, updates both roles, then adjusts difficulty and environment rules.The loop stores measured questions and gated proposals in a bank; one optimizer state persists across roles and rounds.Source: §A.1 · p. 16

A.2 Training Configuration

  • The Questioner and Solver contribute separate reward groups to one LoRA update, while rule selection runs at inference; each question is answered eight times.The configuration is shared by the 4B, 8B, 30B-A3B, and Gemma models.Source: §A.2 · p. 16, Table 5 · p. 17

A.3 Compute Resources

  • Training uses eight GPUs, with a persistent inference engine and asynchronous CPU video rendering scheduled one step ahead.Source: §A.3 · p. 16

A.4 Evaluation Protocol

The evaluation protocol covers video reasoning, temporal grounding, and long-video understanding across twelve benchmarks. The appendix describes deterministic environments and checks that temporal supervision matches sampled frames.

A.4 Evaluation Protocol

  • Twelve benchmarks cover video reasoning, temporal grounding, and long-video understanding, with final evaluation using the fifth-round checkpoint on complete test splits under official metrics.Source: §A.4 · p. 17

B.1 Environment Coverage and Design

  • Environment families use deterministic state transitions, rendering, query templates, and analytical verification; difficulty controls entity interactions and operation complexity, while playback scale sets duration.The task table distinguishes interval questions from multiple-choice questions, and the environment-family table summarizes observable dynamics and reasoning targets.Source: §B.1 · p. 18, Table 6 · p. 18, Table 7 · p. 18

B.2 Video Frame Sampling and Temporal Coordinates

  • Ground-truth criteria are checked against uniformly sampled frames, and intervals use the effective timestamps presented to the model without a second temporal downsampling.Criteria are discarded when their values cannot be recomputed from sampled frames or disagree with the simulation.Source: §B.2 · pp. 18–19

B.3 Verification Checks on the Training Environments

The appendix checks whether training supervision remains verifiable on sampled frames, then details the role protocols, evolving rules, reward terms, question bank, and shared-policy optimization. These mechanisms constrain training signals to executable evidence and control how tasks and environments change.

B.3 Verification Checks on the Training Environments

  • Every key event appears on a sampled frame; recomputation discards 403 of 49,525 criteria, and the static-question screen removes 9.01% of the remainder.The discarded criteria include cases where recomputed values disagree with simulation or cannot be recomputed from frames; the screen excludes questions invariant to reversal and permutation.Source: §B.3 · pp. 19–20

B.4 Role Inputs and Output Protocols

  • The Questioner proposes a verified criterion with evidence lines, while the Solver receives only frames, the question, and any options, then answers with located evidence and an option or interval.Proposals pass a gate requiring one ground-truth option, admissible distractors, and a stem matching the task specification.Source: §B.4 · p. 20

B.5 Rule Selection and Execution Determinism

  • The policy selects from checked rule-change menus and bounded class weights; previews execute proposed configurations before rendering, while invalid changes retain the prior program.Task levels follow pass-rate thresholds, and interval and state-tracking tasks use specified level schedules.Source: §B.5 · pp. 20–21

C.1 Reward Terms

  • Rewards combine Solver learnability, evidence matching and interval grounding, and Questioner novelty credit for evidence intervals not already printed in the question.The learnability reward peaks when half of the Solver rollouts are correct; grounded evidence scores match recorded events and their intervals.Source: §C.1 · pp. 21–22

C.2 Question Bank

  • The 640-question bank stores measured canonical questions with their verification context and uses decayed pass-rate statistics to inform the curriculum and rule-selection prompts.At capacity, eviction first targets overrepresented task classes and families, then older measurements.Source: §C.2 · p. 22

C.3 Group Relative Normalization and Policy Optimization

  • GRPO updates the shared Questioner–Solver policy using separate role-centered advantages and eligible groups from both roles in one combined update per step.Solver groups contain eight rollouts per question, and Questioner groups contain four proposals per instance.Source: §C.3 · pp. 22–23

D Analysis Details

The analyses test majority-vote labels and interval credit, track held-out performance across rounds, and report rule decisions and usable previews. Execution labels are reproducible, while majority voting frequently disagrees with them.

D.1 Majority Voting and Interval Credit by Backbone

  • 56.8% of 4B and 48.2% of 8B Solver groups have wrong majority labels; agreement filtering would retain a wrong label for 21.3% of 4B groups.Across difficulty levels, wrong-majority rates range from 50.8% to 53.5%; grounded scoring keeps 36% to 38% of content-blind interval credit, and only on lines stating the correct event.Source: Table 9 · p. 23, §D.1 · p. 23

D.2 Held-Out Instances

  • All three held-out metrics exceed the base model after every round at both backbones.Evaluation uses 128 instances balanced over families, levels, and playback scales; the table reports accuracy, evidence score, and grounded score.Source: §D.2 · pp. 23–24, Table 10 · p. 24

D.3 Rule Decisions

  • The two training runs make 96 family rule decisions, and all 270 menu alternatives compile into usable instances in runtime previews.Each backbone makes 12 decisions per round.Source: §D.3 · p. 24, Table 11 · p. 24

E.1 Ablation Variants

The ablation variants isolate individual components of the full method under a matched training and evaluation setup.

E.1 Ablation Variants

  • The shared setup makes each variant a single-component comparison against the full method.The passage specifies matched initialization, budget, curriculum constants, and evaluation protocol.Source: §E.1 · p. 24
  • Each ablation variant differs from the full method in exactly one component.Variants share initialization, training budget, curriculum constants, and evaluation protocol with the full method.Source: Table 12 · p. 24, §E.1 · p. 24
  • The ablation table defines the variants but the supplied passages do not report their outcomes.No ablation performance values are included in these passages.Source: Table 12 · p. 24

E.2 Training Value of Evolved Environments

The environment-evolution comparison contrasts training with initial rules against the full method, which evolves rules, while keeping the curriculum and budget fixed. The accompanying prompt and rollout passages describe how the Questioner and Solver provide grounded proposals and answers.

E.2 Training Value of Evolved Environments

  • The evolved-environment comparison uses Qwen3-VL-4B-Instruct fine-tuned on bank questions and recorded Solver responses, contrasting fixed initial rules with the full method.Both runs use the same curriculum and budget and differ only in rule selection, so differences between rows come from the data produced by rule selection.Source: §E.2 · p. 25

F Prompt Templates and Qualitative Rollout

  • The Questioner selects a verified query whose answer depends on timing and supplies located evidence lines before its proposal.Its prompt includes frames, scene and rule information, askable criteria, a bank question of the same task, and Solver pass rate.Source: F Prompt Templates and Qualitative… · p. 25
  • The Solver locates each needed observation before deriving the answer, with each line checked against recorded events and the interval shown in the video.The representative Billiards rollout cites initial and terminal observation intervals before stating the quadrant change.Source: F Prompt Templates and Qualitative… · pp. 25–26

G Limitations and Broader Impact

Tempo-Zero's scope is bounded by pretrained backbones, hand-designed environments, and deterministic temporal supervision. Its executed-program training signal reduces reliance on scraped video and human annotation, while versioned rule configurations make each run's curriculum inspectable.

Related papers

Loading 2610.32433v1…