PaperlayerView exact revision on arXiv

Source-linked AI summary

SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation

Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami

arXiv:2609.37918v1cs.CVcs.LG

TL;DR

Cross-video reasoning requires relating events, identities, motion, and partial observations, yet reliable labels and targeted supervision are difficult to obtain from real footage. SYNCR uses shared simulator-grounded task generators to evaluate and train these capabilities. Models show persistent physical-comparison and scene-integration difficulties, while fine-tuning improves benchmark accuracy and temporal-ordering transfer is most consistent on real footage.

  • Problem

    Real-footage multi-video benchmarks are difficult to annotate precisely, and ambiguous evidence or textual shortcuts complicate interpretation of model performance.

    Source: §1 · p. 1

  • Method

    SYNCR uses shared simulator-grounded task generators to derive answers and support evaluation and targeted supervision across eight cross-video tasks.

    Source: §1 · pp. 1–2, §2 · p. 2

  • Results

    Physical comparison and scene integration remain difficult, while SFT raises Qwen3-VL-8B’s average SYNCR accuracy from 32.6% to 61.6% and temporal-ordering transfer is most consistent.

    Source: §1 · p. 2, §3.2 · p. 4, §6 · p. 10

  • Takeaways & Limitations

    SYNCR provides a controlled setting for diagnosing cross-video reasoning failures, testing their learnability, and identifying where synthetic supervision transfers.

    Source: §6 · p. 10

    SYNCR uses visual-only multiple-choice questions, mostly from simulation, and simulator labels do not guarantee every answer is visible in RGB.

    Source: §6 · p. 10

Abstract

from arXiv · show

Reasoning across videos requires aligning events, matching identities, comparing motion, and integrating partial observations. Evaluating these capabilities and testing how to improve them requires both reliable labels and targeted supervision. We introduce SYNCR, a simulator-grounded framework that connects these two needs through shared task generators. Built on Habitat, Kubric, and CLEVRER, SYNCR derives answers from environment state and provides 4,000 evaluation questions and 15,960 training questions over disjoint videos, spanning eight cross-video reasoning tasks. Visual ablations and human evaluation assess dependence on the supplied evidence and answer recoverability. Evaluation of 22 multimodal large language models reveals persistent difficulties in physical comparison and scene integration that increasing model size does not consistently resolve. Supervised fine-tuning raises Qwen3-VL-8B's average SYNCR accuracy from 32.6% to 61.6%, with gains extending to task configurations and video sources absent from training for those tasks. Transfer to real footage is most consistent for temporal ordering: accuracy improves by 9.0-20.5 percentage points on constructed Assembly101 and Panoptic ordering sets across three checkpoints spanning two model families and two model sizes, with additional gains on existing temporal reasoning benchmarks. These results establish SYNCR as a controlled setting for diagnosing cross-video reasoning failures, testing their learnability, and identifying where synthetic supervision transfers.

1 INTRODUCTION

Cross-video reasoning requires relating events, identities, behavior, and partial observations, but reliable evaluation and targeted learning are difficult when real-footage labels are hard to annotate and evidence can be ambiguous. SYNCR connects simulator-grounded evaluation with training, revealing persistent weaknesses alongside measurable improvements from supervision and transfer to real footage.

1 INTRODUCTION

  • Real-footage benchmarks are difficult to annotate precisely, and ambiguous evidence or textual shortcuts complicate interpretation of model performance.Simulation provides access to underlying state, but answers must still be recoverable from the supplied videos.Source: §1 · p. 1
  • SYNCR uses shared task generators, simulator-derived labels, and disjoint evaluation and training videos to diagnose and train eight cross-video reasoning tasks.Models receive RGB videos, questions, and options; visual ablations and human evaluation probe evidence dependence and answer recoverability.Source: §1 · pp. 1–2
  • 32.6% to 61.6%: supervised fine-tuning raises Qwen3-VL-8B’s average accuracy across eight tasks on matched 200-item-per-task sets.The passage also reports improvements under changes to task structure and video source.Source: §1 · p. 2
  • 9.0–20.5 percentage points: three checkpoints improve on constructed Assembly101 and Panoptic temporal-ordering sets, while results on other real-footage tasks remain mixed.The passage also reports gains on existing temporal reasoning benchmarks.Source: §1 · p. 2

2 SYNCR: TASKS AND DATA CONSTRUCTION

SYNCR reuses task generators to derive simulator-grounded answers for cross-video questions, enabling the same tasks to support evaluation and targeted training. Its eight tasks span four diagnostic families, with item construction grounded in simulator annotations and evaluation videos held apart from training videos.

2 SYNCR: TASKS AND DATA CONSTRUCTION

  • Shared task generators compute answers from simulator state and support testing whether task-specific supervision improves measured failures.Evaluation and training use disjoint videos; later experiments vary task structure and video source.Source: §2 · p. 2

2.1 DIAGNOSTIC TASK FAMILIES

  • The eight tasks are organized into temporal alignment, spatial tracking, comparative reasoning, and holistic synthesis, which expose failures hidden by an eight-task average.The four families are not assumed to be independent abilities.Source: §2.1 · p. 2
  • Temporal tasks test matching event start times across camera views and ordering shuffled, gapped segments from one continuous video.Ordering uses event progression and object motion rather than directly matching adjacent boundary frames.Source: §2.1 · p. 2
  • Spatial Tracking tests re-identifying an object across walkthroughs and reconciling two camera projections to compare 3D distance.A specified object’s departure fixes the time of the spatial comparison.Source: §2.1 · p. 2
  • Comparative tasks compare peak speeds across videos or count collisions and identify the difference, allowing ties.One task estimates motion over time; the other counts discrete events.Source: §2.1 · p. 2
  • Holistic Synthesis counts distinct objects across walkthroughs or joins partial route clips to find the shortest path between rooms.Repeated sightings count once, while route planning joins room connections shown in separate clips.Source: §2.1 · p. 2

2.2 SIMULATOR-GROUNDED ITEM CONSTRUCTION

  • Simulator annotations select clips and determine labels; models receive RGB clips, questions, and options, not the annotations.The framework groups eight tasks into four families covering temporal alignment, identity and geometry, physical comparison, and partial-scene integration.Source: §2.2 · pp. 2–3, Figure 1 · p. 3
  • Habitat, Kubric, and CLEVRER supply walkthrough, multi-camera physical, and annotated collision or trajectory data for different task generators.The passages describe filtering for object visibility, independently cropped camera views, and margins or gaps that shape selected tasks.Source: §2.2 · p. 3

3 EVALUATING CROSS-VIDEO REASONING

Evaluation of 22 multimodal models finds uneven performance, with persistent weaknesses in physical comparison and scene integration and marked differences between temporal tasks. Evidence controls test reliance on videos, while cross-benchmark correlations support SYNCR’s relevance but do not establish that it isolates cross-video reasoning.

3.1 EXPERIMENTAL SETUP

  • The evaluation covers 22 models and uses zero-shot multiple-choice questions, with different item counts for open-weight and proprietary models and a four-person human reference.Standard open-weight checkpoints receive 500 items per task; proprietary models and the human reference receive the same first 100.Source: §3.1 · p. 4

3.2 MAIN RESULTS

  • 64.5% versus 89.5%: GPT-6 Astra trails the human majority vote on matched items; Qwen3-VL 32B leads standard open-weight checkpoints at 37.5%.The human and proprietary results use 100 items per task, while standard open-weight results use 500-item sets, so the groups are not precisely rankable against each other.Source: §3.2 · p. 4
  • Sequential Ordering reaches 100% for GPT-6 Astra, while Multi-Angle Synchronization scores for standard open-weight checkpoints remain near chance at 21.8–31.0%.The two temporal tasks test different relations: ordering gapped segments from one recording versus matching moments across camera views.Source: §3.2 · p. 4
  • Physical comparison and scene integration remain difficult: the highest standard open-weight scores are 29.8% on Kinematic Comparison, 30.4% on Object Counting, and 32.8% on Route Planning.Human majority-vote scores are 68%, 92%, and 88% on those respective matched sets.Source: §3.2 · p. 4
  • Model-size gains vary by family, and Kinematic Comparison remains below chance across Qwen3-VL sizes of 4B, 8B, and 32B.LLaVA-OneVision stays between 23.8% and 25.2% across 0.5B–72B; the Thinking checkpoint comparison does not isolate deliberation because post-training differs.Source: §3.2 · p. 5

3.3 VALIDITY CHECKS

  • Text-only controls remain within chance intervals for tested models, while one-video controls sharply reduce accuracy on several tasks but not universally.For example, one video reduces Qwen3-VL 32B Sequential Ordering from 66.0% to 24.5%; evidence dependence is clearest when full-input accuracy exceeds chance.Source: §3.3 · p. 6
  • Source-video timestamps are removed from cropped inputs, preventing timestamp markers from directly revealing ordering sequences or synchronization offsets.Elapsed time within each clip is retained.Source: §3.3 · p. 6
  • SYNCR averages correlate with MVU-Eval Temporal Reasoning and Comparison scores, but the correlations do not establish that SYNCR isolates cross-video reasoning.The reported Spearman correlations are 0.86 and 0.92, respectively; a general video-understanding factor could also produce them.Source: §3.3 · p. 6

4 LEARNING FROM SYNTHETIC SUPERVISION

SYNCR’s shared task generators provide supervision for testing whether cross-video reasoning failures can be improved. Fine-tuning improves held-out performance and some shifted settings, while transfer to real footage is clearest for temporal ordering.

4 LEARNING FROM SYNTHETIC SUPERVISION

  • The same task generators support diagnosis and learning experiments, testing held-out videos, changed task structures and sources, and transfer to real footage.The Qwen3-VL-8B setup uses one epoch of LoRA training; eight-task SFT uses all 15,960 training examples.Source: §4 · pp. 6–7

4.1 TRAINING GAINS ON SYNCR

  • 32.6% to 61.6%: eight-task SFT raises Qwen3-VL-8B’s average accuracy on matched held-out items, improving seven of eight tasks.Object Counting changes from 30.5% to 34.5%, a smaller gain than on the other tasks.Source: §4.1 · p. 7, Figure 4 · p. 7

4.2 GENERALIZATION TO NEW STRUCTURES AND VIDEO SOURCES

  • Training gains persist under several structural and camera-rig shifts, while transfer to new simulator pairings varies by task and source.For example, trained Sequential Ordering reaches 99.0–99.5% with three or five segments; on a held-out camera rig, Multi-Angle Synchronization reaches 74.5% versus 23.0% for the base.Source: §4.2 · pp. 7–8, Table 2 · p. 8

4.3 SIM-TO-REAL TRANSFER

  • 9.0–20.5 points: three checkpoints improve on constructed Assembly101 and Panoptic ordering sets; existing temporal benchmarks also improve, while other real-footage tasks remain mixed.Ordering gains are measured against each model’s own base; the sets use clips from individual source streams.Source: §4.3 · p. 8, Figure 5 · p. 9

4.4 CONTROLS AND ROBUSTNESS CHECKS

  • Controls show that incorrect-label supervision and the tested spatial baseline do not reproduce temporal-transfer gains, while no-video training leaves seven SYNCR tasks near chance.The authors caution that the video-availability contrast does not isolate a causal visual contribution; main gains persist across seeds and multiple-testing correction.Source: §4.4 · pp. 8–9

5 RELATED WORK

Prior work includes multi-video benchmarks and simulation-based training for single-video skills. SYNCR’s distinction is to couple simulator-state labels and shared generators with matched cross-video training data, controls, and transfer tests.

5 RELATED WORK

  • Multi-video benchmarks extend single-video evaluation to questions relating evidence across videos, while adjacent work studies multi-view understanding, linked activity views, and video-based spatial memory.GameplayQA addresses synchronized agent viewpoints in virtual environments.Source: §5 · p. 9
  • SYNCR couples simulator-grounded labels and shared generators with matched cross-video training data, controls, and tests of transfer to real video.This places diagnosis, supervision, and transfer measurement under one set of task definitions.Source: §5 · p. 9
  • Simulation-based training has addressed single-video reasoning, spatial questions, and kinematics, but the cited work does not supervise relations across separate video streams.In a matched-example comparison, SAT frames rendered as clips did not reproduce SYNCR’s temporal-transfer gains; the passage limits this result to one alternative dataset and recipe.Source: §5 · p. 9

6 CONCLUSION

SYNCR connects simulator-grounded evaluation with targeted supervision across eight cross-video reasoning tasks. Its clearest replicated real-footage transfer is chronological ordering, while transfer varies substantially by task.

6 CONCLUSION

  • Across 22 models, SYNCR finds persistent difficulty in physical comparison and scene integration; eight-task SFT raises Qwen3-VL-8B’s mean accuracy from 32.6% to 61.6%.Ordering gains on constructed Assembly101 and Panoptic sets range from 9.0–20.5 points across three checkpoints; the paper notes limitations in visual-only multiple-choice items and real-footage scope.Source: §6 · p. 10

A THE SYNCR FRAMEWORK: ADDITIONAL DETAILS

  • SYNCR uses eight shared task generators to derive answers from simulator state and test whether supervision on a diagnosed task improves it.Training and evaluation use disjoint videos; experiments also vary task structure and video source.Source: A THE SYNCR FRAMEWORK: ADDITIONAL… · p. 14

A.1 DATASET GENERATION DETAILS

  • The generation pipeline renders or selects clips, filters visibility, computes answers from simulator annotations, and constructs distractors for cross-clip questions.The appendix documents those steps and gives the task prompts.Source: §A.1 · p. 14

A.1.1 HABITAT

  • Habitat supplies egocentric RGB, semantic instance annotations, and navigation meshes; its tasks use visibility windows, distinct-instance counts, and joined route clips.Route Planning asks for the shortest subpath between rooms identified by objects; models receive RGB while semantic tensors determine labels and filters.Source: §A.1.1 · pp. 14–15

A.1.2 KUBRIC

  • Kubric prompt templates cover synchronizing cropped views of one event and finding the object closest to a target in simulator-derived 3D space.For Spatial Measurement, the answer minimizes 3D distance among visible objects at the event frame.Source: Figure 10 · p. 17, §A.1.2 · p. 17, Figure 11 · p. 18

A.1.3 CLEVRER

  • CLEVRER’s annotated collision videos support chronological ordering, peak-velocity comparison, and comparison of collision counts across videos.Ordering uses shuffled gapped segments; numerical comparison asks for the video with most collisions and the difference from the second-highest count.Source: §A.1.3 · p. 18, Figure 12 · p. 18, Figure 13 · p. 19

A.2 DATASET STATISTICS

The appendix reports dataset composition and controls intended to test whether performance depends on video evidence rather than textual cues. Temporal metadata is rebased, and removing videos or retaining only one can substantially reduce performance on some tasks.

A.2 DATASET STATISTICS

  • Evaluation and training videos are disjoint; Table 4 reports per-task video counts, question–answer counts, videos per question, and clip lengths.Unique-video totals deduplicate files reused across tasks.Source: Table 4 · p. 20, §A.2 · p. 20

A.3.1 PREVENTING TEMPORAL METADATA LEAKAGE

  • Per-frame timestamps are rebased so clips start at t=0, preserving elapsed time within each clip but not revealing absolute source position.The protocol is intended to prevent timestamp cues from revealing ordering or crop offsets.Source: §A.3.1 · p. 20

A.3 BENCHMARK VALIDITY

  • Audited text-only rules reach at most 33% on any task, and no-video model results remain within the stated chance intervals.The audit covers first-200 and all-500 item sets; evaluation scenes are disjoint from Habitat training scenes.Source: §A.3.2 · pp. 20–21, §A.3 · p. 20

A.3.2 NO-VIDEO AND SINGLE-VIDEO CONTROLS

  • With one video, Sequential Ordering falls to 24.0% for Qwen3-VL-8B and 24% for GPT-6; Object Re-identification remains above chance for Qwen3-VL-32B.That model scores 39.5% with one video versus 56.0% with all videos.Source: Table 5 · p. 20, §A.3.2 · p. 21

B.1 EXPERIMENTAL DETAILS

The appendix details SYNCR scoring and sampling, reports human and checkpoint comparisons, and notes limits on interpreting them. It also identifies tasks without consistent scaling gains.

B.1 EXPERIMENTAL DETAILS

  • The shared parser accepts a final answer marker, an explicit option commitment, a single option-lettered line, or a response matching an option; unparsed answers are incorrect.Questions have four options, except Numerical Comparison, which has five; stored option order is preserved and each video is prefixed with its index.Source: §B.1 · p. 21, Figure 15 · p. 22
  • Three tasks reuse public CLEVRER videos that may occur in checkpoint pretraining, so visual familiarity could affect their zero-shot results.Newly generated questions and options mean this is not answer leakage; the concern is familiarity with the rendering style.Source: §B.1 · p. 21
  • Reported intervals capture item-sampling uncertainty, not training-seed or API-generation variability; repeated videos and scenes can limit item independence.The appendix reports 95% Wilson intervals and uses exact two-sided McNemar tests for paired model comparisons.Source: §B.1 · pp. 21–22, Table 7 · p. 22
  • Kinematic Comparison, Spatial Measurement, and Object Counting show no consistent improvement across Qwen3-VL 4B, 8B, and 32B.The appendix isolates these task-level exceptions rather than implying consistent scaling gains across tasks.Source: §B.1 · p. 22, Figure 16 · p. 23

B.2 HUMAN BASELINE DETAILS

  • 89.5% is the reported four-person majority-vote average; the protocol does not establish that every answer is recoverable from RGB inputs.The volunteers answered the first 100 items per task without simulator annotations, labels, or generation metadata; Kinematic Comparison reached 68.0% and Spatial Measurement 76.0%.Source: §B.2 · pp. 22–23

B.3.1 REASONING-SPECIALIZED CHECKPOINTS

  • Thinking reaches 76.6% versus Instruct’s 56.4% on Sequential Ordering, but scores 23.8% versus 29.6% on Multi-Angle Synchronization.Its overall average is 34.4% versus 34.1%; these descriptive differences do not isolate deliberation because the checkpoints differ in post-training.Source: §B.3.1 · p. 23, Figure 17 · p. 23

B.3.2 AGREEMENT WITH A REAL-VIDEO BENCHMARK

SYNCR model rankings align with rankings on two MVU-Eval tasks, though this agreement does not establish that SYNCR isolates cross-video reasoning. The appendix also documents the training data and SFT and RL recipes.

B.3.2 AGREEMENT WITH A REAL-VIDEO BENCHMARK

  • SYNCR rankings correlate with MVU-Eval Temporal Reasoning and Comparison rankings (ρ = 0.855 and 0.917, both p < 0.001), but do not show that SYNCR isolates cross-video reasoning.The correlations remain after controlling for log10 model parameters; a general video-understanding factor could produce the same pattern.Source: §B.3.2 · p. 24, Table 8 · p. 24

C LEARNING FROM SYNTHETIC SUPERVISION: ADDITIONAL RESULTS

  • The appendix reports training configurations, complete results, supervision and label controls, and generalization and sim-to-real analyses.These materials support the paper’s learning experiments; the stated correlation analysis compares seventeen open-weight checkpoints on SYNCR and two MVU-Eval tasks.Source: C LEARNING FROM SYNTHETIC SUPERVISION:… · p. 24, §B.3.2 · p. 24

C.1 TRAINING DETAILS

  • SFT uses 2,000 examples per task except Object Counting’s 1,960, while RL uses GRPO with a binary parsed-answer reward.Training mixes range from 7,960 to 15,960 examples; SFT uses LoRA with a frozen vision encoder, and GRPO uses eight rollouts per prompt.Source: §C.1 · pp. 24–25

C.2 FULL SYNCR TRAINING RESULTS

The appendix reports cross-task, held-out-task, and RL results for SYNCR training, alongside results on additional model bases. Transfer and cross-task effects vary by task and training configuration.

C.2 FULL SYNCR TRAINING RESULTS

  • All eight single-task adapters raise average accuracy by 0.1–5.5 points, but 16 of 56 off-diagonal transfer cells decline by more than 1.5 points.The 1.5-point threshold is descriptive, not a statistical-significance criterion; all adapters trained without Multi-Angle Synchronization lower that task by 3.0–8.5 points.Source: §C.2 · p. 25, Figure 18 · p. 27
  • The 4-task mix reaches 29.5% on held-out Spatial Measurement and 24.5% on held-out Kinematic Comparison, while held-out Sequential Ordering falls from 55.0% to 36.0%.The 6-task mix scores below its base on both held-out Kubric tasks; the authors describe the evidence for transfer to excluded tasks as limited.Source: §C.2 · p. 25, Table 9 · p. 26
  • Base-model RL reaches 43.0–47.0% on Object Re-identification and 23.0–29.0% on Kinematic Comparison, versus 71.5–75.5% and 73.5–76.0% for SFT.These comparisons are specific to GRPO with binary correctness rewards and the reported training configuration.Source: §C.2 · p. 25
  • Single-task GRPO raises accuracy on six of seven evaluated trained tasks, most strongly on Kinematic Comparison, from 21.0% to 48.5%; it remains below SFT adapters.Multi-task RL does not match that gain, reaching 23.0–29.0% on Kinematic Comparison; Numerical Comparison evaluation is incomplete.Source: §C.2 · p. 25, Table 10 · p. 26
  • A second 8-task SFT seed reaches 61.8% average accuracy versus 61.6% for the first seed, and gains on ten of eleven real-video benchmark evaluations.It loses 2.0 points on CVBench Scene Recognition; on the first 200 items per SYNCR task, its average is 61.8%.Source: §C.2 · p. 25
  • InternVL3.5-8B improves from 29.3% to 62.8% on matched 200-item sets, with gains on every task; Qwen3-VL-4B reaches 62.6%.The available Qwen3-VL-4B zero-shot result is on all 500 items per task, so a matched training gain is not estimated.Source: §C.2 · p. 25, Table 11 · p. 26

C.3 EVALUATION WITHOUT ANSWER OPTIONS

The appendix tests answer-option-free evaluation, controls on training gains, and generalization beyond training configurations. It also reports the scope of additional sim-to-real analyses.

C.3 EVALUATION WITHOUT ANSWER OPTIONS

  • Without answer options, eight-task SFT raises Object Counting’s mean relative accuracy in all three model families and increases count-rank correlation by 0.11–0.24.The rank-correlation gains have paired bootstrap intervals excluding zero; the evaluation parses a number from each response.Source: §C.3 · p. 27, Table 12 · p. 27
  • Without answer options, Qwen3-VL-4B reaches offset MAE 0.48 s and ρ = 0.75, while InternVL3.5-8B reaches MAE 0.37 s and ρ = 0.90.Their base MAEs are 0.71 s and 0.73 s, close to the 0.70 s constant-zero reference.Source: §C.3 · p. 27, Table 13 · p. 28

C.4.1 ALTERNATIVE SUPERVISION AT A MATCHED BUDGET

  • Matched-budget SAT supervision does not reproduce SYNCR’s transfer gains; on the constructed ordering sets, SAT loses 5.5 points on both while SYNCR gains 9.0 and 19.0.This tests SAT’s image-based content rendered as clips, so the conclusion is limited to that matched-budget comparison.Source: §C.4.1 · p. 28, Table 14 · p. 28

C.4.2 A FORMAT-MATCHED PLACEBO

  • A format-matched placebo trained with randomly wrong labels performs below the base on every SYNCR task and all five reported existing real-benchmark tasks.On Assembly101 ordering, the placebo falls from 35.5% to 28.0%, while correctly labeled SFT reaches 44.5%; this control does not isolate answer-format familiarity.Source: §C.4.2 · p. 28, Table 15 · p. 29

C.4.3 SEED VARIATION

  • Across three 8-task SFT seeds, average accuracy is 61.6%, 61.8%, and 64.6%; Sequential Ordering reaches 98.0% in all three runs.Per-task standard deviation is at most 5.1 points and below 2.5 on six of eight tasks; smaller differences should not be read as effects.Source: §C.4.3 · pp. 28–29, Table 16 · p. 29

C.4.4 VIDEO-DEPENDENCE CONTRAST

  • The full-video gain contrast is 27.1 points for the eight-task mean, versus a 28.9-point full-video gain; Route Planning’s contrast is 10.0 points.The contrast subtracts the no-video gain from the full-video gain and does not partition gains into causal visual and textual components.Source: §C.4.4 · p. 29, Table 17 · p. 29

C.4.5 MULTIPLICITY CORRECTION

  • After Benjamini–Hochberg correction, seven of eight SYNCR SFT task gains and both reported real-benchmark transfer gains remain significant.Object Counting is significant under neither test; the two real-benchmark gains are MVU-Eval Temporal Reasoning and CrossVid Procedural Step Sequencing.Source: §C.4.5 · pp. 29–30

C.5 GENERALIZATION BEYOND THE TRAINING CONFIGURATIONS

  • Generalization tests vary task structure and task–engine combinations; the appendix also reports item counts and paired tests for constructed ordering sets.The task–engine tests do not hold out engines entirely absent from training; some task swaps are omitted where the required ground truth or rendering setup is unavailable.Source: §C.5 · p. 30, §C.6 · p. 30

C.6.1 CONSTRUCTED SETS ON REAL FOOTAGE

Constructed real-video evaluations test whether synthetic cross-video reasoning gains extend to ordering, synchronization, re-identification, and multi-camera tasks. Ordering improves on both real-video sources, while synchronization remains near chance; cross-camera counting gains are less conclusive.

C.6.1 CONSTRUCTED SETS ON REAL FOOTAGE

  • Assembly101 and Panoptic ordering sets each contain 200 shuffled four-segment questions, sampled from 20-second windows with source timestamps removed.Assembly101 uses static RGB recordings; Panoptic uses individual HD-camera streams from six sequences.Source: §C.6.1 · pp. 30–31
  • The EPFL synchronization set assigns three cameras to videos and asks for Videos 2 and 3 start times relative to Video 1; each set has 200 items.Offsets are sampled from 0–24 frames at 12 FPS, and each clip is a three-second crop.Source: §C.6.1 · pp. 31–32
  • Panoptic re-identification asks for a person’s depth rank in a second camera, using persistent 3D pose identity and calibrated camera views.The set uses one qualifying sequence and retains 200 items balanced across the four correct ranks.Source: §C.6.1 · p. 32
  • On Panoptic ordering, Qwen3-VL-8B has 57 correct baseline items versus 95 for SFT and 96 for SFT→RL; both SFT comparisons have exact McNemar p < 0.003 under the least favorable pairing.The pure-RL Assembly101 gain is 3.0 points at p = 0.21.Source: §C.6.1 · p. 32
  • Ordering improves on both real-video sources, while synchronization remains near chance; InternVL3.5-8B gains 14.0 and 20.5 points on the ordering sets.The comparison distinguishes chronological ordering transfer from synchronization across real camera views.Source: §C.6.1 · p. 32, Table 20 · p. 33
  • Every shown checkpoint scores higher with six nuScenes cameras than one, but the enlarged gap is not established across runs and does not establish reliable cross-camera counting transfer.The accuracy gap cannot distinguish identity matching across views from seeing objects absent in the single-camera view.Source: Table 21 · p. 33, §C.6.1 · p. 33

C.6.2 EXISTING MULTI-VIDEO BENCHMARKS

The section tests whether temporal-ordering gains extend beyond SYNCR’s answer format and how supervised fine-tuning compares with other post-training approaches. Gains appear in alternate formats and existing temporal benchmarks, while the size and consistency of transfer vary by model and training mix.

C.6.2 EXISTING MULTI-VIDEO BENCHMARKS

  • On 200 CrossVid PSS items, Qwen3-VL-8B SFT gains 19.0 points in letters and 8.0 in prose, versus 10.0 in SYNCR’s format; InternVL3.5-8B gains 30.5 and 30.0 points in letters and prose.The largest gain occurs in a format absent from training, which the authors treat as evidence of transferred ordering rather than answer-template reuse.Source: Table 22 · p. 34, §C.6.2 · p. 34
  • Base-model GRPO changes MVU-Eval Temporal Reasoning and CrossVid sequencing by 0.5 points each and improves constructed real-video ordering by 3.0–5.0 points, below corresponding SFT gains.VSI-Bench appearance order is an exception: RL gains 13.5 points and SFT→RL 14.0 on that single-video benchmark.Source: §C.6.2 · p. 34
  • After GRPO on the SFT model, MVU-Eval Temporal Reasoning is 76.5% (+8.5 over base) and CrossVid PSS is 84.0% (+9.5); neither changes significantly from its SFT starting point.On PSS, this model is 9.0 points above RL from the base model (p = 0.003).Source: §C.6.2 · p. 34
  • Qwen3-VL-8B-Thinking’s CVBench Temporal Reasoning rise from 50.7% to 66.7% does not survive multiplicity correction; its CrossVid PSS is 81.5%, below the 84.5% for 8-task SFT.The raw CVBench difference is p = 0.029, but the corrected value is pBH = 0.20.Source: §C.6.2 · p. 34
  • InternVL3.5-8B’s eight-task mix produces larger ordering gains than its four-task mix: +28.0 versus +14.0 on PSS and +16.5 versus +8.0 on Temporal Reasoning.The eight-task mix also uses more examples, so the comparison does not isolate the added tasks from extra training exposure.Source: §C.6.2 · p. 34
Loading 2609.37918v1…