Source-linked AI summary
The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics
Xiangbo Gao, Mingyang Wu, Siyuan Yang, Jiongze Yu, Pardis Taghavi, Fangzhou Lin, Zhengzhong Tu
TL;DR
Generative video models can produce smooth motion without a reliable physical time scale because training commonly ignores differing real-world speeds. The paper introduces Visual Chronometer and PhyFPS benchmarks to measure and correct this issue, finding substantial misalignment and instability while showing improved temporal naturalness after correction.
Problem
Video generators lack a consistently grounded and controllable physical time scale because training treats videos with different physical speeds as temporally equivalent.
Method
Visual Chronometer predicts intrinsic Physical Frames Per Second from visual dynamics using controlled temporal resampling, supported by PhyFPS-Bench-Real and PhyFPS-Bench-Gen.
Results
State-of-the-art generators show substantial PhyFPS misalignment and temporal instability, while PhyFPS-guided dynamic retiming improves human-perceived temporal naturalness.
Takeaways & Limitations
Reliable video world models require active, physics-grounded temporal conditioning rather than passive reliance on nominal metadata.
Takeaways & Limitations
The paper does not require every generated video to use 1× physical speed; controllability, including stable default 1× speed, is the stated prerequisite.
Abstract
from arXiv · showhide
While recent generative video models have achieved remarkable visual realism and are being explored as world models, true physical simulation requires mastering both space and time. Current models can produce visually smooth kinematics, yet they lack a reliable internal motion pulse to ground these motions in a consistent, real-world time scale. This temporal ambiguity stems from the common practice of indiscriminately training on videos with vastly different real-world speeds, forcing them into standardized frame rates. This leads to what we term chronometric hallucination: generated sequences exhibit ambiguous, unstable, and uncontrollable physical motion speeds. To address this, we propose Visual Chronometer, a predictor that recovers the Physical Frames Per Second (PhyFPS) directly from the visual dynamics of an input video. Trained via controlled temporal resampling, our method estimates the true temporal scale implied by the motion itself, bypassing unreliable metadata. To systematically quantify this issue, we establish two benchmarks, PhyFPS-Bench-Real and PhyFPS-Bench-Gen. Our evaluations reveal a harsh reality: state-of-the-art video generators suffer from severe PhyFPS misalignment and temporal instability. Finally, we demonstrate that applying PhyFPS corrections significantly improves the human-perceived naturalness of AI-generated videos. Our project page is https://xiangbogaobarry.github.io/Visual_Chronometer/.
1 Introduction
Modern video generators can produce visually smooth motion without grounding it in a consistent physical time scale. The paper defines this failure as chronometric hallucination and proposes Visual Chronometer to recover PhyFPS from visual dynamics.
- Modern generators model spatial realism and fluid kinematics, but often lack a stable relationship between spatial displacement and elapsed real-world time.
- Training on videos with different physical speeds while ignoring their temporal regimes makes generated motion speeds ambiguous, unstable, and uncontrollable.
- Visual Chronometer predicts Physical Frames Per Second directly from visual dynamics, distinguishing intrinsic PhyFPS from nominal metadata FPS.
- Chronometric hallucination denotes generated motion whose physical time scale is not consistently grounded, even without speed-manipulating prompts.
- PhyFPS-Bench-Real and PhyFPS-Bench-Gen evaluate prediction accuracy, generator alignment with physical speed, and temporal stability across windows and outputs.
- PhyFPS-guided correction significantly improves the human-perceived temporal naturalness of AI-generated videos.
2 Related Works
Prior video-generation research emphasizes perceptual quality, spatial layout, and frame-to-frame dynamics, while physical time scale is rarely encoded or supervised. Related work has inferred speed or temporal direction from visual cues, but conventional generation evaluation remains dominated by perceptual and distributional metrics.
- Modern diffusion and autoregressive video models use spatiotemporal operators, causal attention, and temporal latent spaces to achieve perceptual quality and semantic coherence.
- Video generators increasingly serve as proposed world models, yet physical time scale is rarely supervised and models rely on nominal metadata FPS.
- Speed-understanding research progressed from domain-specific slow-motion detection to self-supervised discrimination of normal-rate and artificially sped-up clips.
- Traditional video-generation evaluation relies on frame-level similarity, no-reference perceptual predictors, and distribution-level metrics such as FVD.
3 Data Preparation
The paper constructs PhyFPS training data from temporally reliable sources and controlled resampling. High-rate 240 FPS videos are converted into diverse low-rate variants that model camera capture effects across 18 physical frame rates.
- Training sources are selected where nominal metadata FPS matches physical sampling rate, excluding videos with ambiguous post-hoc time-scale editing.
- The dataset combines high-frame-rate academic benchmarks, raw broadcast footage, synchronized autonomous data, physics-grounded human motion, and verified in-house data.
- All source videos are temporally upsampled to 240 FPS with RIFE before generating target physical frame-rate variants.
- Sharp Capture: Sharp Capture uniformly subsamples the high-rate sequence, preserving sharp boundaries while producing naturally aliased motion such as stutter.
- Motion Blur: Motion Blur averages a temporal exposure window, with long, medium, and short exposures represented by M ∈{N, N/2, N/4}.
- Synthetic Rolling Shutter: Synthetic Rolling Shutter samples different spatial bands at progressively shifted times to model sequential sensor readout.
- The final dataset spans 18 Physical Frame Rates and contains 465,535 clips standardized to 128 frames.
4 Visual Chronometer
Visual Chronometer encodes video clips into spatiotemporal latent representations, pools temporal information with a query-based attention head, and regresses PhyFPS in logarithmic space. It supports variable-length inputs and is trained across distinct frame-rate regimes.
- VideoVAE+ extracts compact spatiotemporal latent tokens from an input clip for Visual Chronometer.
- A learnable query cross-attends to latent tokens, producing a clip-level representation independent of the input frame count.
- The model predicts log PhyFPS and optimizes mean squared error between the ground-truth and predicted logarithmic targets.
- Because PhyFPS values are positive, the loss omits MSLE’s customary +1 offset to represent proportional temporal scaling directly.
- Training uses sliding-window clips with a maximum temporal footprint of T = 32 frames and includes variants for different operational frame-rate regimes.
5 Experiments
The experiments audit physical time-scale alignment in generative videos, validate Visual Chronometer against real dynamics and VLMs, and test its perceptual and methodological utility. Results show pervasive PhyFPS mismatch and instability in generators, strong predictor performance, and improved human preference after PhyFPS correction.
- PhyFPS-Bench-Gen: Most generators exhibit pervasive Meta-vs-PhyFPS mismatch, while LTX models show strong stability despite high absolute alignment errors.Closed-source models slightly outperform open-source models in absolute accuracy, but their inter- and intra-video CV scores are not significantly better.
- Perceptual validation: Both post-processed variants significantly outperform original outputs, with Bradley–Terry preferences of 44.2% for Pred, 36.9% for Pred Dyn, and 19.0% for the original.Pred applies global average correction, whereas Pred Dyn independently re-times temporal segments; the study collected 1,490 pairwise comparisons from over 15 participants.
- Visual Chronometer validation: Visual Chronometers achieve exceptionally low MAE and MAPE on PhyFPS-Bench-Real, whereas tested VLMs fail catastrophically at physical-speed estimation.Seed-1.6-Flash, for example, predicts exactly 30 FPS for every input, regardless of the dynamics.
- Ablation studies: Physics-grounded temporal augmentations reduce MAE from 5.12 to 3.46 compared with uniform temporal subsampling.The ablation attributes the improvement to simulating exposure integration and sequential sensor readout, which helps disentangle physical speed from motion artifacts.
- Ablation studies: Temporal context is optimized at T = 32, remains competitive at T = 64 and 128, but T = 128 loses sliding-window variance reduction and fine-grained fluctuation capture.Very short contexts lack sufficient visual evidence, while extremely large windows restrict evaluation to a single global inference pass.
6 Discussion: Implications and Future Directions
The discussion distinguishes controllable creative retiming from arbitrary temporal hallucination and outlines data, architectural, and optimization strategies for resolving it.
- Inference Context Length: Inference-context ablation compares 32-frame and 128-frame training limits to study the trade-off between temporal receptive field and sliding-window granularity.The evaluation uses the VC-Common model on PhyFPS-Bench-Real across inference patch sizes.
- Q1: Alignment and Creative Retiming: Strict PhyFPS–meta FPS alignment is not always desirable because deliberate slow-motion and time-lapse are important creative tools.The paper instead treats a stable default 1× physical speed as the prerequisite for controllable N× retiming.
- Q1: Alignment and Creative Retiming: Chronometric hallucination is problematic when models arbitrarily assign physical speeds without explicit user instruction.A prompt such as “person walking” may produce 0.5× or 2× physical speed unpredictably.
- Q2: Future Resolution: Future pipelines should treat time as an active condition through PhyFPS-labeled data, temporal conditioning, and physics-grounded preference signals.Visual Chronometer could annotate or filter data and provide reward signals for RLHF or DPO.
7 Conclusion
The paper formalizes chronometric hallucination and introduces Visual Chronometer to recover motion-grounded PhyFPS. Its benchmarks expose inconsistent internal motion pulses, while PhyFPS-guided retiming improves perceived temporal naturalness.
- Contributions: The paper identifies chronometric hallucination as ambiguous and uncontrollable physical speeds caused by reliance on arbitrary metadata containers.This formalizes the mismatch between nominal metadata and the depicted motion’s physical time scale.
- Contributions: Visual Chronometer predicts intrinsic Physical Frames Per Second directly from visual dynamics using physics-grounded temporal resampling.The method recovers temporal scale from motion rather than relying on container metadata.
- Findings: PhyFPS-Bench-Gen and PhyFPS-Bench-Real reveal that state-of-the-art generators and vision-language models struggle to maintain a consistent internal motion pulse.The benchmarks provide the paper’s evaluation basis for temporal instability and misalignment.
- Findings: PhyFPS-guided dynamic retiming significantly improves the human-perceived temporal naturalness of AI-generated videos.The paper presents this correction as an immediate practical mitigation.