PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Fixed Temporal Grids: Frequency-Disentangled Motion Generation

Yunjiao Zhou, Junlang Qian, Gen Li, Xinying Guo, Lihua Xie, Jianfei Yang

arXiv:2610.03012v1cs.CV

TL;DR

Uniform temporal grids give all motion tokens the same temporal support, although human motion combines slow trajectories and rapid events. FreqMo uses wavelets to separate temporal scales and a shared-codebook quantizer for discrete generation. The paper reports strong fidelity and high-frequency preservation, with the decomposition also improving continuous generation.

Abstract

from arXiv · show

Most human motion generation methods encode motion as tokens on a uniform temporal grid, where every token spans the same fixed time window. Human motion, however, is temporally heterogeneous: slowly evolving global trajectories coexist with rapid transient events such as foot contacts and joint impulses. Forcing such multi-scale dynamics onto tokens of identical temporal resolution entangles motion frequencies, leaving slow regions redundant while smoothing out the rapid details that distinguish realistic motion. We propose \textbf{FreqMo}, a scale-adaptive motion representation that decomposes motion into wavelet frequency bands, separating dynamics across temporal scales while preserving temporal localization and exact reconstruction. Unified Frequency Residual Quantization (UFRQ) then encodes all bands within a single shared codebook, compressing the token sequence threefold and enabling stable single-stage generation. Experiments show FreqMo attains SOTA fidelity with substantially improved high-frequency preservation, and the same decomposition transfers to continuous diffusion backbones.

Introduction

Uniform temporal grids mix motion dynamics that unfold at very different speeds, making slow regions redundant and rapid details hard to preserve. FreqMo uses wavelet-based, frequency-adaptive representation and shared-codebook quantization to separate these scales while supporting single-stage generation.

Introduction

  • Identical temporal support entangles slow trajectories and rapid events, producing redundancy in slow regions and smoothing details in fast transients.The paper locates this distortion in single-resolution representation, not model capacity.Source: Introduction · p. 1
  • Wavelet filtering adapts temporal resolution by frequency, giving low-frequency trajectories broad context and high-frequency transients fine precision while preserving event timing.This makes frequency disentanglement a property of the representation rather than the architecture.Source: Introduction · pp. 1–2
  • Coarse-to-fine refinement locks global structure first, confining residual error to high-frequency detail.The wavelet hierarchy runs from high-frequency detail toward coarse global structure, so the refinement order matters.Source: Introduction · p. 2
  • FreqMo decomposes motion into hierarchical frequency bands and uses UFRQ to encode them in one shared codebook through coarse-to-fine residual refinement.A scale-aware masked transformer jointly generates all bands, and the representation also transfers to continuous diffusion.Source: Introduction · p. 2
  • The authors identify single-scale temporal rigidity as structural and report that the representation improves generation under both discrete and continuous paradigms.They describe UFRQ as preserving temporal localization and exact reconstruction while supporting stable single-stage generation.Source: Introduction · p. 2

Related Work

Prior motion representations span continuous latents and discrete codebooks, each with different representational trade-offs. For generation, the paper situates its approach among diffusion, autoregressive, and masked-transformer models, building on bidirectional masked transformers.

Related Work

  • Continuous motion latents offer high precision, but can spend capacity on irrelevant high-frequency noise and minor variations, hindering generalization and complicating generation.The review contrasts this with discrete quantization, which maps poses to a finite codebook and acts as a regularizer.Source: Motion Latent Representation · p. 2

Motion Generative Models

  • Diffusion and autoregressive generators can be slow; masked transformers predict tokens in parallel through iterative refinement, balancing inference speed and quality.The paper builds its generator on a bidirectional masked-transformer framework.Source: Motion Generative Models · p. 2

Method

FreqMo separates motion dynamics across temporal scales before generation, then uses shared-codebook residual quantization and a text-conditioned masked transformer to generate frequency tokens. Its inference process iteratively completes the token sequence before decoding it into motion.

Method

  • FreqMo combines a wavelet-based, frequency-disentangled representation with a text-conditioned masked transformer that jointly generates frequency tokens.The wavelet decomposition is independent of the generator and applies to continuous or discrete latent spaces.Source: Method · p. 2

Frequency-Disentangled Motion Representation

Unified Frequency Residual Quantization

  • UFRQ encodes frequency bands coarse-to-fine in one shared codebook, progressively quantizing residuals so low-frequency structure is captured before finer details.The tokenizer is trained end-to-end to minimize reconstruction error while stabilizing codebook utilization.Source: Unified Frequency Residual Quantization · p. 3

Text-to-Motion Generation Model

  • A bidirectional masked transformer processes all frequency tokens jointly, using text cross-attention, normalized timestamps, and frequency-level embeddings.A scale-aware attention prior biases attention toward coarser levels; the tokenizer remains frozen during this training objective.Source: Text-to-Motion Generation Model · p. 4, Scale-aware attention prior · p. 4

Inference

  • At inference, the model iteratively unmasks and predicts tokens, then decodes the completed sequence into a latent representation and motion.Each refinement step unmasks the top-k_m positions by confidence before re-predicting the remaining tokens.Source: Inference · p. 4

Experiment

FreqMo evaluates motion generation on HumanML3D and KIT-ML, comparing benchmark performance, reconstruction fidelity, frequency preservation, and design choices. Results highlight improved high-frequency fidelity and token efficiency, alongside gains from multi-scale bands and selected generator components.

Setup

  • The benchmarks use HumanML3D and KIT-ML, mirrored for augmentation and split 0.8:0.05:0.15 into training, validation, and test sets.Evaluation reports FID, R-Precision, Multimodal Distance, Multimodality, Acceleration Error, and Jerk Error; FID and R-Precision are prioritized over diversity alone.Source: Setup · pp. 4–5

Generation Performance

  • FreqMo achieves the best HumanML3D FID (0.040) and a competitive KIT-ML FID (0.174), while ReMoDiffuse’s lower KIT-ML FID (0.155) uses retrieval augmentation.The authors describe the latter comparison as complementary rather than direct; Table 1 compares performance on both datasets.Source: Generation Performance · p. 5, Table 1 · p. 5
  • FreqMo-Diff, which replaces UFRQ with a VAE and diffusion decoder, consistently outperforms frame-wise MLD, supporting gains from the frequency-disentangled representation across paradigms.The authors state this confirms the gains stem from the representation rather than the quantization scheme.Source: Generation Performance · p. 5

Representation Quality

  • Against RQ-VAE, FreqMo lowers Acceleration Error by 10.8% (2.15 vs. 2.41) and Jerk Error by 9.2% (532 vs. 586), using 106 rather than 294 tokens.The comparison isolates reconstruction fidelity and high-frequency preservation across quantization methods; FreqMo uses a single shared codebook.Source: Representation Quality · pp. 5–6, Table 2 · p. 6
  • Uniform representations’ power spectral density rolls off sharply above 2 Hz, whereas FreqMo tracks the ground-truth spectrum up to 6 Hz.Per-frame acceleration comparisons also show FreqMo tracking ground-truth transient peaks in timing and magnitude, unlike uniform-grid baselines.Source: Representation Quality · p. 6, Figure 6 · p. 6, Figure 5 · p. 6
  • Adding frequency bands from coarse to fine yields monotonic FID improvement; removing the base layer causes catastrophic degradation, while each finer band contributes complementary detail.Table 3 analyzes the hierarchical frequency contributions.Source: Table 3 · p. 6, Representation Quality · p. 7

Analysis

  • Removing wavelet decomposition or residual quantization substantially degrades FID; removing Scale-Aware Attention raises FID from 0.040 to 0.053, and autoregressive decoding raises it to 0.088.Performance improves with decomposition depth up to L=5 and is stable across wavelet bases.Source: Analysis · p. 7, Table 4 · p. 7
  • FreqMo occupies the optimal bottom-left quadrant of the log-scale FID-versus-inference-speed plot, indicating it is both faster and more accurate.The figure plots generation quality (FID) against average inference time (AIT).Source: Analysis · p. 7, Figure 8 · p. 7

Conclusion

FreqMo decomposes motion into hierarchical frequency bands and encodes them through coarse-to-fine residual quantization in a unified codebook. The paper reports strong fidelity and high-frequency preservation, with gains across discrete and continuous paradigms.

Conclusion

  • FreqMo addresses frequency entanglement from uniform temporal tokenization by decomposing motion into hierarchical frequency bands and encoding them in a unified codebook via coarse-to-fine residual quantization.The conclusion reports SOTA fidelity and faithful high-frequency preservation, with consistent gains across discrete and continuous paradigms.Source: Conclusion · p. 7
Loading 2610.03012v1…