PaperlayerView exact revision on arXiv

Source-linked AI summary

Length-varying Neural Motion Stitching via Cluster Transition Graph

Haemin Kim, Junghyun Nam, Seokhyeon Hong, Vanessa Tan, Junyong Noh

arXiv:2609.37167v1cs.GR

TL;DR

Motion stitching methods can require manual adjustments or fixed transition lengths, limiting their handling of distinct motions. This paper uses cluster-graph pathfinding to guide a variable-length neural stitching generator. The authors report that the guide reduces generation ambiguity and supports transitions across a wide range of motion types, while performance depends on suitable training clusters and pathfinding weights.

  • Problem

    Interpolation can require manual alignment and transition-point selection, while existing neural stitching assumes similar motions and fixed-length training data, limiting transitions between distinct motions.

    Source: §1 · p. 2

  • Method

    The method clusters input motions, finds a connecting path in a cluster transition graph, and uses that path with the inputs to guide a motion generator.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    The authors report that graph-derived guides reduce ambiguity during transition generation and effectively connect a wide range of motion types.

    Source: §9 · p. 16

  • Takeaways & Limitations

    The cluster path determines transition length and supplies per-pose initialization, supporting automatic stitching across varied motion types.

    Source: §1 · p. 2

    Performance depends on diverse, high-quality training clusters, and pathfinding requires manually tuned weights that keep guides consistent with generator training.

    Source: §8.1 · p. 14, §8.2 · p. 15

Abstract

from arXiv · show

Motion stitching aims to create new character animations by seamlessly combining existing motion sequences. Existing approaches often require manual selection of transition range or assume fixed transition length, restricting the types of motions that can be connected. To broaden the diversity of motions that can be synthesized, it is essential to generate transitions of varying lengths, allowing the character sufficient time to adapt its pose when the input motions differ significantly. To this end, we propose a length-varying neural motion stitching method based on a cluster transition graph, which produces naturally connected motion sequences given two distinct input motions. Our framework consists of three stages: motion clustering, cluster pathfinding, and motion generation. First, motion clustering maps input motions to discrete clusters. Next, we identify the corresponding clusters in the cluster transition graph and search for a connecting path. In this graph, nodes represent motion clusters, and directed edges indicate valid transitions between them. The resulting path determines both the transition length and a guide sequence that informs motion generation. Finally, the path and input motions are provided to a Transformer encoder-based motion generator to produce the final transition poses. Experimental results demonstrate that our method adaptively adjusts the motion length and successfully generates plausible transitions between distinct motions, such as crawling, basketball shooting, and slow locomotion. We also show that using a graph structure effectively estimates transition durations and produces high-fidelity results compared to methods that assume a fixed transition length, or directly compute the time.

1 Introduction

Existing stitching struggles with distinct motions: interpolation can require manual adjustments and still produce artifacts, while a neural approach assumes similar inputs and fixed-length data. This paper uses a cluster transition graph to guide variable-length transitions and motion generation.

1 Introduction

  • Interpolation-based stitching can require careful alignment and frame-by-frame transition-point selection, yet still produce foot sliding or body floating for dissimilar motions.Such blending tends to produce natural results when the input motions are sufficiently similar.Source: §1 · p. 2
  • Kim et al. [2023] generate a connected sequence with the aligned input's length, but their KNN-based method assumes sufficiently similar motions and fixed-length training data.These constraints make complex transitions such as running to lying down difficult, where intermediate motions may be needed.Source: §1 · p. 2
  • The proposed method maps inputs to clusters, finds a shortest path using motion-feature costs, and feeds that path with the inputs to a Transformer encoder-based generator.The output transition sequence includes a destination for aligning the second input motion.Source: §1 · p. 2
  • The cluster transition graph guides variable-length stitching by finding intermediate motion states and providing a guide that reduces ambiguity during generation.The path determines the transition length; the generator also predicts the root destination needed to align the second motion.Source: §1 · p. 2

2 Related Work

Prior work addresses motion stitching through blending, variable-length transitions, quantization, graph traversal, and in-betweening. This paper positions cluster-based pathfinding and guide-conditioned generation as alternatives to manual organization, pose-distance graph construction, and explicit timing choices.

2.1 Motion Stitching

  • Blend durations matter: overly short durations cause abrupt velocity changes, while overly long durations smooth high-frequency details; this method estimates transition length through graph traversal.Prior work computes blend lengths using geodesic distance or joint velocity [Wang et al. 2008].Source: §2.1 · p. 2
  • Li et al. [2008] minimize an elastic L-score and Koyama et al. [2019] insert one-hop pose sequences; this method generalizes the idea with multiple motion clips selected through a cluster graph.This is presented as accommodating a wider range of motion types.Source: §2.1 · p. 2
  • Kim et al. [2023] use KNN-generated locomotion pairs for an RNN refiner, whereas this method targets diverse motion pairs using dataset-derived clusters and variable-length transitions.Interpolation-based pose blending remains a widely adopted practical technique, with Inertialization Blending applying scaled pose displacement during a user-set interval [Bollo 2018].Source: §2.1 · pp. 2–3

2.2 Motion Quantization

  • VQ-VAE has been used to tokenize motion for text or control tasks; here it groups short motion segments into finite clusters without manual labeling.This clustering use follows Cho et al. [2021].Source: §2.2 · p. 3

2.3 Graph-based Motion Synthesis

  • Unlike motion graphs built from pairwise pose distances, this method uses one node per motion cluster and connects clusters when a valid transition exists.Other approaches manually annotate or divide motions into groups [Min et al. 2012; Tao et al. 2023].Source: §2.3 · p. 3

2.4 Motion In-betweening

  • In-betweening methods range from RNNs using context frames and a target keyframe to sparse-context, mixture-of-experts, skeletal Transformer, and diffusion approaches.Examples include Harvey et al. [2020], He et al. [2022], Starke et al. [2023], and Bae et al. [2025].Source: §2.4 · p. 3
  • Transformer in-betweening work predicts whole sequences or refines coarse poses; this generator instead incorporates a guide from pathfinding rather than zeroing all in-between frames.Other work learns time warping jointly with pose predictions, while this approach automatically determines total motion length and the second motion's transformation [Goel et al. 2025].Source: §2.4 · p. 3

3 Overview

Neural Motion Stitching connects two approximately aligned canonical-space motions through clustering, cluster-graph pathfinding, and motion generation. The path sets the number of motion blocks, while the generator combines the inputs and path into the output.

3 Overview

  • Inputs are raw motions of T_l + 1 frames in canonical space, approximately identity-aligned, with the first pose facing forward and the root beginning at the ground-plane origin.The extra frame supports displacement computation across T_l frames.Source: §3 · pp. 3–4
  • Motion clustering maps each input to a cluster, then pathfinding connects their cluster IDs as start and goal nodes in a transition graph.A shortest-path algorithm finds the cluster path m.Source: §3 · p. 4, Figure 2 · p. 4
  • The cluster path determines the number of motion blocks, and the generator uses both input motions and the path to produce the output motion.The output length is T = T_l · N_m; each motion block has 16 frames, approximately 0.3 seconds, at 60 FPS.Source: §3 · p. 4
  • The framework uses 22 joints for full-skeleton generation and 9 key joints for VQ-VAE training; orientations are represented in 6D format.The joint selection is illustrated in Figure 3.Source: Figure 3 · p. 4, §3 · p. 4
  • Figure 4 presents CVQ-VAE training for cluster construction and the later cluster-assignment process.These are the clustering stage's training and inference steps.Source: Figure 4 · p. 4

4 Motion Clustering

Motion clustering uses an unsupervised CVQ-VAE to assign motion blocks to discrete codes, with decoded cluster clips serving as representatives. New motions are assigned among candidate codes using both latent similarity and root-position comparison.

4 Motion Clustering

  • CVQ-VAE encodes each motion block, assigns it to the nearest code by cosine similarity, and decodes that code into a representative cluster clip.Each code denotes a unique motion cluster.Source: §4 · p. 4
  • New motion blocks are encoded and their top three cosine-similar codes are decoded; root-position comparison selects the final cluster ID.This candidate-based assignment is intended to reduce outliers and false graph-edge connections.Source: §4 · p. 5
  • The codebook uses 2048 codes with 100% utilization, yielding 2048 clusters in the reported implementation.Online reinitialization is used to maximize codebook usage.Source: §4 · p. 5

4.1.1 Data preprocessing.

  • Preprocessing uses 9 selected joints, including knees, and represents each pose through joint translational and angular displacements across frames.Clips contain T_l + 1 frames and are windowed with a stride of T_l/2.Source: §4.1.1 · p. 5

4.1.2 Network architecture.

  • The CVQ-VAE uses a convolutional autoencoder with nine residual 1D convolutional layers to compress the time dimension to T_l/4 before codebook quantization.The 64-dimensional selected code is decoded through a symmetric architecture to reconstruct the motion block.Source: §4.1.2 · p. 5

4.1.3 Loss function.

  • Training combines reconstruction, world-position, vector-quantization, and contrastive losses, weighted respectively by 0.01, 1.5, 1.0, and 1.0.The contrastive loss promotes sparse codes and distinct clusters; its negatives are the batch's bottom 50% latent vectors by similarity.Source: §4.1.3 · p. 5

5 Cluster Pathfinding

The cluster transition graph encodes observed transitions between motion clusters, and A* finds a low-cost path whose intermediate nodes determine the transition sequence. Edge and path costs account for cluster-code similarity and boundary pose differences.

5.1 Graph Construction

  • Consecutive non-overlapping motion-capture blocks define directed edges between their assigned clusters, treating observed adjacency as a plausible transition.Repeating this construction across training sequences yields the graph.Source: §5.1 · pp. 5–6

5.2 Graph Traversal

  • A* selects a path from the start cluster to the goal using accumulated edge costs and a heuristic estimate of remaining cost.The heuristic compares the current cluster with the goal using the same latent-code and boundary terms.Source: §5.2 · p. 6
  • The shortest path inserts its intermediate nodes as natural motion blocks, determining transition length while avoiding unnecessary states.Figure 5 marks the selected shortest-path edges with yellow arrows.Source: §5.2 · p. 6, Figure 5 · p. 6
  • Each edge cost combines latent-space cluster-code dissimilarity with a boundary mismatch between neighboring clips.The boundary term measures root height, orientation, translational displacement, and angular displacement differences.Source: §5.2 · p. 6

6 Motion Generation

Motion generation combines the input motions and a cluster-derived guide in a Transformer encoder to synthesize transition poses and predict the destination root transformation. Its inputs, variable-length training pairs, and losses support the generation and alignment process.

6 Motion Generation

  • The Transformer processes input motions and a cluster path, predicts transition poses and a root destination vector, then combines these with the original motions.The destination vector is used to transform and align the second input motion when assembling the connected result.Source: Figure 6 · p. 6, §6 · pp. 6–7

6.1.1 Data format.

  • Each motion is represented as a sequence of pose vectors, using a full-joint representation for both inputs, generated transitions, and final output.The pose representation includes root-relative joint orientations.Source: §6.1.1 · p. 7

6.1.2 Input processing.

  • Input processing combines a path-derived guide with a target sequence that preserves the first motion, zeros the transition frames, and aligns the second motion.The guide's root transformation at T_dest is copied to align the second motion; guide length determines final sequence length.Source: §6.1.2 · p. 7, Figure 7 · p. 7

6.1.3 Training dataset.

  • Training pairs use motion-capture clips of 3–8 motion blocks, allowing supervised learning across varying sequence lengths.The clips are segmented with a sliding window, and ground truth is used instead of cluster pathfinding to construct paired inputs.Source: §6.1.3 · p. 7

6.2 Motion Generator

  • A single Transformer encoder maps the input sequence to transition frames and a root destination vector; adversarial training targets boundary popping and foot sliding.Relative positional encoding is used without absolute positional encoding, allowing inputs longer than training samples.Source: §6.2 · pp. 7–8, §6.2.2 · p. 8

6.2.2 Loss function.

  • The generator loss includes transition and forward-kinematics terms, while adversarial training uses a discriminator over newly segmented motion blocks.The transition loss is evaluated on transition frames, and the destination loss supervises the predicted root transformation.Source: §6.2.2 · p. 8

7 Experiments

Experiments evaluate adaptive stitching across diverse and unseen motions, direction control, ablations, scalability, and comparisons with prior methods. The reported results show plausible transitions and strong performance across the tested comparisons.

7.1 Motion Stitching on Diverse Motions

  • Transition length adapts to the input pair and foot phase: starting the second motion with the right foot yielded 80 frames, versus 64 frames with the left foot.The longer case inserts an additional step; single-pose inputs also produced longer transitions as the character gradually adjusted its joints.Source: §7.1 · pp. 8–9, Figure 9 · p. 9, Figure 10 · p. 9

7.2 Motion Stitching on Unseen Motions

  • Without additional training, the method generated plausible transitions between unseen motions from six datasets, including locomotion, dance, gestures, and motions with rich ground contact.The authors attribute robustness to mapping unseen motions to known clusters and using training-observed cluster clips as guides.Source: §7.2 · p. 9, Figure 12 · p. 10

7.3 Transition Direction Control

  • Changing the guide through direction-aware cluster pathfinding steered identical input pairs forward, left, or right while matching the second motion’s phase.The direction control term was added to the pathfinding cost and heuristic, without retraining the generator.Source: §7.3 · p. 9, Figure 13 · p. 10

7.4 Ablation Study

  • Removing guide input led to sliding and less accurate results, while removing destination loss caused boundary popping from root-transformation discontinuity.The destination loss predicts the next-frame root; the full model was closer to ground truth than the model without that loss.Source: §7.4 · p. 10, §7.4.1 · p. 10, Table 1 · p. 10

7.5.1 Scalability of cluster transition graph.

  • As training data grew across LAFAN1, InterAct, and 100Style, codebook size increased only moderately and graph memory and pathfinding time scaled with codebook size.The authors report that CVQ-VAE summarizes motion from multiple sources into a finite set of codes, keeping graph size manageable.Source: §7.5.1 · pp. 10–11, Table 2 · p. 11

7.5.2 Utilizing large dataset.

  • NMS_large trained on InterAct and 100Style without LAFAN1 connected unseen input combinations, including sitting-to-dinosaur-walking and kicking-to-shrugging transitions.The codebook size increased modestly despite the larger dataset; the authors attribute this to relatively limited root-movement variation in those datasets.Source: §7.5.2 · p. 11, Table 3 · p. 11, Figure 14 · p. 11

7.6.3 Qualitative comparison.

  • In qualitative comparisons, our method produced gradual intermediate poses, while fixed-length methods showed abrupt changes or smoothed, discontinuous, or implausible results on challenging and unseen inputs.BC-based methods could also underestimate transition time; even with similar predicted lengths, our method produced smooth transitions without discontinuities.Source: §7.6.3 · p. 13

7.6.4 Quantitative comparison.

  • Our method achieved the best scores on LAFAN1_within and significantly lower jitter on LAFAN1_random and Mixamo + 100Style, while maintaining comparable FID and detail distance.Against BC-based approaches, it outperformed all except SILK + BC on Mixamo + 100Style; BC-based combinations did not yield significant performance improvements.Source: Table 4 · p. 13, §7.6.4 · p. 14

8 Discussion

The discussion identifies dataset coverage, pathfinding choices, and physical constraints as important boundaries on motion quality and applicability. Longer transitions generally correspond to more dissimilar inputs, but can lose detail; graph memory also grows with dataset size.

8.1 Effect of Training Dataset

  • Limited training-data diversity can map inputs to unsuitable clusters, while out-of-distribution motions can cause popping at the first boundary.The LAFAN1-trained example lacked clusters related to swimming; postprocess blending can mitigate first-boundary popping.Source: §8.1 · p. 14

8.2 Pathfinding-based Guide Generation

  • Pathfinding is deterministic, lacks semantic conditioning, and requires manually tuned weights so guides resemble training samples; root-related features were prioritized.With fixed cost settings, the method produces the same transitions, although adjusting costs can produce diverse transitions from the same inputs.Source: §8.2 · pp. 14–15

8.3 Realtime Application

  • A* length estimation is the main inference-time bottleneck against TS- and SILK-based methods, motivating a learned graph-and-search replacement.The proposed future design could also automate weight tuning.Source: §8.3 · p. 15

8.4 Variation of Transition Lengths

  • Output length generally increased with input dissimilarity, while quality metrics worsened in longer bins; the longest-bin transitions remained plausible but sometimes lost detail or exhibited foot sliding.The longest bin included challenging pairs such as lying down to aiming and kneeling to running.Source: §8.4 · p. 15, Table 6 · p. 15

8.5 Graph Scalability

  • Graph memory grows with training-dataset size, though the method generalizes to unseen motion combinations when trained on sufficiently diverse data.Pruning rarely used edges or using a hierarchical graph are proposed ways to reduce memory.Source: §8.5 · pp. 15–16

8.6 Physical Plausibility

  • Without physics constraints, transitions may violate real-world physical principles, including lifting the hips without appropriate support from other body parts.The authors suggest physics-based terms or simulation to improve realism.Source: §8.6 · p. 16

9 Conclusion

The conclusion presents variable-length, graph-guided stitching as a way to connect a wide range of motions, with guides reducing generation ambiguity and pathfinding costs remaining adjustable without additional training. The appendices describe root-position-based cluster assignment and a special guide construction for directly connected paths.

9 Conclusion

  • The framework uses cluster pathfinding to identify intermediate motion states and guide variable-length transitions, reducing generation ambiguity and allowing path-cost control without additional training.The authors identify more diverse, balanced training data and physics-based constraints as future directions for output quality and realism.Source: §9 · p. 16

A Cluster Assignment

  • Cluster assignment selects among the nearest code candidates by comparing their clips’ root positions with the input, reducing outlier assignments that can create false graph edges.The reported candidate count was n = 3.Source: A Cluster Assignment · p. 18

B Directly Connected Paths

  • When start and goal clusters connect directly, the guide uses an interpolated block of the minimum transition length instead of cluster clips.The resulting guide contains N_b + 1 motion blocks.Source: B Directly Connected Paths · p. 18

C.0.1 CVQ-VAE training details.

The appendix reports the CVQ-VAE and motion-generator training setup and describes how pathfinding weights were tuned using normalized feature distances and generated-motion quality.

C.0.1 CVQ-VAE training details.

  • CVQ-VAE was trained on LAFAN1 motion blocks for 100 epochs with batch size 64, taking about 2.2 hours on an RTX 3090.Training used AdamW with a 10^-4 learning rate, cosine decay, and a 10^-5 minimum.Source: §C.0.1 · p. 18

C.0.2 Cluster pathfinding weight tuning.

  • Pathfinding weights were tuned by evaluating generated-motion quality after normalizing precomputed pairwise distances for cluster codes and clips.The motion generator was trained on variable-length LAFAN1 subsequences for 100 epochs; adversarial training began after warm-up.Source: §C.0.2 · p. 18, §C.0.3 · p. 18

D Effect of Guide for Direction Control

The guide input improves responsiveness to directional control when significant turns are required. With identical input motions and control vectors, the guided model follows the control more closely than the model trained without the guide.

D Effect of Guide for Direction Control

  • With significant directional changes, the guided model adhered more closely to control vectors and made sharper body turns than the model trained without S_g.Both models performed similarly under forward control; the comparison visualizes sequences every 16 frames from a top view.Source: D Effect of Guide for Direction Control · p. 18, Figure 18 · p. 18

E Additional Comparison

The comparison adapts a diffusion model for in-betweening and evaluates it against the proposed method. The reported results favor the proposed method in visual quality and in connecting inputs with minimal jitter while maintaining low FID and detail distance.

E Additional Comparison

  • CAMDM_inb produced overly smoothed motion and boundary discontinuities, and adding boundary control did not improve its performance.It also struggled to match the second motion in one sample and retained boundary discontinuities in another.Source: E Additional Comparison · p. 19
  • The proposed method connects the inputs with minimal jitter while maintaining low FID and detail distance across datasets, whereas CAMDM_inb exhibits high jitter.CAMDM_inb has moderate FID and detail distance despite its high jitter.Source: E Additional Comparison · p. 19
  • The proposed method achieved the highest user-study score, while CAMDM_inb scored similarly to CondMDI.The authors report better overall visual quality for their method than for CAMDM_inb.Source: E Additional Comparison · p. 19

F Hyperparameter Ablation

The ablation tests how cluster-pathfinding cost weights affect evaluation metrics and transition lengths. Although no tested configuration consistently outperforms the default across metrics, weight changes produce different paths and motion lengths.

F Hyperparameter Ablation

  • The sensitivity test independently halved and doubled five pathfinding weights across 1,000 sampled LAFAN1_within motion pairs, with λ_bnd fixed at 1.The remaining weights were held fixed during each change.Source: F Hyperparameter Ablation · p. 19
  • No tested cost-weight configuration consistently outperformed the default across all evaluation metrics.Some alternatives scored slightly better on individual metrics.Source: F Hyperparameter Ablation · p. 19, Table 8 · p. 21
  • Changing cost weights altered selected cluster paths and transition lengths; High λ_h produced 112 frames versus 64 for the default and often lingered before the second input.The default was selected over High λ_h because it achieved the best detail distance score.Source: F Hyperparameter Ablation · p. 19, Figure 20 · p. 19
Loading 2609.37167v1…