PaperlayerView exact revision on arXiv

Source-linked AI summary

Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animation

Shaowei Liu, Chuan Guo, Bing Zhou, Jian Wang

arXiv:2510.14976v1cs.CVcs.GRcs.RO

TL;DR

Close-contact interactive poses carry cues about interaction dynamics, while existing image-based approaches may model static poses without their temporal dynamics. Ponimator uses conditional diffusion models to generate motion from interactive poses and to generate interactive poses from flexible inputs; experiments report generalization and realism across datasets and applications. Its scope includes short interaction segments and pose-only inputs that omit scene context.

  • Problem

    Close-contact human interactions carry contextual cues, but image-based methods cited by the paper often model static poses without temporal dynamics.

    Source: §1 · p. 2

  • Method

    Ponimator uses a temporal-prior pose animator and a spatial-prior pose generator to create motion from interactive poses and interactive poses from a single pose, text, or both.

    Source: §5 · p. 9

  • Results

    Across datasets and applications, Ponimator exhibits strong generalization and realism; on Inter-X, it achieves FID of 22.6 and contact ratio of 68.1.

    Source: §4.1 · p. 7, §5 · p. 9

  • Takeaways & Limitations

    The interactive-pose prior supports open-world image interaction animation, single-pose interaction generation, and text-to-interaction synthesis.

    Source: §5 · p. 9

    The method focuses on short interaction segments and relies solely on human poses, ignoring scene context.

    Source: §4.5 · p. 9

Abstract

from arXiv · show

Close-proximity human-human interactive poses convey rich contextual information about interaction dynamics. Given such poses, humans can intuitively infer the context and anticipate possible past and future dynamics, drawing on strong priors of human behavior. Inspired by this observation, we propose Ponimator, a simple framework anchored on proximal interactive poses for versatile interaction animation. Our training data consists of close-contact two-person poses and their surrounding temporal context from motion-capture interaction datasets. Leveraging interactive pose priors, Ponimator employs two conditional diffusion models: (1) a pose animator that uses the temporal prior to generate dynamic motion sequences from interactive poses, and (2) a pose generator that applies the spatial prior to synthesize interactive poses from a single pose, text, or both when interactive poses are unavailable. Collectively, Ponimator supports diverse tasks, including image-based interaction animation, reaction animation, and text-to-interaction synthesis, facilitating the transfer of interaction knowledge from high-quality mocap data to open-world scenarios. Empirical experiments across diverse datasets and applications demonstrate the universality of the pose prior and the effectiveness and robustness of our framework.

1. Introduction

Close-contact poses provide cues about spatial relationships, constraints, intent, and likely motion, making them useful anchors for modeling interaction dynamics. Ponimator learns from mocap interactions to animate such poses and generate them from flexible inputs, supporting image-based and text-driven applications.

1. Introduction

  • Close-proximity interactive poses encode spatial relationships, constraints, and intent, often suggesting probable past and future motion.The paper contrasts these poses with non-interactive poses, which are more ambiguous and harder to interpret.Source: §1 · p. 2, Figure 2 · p. 2
  • Ponimator trains on close-contact poses and their surrounding motions from Inter-X [65] and Dual-Human to generate contextual interaction dynamics from interactive poses.The framework uses a conditional diffusion model for pose-to-dynamics generation.Source: §1 · p. 2
  • A second conditional diffusion model generates interactive poses from text, a single pose, or both, enabling partner-pose generation when an image shows only one person.The generated interactive poses can then be animated over time.Source: §1 · p. 2
  • Ponimator applies learned interaction priors to open-domain images and supports short-clip two-person motion generation directly from text.The paper also identifies reaction animation among its diverse applications.Source: §1 · p. 2
  • The stated contributions include generalization of the learned prior to poses extracted from open-world images and interactive-pose generation for diverse applications.These applications include reaction animation and text-to-interaction synthesis.Source: §1 · p. 2

2. Related work

Prior work addresses interactive pose estimation, motion reconstruction, synthesis, and prediction, but the cited approaches leave gaps in modeling close-contact dynamics and flexible interaction inputs. Ponimator positions interactive poses as a bridge between pose understanding and interaction motion.

2. Related work

  • Image-based interaction work includes static-pose estimation and sequence reconstruction, while BUDDI is limited to static pose modeling and overlooks interaction dynamics.The related-work discussion also contrasts Ponimator with methods using physical simulation or specialized interaction-prior designs.Source: §2 · p. 2
  • Ponimator’s overview connects a pose generator, which accepts a single pose, text, or both, with an animator that generates interaction dynamics from static poses.Interactive poses bridge the two components.Source: Figure 3 · p. 3
  • Many two-person motion-generation studies use text but often overlook close-contact dynamics; cited prediction methods also rely on long histories or full individual motions.The paper frames its approach as bridging interactive poses and dynamics forecasting with more flexible inputs.Source: §2 · p. 3

3. Approach

Ponimator organizes interaction animation around close-contact two-person poses, using conditional diffusion models to learn temporal motion and spatial pose priors. Its animator turns interactive poses into motion, while its generator supplies interactive poses from pose or text inputs for several application routes.

3.1. Interactive Pose and Motion Modeling.

  • Close-contact poses at an interaction moment anchor a sequence of past and future poses, providing spatial and temporal cues for modeling interaction motion.The framework factorizes motion modeling into a temporal prior conditioned on the interactive pose and a spatial prior over the pose and body shape.Source: §3.1 · p. 3

3.2. Unveiling Dynamics from Interactive Poses

  • The animator predicts motion residuals around the interactive pose, imputes that pose at its interaction-time index, and uses contact and velocity losses alongside noise augmentation.At inference, the predicted residuals are added back to the interactive pose; the stated losses encourage contact and motion coherence, and pose noise targets robustness to noisy real-world poses.Source: §3.2 · pp. 4–5

3.3. Interactive Pose Generator

  • The pose generator models the spatial prior and uses text and single-person pose conditions, with independently sampled masks allowing different condition combinations.Its training data combines close-proximity poses and surrounding motion from Inter-X and Dual-Human, and diffusion models learn the priors from mocap data.Source: §3.1 · pp. 3–4, §3.3 · p. 5

3.4. Applications

  • Applications estimate an interactive pose from a two-person image, generate a counterpart pose from a single-person image, or synthesize one from text, then animate the result.For image-based animation, adjusting the interaction-time index controls whether the animator predicts future motion, reconstructs past motion, or produces symmetric animation.Source: §3.4 · pp. 5–6

4. Experiments

Experiments assess interactive-pose anchoring across generation, animation, and video applications, reporting benchmark performance and cross-dataset generalization while identifying scope limitations.

4. Experiments

  • Experiments use Inter-X and Dual-Human; evaluation includes FID, precision, recall, diversity, contact-frame ratio, and inter-person penetration.Inter-X uses its official split, while Dual-Human uses a 3:1 training-testing split after non-interactive sequences are excluded.Source: §4 · pp. 6–7

4.1. Effectiveness of Anchoring on Interactive Poses

  • On Inter-X unconstrained synthesis, Ponimator achieves FID of 22.6 and contact ratio of 68.1, outperforming prior methods in motion realism and contact.The comparison evaluates motion quality and physical contact when generating interactions without input conditions.Source: Table 1 · p. 7, §4.1 · p. 7

4.2. Interactive Pose Animation

  • Interactive-pose anchoring, condition encoding, and interactive frames are crucial; the prior also animates unseen-dataset and composed multi-person poses without retraining.Figure 7 shows in-domain and out-of-domain examples, including a composed multi-person interaction.Source: Table 2 · p. 7, Figure 7 · p. 7, §4.2 · p. 8

4.3. Interaction Motion Generation

  • For short-term text-to-interaction synthesis, Ponimator outperforms the end-to-end no-anchor baseline, with better contact and more realistic dynamics than that baseline and InterGen.The evaluation considers 3-second generation, FID, Diversity, and MModality.Source: Table 3 · p. 8, Figure 8 · p. 8, §4.3 · p. 8
  • For single-pose interaction synthesis, the no-anchor baseline struggles while Ponimator generates more natural, diverse motions; it also animates open-world single-person images.The image pipeline estimates a pose, generates an interactive counterpart, and animates the motion.Source: Figure 9 · p. 8, Table 4 · p. 8, §4.3 · p. 8

4.4. Interaction Video Generation

  • Generated interactive motions can serve as intermediate outputs for video synthesis, paired with a textured person reconstructed from an image and an arbitrary second-person texture.The figure illustrates videos produced by applying textures to the generated motion.Source: §4.4 · p. 9, Figure 11 · p. 9

4.5. Limitations

  • The method focuses on short interactions, omits scene context, and can produce contact errors, foot sliding, or inter-person penetration.The authors identify pose inaccuracies and close interactions as sources of the listed issues.Source: §4.5 · p. 9

5. Conclusion

Ponimator combines pose animation and pose generation using conditional diffusion models. It supports several interaction-animation tasks and reports generalization and realism across datasets and applications.

5. Conclusion

  • Ponimator integrates a pose animator and pose generator, using conditional diffusion models for interactive pose animation and generation.The animator uses temporal priors for motion, while the generator applies spatial priors to synthesize interactive poses.Source: §5 · p. 9
  • The framework generates interactive poses from a single pose, text, or both, and supports open-world image animation, reaction animation, and text-to-interaction synthesis.These capabilities combine pose generation with interactive pose animation.Source: §5 · p. 9
  • Across datasets and applications, Ponimator exhibits strong generalization and realism.Source: §5 · p. 9

Abstract

The supplementary material outlines implementation details, limitation analysis, and additional qualitative results across interaction-generation tasks.

Abstract

  • Section A provides implementation details and model architectures for the interactive pose animator and generator.Source: Abstract · p. 13
  • Section B provides a limitation analysis of the current approach.Source: Abstract · p. 13
  • Section C collects qualitative results on long interactions, image animation, and text- and single-pose-to-interaction synthesis.It also includes complex interaction synthesis and interactive pose animation examples.Source: Abstract · p. 13

A. Implementation details

The implementation details describe how interactive poses are identified, how the animator and generator are built, and how training and inference are configured.

A. Implementation details

  • A pose is classified as interactive when the minimum distance between downsampled contact-region SMPL-X mesh vertices is below 1.3 cm.The extracted pose and its surrounding interaction motion are used to train interaction dynamics.Source: A. Implementation details · p. 13
  • Both models use eight Transformer layers and latent dimension 1024; the animator adds temporal attention, while the generator uses spatial attention only.The models use eight-head attention, and the animator’s output residuals are added to the interactive pose.Source: A. Implementation details · p. 13
  • Animator training adds random noise at scale 0.02 to interactive poses and uses contact, relative-orientation, and velocity losses.The noise augmentation is intended to account for real-world pose-estimation inaccuracies.Source: A. Implementation details · p. 13
  • On one A100, interactive pose generation takes 0.21 seconds and the animator generates 3-second motion at 10 fps in 0.24 seconds.The animator speed is compared with InterGen’s 0.76 seconds for the same motion length.Source: A. Implementation details · p. 13

B. Limitation Analysis

Ponimator is constrained by short-horizon modeling and pose-only inputs, with limitations in physical and scene awareness and sensitivity to inaccurate interactive poses.

Short motion modeling

  • Short segments can be chained by reusing the last pose and resetting time, but the interactive-pose prior weakens over longer sequences.The paper also notes that text describing extended interactions can yield partial motions.Source: B. Limitation Analysis · p. 13, Figure 13 · p. 15
  • Without text conditioning, the pose animator cannot capture semantic context or temporal ordering, such as distinguishing “lifting up” from “putting down.”Text-to-interaction prompts describing extended human interactions may therefore produce partial rather than complete sequences.Source: B. Limitation Analysis · pp. 13–14

Inter-person penetrations

  • Without explicit inter-person penetration modeling, close-contact outputs may interpenetrate; strict physical constraints could compromise natural motion quality.The authors identify balancing realistic contact and preventing interpenetration as a challenge.Source: B. Limitation Analysis · p. 14, Figure 12 · p. 14

Lack of scene awareness

  • Without scene awareness, in-the-wild motions can collide with surroundings and appear physically implausible.The method relies solely on human pose information and ignores the surrounding environment.Source: B. Limitation Analysis · p. 14, Figure 12 · p. 14

Inaccurate contact

  • Inaccurate estimated or generated interactive poses can yield poor contact that the pose animator cannot correct, while foot sliding may remain unmitigated without post-processing.The animator primarily models temporal dynamics and depends on the interactive pose for spatial information.Source: B. Limitation Analysis · p. 15

C. Qualitative results

The qualitative results illustrate pose animation across complex, in-the-wild, and out-of-domain settings, alongside pose- and text-conditioned interaction synthesis. They also show the chaining procedure for longer sequences and a comparison in which Ponimator produces better contact and human dynamics than the supplied baselines.

Longer and complex motion generation

  • Longer motions are synthesized by reusing each segment’s last pose as the next starting pose, resetting interactive time to zero, and continuing generation.The visualized results also include high-dynamics and close-contact interactions.Source: C. Qualitative results · p. 15

Two-person image animation

  • From in-the-wild two-person images, Ponimator animates extracted interactive poses into realistic human-human interactions.The described pipeline uses an off-the-shelf pose model and inpainting model for rendering.Source: C. Qualitative results · p. 15, Figure 15 · p. 16

Single-person image interaction generation

  • For single-person images, the pose generator produces plausible partner poses under varied inputs, after which the animator synthesizes interaction dynamics.The examples include Motion-X inputs with optional accompanying text.Source: C. Qualitative results · p. 16, Figure 16 · p. 17

Out-of-domain animation

InterHuman evaluation

  • On InterHuman, Ponimator produces realistic, coherent interactions despite annotation noise and diverse pose distributions, though contact inaccuracies remain.The dataset’s SMPLH annotations are converted to SMPLX and interactive poses are extracted from test sequences.Source: C. Qualitative results · p. 17, Figure 18 · p. 19

Interactive pose animation

  • Compared with InterGen and the random-pose variant, Ponimator produces better contact and human dynamics, while both baselines show severe body penetration and less accurate contact.The comparison is qualitative and shown on Inter-X.Source: C. Qualitative results · p. 17, Figure 19 · p. 19

Text-to-interaction synthesis

  • Short phrases or single words produce realistic two-person interactions, with the intermediate interactive pose supporting consistent interaction and contact.The text-to-interaction visualizations are presented in Figure 20.Source: C. Qualitative results · p. 17, Figure 20 · p. 20

Single-pose-to-interaction synthesis

  • From varied single-person poses, Ponimator generates appropriate interactive poses and captures vivid underlying human dynamics on Inter-X and Dual-Human.The single-pose-guided results are shown in Figure 21.Source: C. Qualitative results · p. 17, Figure 21 · p. 21

Related papers

Loading 2510.14976v1…