4DEquine: Disentangling Motion and Appearance for 4D Equine Reconstruction from Monocular Video

• Prior animal reconstruction methods either lack explicit structural priors or depend on image-aligned textures, while optimization-based approaches are costly and require clean 360-degree turntable sequences.Template-free radiance-field and point-cloud methods [45] [46] [11] can lack geometric detail; SMAL-based methods [58] [60] have strong priors but alignment-sensitive textures, while implicit duplex fields [27] [55] and mesh-bound Gaussians [13] remain optimization-based.Source: §1 · p. 1
• 4DEquine disentangles 4D reconstruction into per-frame dynamic motion and a canonical static appearance, connecting them through VAREN and binding Gaussian point clouds to its mesh faces.Source: §1 · p. 1
• AniMoFormer uses a spatio-temporal transformer over consecutive frames plus post-optimization for smooth, pixel-aligned motion, and trains on VarenPoser, a synthetic 4D VAREN dataset with diverse virtual camera trajectories.VarenPoser meshes are fitted to marker-based PFERD motions and textured using MV-Adapter [6]; training only on this dataset yields realistic motion on real-world data.Source: §1 · pp. 1–2
• EquineGS predicts an animatable Gaussian avatar from one representative image, trained on VarenTex multi-view diffusion images, then animates VAREN-attached Gaussian primitives with linear blend skinning.The single-image design exploits within-clip appearance consistency and incomplete real-world viewpoint coverage; VarenTex provides higher-quality appearance images than VarenPoser.Source: §1 · p. 2
• On AiM, full 4D reconstruction by 4DEquine is superior to optimization-based GART [13] and 4D-Fauna, and outperforms GVFDiffusion [50].Source: §1 · p. 2
• On unseen equine videos from APT-36K [47] and AiM, 4DEquine achieves state-of-the-art motion recovery on PCK@0.05, PCK@0.1, Accel, and Chamfer Distance.Source: §1 · p. 2

4D Reconstruction Methods

• Even feed-forward 4D methods such as Monst3R [51] and Page-4D [56] struggle with incomplete observations; full horse geometry and appearance require 360-degree video.Earlier scene-specific methods are time-consuming and fail to recover unseen surfaces.Source: §2 · p. 2

Equine Geometry Estimation

• Image-based methods including SMALify [2], AniMer [1] [22], and GenZoo estimate SMAL pose and shape but do not address temporal 4D reconstruction or use VAREN.hSMAL [14] and Dessie [16] add horse-specific details to SMAL but still lack realism; VAREN is learned from multiple high-resolution horse scans.Source: §2 · p. 2

Animal Avatar Reconstruction

• Per-instance avatar methods such as BANMo [45] and RAC [46] are slow and lack fine detail, while scalable feed-forward approaches trade shape realism for diversity, leaving high-quality equine avatar generation lacking.Optimization-based avatar methods also require substantial viewpoint coverage; 3D-Fauna [17] and 4D-Fauna produce diverse quadruped avatars, while MagicPony [39] and Ponymation [29] learn animatable horses.Source: §2 · p. 2

The VAREN Model

• VAREN learns pose-dependent muscle deformation from scans of 50 horses and maps shape, pose, and translation parameters to a high-resolution mesh.It uses shape β ∈ R39, pose θ ∈ R38×3 for 38 joints, and translation γ ∈ R3; unlike SMAL [58], it models non-rigid muscle deformation.Source: §3 · p. 3

3D Gaussian Splatting (3DGS)

• 3DGS [12] represents scenes as 3D Gaussians whose projected primitives are composited by depth-sorted, view-ordered alpha blending.Each primitive has position, opacity, scale, rotation, and directional appearance represented by Spherical Harmonics coefficients.Source: §3 · p. 3

4.1.1. VarenPoser Dataset Construction

• VarenPoser provides 1,171 synthetic video clips, built from PFERD horse motions with randomized shapes, generated textures, and simulated camera trajectories.Clips are segmented into 600-frame sequences and rendered at 512 × 512 resolution and 60 FPS; camera paths include fixed, dolly, and orbit trajectories.Source: §4.1.1 · p. 3

4.1.2. AniMoFormer

• AniMoFormer combines a spatio-temporal transformer with post-optimization to recover temporally smooth, image-aligned pose and shape sequences from video.The transformer uses a fixed 16-frame training window and sliding windows for arbitrarily long inference; post-optimization aligns rendered meshes to pseudo-ground-truth keypoints and masks while regularizing pose.Source: §4.1.2 · pp. 3–4

4.2.1. EquineGS

• EquineGS predicts canonical Gaussian attributes from a single image by fusing image and VAREN-template point-cloud features with a dual-stream transformer decoder.The upsampled template supplies 55,486 initial Gaussian positions; predicted Gaussians are deformed using VAREN pose and shape parameters, then trained with image, silhouette, and regularization losses.Source: §4.2.1 · pp. 4–5

4.2.2. VarenTex Dataset

• VarenTex addresses the need for photorealistic multi-view training images by using UniTex to generate 150K images at 512 × 512 resolution.The generation pipeline conditions UniTex on mesh-derived normal and canonical-coordinate maps plus a ControlNet-generated reference image.Source: §4.2.2 · p. 5

5.1.1. Datasets

• AniMoFormer is trained only on VarenPoser and evaluated on horse subsets of APT-36K and AiM plus synthetic VarenPoser; EquineGS trains only on VarenTex and is tested on AiM horses and zebras.The geometry test split contains three clips and 543 frames; the AiM pose and shape subsets contain 30 APT-36K clips (402 frames) and 10 AiM clips (453 frames). Avatar tests include 35 horse sequences and 10 zebra sequences.Source: §5.1.1 · pp. 5–6

5.1.2. Evaluation Metrics

• Pose and shape evaluation uses PCK, acceleration error, and Procrustes-aligned Chamfer Distance; avatar rendering uses PSNR, SSIM, and LPIPS.Source: §5.1.2 · p. 6

5.1.3. Baselines

• AniMoFormer is compared with Dessie, AniMer, GenZoo, 3D-Fauna, and 4D-Fauna; EquineGS is compared with 3D-Fauna, 4D-Fauna, GVFDiffusion, and GART variants.GART uses all training frames, while GART∗ uses four; EquineGS uses only the first training frame, and all methods receive AniMoFormer pose and shape sequences with a VAREN GART template.Source: §5.1.3 · p. 6

5.1.4. Training Details of 4DEquine

• Detailed 4DEquine training settings are provided in the supplementary materials.Source: §5.1.4 · p. 6

5.2. Qualitative Evaluation of 4DEquine

• Qualitatively, 4DEquine recovers better-aligned geometry and stable appearance from limited views, while EquineGS generates a novel-pose avatar from only the video’s first frame.Compared methods show distorted shapes, artifacts, floaters, or pose ambiguities; 4DEquine also produces plausible donkey reconstructions from single images despite horse-only training.Source: §5.2 · pp. 6–7, Figure 4 · p. 6, Figure 5 · p. 7

5.4. Ablation Study

• Removing post-optimization worsens image alignment and rendering metrics, while removing the temporal transformer increases acceleration error.Post-optimization aligns reconstructions more closely to 2D image evidence; the temporal transformer contributes to motion smoothness.Source: §5.4 · p. 7, Table 3 · p. 8, Table 4 · p. 8
• DSTG consistently outperforms cross-attention, while the sparse 13,873-point avatar can have slightly higher PSNR but renders porous and incomplete.The full model uses 55,486 subdivision-generated points; the base-mesh variant lacks enough points to form a contiguous surface.Source: §5.4 · pp. 7–8, Table 4 · p. 8, Figure 6 · p. 8

5.3. Quantitative Analysis

• AniMoFormer achieves the highest PCK and lowest Accel and CD, while 4DEquine improves horse-subset SSIM and LPIPS and beats Few-shot GART across all metrics.On the horse subset, fully optimized GART has slightly higher PSNR; on zebra, 4DEquine outperforms all competitors, including GART, across all three metrics. Reconstruction takes 11 seconds per frame on one NVIDIA A100, compared with GART’s fixed 15 minutes.Source: §5.3 · p. 7, Table 1 · p. 8, Table 2 · p. 8

Summary

• 4DEquine disentangles reconstruction into AniMoFormer motion estimation and single-image EquineGS appearance reconstruction, supported by the VarenPoser and VarenTex synthetic datasets.AniMoFormer combines smooth spatio-temporal regression with post-optimization for 2D alignment; EquineGS generates a high-fidelity, animatable 3D Gaussian avatar without costly per-video optimization.Source: §6 · p. 8
• The authors report state-of-the-art performance and strong zero-shot generalization to donkeys and zebras.Source: §6 · p. 8

Limitations and Future Work

• The synthetic datasets inherit VAREN's limitations in tail and mane physics and appearance, and the approach does not handle dynamic lighting.The authors plan physics-based tail and mane representations, a relighting module, and keyframe fusion in EquineGS to capture unique markings efficiently.Source: §6 · p. 8

7. More Qualitative Results

• For Fig. 7, 4DEquine uses each Internet-collected video’s first frame as input and shows novel-pose animations.Source: §7 · p. 12

8.1.1. Training Stage

• Motion and appearance are trained separately on VarenPoser and VarenTex: AniMoFormer regresses VAREN pose and shape, while EquineGS predicts canonical Gaussian attributes.The paper attributes this synthetic-only training strategy to the scarcity of high-quality 4D equine data.Source: §8.1.1 · p. 12

8.1.2. Inference Stage

• At inference, AniMoFormer processes video with sliding windows, post-optimization aligns geometry to 2D evidence, and EquineGS generates an avatar from one keyframe for LBS-driven animation.The post-optimization uses 2D keypoints and masks; optimized VAREN parameters drive the deformation of canonical Gaussian points into each frame's pose space.Source: §8.1.2 · p. 12

8.2. More Details about VarenPoser Dataset

• VarenPoser assigns generated textures from 500 Internet horse images across 1,171 clips, limiting each texture to three clips, and varies camera viewpoints and trajectories.Initial pitch and azimuth are sampled from (−15°, 15°) and (−180°, 180°); fixed, orbit, and dolly trajectories have probabilities 0.4, 0.3, and 0.3. Videos are 60 fps at 512×512, with VAREN parameters, 21 3D and 21 2D keypoints, visibility flags, and masks annotated per frame.Source: §8.2 · p. 12

8.3.1. Implementation Details

• AniMoFormer uses a ViT-H spatial transformer initialized from AniMer, temporal encoding, occlusion and frame-rate augmentation, and two-stage keypoint-then-silhouette post-optimization.Training takes 100,000 steps on one RTX 4090 GPU (approximately 10 hours); each post-optimization stage uses 100 iterations, first refining VAREN parameters and camera against keypoints, then refining shape against the silhouette.Source: §8.3.1 · pp. 12–13

8.4. More Details about EquineGS

• EquineGS keeps its DINOv3 backbone frozen and trains from one randomly selected view with four additional views for supervision.Training uses 448×448 images for 100,000 steps with gradient accumulation of 4 across eight RTX 4090 GPUs, requiring approximately three days.Source: §8.4 · pp. 13–14

8.5. Visualization for VarenTex

• The paper presents representative visual samples from VarenTex.These samples are shown in Figure 9.Source: §8.5 · p. 14, Figure 9 · p. 14

9.1. Comparison with GART

• On AiM, AniMoFormer outperforms GART [13] across all metrics; even initialized from AniMoFormer, GART’s photometric optimization degrades pose by favoring pixel-color alignment over geometric fidelity.The authors present this as evidence for their decomposition design.Source: Table 6 · p. 14, §9.1 · p. 14

9.2. Comparison with AniMer + PO

• AniMoFormer outperforms AniMer + PO [22], which the authors attribute to temporal attention and VAREN’s greater expressiveness than SMAL.The comparison is reported in Table 7.Source: §9.2 · p. 14, Table 7 · p. 14

9.3. Ablation on the Number of Input Frames

• As window size N increases from 4 to 8 to 16, PCK@0.05 rises from 46.8 to 47.5 to 47.8, while acceleration error falls from 27.0 to 25.8 to 25.7.N = 32 causes an out-of-memory error, so sliding-window inference enables processing videos longer than 16 frames.Source: §9.3 · p. 14

9.4. Qualitative Results of Challenging Pose

• Figure 10 presents rearing and sitting poses as challenging examples used to demonstrate 4DEquine’s robustness.The rearing pose is shown on the left and the sitting pose on the right.Source: §9.4 · p. 14, Figure 10 · p. 14

10. Failure Cases and Discussion

• Under severe occlusion or truncation, AniMoFormer still reconstructs geometry, but EquineGS degrades appearance at unseen areas.Figure 11 contrasts the truncated or occluded input, AniMoFormer output, EquineGS output, and a reference frame showing the horse’s true appearance.Source: §10 · p. 14, Figure 11 · p. 15
• EquineGS keyframes should avoid images in which the horse is significantly occluded or truncated.The authors identify this as necessary when selecting keyframes for EquineGS.Source: §10 · p. 15
• Future work will develop a method for the efficient fusion of multiple keyframes to address these challenges.Source: §10 · p. 15