EgoX: Egocentric Video Generation from a Single Exocentric Video

• Egocentric perspectives offer immersive entertainment and support robotics and AR/VR tasks such as imitation, reasoning, and interaction.Source: §1 · pp. 1–2
• Extreme camera shifts create large unseen regions and little view overlap, requiring plausible synthesis and selective use of view-related rather than unrelated content.Source: §1 · p. 2
• Prior approaches simplify generation by requiring a first egocentric frame or four simultaneous exocentric views.Source: §1 · p. 2
• EgoX generates from one exocentric video using pretrained video-diffusion knowledge, unified exo/ego conditioning, lightweight LoRA adaptation, and geometry-guided self-attention.Source: §1 · p. 2
• EgoX is reported to outperform previous approaches by a large margin and achieve state-of-the-art performance on diverse, challenging exo-to-ego benchmarks.Source: §1 · p. 2

2.1. Exo-to-Ego View Generation

• Channel-wise exocentric feature concatenation [28] [32] faces pixel-wise misalignment, weakening conditioning use and risking poor geometry understanding, overfitting, or output degradation.Source: §2.1 · pp. 2–3
• Cross-attention conditioning in 4Diff prevents use of powerful pretrained diffusion weights, limiting generalizability and synthesis quality.Source: §2.1 · p. 3
• EgoExo-Gen [44] needs the first egocentric frame, while Exo2Ego-V uses four exocentric videos and separate spatial and temporal modules; EgoX instead uses pretrained weights with one input.Source: §2.1 · p. 3

2.2. Video Diffusion Models

• Video diffusion advances yield realistic, coherent sequences [1, 5, 6, 14, 40, 46], with conditional generation demonstrated using depth maps or static images [19] [22] [45].Source: §2.2 · p. 3
• Camera-controlled video methods condition directly on camera extrinsics [3] [4] [30] [43] [47] or render a lifted 3D scene from a target pose [18] [25] [36] [42] [50].Source: §2.2 · p. 3
• These camera-control methods target modest viewpoint changes and struggle with extreme exocentric-to-egocentric pose differences.Source: §2.2 · p. 3

3.1. Egocentric Point Cloud Rendering

• Per-frame monocular depth is aligned to temporally smooth video depth; dynamic objects are masked before aligned depth is converted to a point cloud and rendered from the egocentric viewpoint.The rendered prior provides pixel-wise RGB information and camera-trajectory cues for viewpoint alignment.Source: §3.1 · p. 3

3.2. Exo-to-Ego View Generation with VDM

• The diffusion model encodes both videos as latents, concatenating the ego prior channel-wise and the exocentric latent width-wise with the noisy target latent.The clean exocentric latent stays fixed during denoising; after sampling, only the egocentric part is decoded.Source: §3.2 · pp. 3–4

3.3. Geometry-Guided Self-Attention

• Geometry-guided self-attention adds 3D direction-similarity bias between egocentric queries and exocentric keys, recomputing pairwise directions as the ego camera center changes across frames.Similar directions receive higher scores, while opposite directions receive lower scores.Source: §3.3 · pp. 4–5, Figure 4 · p. 5

4.1. Experimental Setup

• The evaluation compares representative exo-to-ego and camera-control baselines, using image, object-level, and video criteria.The setup also includes 4,000 Ego-Exo4D clips, with 3,600 for training, 400 for testing, and 100 unseen clips for generalization evaluation.Source: §4.1 · pp. 5–7, §4 · p. 5

4.2. Qualitative Results

• Qualitative comparisons report that EgoX produces geometrically accurate, coherent videos and generalizes to unseen in-the-wild scenes from one exocentric view.Exo2Ego-V is inapplicable in the in-the-wild setting because it requires four exocentric views; Trajectory Crafter struggles with large camera translations, while Wan VACE and Wan Fun Control mismatch geometry and include irrelevant exocentric content.Source: Figure 5 · p. 6, §4.2 · p. 7

4.3. Quantitative Results

• EgoX achieves the best overall image- and object-criteria performance; Wan VACE scores highest on temporal smoothness and flicker but produces overly static videos.EgoX is reported to balance spatial fidelity and motion realism; it also achieves the best results across image metrics despite the challenge of synthesizing unseen regions.Source: §4.3 · p. 7, Table 1 · p. 7

4.4. Ablation Study

• Removing GGA, the egocentric prior, or the clean latent degrades generation quality and geometric consistency in the ablation comparisons.Without GGA, attention spreads to unrelated regions; without the ego prior, the model lacks pixel-wise and camera-trajectory guidance; without the clean latent, object details can blur or disappear.Source: Table 2 · p. 7, §4.4 · p. 8, Figure 6 · p. 8

5. Conclusion

• EgoX generates egocentric videos from a single exocentric input and generalizes across diverse scenes.Its conditioning combines exocentric and egocentric priors, while lightweight LoRA adaptation, clean latent representations, and geometry-guided self-attention support coherent, high-fidelity generation.Source: §5 · p. 8
• The method combines viewpoint-aligned guidance with broader exocentric scene context to support geometric consistency.The conclusion describes width- and channel-wise concatenation of the two priors alongside geometry-guided attention.Source: §5 · p. 8
• The framework requires an egocentric camera pose, which users can provide interactively.Automatic head-pose estimation is identified as a valuable future direction.Source: §5 · p. 9

F.1. GGA Implementation Detail

• GGA uses pixel-level 3D directions averaged over each 4×16×16 patch to provide geometric cues in latent-space attention.The geometric terms are precomputed, and separate ego-to-exo and exo-to-ego attention kernels help limit memory and computational costs.Source: §F.1 · p. 12

F.2. Ego Camera Pose for In-the-wild Example

• For in-the-wild rendering, camera extrinsics are selected interactively with Viser after lifting the exocentric video into a 3D point cloud.The paper notes that automatic head-pose trackers or SMPL-based pose estimators could eliminate manual intervention.Source: §F.2 · p. 12, Figure 8 · p. 12
• Without depth alignment, inconsistent frame-to-frame depth values lead to unstable and unexpected camera movements.The comparison renders the egocentric view from 3D point clouds across frames.Source: Figure 9 · p. 12

F.3. Evaluation Detail

• Object-level evaluation segments objects with SAM2, encodes their crops with DINOv3, and matches generated and ground-truth objects using cosine similarity.Only pairs above the appearance threshold τsim = 0.9 are treated as valid correspondences for downstream metrics.Source: §F.3 · p. 12, Object Criteria · pp. 12–13
• Table 3 reports that ablation performance trends on unseen scenes are consistent with those on seen scenes.The table compares results when core components are removed.Source: Table 3 · p. 13
• Location error measures the distance between matched-object bounding-box centers, while bounding-box IoU measures coarse geometric consistency.Contour Accuracy evaluates fine-grained geometric consistency using contours from the ground-truth and generated frames.Source: Object Criteria · p. 13

F.4. Text Prompts

• Text prompts are generated with GPT-4o because the pretrained diffusion model requires prompts for conditioning.The prompt-generation system is listed in Table 6, with examples in Figure 18.Source: §F.4 · p. 13
• Without GGA, attention to events outside the visible region can introduce unwanted events into the generated ego view.The figure contrasts this with GGA's focus on the visible region.Source: Figure 10 · p. 13

G.1. Ablation on Unseen Scene

• Removing geometry-guided attention, the egocentric prior, or clean latents degrades unseen-scene geometric consistency, fidelity, or temporal coherence.The reported trends follow those in seen scenes, and the authors identify all three components as essential for coherent, high-fidelity generation.Source: §G.1 · p. 13

G.2. Point cloud rendering

• Depth alignment corrects frame-wise scale inconsistencies in monocular depth, keeping rendered point clouds stable over time for downstream generation.Without alignment, even static backgrounds shift and can introduce artificial camera motion.Source: §G.2 · pp. 13–14
• Table 4 reports conditioning-strategy and GGA-training ablations, with the integrated approach highest across evaluated metrics.Source: Table 4 · p. 14

G.3. Conditioning Strategy Ablation

• Width-wise fusion preserves the exocentric view’s global structure, while channel-wise fusion injects pixel-aligned egocentric viewpoint information.Source: §G.3 · p. 14
• Reversing the fusion modes weakens warping; fusing both inputs width-wise dilutes the prior, while channel-wise exocentric fusion prevents GGA from operating.Both-width fusion also exceeds memory limits and requires resizing; both-channel fusion was not tested because it requires additional network modules.Source: §G.3 · p. 14
• The proposed conditioning strategy delivers the strongest results, with the best geometric alignment, conditioning reliability, and visual quality.Source: §G.3 · p. 14

H.1. GGA Training Ablation

• Applying GGA during training as well as inference yields sharper details, more stable reconstructions, and more accurate geometry than inference-only GGA.The inference-only setting introduces attention distributions the model was not trained to interpret.Source: §H.1 · p. 14

H.2. Runtime

• GGA adds moderate overhead from computing attention bias at each attention layer, while the egocentric prior incurs a similar cost to the image-to-video versus text-to-video input difference.The clean latent strategy adds no computation; it changes only the noise schedule during denoising.Source: §H.2 · pp. 14–15
• GGA can prevent generation of events outside the visible ego-view region by steering attention away from irrelevant regions.The authors consider this guidance worthwhile given observed qualitative and quantitative improvements, particularly in geometry and appearance.Source: §H.2 · p. 15

H.3. User Study

• EgoX received the most selections across reconstruction accuracy, motion/camera consistency, and overall quality, significantly outperforming all baselines.The study included 19 participants evaluating five methods on unseen-scene and in-the-wild videos.Source: §H.3 · p. 15

H.4. Additional qualitative results

• Time-axis comparisons show natural, high-fidelity egocentric videos with accurate geometry and stable motion; baselines show static output, conditioning failures, artifacts, or distortions.The paper includes additional in-the-wild time-axis examples.Source: §H.4 · p. 15
• EgoX received the highest number of selections across all user-study questions and significantly outperformed the baselines.Source: Figure 11 · p. 15

H.5. Failure example

• Ambiguous, partly occluded arm poses can be misread, such as generating both arms extended when one arm is bent.The caption attributes this to sparse visual evidence and task ambiguity that may challenge human observers too.Source: Figure 12 · p. 15, §H.5 · p. 16
• The qualitative ablations show model variants losing geometric fidelity or detail reproduction, while EgoX produces the highest-quality output.Source: Figure 13 · p. 16
• The example prompt describes both exocentric and egocentric views because the exocentric views are concatenated width-wise.Source: Figure 18 · p. 21