PaperlayerView exact revision on arXiv

Source-linked AI summary

WorldCast: World-Consistent Character Video Generation with Large-Angle Camera Control

Lei Cao, Yu Liu, Wenhua Liao, Peng Zhang, Fei Wang, Hao Zhang, Jiajun Le, Daiguo Zhou, Jian Luan, Jiayi Ma

arXiv:2610.33135v1cs.CV

TL;DR

Cinematic character video needs precise camera control as well as character consistency across substantial viewpoint changes. This paper introduces CLACT and proposes WorldCast, combining geometric guidance with geometry-aligned multi-view character memory. Across synthetic and real-world scenes, the authors report more accurate camera control and stronger preservation of character geometry, appearance, and identity, with competitive visual quality.

  • Problem

    Cinematic character video requires precise camera control as well as consistent character geometry, appearance, and identity across substantial viewpoint changes.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces CLACT for training and evaluation and proposes WorldCast, which combines scale-aligned geometric guidance with GAMA’s geometry-aligned retrieval from multi-view character memory.

    Source: §1 · p. 3

  • Results

    Experiments on synthetic and real-world scenes demonstrate more accurate camera control and stronger preservation of character geometry, appearance, and identity, while maintaining competitive visual quality.

    Source: §1 · p. 3, §5 · p. 11

  • Takeaways & Limitations

    The reported results show joint improvement in trajectory accuracy and character preservation under substantial viewpoint changes.

    Source: §5 · p. 11

    The training benchmark is constructed primarily from controlled simulations, and released real-world evaluation data will exclude personally identifiable private individuals and follow applicable data licenses and usage terms.

    Source: Ethics Statement · p. 16

Abstract

from arXiv · show

Precise camera control is essential for cinematic character video generation, yet existing methods struggle to follow large-angle trajectories with combined rotation, translation, and radius variation from arbitrary initial azimuths. Such viewpoint changes also reveal character regions unseen in the initial frame, causing geometric distortions, appearance degradation, and identity drift. To systematically study these coupled challenges, we introduce CLACT, a large-scale Character-centric dataset and benchmark covering diverse Large-Angle Camera Trajectories, with accurate camera geometry and multi-view character observations for training and evaluation. We further propose WorldCast, a framework for world-consistent character video generation under diverse large-angle camera motions. Specifically, we back-project the initial frame into a 3D point cloud using estimated depth, align the point-cloud scale with the target camera trajectory, and render it along the trajectory to obtain explicit geometric guidance and coverage masks. To complement the incomplete character information, we introduce Geometry-Aligned Memory Attention (GAMA). Using relative camera poses estimated jointly from the initial frame and multi-view character anchors, we express anchor and target-view geometry in a common coordinate frame. GAMA uses this geometric alignment to guide the retrieval and injection of view-relevant character features during generation. Evaluations across diverse large-angle camera trajectories demonstrate accurate trajectory following while preserving character geometry, appearance, and identity.

1 Introduction

Cinematic character video requires precise camera motion alongside consistent character geometry, appearance, and identity across substantial viewpoint changes. The introduction presents CLACT and WorldCast to address the coupled challenges of large-angle camera control and character consistency.

1 Introduction

  • Visual fidelity alone is insufficient for cinematic content creation, which also requires precise camera-motion control [1].Character videos also use viewpoint changes to reveal subjects, establish spatial relationships, and direct visual attention.Source: §1 · pp. 1–2
  • Large-angle trajectories challenge camera control, while changing viewpoints expose character regions unseen in the initial frame and weakly constrain their geometry, appearance, and identity [16] [17].The target trajectory determines both the geometry to follow and the unseen character content to synthesize.Source: §1 · p. 2
  • Existing camera-motion datasets have limited coverage of complex trajectories and are not consistently paired with complementary views of the same character [20].This limits joint study of trajectory accuracy and cross-view character consistency.Source: §1 · pp. 2–3
  • 50,000 Unreal Engine 5 camera-motion sequences, 30,000 Blender-rendered multi-view character samples, and 42 synthetic and real-world scenes make up CLACT’s training and evaluation data.The benchmark supports evaluation across multiple trajectory families and substantial viewpoint changes.Source: §1 · p. 3
  • WorldCast combines scale-aligned geometric guidance with multi-view character memory to address large-angle camera control and character consistency.It back-projects the initial frame into a point cloud, aligns its scale with the camera trajectory, and renders geometric guidance and coverage masks; GAMA aligns anchor and target-view geometry for view-relevant feature retrieval.Source: §1 · p. 3
  • WorldCast improves camera-control accuracy and cross-view character consistency while maintaining competitive visual quality.Figure 1 depicts tilted circular and lemniscate trajectories and compares ATE, MEt3R, and IQ; its axes are reversed, placing lower ATE and MEt3R toward the upper right.Source: Figure 1 · p. 2, §1 · p. 3

2 Related Work

Related work spans camera-conditioned video generation, geometry-aware novel-view synthesis, and identity-consistent generation. WorldCast’s stated distinction is to relate multi-view character-anchor geometry to the target trajectory for view-relevant guidance.

2 Related Work

  • Camera-controlled video generation progresses from pose and ray conditioning to geometric guidance, expressive camera representations, and video re-rendering methods [6] [21] [11] [15] [24].The cited approaches include camera/object-motion disentanglement, camera geometry and epipolar constraints, and joint RGB-depth generation.Source: §2 · p. 3
  • Novel-view synthesis methods use one or a few observations, while other approaches leverage geometry or multiple input views to synthesize target viewpoints [17].For camera-controlled generation, other work introduces explicit scene geometry or trajectory-aligned 3D guidance; ConfCtrl addresses unreliable projections, while Lyra 2.0 and WorldStereo explore persistent geometry and geometry-constrained memory retrieval [16] [28] [29].Source: §2 · pp. 3–4
  • WorldCast distinguishes itself by jointly addressing character consistency and large-angle camera control through geometric guidance and view-aware retrieval from multi-view character memory.Its approach relates character-anchor geometry to the target trajectory.Source: §2 · p. 4
  • Subject-conditioned video generation and identity-preservation methods seek to maintain appearance and identity, with some approaches using complementary views or content anchors.Virtually Being combines viewpoint identity preservation with camera control through customization based on performance captures.Source: §2 · p. 4

3 Method

WorldCast formulates character video generation as following a prescribed camera sequence while preserving character geometry, appearance, and identity. It combines scale-aligned geometric guidance, geometry-aligned multi-view memory, and a training corpus built for large-angle camera motion.

3.1 Preliminaries: Video Diffusion Models

  • The video-diffusion backbone encodes videos into latents and learns a conditional velocity field, then integrates an ODE to decode generated video.WorldCast adopts Wan2.1-I2V-14B as its generative backbone.Source: §3.1 · p. 4

3.2 Problem Formulation

  • WorldCast conditions generation on an initial character image, text prompt, target camera sequence, and multi-view observations, aiming to follow the trajectory while preserving character geometry, appearance, and identity.Large-angle views can reveal surfaces unobserved in the initial image.Source: §3.2 · pp. 4–5

3.3 Geometric Camera Conditioning

  • For geometric camera conditioning, candidate camera-translation scales are rendered from a back-projected initial-view point cloud, and the scale whose image motion best matches the reference is selected.The resulting renderings and coverage masks provide geometric guidance along the target trajectory.Source: §3.3 · pp. 5–6

3.4 Geometry-Aligned Memory Attention

  • GAMA aligns anchor-camera geometry with the target trajectory in a shared normalized frame, then forms memory tokens from masked observations, their aligned camera geometry, and ray grids.Video queries retrieve view-relevant character evidence from the memory while retaining video context.Source: §3.4 · pp. 6–7
  • Memory-attention blocks inject a refined residual update into video tokens, with their output projections zero-initialized to preserve the pretrained backbone at adaptation start.The blocks perform joint self-attention over memory and video tokens.Source: §3.4 · p. 7

3.5 Training Data Construction

  • The 80,000-sample training corpus combines 50,000 UE5 camera-motion sequences with 30,000 Blender-rendered samples and multi-view character anchors.Both collections cover 99 characters and 33 environments, with varied camera paths and initial viewpoints.Source: §3.5 · p. 8

4 Experiments

Experiments assess camera control, cross-view character consistency, and visual quality on CLACT, then compare WorldCast with prior methods and examine its components, training data, camera conditioning, and anchor count.

4.1 Experimental Setup

  • The evaluation benchmark covers camera-control accuracy, cross-view character consistency, and visual quality under viewpoint changes, using 42 scenes and six trajectories per scene.The scenes comprise 17 synthetic and 25 real-world cases.Source: §4.1 · pp. 8–9

4.2 Comparison with State-of-the-Art Methods

  • WorldCast achieves superior camera control and character consistency while maintaining competitive visual quality, with gains spanning trajectory accuracy, cross-view geometry, and identity and appearance preservation.The comparison uses CLACT’s 42 synthetic and real-world scenes across six camera trajectories.Source: §4.1 · p. 8, §4.2 · p. 9, Table 1 · p. 9
  • On a large-angle orbit-and-dolly trajectory, Uni3C distorts characters, while WorldCast more reliably follows camera motion and preserves identity, 3D structure, and appearance.MVGenMaster also loses fine character details and appearance cues despite using multi-view references.Source: §4.2 · pp. 9–10, Figure 4 · p. 10

4.3 Ablations

  • The full configuration performs best across metrics; removing GAMA, anchor geometry alignment, or foreground masking degrades the corresponding control or character-consistency outcomes.Without anchor geometry alignment, rotation error rises markedly while identity similarity remains relatively high.Source: §4.3 · p. 10, Table 2 · p. 10
  • Excluding UE5 training data increases all three camera errors, and removing Plücker conditioning degrades camera accuracy in the first-stage model before GAMA adaptation.The study attributes the data benefit to large-angle training sequences and says camera-ray information complements projected geometric guidance.Source: Table 3 · p. 10, §4.3 · p. 11
  • Increasing character anchors from one to four improves cross-view geometry, identity preservation, and appearance similarity, with smaller gains from three to four.The passage attributes the improvements to complementary views supplying character information missing from the initial view.Source: §4.3 · p. 11, Figure 5 · p. 11

5 Conclusion

The paper concludes that CLACT and WorldCast address large-angle camera-controlled character video generation through trajectory guidance and multi-view character memory. Experiments support improved camera control and character preservation, with future work targeting more flexible anchors, dynamic scenes, and longer or interactive trajectories.

5 Conclusion

  • CLACT provides a character-centric benchmark, while WorldCast jointly improves trajectory accuracy and character preservation under substantial viewpoint changes.The framework combines scale-aligned geometric guidance with GAMA, which aligns multi-view observations to target camera geometry and retrieves view-relevant features.Source: §5 · p. 11
  • Experiments on synthetic and real-world scenes show more accurate camera control and stronger preservation of character geometry, appearance, and identity, with competitive visual quality.Source: §5 · p. 11
  • Future work will explore more flexible anchor acquisition, more complex dynamic scenes, and longer or interactive camera trajectories.Source: §5 · p. 11

AI Use Statement

This material documents the authors’ AI-use and ethics practices, reproducibility resources, and a scale-calibration procedure for aligning rendered point-cloud motion with reference-video motion.

AI Use Statement

  • The authors used AI for language polishing and revision, reviewed its contributions, and state it did not conduct experiments or determine results or conclusions.The authors take responsibility for the final content, claims, code, and artifacts.Source: AI Use Statement · p. 15

Ethics Statement

  • The benchmark uses primarily controlled simulations and licensed or public assets; real-world evaluation data will exclude identifiable private individuals and follow applicable licenses.The authors also identify potential misuse for deceptive impersonation and say they will provide provenance and documentation where applicable.Source: Ethics Statement · p. 16

Reproducibility Statement

  • The authors document architectural, conditioning, training, and evaluation details and plan to release benchmark-generation and evaluation resources upon publication.Planned releases include alignment and identity-evaluation procedures, trajectory and configuration settings, and materials for reproducing results with the checkpoint.Source: Reproducibility Statement · p. 16

A Depth–Trajectory Scale Calibration

  • Depth and camera translation are calibrated by matching point-cloud-rendered image motion to reference-video motion, reducing parallax mismatch during training.At inference, translation is interpreted relative to estimated scene depth, without reference-based scale calibration.Source: A Depth–Trajectory Scale Calibration · pp. 17–18

B Training Data Construction

The training data combine camera-motion sequences with multi-view character observations, using complementary UE5 and Blender collections for camera-control and character-memory learning.

B Training Data Construction

  • The training corpus contains 80,000 character-centric samples, with UE5 sequences for large-angle camera-control learning and Blender samples with multi-view anchors for character-memory learning.The two collections share character and environment counts, while the figure summarizes their sizes, roles, and trajectory configurations.Source: B Training Data Construction · p. 18, Figure B.1 · p. 18

B.1 UE5 Character-Centric Camera-Motion Data

  • The UE5 collection renders 50,000 sequences across 99 character models, 33 environments, and 96 actions, with varied camera configurations and complementary front, back, left, and right views.Its trajectories include arcs, large-angle orbits, orbit-and-dolly paths, and free-form paths, with constant or variable speed.Source: §B.1 · pp. 18–19

B.2 Blender Data with Multi-View Character Anchors

  • The Blender pipeline generates 30,000 samples and supports large or partial orbits, translations, combined rotation–translation paths, and radius variation.It supplies four-view character anchors, reference videos, initial RGB-D observations, camera parameters, projected RGB guidance, and masks for regions unsupported by the initial view.Source: §B.2 · pp. 19–20

C.1 Benchmark Construction

The benchmark evaluates camera control and character consistency across disjoint synthetic and real-world scenes, prescribed large-angle trajectories, and standardized initial-image inputs.

C.1 Benchmark Construction

  • The evaluation set contains 42 disjoint character-centric scenes: 17 synthetic UE5 scenes and 25 scenes collected from real-world imagery.The synthetic and real-world subsets provide controlled configurations and natural-image settings, respectively.Source: §C.1 · p. 20
  • Each scene is evaluated under six prescribed trajectories spanning large-angle orbits, pure translations, and combined rotation–translation motions from diverse initial azimuths.The trajectories vary which character content is initially visible, supporting joint evaluation of trajectory adherence and character consistency as unseen regions appear.Source: §C.1 · p. 20
  • Each case provides an initial character image and target trajectory, with multi-view observations additionally supplied to methods that support or require them.All methods follow the same target trajectories, with camera parameters adapted to native representations where needed.Source: §C.1 · p. 20

C.2 Baseline Evaluation Protocol

The evaluation adapts inputs to three generation paradigms while retaining prescribed camera motion, then measures camera adherence, character consistency, and visual quality.

C.2 Baseline Evaluation Protocol

  • The protocol adapts benchmark inputs to direct camera-controlled generation, novel-view synthesis, and video re-rendering while preserving prescribed target motion.Novel-view methods are tested in native single-image and supported multi-image settings; re-rendering methods receive a static video made by repeating the initial image.Source: §C.2 · pp. 20–21

C.3 Evaluation Metrics

  • Camera adherence is assessed with aligned-trajectory ATE, RTE, and RRE, with lower values indicating better adherence.Generated-video camera trajectories are recovered using Depth Anything 3 and aligned to targets with a Sim(3) Umeyama transformation.Source: §C.3 · p. 21
  • Character consistency is measured with MEt3R geometry and ArcFace, CLIP, and DINO similarities against the closest-pose observation anchor; visual fidelity uses subject-consistency and image-quality measures.Lower MEt3R and higher ArcFace, CLIP, and DINO similarities indicate stronger consistency.Source: §C.3 · p. 21

D Additional Analyses and Visualizations

D.1 Analysis of Geometry-Aligned Anchor Routing

Under a 360° orbit, attention routing usually selects the character anchor whose camera pose is closest to the target viewpoint. Anchor geometry alignment improves routing accuracy during training and after convergence.

D.1 Analysis of Geometry-Aligned Anchor Routing

  • 19/21 latent positions route most strongly to the anchor with the closest camera pose to the target viewpoint.The 81 generated frames correspond to 21 temporal latent positions.Source: Figure D.1 · p. 22, §D.1 · p. 22
  • As the camera moves around the character, dominant attention shifts across anchors with different viewpoints.This pattern indicates retrieval according to the evolving target camera geometry.Source: §D.1 · p. 22
  • Geometry alignment improves routing accuracy faster during training and yields higher final accuracy than training without alignment.The authors attribute this to a stronger geometric prior for learning view-aware memory retrieval.Source: §D.1 · p. 22, Figure D.2 · p. 22

D.2 Additional Qualitative Results

Additional comparisons cover diverse camera paths and initial azimuths, while trajectory-generalization examples reuse fixed character anchors across varied target motions.

D.2 Additional Qualitative Results

  • Additional comparisons span large-angle orbits, orbit-and-dolly motions, spiral and hyperbolic paths, and conventional camera motions.Examples cover different initial azimuths and evaluate camera control and character preservation under substantial viewpoint changes.Source: §D.2 · p. 22, Figure D.3 · p. 24, Figure D.4 · p. 25
  • With the same character anchors fixed, examples show support for diverse target motions, including orbits, translations, and combined motions.The authors report that the anchors can be reused across different target trajectories.Source: §D.2 · p. 23, Figure D.8 · p. 29, Figure D.9 · p. 30
  • The trajectory-generalization figures compare diverse target camera motions while keeping the character-anchor set unchanged.The captions identify Figures D.8 and D.9 as using the same anchors across diverse target motions.Source: §D.2 · p. 23, Figure D.8 · p. 29, Figure D.9 · p. 30

Related papers

Loading 2610.33135v1…