AESOP: Asymmetric Human-Camera Generation with Translation-Intensity Control

1 INTRODUCTION

• Human motion defines the action, while camera trajectory presents it; both are complementary parts of motion storytelling.Cinematography uses framing and camera movement to organize screen space, direct attention, and shape how viewers experience an event.Source: §1 · p. 1
• AESOP shares a camera generator conditioned on supplied or generated human motion, reflecting that human action can be determined independently of the camera.Its asymmetric architecture preserves the supplied or generated human action during camera generation.Source: §1 · pp. 1–2
• Intensity-paired augmentation varies camera translation magnitude while keeping human motion and camera text fixed; direction-paired examples add auxiliary supervision.Intensity pairs contract or expand smooth displacement about the camera center at the first frame, with geometry and visibility checks screening targets.Source: §1 · p. 2
• AESOP reduces given-human r-FPD by 49.9% relative to DIRECTOR-C and joint camera FDC by 87.8% relative to PulpMotion DiT on PulpMotion.The paper also reports strong camera distributional and framing quality in both tasks and effective control over camera travel intensity.Source: §1 · p. 2

2.1 HUMAN MOTION GENERATION

• Human-motion research broadened from plausible action clips toward semantic fidelity, motion quality, finer instructions, streaming, larger action coverage, and video-generation priors.Paired datasets include HumanML3D (Guo et al., 2022) and MotionMillion; methods include diffusion and discrete generation (Zhang et al., 2023; Guo et al., 2024).Source: §2.1 · pp. 2–3

2.2 HUMAN-AWARE CAMERA GENERATION

• Human-aware camera methods condition viewpoints and trajectories on available performances, while joint systems also synthesize human motion and camera behavior.DIRECTOR and DanceCamera3D use performance context; PulpMotion and Towards Storytelling Animations study joint generation.Source: §2.2 · p. 3

2.3 EXPLICIT AND GEOMETRY-GROUNDED CAMERA CONTROL

• Explicit trajectory interfaces address ambiguity by supplying camera geometry, whereas AESOP varies direction or translation intensity while holding human motion fixed.MotionCtrl and CameraCtrl condition video generation on camera paths, shifting trajectory authoring to the user; AESOP instead uses paired directional instructions and a continuous intensity scalar.Source: §2.3 · p. 3

3.1 PRELIMINARIES

• Given-human generation maps human motion H and camera text TC to camera trajectory C, while joint generation maps human text TH and camera text TC to both H and C.Both tasks accept optional translation intensity a: a = 1 specifies default behavior, and smaller or larger values request weaker or stronger camera-center translation.Source: §3.1 · p. 3
• The flow-matching setup interpolates clean latents and Gaussian noise, and sampling integrates predicted velocity from noise at σ = 1 to a clean latent at σ = 0.The camera flow receives the complete, fixed human latent sequence.Source: §3.1 · p. 3

3.2 UNIFIED ASYMMETRIC HUMAN–CAMERA GENERATION

• AESOP uses an independent human pathway and a camera generator conditioned on the complete human latent sequence; human reconstruction depends only on the human latent.The same camera flow and decoder serve both tasks: given-human generation encodes supplied motion, while joint generation reuses the completed human latent for camera conditioning and human decoding.Source: Figure 2 · p. 4, §3.2 · pp. 4–5
• Human generation samples a complete latent sequence and decodes it, then freezes the human pathway before camera training so camera guidance and intensity do not change the generated human motion.During camera training, the camera flow is conditioned on the complete encoded ground-truth human sequence.Source: §3.2 · pp. 4–5

3.3 GEOMETRY-GROUNDED CAMERA CONTROL

• DPA creates opposite-direction targets, while IPA creates different translation intensities in physical camera space; both constructions fix human motion and screen targets with geometry and visibility checks.Directional pairs differ in direction while preserving event timing, displacement magnitude, initial camera state, rotation, and field of view.Source: §3.3 · p. 5
• IPA preserves camera text, rotation, field of view, and sequence timing, recovers the original path at a = 1, and amplifies only smooth translation when a > 1.Intensity is embedded into the camera-flow timestep condition; unchanged low-translation targets are also paired across default and non-default intensity values.Source: §3.3 · pp. 5–6

4.1 EXPERIMENTAL SETUP

• Evaluation uses the public PulpMotion split of 162,760 training and 4,053 test sequences, with three seeds for generation and reconstruction comparisons.The study reports camera and human metrics, framing, trajectory error, and geometry-grounded control.Source: §4.1 · p. 6, §4 · p. 6

4.2 COMPARISON WITH PRIOR METHODS

• 7.29 to 5.97: given-human FDC falls against DIRECTOR-C, while joint FDC falls from DiT's 83.80 to AESOP's 10.18.AESOP also lowers r-FPD from 1.993 to 0.998 in given-human generation and from 7.772 to 0.581 in joint generation.Source: §4.2 · pp. 6–8
• AESOP reaches 0.835 PRDC recall in joint generation, versus 0.215 for DiT and 0.312 for MAR, while improving all four PRDC measures.Higher precision accompanies broader coverage and lower human distribution distance in Table 1.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 REPRESENTATION AND CAMERA CONTROL

• AESOP reconstructs camera centers with 0.029 m ADE and 0.41° rotation error, versus PulpMotion's 0.110 m and 1.37°.PulpMotion has higher human-text alignment, 20.15 versus 17.56.Source: §4.3 · p. 8
• Mean path length rises from 0.647 to 1.023 m in given-human generation and from 0.367 to 0.612 m in joint generation as intensity varies from 0.5 to 1.5.Travel is nondecreasing across all ten adjacent intensity intervals in 81.50%/81.37% of clips for path length and 78.61%/78.46% for net displacement, respectively; path-length rates reach 99.19%/98.86% among clips with reference net displacement ≥0.2 m.Source: §4.3 · pp. 8–9
• As camera travel increases, CLaTr and F1 rise alongside worse r-FPD and Out, while given-human ADE and rotation error in both tasks vary little.Given-human FDC is lowest near a = 0.8, whereas joint FDC decreases across the tested grid.Source: §4.3 · p. 9
• Direction-pair continuation reaches 84.33%/83.11% combined accuracy in given-human/joint generation, beating matched factual-pair continuation by 1.67/1.38 percentage points.IPA then adds continuous intensity control while maintaining camera quality similar to the +DPA endpoint.Source: §4.3 · p. 9

4.4 USER STUDY

• In an anonymized paired study with 29 participants, AESOP receives 87.0% preference for given-human generation and 89.3% for joint generation.The respective 95% confidence intervals are 83.8–90.8 and 84.4–93.6.Source: §4.4 · p. 9, Table 3 · p. 9

5 CONCLUSION

• AESOP provides strong camera distributional and framing quality in both tasks, while its learned intensity condition adjusts camera travel.Source: §5 · p. 10
• Sequence-level intensity limits control over individual events, while complete human context and offline sampling limit streaming generation.The authors identify event-wise intensity control and causal, incremental camera generation as future directions.Source: §5 · p. 10

A DISCLOSURE OF AI USE

• Generative AI tools assisted with supporting code, experimental execution, DPA/IPA threshold selection, and language refinement under the authors’ direction.The authors state that they made the final research decisions and take responsibility for the work.Source: A DISCLOSURE OF AI USE · p. 14

B.1 SYMMETRIC AND ASYMMETRIC REPRESENTATIONS

• Unlike AESOP, PulpMotion’s shared encoder lets both latents depend on human and camera inputs, coupling human representation learning to camera training.PulpMotion reconstructs human motion and camera trajectory from split components of a shared latent and reconstructs framing from an auxiliary latent.Source: §B.1 · p. 14
• AESOP’s human decoder reads only the human latent, while its camera decoder combines the human and camera latents.Human representation training is completed before camera training, and the camera pathway uses frozen human latents.Source: §B.1 · p. 14, Figure S1 · p. 14
• AESOP has lower human positional, translation-velocity, and jerk reconstruction errors, while PulpMotion has lower rotation-velocity error.The comparison concerns deterministic encoding and decoding on the test split; Table S2 defines the reported error units and metrics.Source: §B.1 · p. 14, Table S2 · p. 15
• Root-aligned human error is smaller than global error for both representations, indicating a contribution from root-trajectory deviations.Table 1 reports distributional and camera-pose reconstruction quality.Source: §B.1 · p. 15

B.2 IMPLEMENTATION, TRAINING OBJECTIVES AND INFERENCE

• Each human frame has 199 features and each camera frame has 14, including relative position, rotation, and translation increments.For a sequence of T frames, each encoder produces ⌈T/4⌉ latent tokens; the human and camera latents have 128 and 64 channels, respectively.Source: §B.2 · p. 15
• The camera Transformer attends to camera text and the full human sequence, while the human and camera flow Transformers each use 12 layers and width 512.Both use eight attention heads, an FFN multiplier of four, and dropout 0.1; camera text features are frozen 512-dimensional CLIP embeddings.Source: §B.2 · p. 15
• Latents are standardized and whitened with training-only statistics; the inverse transform is applied before decoding.Cholesky whitening uses ridge 10^-4, and invalid positions are zeroed after normalization or its inverse.Source: §B.2 · p. 15
• Stage 2 samples noise levels and trains velocity prediction, with continuation batches weighted by clip length rather than equally.Original training weights clips equally; the schedule adds DPA and then IPA continuation, and the final phase trains the intensity MLP at 10^-4.Source: §B.2 · pp. 15–16, Table S3 · p. 16
• Inference uses solver shift five and camera-text guidance, with an intensity MLP output added to the camera timestep condition.Source: §B.2 · p. 16

C DPA AND IPA DATA CONSTRUCTION

• Both augmentation schemes preserve human motion, camera rotation, field of view, and timing; DPA changes direction and its text, while IPA changes translation intensity with text fixed.The human latent and decoded output remain unchanged across each constructed pair.Source: C DPA AND IPA DATA CONSTRUCTION · p. 16

C.1 DIRECTION-PAIRED AUGMENTATION

• DPA selects Truck or Dolly shots with sufficient stable translation, then reflects the signed event coordinate to create an opposite-direction trajectory.Truck preserves orthogonal components, while Dolly preserves the human-to-camera unit direction.Source: §C.1 · p. 16

Direction-paired augmentation

• Post-checks retain 601 DPA sources, yielding 601 original/opposite training pairs; the checks include decoded direction and visibility.The retained set comprises 375 Dolly and 226 Truck sources.Source: §C.1 · p. 16, Figure S2 · p. 17

C.2 INTENSITY-PAIRED AUGMENTATION

• IPA screens active sources for sufficient travel and pairs low-translation null sources with unchanged targets to teach retention under intensity changes.The active branch transforms camera-center displacement with a smoothed trajectory; relative-center and velocity features are recomputed.Source: §C.2 · p. 17

Intensity-paired augmentation

• IPA samples weaker labels below a = 1 and stronger labels above a = 1, favoring values closer to one; evaluation sweeps a ∈ [0.5, 1.5] in 0.1 steps.Source: §C.2 · p. 17

C.2 INTENSITY-PAIRED AUGMENTATION

• The final IPA training set uses 8,000 active sources for 16,000 targets and 2,000 null sources.Active sources must have an accepted target on both sides of a = 1; development variants are kept separate from control training and test data.Source: §C.2 · p. 17, Figure S3 · p. 18

Intensity-paired augmentation

• IPA acceptance combines physical and construction-representation reconstruction checks, with null targets left unchanged.The listed criteria cover source selection, target geometry, framing, and reconstruction.Source: Table S4 · p. 18, Intensity-paired augmentation · p. 19

D.1 BASELINE PROTOCOLS

• Given-human evaluation supplies reference motion, whereas joint evaluation uses each method’s generated motion; baselines differ in inputs and sampling configurations.AESOP and PulpMotion anchor camera position to the human root and accumulate world-space increments, unlike CCD and CCD-H’s human-relative decoding.Source: §D.1 · p. 19, Table S5 · p. 19

D.2 DIRECTION-PAIRED TRAINING

• Each direction-prompt arm uses the same 3,194 clips with active reference translation, totaling 6,388 clip–prompt cases per task.Pairs share text encoding, human latents, camera noise, and valid-length masks; joint generation samples the human first.Source: §D.2 · p. 19
• Direction accuracy scores reference-active translation-axis signs at fixed reference times, excludes inactive positions, and counts still predictions at active positions as errors.Reversed targets negate the corresponding reference signs; clip scores average over positions and axes before equal-weight averaging across clips.Source: §D.2 · p. 19
• Table S6 reports direction accuracy, comparing DPA with factual continuations matched on checkpoint, updates, batch allocation, and sampling schedule.The factual-pair control replaces directional counterfactuals; the passage does not provide table values.Source: Table S6 · p. 20

E TRANSLATION-INTENSITY CONTROL

• 97.31% of given-human and 97.01% of joint test outputs have nondecreasing path length when intensity changes from 1 to 1.5.The measure concerns endpoint change in translation intensity.Source: E TRANSLATION-INTENSITY CONTROL · p. 20

• Trials assess camera-text alignment, camera motion quality, and framing; joint trials also assess human-text alignment and human motion quality.Figure S4 shows the generated camera view above an external spatial view.Source: F USER STUDY · p. 20, Figure S4 · p. 21
• Overall camera quality is the primary outcome for given-human trials, and overall human–camera quality is primary for joint trials.Preference averages equally over clips and opponents; 95% confidence intervals use 10,000 participant-cluster bootstrap resamples.Source: F USER STUDY · p. 20
• The study includes 29 complete questionnaires, comprising 580 given-human and 522 joint trials across all 120 and 60 clip–pair comparisons.Of 5,452 criterion responses, 5,238 were judgeable and 214 were “Cannot judge.”Source: F USER STUDY · p. 20
• 80.4% joint human-text alignment and 80.9% joint motion-quality preference, while camera criteria range from 87.5% to 89.9%.AESOP has the highest preference on each reported criterion.Source: F USER STUDY · p. 20, Figure S5 · p. 21

• AESOP is demonstrated in camera generation from HumanML3D motions, joint generation from HumanML3D action descriptions, and camera generation from HY-Motion outputs.The examples pair global views with successive camera views.Source: G QUALITATIVE RESULTS ON EXTERNAL… · p. 22
• With ground-truth HumanML3D motions, examples show static, pull-out, and trucking camera behavior for running, walking and turning, and jumping and spinning.Figure S6 shows a static view of running in place, a pull-out for walking and turning, and trucking with jumping and spinning.Source: G QUALITATIVE RESULTS ON EXTERNAL… · p. 22, Figure S6 · p. 22
• For joint generation from HumanML3D action descriptions, examples include a pull-in during arm stretching and a static view of an approaching runner.Figure S7 presents generated human motion and camera trajectory in global and successive camera views.Source: G QUALITATIVE RESULTS ON EXTERNAL… · p. 22, Figure S7 · p. 23
• Conditioned on HY-Motion outputs, examples pair static views with bowing and rhythmic stepping, a pull-out with a surfboard pop-up, and a pull-in with stepping back and blocking.Figure S8 shows a global view and successive camera views.Source: G QUALITATIVE RESULTS ON EXTERNAL… · p. 22, Figure S8 · p. 23

H MORE QUALITATIVE RESULTS

• Figures S9 and S10 add two examples for each task, showing a global view and three camera projections in temporal order.The global views are fitted separately to each trajectory.Source: H MORE QUALITATIVE RESULTS · p. 24, Figure S9 · p. 24, Figure S10 · p. 25

DIRECTOR-C

• DIRECTOR-C examples pair a forward walk with a rightward dolly or truck, and an instrumental performance with a gradual camera move closer to the subject.In the second example, the person moves a hand toward their face and then returns it to the instrument.Source: DIRECTOR-C · p. 25