PaperlayerView exact revision on arXiv

Source-linked AI summary

OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images

Junyeong Ahn, Geonwoo Kim, Kinam Kim, Gihwi Kim, Hoonjin Jung, Hyojin Jang, Jaegul Choo

arXiv:2610.35684v1cs.CVcs.AI

TL;DR

Single-view S2V leaves unseen surfaces unconstrained, and multiple reference views do not explicitly specify the 3D structure connecting them. OmniShowcase pairs geometry-consistent training-data synthesis with reference-derived surface-normal conditioning that requires no frame-wise alignment. It achieves the best reported DINO, CLIP, MEt3R, and GT-CD scores on both benchmarks while keeping text alignment and video quality competitive.

  • Problem

    Single-view S2V leaves unseen surfaces unconstrained, while multi-view methods do not explicitly specify the 3D structure connecting reference views.

    Source: §1 · p. 2

  • Method

    OmniShowcase synthesizes training videos from 3D assets and conditions generation on reference images plus a reconstructed surface-normal turntable without frame-wise alignment to the target.

    Source: §1 · p. 2, §3.3 · p. 6

  • Results

    On both benchmarks, OmniShowcase achieves the best DINO, CLIP, MEt3R, and GT-CD scores while keeping text alignment and video quality competitive.

    Source: §4.2 · p. 8

  • Takeaways & Limitations

    The evaluation supports reconstructed geometry as a cue for subject identity and cross-view 3D consistency on SHOWCASEBENCH and NAVI.

    Source: §5 · p. 9

    Training geometry is reconstructed from the four reference views rather than taken from the source mesh, matching the conditioning available at inference.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Generating videos of a specific subject from a few reference images remains challenging when the camera reveals surfaces that are not visible in the input. Early subject-to-video (S2V) methods condition on a single reference image and must therefore infer the subject's unseen appearance and geometry as the viewpoint changes. To address this limitation, recent methods extend S2V to multiple reference views, providing more information about the subject from different viewpoints. However, they do not explicitly specify the 3D structure that connects these observations. In addition, existing training data do not guarantee that viewpoint changes preserve the same underlying 3D subject. We present OmniShowcase, built on two components: (i) a fully automated data synthesis pipeline that produces training videos along diverse camera trajectories, with viewpoint variation driven by underlying subject geometry, even spanning full 360-degree orbits; and (ii) a geometry-conditioning method that provides the video model with a turntable of surface-normal maps reconstructed from the reference views without requiring temporal alignment with the generated frames. We construct Showcase-4K, a corpus of 3D-asset-based videos covering twelve camera motions across 88 trajectory configurations with reconstruction-derived surface-normal conditioning, and ShowcaseBench, a held-out evaluation set with the source meshes and camera parameters used to render each video. Experiments show state-of-the-art subject identity and 3D consistency while maintaining competitive video quality and motion. Project page: justin4ai.github.io/OmniShowcase

1 INTRODUCTION

Single-view S2V leaves unseen surfaces unconstrained, while multiple reference views still do not explicitly specify the 3D structure connecting them. OmniShowcase combines automated, geometry-consistent training-data synthesis with surface-normal conditioning reconstructed from reference views.

1 INTRODUCTION

  • Single-view S2V leaves unseen surfaces unconstrained, so newly revealed viewpoints can produce inconsistent geometry, appearance, silhouettes, and surface details.The central difficulty is inferring 3D structure absent from the input, not merely matching appearance.Source: §1 · p. 2
  • Recent multi-view S2V methods provide more appearance information, but do not explicitly specify the 3D structure connecting the views.Prior approaches include feed-forward training on generated multi-view videos and per-subject optimization on static multi-view images.Source: §1 · p. 2
  • OmniShowcase combines automated 3D-asset-based video synthesis with geometry-aware conditioning reconstructed from reference views.The method uses a surface-normal sequence as its geometry condition.Source: §1 · p. 2
  • The paper introduces SHOWCASE-4K training videos and SHOWCASEBENCH, a held-out benchmark with ground-truth geometry and camera parameters.The paper reports state-of-the-art subject identity preservation and 3D consistency with competitive video quality.Source: §1 · p. 2

2 RELATED WORK

The related work spans video diffusion, 3D generation and novel-view synthesis, and subject-driven video generation. OmniShowcase builds on these lines while addressing explicit geometry and 3D-consistent viewpoint changes for S2V.

2.1 VIDEO GENERATION DIFFUSION MODELS

  • Diffusion transformers and rectified-flow training advanced video generation, while pretrained image-to-video models provide a starting point for OmniShowcase (Hu et al., 2022).Earlier video diffusion work added temporal layers to image backbones; later systems use large diffusion transformers trained on web-scale corpora.Source: §2.1 · p. 2

2.2 3D GENERATION AND NOVEL VIEW SYNTHESIS

  • Multi-view diffusion methods encourage cross-view coherence, and feed-forward estimators recover dense 3D structure from sparse uncalibrated images (Wang et al., 2024a; 2025; 2026; Lin et al., 2026).The multi-view generators are mostly limited to isolated objects on clean backgrounds and do not model scene-level dynamics.Source: §2.2 · p. 3

2.3 SUBJECT-DRIVEN VIDEO GENERATION

  • S2V methods primarily encode subjects through 2D appearance, leaving viewpoints outside the references unconstrained (Jiang et al., 2024; Wei et al., 2024; He et al., 2024; Yuan et al., 2025).MV-S2V uses multiple views but lacks explicit 3D surface structure and training viewpoint variation tied to a shared asset; 3DreamBooth requires per-subject optimization (Song et al., 2026; Ko et al., 2026).Source: §2.3 · p. 3

3 METHOD

OmniShowcase generates training videos from 3D assets and conditions video generation on reference images plus a reconstructed surface-normal turntable. Its conditioning treats the turntable as a separate sequence, so it need not align frame by frame with the target trajectory.

3 METHOD

  • OmniShowcase reconstructs a subject from reference views and renders a surface-normal turntable to supply explicit geometry to a pretrained video diffusion model.The method uses the reference images for appearance and the normal condition for 3D structure.Source: §3 · p. 3, Figure 4 · p. 6

3.1 PRELIMINARIES AND PROBLEM SETUP

  • The task is to generate a prompted video that follows a specified camera trajectory while preserving subject appearance and 3D structure across viewpoints.The experiments use four reference views, and the reconstructed geometry reduces reliance on model priors for unobserved viewpoints.Source: §3.1 · p. 4

3.2 AUTOMATED 3D-CONSISTENT DATA SYNTHESIS

  • The synthesis pipeline uses one mesh for target and reference renders, producing 81-frame videos and four views at 90° yaw intervals across 88 trajectory configurations.The trajectories include orbit-family motions and additional arc, truck, dolly, and zoom variants.Source: §3.2 · pp. 4–5
  • Video inpainting generates a coherent prompt-driven background while leaving rendered subject pixels unchanged; this contrasts with Uni3C generating newly revealed subject surfaces.A vision-language gate discards clips whose subject floats, deforms, or departs from the references.Source: §3.2 · p. 5, Figure 3 · p. 5
  • Training geometry is reconstructed from the four reference views, then fused and rendered as a yaw-orbit surface-normal turntable rather than taken from the source mesh.This makes the conditioning signal available during training match the one available at inference; reconstruction failures are filtered.Source: §3.2 · p. 5

3.3 GEOMETRY-AWARE CONDITIONING

  • The model encodes references independently and normals as a temporal sequence, then concatenates reference, normal, and target latents for transformer attention.The normal turntable is a trajectory-independent cue, so no frame-wise correspondence with the target sequence is imposed.Source: §3.3 · pp. 5–6

3.4 TRAINING AND INFERENCE

  • Training applies the rectified-flow loss only to target-video latent elements, while reference and normal latents remain clean context.At inference, 7 normal-condition and 20 target latents decode into 25 normal frames followed by 80 target frames.Source: §3.4 · pp. 6–7

4 EXPERIMENT

OmniShowcase leads the reported identity and 3D-consistency metrics on both benchmarks, with competitive text alignment and video quality. Ablations show that geometry conditioning and turntable coverage matter, though the outcomes vary by metric and benchmark.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses 50 SHOWCASEBENCH subjects and 35 NAVI subjects, with all main-comparison methods receiving the same four reference images.Identity and 3D metrics use masked subjects; text alignment and video quality use full frames.Source: §4.1 · p. 7

4.2 QUALITATIVE AND QUANTITATIVE COMPARISON

  • Phantom-MV and MAGREF-MV barely orbit, MV-S2V stops short and changes the teapot structure, and 3DreamBooth misses the target identity and geometry.The qualitative comparison evaluates trajectory following, identity, and 3D consistency for OmniShowcase and four baselines.Source: §4.2 · p. 7, Figure 5 · p. 8
  • OmniShowcase achieves the best DINO, CLIP, MEt3R, and GT-CD scores on both SHOWCASEBENCH and NAVI while keeping text alignment and video quality competitive.Table 2 compares nine metrics across the two benchmarks.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 ABLATION

  • Removing geometry worsens identity, MEt3R, and GT-CD; Textured has the largest text-alignment drop on SHOWCASEBENCH, while Sparse has the lowest auxiliary RotCoverage there.Textured has the lowest GT-CD among the three ablations on SHOWCASEBENCH, while removing geometry yields the highest GT-CD on NAVI; thinning the turntable's cost depends on reconstruction reliability.Source: §4.3 · pp. 8–9, Table 3 · p. 9, Figure 6 · p. 9

A ADDITIONAL QUALITATIVE RESULTS

Additional examples test trajectory following across subjects and camera motions. In a fixed-scene comparison, the subject retains its identity and shape as the camera path changes, while point-cloud visualizations expose differences in reconstructed geometry.

A ADDITIONAL QUALITATIVE RESULTS

  • Across four additional subjects and trajectories, OmniShowcase completes each commanded path while preserving the object; baselines either rotate less, reconstruct the subject during travel, or lose identifying surface detail.The comparison uses the same method ordering as Figure 5.Source: A ADDITIONAL QUALITATIVE RESULTS · p. 14, Figure 7 · p. 14
  • With the subject and background fixed, the subject keeps its identity and shape as the camera trajectory changes.The examples include arc, zoom-in, truck, full orbits, and zoom-out configurations.Source: A ADDITIONAL QUALITATIVE RESULTS · pp. 14–15, Figure 8 · p. 15, Figure 9 · p. 15
  • Point clouds from generated orbits show baselines covering only their camera-swept arcs, while OmniShowcase closes the volume measured against ground-truth geometry.Figure 10 shows each reconstruction from four viewpoints.Source: A ADDITIONAL QUALITATIVE RESULTS · p. 15

B USER STUDY

In a preference study of unseen videos, OmniShowcase was selected most often across all four criteria. The accompanying reconstruction visualization contrasts baselines’ thin swept shells with OmniShowcase’s closed volume.

B USER STUDY

  • The study asked 44 participants to choose among five results for 24 videos not seen during training.Participants saw the reference views and prompt alongside the results.Source: B USER STUDY · p. 15
  • Selections covered identity preservation, 3D consistency, prompt alignment, and overall quality.The criteria asked viewers to judge identity across viewpoints, consistent shape while turning, prompt following, and overall quality.Source: B USER STUDY · pp. 15–16
  • Reconstructions from generated orbits show baselines recovering thin shells of swept surfaces, whereas OmniShowcase closes the volume.Subjects are segmented from generated orbits and lifted to colored point clouds, the same reconstruction evaluated by GT-CD.Source: Figure 10 · p. 16
  • OmniShowcase received 70.9%–79.4% of votes across all four criteria, while no baseline exceeded 12.7%.Its largest margin was on prompt alignment, where it alone completed the commanded orbit rather than moving only a few degrees or remaining still.Source: B USER STUDY · p. 16, Figure 11 · p. 16

C IMPLEMENTATION DETAILS

The implementation fine-tunes Wan2.1 with LoRA, uses separate normal-condition and target segments, and jointly decodes them at inference. Geometry evaluation uses point-cloud distance and an alignment-dependent viewpoint-coverage diagnostic.

C IMPLEMENTATION DETAILS

  • Training fine-tunes Wan2.1 I2V-14B-480P with rank-128 LoRA while freezing other weights, using 25 normal-condition and 80 target frames per sample.The segments are encoded into 7 and 20 latent frames, respectively, then concatenated into a 27-frame sequence; training runs for 33,000 steps over four B200 GPUs in about 50 hours.Source: C IMPLEMENTATION DETAILS · pp. 16–17
  • Inference jointly decodes the 27-frame latent sequence into 105 frames: 25 normal-condition frames followed by 80 target frames.Sampling uses 50 steps with three-way classifier-free guidance, weighting reference and text streams at 5.0 each; clips are 480 × 832 at 30 fps.Source: C IMPLEMENTATION DETAILS · p. 17
  • GT-CD sums prediction-to-ground-truth and ground-truth-to-prediction Euclidean distances between point clouds.The two directed terms measure accuracy and completeness, respectively.Source: C IMPLEMENTATION DETAILS · p. 17
  • RotCoverage is an alignment-dependent viewpoint-coverage diagnostic that averages each reference pose’s nearest generated-pose distance.It compares normalized camera poses represented by five frustum vertices and reports a dimensionless score.Source: C IMPLEMENTATION DETAILS · p. 17

D REFERENCE-COUNT ABLATION

The full four-view setting performs best overall, with the clearest reference-count benefit appearing in GT-CD. The reported identity and prompt-adherence metrics vary less with reference count in the specified NAVI setting.

D REFERENCE-COUNT ABLATION

  • For fewer than four references, condition slots cycle the available views and the surface-normal turntable is rebuilt from exactly those views.The table caption states that missing views do not leak into the condition.Source: Table 4 · p. 17
  • The full four-view setting gives the lowest GT-CD on both SHOWCASEBENCH and NAVI.The table compares reference counts using the same checkpoint and evaluation protocol; its four-view row repeats the Table 2 OmniShowcase result.Source: D REFERENCE-COUNT ABLATION · p. 17, Table 4 · p. 17
  • NAVI identity scores stay nearly flat across reference counts, while ViCLIP is saturated for every count on both benchmarks.The passage attributes the NAVI identity pattern to small, largely convex subjects, for which one view already fixes appearance and extra references instead buy cross-view agreement.Source: D REFERENCE-COUNT ABLATION · p. 17

E TRAINING DATA COMPARISON

The paper compares paired training-data generations, tests single- versus four-view baseline inputs, documents the corpus curation yield, and specifies an 88-trajectory camera vocabulary.

E TRAINING DATA COMPARISON

  • Each corpus clip is paired with a Uni3C clip for the same subject and camera path, then scored with the same metrics and reference views.GT-CD uses the same source mesh, and Table 5 reports the paired comparison; the supplied passages do not state its outcome.Source: E TRAINING DATA COMPARISON · pp. 17–18, Table 5 · p. 18

F BASELINES EXTENSION

  • Phantom and MAGREF are compared with one reference view and four while checkpoint, sampler, trajectory sentence, and evaluation protocol remain fixed.The four-view setting follows MV-S2V; Tables 6 and 7 report the two settings side by side, with GT-CD using ground-truth geometry.Source: F BASELINES EXTENSION · p. 18, Table 6 · p. 18, Table 7 · p. 18

G.1 CURATION AND COMPOSITION

  • The released SHOWCASE-4K corpus contains 4,000 clips from 2,593 assets across 620 categories, after only 28% of recent-round composited clips survived G2 and G3.Across all rounds, 4,829 clips passed; G2 was the binding constraint, with most rejections involving changed structure or implausible support.Source: §G.1 · pp. 18–19

G.2 CAMERA TRAJECTORIES

  • The camera vocabulary defines 88 trajectories: 16 orbit-family paths and 72 paths from eight other motions, with rotation or framing-transition suffixes.Figure 12 draws 16 representative trajectories, showing sampled camera frusta, travel direction, and subject position.Source: §G.2 · p. 19, Figure 12 · p. 20

H FAILURE ANALYSIS

The failures arise from separate sources: generated backgrounds inherit limitations of the compositing data, while a static geometry condition cannot represent subjects changing pose. The appendix also specifies curation gates, caption formats, and inference prompts.

H FAILURE ANALYSIS

  • The backgrounds can look synthetic because training pairs a physically rendered subject with an inpainted scene whose light transport is inconsistent.The model reproduces the flat shading and sparse material detail of those generated surroundings.Source: H FAILURE ANALYSIS · pp. 20–21, Figure 13 · p. 21
  • Pose-changing references can fuse into a rigid reconstruction matching no actual pose; static conditioning then yields averaged poses or 3D tangles, a limitation left for future work.A motion prompt does not resolve the mismatch because the geometry condition does not encode pose changes.Source: H FAILURE ANALYSIS · p. 21

I PROMPTS AND RUBRICS

  • The final pipeline configuration covers curation, scene authoring, captioning, and inference prompts; quantitative evaluation uses no LLM judge.G0 screens meshes and quick renders for degeneracy; G1–G3 use criteria, and samples pass only when every criterion scores 1.Source: I PROMPTS AND RUBRICS · p. 21, §I.1 · pp. 21–22, Table 11 · p. 22

I.2 CATEGORY DESCRIPTIONS AND SCENE VOCABULARY

  • Category descriptions pair an appearance sentence and coarse type with compatible scene locations drawn from a 2,000-location scene bank.The bank includes surfaces, props, depth cues, and lighting.Source: §I.2 · p. 24

I.3 CAPTIONING

  • Tarsier2-7B captions the reconstruction turntable and composited target separately, with reconstruction-specific narration removed from the condition caption.The stripping rules retain visible appearance and distinctive parts while removing turntable, white-background, and reconstruction references.Source: §I.3 · p. 24, Condition Caption Format · p. 24

Condition Caption Format

  • The condition caption describes the object's visible appearance and distinctive features, beginning with “The video starts with”.Optional appearance details may follow; turntable rotation, white-background phrases, and reconstruction narration are removed.Source: Condition Caption Format · pp. 24–25

Target Caption Prompt

  • Target captions begin with the object, placement, and scene, then may describe camera motion, scene changes, or other visible action in two to three sentences.The prompt directs descriptions toward the object, its placement, surrounding scene, and visible action or change.Source: Target Caption Prompt · p. 25

I.4 INFERENCE PROMPTS

  • Because the captioner may omit camera-turn extent, a camera-mode sentence is appended; the model also receives condition and target captions, while baselines receive only the target caption.The trajectory sentence is selected from the motion and suffix vocabulary.Source: §I.4 · p. 25, Inference Prompt · p. 25
Loading 2610.35684v1…