PaperlayerView exact revision on arXiv

Source-linked AI summary

Depth-to-RGB: Repurposing a Frozen Depth Estimator for Geometry-Guided Compositing

Sanghyun Jo, Chae Yeon Lim, Donghwan Lee, Sihyun Kim, Soo Ye Kim, Kyungsu Kim

arXiv:2610.09125v1cs.CV

TL;DR

Reference-based object compositing lacks an explicit geometric condition for the completed scene, leaving the intended object shape and placement unspecified by its standard inputs. D2R predicts composite depth from those inputs with a frozen depth estimator and holds that condition fixed during RGB synthesis. The reported system improves geometry, photometric quality, and reference identity across its evaluations.

  • Problem

    Background, reference, and 2D mask inputs do not specify the reference object’s depth and shape in the completed scene, leaving compositing without an explicit geometric condition.

    Source: §1 · p. 1

  • Method

    D2R learns reference-conditioned corrections to a frozen depth estimator from completed-scene encoder features, then uses the predicted depth as a fixed condition for RGB synthesis.

    Source: §1 · p. 2

  • Results

    The complete pipeline improves intermediate depth and final RGB geometry, photometric quality, and reference identity; feature targets also reduce OOD Stage-1 AbsRel by 31.4% versus decoded-depth supervision.

    Source: §5 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    The approach extends depth estimation from observed images to intended composites, providing a reusable geometric condition for RGB synthesis.

    Source: §6 · p. 9

    Because predicted depth stays fixed during RGB synthesis, errors in object shape or placement can persist, and depth does not specify color or texture changes needed to match the background.

    Source: §5 · p. 9

Abstract

from arXiv · show

Reference-based object compositing inserts or replaces an object using a background image, a reference image, and a 2D compositing mask. These inputs guide appearance and placement but leave the completed scene's geometry implicit, which can distort object structure or alter the surroundings. Our Depth-to-RGB (D2R) framework predicts composite depth for a scene not yet observed in the RGB inputs. It learns reference-conditioned corrections to a frozen depth estimator using encoder features of paired completed scenes as targets. The unchanged decoder maps the corrected representation to the intended scene's depth, which a separately trained renderer holds fixed during RGB synthesis. Under matched architecture and training, encoder-feature supervision reduces OOD Stage-1 AbsRel by 31.4% relative to decoded-depth supervision. We also introduce AnyInsertion++ with paired in-distribution and category-disjoint splits to evaluate generalization beyond compositing training categories. The complete D2R system leads 12 open-source and 3 closed-source baselines in estimator-derived geometry and photometric quality on both paired splits. On category-disjoint data, D2R reduces AbsRel by 43.7% and improves PSNR by 2.4 dB over the matched RGB baseline. Across three unpaired benchmarks, D2R leads both identity metrics and reduces mean CLIP reference cosine distance by 55% relative to the strongest baseline. Project page: https://shjo-april.github.io/Depth2RGB/

1 Introduction

Reference-based compositing lacks an explicit description of the completed scene’s geometry, although the background, reference, and mask do not determine the intended object depth and shape. D2R predicts composite depth before RGB synthesis; it leads the reported baselines on paired and unpaired evaluations.

Problem

  • The background, reference image, and 2D mask leave the completed scene’s geometry implicit, so they do not specify the reference object’s shape and placement in the target scene.Existing approaches use observed-image depth, assembled foreground and background maps, or a proxy rendered from a user-posed 3D object.Source: §1 · p. 1

Evaluation

  • AnyInsertion++ expands paired evaluation from 38 to 426 cases and adds a category-disjoint split; three unpaired benchmarks assess reference identity across photographic, cartoon, and artistic backgrounds.The unpaired evaluations use TF-ICON, AIComposer, and ComplexCompo.Source: §1 · p. 2

Approach

  • D2R learns reference-conditioned corrections to a frozen depth estimator using completed-scene encoder features, then keeps the predicted depth fixed during RGB synthesis.Under matched architecture and training, encoder-feature supervision reduces OOD Stage-1 AbsRel by 31.4% versus decoded-depth supervision.Source: §1 · p. 2

Results

  • D2R outperforms 15 open- and closed-source baselines on reported metrics across both paired splits; on unpaired benchmarks, it leads both identity metrics and reduces mean reference cosine distance by 55%.The category-disjoint comparison reports 43.7% lower AbsRel and 2.4 dB higher PSNR than the matched RGB baseline.Source: §1 · p. 2

2 Related Work

Prior compositing work transfers reference appearance or uses other geometric conditions, while D2R predicts completed-scene geometry from standard compositing inputs. Its direct baseline enables a matched evaluation of adding depth prediction and supervision.

Implicit Geometry in RGB Compositing

  • Insert-Anything provides a direct baseline because its in-context canvas and LoRA support a matched evaluation of adding depth prediction and supervision.D2R predicts composite depth before RGB generation, unlike 2D structural cues that can preserve local shape but do not specify scene-wide depth.Source: §2 · p. 2

Geometry Before RGB Synthesis

  • Prior geometry methods use input-background depth, assembled reference and scene depth, or a posed 3D proxy; D2R instead predicts completed-scene depth from standard compositing inputs.Person-In-Situ predicts completed-scene depth for human insertion using rendered body depth, while 3DIS generates scene depth from instance layouts before RGB.Source: §2 · pp. 2–3

Sampling and Conditioning

  • Other approaches adjust RGB generation during inference or augment encoders and generators, whereas D2R predicts a requested static composite representation from background, reference, and mask.VGGT-World predicts future geometry features from video observations and decodes them through frozen geometry components.Source: §2 · p. 3

3 Method

D2R factorizes compositing into predicting composite depth from the background, reference, and mask, then generating RGB conditioned on that prediction. The depth estimator stays frozen while Stage 1 learns feature corrections and Stage 2 trains a renderer to use depth as a fixed condition.

3 Method

  • D2R predicts one composite-depth estimate from the background, reference, and mask, then generates RGB conditioned on that estimate.Stage 2 uses pθ(I | ˆD, X), with trainable LoRA parameters.Source: §3 · p. 3

3.1 Stage 1: Predicting Composite Depth from Reference and Background

  • Stage 1 freezes the depth estimator’s encoder and decoder, training mask-conditioned injectors to correct background features toward encoder features from paired composites.Corrections span the token grid and propagate through later frozen blocks, allowing object and background geometry to be inferred jointly; zero-initialized output projections initially recover the frozen estimator on the masked-background input.Source: §3.1 · pp. 3–4

3.2 Stage 2: Synthesizing RGB with Fixed Composite Depth

  • Stage 2 arranges the masked background, reference, and composite depth on an input canvas, then generates RGB only in the masked composite region while keeping the depth patch fixed.Training uses a separate supervision canvas containing the paired composite, reference, and target depth D; depth denoising and RGB-depth consistency target D even when the input depth patch contains ˆD.Source: §3.2 · p. 5

4 Experiments

D2R leads the reported baselines on paired geometry, photometric quality, and unpaired identity evaluations. AnyInsertion++ supplies paired in-distribution and category-disjoint cases, while the unpaired benchmarks test identity across varied visual styles.

Datasets

  • AnyInsertion++ contains 52.9K curated training samples and 426 paired evaluation cases: 287 in-distribution and 139 from categories absent from compositing training.The unpaired sets provide 332 TF-ICON, 272 AIComposer, and 300 ComplexCompo cases.Source: §4 · p. 5

Metrics

  • Paired depth and RGB metrics compare generated composites with targets, while unpaired CLIP-I and DINO-I compare generated object crops with reference crops.AnyInsertion++ uses object masks, whereas unpaired benchmarks use coarse box-shaped target masks.Source: §4 · p. 5

Paired and Unpaired Performance

  • D2R outperforms all 15 open- and closed-source baselines on every reported geometric, photometric, and identity metric across both paired splits.Relative to the strongest baseline for each metric, it reduces AbsRel by at least 22% and LPIPS by at least 19%, while improving PSNR by at least 1.0 dB.Source: §4 · p. 5, Table 2 · p. 6, Table 3 · p. 6
  • On three unpaired benchmarks, D2R leads both identity metrics, reducing mean reference cosine distance by 55.4% with CLIP and 22.6% with DINO over Insert-Anything∗.The benchmarks cover photographic, cartoon, and artistic backgrounds.Source: §4 · pp. 5–6, Table 4 · p. 7

Geometry from Standard Compositing Inputs

  • Replacing Stage-1 predictions with ground-truth composite depth lowers rendered-image AbsRel by about 8 to 12% and raises PSNR by 0.5 to 0.8 dB on paired splits.D2R’s predicted condition already supports leading performance on unseen categories without a completed target image, measured depth, or user-posed 3D proxy at inference.Source: §4 · p. 6

5 Ablation Study and Discussion

The ablations compare ways to predict and use composite depth, finding that D2R’s fixed geometry condition improves both geometric and photometric metrics across the paired splits. Further tests examine supervision, depth conditioning, robustness, and generalization.

Learning the Completed Scene’s Depth Features

  • Feature targets lower ID and OOD Stage-1 AbsRel by 30.9% and 31.4%, and lower OOD rendered-image AbsRel from 0.089 to 0.067.Adding decoded-depth loss improves ID by about 5% but worsens OOD by about 3% and reduces the largest single-GPU batch by about 30%.Source: §5 · p. 7, Table 5 · p. 7

Feature Correction Versus Diffusion-Based Depth Generation

  • D2R reduces OOD AbsRel from 0.1740 to 0.0751 (56.8%) versus a diffusion-based depth generator, using one feed-forward pass instead of 28 denoising steps.The comparison evaluates depth-prediction strategies; Table 5 instead isolates supervision within the same predictor.Source: §5 · p. 7

Predicting Geometry Before RGB Synthesis

  • D2R predicts geometry before RGB synthesis and improves both metrics on both splits, but the compared implementations also differ in geometry construction and training.These comparisons therefore do not isolate generation order.Source: §5 · p. 8, Table 6 · p. 8

Learning to Use the Depth Condition

  • With depth supply fixed, combining depth-patch denoising and RGB-depth consistency improves on either objective alone; uniform consistency weighting yields 0.075 AbsRel and 22.8 dB PSNR, versus 0.067 and 24.0 dB with 1 − t weighting.All conditioned variants use Stage-1 predictions at evaluation.Source: §5 · p. 8, Table 7 · p. 8

Learning from Accurate and Predicted Depth

  • Selection training gives the lowest OOD AbsRel under both GT and predicted depth inputs: 0.059 with GT depth and 0.067 with predictions.Selection pairs either intact input map with the same GT target, exposing the renderer to prediction errors without using them as supervision targets.Source: §5 · p. 8, Table 8 · p. 8

Correcting Object and Scene Features Together

  • Restricting correction to masked tokens raises average Stage-1 AbsRel by about 2.5%, while sharing one injector across four layers raises ID and OOD error by about 10% and 3%.Injection also improves all six depth backbones against their own targets, supporting reuse across estimators without ranking absolute accuracy.Source: §5 · p. 9

Adapting Different RGB Generators

  • Across SD2, SDXL, Qwen, and FLUX.1-Fill, adaptation with identical depth input reduces OOD AbsRel by 18% to 32% and increases PSNR by 1.08 to 2.13 dB.The Stage-1 predictor takes 56.5 ms and 5.21 GB peak inference memory on RTX PRO 6000; depth errors can persist, and depth does not specify background-matching color or texture changes.Source: §5 · p. 9

Consistency and Generalization

  • D2R improves OOD full-frame AbsRel over Insert-Anything∗ in 103 of 139 cases, lowers non-rigid AbsRel from 0.133 to 0.071 across 20 cases, and ranks first under five depth scorers.Gains occur inside objects and along their boundaries; five default training runs show about 1% average relative variation across metrics.Source: §5 · p. 9

6 Conclusion

D2R predicts features of an intended composite from its background and reference, then uses the resulting depth as a fixed geometric condition for RGB synthesis.

6 Conclusion

  • D2R predicts intended-composite features with a frozen depth estimator and provides a reusable geometric condition for RGB synthesis without a depth sensor or user-posed 3D proxy.The conclusion also reports that feature correction adapts multiple depth estimators and the resulting condition supports different RGB generators.Source: §6 · p. 9

A.1 Geometric Conditions for Reference-Based Compositing

  • The comparison criteria distinguish whether a method predicts intended composite depth jointly across the scene, uses standard inputs, and fixes geometry before RGB synthesis.A full-frame map assembled from separate depths does not count as jointly predicted scene geometry.Source: §A.1 · p. 17
  • D2R makes scene-wide relative depth explicit before RGB synthesis, unlike compositing methods whose features can encode shape and geometry implicitly.Insert-Anything is the matched baseline because D2R retains its rendering interface while adding depth prediction and supervision.Source: §A.1 · p. 17
  • Person-In-Situ predicts completed-scene depth for human insertion using rendered body depth, whereas D2R uses a frozen general depth prior and needs no human-body representation.The distinction is in inputs and learned representation, not in the depth-before-RGB ordering.Source: §A.1 · p. 18

A.2 Scene Synthesis and Geometry Control in Related Tasks

D2R differs from related geometry-control methods by predicting composite depth from standard compositing inputs before RGB synthesis. Its appendix describes the feature-prediction and rendering setup, including a width trade-off for the depth injector.

A.2 Scene Synthesis and Geometry Control in Related Tasks

  • Related scene-synthesis methods use foreground/background depth, instance layouts, explicit transformations, existing 3D scenes, or test-time control; D2R predicts composite depth from standard compositing inputs.D2R predicts depth once and holds it fixed throughout RGB sampling.Source: §A.2 · pp. 18–19

A.3 Representation Prediction and Joint RGB-Depth Modeling

  • D2R predicts features of a requested static composite from background, reference, and mask, then decodes them with the unchanged depth decoder.Its target differs from future-geometry features predicted from video or representations aligned to RGB-generator hidden states.Source: §A.3 · pp. 19–20

B.1 Notation and Denoising Objective

  • Training corrupts a supervision canvas containing the paired composite, reference, and target depth, while a separate input canvas supplies the masked background, reference, and conditioning depth.At inference, the reference, depth, and unmasked background remain observed inputs and are restored after each update.Source: §B.1 · pp. 20–21

B.2.1 Encoder Corrections and Capacity

  • At decoder-read encoder taps, mask-conditioned injectors attend to background and reference features and add residual corrections; the frozen blocks propagate corrections to higher taps.Each tap has independent injector parameters, and the injector is the only trainable part of Stage 1.Source: §B.2.1 · p. 21

B.2.2 Injector Width and Cost

  • 0.0462 ID and 0.0751 OOD AbsRel at width 640, compared with 0.0445 ID and 0.0755 OOD AbsRel at width 896; the larger injector costs more without improving OOD accuracy.Parameters rise from 272.4M to 529.2M, and latency from 56.5 to 67.6 ms.Source: §B.2.2 · p. 21

B.2.3 Feature Supervision and Structural Ablations

Feature supervision targets representations the frozen decoder already maps to composite depth, while the structural ablations examine correction scope and attention design.

B.2.3 Feature Supervision and Structural Ablations

  • Feature supervision asks the injector to recover paired-composite encoder features, retaining the pretrained feature-to-depth mapping that decoded-depth supervision does not constrain directly.Different feature configurations can produce similar depth, so a close output match need not recover the paired representation.Source: §B.2.3 · p. 21
  • The feature-matching argument gives no numerical bound on test depth error or local contact accuracy and does not establish generalization.The Lipschitz constant is not estimated, and the training loss averages per-tap, sample-normalized errors rather than the assumed unnormalized norm.Source: §B.2.3 · p. 22
  • Full-grid corrections outperform the masked-token ablation in the tested setting; one proposed explanation is that object and background features adapt together to represent relative depth.The authors caution that this does not mean insertion physically moves unchanged background surfaces.Source: §B.2.3 · p. 22
  • Joint attention lowers full-frame AbsRel by approximately 2% on ID and 1% on OOD versus reference-only cross-attention; adding the gate lowers error by a further 1% and 3%, respectively.The ungated variant varies more across checkpoints.Source: §B.2.3 · p. 22

B.2.4 Feature Targets Versus Decoded-Depth Targets

Encoder-feature supervision improves composite-depth prediction over decoded-depth supervision under matched training, and its advantage also carries through the same renderer. The appendix specifies the backbone interfaces, canvas, training and inference procedures, and computational costs.

B.2.4 Feature Targets Versus Decoded-Depth Targets

  • Encoder-feature supervision lowers ID and OOD AbsRel by 30.9% and 31.4%, respectively, relative to decoded-depth-only supervision; OOD error falls from 0.1094 to 0.0751.The comparison holds architecture and training iterations fixed; feature supervision also avoids the decoder backward pass required by the alternative.Source: §B.2.4 · pp. 22–23
  • Adding decoded-depth supervision to feature-only training lowers ID full-frame AbsRel by about 5% but raises OOD error by about 3%.It also reduces the largest single-GPU batch by about 30%; in this setting, the ID/OOD tradeoff and memory cost favor feature-only training.Source: §B.2.4 · p. 23
  • With the same trained renderer, feature-supervised depth yields lower rendered-image AbsRel than decoded-depth-supervised depth.This tests predictor utility for a renderer trained with the default depth-supply strategy, not renderers retrained for each predictor.Source: §B.2.4 · p. 23, Table B3 · p. 23

B.2.5 Backbone Interfaces and Parameter Sharing

  • Across tested depth backbones, tap placement and feature dimensions are adapted to each interface, while independent tap-specific injectors let each encoder layer learn its own correction.The weight-sharing alternative reduces parameter count but uses the same number of attention blocks.Source: §B.2.5 · p. 23

B.3.1 Canvas and Fixed Observations

  • The input canvas places the masked background beside the reference and conditioning-depth patches, while inference restores observed latent positions after every sampling update.The supplied reference-object mask selects the source object; the target mask marks the editable region, and the evaluations use both silhouette and coarse box masks.Source: §B.3.1 · pp. 23–24

B.3.2 Training Objectives and Depth Selection

  • Stage-2 training uses equally weighted losses, keeps target depth fixed even when conditioning depth is predicted, and selects predicted or target depth per sample with equal probability.RGB-depth consistency compares normalized depth estimated from the generated composite with target depth; its contribution is reduced for noisy early estimates.Source: §B.3.2 · p. 24

B.3.3 Training and Inference Algorithms

  • Inference predicts composite depth once, samples the masked composite region for T steps while restoring observed canvas positions, then decodes and resizes the result.FLUX.1-Fill receives the reference through Redux; LoRA is merged before deployment.Source: §B.3.3 · p. 25, §B.3.2 · p. 25

B.4 Computational Cost

  • On the RTX PRO 6000, peak training memory is 72.4 GiB for the width-640 injector and 66.8–94.0 GiB across the reported Stage-2 renderer setups.The Stage-2 readings vary by backbone and configuration and do not isolate backbone memory efficiency.Source: §B.4 · p. 26

C.1.1 Paired Data and Depth Targets

This part describes AnyInsertion++ paired data and depth targets, evaluation and training protocols, and results for depth prediction and reference agreement. Across 139 OOD cases, all four D2R-adapted renderers improve AbsRel and PSNR over their same-backbone pretrained counterparts; five-run checks found approximately 1% average relative changes across applicable metrics.

C.1.1 Paired Data and Depth Targets

  • AnyInsertion++ retains 52,918 of 77,745 training samples and evaluates 287 ID and 139 OOD cases; D2R lowers AbsRel in all four rigidity groups.OOD categories are absent from training; the split has 119 rigid and 20 non-rigid OOD cases. The paired-composite depth target is estimator-derived, not sensor ground truth.Source: §C.1.1 · pp. 26–27, Table C3 · p. 28

C.1.2 Dataset Quality Control Prompt

  • The quality-control prompt checks object presence, mask alignment and validity, instance identity, and background/composite agreement, retaining samples scoring at least 8.The threshold was informed by human inspection of 300 randomly sampled images per score tier; usability required agreement from at least two of three annotators.Source: §C.1.1 · pp. 26–27, §C.1.2 · p. 28

C.1.3 Unpaired Evaluation and Identity Margins

  • Mean CLIP cosine distance falls from 0.215 to 0.096 (55.4%), and mean DINO distance from 0.268 to 0.208 (22.6%) across the unpaired benchmarks.Matched Insert-Anything* is the strongest baseline in all six benchmark-and-encoder comparisons; these embedding scores measure reference agreement, not identity accuracy or compositing-failure frequency.Source: §C.1.3 · p. 29

C.1.4 Baseline Evaluation Protocol

  • The matched Insert-Anything* baseline is retrained on AnyInsertion++ with D2R’s data and configuration, but uses an empty depth patch and only RGB denoising loss.The released checkpoint is excluded because its original training corpus can overlap the evaluation splits; commercial editors produce one valid output per case and are scored on the same cases.Source: §C.1.4 · p. 29

C.2.1 Depth Backbones and Training Settings

  • Adding the injector lowers AbsRel and raises δ1 in every within-estimator comparison across the tested depth backbones.Absolute scores do not support a controlled architecture ranking because targets and sample exposure differ across estimators.Source: §C.2.1 · p. 30, Table C4 · p. 31

C.2.2 Feature Correction Versus Diffusion-Based Composite Depth Generation

  • The injector reaches 0.0462 ID and 0.0751 OOD AbsRel, reducing error versus the adapted FLUX route by 56.7% and 56.8%, respectively.The comparison evaluates the specified FLUX.1-Fill and Redux implementation; architecture, conditioning, and objective vary together. The injector predicts depth in 56.5 ms with 5.21 GB peak inference memory on RTX PRO 6000.Source: §C.2.2 · p. 31, Table C5 · p. 32

C.3.1 Backbones and Prompt Conditioning

  • The renderer comparison tests SD2-Inpainting, SDXL-Inpainting, Qwen-Inpainting, and FLUX.1-Fill with the same three-patch canvas and depth conditioning throughout inference.FLUX.1-Fill receives reference appearance through the canvas and Redux, without a text prompt; the other three use text instructions without a separate image-prompt encoder.Source: §C.3.1 · p. 32

C.3.2 Training and Sampling Settings

  • Stage 2 ablation variants are evaluated after 100K training iterations, with rank-256 LoRA used for SD2-Inpainting, Qwen-Inpainting, and FLUX.1-Fill.The variants use exported Stage-1 predictions, auxiliary objectives, and per-sample depth selection; SDXL uses separate AdamW settings.Source: §C.3.2 · pp. 32–33
  • Across five training seeds, relative changes in applicable geometric, photometric, and identity metrics averaged approximately 1% overall.Sampling-seed checks separately indicated stable performance.Source: §C.3.2 · p. 33

C.3.3 Comparison Across Diffusion Backbones

  • All four adapted renderers improve AbsRel and PSNR over same-backbone pretrained renderers on 139 OOD cases receiving identical Stage-1 predicted depth.Resolution, conditioning, and training differences prevent a controlled ranking across architectures.Source: §C.3.3 · p. 33

C.3.4 Loss Ablations and Sampling Comparisons

The section lays out comparisons among renderer losses, depth-input strategies, and alternative geometry-provisioning implementations. It also specifies how depth predictions are aligned and scored, but the supplied passages do not report the ablation outcomes.

C.3.4 Loss Ablations and Sampling Comparisons

  • Table 7 compares renderer loss configurations, including depth denoising, RGB-depth consistency, and uniform versus 1 − t consistency weighting, under matched training settings.Conditioned rows use per-sample 50:50 predicted-or-target depth during training and Stage-1 predicted depth at evaluation.Source: §C.3.4 · p. 33
  • Table 8 varies training-time depth input among target-only, predicted-only, blending, and per-sample selection, then evaluates each renderer with target and predicted depth.The other training settings and target-depth supervision are retained.Source: §C.3.4 · p. 33
  • Latent optimization applies the same correction at all 28 steps, while joint denoising generates depth alongside RGB; Figure C1 shows one-step trajectories, whereas Table 6 evaluates D2R with Stage-1 predicted depth.The geometry-provisioning strategies differ in both depth construction and training, so their results compare complete implementations rather than isolating conditioning time.Source: §C.3.4 · pp. 33–34, Figure C1 · p. 34

C.4.1 Depth Alignment and Geometric Metrics

  • Depth metrics align predictions to targets by ordinary least squares on background pixels outside a dilated target mask, then score the aligned maps over the full frame.If fewer than 100 background pixels remain, fitting uses the full frame; reported δ1 uses a threshold of 1.25, and the metrics compare aligned estimator outputs rather than calibrated physical depth.Source: §C.4.1 · pp. 34–35

C.4.2 Paired Uncertainty and Regional Evaluation

D2R’s paired depth advantage persists under alternative bootstrap procedures and extends to the inserted object and nearby scene content. It ranks first under every tested depth scorer, while identity and geometry metrics use complementary evaluation protocols.

C.4.2 Paired Uncertainty and Regional Evaluation

  • Both alternative bootstrap intervals exclude zero for the OOD comparison with Insert-Anything∗, supporting persistence of the improvement.The cluster-bootstrap interval over 82 image IDs is [−0.0785, −0.0286], and the BCa interval is [−0.0749, −0.0354].Source: §C.4.2 · p. 35, Table C6 · p. 35
  • Removing the 14 most D2R-favorable OOD cases or 29 ID cases leaves a negative mean difference in all four comparisons.The largest gains contribute to, but do not account for, the full-frame advantage.Source: §C.4.2 · p. 35, Table C6 · p. 35
  • On 139 OOD cases, D2R lowers interior and boundary-band AbsRel versus both comparators, with values of 0.0509 and 0.1491, respectively.All four paired intervals exclude zero, but the OOD interior δ1 gain over Insert-Anything∗ reverses after excluding the largest 10% of improvements.Source: §C.4.2 · p. 35, Table C7 · p. 36

C.4.3 Cross-Scorer Evaluation

  • Cross-scorer evaluation holds generated images fixed, scores generated and ground-truth images with each estimator, and retains D2R’s Stage-1 predictions.Absolute errors are compared within each scorer because depth representations and scales differ.Source: §C.4.3 · p. 36
  • D2R ranks first under every scorer on both splits, although the remaining methods change order.Across four alternative scorers, Spearman correlations with the Depth Anything V2 ordering range from 0.888 to 0.985 on ID and 0.824 to 0.976 on OOD.Source: §C.4.3 · p. 36, Table C8 · p. 36, Table C9 · p. 37

C.4.4 Photometric and Identity Metrics

  • Identity and paired geometry metrics use complementary targets and crops, so their scores are not directly comparable across protocols.Paired identity compares ground-truth and predicted composite crops; unpaired identity compares the reference object with the predicted composite.Source: §C.4.4 · pp. 36–37

C.5 Limitations

The limitations concern geometry errors passed into rendering, appearance mismatches that depth cannot resolve, and boundaries on what the evaluation establishes.

C.5 Limitations

  • An incorrect footprint, contact height, or occlusion order in fixed depth can pass into RGB, and relative depth leaves metric scale undetermined.Multiple geometries may fit the inputs even though paired evaluation uses one target; thin and transparent structures remain difficult.Source: §C.5 · p. 37
  • Geometric plausibility does not ensure appearance harmonization: larger palette and texture changes remain difficult, and depth alone cannot resolve RGB-to-grayscale mismatch.The passage notes photographic fur shading in an ink drawing and retained color in a grayscale scene; geometry-guided harmonization and relighting are future work.Source: §C.5 · p. 37
  • Agreement across depth scorers supports method ranking, but does not establish physical accuracy or local contact correctness.The matched RGB baseline also cannot separate the pretrained depth prior from 272.4M added injector parameters.Source: §C.5 · p. 37

D Additional Qualitative Results

The appendix adds qualitative composites across paired splits, external benchmarks, illustrated scenes, and point-cloud views. The figures show predicted depth alongside RGB outputs, while the point-cloud views expose inconsistencies in monocular predictions rather than independently validating 3D geometry.

D Additional Qualitative Results

  • Figures D1–D5 show predicted depth and resulting RGB in columns seven and eight; OOD examples use object tags absent from compositing training.Figure D1 covers the ID split, Figure D2 the OOD split, and Figures D3–D5 TF-ICON, AIComposer, and ComplexCompo.Source: D Additional Qualitative Results · p. 38, Figure D1 · p. 38, Figure D2 · p. 39
  • Illustrated-scene examples show predicted shape and placement translated into composites, while retaining palette or texture mismatches in some cases.The examples include cartoon and illustrated scenes; the appendix discusses remaining harmonization errors.Source: D Additional Qualitative Results · p. 40, Figure D3 · p. 40, Figure D4 · p. 41
  • Figure D6 back-projects outputs using their own estimated depth and background alignment; off-axis views expose inconsistencies but do not independently measure 3D geometry.Pale regions mark unsupported disocclusions, and dotted circles indicate the stated large inserted-object disparity criterion; no displayed D2R panel exceeds it.Source: D Additional Qualitative Results · p. 43, Figure D6 · p. 43
Loading 2610.09125v1…