PaperlayerView exact revision on arXiv

Source-linked AI summary

RADIANT: Object-Centric Feed-Forward Global Illumination

Sarah Zhang, Zhiyang Dou, Minghao Guo, Benjamin Tod Jones, Wojciech Matusik

arXiv:2610.32075v1cs.CVcs.GRcs.LG

TL;DR

Triangle-token feed-forward renderers face attention costs that grow with mesh resolution, despite transport being governed by surface geometry and object arrangement. RADIANT uses object-centric patch tokens and 3D scene transport with ray-guided view decoding. On the 1.03M-scene benchmark it reaches 25 dB in one-fifth the steps of RenderFormer and achieves 32.2 dB at 600k steps.

  • Problem

    Triangle-token renderers have O(T^2) self-attention, while macroscopic transport depends on surface geometry and object arrangements rather than mesh discretization.

    Source: §1 · pp. 1–2

  • Method

    RADIANT pools point inputs into 32 surface-patch tokens per object, models inter-object transport with invariant form-factor attention, and decodes camera rays guided by point-splat visibility.

    Source: §1 · pp. 2–3, §3.4 · p. 5, §3.5 · p. 6

  • Results

    32.2 dB tone-mapped PSNR at 600k steps, +3.3 dB over RenderFormer and +7.6 dB over RenderFormer-V2; RADIANT reaches 25 dB in 5× fewer steps than RenderFormer.

    Source: §1 · p. 2, §5 · p. 11

  • Takeaways & Limitations

    RADIANT's transport sequence length scales with object count rather than polygon count, and its 500-object runtime-only forward pass takes 0.83 s.

    Source: §1 · p. 2, §5 · p. 11

    The uniform 32-token budget can over-allocate tokens to planar walls and under-sample geometrically complex objects.

    Source: §5 · p. 11

Abstract

from arXiv · show

Feed-forward neural renderers synthesize global illumination across novel 3D scenes without per-scene optimization or runtime ray tracing. Existing transformer-based renderers tokenize individual triangles, so self-attention scales quadratically with mesh resolution. Yet macroscopic radiative transport is governed by the underlying surface geometry and object arrangements rather than a particular mesh discretization; finer tessellation can thus inflate token counts without changing the underlying transport physics, motivating a discretization-agnostic and object-centric representation. We introduce RADIANT (RADiance-based Inter-Actor Neural Transformer), an object-centric architecture for feed-forward global illumination from point inputs. By representing each object with a compact budget of 32 surface-patch tokens, RADIANT decouples attention sequence length from polygon counts. In a single network, an inter-object scene transformer models 3D radiative exchange and a view decoder renders images at target camera poses. To capture light transport without materializing O(N^2) bias matrices, invariant form-factor attention embeds geometric distances and surface orientations directly into query-key dot products as SE(3)-invariant inner products weighted by learned area and material gains. For primary visibility without mesh connectivity, camera rays cross-attend scene tokens guided by depth-tested point splats in a shared spherical frame. Across a 1.03M-scene benchmark under matched model capacity, RADIANT reaches 25 dB tone-mapped PSNR in 5x fewer training steps than RenderFormer, and achieves 32.2 dB at 600k steps (+3.3 dB over RenderFormer and +7.6 dB over RenderFormer-V2). We further demonstrate the scalability of RADIANT, which completes a 512x512 forward pass of a 500-object scene in 0.83 s, with computational cost scaling with the number of objects rather than polygons.

1 INTRODUCTION

RADIANT represents objects with compact surface-patch tokens to decouple global-transport attention from polygon tessellation. It combines 3D scene transport with camera-ray view decoding and improves benchmark quality and runtime over RenderFormer.

1 INTRODUCTION

  • Existing triangle-token renderers incur O(T^2) self-attention, so latency and GPU memory grow rapidly with mesh resolution.Windowed attention and object-level pooling still require explicit triangle geometry and primitive-level attention during encoding.Source: §1 · p. 1
  • RADIANT uses 32 surface-patch tokens per object, decoupling transport sequence length from polygon counts and supporting meshes and point clouds.The representation follows the premise that macroscopic light transport is unchanged when fixed geometry and materials are represented by 50 or 50,000 triangles.Source: §1 · p. 2
  • A point encoder extracts 32 patches per object, and an inter-object transformer models radiative exchange with attention over 32N tokens for N scene entities.Invariant form-factor attention incorporates distance and surface orientation into query-key products with learned area and material gains, avoiding explicit pairwise bias matrices.Source: §1 · pp. 2–3
  • For novel views, camera rays cross-attend scene tokens, while point-splat depth, normals, and materials guide primary visibility without mesh connectivity.The view decoder can then focus on multi-bounce interreflections, soft contact shadows, and color bleeding.Source: §1 · pp. 2–3
  • 32.2 dB tone-mapped PSNR at 600k steps, +3.3 dB over RenderFormer and +7.6 dB over RenderFormer-V2; RADIANT reaches 25 dB in 5× fewer steps than RenderFormer.The 500-object runtime-only benchmark takes 0.83 s, 8.5× faster than RenderFormer with FlashAttention; rendering quality is evaluated on scenes with up to 50 entities.Source: §1 · p. 2

2 RELATED WORK

Prior work spans per-scene fitting, triangle-based feed-forward rendering, and meshless or alternative representations for global illumination. RADIANT instead pools point inputs into object-level surface patches and models transport in 3D before ray-based decoding.

Per-scene neural rendering

  • Per-scene neural rendering avoids path tracing after training but requires scene-specific fitting and does not generalize to novel geometric arrangements.RADIANT targets feed-forward prediction across novel scenes without test-time optimization.Source: §2 · p. 3

Feed-forward rendering of explicit geometry

  • RenderFormer and RenderFormer-V2 tokenize triangle meshes, while RenderFormer++ pools triangle tokens only after full self-attention over each object's triangles.RADIANT instead operates directly on point inputs via set abstraction without primitive attention.Source: §2 · p. 3

Point-based and alternative representations for global illumination

  • Point-based prior work learns direct-to-indirect illumination or diffuse transport, while other methods aggregate virtual point lights or predict screen-space shading from G-buffers.RADIANT pools points into 32 surface-patch tokens per object, models multi-bounce transfer in 3D, and decodes HDR radiance along camera rays guided by a point-splat G-buffer.Source: §2 · p. 3

Rotary embeddings and attention biases

  • RADIANT's invariant form-factor attention encodes distance and surface-facing angles as exact inner products weighted by gains predicted from surface area and material attributes.This follows the inner-product approach to avoid explicit O(N^2) pairwise bias matrices; the framework also uses CamRay parameterization, FiLM, and register tokens.Source: §2 · p. 3

3 METHOD

RADIANT processes point-based scenes through compact object tokens, models radiative exchange once per scene, and decodes HDR images for query viewpoints. Its geometry-aware attention and rasterizer-guided ray decoding support mesh-free transport and visibility.

3.1 ARCHITECTURE

  • The architecture computes scene transport once, then uses a viewpoint-conditioned decoder to cross-attend camera-ray tokens to scene tokens and produce full-resolution HDR radiance.The scene transformer models inter-object transport; rasterized geometry channels guide view decoding.Source: Figure 1 · p. 4, §3.1 · p. 4

3 METHOD

  • Each object is represented by 2,048 surface samples with geometry and material attributes, and the network predicts the HDR radiance image for a novel scene in one forward pass.Raw point clouds are ingested directly without surface reconstruction; mesh-derived samples are used in the primary benchmarks.Source: §3 · p. 4

3.2 OBJECT TOKENIZATION

  • PointNet++ abstraction selects 32 centroids per object and pools their neighborhoods into 1,024-wide patch tokens; scene-transformer outputs are invariant to object input ordering.Sixteen learnable register tokens absorb extraneous attention weight.Source: §3.2 · p. 4

3.3 RADIOSITY STATISTICS AND CONDITIONING

  • Shared per-object radiosity statistics condition attention through learned receiver and sender encodings, while FiLM pathways add shape and camera-relative pose conditioning.The statistics include a log area proxy, mean diffuse albedo, mean emission, and mean specular reflectance.Source: §3.3 · p. 5

3.4 INVARIANT FORM-FACTOR ATTENTION

  • Invariant form-factor attention factorizes distance and orientation terms into query-key features, adding a learned, material-dependent bias without a pairwise bias matrix.The factorization captures exact O(3)-invariant quadratic arguments of the differential radiative form factor; cosine clamping, inverse-square attenuation, and mutual visibility are left to learned transformer layers.Source: §3.4 · p. 5

3.5 VIEW-DEPENDENT DECODING: RAY TOKENS, RASTERIZER CHANNELS, AND SPHERICAL CROSS-ATTENTION

  • Depth-tested point splats provide mesh-free primary-hit geometry, while spherical cross-attention RoPE encodes angular ray-object offsets and explicit depth for occlusion reasoning.The rasterizer supplies visible object identity, depth, normal, and material per pixel; it does not explicitly encode incident illumination or shadows.Source: §3.5 · p. 6

3.6 TRAINING OBJECTIVE

  • Training combines mean absolute error in log-radiance space with a perceptual term on tone-mapped images, using the same loss and optimizer setup across models except for the stated RenderFormer-V2 difference.The tone map is τ(L) = clip[0,1](log2(1 + L)); the perceptual term has weight 0.05.Source: §3.6 · p. 6

4 EXPERIMENTS

Under a matched training protocol, RADIANT converges faster and achieves higher held-out rendering quality than the RenderFormer baselines. Ablations and runtime tests examine its transport, visibility, token, and scaling properties.

4.2 RENDERING QUALITY

  • RADIANT reaches 25 dB in 5× fewer optimizer steps than RenderFormer: 16k versus 80k at the same global batch size.At step 6k, RADIANT leads RenderFormer by +7.30 dB (23.45 vs. 16.15 dB).Source: §4.2 · p. 7, Figure 2 · p. 7
  • RADIANT maintains a 3.1–3.5 dB margin across entity counts and leads on 500 novel assets and crowded scenes with 25–50 entities.On crowded held-out scenes, it resolves contact shadows, fine silhouettes, and color bleeding where both baselines show artifacts.Source: §4.2 · pp. 7–8, Table 3 · p. 8, Table 4 · p. 8
  • At 600k steps, RADIANT reaches 32.21 dB per-image PSNR, exceeding RenderFormer by +3.26 dB and RenderFormer-V2 by +7.63 dB.Table 4’s held-out row repeats the 600k evaluation.Source: §4.2 · p. 7, Table 4 · p. 8
  • With partial point clouds missing 52%–72% of foreground-object inputs, RADIANT reaches 26.33 dB zero-shot; 400-step fine-tuning raises novel-view PSNR to 30.30 dB.Fine-tuning uses four reference views; the novel views are not used for fitting.Source: §4.2 · p. 8, Table 5 · p. 8, Figure 4 · p. 10

4.3 ABLATIONS

  • Restricting scene attention to intra-object tokens costs 2.46 ± 0.66 dB, while removing invariant form-factor attention costs 0.52 ± 0.08 dB.The precision control reports a 0.53 ± 0.02 dB drop when invariant terms are zeroed while float32 attention is retained.Source: §4.3 · p. 8, Table 6 · p. 10
  • Removing rasterizer channels lowers PSNR by 3.12 ± 0.08 dB and raises LPIPS by 56%, despite those channels carrying only unshaded geometry.The ablation also reports a 5.66 ± 0.06 dB drop when decoding the G-buffer without 3D scene tokens in a model lacking invariant form-factor attention.Source: §4.3 · p. 10, Table 6 · p. 10
  • Varying the patch budget from 8 to 64 changes PSNR by at most 0.04 dB, while pooling RenderFormer triangles into 32 tokens per object lowers its PSNR by 0.38 ± 0.09 dB.Sixteen tokens achieve 29.47 dB versus 29.43 dB with 32 tokens and reduce training step time by 17%.Source: §4.3 · p. 10

4.4 INFERENCE EFFICIENCY AND SCALING WITH SCENE COMPLEXITY

  • RADIANT is slower on scenes up to 20 objects, becomes faster beyond 25 (35 with FlashAttention), and runs 8.5× faster than RenderFormer with FlashAttention on synthetic 500-object scenes.In crowded regimes of 25–50 entities beyond the training distribution, rendering quality drops for all evaluated models.Source: §4.4 · p. 10

5 CONCLUSION

RADIANT learns feed-forward global illumination from point inputs using compact object-centric surface patches, and its attention length is decoupled from polygon counts. The conclusion reports benchmark performance and runtime results alongside limitations and reproducibility details.

5 CONCLUSION

  • RADIANT reaches 25 dB in one-fifth the steps of RenderFormer, achieves 32.2 dB at 600k steps, and runs 8.5× faster on 500-object scenes.The speed comparison is from a runtime-only benchmark.Source: §5 · p. 11
  • Compact object-centric surface patches decouple attention sequence length from polygon counts, while invariant form-factor attention embeds radiative-transfer geometry in query-key inner products.The method avoids explicit pairwise bias tables and does not require mesh connectivity or per-scene optimization.Source: §5 · p. 11
  • A uniform budget of 32 patch tokens can over-allocate capacity to planar walls and under-sample geometrically complex objects.Full self-attention over view-decoder ray tokens limits scaling beyond 1024² resolution; the point-splat rasterizer handles only direct surface intersections.Source: §5 · p. 11
  • The work uses synthetic scenes assembled from publicly available Objaverse assets, with no human subjects or personal data.The stated scene licenses include Creative Commons terms, which apply to scenes containing assets with non-commercial or share-alike conditions.Source: ETHICS STATEMENT · p. 12
  • The authors state that they will release the data-generation pipeline, scene corpus, training configurations, and evaluation code.The matched-budget protocol and appendices specify the model architectures, parameter counts, dataset generation, losses, optimizers, and evaluation schedules.Source: REPRODUCIBILITY STATEMENT · p. 12

A ERROR ANALYSIS AND SCENE COMPLEXITY

Rendering-quality comparisons remain consistent across scene-complexity subsets, while the analysis identifies concentrated error regions and states geometric and permutation properties of the method. The appendix proofs explain how these properties follow from the architecture and attention construction.

A ERROR ANALYSIS AND SCENE COMPLEXITY

  • Errors concentrate on thin structures, which 7×7 rasterizer dilation renders slightly thicker, and at object–cast-shadow boundaries where hit channels and routing bias change discontinuously.These error locations are reported in Fig. 3.Source: A ERROR ANALYSIS AND SCENE COMPLEXITY · p. 15
  • At 600k steps, RADIANT leads RenderFormer by 3.1–3.5 dB and RenderFormer-V2 by 7.3–7.9 dB across every scene-complexity bucket.Absolute PSNR varies by at most 1.1 dB with entity count; LPIPS rises from the sparsest to most crowded bucket by 37% for RADIANT, 47% for RenderFormer, and 31% for RenderFormer-V2.Source: A ERROR ANALYSIS AND SCENE COMPLEXITY · p. 15

B.1 STATEMENTS

  • The scene transformer is object-permutation equivariant, and the rendered image is invariant to object relabeling when rasterization selects the same surface at every pixel.The condition excludes depth ties between different objects.Source: §B.1 · p. 15, §B.2 · pp. 15–16
  • Rotary-block attention logits depend on patch positions through their relative offset, so a common translation of all centroids leaves those logits unchanged.The proof also states that content vectors are translation-invariant because the encoder uses centered samples and the FiLM and radiosity statistics are translation-invariant.Source: §B.1 · p. 15, §B.2 · p. 16
  • For camera rays and object centroids, the first two spherical coordinates coincide when a ray passes through the centroid, and their difference gives the view-angle offsets.The coordinates remain unchanged under changes in intrinsics or resolution that preserve the physical rays.Source: §B.1 · p. 15, §B.2 · p. 16
  • Invariant form-factor attention factorizes geometric terms into per-token inner products and remains invariant to rigid motions, with logits permuting alongside the tokens.The stated degree-two O(3)-invariant polynomials reduce to entries of Gij and the squared normal magnitudes; the latter contribute query-only or key-only terms.Source: §B.1 · p. 15, §B.2 · p. 16

C TRAINING DETAILS

The training details describe shared-resolution, matched-configuration experiments, a staged learning-rate schedule, and a checkpoint-selection procedure applied consistently across the three models.

C TRAINING DETAILS

  • All models train from scratch at 512² on NVIDIA A100 GPUs with global batch size 24 and no data augmentation.RADIANT and RenderFormer use a shared training loop; RenderFormer-V2 uses its released loop with the data loader replaced for these scenes.Source: C TRAINING DETAILS · pp. 16–17
  • The learning rate warms up for 8,000 steps to 5 × 10^-5, then cosine-anneals to 10^-6 at 100k steps, with continued schedules through 600k.The main comparison is at 600k steps; continuation carries over model and optimizer state.Source: C TRAINING DETAILS · p. 17
  • Checkpoints are evaluated on all 4,112 held-out images with identical metric definitions, and the best checkpoint is selected by highest per-image PSNR.Random-half selection checks chose the 600k checkpoint in every halving for both RenderFormer generations and in 99.9% of choices for RADIANT.Source: C TRAINING DETAILS · p. 17

D DATA STATISTICS

The corpus uses independently sampled scene compositions from curated Objaverse meshes, with a fixed held-out boundary and path-traced image generation. The accompanying table captions specify corpus-statistics and evaluation-point scope.

D DATA STATISTICS

  • Scenes contain one to six foreground objects from 18,085 curated Objaverse meshes in a template room, with randomized lights, materials, and camera viewpoints.Each scene is rendered from four cameras at 512² with 512 samples per pixel; held-out scenes use the same mesh pool but independently sampled compositions, arrangements, materials, emitters, and viewpoints.Source: D DATA STATISTICS · p. 17, Table 9 · p. 18
  • The held-out split is the last 0.1% of scenes in a fixed order, with point-sampled and triangle representations verified to have identical scene sets and ordering.The split boundary falls on the same scene for both representations.Source: D DATA STATISTICS · p. 17
  • Table 8 reports per-image PSNR and LPIPS on 4,112 held-out images at step 25k, then every 50k steps, plus early evaluations at 6k, 16k, and 80k.The table caption marks unevaluated entries with a dash and bolds the best value of each run.Source: Table 8 · p. 18

E RUNTIME ANALYSIS

RADIANT’s runtime scales with object count rather than mesh triangle count, but its speed advantage depends on scene size, resolution, and whether views can share scene computation. In the reported large-scene tests it is faster at inference, while small-scene and high-resolution multi-view workloads expose costs from its ray-token decoder.

Complexity

  • At 512², RADIANT’s scene transformer attends over 32N + 16 tokens at O(S²), while its predictor costs O(QS) + O(Q²); these terms do not depend on triangle count.Rasterization is one splat pass over NP point inputs, and the DPT head is linear in image area; triangle-level attention instead costs O(T²).Source: E RUNTIME ANALYSIS · p. 18

Component breakdown

  • From 10 to 200 objects, RADIANT’s view-dependent portion dominates at 10 and 50 objects (68% and 58%) and grows from 50 to 75 ms, while its encoder and scene transformer grow with object count.RenderFormer’s triangle transformer grows from 19 ms to 1.20 s with FlashAttention and reaches 94% of its 200-object forward pass (96% with PyTorch attention).Source: E RUNTIME ANALYSIS · pp. 18–19, Table 10 · p. 19

Crossover

  • At 512², RenderFormer with SDPA is faster through 20 objects (70.9 versus 77.2 ms at 20), while RADIANT is faster from 25 objects; against FlashAttention, the crossover is between 30 and 35 objects.At 128² and 256², RADIANT is faster from 15 objects, or from 10 objects at 128² against FlashAttention.Source: E RUNTIME ANALYSIS · p. 19

Tessellation

  • With ten objects, RenderFormer’s time rises with tessellation while RADIANT’s input and time remain unchanged; RADIANT is 1.5–2.5× faster from 2,048 triangles per object and 155×–412× faster at 32,768 triangles.The upper speedup is 412× with SDPA and the lower is 155× with FlashAttention.Source: E RUNTIME ANALYSIS · p. 19, Table 12 · p. 19

Training-scene scale

  • On small corpus scenes using one A100, RADIANT takes 130 ms per single-view forward versus RenderFormer’s 42–49 ms, and 1.09 s per training step versus 0.72–0.91 s.Despite the slower steps, reaching 25 dB takes 4.8 h for RADIANT versus 15.9–20.3 h for RenderFormer.Source: E RUNTIME ANALYSIS · p. 19, Table 13 · p. 20

Large-scene latency

  • In 50–500-object scenes, RADIANT is up to 22.2× faster at inference than RenderFormer with SDPA and 8.5× faster than with FlashAttention.At 50–200 objects, RADIANT training steps are 4.3–18.0× faster than SDPA and 2.3–7.5× faster than FlashAttention; at 500 objects RADIANT exceeds 80 GB of GPU memory in training, while RenderFormer needs 32.7 GiB.Source: Figure 5 · p. 20

Batched throughput

Multi-view amortization

  • At 512², RADIANT’s view-dependent part costs 52–58 ms, 2.1–3.3 times RenderFormer’s, so RenderFormer has lower amortized per-view cost on the tested 8- and 50-object scenes at 64 views.At 256² the view-dependent parts are comparable, and the view-independent component determines the per-view cost; RADIANT is ahead at 50 objects.Source: E RUNTIME ANALYSIS · p. 20, Table 14 · p. 21

F ARCHITECTURE

The architecture uses object-level patch tokens for scene transport and camera-ray tokens for view synthesis. Its implementation combines geometric descriptors, radiosity-conditioned attention, and point-rasterizer guidance with a DPT image head.

View decoding

  • The predictor uses spherical ray/object coordinates and a learned routing-bias gate per layer; CamRay assumes square images with a centered principal point.Its DPT head has 256 features and output channels 128, 256, 512, and 1,024.Source: F ARCHITECTURE · pp. 21, 23

Module configuration

  • RADIANT has 205,942,972 parameters; its scene transformer uses 16 register tokens, while the predictor fuses the last three scene-transformer layers and decodes 4 × 4 pixel-patch ray tokens with a DPT head.The predictor also receives 14 rasterized surface channels through a zero-initialized 1 × 1 convolution.Source: F ARCHITECTURE · p. 21, Table 18 · p. 22

Geometric encoding

  • Each scene-transformer patch token carries a centroid and the principal axes of its parent object’s point samples, forming a 12-coordinate descriptor encoded with rotary frequencies.The descriptor is split into a 96-dimensional rotary block and a 32-dimensional pass-through block; the object axes are covariance eigenvectors scaled by their standard deviations.Source: F ARCHITECTURE · p. 22

Invariant form-factor attention

  • Invariant form-factor attention appends 16 features per head to queries and keys, using learned receiver and sender gains over eight radiosity statistics; its added gain networks contribute 0.05M parameters.The invariant attention layers run in float32, with scene-centred, RMS-scaled centroids; the gain MLPs’ final sender layer starts at zero.Source: F ARCHITECTURE · p. 23

G SEED REPLICATES OF THE ABLATIONS

The replicate analyses assess uncertainty in the headline comparison and test which architectural components contribute to image quality. The reported ablations distinguish rasterizer guidance, inter-object attention, and invariant form-factor attention from auxiliary conditioning and token-budget choices.

Rasterizer controls

  • Removing scene tokens while retaining rasterizer channels and the splat costs 5.66 ± 0.06 dB and triples log-radiance error (0.062 versus 0.021), more than removing rasterizer channels alone.Removing the splat into the output head leaves PSNR unchanged (+0.01 ± 0.07 dB); the authors identify rasterizer channels as surface cues and scene tokens as global shading context.Source: G SEED REPLICATES OF THE ABLATIONS · p. 23, Table 20 · p. 24

Replicated ablations

  • Removing rasterizer channels costs 3.12 ± 0.08 dB and removing invariant form-factor attention costs 0.52 ± 0.08 dB; removing FiLM costs 0.15 ± 0.02 dB and removing register tokens costs 0.13 ± 0.01 dB.The first two drops have the same sign in every seed; the full model’s seed-to-seed standard deviation is 0.01 dB.Source: G SEED REPLICATES OF THE ABLATIONS · pp. 23–24

Inter-object attention

  • Restricting scene-transformer attention to within-object tokens lowers per-image PSNR by 2.46 ± 0.66 dB across seeds 42–47, with the largest reported drop (2.87 dB) in scenes with at most seven entities.The pooled drop is 1.27 ± 0.36 dB; the loss is present at every scene size and is larger in sparser scenes.Source: G SEED REPLICATES OF THE ABLATIONS · p. 24

Invariant form-factor attention

  • The full model reaches 29.96 ± 0.01 dB per-image PSNR versus 29.43 ± 0.07 dB without invariant channels, a paired gain of 0.52 ± 0.08 dB across seeds 42–44.Freezing invariant gains at zero yields 29.43 ± 0.03 dB, while removing OBB-RoPE yields 29.87 ± 0.04 dB; the invariant channels’ gain is not explained by float32 precision alone.Source: G SEED REPLICATES OF THE ABLATIONS · pp. 24–25

Token-budget control

  • Changing the per-object token budget among 8, 16, 32, and 64 changes the three-seed mean by at most 0.04 dB on the model without invariant form-factor attention.The fixed 32-token budget is not the binding constraint at this ablation scale.Source: G SEED REPLICATES OF THE ABLATIONS · p. 25

Pooling control

  • Pooling RenderFormer’s triangle tokens to 32 per object lowers its PSNR from 23.56 ± 0.21 to 23.19 ± 0.15 dB, while RADIANT reaches 29.96 ± 0.01 dB under the same schedule.The paired pooling drop is 0.38 ± 0.09 dB; the authors conclude the architectural difference is not explained by token budget alone.Source: G SEED REPLICATES OF THE ABLATIONS · p. 25, Table 22 · p. 25

Uncertainty of the main comparison

  • At 600k steps, RADIANT leads RenderFormer by +3.26 dB (95% CI [+3.12, +3.39]) and RenderFormer-V2 by +7.63 dB (95% CI [+7.45, +7.81]) on paired per-image PSNR.The intervals bootstrap held-out scenes, not training variance; each model was trained once, and per-image PSNR standard deviations across three ablation seeds were 0.01 dB for RADIANT and 0.21 dB for RenderFormer.Source: G SEED REPLICATES OF THE ABLATIONS · p. 25

H DESIGN TRADE-OFFS

RADIANT’s fixed object-level token budget makes computation independent of mesh resolution, while its design trade-offs include difficult objects with strongly varying surface appearance. On unseen objects it retains an advantage over both baselines, but scenes with many entities reduce accuracy for all three models.

H DESIGN TRADE-OFFS

  • The fixed token budget makes computation independent of mesh resolution, but objects whose appearance varies strongly across their surface remain a hard case; a variable per-object budget is proposed as a natural extension.On the model without invariant form-factor attention, varying the budget from 8 to 64 tokens moves the mean by at most 0.04 dB, and 16 tokens match 32 at 17% lower step time.Source: H DESIGN TRADE-OFFS · p. 25, Table 23 · p. 26
  • With rasterizer and decoder fixed, scene-transformer variants span 2.5 dB (27.50 to 29.96 dB): removing scene tokens costs 5.66 ± 0.06 dB, while restricting attention within objects costs 2.46 ± 0.66 dB.The rasterizer supplies local surface grounding but no indirect illumination or shadows; removing invariant form-factor attention costs 0.52 ± 0.08 dB.Source: H DESIGN TRADE-OFFS · p. 26
  • At 512², reducing RADIANT’s ray-token density from 4 × 4 to RenderFormer’s 8 × 8 patches costs 0.60 ± 0.09 dB, less than the 1.59 ± 0.25 dB cost of removing inter-object attention in that ablation model.Removing the rasterized splat leaves PSNR unchanged, so late convolutional upsampling does not bypass cross-attention transport.Source: H DESIGN TRADE-OFFS · p. 26

I UNSEEN ASSETS AND LARGER SCENES

  • On unseen assets with 25–50 entities, RADIANT loses 7.3 dB relative to subset A yet leads RenderFormer by +2.81 dB and RenderFormer-V2 by +5.26 dB.Smaller objects alone are easier for all three models; the authors attribute the subset-B accuracy drop to entity count rather than object size.Source: I UNSEEN ASSETS AND LARGER SCENES · p. 27
  • On unseen assets at the training scene size, each model scores 0.32–0.41 dB above its held-out result; RADIANT leads both baselines on subsets A and C as well as B.RADIANT’s margins are +3.23 and +7.69 dB on A, and +2.61 and +8.52 dB on C, versus RenderFormer and RenderFormer-V2 respectively.Source: I UNSEEN ASSETS AND LARGER SCENES · pp. 26–27

J IMPERFECT GEOMETRY AND ROBUSTNESS

Tests on incomplete and corrupted point inputs show how RADIANT’s rendering quality changes with missing geometry and input noise. Related-work comparisons distinguish learned transport for unseen scenes from methods computing transport for a given scene, while the limitations bound geometry detail and benchmark conditions.

J IMPERFECT GEOMETRY AND ROBUSTNESS

  • With single-camera partial scans, only 28%–48% of point inputs survive visibility testing; zero-shot PSNR is 26.33 dB on new views versus 34.24 dB with complete geometry.The scans retain only observed portions of objects; unscanned surfaces render as voids.Source: J IMPERFECT GEOMETRY AND ROBUSTNESS · p. 27
  • After 400-step adaptation, partial scans reach 37.51 dB on fitted views versus 39.02 dB with complete geometry, and 30.30 dB on new views.On new views, partial-scan fine-tuning recovers 50% of the 7.91 dB lost to missing surfaces.Source: J IMPERFECT GEOMETRY AND ROBUSTNESS · p. 27
  • With foreground-only corruption, 512 points per entity or position noise σ = 0.01 yields 27.1 and 28.5 dB, versus RenderFormer’s 28.7 dB on clean input.Corrupting all entities degrades performance more steeply because rasterized hit channels propagate geometric noise into the view decoder.Source: J IMPERFECT GEOMETRY AND ROBUSTNESS · p. 27, Table 24 · p. 28

K ADDITIONAL RELATED WORK

  • Unlike hierarchical radiosity and meshless hierarchical bases, which compute transport for a given scene, RADIANT learns one mapping for unseen scenes.Prior neural approaches include online path-tracer adaptation, generators for one parameterized scene, mesh-vertex features, and object-centric feature grids; RADIANT is trained across a million scenes.Source: K ADDITIONAL RELATED WORK · pp. 27–28
  • RADIANT uses depth-tested point splats with 7 × 7 dilation for its primary G-buffer, predicts lighting in one pass, and traces no rays.Pointersect infers point-cloud surface intersections to enable ray tracing, including global illumination.Source: K ADDITIONAL RELATED WORK · p. 28

L LIMITATIONS, FUTURE WORK, AND BROADER IMPACT

  • A fixed per-object sample and token budget limits representable geometric detail, and small batch-size-1 scenes favor triangle baselines before quadratic scaling dominates.Baselines use matched capacity and budget rather than their authors’ schedules; training uses synthetic scenes assembled from public Objaverse assets.Source: L LIMITATIONS, FUTURE WORK, AND… · p. 28

Related papers

Loading 2610.32075v1…