Source-linked AI summary
Dior: Drawing the Light of Image via Material-Decoupled Illumination Representation
Xuanpu Zhang, Xuesong Niu, Haoxiang Cao, Ruidong Chen, Jianhao Zeng, Changqian Yu
TL;DR
Image relighting needs precise, controllable illumination changes, yet existing scribble-driven methods do not consistently map sparse inputs to intensity, chromaticity, and spatial effects. Dior-Light introduces the material-decoupled Lumi Map and trains with synthetic triplets plus real-pair reconstruction; experiments report improved accuracy over evaluated baselines and consistent sparse-stroke control on in-the-wild images.
Problem
Existing relighting methods provide limited explicit control over illumination intensity, chromaticity, and complex spatial distributions, while accurate illumination representations are difficult to obtain for real photographs.
Method
Dior-Light uses a material-decoupled Lumi Map with synthetic source–condition–target triplets and two-stage training combining rendered supervision with estimator-guided reconstruction on real relighting pairs.
Results
Experiments on three paired benchmarks improve relighting accuracy over evaluated baselines, while qualitative results show consistent responses to sparse hand-drawn controls on in-the-wild images.
Takeaways & Limitations
Dior-Light provides a practical approach to controlling spatial distribution, relative intensity, and chromaticity of direct illumination through hand-drawn controls.
Takeaways & Limitations
The Lumi Map retains only the direct-source component and is not asserted to approximate the complete image; indirect illumination can still depend on earlier-surface materials.
Abstract
from arXiv · showhide
Controllable image relighting is an important problem in image editing, and hand-drawn scribbles provide an intuitive interface for specifying the desired illumination. However, existing methods do not establish a consistent and effective mapping between scribble inputs and relighting results, limiting their ability to control illumination intensity, chromaticity, and complex spatial distributions. We address this limitation by introducing a material-decoupled illumination representation, termed the Lumi Map, which establishes an explicit mapping between user scribbles and the resulting illumination, thereby improving both relighting accuracy and controllability. Specifically, we use a renderer to synthesize source image-Lumi Map-relit image triplets and train the model to predict the target relighting result conditioned on the Lumi Map. To mitigate the domain gap introduced by synthetic data, we further perform reconstruction training on real relighting pairs, improving the model's generalization to real-world images. Finally, we present Dior-Light, an image relighting method controlled by hand-drawn strokes. Extensive experiments demonstrate that our method outperforms existing approaches in relighting accuracy and enables effective control over illumination intensity and chromaticity on in-the-wild images.
1 Introduction
Image relighting seeks illumination changes that preserve scene content, but scribble-based control remains limited in intensity, chromaticity, and complex spatial distributions. Dior-Light addresses this with the material-decoupled Lumi Map and training on synthetic and real relighting data.
- Motivation: Image relighting changes illumination while preserving viewpoint, geometry, reflectance, and subject identity, but accurate illumination control remains difficult.Decomposition-based approaches depend on accurate illumination representations, which are difficult to obtain for real photographs.
- Motivation: Existing scribble-driven methods primarily control illumination spatially, leaving intensity and chromaticity less explicit.Hand-drawn strokes provide an intuitive interface, but the mapping from sparse cues to complete illumination remains underdetermined.
- Method: The Lumi Map is a material-decoupled representation that encodes illumination spatial distribution, intensity, and chromaticity for controllable relighting.Unlike shading and Phong representations, it is designed to preserve material consistency and capture source-illumination structure.
- Method: The method synthesizes diverse, physically consistent source image–Lumi Map–relit image triplets and trains a Lumi Map-conditioned relighting model.It varies exposure, materials, viewpoints, and light-source attributes while exporting the Lumi Map in linear color space.
- Results: Experiments report state-of-the-art relighting accuracy and continuous, consistent illumination control on real-world images.Evaluation covers paired data with ground-truth targets and manually annotated real-world images.
- Contribution: The paper presents a unified framework combining material-decoupled illumination representation, relighting, and illumination estimation across synthetic and real-world data.Its final system is Dior-Light, a scribble-driven relighting method for real-world images.
2 Related Work
Prior relighting methods use diffusion models and different lighting representations, but scribble-guided approaches often lack complete control over light transport, magnitude, or chromaticity. Dior-Light positions the Lumi Map as a more expressive material-decoupled alternative.
- Diffusion-based relighting: Diffusion-based relighting improves realism by combining image priors with conditioning signals such as text or background images.These methods provide different degrees of illumination control but are not described here as resolving all scribble-control requirements.
- Scribble-guided relighting: SmartShadow, LightPainter, and ScribbleLight progressively support sparse lighting guidance but remain limited by artistic shadows, Phong assumptions, portrait scope, or binary annotations.The cited limitations concern photorealism, visibility and distance attenuation, unconstrained photographs, and continuous magnitude or chromaticity.
- Illumination representations: Phong, intrinsic-image shading, and related decomposition methods expose illumination at different abstraction levels but can conflate lighting with material or geometry.These representations provide editable image-space layers or appearance terms without fully decoupling illumination from scene properties.
3 Illumination Representation with Material Decoupling
The Lumi Map is derived by canonicalizing receiver materials and retaining only direct-source illumination, yielding a pixel-aligned signal independent of the original material field. It preserves direct-light support and shadow geometry while encoding illumination magnitude and chromaticity as relative control values.
- Design goals: The derivation seeks a dense, sparsifiable signal that preserves image-space support, relative magnitude, and chromaticity while excluding the original material response.The signal retains geometric factors governing direct illumination and cast shadows under a steady-state, surface-only light-transport model.
- Material decoupling: Canonicalizing the receiver replaces its BRDF with a white, unit-albedo Lambertian constant, removing the receiver’s original material response.This step alone does not remove material dependence from indirect illumination caused by earlier scattering surfaces.
- Direct illumination: The Lumi Map retains only the direct-source component rather than approximating complete image formation, using each pixel’s first camera-visible surface intersection.This deliberately removes material-dependent interreflection from the representation.
- Material independence: The resulting signal depends on geometry, target lighting configuration, and camera, but contains no factor from the original material field.It preserves direct-illumination and cast-shadow support while excluding receiver response and material-colored interreflection.
- Control semantics: Pixel-aligned RGB values encode relative direct-light magnitude and chromaticity, while visibility and attenuation determine light-and-shadow support.Adjusting Lumi Map values linearly modulates the encoded direct illumination, but the values are relative control signals rather than calibrated radiometric measurements.
4 Method
Dior-Light trains a hand-drawn relighting model from pixel-aligned rendered supervision and sparsified Lumi Maps, then adapts it to real image pairs through estimator-guided reconstruction. A shared network learns both relighting and Lumi Map estimation across synthetic and real-data objectives.
- 4.1 From Dense Rendering Supervision to Sparse Strokes: Rendered supervision pairs each sRGB image with a pixel-aligned canonical-material Lumi Map under shared scene, camera, geometry, and source conditions.The rendering pipeline varies scenes, viewpoints, source positions, and chromaticity while preserving pixel correspondence.
- 4.1 From Dense Rendering Supervision to Sparse Strokes: Dense Lumi Maps are coarsened, masked, or reduced to freehand strokes, while retained pixels preserve local relative magnitude and chromaticity.Masked regions remain unspecified, creating a sparse-to-dense spectrum of lighting controls.
- 4.2 Two-Stage Training with Rendered Supervision and Real Reconstruction: Stage 1 jointly trains a shared network for relighting from target Lumi Maps and estimating Lumi Maps from images using dense and sparsified rendered pairs.The relighting mode maps source image and target illumination to the target image, while the estimation mode maps an image to its Lumi Map.
- 4.2 Two-Stage Training with Rendered Supervision and Real Reconstruction: Stage 2 adds real-pair reconstruction by estimating a target-derived pseudo-control with stopped gradients, while synthetic relighting and estimation supervision remains active.Continuing synthetic supervision limits exposure to erroneous estimated conditions during real-data training.
5 Experiments
Experiments evaluate Dior-Light on paired benchmarks, in-the-wild photographs, ablations, and freehand lighting control. The method achieves strong relighting accuracy and supports consistent manipulation of illumination properties.
- Experimental Setup: Experiments use full-reference comparisons on three paired benchmarks, qualitative comparisons on in-the-wild photographs, component ablations, and freehand lighting-control applications.The benchmarks are ReNe, MultiIllum, and SynthBench; each contains 600 held-out image pairs.
- Experimental Setup: The comparison includes ScribbleLight, IC-Light, Banana, and Qwen-Edit, with each baseline evaluated using its most suitable conditioning format.
- Comparison with Existing Methods: Our method achieves the best relighting accuracy across all evaluation settings while enabling effective illumination control.Existing methods show limited illumination control and substantial discrepancies from target relit images.
- Comparison with Existing Methods: Across ReNe, MultiIllum, and complex synthetic scenes, Dior-Light better matches shadow and highlight locations, illumination intensity, material appearance, and chromaticity.Competing methods struggle particularly in complex synthetic scenes because of insufficient control.
- Comparison with Existing Methods: On in-the-wild images, Dior-Light remains responsive to scribbles and produces realistic, physically consistent relighting, including shadows and water-surface reflections.Existing methods may miss the desired illumination direction or color or generate low-quality images.
- Ablation Study: Replacing the Lumi Map degrades performance, especially on SynthBench, while removing Stage 2 causes clear drops on real-image benchmarks.Qualitative ablations show less reliable scribble following, color shifts, abrupt facial transitions, and inconsistent foreground-background blending.
- Application: Drawing Light in Image Space: Users can progressively add strokes while adjusting illumination chromaticity and intensity to create customized and complex relighting effects.
6 Conclusion
Dior-Light combines the Lumi Map with synthetic triplet supervision and real-pair reconstruction training for hand-drawn image relighting. Experiments show improved accuracy and consistent sparse-control responses without explicit geometry or material estimation.
- Dior-Light uses the material-decoupled Lumi Map to encode direct illumination’s spatial distribution, relative intensity, and chromaticity.
- The method combines pixel-aligned synthetic training triplets with estimator-guided reconstruction on real relighting pairs.
- Across three paired benchmarks, the representation and training strategy improve relighting accuracy over evaluated baselines, while qualitative results show consistent sparse hand-drawn control on in-the-wild images.
- Dior-Light controls illumination distribution, intensity, and chromaticity without explicit geometry or material estimation or per-image optimization.