Source-linked AI summary
DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network
Yeying Jin, Aashish Sharma, Robby T. Tan
TL;DR
Single-image shadow removal lacks scalable paired training data, and Mask-ShadowGAN cannot handle soft shadows because it uses binary masks. DC-ShadowNet uses domain-classifier guidance and complementary unsupervised losses, and experiments report robust hard- and soft-shadow handling with improved performance over existing state-of-the-art methods.
Problem
Existing learning-based shadow-removal methods typically require costly paired shadow and non-shadow images, while Mask-ShadowGAN cannot handle soft shadows because it uses binary masks.
Method
DC-ShadowNet integrates shadow/shadow-free domain classifiers into its generator and discriminator and adds chromaticity, shadow-robust feature, and boundary-smoothness losses.
Results
Experiments report that DC-ShadowNet handles both hard and soft shadows and outperforms existing state-of-the-art shadow-removal methods quantitatively and qualitatively.
Takeaways & Limitations
The unsupervised network can learn from unlabeled real shadow images and can be further improved through test-time refinement.
Takeaways & Limitations
Entropy minimization can fail on surfaces close to achromatic, although the other unsupervised losses can still produce proper shadow-removal results.
Abstract
from arXiv · showhide
Shadow removal from a single image is generally still an open problem. Most existing learning-based methods use supervised learning and require a large number of paired images (shadow and corresponding non-shadow images) for training. A recent unsupervised method, Mask-ShadowGAN~\cite{Hu19}, addresses this limitation. However, it requires a binary mask to represent shadow regions, making it inapplicable to soft shadows. To address the problem, in this paper, we propose an unsupervised domain-classifier guided shadow removal network, DC-ShadowNet. Specifically, we propose to integrate a shadow/shadow-free domain classifier into a generator and its discriminator, enabling them to focus on shadow regions. To train our network, we introduce novel losses based on physics-based shadow-free chromaticity, shadow-robust perceptual features, and boundary smoothness. Moreover, we show that our unsupervised network can be used for test-time training that further improves the results. Our experiments show that all these novel components allow our method to handle soft shadows, and also to perform better on hard shadows both quantitatively and qualitatively than the existing state-of-the-art shadow removal methods. Our code is available at: \url{https://github.com/jinyeying/DC-ShadowNet-Hard-and-Soft-Shadow-Removal}.
1. Introduction
Single-image shadow removal remains difficult: supervised methods require costly paired data, while existing unsupervised approaches struggle with soft shadows. DC-ShadowNet addresses this by combining domain-classifier guidance with unsupervised losses for robust hard- and soft-shadow removal.
- Single-image shadow removal remains an open problem, despite applications including image editing and scene relighting.
- Supervised deep-learning methods require large collections of paired shadow and non-shadow images, which are expensive to obtain.
- Mask-ShadowGAN uses binary shadow masks and can produce incorrect outputs, making it unsuitable for soft-shadow images.
- DC-ShadowNet integrates shadow/shadow-free domain classifiers into its generator and discriminator so both focus on shadow regions.
- Its unsupervised losses use physics-based shadow-free chromaticity, shadow-robust perceptual features, and boundary smoothness.
- The method is designed to remove both hard and soft shadows from a single image.
2. Related work
Prior shadow-removal methods rely on restrictive assumptions, user interaction, annotations, or supervised data, while Mask-ShadowGAN remains limited by adversarial guidance and binary masks. DC-ShadowNet instead uses domain-classifier guidance and complementary unsupervised losses to target shadow regions and support soft-shadow removal.
- Physics-based methods assume chromatic surface colors and are designed for hard shadows, so they are vulnerable on achromatic surfaces.
- Some non-learning methods require user brush strokes or annotated ground truth to model or relight shadow regions.
- Many deep-learning methods address shadow detection and removal through architectures such as conditional GANs, image decomposition, context modules, or recurrent attention.
- Mask-ShadowGAN relies mainly on adversarial training without strong guidance and uses binary masks, limiting shadow-free output reliability and soft-shadow handling.
- DC-ShadowNet guides its generator and discriminator with domain classifiers that distinguish shadow from shadow-free inputs.
- The network adds chromaticity, shadow-robust feature, and boundary-smoothness losses to guide shadow removal beyond adversarial training.
3. Proposed Method
DC-ShadowNet transforms shadow or shadow-free inputs into shadow-free outputs using domain classification to focus the generator and discriminator on shadow regions. It combines chromaticity, feature, attention, and consistency mechanisms to guide unsupervised shadow removal.
- Network Architecture: DC-ShadowNet uses a generator to remove shadows from shadow inputs while reconstructing shadow-free inputs unchanged.A second generator, discriminator, and shadow mask support reconstruction consistency and shadow-region localization.
- Shadow-Free Chromaticity Loss: The shadow-free chromaticity loss guides output chromaticity toward a physics-based target obtained through entropy minimization and illumination compensation.Entropy minimization removes shadows in log-chromaticity space but can introduce color shifts that illumination compensation corrects.
- Shadow-Robust Feature Loss: A VGG-16 feature loss matches output and input-shadow features because selected pretrained features are less affected by shadows and retain structural information.The loss compares Conv22 feature maps from the shadow input and generated shadow-free output.
- Limitations: Entropy minimization can fail on nearly achromatic surfaces, but the other unsupervised losses can still support proper shadow removal.This limitation concerns recovery of the physics-based shadow-free chromaticity map rather than necessarily the final output.
- Domain Classification: Domain classifiers in the generator and discriminator highlight shadow regions and let the network selectively focus on them.The generator classifier produces an attention map, while the discriminator classifier is pretrained because generated-image domains can be ambiguous.
4. Experiments
DC-ShadowNet is evaluated on hard- and soft-shadow datasets against unsupervised, weakly supervised, supervised, and traditional methods. It achieves strong quantitative and qualitative results, while test-time training further improves soft-shadow performance.
- Results on Hard Shadows: DC-ShadowNet achieves the best performance among baseline methods on the SRD and AISTD hard-shadow datasets.Compared with Mask-ShadowGAN, its shadow-region results improve by approximately 33% on SRD and 18% on AISTD.
- Results on Hard Shadows: Approximately 33% and 18% improvements over Mask-ShadowGAN are reported for shadow regions on SRD and AISTD, respectively.These comparisons use the state-of-the-art unsupervised method as the baseline.
- Qualitative Results: Qualitative comparisons show more proper shadow removal than competing methods on diverse hard-shadow scenes, including images with shadows on semantic objects.The method [13] alters non-shadow colors and mishandles boundaries, while Gong et al. produces unsatisfactory recovery.
- Results on Soft Shadows: On LRSS, a soft-shadow dataset, DC-ShadowNet is trained unsupervised using 100 images and tested on 34 paired images.The evaluation compares the method with unsupervised and supervised baselines using RMSE and PSNR.
- Results on Soft Shadows: On soft-shadow images, competing methods exhibit binary-mask failures, artifacts, domain-gap effects, or difficulty identifying soft-shadow regions, whereas DC-ShadowNet produces more proper results.The comparison includes Mask-ShadowGAN, DHAN, SP+M-Net, and Guo et al.
- Test-Time Training: Test-time training improves RMSE from 3.48 to 3.36 and PSNR from 31.01 to 31.31 on 34 soft-shadow test images.The authors use the unsupervised losses to adapt the model on the test images.
5. Ablation Study
The ablation study evaluates the shadow-invariant chromaticity, shadow-robust feature, boundary-smoothness, and domain-classifier components on SRD, finding that each contributes to performance.
- Component analysis: The ablation study evaluates four components: Lchroma, Lfeature, Lsmooth, and the domain classifier Φg_sf.These components represent chromaticity, feature, smoothness, and domain-classification contributions.
- Findings: Each component of DC-ShadowNet is important and contributes to better performance on the SRD dataset.The corresponding quantitative results are reported in Table 4.
6. Conclusion
The conclusion presents DC-ShadowNet as an unsupervised method for hard and soft shadow removal, using domain classification and several specialized losses. It also reports that test-time refinement further improves performance and that the method outperforms state-of-the-art shadow removal methods.
- Method: DC-ShadowNet combines domain classification with shadow-free chromaticity, shadow-robust feature, and boundary-smoothness losses.The method learns from unlabeled real shadow images without ground truth.
- Capabilities: The method robustly handles both hard and soft shadow images through its integrated domain classifier and unsupervised losses.The domain classifier enables the network to focus on shadow regions.
- Refinement: Test-time refinement further improves the method’s performance.This refinement is presented as an additional use of the unsupervised network.
- Results: Experimental results confirm effectiveness and outperformance of state-of-the-art shadow removal methods.The conclusion reports this result without specifying a metric in the supplied passage.