Source-linked AI summary
Shadow Detection: A Survey and Comparative Evaluation of Recent Methods
Andres Sanin, Conrad Sanderson, Brian C. Lovell
TL;DR
Moving shadows are frequently included in foreground masks, reducing detection and tracking reliability. The paper surveys recent methods using a four-category feature taxonomy and compares selected techniques quantitatively, qualitatively, and through tracking performance. The large-region texture-based method performs best in the reported comparisons, while methods exhibit distinct trade-offs and limitations.
Problem
Moving shadows are often misclassified as foreground, causing inaccurate detection and reduced tracking performance.
Method
The paper surveys methods from the last decade, organizes them into four feature-based categories, and compares selected methods using detection, discrimination, desaturation, qualitative, and tracking evaluations.
Results
The large-region texture-based method performs considerably better than the others, with high detection and discrimination rates across all test sequences.
Takeaways & Limitations
Tracking performance provides an application-level measure for judging the practical usefulness of shadow detection methods.
Takeaways & Limitations
Geometry-based methods impose scene-specific assumptions, and low shadow-detection performance can make shadow removal worsen tracking.
Abstract
from arXiv · showhide
This paper presents a survey and a comparative evaluation of recent techniques for moving cast shadow detection. We identify shadow removal as a critical step for improving object detection and tracking. The survey covers methods published during the last decade, and places them in a feature-based taxonomy comprised of four categories: chromacity, physical, geometry and textures. A selection of prominent methods across the categories is compared in terms of quantitative performance measures (shadow detection and discrimination rates, colour desaturation) as well as qualitative observations. Furthermore, we propose the use of tracking performance as an unbiased approach for determining the practical usefulness of shadow detection methods. The evaluation indicates that all shadow detection approaches make different contributions and all have individual strength and weaknesses. Out of the selected methods, the geometry-based technique has strict assumptions and is not generalisable to various environments, but it is a straightforward choice when the objects of interest are easy to model and their shadows have different orientation. The chromacity based method is the fastest to implement and run, but it is sensitive to noise and less effective in low saturated scenes. The physical method improves upon the accuracy of the chromacity method by adapting to local shadow models, but fails when the spectral properties of the objects are similar to that of the background. The small-region texture based method is especially robust for pixels whose neighbourhood is textured, but may take longer to implement and is the most computationally expensive. The large-region texture based method produces the most accurate results, but has a significant computational load due to its multiple processing steps.
1. Introduction
Moving cast shadows contaminate foreground detection and can degrade object tracking, motivating shadow removal. This paper updates prior work with a feature-based survey and comparative evaluation of recent methods.
- Shadows are often classified as foreground because they share objects’ movement patterns and similar intensity changes.
- Shadow inclusion can merge people and weaken appearance models, increasing tracking loss.
- The 2003 review found no single robust technique, with generalisation typically requiring application-specific assumptions.
- The paper surveys methods from the last decade and compares prominent techniques through qualitative, quantitative, and applied experiments.
2. Detecting Moving Cast Shadows
The paper organizes shadow-detection methods by the features contributing most to their results, rather than primarily by algorithm type.
- The taxonomy prioritizes feature choice because it has greater impact on detection results than algorithm choice.
- Spectral features are divided into intensity, chromacity, and physical properties, while spatial features are divided into geometry and textures.
- The review uses a feature-based taxonomy with secondary mention of algorithm types.
2.1. Useful features for shadow detection
Useful shadow cues include background comparisons, intensity and chromacity changes, geometry, texture, and temporal consistency, each carrying distinct assumptions and limitations.
- The reviewed methods generally assume that a background reference can be obtained or estimated from video sequences.
- Intensity: Intensity cues identify regions that become darker under blocked illumination, mainly serving as an initial rejection stage.
- Chromacity: Chromacity methods model shadows as darker regions that retain colour, but pixel-level comparisons are susceptible to noise and illumination changes.
- Geometry: Geometry methods avoid dependence on accurate background estimation but impose constraints on object types, shadow orientation, lighting, surfaces, and multiple shadows.
- Textures: Texture methods classify candidate regions using texture correlation, which is robust to colour and illumination changes but computationally slow.
- Temporal features: Temporal consistency filters retain shadow pixels consistent over time, although no reviewed method relies primarily on temporal features.
2.2. Taxonomy of recent shadow detection methods
Recent shadow-detection methods are grouped into chromacity-based, physical, geometry-based, and texture-based categories according to their dominant features.
- Methods are classified by the feature making the dominant contribution to detection results or paper novelty.
- The four categories are chromacity-based, physical, geometry-based, and texture-based methods.
- Highlighted methods in the taxonomy were selected for comparative evaluation.
- The taxonomy table groups methods by year and category while recording secondary distinctions within each category.
3. Methods Selected for Implementation
The paper implements representative chromacity, physical, geometry, and texture-based shadow detectors for comparative evaluation. These methods use distinct colour, illumination, shape, and texture cues, with texture approaches operating on small neighbourhoods or larger candidate regions.
- Method selection: The evaluation implements one or more prominent methods from each reviewed feature-based category for comparison.The categories represented include chromacity, physical, geometry, and textures.
- Chromacity-based method: The HSV chromacity method detects shadows using lower intensity, unchanged hue, and reduced saturation relative to the background.HSV separates chromacity from luminosity, with empirically optimised thresholds applied to the component values.
- Physical-model method: The physical method learns local 3D colour features unsupervised from candidate shadow pixels and uses attenuation to update a Gaussian mixture model.A weak detector identifies reduced luminance and different saturation before model updating penalises pixels with stronger gradients than the background.
- Geometry-based method: The geometry method separates candidate blobs using vertical changes and an orientation parameter before modelling the candidate shadow region with a Gaussian intensity model.The model incorporates elliptical pixel coordinates and intensity, then classifies pixels in the original region as object or shadow according to model agreement.
- Texture-based methods: The small-region texture method correlates neighbourhood textures through Gabor functions, while the large-region method first forms HSV candidate regions and then compares their gradient textures.Large-region candidates are intended to contain shadow or object pixels, but not both; gradient-direction correlation determines whether a candidate is removed as shadow.
- Large-region texture method: The large-region texture method estimates gradient magnitude and direction, compares frame and background directions angularly, and uses the matching fraction c for classification.Pixels below the magnitude threshold are excluded to reduce noise, and regions with c above threshold τc are classified as shadows.
4. Comparative Evaluation
The comparative evaluation combines direct shadow-detection experiments with an applied study of tracking performance. It tests five methods across varied scenes and constructs foreground-based ground truth masks for evaluation.
- Experimental design: Two experiments compare the five selected methods through direct shadow-detection measures and tracking-performance improvement after shadow removal.The first experiment reports quantitative and qualitative results; the second is an applied empirical study.
- Sequences: The test sequences vary object and shadow type and size, with each sequence presenting a different robustness challenge.Campus contains noisy outdoor scenes and extremely long shadows, while Hallway combines textured backgrounds with substantial changes in people’s apparent size.
- Sequences: The evaluation uses a broad set of scenes, including 25 sequences summarised from the CAVIAR dataset.The first six sequences were introduced in earlier shadow-detection work and have been widely used for testing.
- Ground truth: Ground-truth construction begins by applying OpenCV 2.0’s standard Gaussian mixture model foreground extraction to each frame.The resulting foreground mask is intended to contain object and shadow pixels before shadow labels are established.
CAVIAR
The CAVIAR evaluation compares shadow detectors across varied sequences using detection, discrimination, colour-desaturation, qualitative, processing-time, and tracking experiments. Large-region texture methods generally perform best, while each method exhibits distinct environmental and computational trade-offs.
- Evaluation setup: The evaluation covers sequences varying in scene, object, shadow, noise, and manually labelled-shadow characteristics, including 25 recordings from the same CAVIAR scene.Ground truth masks distinguish object pixels from shadow pixels after GMM foreground extraction and manual labelling.
- Quantitative evaluation: The average of shadow detection and discrimination rates is used as the principal quantitative performance measure.Detection rate targets recovered shadow pixels, whereas discrimination rate preserves moving-object pixels as foreground.
- Quantitative evaluation: Large-region texture detection performs considerably better than the other methods across test sequences, with high detection and discrimination rates.The same method also performs better across all qualitative examples, although geometry-based comparisons exclude non-pedestrian sequences because of its design assumptions.
- Computational cost: Chromacity is quickest to implement and run, whereas texture-based methods impose greater computational demands, especially the small-region method and the large-region method’s extra processing steps.The small-region method remains computationally expensive even when kernel number and size are minimised.
- Colour desaturation: Colour-based methods lose performance as colour information is removed, whereas locally adaptive physical and globally adaptive large-region texture methods decline less.The large-region texture method remains best at every desaturation level and performs well without colour by relying on texture.
- Qualitative comparison: Geometry-based detection works for shadows with distinct orientations but fails for matching or multiple orientations, while chromacity and physical methods are vulnerable to pixel noise and similar object-background colours.Small-region texture detection fails in non-textured neighbourhoods; large-region texture detection works reasonably well across the examples.
- Tracking impact: Shadow removal generally improves tracking across algorithms, and the large-region texture method yields the best tracking performance with considerable gains over the small-region method.Geometry-based removal can reduce tracking performance when its orientation assumption is not met, while better shadow-removal performance can produce proportional tracking improvement.
5. Main Findings and Future Directions
The paper updates the shadow-detection literature through a feature-based taxonomy and comparative evaluation, finding distinct method trade-offs and practical limits. It also motivates tracking performance as an applied measure and proposes combining complementary features as a future direction.
- Main Findings: The survey classifies recent shadow-detection methods into chromacity-based, physical, geometry-based, and texture-based groups, selecting prominent representatives for comparison.The texture category includes separate small-region and large-region methods.
- Main Findings: The comparative results show that methods make different contributions, so selection should depend on the specific purpose rather than a single universally best technique.The paper presents the results as a guide for practitioners and researchers choosing among methods.
- Main Findings: The large-region texture-based method produces the most accurate results but incurs a significant computational load from its multiple processing steps.The method performs better across the reported examples, while its processing cost remains substantial.
- Main Findings: Tracking experiments test whether higher pixel-level shadow detection performance translates into improved object tracking, rather than treating detection accuracy as sufficient.The paper applies each method to remove shadows from foreground masks before measuring tracking performance.
- Future Directions: Future work could combine geometry and temporal features with physical or texture-based approaches, or use physical and texture cues to pre-select candidates for geometry-based remodelling.The proposed combinations reflect the observation that current features provide largely independent contributions.