Source-linked AI summary
A Large Contextual Dataset for Classification, Detection and Counting of Cars with Deep Learning
T. Nathan Mundhenk, Goran Konjevod, Wesam A. Sakla, Kofi Boakye
TL;DR
Overhead-car analytics lack large, diverse public datasets, while existing counting approaches may depend on localization or scene-specific assumptions. The paper introduces COWC and a one-look, large-stride neural counter, demonstrating strong detection and efficient automatic counting on held-out scenes. The approach also preserves context and avoids explicit assumptions about car locations, though stride overlap cannot perfectly resolve split-car counting errors.
Problem
Large standardized public datasets for overhead-car analytics are scarce, and existing counting approaches can be tied to localization or the training scene.
Method
The paper constructs COWC with contextual car and confounder patches and trains a neural network to count cars in extended patches scanned across scenes with a large stride.
Results
Detection achieved an F-score of 94.37% over an unconstrained area of approximately 1 km2, while counting was completely automatic and had mean accuracy comparable to interactive density estimation.
Takeaways & Limitations
The counting approach can process new scenes without special interactive configuration or explicit assumptions about car locations.
Takeaways & Limitations
Stride overlap cannot perfectly resolve cars split across patches because the method does not localize cars.
Abstract
from arXiv · showhide
We have created a large diverse set of cars from overhead images, which are useful for training a deep learner to binary classify, detect and count them. The dataset and all related material will be made publically available. The set contains contextual matter to aid in identification of difficult targets. We demonstrate classification and detection on this dataset using a neural network we call ResCeption. This network combines residual learning with Inception-style layers and is used to count cars in one look. This is a new way to count objects rather than by localization or density estimation. It is fairly accurate, fast and easy to implement. Additionally, the counting method is not car or scene specific. It would be easy to train this method to count other kinds of objects and counting over new scenes requires no extra set up or assumptions about object locations.
1 Introduction
The paper introduces COWC, a large, diverse, difficult public dataset designed to support overhead-car classification, detection, and counting. It addresses the scarcity and geographic or sensor homogeneity of standardized public datasets by combining many unique cars, confounders, and contextual information.
- Motivation: Automated overhead-car analytics can support commercial monitoring and government traffic or parking analysis.Satellite imagery may be more cost effective than road sensors when sufficiently cheap and plentiful.
- Dataset gap: Existing public datasets are small and often limited to repeated regions or sensors.OIRDS contains 180 unique cars, while VEDAI contains 2,950 and draws all images from Utah’s AGRC collection.
- Release: All datasets and related scripts are intended to be publicly available.The paper provides a URL for the datasets, annotations, networks, and scripts.
- Dataset contribution: COWC contains 32,716 unique cars from six geographically distinct image sets produced by different imagers.The sources include locations in Canada, New Zealand, Germany, and the United States.
- Dataset contribution: The dataset includes 58,247 usable negative targets selected from objects that can resemble cars, including boats, trailers, bushes, and A/C units.Context around targets helps the learner use surroundings to reject confounders or recognize cars in unusual settings.
2 Related Work
The paper evaluates classification, detection, and a one-look counting approach on difficult overhead-car imagery. Its counting method predicts object counts in extended patches and scans scenes with a large stride without explicit localization or scene-specific assumptions.
- Classification: Classification is treated as a deliberately difficult two-class task, with context helping mainly on special difficult cases.The paper contrasts this goal with near-100% classification reported on another dataset.
- Detection: Detection methods commonly combine classifiers with spatial testing, heat maps, or contextual cues such as roads and motion.The paper positions its detector among approaches using deep networks and contextual assistance.
- Counting background: Counting methods generally use either density estimation or instance detection, with density estimators often constrained to their training scene.Detection counters localize individual cars before counting them.
- Counting method: The proposed counter learns the number of cars in an extended patch as a class, then counts groups across a scene using a large stride.It receives no information about object locations or expected features.
- Counting method: The method is presented as applicable beyond roads or a particular scene because it counts by appearance without assuming car locations.The paper states that cars can be counted even when they are on lawns, rather than only on roads.
3 Data Set Details
The dataset standardizes overhead imagery while preserving variation in color, artifacts, rotation, and difficult context. Annotations emphasize confounders and task-specific ambiguity, and geographically separated held-out scenes test generalization.
- Image standardization: Imagery is standardized to 15 cm per pixel, making cars range from 24 to 48 pixels.The sources include both grayscale and RGB imagery, with remaining variation in quality, appearance, rotation, and artifacts.
- Annotation: Cars are annotated with single-pixel center points, while large trucks are omitted and vans or pickups remain included as cars.Boats, trailers, and construction vehicles are always labeled as negatives.
- Task-specific labeling: Classification excludes highly ambiguous items, whereas counting uses a best guess for distorted items that still appear to be cars.These inclusion decisions are explicitly described as judgment calls.
- Patch extraction: Training and testing patches are assigned to separate 1024x1024 grid regions so the two patch types cannot overlap.Overlaps are permitted within training regions and within testing regions, including crowded parking lots.
- Held-out evaluation: Final evaluation uses geographically separated 2048x2048 held-out scenes containing varied settings such as forests, water parks, junkyards, and utility plants.The held-out locations are outside the Salt Lake City metropolitan area used for patch data.
4 Classification and Detection
The paper evaluates contextual car-patch classification and scene detection using Inception-based models, including the residual-projection ResCeption architecture. It examines context effects and uses heat-map scanning with thresholded, fixed-size detections.
- Classification: 308,988 training patches and 79,447 testing patches were created from 256x256 contextual patches, labeling cars within a central 48x48 region.Cars outside the central region were treated as context, so negative patches could still contain cars elsewhere.
- Classification: ResCeption replaces each Inception layer’s 1x1 convolutions with a residual projection shortcut.The ResCeption GoogLeNet variant has about 5% more operations but runs about 5% faster than Inception.
- Classification: Inception and ResCeption classify test patches noticeably better than AlexNet, while all three models perform well.Table 1 also includes a Non-Utah model trained on five datasets and tested on Utah to assess generalization.
- Classification: Classification accuracy increases as more context is included, with a small amount of context appearing optimal.The context experiment masks patch margins in 32-pixel increments and evaluates fine-tuned GoogLeNet models.
- Detection: For detection, 10 labeled 2048x2048 scenes were scanned at stride eight using 224x224 patches formed from 192x192 regions plus a 32-pixel margin.Softmax outputs were raised to the power of 16 to produce a wider heat-map gradient.
- Detection: Detection uses a heat map, fixed 48-pixel bounding boxes, non-maximal suppression, and a 0.75 threshold; a detection requires at least half the car inside the box.The threshold and overlap settings were established on a validation scene rather than the held-out scenes.
5 Counting
The paper frames counting as a two-stage task: a network learns car counts for large patches, then a stride aggregates counts across a scene. It evaluates patch and held-out-scene counting, including overlap, offset averaging, comparisons, and efficiency.
- Counting approach: Counting is split into learning one-look counts over large patches and scanning scenes one patch at a time with a large stride.The network predicts the number of cars in an extended patch without location or feature annotations, then aggregates counts across stride locations.
- Patch-based counting: Patch labels are car counts, with boundary cars included only when their point annotations are at least 8 pixels inside the visible patch.This ensures counted cars are mostly inside the patch, while ambiguous incidental objects must still be assigned a car or non-car label.
- Network design: 64 softmax outputs cover the full observed counting interval, which reached 61 cars per training patch.The output space was rounded up from the observed maximum to accommodate larger patches during later fine-tuning.
- Network design: A 22-layer double-tall ResCeption network repeats each ResCeption layer twice, combining Inception-like stacking with residual learning.The authors report that the tall network could be trained with one error output, suggesting the residual component supported the deeper architecture.
- Counting scenes: Stride overlap reduces split or double-counted cars, but without localization the method cannot perfectly resolve boundary cases and may omit some split cars.A validation scene with 628 cars produced a selected stride of 167, which was then used for other counting scenes.
- Counting scenes: Four slightly offset strides improved 19 of 20 scene error statistics, while the held-out evaluation used 20 unique Utah scenes containing 10–881 cars.The held-out locations were outside the Salt Lake City metropolitan training region and included difficult settings such as forests, a water park, and a junkyard.
- Comparison and scope: Mean accuracy was comparable to interactive density estimation, while this method was automatic and made no explicit assumption about car location.The authors note that the learned context could nevertheless bias counting against cars on water or building tops.
- Counting efficiency: 1 km2 per second is the reported counting rate for the AlexNet version, with 8.46% mean absolute error for trend-oriented analysis.The paper estimates that processing 4 trillion pixels would take 23 hours on one GPU under its stated assumptions.
6 Conclusion
The paper presents a large, difficult overhead-car dataset for classification, detection, and counting. It reports strong classification, apparently improved detection, and efficient counting with results similar to scene-constrained or fine-tuned methods.
- 6 Conclusion: The dataset supports classification, detection, and counting of cars in large, difficult overhead imagery.The conclusion describes the counting method as efficient and reports similar results to methods constrained to scenes or requiring fine-tuning.
- 6 Conclusion: Classification results were described as excellent, while detection appeared better than methods that constrain car locations.The conclusion also characterizes counting as efficient and comparable to scene-constrained or scene-fine-tuned approaches.