Source-linked AI summary
CoSA: Correlation-Guided Change A ttention with Learnable Residual Gating for Remote Sensing Change Detection
Abdirashid Omar, Jonghyuk Park
TL;DR
The paper addresses crosswalk segmentation under FPV-to-CCTV domain shift when dense target-domain annotation is expensive. It trains a supervised CCTV model and filters pseudo-labels using certainty and foreground geometry. The defensible target-domain result is 88.91% IoU on 40 human-annotated validation images, while the 98.52% second-stage score is pseudo-label agreement rather than human-ground-truth accuracy.
Problem
Fixed traffic-camera imagery is expensive to annotate, and street-level crosswalk models face substantial viewpoint and appearance shifts when applied to elevated CCTV.
Method
The pipeline trains on 201 labeled CCTV images, predicts 5,926 unlabeled frames, and ranks pseudo-labels using certainty and a 5–40% largest-component area prior.
Results
88.91% IoU is obtained on 40 human-annotated CCTV validation images, while 98.52% measures agreement with teacher-generated pseudo-masks in a pseudo-only split.
Takeaways & Limitations
Confidence-and-geometry filtering is practical for generating visually plausible masks at scale, but pseudo-label evaluation must remain separate from human-ground-truth accuracy.
Takeaways & Limitations
The executed second stage contained no manual masks, and nearby-frame random splitting may leak scene and temporal redundancy.
Abstract
from arXiv · showhide
Pixel-level annotation of fixed traffic-camera imagery is expensive, while crosswalk models trained from street-level imagery face a substantial viewpoint and appearance shift when applied to elevated CCTV. We investigate a data-efficient target-domain pipeline using 241 manually annotated CCTV images and 5,926 unlabeled CCTV frames. A source-domain experiment trains a 31.0M-parameter custom U-Net on 3,300 first-person-view (FPV) images and obtains 93.05% IoU on its 330-image FPV test split. This result is a source baseline, not transferred performance: the released CCTV notebook instantiates a 42.0M-parameter DeepLabV3-ResNet50 from torchvision weights, and no compatible mapping from the U-Net checkpoint is implemented. Training on 201 manual CCTV images and selecting on 40 held-out manual masks yields 88.91% IoU. The model then predicts all unlabeled frames; image-level certainty and a largest-component area prior rank the candidates, and the top 1,000 attain mean certainty 0.976 and mean combined score 0.988. A repository audit shows that the reported second-stage 98.52% IoU was measured on a 150-image split containing only teacher-generated pseudo-masks. Because of a directory-layout mismatch, the executed combined-data loader found zero manual samples and split 1,000 pseudo-labeled samples into 850 training and 150 evaluation samples. We therefore report 98.52% as internal pseudo-label agreement rather than human-ground-truth accuracy. The defensible target-domain result is 88.91% IoU on the 40 manual validation images. Batch-one FP32 inference at 512 x 512 requires 12.98 ms, corresponding to 77.03 FPS, on an NVIDIA RTX A6000 48 GB GPU. These findings support the practicality of confidence-and-geometry filtering while also showing why pseudo-label evaluation must remain isolated from the labels used for self-training.
1 Introduction
The paper frames fixed-CCTV crosswalk segmentation as a target-domain problem caused by viewpoint and appearance differences, and tests whether limited labels plus unlabeled frames can support useful segmentation. It emphasizes conservative evaluation, distinguishing human-mask validation from pseudo-mask agreement.
- Fixed overhead cameras differ sharply from pedestrian- and vehicle-level imagery through perspective compression, occlusion, shadows, illumination, and camera-specific backgrounds.
- 5,926 unlabeled CCTV frames are filtered using prediction confidence and foreground-area checks before selecting masks for second-stage training.
- The evaluation separates source-domain FPV performance, held-out manual CCTV validation, and agreement with generated pseudo-masks.
- The study contributes an empirical FPV-to-CCTV viewpoint-gap analysis and a confidence-plus-area ranking rule that selects 1,000 pseudo-labels.
2 Related Work
Prior crosswalk research spans mobile, street-level, overhead, and satellite viewpoints, while fixed CCTV introduces distinct geometric and occlusion challenges. The audited pipeline separates source-domain reference modeling, human-annotated target data, and pseudo-label generation.
- Crosswalk detection has been studied from mobile, vehicle, street-level, overhead, and satellite viewpoints.Prior methods include line grouping, cross-database evaluation, residual segmentation, and satellite-based crossing detection.
- The audited pipeline treats the FPV model as a source-domain reference and separates human-annotated target training and validation from pseudo-label generation.The figure also records the executed second-stage protocol and notes that no verified U-Net-to-DeepLab parameter transfer is implemented.
- Fixed urban CCTV presents distant, oblique, partially framed, and repeatedly occluded crosswalks.
- U-Net, residual learning, DeepLabV3, and DeepLabV3+ provide encoder-decoder, backbone, atrous-context, and decoder designs for dense prediction.
- Pseudo-labeling and semi-supervised segmentation methods use confident predictions, stabilized targets, augmentation, and selective self-training.Domain-adaptation approaches additionally align structured outputs, use cycle consistency, or mix source and target data.
3 Method
The method establishes a source-domain baseline, trains a separately initialized supervised CCTV model, and ranks unlabeled frames using prediction certainty and foreground geometry before second-stage training. A repository audit shows that the executed second stage used only pseudo-labeled samples, while unsynchronized augmentation introduces alignment noise.
- Source-domain baseline: 93.05% IoU is obtained by a custom 31,043,521-parameter U-Net on the FPV test set, serving only as a source-domain baseline.The FPV notebook partitions 3,300 images into training, validation, and test splits; the restored checkpoint achieves 93.05% on 330 test images.
- Supervised CCTV model: The CCTV notebook uses a separately initialized DeepLabV3-ResNet50 and does not establish meaningful transfer from the FPV U-Net checkpoint.The architectures have incompatible namespaces and tensors, and no parameter mapping or matched-key analysis is implemented.
- Supervised CCTV model: 201 manual CCTV images train the target model, while 40 held-out images provide validation after resizing inputs and masks to 512×512.The target model uses DeepLabV3 with a ResNet50 backbone and binary supervision; the training loader processes 200 examples per epoch because it drops one incomplete sample.
- Confidence- and geometry-guided selection: Image-level certainty is computed from prediction distance to the binary decision boundary, and the largest predicted contour supplies a foreground-area ratio.The teacher produces a probability map and binary pseudo-mask for each unlabeled frame before these quantities are used for selection.
- Confidence- and geometry-guided selection: 1,000 candidates satisfy the certainty threshold and 5–40% area rule, with mean certainty approximately 0.976 and mean combined score 0.9882.Candidates with s_j ≥ 0.7 are sorted, and all retained samples satisfy the largest-component area constraint.
- Second-stage training and protocol audit: The executed second-stage loader found zero manual samples and split 1,000 pseudo-labeled samples into 850 training and 150 validation images.The later “1,241 samples” message is hard-coded, and the second-stage score therefore measures agreement with teacher-generated pseudo-masks rather than independent manual accuracy.
4 Experimental Setup
The experiments use unlabeled CCTV frames only for teacher prediction and pseudo-label selection, while evaluating IoU and implementation details on the archived CCTV pipeline.
- Data: 5,926 unlabeled CCTV frames are used only for teacher prediction and pseudo-label selection.
- Pseudo-labels: 0.976 pixel certainty and 1.0 binary geometry score combine to about 0.988 for displayed high-ranked pseudo-labels.The red overlays are automatically generated masks rather than human ground truth.
- Metrics: IoU is reported as the binary-mask intersection-over-union metric.Notebook validation averages batch-level IoUs rather than accumulating a dataset-wide confusion matrix.
- Implementation: 42.0M trainable parameters are reported for the CCTV notebook model.The timing setup uses an NVIDIA RTX A6000 with 48 GB VRAM and FP32 inference.
5 Results
The results separate supervised CCTV performance from source-domain and pseudo-label outcomes. The supervised model reaches 88.91% IoU on human-masked validation data, while the 98.52% second-stage score measures agreement with teacher-generated targets.
- 88.91% IoU is the strongest target-domain result backed by human masks on the fixed 40-image validation set.Because the same split selects the checkpoint, it is validation rather than an independent test performance.
- 98.52% IoU measures continued-model overlap with teacher-generated pseudo-masks in a pseudo-only split.It is a self-training diagnostic, not evidence of a 9.61-point target-domain improvement.
- The supervised CCTV model follows oblique boundaries, crowded scenes, and nighttime imagery in qualitative validation examples.The figure’s upper grid contains inputs, human ground-truth overlays, and prediction overlays from the 40-image validation set.
- The FPV-only source model produces an almost empty response on a difficult night CCTV frame, illustrating the viewpoint and appearance shift.That lower-strip example has no human mask and is not an iteration-two evaluation.
6 Discussion and Limitations
The pipeline’s practical promise is bounded by limited manual validation, a pseudo-only second stage, possible frame leakage, augmentation noise, and incomplete deployment timing. These constraints motivate camera-disjoint evaluation and corrected mixed-data experiments.
- High certainty and a 5–40% largest-component prior efficiently select plausible masks but cannot guarantee correctness.Certainty may reflect calibration or class imbalance, while the area prior can accept a confidently segmented road region of plausible size.
- 40 validation images are too few for a definitive estimate and are also used for checkpoint selection.The split should therefore be interpreted as validation rather than independent testing.
- The executed second stage contains no manual masks, so it is not the intended semi-supervised mixture.Nearby-frame random splits may also leak scene and temporal redundancy, and unsynchronized image/mask flipping adds training noise.
- A corrected study should train on exactly 201 manual plus 1,000 pseudo-labeled samples and evaluate camera-disjoint manual data.It should also synchronize transforms, assess calibration and pseudo-mask quality, report per-camera uncertainty, and compare selection ablations.
- 77.03 FPS demonstrates server-GPU feasibility rather than embedded deployment readiness.The timing excludes decoding, resizing, transfer, and post-processing and should not be extrapolated to edge devices.
7 Conclusion
The study examines crosswalk segmentation under FPV-to-CCTV shift using supervised target-domain training and filtered pseudo-label generation. Its defensible evidence supports strong CCTV validation performance and practical pseudo-label generation, but not a proven human-ground-truth gain from self-training.
- 88.91% IoU is achieved by DeepLabV3-ResNet50 on 40 human-annotated CCTV validation images after training on 201 manual examples.
- 1,000 visually plausible pseudo-labels are selected from 5,926 unlabeled frames using confidence and a 5–40% largest-component area prior.
- 98.52% is pseudo-only internal agreement because the executed loader omitted all manual samples.The study therefore does not establish a human-ground-truth accuracy gain from self-training.
Data and Code Availability
The paper’s code, notebooks, numerical summaries, pseudo-label metadata, and figures are publicly available, but the large datasets and trained checkpoints are excluded.
- The public repository contains the code, notebooks, numerical summaries, pseudo-label metadata, and experiment figures used in the paper.The audited repository commit is 7e4a9d7f59e7de56f867a2e49ab971408e5007a0.
- Large datasets and trained checkpoints are excluded, preventing independent rerun of the final checkpoint on manual CCTV masks from the public artifacts alone.