Source-linked AI summary
CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning
Eunki Cho, Hyeon Bae Kim, Seong Tae Kim
TL;DR
CDC methods commonly use shared representations or jointly optimized architectures, while detection and captioning have different supervision granularity and task-specific reliability. CoDeRNet selectively routes information between preserved task branches using receiver need and sender reliability cues. On LEVIR-MCI and WHU-CDC, it achieves strong overall performance, particularly improving caption generation while preserving competitive localization.
Problem
CDC methods emphasize shared representations or joint optimization, while task-specific communication under pixel-level detection and sentence-level caption supervision remains less explored.
Source: §1 · p. 1Method
CoDeRNet preserves task-specific branches and uses Confidence-Decomposed Routing to selectively transfer information according to receiver need and sender reliability.
Source: §1 · p. 2Results
CoDeRNet achieves strong CDC performance on LEVIR-MCI and WHU-CDC, with notable caption-generation gains while maintaining competitive change localization.
Source: §1 · p. 2Takeaways & Limitations
Systematic analyses show that the reported benefits arise from explicitly modeling selective cross-task routing, rather than merely joint learning or generic interaction.
Source: §1 · p. 2Evaluation covers two bi-temporal remote-sensing CDC benchmarks; effectiveness in multitemporal scenarios and broader dense-prediction-and-language settings remains unverified.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Change Detection & Captioning (CDC) aims to jointly localize changed regions and describe their semantic content from bi-temporal remote sensing images. While recent approaches have demonstrated the effectiveness of unified modeling, they primarily rely on shared representations or jointly optimized architectures, leaving task-specific communication under heterogeneous supervision less explored. However, detection and captioning differ fundamentally in their supervision granularity, leading to heterogeneous representations with uneven reliability across tasks and spatial locations. In this paper, we reformulate CDC as a selective information routing problem between heterogeneous task branches, where the key challenge is to determine when and where information should be transferred. To this end, we propose CoDeRNet, a unified framework that preserves task-specific representations while enabling bidirectional exchange of complementary information. CoDeRNet introduces a Confidence-Decomposed Routing (CoDeR) mechanism, which decomposes the routing condition into receiver-side need for support and sender-side reliability cues to guide selective transfer. Experiments on LEVIR-MCI and WHU-CDC show that CoDeRNet achieves strong overall CDC performance, particularly improving caption generation while preserving competitive change localization. Extensive analyses further show that the benefits arise not merely from joint learning or generic cross-task communication, but from explicitly modeling selective cross-task information routing. These results highlight the importance of conditional information transfer for unified CDC under heterogeneous supervision.
1 Introduction
Detection and captioning share a change event but differ in supervision granularity, creating a need for selective communication between their task-specific representations. CoDeRNet routes information according to receiver need and sender reliability, with strong CDC performance and notable captioning gains.
1 Introduction
- Detection is pixel-level supervised, whereas captioning is sentence-level supervised, producing task representations with different reliability and support needs.Existing CDC approaches emphasize shared representations or jointly optimized architectures, leaving task-specific communication under heterogeneous supervision less explored.Source: §1 · pp. 1–2
- Detection features can provide spatial evidence but lack sentence-level semantic organization, while caption features capture context but lack explicit spatial precision.The paper therefore argues that transfer should be conditional on the receiving branch's insufficiency and the other branch's reliable support.Source: §1 · p. 2
- CoDeRNet preserves task-specific branches while enabling bidirectional transfer through Confidence-Decomposed Routing, which uses receiver need and sender reliability cues.The framework reformulates CDC as selective information routing between heterogeneous task branches.Source: §1 · p. 2
- CoDeRNet achieves strong CDC performance on LEVIR-MCI and WHU-CDC, with notable caption-generation gains while maintaining competitive change localization.Systematic analyses indicate that benefits arise from explicitly modeling selective cross-task routing, not merely joint learning or generic interaction.Source: §1 · p. 2
2 Related Work
Related work spans separate change-detection and change-captioning methods as well as unified CDC models. The unified task combines pixel-level localization with semantic descriptions, building on dual-branch and shared-multitask architectures.
Remote Sensing Change Detection
- RSCD methods progressed from CNN fully convolutional and Siamese encoder-decoder models [8] [9] [13] to Transformers [1] [6] [12] [30] [34] and Mamba models [7] [46].The cited developments address multi-scale representation, temporal comparison, contextual reasoning, and scalable spatiotemporal modeling.Source: §2 · pp. 2–3
Remote Sensing Image Change Captioning
- RSICC work includes encoder-decoder [32], Transformer [23], pretrained-language-model, attentive or lightweight [4] [36], pretraining, and diffusion approaches [37].These methods strengthen visual change cue localization, temporal-difference modeling, and semantic description generation.Source: §2 · p. 3
Remote Sensing Change Detection & Captioning
- Unified CDC combines pixel-level localization and semantic description; MCINet introduced LEVIR-MCI with a dual-branch model, and CTMTNet developed a shared multitask architecture.These studies established unified CDC as a direction for jointly modeling spatial and semantic change information.Source: §2 · p. 3
3 Method
CoDeRNet builds separate detection and captioning branches from shared change features, then selectively routes complementary information between them before task-specific decoding. Its routing gates transfer using receiver-side need and sender-side reliability, while the branches retain task-specific representations.
3.1 Overview
- Given a bi-temporal image pair, the model extracts multi-scale visual features, constructs shared change representations, routes information between branches, and decodes a mask and description.The framework is optimized end-to-end under a joint training objective.Source: §3.1 · p. 3, §3.2 · p. 3
3.2 Shared Change Representation
- Shared change representations combine temporal features with magnitude cues for change strength and signed cues for directional temporal differences.Source: §3.2 · p. 3
3.3 Branch-Specific Feature Interface
- Branch-specific interfaces provide separate detection and captioning feature spaces for estimating routing cues and applying cross-task transfer.The same interfaces are used across compared variants so comparisons focus on routing rather than branch construction.Source: §3.3 · p. 3
3.4 Confidence-Decomposed Routing
- CoDeR gates residual transfer by receiver need and sender reliability, operating bidirectionally with task-specific estimators and adapters.Detection receives routed information at a relatively higher-resolution scale, while captioning receives it at a lower-resolution scale.Source: §3.4 · pp. 4–5
3.5 Task-Specific Decoders
- Task-specific decoders retain multi-scale hierarchies: an FPN-style segmentation head predicts pixel logits, while captioning uses global, region-level, and local memory tokens.Source: §3.5 · pp. 5–6
3.6 Training Objective
- Joint training combines pixel-wise cross-entropy and Dice for segmentation with token-level caption cross-entropy; detached-magnitude normalization balances the task-loss scales.Source: §3.6 · p. 6
4 Experiments
Experiments evaluate CoDeRNet on LEVIR-MCI and WHU-CDC using task-specific localization and captioning metrics, with controlled ablations and human ratings assessing the routing design and caption quality. CoDeRNet performs strongly across both benchmarks, and the analyses support selective routing over joint learning alone or generic interaction.
4.1 Datasets and Evaluation Metrics
- LEVIR-MCI provides multi-class building and road masks with five captions per image pair, while WHU-CDC provides binary building-change masks and captions spanning multiple change categories.The experiments use mIoU for LEVIR-MCI localization, change-class IoU for WHU-CDC, and standard caption-generation metrics.Source: §4.1 · pp. 6–7
4.2 Implementation Details
- The experiments use a weight-shared ConvNeXt-Base backbone, Adam optimization, and greedy autoregressive decoding, training for 100 epochs on LEVIR-MCI and 150 on WHU-CDC.Both datasets use 256 × 256 inputs; one training run takes approximately 12 hours.Source: §4.2 · p. 7
4.3 Comparison with State-of-the-Art Methods
- CoDeRNet leads overall on both benchmarks, with LEVIR-MCI scores of BLEU-4 66.27, CIDEr-D 141.79, and mIoU 87.79, and WHU-CDC scores of IoU 90.25, CIDEr-D 163.37, and ROUGE-L 83.58.On WHU-CDC it remains competitive on BLEU-based metrics.Source: §4.3 · p. 8
4.4 Ablation Study
- Joint training with CoDeR outperforms single-task training and joint training without routing, while bidirectional routing gives the best overall balance among the compared directions.The ablations indicate that joint learning helps but is insufficient alone, and mutual exchange is more effective than one-way transfer.Source: §4.4 · p. 8, Table 3 · p. 8
- The full decomposed routing mask performs best; removing either the target-dependent or source-dependent term, or replacing them with a single gate, weakens performance.The comparison supports modeling receiver-side need and sender-side reliability rather than relying on gating alone.Source: §4.4 · p. 8, Table 4 · p. 9
- Generic interaction operators improve cross-branch communication, but CoDeR yields stronger captioning while preserving competitive detection accuracy by conditioning transfer on support need and evidence reliability.Operators were compared within the same CoDeRNet pipeline with other components fixed.Source: §4.4 · pp. 8–9
4.5 Human Evaluation of Caption Grounding and Cross-Output Consistency
- On LEVIR-MCI, CoDeRNet improves factual grounding and cross-output consistency over MCINet by 0.138 and 0.181, respectively, with sign-test p-values of 0.045 and 0.0004.The blind human evaluation covers 660 change-containing image pairs and uses a five-point rating scale.Source: §4.5 · p. 9
5 Discussion
The discussion frames conditional routing as a way to coordinate branches with heterogeneous supervision rather than assuming uniform feature sharing. It notes asymmetric benefits and boundaries in the evaluation and suggests semantic correspondence as a direction for further work.
5 Discussion
- Conditional information routing is presented as a more suitable coordination approach than uniform feature sharing for heterogeneous task branches.The discussion connects this challenge to dense prediction and language generation requiring different forms of representation.Source: §5 · p. 9
- Captioning benefits significantly from detection cues, whereas the reverse benefit is more constrained by the semantic gap.The authors suggest spatially grounded detection cues are particularly effective for semantic change description.Source: §5 · pp. 9–10
- Evidence is limited to two bi-temporal remote-sensing CDC benchmarks; multitemporal scenarios and broader dense-prediction-and-language settings remain unverified.The authors also note that routing cues lack explicit region-language correspondence supervision, which may limit faithfulness for subtle or linguistically ambiguous changes.Source: §5 · p. 10
- CoDeRNet may support urban monitoring, disaster assessment, and environmental analysis, but incorrect localization or captions could mislead downstream decisions, so human verification is recommended for high-stakes use.Source: §5 · p. 10
- Region-language alignment or class-aware semantic transfer are proposed as ways to strengthen correspondence between branches and reflect shared semantic structure in routing.The authors suggest this may help translate semantic cues into region-level predictions.Source: §5 · p. 10
6 Conclusion
The conclusion recasts change detection and captioning as conditional information routing under heterogeneous supervision. CoDeRNet uses selective bidirectional exchange, and experiments report improved semantic description while preserving strong localization.
6 Conclusion
- CoDeRNet preserves task-specific representations while selectively routing complementary information bidirectionally between detection and captioning branches.Source: §6 · p. 10
- Confidence-Decomposed Routing conditions transfer on receiver-side support needs and sender-side reliability cues.The conclusion characterizes this as transferring information only when it is expected to be beneficial.Source: §6 · p. 10
- Experiments on LEVIR-MCI and WHU-CDC show improved semantic description quality while preserving strong localization accuracy, with gains attributed to selective routing rather than generic interaction alone.Source: §6 · p. 10
7 Appendix
The appendix probes routing scale and backbone robustness, tests region-token, augmentation, and change-fusion choices, and formulates feature-interaction operators. It also evaluates change-class localization and the statistical evidence for detection gains.
7.1 Scale Selection for CoDeR
- cap →seg performs best at 1/8 and seg →cap at 1/16; the default asymmetric single-scale design is more consistent overall than multi-scale variants.Fully multi-scale routing degrades all caption metrics, while direction-specific multi-scale variants show no clear advantage over the default.Source: §7.1 · p. 14, Table 6 · p. 15, Table 8 · p. 16
7.2 Backbone-Level Generalizability of CoDeR
- Across tested backbone families and ConvNeXt sizes, CoDeR generally improves most caption metrics while maintaining detection performance, although mIoU gains are smaller and not always positive.The reported families include image-only and vision-language pretrained backbones.Source: §7.2 · p. 14, Table 7 · p. 15
7.3 Additional Ablation Studies
- Pooling size 2 performs best on most metrics; geometric augmentation with directional remapping gives the best results for both tasks, and signed plus magnitude change cues outperform temporal features alone.The augmentation result highlights caption-image consistency under direction-sensitive transformations.Source: §7.3 · pp. 14–15, Table 8 · p. 16, Table 9 · p. 16
7.4 Detailed Formulation of Feature Interaction Operators
- In controlled operator comparisons, CoDeR separates target-side sufficiency from source-side reliability, unlike co-attention’s mixed-feature gating and cross-attention’s token-wise relations.Its routing mask is large when the receiving branch needs support and the sending branch provides reliable evidence.Source: §7.4 · pp. 15–18
7.6 Statistical Analysis
- 1.36 percentage points in mIoU and 1.79 in cIoU on LEVIR-MCI, plus 2.02 in cIoU on WHU-CDC, over MCINet; all reported 95% confidence intervals lie above zero.Differences are from full-test-set comparisons using image-level paired bootstrap analysis.Source: Table 11 · p. 18, §7.6 · p. 18
7.5 Change-Class IoU Evaluation
- CoDeRNet achieves the highest mIoU and cIoU among the three settings; cIoU rises from 82.05 to 82.97 with joint training and to 83.07 with CoDeR.The incremental detection gains over joint training are modest: 0.12 percentage points in mIoU and 0.10 in cIoU.Source: Table 11 · p. 18, §7.5 · p. 18