What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation

1. Introduction

• Unsupervised domain adaptation methods have been developed to narrow distribution divergence between labeled source and unlabeled target datasets for semantic segmentation.The introduction situates this work within computer vision applications of domain adaptation.Source: §1 · p. 1
• Diverse lesion appearances across gastroscope and enteroscopy datasets make transferable visual characterizations difficult to identify, while using irrelevant representations can degrade transfer.The paper frames automatically retaining transferable knowledge and neglecting irrelevant knowledge as its central challenge.Source: §1 · pp. 1–2
• The proposed unsupervised semantic transfer model uses complementary TD and TF modules to determine where and how to explore transferable knowledge between labeled source and unlabeled target datasets.TD highlights transferable visual information while preventing irrelevant translation; TF augments transferable semantic features and supports TD.Source: §1 · p. 2
• The paper provides theoretical analysis of narrowing source-target domain discrepancy and evaluates the model on medical endoscopic and several non-medical public datasets.The contributions describe the work as an early attempt to automatically highlight transferable semantic knowledge for endoscopic lesion segmentation.Source: §1 · p. 2

Semantic Segmentation of Lesions

• Medical lesion segmentation uses deep neural networks, while traditional models rely on handcrafted lesion features; deep networks need large-scale pixel labels.Unsupervised learning has been applied to reduce annotation costs.Source: §2 · p. 2
• Unsupervised lesion-segmentation methods may require prior information or distribution hypotheses to generalize to unseen diseases, producing inaccurate and coarse predictions.Performing well on unseen target lesions after training on source disease data remains a challenge [8] [16] [17].Source: §2 · p. 2

Unsupervised Domain Adaptation

• Domain adaptation for semantic segmentation includes adversarial strategies [11] [34] [35] [37] [41], curriculum learning [27] [43], self-training, domain-flow translation [18], bidirectional learning [26], and adaptation losses [25] [38].Other work uses pseudo-pixel labels for hard-to-transfer target samples [16] or explores discriminative semantic knowledge [28] [30] [36].Source: §2 · p. 2

3.1. Overview

• TD and TF form a closed loop: translation informs feature augmentation, which reduces feature-space domain gaps and feeds back to improve translation.The process also mines confident target pseudo-labels to fine-tune the segmentation model.Source: §3.1 · p. 3

3.2. Quantified Transferability Perception

• The discriminator's output probabilities are used to quantify feature transferability with entropy, and residual transferability perception feeds back to TD and the feature augmentor.The paper notes that false transferability perception can harm semantic transfer.Source: §3.2 · p. 3

3.3. Transferable Data Translation (TD)

• TD translates source and target in both directions, using a transferability-weighted information bottleneck to retain transferable visual features and suppress irrelevant representations.The bottleneck weights KL divergence by transferability perception; the paper describes this as neglecting untransferable representations.Source: §3.3 · pp. 3–4

3.4. Transferable Feature Augmentation (TF)

• TF trains the segmenter with labeled translated sources and confident target pseudo-labels, while AF selectively augments features to reduce residual domain mismatch.RA2B gates attention features against the input because attention alone can produce an average weighted feature map even without relevant transferable features.Source: §3.4 · pp. 4–5

3.5. Implementation Details

• The implementation uses CycleGAN for TD and DeepLab-v3 with ResNet-101 for TF; target images are forwarded directly to the segmentation network at test time.Target pseudo-labels use a confidence threshold β = 0.9.Source: §3.5 · pp. 5–6, §3.4 · p. 5

3.6. Theoretical Analysis

• The theoretical analysis argues that bringing both domain distributions toward a standard Gaussian makes discrepancy terms approach zero and tightens the target-error upper bound.The paper concludes that this reduces the source–target domain discrepancy.Source: §3.6 · p. 6

4.1. Datasets and Evaluation

• The medical evaluation trains on 2,969 labeled gastroscope images and 300 unlabeled enteroscopy images, then tests on 390 other enteroscopy images.The non-medical experiments train with GTA or SYNTHIA as source and Cityscapes training images as target, evaluating on Cityscapes validation images.Source: §4.1 · p. 6, §4.3 · p. 8

4.2. Experiments on Medical Endoscopic Dataset

• On the medical dataset, the model mitigates the source–target domain gap by 11.02% versus baseline BL.Competing methods in Table 1 use ResNet-101 as their backbone for a fair comparison.Source: §4.1 · p. 6, §4.2 · p. 7
• On the medical dataset, T_D and T_F mutually promote one another and progressively narrow the domain gap, reaching efficient convergence after three iterations.T_D translates transferable visual information; T_F augments transferable semantic features and promotes T_D in return.Source: §4.2 · p. 7, Figure 7 · p. 7
• Removing any tested component degrades medical-dataset performance by 1.52%–3.23%; the study also sets the number of RA2Bs to 16.Confident pseudo-labels are generated progressively during the iteration process to further bridge dataset shift.Source: §4.2 · p. 7, Figure 5 · p. 7, Table 2 · p. 7
• Medical-dataset performance remains stable across a wide range of tested hyperparameters, supporting the residual transferability-aware bottleneck’s ability to purify representations with high transfer scores.The tested groups are {γ, λs(λt)} and {α, T}; λs and λt share a value.Source: §4.2 · p. 7, Figure 6 · p. 7

4.3. Experiments on Benchmark Datasets

• On GTA→Cityscapes and SYNTHIA→Cityscapes, the model outperforms existing transfer models by 0.5%–11.1%.The reported comparison covers both benchmark transfers; the tables report their performance results.Source: §4.3 · p. 8, Table 3 · p. 8, Table 4 · p. 8
• The benchmark experiments report larger improvements for hard-to-transfer classes with varied appearances, including sidewalk, wall, motorbike, rider, sky, and terrain.The paper also reports iteratively generated pseudo-labels for GTA→Cityscapes, which narrow distribution divergence.Source: §4.3 · p. 8

• Theory analysis and experiments on medical endoscopic and several non-medical benchmark datasets validate the model's effectiveness.Source: §5 · p. 8
• T_D identifies where to translate transferable visual characterizations while preventing untransferable translation.Source: §5 · p. 8
• T_F augments semantic representations with high transferability scores, which in turn promotes T_D's translation performance.Source: §5 · p. 8
• The modules explore domain-invariant knowledge between labeled gastroscope source lesions and unlabeled enteroscopy target diseases.Source: §5 · p. 8