Transformer-Based Visual Segmentation: A Survey

Background

• Segmentation research has moved from hand-crafted features and classical models to CNN/FCN approaches, while recent state-of-the-art methods use transformers with simpler pipelines and stronger performance than CNN approaches.Segmentation serves applications including robotics, surveillance, image and video editing, social media, and autonomous driving.Source: §1 · p. 1
• Self-attention transformers moved from NLP into vision; ViT processes image-patch sequences for classification, while DETR uses object queries to simplify detection and segmentation pipelines.Earlier vision approaches either added self-attention to CNNs or replaced convolution layers with pure self-attention.Source: §1 · p. 1

Motivation

• The survey fills a gap because earlier vision-transformer surveys focus mainly on general design and selected tasks, while deep-learning segmentation surveys do not focus on transformer-based segmentation or query-based detection.The authors state that, to their knowledge, no prior survey focused on these topics.Source: §1 · p. 1

Contribution

• The review extends DETR into a meta-architecture and groups methods into six categories, including representation learning, decoder interaction, query optimization and association, and conditional query generation.The survey also defines tasks, datasets, and CNN-based approaches before reviewing transformer methods and future directions.Source: §1 · p. 2
• It also covers point-cloud segmentation, foundation-model tuning, domain-aware and efficient segmentation, class-agnostic segmentation and tracking, and medical segmentation, and evaluates influential works on widely used benchmarks.Its scope includes semantic, instance, and panoptic segmentation and variants such as video and point-cloud segmentation; it focuses on representative transformer methods, with a few CNN approaches for reference.Source: §1 · p. 2

2.1 Problem Definition

• Image segmentation distinguishes semantic masks, potentially overlapping instance masks, and non-overlapping panoptic masks, which combine semantic and instance segmentation.It can also be represented as a dense map of pixel identities and class labels.Source: §2.1 · p. 2
• Video segmentation produces mask tubes; VPS includes temporal consistency and tracking, while related tasks include video detection, point-cloud, referring, and video-object segmentation.VSS and VIS are special cases of VPS with different class and identity requirements.Source: §2.1 · pp. 2–3

2.2 Datasets and Metrics

• Common image datasets include COCO, ADE20k, and Cityscapes; video datasets include VSPW and Youtube-VIS, with metrics chosen by task.The primary metrics are mIoU for SS/VSS, mask mAP for IS, 3D mAP for VIS, PQ for PS, and VPQ or STQ for VPS.Source: §2.2 · p. 3

2.3 Segmentation Approaches Before Transformer

• Before transformers, image methods used FCN-based semantic segmentation, top-down or bottom-up instance segmentation, and panoptic pipelines that fused semantic and instance results.Semantic designs included encoder-decoder frameworks,, multiscale pooling, [62], and boundary delineation,,,; instance methods used mask heads, or clustering, [79]. Panoptic methods often used complex, engineered pipelines,.Source: §2.3 · pp. 3–4
• Pre-transformer video methods addressed spatial-temporal fusion, instance relations, and association, while point-cloud methods grouped points using point-based, voxel-based, or semantic-first pipelines.Video systems could be highly engineered, as in MaskPro; indoor point-cloud instance methods used top-down boxes and masks, or bottom-up point grouping,.Source: §2.3 · p. 4

2.4 Transformer Basics

• Vanilla transformers stack attention and feed-forward blocks with residual connections and normalization; self-attention enables global token interactions, while multi-head attention combines parallel heads.The decoder uses a future-token mask and positional embeddings; the feed-forward network adds nonlinearity.Source: §2.4 · pp. 4–5

3.1 Meta-Architecture

• The DETR-like meta-architecture combines a feature extractor, object queries, and a transformer decoder to produce task-specific segmentation predictions.Object queries represent instances and are refined through cross-attention with image or video features; iterative decoder stages produce predictions, with intermediate outputs used for auxiliary training losses.Source: §3 · p. 5, §3.1 · pp. 5–6
• The meta-architecture uses pixel-wise masks for semantic-aware tasks and per-mask predictions for instance-aware tasks, with Hungarian matching assigning queries to ground truth during training.The survey also treats methods such as SegFormer and SETR as a basic form of this architecture, with category queries and no cascaded cross-attention or bipartite matching.Source: §3.1 · p. 6

3.2 Method Categorization

• The survey categorizes methods by shared techniques rather than task settings, summarizing representative approaches, techniques, tasks, and references in Tables 3 and 4.Most approaches follow the meta-architecture described in Sec. 3.1.Source: §3.2 · p. 6, Table 3 · p. 7, Table 4 · p. 9

3.2.1 Strong Representations

• Strong representations are pursued through improved ViTs, hybrid CNN-transformer or MLP designs, and self-supervised learning; SETR was the first to replace a CNN backbone with ViT for semantic segmentation.The survey describes multiscale and cross-scale ViT designs, lightweight or CNN-like alternatives, and self-supervised methods including masked image modeling; it reports that SETR achieved state-of-the-art results on ADE20k without bells and whistles.Source: §3.2.1 · pp. 6–7

3.2.2 Cross-Attention Design in Decoder

• Image decoder designs range from deformable or RoI-based cross-attention to pure-mask approaches that remove box heads and can unify semantic, instance, and panoptic segmentation.Mask2Former uses masked cross-attention so queries attend to object regions indicated by earlier mask outputs; RoI-based methods can have limited mask quality because resolution is constrained within boxes.Source: §3.2.2 · pp. 7–8
• Video methods extend queries across frames to model temporal context and object association, with approaches including direct spatiotemporal masks and unified video segmentation frameworks.VisTR lets one query output masks across a clip without extra tracking, while Video K-Net unifies video tasks via tracking and linking kernels; Tube-Link is reported to perform better than task-specific methods in VSS, VIS, and VPS.Source: §3.2.2 · p. 8

3.2.3 Optimizing Object Query

• Query optimization is organized around positional cues and extra supervision; Anchor DETR and DAB-DETR use anchor priors, while DN-DETR [178] adds noisy boxes and denoising loss.The survey says positional information provides cues for sampling query features to accelerate training, while extra supervision adds losses beyond DETR’s defaults.Source: §3.2.3 · pp. 8–9
• Query optimization adds denoising, contrastive, and one-to-many supervision to improve query learning, with extra supervision heads removed at inference.DN-DETR denoises ground-truth boxes, while MP-Former reconstructs masks and labels; DINO adds contrastive denoising and mixed query selection, and Mask DINO adds mask decoding. DE-DETR, Group DETR [183], H-DETR, and Co-DETR [184] use one-to-many assignments or auxiliary heads; these designs extend to query-based segmentation.Source: §3.2.3 · pp. 9–10

3.2.4 Using Query For Association

• Queries support instance-level association in video and task-level feature fusion, serving as matching representations or links across tasks.Tracking methods use extra tracking queries or object queries directly, with image-dataset pretraining and video-dataset fine-tuning; multitask methods use query fusion for segmentation, depth, fashion attributes, or language generation.Source: §3.2.4 · p. 10

3.2.5 Conditional Query Fusion

• Conditional query fusion incorporates language or image features to generate masks for referring, multi-image, video, few-shot, and matting tasks.Language-conditioned designs include query generation and balance, gated cross-attention, and language-conditioned object queries; image-conditioned designs use support-image features, prior tokens, or video features to refine main-image outputs.Source: §3.2.5 · pp. 10–11

4.1 Segmentation

• Point-cloud methods use transformers for semantic representations and instance queries, including super-point grouping and unified query-based systems for outdoor scenes,,,,.Stratified-Transformer expands context through mixed key sampling and merged cube outputs, while PointBert and Point-MAE use masked point modeling for pretraining.Source: §4.1 · p. 11

4.2 Tuning Foundation Models

• Foundation-model adaptation uses frozen models with learnable adapters or prompts, while open-vocabulary methods align visual regions or pixels with language features to segment novel classes,,,,.ViT-Adapter adds spatial priors and deformable attention; CLIPSeg supports text or image prompts, and OpenSeg aligns caption words with predicted masks.Source: §4.2 · p. 12

4.3 Domain-aware Segmentation

• Domain-aware methods address source-to-target adaptation and multi-dataset learning through training strategies, feature alignment, taxonomy handling, and shared models,, [300],,.HRDA uses multi-resolution training and crops to preserve detail and long-range context; OMG-Seg unifies multi-dataset, image/video, and open-vocabulary segmentation in one model.Source: §4.3 · pp. 12–13

4.4 Label and Model Efficient Segmentation

• Label- and model-efficient methods learn from weak or absent labels and target mobile constraints, using transformer features, pseudo-labels, and efficient attention designs,,,,,.DINO features support unsupervised object discovery, while CutLER generates coarse masks then recalls missing masks; SeaFormer combines squeeze axial attention with detail enhancement.Source: §4.4 · pp. 13–14

4.5 Class Agnostic Segmentation and Tracking

• Fine-grained segmentation methods refine mask boundaries with multiscale features, while video object segmentation uses memory matching and cross-frame attention to track objects,,,,.Transfiner refines coarse masks with a quad-tree transformer; AOT jointly matches and decodes multiple objects, while XMem uses mixed memory for long videos.Source: §4.5 · p. 14

4.6 Medical Image Segmentation

• Medical segmentation adapts U-Net-like designs by combining transformer global context with CNN spatial detail or multilevel skip-connected features [346],,,.TransUNet [346] encodes image patches and combines decoded features with high-resolution CNN maps; UNETR applies a similar approach to 3D medical images.Source: §4.6 · p. 14

5.1 Main Results on Image Segmentation Datasets

• Mask2Former and OneFormer lead semantic segmentation on Cityscapes and ADE20K, SegNext leads on COCO-Stuff and Pascal-Context, and Mask DINO leads COCO instance segmentation with ResNet and Swin-L backbones.The reported semantic metric is mIoU; the instance metric is mAP.Source: Table 5 · p. 14, Table 6 · p. 14, §5.1 · p. 14
• For panoptic segmentation, Mask DINO and K-Max Deeplab lead on COCO, K-Max Deeplab leads on Cityscapes, and OneFormer leads on ADE20K.The reported metric is PQ.Source: §5.1 · p. 14, Table 7 · p. 15

5.3 Main Results for Video Segmentation Datasets

• TubeFormer achieves the best results on VPSW video semantic segmentation; for video instance segmentation, CTVIS is best on YT-VIS-2019 and YT-VIS-2021 with ResNet50 and achieves the best results with Swin-L.GenVIS achieves better results on OVIS with ResNet50.Source: §5.3 · p. 15

5.2 Re-Benchmarking For Image Segmentation

• With a shared neck, Segformer+ [123] leads semantic segmentation on COCO-Stuff and Cityscapes, while Mask2Former leads on ADE-20k; Mask2Former also leads COCO instance segmentation, where K-Net and MaskFormer improve over their original results.The re-benchmark uses the same encoder and neck architecture for a fair comparison.Source: Table 9 · p. 15, Table 10 · p. 15, §5.2 · p. 15
• Mask2Former leads panoptic segmentation on all three re-benchmark datasets; K-Net+ and MaskFormer+ improve over K-Net and MaskFormer by over 3-4%, with a stronger neck closing their gaps to Mask2Former.Source: Table 11 · p. 15, §5.2 · p. 15

5.3 Main Results for Video Segmentation Datasets

• For video panoptic segmentation, SLOT-VPS leads on Cityscapes-VPS, TubeLink on VIP-Seg, and Video K-Net on KITTI-STEP.The reported metrics are VPQ and STQ.Source: Table 13 · p. 15, §5.3 · p. 15

General and Unified Image/Video Segmentation

• Universal models could unify image and video segmentation across datasets and support robust performance in scenarios such as rare-class detection for robotics and autonomous vehicles,, [129],,,,.The proposed direction integrates image and video segmentation tasks in one model.Source: §6 · p. 16

Joint Learning with Multi-Modality

• Transformers could jointly learn segmentation and vision-language tasks, using pixel-level information to enhance text-image retrieval and caption generation,,, [363], [364].Recent work suggests a universal transformer architecture can learn these tasks concurrently.Source: §6 · p. 16

Life-Long Learning for Segmentation

• Lifelong segmentation should continually incorporate novel concepts because real-world settings are open-world and non-stationary, unlike closed-world benchmarks with predefined categories.The paper points to unexpected unseen situations in self-driving and medical diagnosis as examples.Source: §6 · p. 16

Long Video Segmentation in Dynamic Scenes

• Long-video segmentation needs long-term memory, temporal consistency, occlusion reasoning and detection, and domain adaptation to associate instances and handle changing scenes,, [366].Masks must remain consistent as instances move in and out of view, while models cope with heavy occlusion and diverse scene inputs.Source: §6 · p. 16

Generative Segmentation

• Generative segmentation can simplify frameworks by avoiding transformer decoders and object queries, but its complicated training pipeline motivates simpler alternatives [367], [368], [369],.The direction is inspired by stronger generative models, transformer decoders, and high-resolution diffusion-model representations.Source: §6 · p. 16

Segmentation with Visual Reasoning

• Joint segmentation and visual reasoning could mutually benefit segmentation accuracy and relation classification, supporting applications such as object tracking and scene understanding,,,, [375].Segmentation can provide better input for reasoning, while reasoning can improve segmentation.Source: §6 · p. 16

7 CONCLUSION

• The survey reviews more than 120 deep-learning models for segmentation, grouping recent works into six categories based on the segmenter meta-architecture.It also covers essential background and works predating transformers.Source: §7 · p. 16
• The survey reviews five specific subfields and reports representative segmentation methods’ results on widely used datasets.It re-benchmarks several representative works under the same settings to ensure fair comparisons.Source: §7 · p. 16
• The survey concludes by identifying future research directions for transformer-based visual segmentation.Source: §7 · p. 16

APPENDIX

• It expands task-metric descriptions and identifies representative works from the main paper’s “Specific Subfields” section.These topics are covered in Appendix Sections A and B, respectively.Source: APPENDIX · p. 16
• It provides detailed benchmark and re-benchmark results for remaining segmentation tasks, alongside experiment settings and implementation details.These topics are covered in Appendix Sections C and D, respectively.Source: APPENDIX · p. 16
• It presents additional future directions in Appendix Section E.Source: APPENDIX · p. 16

Mean Intersection over Union (mIoU)

• mIoU averages per-class intersection-over-union scores and measures semantic segmentation overlap between predicted and ground-truth masks.A score of 1 indicates perfect overlap; the metric is also used for video and point-cloud semantic segmentation.Source: Task Metrics · pp. 16–17

Mean Average Precision (mAP)

• mAP averages category-level AP scores based on mask IoU; COCO reports it across IoU thresholds 0.5, 0.75, and 0.95.For video instance segmentation, AP scores are averaged across object categories and frames.Source: Task Metrics · pp. 17–18

Panoptic Quality (PQ) and Video Panoptic Quality (VPQ)

• PQ evaluates matched thing and stuff segments using an IoU threshold above 0.5, while VPQ extends PQ across temporal windows for video panoptic segmentation.VPQ averages results across window sizes; a window size of 1 yields PQ, and cross-frame label inconsistency can lower tube IoU.Source: Task Metrics · p. 18

Segmentation and Tracking Quality (STQ)

• STQ combines pixel-level association quality and segmentation quality, with the overall score defined as their geometric mean.Its association quality considers correct and incorrect associations across the full video independently of an IoU threshold; segmentation quality uses the mIoU definition.Source: Task Metrics · p. 18

Video Object Segmentation Metrics

• For video object segmentation, region similarity J uses mask intersection-over-union, while contour accuracy F uses the F-measure between predicted and ground-truth contour points.Source: Task Metrics · p. 18

• Tab. 14 lists representative works to augment the specific-subfields discussion, addressing the main paper’s space limitations.Source: Representative Works in Specific… · p. 18
• The section lists detailed improvements from different techniques for K-Net on COCO-panoptic datasets and uses default settings for re-benchmarking.Source: Representative Works in Specific… · p. 18

Results on Point Cloud Segmentation

• OneFormer3D [377] achieves the best results on three point-cloud segmentation subtasks across ScanNet and S3DIS.The benchmark results are listed in Table 15.Source: More Benchmark Results · p. 18, Table 15 · p. 18

Results on Open Vocabulary Semantic Segmentation

• In open-vocabulary semantic segmentation, FreeSeg achieves the best self-evaluation results, while X-Decoder leads in cross-evaluation.Self-evaluation trains with base classes, treats novel classes as background, and tests on both; cross-evaluation methods differ in datasets and pre-training and co-training supervision.Source: More Benchmark Results · p. 18

Results on Open Vocabulary Instance Segmentation

• CGG achieves the best performance among the reported open-vocabulary instance segmentation methods on COCO.The results are listed in Table 18.Source: More Benchmark Results · p. 18

Results on Open Vocabulary Panoptic Segmentation

• FreeSeg achieves the best performance among the reported open-vocabulary panoptic segmentation methods on COCO.The results are listed in Table 19.Source: More Benchmark Results · p. 18

Results on Weakly-Supervised Semantic Segmentation

• WeakTr [400] achieves the best reported weakly supervised semantic segmentation performance on VOC and COCO.The methods are listed in Table 20.Source: More Benchmark Results · p. 18

Results on Unsupervised Semantic Segmentation

• CLAUSE [401] achieves the best results in both regular and linear-probing unsupervised semantic segmentation settings, using ViT-S and ViT-B backbones.The methods and settings are listed in Table 21.Source: More Benchmark Results · p. 18

Implementation details

• Semantic segmentation re-benchmarks use MMSegmentation with AdamW for 160K iterations on ADE20K and Cityscapes and 80K on COCO-Stuff.Cityscapes uses a 1024 crop; the other datasets use 512 crops.Source: Details of Benchmark Experiment · p. 18
• Instance and panoptic re-benchmarking uses MMDetection, follows Mask2Former settings, and applies a 0.1 learning-rate multiplier to the backbone.The described augmentation and training settings include large-scale jittering from scales 0.1 to 2.0; COCO uses 1024 × 1024 crops, ADE20k uses 640 crops and 160k iterations, and Cityscapes uses 512 × 1024 crops and 90k iterations.Source: Details of Benchmark Experiment · pp. 19–20

Benchmark findings

• On COCO, LSJ improves performance by 0.5%, deformable FPN by 1.0%, and combining them performs better; extending Mask2Former training to 50 epochs gives the best performance.Both techniques also work effectively for Mask2Former semantic segmentation on ADE-20k.Source: Table 22 · p. 19, Details of Benchmark Experiment · p. 20, Table 23 · p. 20
• For video segmentation, deformable FPN works across tasks, while LSJ adds no gains on VSPW and Youtube-VIS-2019 and is less effective than on COCO.The paper attributes LSJ's weaker effect to less diversity and fewer small objects in those video datasets.Source: Details of Benchmark Experiment · p. 20, Table 24 · p. 20, Table 25 · p. 20
• Across settings, deformable FPN strengthens multi-scale feature representation, while LSJ is more effective in complex scenes such as COCO.This summarizes the reported contrast between the consistently effective feature-pyramid change and the dataset-dependent augmentation effect.Source: Details of Benchmark Experiment · p. 20

Mobile Segmentation

• Mobile segmentation research should extend beyond semantic image inputs to efficiently segment and track instances across video clips on mobile devices.Existing transformer-based mobile methods, mainly focus on semantic segmentation with pure image inputs.Source: More Future Directions · p. 20

Using Synthetic Datasets for Joint Training

• Synthetic datasets could reduce annotation costs and support joint training, especially when diffusion models generate high-quality images and masks tailored to few-shot or long-tail tasks.Image-text pairs and open-vocabulary approaches can reduce annotation costs but still require substantial manual data collection; synthetic data may also reduce domain gaps from natural images.Source: More Future Directions · p. 20

Weakly Supervised or Unsupervised Segmentation

• Annotation-efficient methods should replace masks with weaker labels, build on box-supervised mask classification [406], and explore vision transformers as unsupervised mask generators after contrastive pre-training.The section motivates these directions by the substantial manual costs of collecting mask annotations.Source: More Future Directions · p. 20

Domain Generation

• Domain generation should test transformer methods beyond similar driving scenes, establish stronger foundation-model baselines, and adapt across outdoor and indoor domains without training on unseen images.Prior designs include data augmentation, feature distillation, and feature whitening, while only a few works, [409] test transformer segmentation with data augmentation.Source: More Future Directions · pp. 20–21

4D Point Cloud Panoptic Segmentation

• A unified transformer for 4D point-cloud panoptic segmentation could simplify pipelines for segmenting and tracking points while associating tasks.Current methods,,, [410] typically use separate point segmentation, clustering, and association stages.Source: More Future Directions · p. 21