PaperlayerView exact revision on arXiv

Source-linked AI summary

Transformer-Based Visual Segmentation: A Survey

Xiangtai Li, Henghui Ding, Haobo Yuan, Wenwei Zhang, Jiangmiao Pang, Guangliang Cheng, Kai Chen, Ziwei Liu, Chen Change Loy

arXiv:2304.09854v4cs.CV

TL;DR

Visual segmentation requires methods that partition images, videos, and point clouds across many applications and task settings. This survey synthesizes transformer-based approaches through a unified meta-architecture, reviews related subfields, and re-evaluates representative methods. Its benchmark compilation identifies different leading methods across image and video segmentation datasets, while highlighting open directions such as unified models and visual reasoning.

  • Problem

    Visual segmentation spans diverse tasks, but prior surveys did not specifically focus on vision transformers for visual segmentation or query-based object detection.

  • Method

    The survey reviews transformer-based segmentation through a DETR-extended meta-architecture, covering method categories, related subfields, and representative benchmark evaluations.

  • Results

    The survey reports task- and dataset-specific leaders, including Mask2Former and OneFormer on selected semantic datasets, Mask DINO on COCO instance segmentation, and several leading video methods.

  • Takeaways & Limitations

    Transformer-based segmentation is developing toward unified architectures spanning diverse image and video tasks and applications.

  • Takeaways & Limitations

    Generative segmentation models can simplify the framework by avoiding transformer decoders and object queries but typically introduce complicated training pipelines.

Abstract

from arXiv · show

Visual segmentation seeks to partition images, video frames, or point clouds into multiple segments or groups. This technique has numerous real-world applications, such as autonomous driving, image editing, robot sensing, and medical analysis. Over the past decade, deep learning-based methods have made remarkable strides in this area. Recently, transformers, a type of neural network based on self-attention originally designed for natural language processing, have considerably surpassed previous convolutional or recurrent approaches in various vision processing tasks. Specifically, vision transformers offer robust, unified, and even simpler solutions for various segmentation tasks. This survey provides a thorough overview of transformer-based visual segmentation, summarizing recent advancements. We first review the background, encompassing problem definitions, datasets, and prior convolutional methods. Next, we summarize a meta-architecture that unifies all recent transformer-based approaches. Based on this meta-architecture, we examine various method designs, including modifications to the meta-architecture and associated applications. We also present several closely related settings, including 3D point cloud segmentation, foundation model tuning, domain-aware segmentation, efficient segmentation, and medical segmentation. Additionally, we compile and re-evaluate the reviewed methods on several well-established datasets. Finally, we identify open challenges in this field and propose directions for future research. The project page can be found at https://github.com/lxtGH/Awesome-Segmentation-With-Transformer. We will also continually monitor developments in this rapidly evolving field.

1 INTRODUCTION

Visual segmentation has advanced from handcrafted and CNN-based methods toward transformer architectures, which offer simpler pipelines and stronger performance across diverse segmentation tasks. This survey organizes the field around a unified meta-architecture and reviews representative methods, applications, and future directions.

  • Visual segmentation groups image or video pixels into semantic regions for applications including robotics, surveillance, editing, and autonomous driving.
  • CNNs and FCNs achieved major gains and became foundational components of segmentation research because of their stronger generalization than traditional approaches.
  • Transformers originated in NLP with self-attention and parallel token processing before being applied to computer vision.
  • Transformer-based segmentation methods generally combine simpler pipelines with stronger performance than CNN-based approaches.
  • The survey organizes recent methods using a DETR-extended meta-architecture and categorizes designs by changes to its components.
  • Its coverage spans mainstream segmentation tasks, related subfields, representative methods, and future research directions.

2 BACKGROUND

The background defines image, video, and point-cloud segmentation tasks, reviews datasets, metrics, pre-transformer approaches, and transformer fundamentals. It also introduces a DETR-like meta-architecture for organizing transformer-based segmentation methods.

  • 2.1 Problem Definition: Image segmentation outputs masks with class labels and includes semantic, instance, and panoptic segmentation according to mask and class scope.Semantic segmentation assigns one non-overlapping mask per class; instance segmentation allows multiple, potentially overlapping masks for foreground objects; panoptic segmentation combines both settings.
  • 2.1 Problem Definition: Video segmentation predicts temporally extended mask tubes, while video panoptic segmentation additionally requires temporally consistent tracking identities.Video semantic and instance segmentation arise from video panoptic segmentation under restricted class and identity settings.
  • 2.2 Datasets and Metrics: Common benchmarks include COCO, ADE20k, Cityscapes, VSPW, and Youtube-VIS, with task-specific metrics such as mIoU, mask mAP, 3D mAP, PQ, VPQ, and STQ.The metrics cover semantic, instance, panoptic, and video panoptic segmentation, with VPQ extending PQ temporally and STQ separating segmentation and tracking.
  • 2.3 Segmentation Approaches Before Transformer: Before transformers, semantic segmentation was primarily dense pixel classification within FCN-based frameworks, while instance segmentation used top-down detection extensions or bottom-up clustering.The surveyed pre-transformer literature also includes point-cloud segmentation methods for semantic and instance categories.
  • 2.4 Transformer Basics: Transformer basics center on encoder-decoder blocks with multi-head self-attention and feed-forward networks, where attention provides global token interactions.Queries, keys, and values are linearly projected from token embeddings before self-attention is computed.
  • 2.4 Transformer Basics: The survey organizes transformer-based segmentation around a DETR-like meta-architecture containing a feature extractor, object queries, and a transformer decoder.Decoder cross-attention and feed-forward operations iteratively refine object queries; segmentation uses output embeddings with feature maps to produce binary mask logits.

3 METHODS: A SURVEY

The survey organizes transformer-based segmentation around a DETR-like meta-architecture and reviews improvements to its feature extractor, queries, decoder, and prediction representation. It also categorizes methods by shared technical techniques and covers image and video segmentation extensions.

  • 3.1 Meta-Architecture: The unified meta-architecture contains a feature extractor, object query, and transformer decoder, with methods grouped by modifications to these components.The survey also discusses applications built on this architecture.
  • 3.1 Meta-Architecture: ViT extracts dense segmentation features by embedding image patches with positional information, transformer encoding, and resizing the output into a dense map.For an image I ∈ R^H×W×3, patch embeddings produce token features that are reshaped into F ∈ R^H×W×C.
  • 3.1 Meta-Architecture: During decoding, cross-attention and feed-forward operations iteratively refine object queries; segmentation uses the query embedding’s dot product with feature F to produce mask logits.Intermediate predictions support auxiliary training losses, while the final decoder stage supplies inference outputs.
  • 3 METHODS: A SURVEY: The survey categorizes transformer segmentation methods by five shared technical aspects rather than by task setting and summarizes representative methods in comparison tables.The stated goal is to extract essential techniques common across the literature.
  • 3.2.1 Strong Representations: Strong feature representations improve segmentation, motivating better ViT designs, CNN-transformer or MLP hybrids, and self-supervised learning approaches.Examples include multiscale representations, pooling strategies, and masked image modeling.
  • 3.2.2 Cross-Attention Design in Decoder: Decoder studies improve cross-attention for image segmentation or extend query-based detection and segmentation into video through spatial-temporal attention and temporal association.VisTR directly predicts spatial-temporal masks from object queries, while other methods link queries and features across time.
  • 3.2.3 Optimizing Object Query: MaskFormer removes the box head and converts object queries into mask queries, while K-Net unifies semantic, instance, and panoptic image segmentation with kernels.Pure mask-based approaches avoid extra box heads and generate masks from high-resolution features.

4 SPECIFIC SUBFIELDS

The survey examines transformer segmentation beyond standard image tasks, covering point clouds, foundation-model adaptation, domain adaptation, weak supervision, efficient models, fine-grained boundaries, and medical imaging. Across these settings, methods use attention, pretrained knowledge, multiscale features, or CNN-transformer combinations to address task-specific constraints.

  • Point Cloud Segmentation: Point-cloud methods transfer transformer representation learning to 3D data, while Stratified-Transformer uses cube partitioning, mixed key sampling, and merged outputs to enlarge receptive fields.The passage identifies limited long-range context and cross-scale interaction as challenges for some point-cloud approaches.
  • Foundation Model Tuning: Foundation-model segmentation research develops vision adapters for downstream tasks and open-vocabulary methods for segmenting unknown objects through vision-language knowledge transfer.Open-vocabulary approaches use pretrained vision-language models and zero-shot transfer on unseen segmentation datasets.
  • Domain Adaption: Unsupervised domain adaptation transfers models from synthetic source domains to real target domains without target labels, using transformer backbones, masked-image consistency, or domain-query alignment.SFA combines domain-query-based and token-wise feature alignment for detection transformers under domain adaptation.
  • Weakly Supervised Segmentation: Weakly supervised segmentation uses image labels or boxes instead of full masks, with transformer methods generating localization maps from class-token and patch-token interactions.Multiple class tokens can model interactions with patch tokens to generate segmentation labels.
  • Efficient Segmentation: Efficient transformer segmentation includes token pyramids, squeeze-enhanced axial attention, and detail-enhancement modules for scale-aware features, global context, and preserved spatial detail.These designs target mobile semantic segmentation and unified efficient settings.
  • Fine-grained Object Segmentation: Fine-grained segmentation methods refine coarse masks with quad-tree transformers, multiscale point features, or patch-based DCT mask representations to recover object-boundary details.Video-Transfiner extends boundary refinement to spatial-temporal masks.
  • Medical Segmentation: Medical segmentation methods combine transformers with U-Net architectures, parallel CNN branches, or symmetric transformer decoders to capture global dependencies and recover precise localization.UNETR applies a U-Net-like transformer architecture to 3D medical images.

5 BENCHMARK RESULTS

The survey benchmarks transformer-based segmentation methods across image and video tasks, including both published results and controlled re-benchmarks. Performance leaders vary by task, dataset, backbone, and evaluation setting.

  • 5.1 Main Results on Image Segmentation Datasets: Published benchmarks identify different leaders across semantic, instance, and panoptic image segmentation datasets.Mask2Former and OneFormer lead on Cityscapes and ADE20K, SegNext leads on COCO-Stuff and Pascal-Context, Mask DINO leads COCO instance segmentation, and Mask DINO or K-Max Deeplab lead panoptic segmentation.
  • 5.1 Main Results on Image Segmentation Datasets: Mask DINO achieves the best COCO instance-segmentation results with both ResNet and Swin-L backbones.The benchmark reports this result using the mAP metric on COCO validation data.
  • 5.1 Main Results on Image Segmentation Datasets: Mask DINO and K-Max Deeplab lead COCO panoptic segmentation, while K-Max Deeplab and OneFormer lead Cityscapes and ADE20K, respectively.These benchmark results use the PQ metric for panoptic segmentation.
  • 5.2 Re-Benchmarking For Image Segmentation: Controlled re-benchmarking standardizes encoder, neck, and data-augmentation settings to examine transformer-decoder effects across segmentation tasks.The study covers semantic and panoptic segmentation on ADE20K, COCO, Cityscapes, and COCO-Stuff, with additional instance results on COCO.
  • 5.2 Re-Benchmarking For Image Segmentation: Under matched settings, Mask2Former leads ADE20K semantic segmentation, COCO instance segmentation, and all three reported panoptic datasets.Segformer+ leads COCO-Stuff and Cityscapes semantic segmentation, while K-Net+ and MaskFormer+ gain over 3-4% from a stronger neck.
  • 5.3 Main Results for Video Segmentation Datasets: TubeFormer leads video semantic segmentation on VPSW, while different methods lead video instance and panoptic benchmarks across datasets and backbones.CTVIS leads YT-VIS and Swin-L video instance results, GenVIS leads OVIS with ResNet50, and SLOT-VPS, TubeLink, and Video K-Net lead separate video panoptic datasets.

6 FUTURE DIRECTIONS

The survey identifies future directions that extend transformer segmentation toward unified image-video systems, multimodal learning, simpler generative designs, and joint visual reasoning. These directions target broader capabilities while retaining attention to training complexity and task interaction.

  • General and Unified Image/Video Segmentation: Unified image-video segmentation models could support robust segmentation across multiple scenarios and datasets.The survey connects this direction to rare-class detection and applications such as robot navigation and autonomous vehicles.
  • Joint Learning with Multi-Modality: Transformers could jointly learn segmentation with vision-language tasks because segmentation supplies pixel-level information.The cited applications include text-image retrieval and caption generation.
  • Generative Segmentation: Generative diffusion designs avoid transformer decoders and object queries, but typically require more complicated training pipelines.The survey calls for simpler training pipelines for further research.
  • Segmentation with Visual Reasoning: Joint segmentation and visual reasoning could provide mutual benefits for segmentation accuracy and relation classification.Segmentation can improve inputs to visual reasoning, while reasoning can be incorporated into segmentation.

7 CONCLUSION

The survey synthesizes transformer-based visual segmentation through a broad review, structured categorization, benchmark evaluation, controlled re-benchmarking, and proposed future directions. It presents the field as spanning diverse tasks and specialized subfields.

  • 7 CONCLUSION: The survey reviews more than 120 deep-learning models for diverse visual segmentation tasks.It groups representative works into six categories based on a segmenter meta-architecture.
  • 7 CONCLUSION: The paper covers five specific subfields and reports representative-method results on widely used datasets.It also includes background on pre-transformer work and related areas.
  • 7 CONCLUSION: Controlled re-benchmarking evaluates representative methods under the same settings to support fair comparison.The conclusion identifies future research directions for transformer-based visual segmentation.

APPENDIX

The appendix extends the main paper with supplementary metric explanations, subfield descriptions, benchmark results, experimental details, and additional future directions.

  • APPENDIX: The appendix supplies more future directions as a supplementary adjunct to the main paper.The overview describes the appendix as an extension of the main paper.
  • APPENDIX: The appendix provides more descriptions of task metrics and representative works from the specific-subfields section.These materials are organized in Sections A and B.
  • APPENDIX: Additional benchmark and re-benchmark results cover segmentation tasks not detailed in the main paper.The appendix also includes experiment settings and implementation details for re-benchmarking.

Task Metrics

The survey describes segmentation metrics based on overlap between predicted and ground-truth masks, including mIoU, mAP, PQ, and VPQ. These metrics cover semantic, instance, panoptic, and video panoptic settings.

  • Metrics: mIoU averages per-class intersection-over-union scores to evaluate semantic segmentation masks.Higher mIoU indicates better overlap between predicted and ground-truth masks across classes.
  • Metrics: mAP averages category-level average precision scores computed from predicted and ground-truth mask overlap.The underlying IoU measures the match between the two masks.
  • Metrics: PQ evaluates panoptic segmentation at segment level by matching predicted and ground-truth thing and stuff masks.Matching uses an IoU threshold greater than 0.5, and the final score averages results across classes.
  • Metrics: VPQ extends PQ to video by measuring spatial-temporal mask IoU across temporal window sizes.With a temporal window size of 1, VPQ equals PQ; tracked thing objects must retain matching semantic classes.

Representative Works in Specific Subfields

The survey supplements its main review with representative works across specific segmentation subfields and documents technical improvements used in re-benchmarking.

  • Specific subfields: Representative works are listed to augment the specific-subfields section because the main paper has limited space.The detailed list appears in Table 14.
  • Scope: The specific-subfields material combines representative-work coverage with implementation-oriented re-benchmarking details.These components extend the survey beyond its core method review.
  • Re-benchmarking: The survey also lists detailed improvements for different techniques used to improve K-Net on COCO-panoptic datasets.The re-benchmarking adopts the default settings.

More Benchmark Results

The survey reports benchmark comparisons for transformer-based segmentation beyond standard image tasks, including point-cloud and open-vocabulary settings. The reported leaders differ across subtasks and evaluation protocols.

  • Point cloud segmentation: OneFormer3D achieves the best results on all three point-cloud segmentation subtasks evaluated on ScanNet and S3DIS.The comparison is reported in Table 15.
  • Open-vocabulary segmentation: FreeSeg achieves the best results in open-vocabulary semantic segmentation under self-evaluation.This setting trains on base classes while treating novel classes as background, then tests on both base and novel classes.
  • Open-vocabulary segmentation: X-Decoder leads the reported cross-evaluation comparison, while CGG and FreeSeg lead open-vocabulary instance and panoptic segmentation, respectively.The instance and panoptic comparisons use COCO datasets; cross-evaluation methods differ in pre-training data and supervision.

Details of Benchmark Experiment

The survey re-benchmarks segmentation methods under standardized codebases and training settings, then studies augmentation and feature-pyramid improvements across image and video tasks. Deformable FPN is broadly effective, whereas LSJ is more dependent on scene complexity.

  • Experiment setup: Semantic-segmentation re-benchmarking uses MMSegmentation with AdamW for 160K iterations on ADE20K and Cityscapes and 80K on COCO-Stuff.Cityscapes uses 1024 crops; the other datasets use 512 crops.
  • Experiment setup: Instance and panoptic re-benchmarking uses MMDetection, follows Mask2Former settings, and applies a 0.1 learning-rate multiplier to the backbone.COCO uses large-scale jittering with scales from 0.1 to 2.0 and 1024 × 1024 crops; ADE20K uses 640 crops.
  • Image segmentation ablations: On COCO, LSJ improves performance by 0.5% and deformable FPN by 1.0%, while combining both and extending training to 50 epochs gives the best result.The survey concludes that stronger augmentation and feature pyramids are important for COCO.
  • Video segmentation ablations: In video segmentation, LSJ adds no extra gains, whereas deformable FPN works across nearly all evaluated video tasks.The survey attributes LSJ’s weaker effect to lower diversity and fewer small objects in VSPW and YouTube-VIS-2019 than in COCO.
  • Interpretation: Across settings, deformable FPN improves multi-scale feature representation, while LSJ is more effective in complex scenes such as COCO.This conclusion summarizes the image and video ablation patterns.

More Future Directions

The survey identifies future directions spanning mobile video segmentation, synthetic-data training, domain generalization, and unified 4D point-cloud panoptic segmentation.

  • Mobile Segmentation: Mobile segmentation research should extend beyond semantic image inputs toward efficient instance-level video segmentation and tracking on devices.The survey highlights short-video applications as an underexplored setting requiring further research.
  • Using Synthetic Datasets for Joint Training: Synthetic datasets generated by diffusion models may help reduce the substantial manual effort required for pixel-wise segmentation annotations.Image-text pairs and open-vocabulary methods reduce annotation costs but still require significant manual data collection.
  • Domain Generation: Future methods should evaluate generation ability across varied domains because models tuned to similar driving-scene domains may not generalize to real applications.The survey specifically contrasts outdoor and indoor scene inputs for robotic adaptation.
  • 4D Point Cloud Panoptic Segmentation: A unified transformer solution could simplify 4D point-cloud panoptic segmentation by associating point segmentation, clustering, and tracking within one framework.Current methods typically use separate point segmentation, point clustering, and point association pipelines.
Loading 2304.09854v4…