Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation

1 Introduction

• 43.6% mIoU on ADE20K with 4.0 GFLOPs is 0.9% and 2.5% mIoU better than SeaFormer and SegNeXt, respectively, with about 38.0% fewer GFLOPs.Fig. 1 compares ADE20K performance against FLOPs and throughput.Source: Figure 1 · p. 2, §1 · p. 3
• Lightweight models can struggle to represent boundaries and distinguish foreground categories, leading to segmentation inaccuracies and misclassification.The paper frames this as a challenge for achieving advanced performance with limited computing resources.Source: §1 · p. 2
• RCM uses horizontal and vertical pooling for axial global context, then calibrates rectangular attention toward foreground features for spatial reconstruction and pyramid context extraction.Large-kernel strip convolutions adjust the attention shape, while feature fusion adds local detail.Source: §1 · pp. 2–3
• The DPG head embeds class information explicitly to improve foreground-object classification and class discrimination.It derives a dynamic prototype from class-space and pixel-space features, then projects a class embedding back into pixel features.Source: §1 · p. 3

2 Related Work

• CNN and Transformer segmentation methods pursue efficient performance through designs including two-path networks, lightweight decoders, squeeze-enhanced axial attention, and inexpensive convolutions.Examples include BiSeNet, SegFormer, SeaFormer, and SegNeXt.Source: §2 · p. 3
• Context-extraction approaches include coordinate attention and GatherExcite; CGRSeg's RCM adds rectangular key-region modeling and shape self-calibration toward foreground features.The passage also discusses large-kernel convolutional architectures such as ConvNeXt and InceptionNeXt.Source: §2 · pp. 3–4
• Segmentation heads capture multiscale or long-range context, use matrix decomposition, or replace pixel-wise classification with prototypes; the paper characterizes existing heads as costly and poorer-performing.Examples include PSPNet, DeepLab, OCRHead, DAHead, Hamburger Head, and ProtoSeg.Source: §2 · p. 4

3.1 CGRSeg

• CGRSeg has three parts: pyramid context extraction, spatial feature reconstruction, and a lightweight head; RCM supports the first two, while DPG improves foreground classification through explicit class embedding.The architecture is shown in Fig. 2.Source: §3.1 · p. 4, Figure 2 · p. 5
• Pyramid context extraction drops the largest-scale feature, downsamples F2–F4, concatenates them into F5, and processes F5 with stacked RCMs before splitting and upsampling.Horizontal and vertical pooling capture axial context, and MLP further enhances feature representation.Source: §3.1 · pp. 4–5
• Spatial reconstruction fuses encoder spatial features with corresponding decoder features, then uses RCM and pyramid features to produce foreground-focused, multiscale representations.RCM models rectangular critical regions and calibrates their attention toward foreground objects.Source: §3.1 · pp. 5–6

3.2 Rectangular Self-Calibration Module

• RCM combines rectangular self-calibration attention with batch normalization and MLP, followed by a residual connection to refine features and reuse them.The architecture is inspired by the MetaNeXt structure.Source: §3.2 · pp. 6–7
• RCA adds horizontal- and vertical-pooled axis vectors to model a rectangular region, then uses learnable strip convolutions to calibrate its shape toward foreground objects.The two directional convolutions are decoupled, allowing the attention shape to adapt; Fig. 3 illustrates shape change during training.Source: §3.2 · p. 6, Figure 3 · p. 6

3.3 Dynamic Prototype Guided Head

• The DPG head forms an image-dependent prototype from class-space and pixel-space features, compresses it into class embeddings, and weights pixel features to enhance class distinction.Its prototype compression and broadcast product keep computation low, making the head more suitable for efficient segmentation than high-complexity heads such as OCRHead.Source: §3.3 · p. 8

4.1 Datasets

• The experiments use ADE20K, COCO-Stuff, and Pascal Context, with 150, 172, and 59 categories, respectively.ADE20K has more than 20,000 images; COCO-Stuff has 164K images; Pascal Context has 4,996 training and 5,104 test images.Source: §4.1 · pp. 8–9

4.2 Implementation details

• Training uses EfficientFormerV2 and mmsegmentation, with dataset-specific schedules, batch sizes, learning rates, and image resolutions.ADE20K uses 160K iterations at 1024×512; COCO-Stuff uses 80K at 1024×512; Pascal Context uses 80K at 480×480.Source: §4.2 · pp. 9–10

4.3 Comparisons with the State-of-the-art Methods

• 43.6% mIoU on ADE20K with 4.0 GFLOPs; CGRSeg-T exceeds SeaFormer-L by 0.9% mIoU at 2.5 GFLOPs lower cost, and the framework reports gains on COCO-Stuff and Pascal Context.On COCO-Stuff, tiny, base, and large variants exceed the best-compared method by at least 3.5%, 1.3%, and 0.2% mIoU with 61%, 48%, and 43% FLOPs; on Pascal Context, CGRSeg-T reaches 54.1% mIoU at 4.0 GFLOPs.Source: Table 1 · p. 9, §4.3 · pp. 10–11, Table 2 · p. 10

4.4 Ablation Study

• 43.60% mIoU with RCM and DPG improves 2.74% over the 40.86% mIoU base; RCM for PCE and SFR adds 1.23% and 1.13% mIoU, respectively.PCE adds 0.19 GFLOPs and SFR adds 0.17 GFLOPs; adding DPG alone raises performance to 41.34% mIoU.Source: Table 4 · p. 11, §4.4 · p. 12
• CGRSeg-T’s DPG head exceeds HamHead by 1.5% mIoU while reducing FLOPs by 0.05 G; GCHead costs less but has poorer mIoU.The comparison replaces DPG with alternative segmentation heads in CGRSeg-T.Source: §4.4 · p. 12, Table 5 · p. 12
• Replacing RCM with alternatives, CGRSeg improves mIoU over CoordAtt and GatherExcite by 2.1% and 1.9%, respectively.With MSCAN as backbone, CGRSeg increases mIoU by 1.5% and reduces FLOPs by 0.7 G versus SegNeXt.Source: Table 6 · p. 12, §4.4 · p. 13, Table 7 · p. 13
• Broadcast addition outperforms multiplication by 0.53% mIoU in rectangular self-calibration attention.The comparison reports that explicit modeling of key regions improves segmentation performance.Source: §4.4 · p. 13, Table 8 · p. 13
• Shape self-calibration performs best with 1×11 and 11×1 kernels; fusion performs best with a 3×3 kernel.Larger shape-calibration kernels increase FLOPs, so the selected strip kernels are not increased further.Source: Table 9 · p. 13, Table 10 · p. 13, §4.4 · p. 14

• CGRSeg achieves state-of-the-art segmentation performance on ADE20K, COCO-Stuff, and Pascal Context.Source: §5 · p. 14
• The framework combines pyramid context extraction and spatial feature reconstruction, using an RCM to focus on foreground regions and capture axial global context.Source: §5 · p. 14
• A DPG head improves feature representation with only a small amount of computation.Source: §5 · p. 14