PaperlayerView exact revision on arXiv

Source-linked AI summary

Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation

Sanghun Jung, Jungsoo Lee, Daehoon Gwak, Sungha Choi, Jaegul Choo

arXiv:2107.11264v4cs.CV

TL;DR

Urban-scene segmentation can assign unexpected road objects to known classes, a safety-relevant gap, and class-specific max-logit ranges complicate score-based detection. The paper standardizes max logits by predicted class and processes local regions, achieving state-of-the-art performance on Fishyscapes Lost & Found.

  • Problem

    Urban-scene segmentation assigns unexpected objects to predefined classes, while class-specific max-logit ranges limit score-based detection; identifying such objects matters for safety-critical applications.

    Source: §1 · pp. 1–2

  • Method

    The approach standardizes max logits within each predicted class and uses local-region processing, including iterative boundary suppression and dilated smoothing.

    Source: §1 · p. 2, §6 · p. 8

  • Results

    The approach achieves state-of-the-art performance on the Fishyscapes Lost & Found leaderboard with a large margin.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The method applies to existing segmentation models without additional training or external datasets.

    Source: §1 · p. 2

Abstract

from arXiv · show

Identifying unexpected objects on roads in semantic segmentation (e.g., identifying dogs on roads) is crucial in safety-critical applications. Existing approaches use images of unexpected objects from external datasets or require additional training (e.g., retraining segmentation networks or training an extra network), which necessitate a non-trivial amount of labor intensity or lengthy inference time. One possible alternative is to use prediction scores of a pre-trained network such as the max logits (i.e., maximum values among classes before the final softmax layer) for detecting such objects. However, the distribution of max logits of each predicted class is significantly different from each other, which degrades the performance of identifying unexpected objects in urban-scene segmentation. To address this issue, we propose a simple yet effective approach that standardizes the max logits in order to align the different distributions and reflect the relative meanings of max logits within each predicted class. Moreover, we consider the local regions from two different perspectives based on the intuition that neighboring pixels share similar semantic information. In contrast to previous approaches, our method does not utilize any external datasets or require additional training, which makes our method widely applicable to existing pre-trained segmentation models. Such a straightforward approach achieves a new state-of-the-art performance on the publicly available Fishyscapes Lost & Found leaderboard with a large margin. Our code is publicly available at this $\href{https://github.com/shjung13/Standardized-max-logits}{link}$.

1. Introduction

The paper proposes standardized max logits and local-region processing to detect unexpected road objects in urban-scene segmentation without external data or additional training. It reports state-of-the-art performance on the Fishyscapes Lost & Found leaderboard.

Introduction

  • Existing segmentation models assign pixels to predefined classes, so unexpected objects such as a dog on the road may be mislabeled, posing a safety risk for autonomous driving.Source: §1 · p. 1
  • MSP can be overconfident on out-of-distribution samples, while max-logit ranges differ by predicted class, undermining detection metrics that apply one threshold across classes.Unexpected objects predicted as one class can have higher max logits than in-distribution objects predicted as another.Source: §1 · p. 2
  • Standardized max logits align score distributions across predicted classes, reflecting within-class relative values and reducing false positives under a single threshold.The method standardizes max logits class-wise using training-sample statistics.Source: §1 · p. 2
  • Iterative boundary suppression replaces high anomaly scores at class boundaries with lower scores from neighboring non-boundary pixels, while dilated smoothing uses local consistency to reduce remaining false positives.Source: §1 · p. 2
  • The approach is designed for existing segmentation models and requires neither additional training nor external datasets.Source: §1 · p. 2
  • The method achieves a new state-of-the-art result on the Fishyscapes Lost & Found leaderboard, with a large margin and negligible computation overhead.It does not require additional training or out-of-distribution data.Source: §1 · pp. 2–3

2. Related Work

Urban-scene segmentation has advanced, but existing models do not identify unexpected objects. Prior detection approaches require external training data or an additional model, motivating a method that works with a given segmentation model without extra training or datasets.

2.1. Semantic segmentation on urban driving scenes

  • Urban-scene segmentation studies [8] [9] [22] [43] [44] [12] [6] improve accuracy or inference efficiency, but do not identify unexpected objects.Their techniques include scale changes [22], urban-scene geometry and positional patterns [9], and efficient architectures [6].Source: §2.1 · p. 3

2.2. Detecting unexpected objects in semantic segmentation

  • Methods using unexpected-object samples from external datasets [2] [5] [3] require retraining, limiting direct use of a pre-trained segmentation network.Examples include overlaying ImageNet objects [2] or using COCO objects [5] as unexpected-object samples.Source: §2.2 · p. 3
  • Image-resynthesis methods [26] [39] [25] detect unexpected objects from reconstruction discrepancies but add lengthy inference time.These methods rely on an extra image-resynthesis model that fails to reconstruct regions containing unexpected objects.Source: §2.2 · p. 3
  • The proposed method targets real-time detection of unexpected objects using a given segmentation model, without additional training or external datasets.The paper identifies real-time detection as important for applications such as autonomous driving.Source: §2.2 · p. 3

3. Proposed Method

The method detects unexpected road obstacles by class-wise standardizing max logits, then refining the resulting anomaly scores using local pixel relationships. It requires no additional training or external data, and uses the negative final SML as the anomaly score.

3.1. Method Overview

  • Class-wise standardization addresses differing max-logit ranges, which can make unexpected objects appear more confident than in-distribution pixels from other predicted classes.The method aligns the distributions and reflects the relative meaning of max logits within each predicted class.Source: §3.1 · p. 3

3.2. Standardized Max Logits (SML)

  • SML standardizes each test pixel’s max logit using the training-sample mean and variance for its predicted class.These class-specific statistics are computed from the max logits and predicted classes of training samples.Source: §3.2 · p. 4

3.3.1 Iterative boundary suppression

  • Iterative boundary suppression replaces boundary-pixel SMLs with averages from neighboring non-boundary pixels while progressively shrinking the boundary region.It uses a 3 × 3 receptive field, four iterations, an initial boundary width of 8, and reduces boundary width by 2 per iteration.Source: §3.3.1 · p. 4, Figure 4 · p. 4

3.3.2 Dilated smoothing

  • Dilated Gaussian smoothing addresses irregular scores in non-boundary regions and residual boundary errors by incorporating information from beyond nearby pixels.The kernel size is 7, its standard deviation is 1, and the dilation rate is 6.Source: §3.3.2 · pp. 4–5

4. Experiments

The experiments evaluate standardized max logits on Fishyscapes and Road Anomaly, comparing benchmark performance and qualitative detections. They report a new state-of-the-art result on Fishyscapes Lost & Found and reduced false positives in visual comparisons.

4.1. Datasets

  • Evaluation datasets comprise real road obstacles in Fishyscapes Lost & Found, overlaid PASCAL VOC objects in Fishyscapes Static, and 60 web-collected anomalous-road images in Road Anomaly.Lost & Found includes 100 public validation images and 275 hidden test images; Static includes 30 public validation samples and 1,000 hidden test images. Road Anomaly images are 1280 × 720 and include diverse object scales and adverse road conditions.Source: §4.1 · p. 5

4.2. Experimental Setup

  • Experiments use a Cityscapes-trained DeepLabv3+ with ResNet101 backbone and output stride 8; quantitative metrics are AUROC, AP, and FPR95, with qualitative predictions visualized at TPR95.Comparisons include Fishyscapes leaderboard approaches, prior methods without external datasets or additional training, max logit [16], and SynthCP [39], which requires training additional networks.Source: §4.2 · pp. 5–6

4.3.2 Comparison on Fishyscapes validation sets and Road Anomaly

  • On Fishyscapes validation and Road Anomaly, the method outperforms previous approaches across all three datasets by a large margin and achieves significantly lower FPR95.The comparison is with methods that do not require additional training or OoD data.Source: Table 2 · p. 6, §4.3.2 · p. 6

4.3.1 Comparison on Fishyscapes Leaderboard

  • The method achieves new state-of-the-art performance on Fishyscapes Lost & Found by a large margin, outperforming six prior approaches there and five on Fishyscapes Static that meet at least one specified constraint.The comparison concerns approaches that do not require segmentation-network retraining or external OoD data; the leaderboard separately categorizes extra networks under additional training.Source: §4.3.1 · p. 6

4.3.3 Qualitative Analysis

  • At TPR95, the method reduces false-positive white regions compared with MSP [18] and max logit [16], which mark numerous less-confident in-distribution pixels as unexpected.The comparison is visualized in Fig. 5.Source: §4.3.3 · p. 6

5. Discussion

The discussion reports that standardized max logits (SML) support effective boundary processing, preserve reasonable segmentation performance, generalize across backbones, and add little computational cost. It explains that standardizing class-dependent max-logit distributions enables neighboring-pixel operations that do not help, and can hurt, other scores.

5.1. Ablation Study

  • 4% increase in AP and 1% decrease in FPR95 result from iterative boundary suppression on SMLs; dilated smoothing helps after suppression.Without iterative boundary suppression, dilated smoothing slightly increases FPR95, as boundary scores can propagate to non-boundary pixels; suppression first updates boundary regions using neighboring non-boundary regions. Figure 5 also shows fewer false-positive pixels than MSP and max logit.Source: Table 3 · p. 6, §5.1 · p. 6, Figure 5 · p. 7

5.2.1 Effects on the segmentation performance

  • At the TPR95 detection threshold, all methods reduce Cityscapes mIoU from 80.33%, but the approach retains reasonable segmentation performance alongside obstacle detection.MSP and max logit cause significant segmentation-performance degradation.Source: Table 4 · p. 7, §5.2.1 · p. 7

5.2.2 Comparison with various backbones

  • Across MobileNetV2, ShuffleNetV2, and ResNet50 backbones, the method significantly outperforms MSP and max logit in AP, demonstrating applicability across backbones.The method requires neither additional training nor extra OoD datasets.Source: §5.2.2 · pp. 7–8, Table 5 · p. 7

5.2.3 Comparison on computational cost

  • The method adds minimal GFLOPs and inference time relative to ResNet-101, and substantially less computational cost than SynthCP.Measurements use 2048 × 1024 images on an NVIDIA GeForce RTX 3090 GPU, with inference time averaged over 100 trials.Source: Table 6 · p. 8, §5.2.3 · p. 8

5.3. Effects of Standardized Max Logit

  • Iterative boundary suppression and dilated smoothing improve SML but do not improve, or can aggravate, MSP and max logit performance because SML aligns class-dependent score distributions.The alignment scales values so neighboring pixels can be used in computations; the overconfidence of softmax makes MSP anomaly scores insufficiently distinguishable from in-distribution scores.Source: §5.3 · p. 8, Table 7 · p. 8

6. Conclusions

The paper concludes that standardizing max logits and applying local-region smoothing can identify unexpected road obstacles without external datasets or additional training. The method achieves state-of-the-art performance on Fishyscapes Lost & Found, but may remove very small obstacles and depends on the main segmentation network’s max-logit distribution.

  • The method achieved new state-of-the-art performance on Fishyscapes Lost & Found, and experiments across diverse datasets showed superiority to prior approaches.Source: §6 · p. 8
  • The approach standardizes max logits within each predicted class and uses iterative boundary suppression plus dilated smoothing to remove irregular pixels based on neighboring values.Source: §6 · p. 8
  • Dilated smoothing may remove unexpected obstacles as small as noise, and performance depends on the max-logit distribution of the main segmentation network.Source: §6 · p. 8

A.1. Effects on Different Architecture and Backbone

On Fishyscapes Lost & Found validation, the approach outperforms all other methods with both EfficientPS and DeeplabV3+ with a ResNeSt backbone. The performance gain is not strictly correlated with downstream task performance, as prior work also noted [17] [19].

A.1. Effects on Different Architecture and Backbone

  • The approach outperforms all other methods with both EfficientPS and DeeplabV3+ with a ResNeSt backbone on Fishyscapes Lost & Found validation.Table 8 reports results for these architecture and backbone settings; the supplied passages do not provide numeric values.Source: §A.1 · p. 11, Table 8 · p. 11
  • The amount of performance increase is not strictly correlated with downstream task performance, as also noted in prior work [17] [19].Source: §A.1 · p. 11

A.2. Analysis on Hyper-parameters

Ablations on the Fishyscapes Lost & Found validation set examine how iteration count and dilation rate affect the proposed method.

Number of iterations n

  • At n = 4, AP is the primary selection criterion: it worsens at n = 5 even as AUROC and FPR95 improve.The ablation sets r0 = 2n so Δr = 2, reducing boundary width by one on each side per iteration; AP is chosen because it is invariant to pixel imbalance.Source: §A.2 · p. 11, Table 9 · p. 11

Dilation rate d

  • AP improves with dilation through d = 6, but performance degrades beyond that (e.g., d = 7), indicating that receptive-field size matters.The dilation rate controls the receptive field used in dilated smoothing.Source: §A.2 · p. 11, Table 10 · p. 11

A.3. Further Implementation Details

The implementation uses DeepLabV3+ with multiple backbones and specifies both the training recipe and boundary-processing parameters.

A.3. Further Implementation Details

  • With MobileNetV2 [37], ShuffleNetV2 [28], and ResNet50 [15], the method achieves best performance versus MSP [18] and max logit [16].The segmentation architecture is DeepLabV3+ [7], with ResNet101 [15] used as the backbone for most experiments; the method is described as model-agnostic.Source: §A.3 · p. 11
  • Training uses output stride 8, batch size 8, 60,000 iterations, initial learning rate 1e-2, and momentum 0.9.The recipe also uses polynomial learning-rate scheduling [27] with power 0.9, cross-entropy with PSPNet auxiliary loss [21] weighted λ = 0.4, color and positional augmentations, and class-uniform sampling [43] [4] at rate 0.5. Augmentations include color jittering, Gaussian blur, random scaling from [0.5, 2.0], horizontal flipping, and random cropping.Source: §A.3 · p. 11
  • Boundary processing uses n = 4 iterations, initial width r0 = 8, dilation d = 6, 3 × 3 boundary-aware pooling, and 7 × 7 smoothing.Source: §A.3 · pp. 11–12

A.4. Qualitative Results

At TPR95, qualitative visualizations show that the proposed methods reduce false positives, with iterative boundary suppression addressing boundary regions and dilated smoothing addressing non-boundary regions. Compared with MSP [18] and max logit [16], the method yields far fewer false positives while identifying most unexpected-object pixels.

Qualitative results

  • Visualizations use the TPR95 threshold, showing predicted in-distribution pixels in black and unexpected pixels in white.At this threshold, most pixels in unexpected objects are identified.Source: §A.4 · p. 12
  • Iterative boundary suppression significantly removes false positives in boundary regions; Figure 6 shows results on Fishyscapes Lost & Found for iterations 2 and 4.Red-box zooms illustrate the boundary-region changes.Source: §A.4 · p. 12, Figure 6 · p. 13
  • Applying all methods removes boundary-region false positives with iterative boundary suppression and non-boundary false positives with dilated smoothing.Figure 7 highlights these effects in yellow and green boxes, respectively.Source: §A.4 · p. 12

Comparison with other approaches

  • At TPR95, our method produces negligible false positives compared with the substantial false positives from MSP [18] and max logit [16], while identifying most unexpected-object pixels.The comparisons cover Fishyscapes Lost & Found and Fishyscapes Static.Source: §A.4 · p. 12

Iterative Boundary

The figures present qualitative unexpected-pixel predictions for boundary suppression and dilated smoothing, alongside comparisons with MSP and max logit on Fishyscapes datasets.

  • Figure 7 shows false positives effectively removed by iterative boundary suppression and dilated smoothing in Fishyscapes Lost & Found examples.Source: Figure 7 · p. 14
  • Figure 8 compares unexpected-pixel predictions from MSP, max logit, and the proposed method on Fishyscapes Lost & Found.Source: Figure 8 · p. 15
  • Figure 9 compares unexpected-pixel predictions from MSP, max logit, and the proposed method on Fishyscapes Static.Source: Figure 9 · p. 15
Loading 2107.11264v4…