PaperlayerView exact revision on arXiv

Source-linked AI summary

Boosting Unsupervised Video Instance Segmentation with Automatic Quality-Guided Self-Training

Kaixuan Lu, Mehmet Onurcan Kaya, Dim P. Papadopoulos

arXiv:2512.06864v1cs.CV

TL;DR

VIS annotation is costly because it requires pixel masks and temporal consistency labels, while synthetic-data methods such as VideoCutLER face a synthetic-to-real gap. AutoQ-VIS uses quality-guided self-training and achieves 52.6% AP50 on YouTubeVIS-2019 validation, 4.4% AP50 above VideoCutLER.

  • Problem

    VIS requires pixel-level masks and temporal consistency labels, making annotation costly, while VideoCutLER’s synthetic-video approach remains constrained by the synthetic-to-real domain gap.

    Source: §1 · pp. 1–2

  • Method

    AutoQ-VIS iteratively scores pseudo-labels generated on unlabeled videos with a quality predictor and adds high-quality labels to the VIS model’s training data.

    Source: §1 · p. 2

  • Results

    52.6% AP50 on YouTubeVIS-2019 validation set, 4.4% AP50 above VideoCutLER.

    Source: §1 · p. 2

  • Takeaways & Limitations

    AutoQ-VIS achieves state-of-the-art YouTubeVIS-2019 validation performance without human annotations, and its quality predictor is effective in pseudo-label assessment.

    Source: §5 · p. 8

    The paper reports no results on YouTubeVIS-2021 or OVIS because their validation splits lack ground-truth labels needed for class-agnostic VIS evaluation.

    Source: §4.3 · p. 8

Abstract

from arXiv · show

Video Instance Segmentation (VIS) faces significant annotation challenges due to its dual requirements of pixel-level masks and temporal consistency labels. While recent unsupervised methods like VideoCutLER eliminate optical flow dependencies through synthetic data, they remain constrained by the synthetic-to-real domain gap. We present AutoQ-VIS, a novel unsupervised framework that bridges this gap through quality-guided self-training. Our approach establishes a closed-loop system between pseudo-label generation and automatic quality assessment, enabling progressive adaptation from synthetic to real videos. Experiments demonstrate state-of-the-art performance with 52.6 $\text{AP}_{50}$ on YouTubeVIS-2019 $\texttt{val}$ set, surpassing the previous state-of-the-art VideoCutLER by 4.4%, while requiring no human annotations. This demonstrates the viability of quality-aware self-training for unsupervised VIS. We will release the code at https://github.com/wcbup/AutoQ-VIS.

1. Introduction

VIS requires detecting, segmenting, and tracking instances across video, while its pixel-level masks and temporal labels are costly to annotate. AutoQ-VIS addresses VideoCutLER’s synthetic-to-real gap with quality-guided self-training and reports a new YouTubeVIS-2019 validation result.

1. Introduction

  • VIS combines detecting, segmenting, and tracking instances, but training high-performance models typically requires costly pixel-level annotations across frames.Annotation work includes maintaining temporal consistency and instance identities.Source: §1 · p. 1
  • VideoCutLER avoids optical flow but remains limited by synthetic videos that lack natural motion patterns and real-world appearance variation.AutoQ-VIS builds on its synthetic-data paradigm and adds quality-filtered pseudo-labels from unlabeled real videos.Source: §1 · pp. 1–2
  • AutoQ-VIS scores pseudo-labels generated on unlabeled videos, adds high-quality labels to training, and retrains the VIS model alongside synthetic data.A frozen quality predictor estimates mask IoU; training alternates with quality-based dataset expansion.Source: Figure 1 · p. 1, §1 · p. 2
  • AutoQ-VIS establishes a new state-of-the-art in unsupervised video instance segmentation, achieving 52.6% AP50 on the YouTubeVIS-2019 validation set [39], a +4.4% AP50 improvement over previous best-performing VideoCutLER [31].Source: §1 · p. 2

2. Related Work

Related work distinguishes unsupervised video instance segmentation from video object segmentation and describes limitations of prior approaches. It also situates looped self-training and presents the quality predictor’s architecture.

2. Related Work

  • Unsupervised VIS distinguishes multiple instances without human labels, while many earlier VOS methods rely on annotated-data-trained optical flow estimators or track only one object.VideoCutLER introduced unsupervised multi-instance segmentation without optical flow, but its static synthetic videos miss natural motion and retain rigid instance modeling.Source: §2 · p. 2
  • The quality predictor combines mask predictions with pixel-decoder features and processes them through four convolutional and three fully connected layers to predict mask IoU.The final convolution layer uses stride 2 for spatial reduction.Source: Figure 2 · p. 3
  • Looped self-training retrains a model using pseudo-labels generated by that model.The passage describes its use across unsupervised and semi-supervised learning areas.Source: §2 · p. 3

3. Method

AutoQ-VIS initializes from VideoCutLER synthetic videos, then iteratively trains on quality-filtered pseudo-labels from unlabeled real videos. Its pipeline combines quality prediction, DropLoss, and adaptive dataset fusion.

3. Method

  • The pipeline combines synthetic-data initialization, multi-round quality-filtered self-training, DropLoss, and adaptive fusion without human annotations.Newly curated annotations augment the training dataset dynamically.Source: §3 · p. 3

3.1. Initial training stage

  • VideoMask2Former with a ResNet-50 backbone is initialized from VideoCutLER, while a quality predictor regresses mask IoU using frame features and raw predicted masks.The predictor is supervised with threshold-binarized (0.5) mask IoU and an ℓ2 regression loss.Source: §3.1 · p. 3

3.2. Multi-round self-training

  • Each round selects real-video pseudo-labels with confidence over 0.25 when predicted IoU times class-head confidence meets τ_th, then adds them to the training dataset.The product is the label's quality score.Source: §3.2 · p. 3

3.3. DropLoss for mask head

  • DropLoss omits a mask's loss contribution when its maximum ground-truth IoU falls below τ_IoU; τ_IoU = 0.01 filters near-zero-overlap predictions.The method modifies the mask-head loss.Source: §3.3 · pp. 3–4

3.4. Adaptive fusion of new curated annotations and old curated annotations

  • For new videos, qualified detections are inserted in bulk; for existing videos, overlapping detections are fused frame by frame, using IoU ≥ 0.5 to identify overlap.When detections overlap, the fusion uses the existing label only when it is selected and the new label is not.Source: §3.4 · p. 4
  • Training samples three frames only from videos' eligible frames, where all detections are selected, while all pseudo-labels remain in the training set regardless of selection.Uniform sampling is used for the three-frame input.Source: §3.4 · pp. 4–5

3. Method

  • 4.4 AP50: AutoQ-VIS outperforms VideoCutLER [31] on the class-agnostic YouTubeVIS-2019 val split.Table 1 reports the comparison.Source: Table 1 · p. 5

4. Experiments

AutoQ-VIS outperforms VideoCutLER on YouTubeVIS-2019 and UVO-Dense. Ablations identify strong component gains, while discussion finds quality-IoU alignment alongside a limited large-object bias.

4. Experiments

  • Training combines VideoCutLER synthetic videos with unlabeled YouTubeVIS-2019 training videos, using class-agnostic evaluation on the validation split.Batches are sampled equally from synthetic and pseudo-labeled videos.Source: §4 · p. 5

4.1. Experimental results

  • AutoQ-VIS improves over VideoCutLER by about 4.4% AP50 on YouTubeVIS-2019 and by 5.3% AP75.The reported gains are strongest for medium and large objects.Source: §4.1 · p. 5
  • AutoQ-VIS discovers more instances and produces higher-quality masks than VideoCutLER in qualitative comparisons.Examples include detecting missed humans, identifying semantically distinct objects, and delineating boundaries more precisely.Source: §4.1 · p. 5, Figure 4 · p. 6

4.2. Ablation studies

  • DropLoss contributes the largest reported component gain, improving performance by +4.6% AP50.It replaces the vanilla mask-head loss and was absent from VideoCutLER’s video pipeline.Source: §4.2 · p. 6, Table 3 · p. 6
  • Round 2 peaks at 52.6 AP50 before pseudo-label noise degrades later rounds; τth = 0.75 is optimal, while τth = 0.50 admits noisy supervision.Higher thresholds can oversuppress valid samples.Source: Table 4 · p. 7, Table 5 · p. 7, §4.2 · p. 7
  • Freezing the quality predictor contributes +3.4% AP50, adaptive fusion +2.9% AP50, and the quality predictor +2.1% AP50.The paper attributes freezing’s benefit to preventing mutual confirmation of pseudo-label errors.Source: §4.2 · p. 7

4.3. Discussion

  • Quality scores align with ground-truth IoU across object sizes, while pseudo-label area distributions stay stable but skew toward larger objects.The score slightly overestimates large instances, a bias reported not to hinder iterative self-training.Source: §4.3 · pp. 7–8, Figure 6 · p. 8, Figure 7 · p. 8
  • On UVO-Dense, AutoQ-VIS beats VideoCutLER by 1.1 AP50; YouTubeVIS-2021 and OVIS are omitted because validation labels needed for class-agnostic evaluation are unavailable.For UVO-Dense, training combines synthetic videos with unlabeled training videos and uses τth = 0.3 to accommodate higher object density.Source: Table 6 · p. 8, §4.3 · p. 8

5. Conclusion

AutoQ-VIS uses iterative pseudo-label refinement with automatic quality control for unsupervised video instance segmentation. It reports state-of-the-art performance on YouTubeVIS-2019 validation without human annotations.

  • 52.6 AP50 on YouTubeVIS-2019 val split establishes the reported state-of-the-art performance.Source: §5 · p. 8
  • AutoQ-VIS iteratively refines pseudo-labels through a closed loop of pseudo-label generation and automatic quality assessment.Source: §5 · p. 8
  • The framework requires no human annotations, and its simple quality predictor is reported effective for pseudo-label quality assessment.Source: §5 · p. 8

6. Detailed methodology

The detailed methodology filters model predictions into pseudo-labels using confidence, spatiotemporal overlap, and predicted mask quality. Retention depends on quality scores passing a threshold in at least one frame.

6.1. Automated pseudo-annotation with spatiotemporal NMS

  • Predictions with confidence scores of at least 0.25 are retained as candidate pseudo-labels for unlabeled videos.Source: §6.1 · p. 12
  • Spatiotemporal NMS suppresses a lower-confidence detection when its mask has IoU ≥ 0.5 with a higher-confidence detection in any video frame.Source: §6.1 · p. 12

6.2. Confidence-aware filtration via quality predictor

  • For each detection and frame, the quality score combines the VIS confidence score with the quality predictor’s estimated mask IoU, and the score is compared with a fixed threshold.Source: §6.2 · p. 12
  • A detection is retained if it passes the quality threshold in at least one frame; detections with no selected frames are discarded.Source: §6.2 · p. 12

7. Additional qualitative visualizations

The section provides additional qualitative visualizations of the VIS model and quality predictor. The model examples are from YouTubeVIS-2019 validation, while predictor scores are shown for the training split.

7. Additional qualitative visualizations

  • Figure 8 presents qualitative results of the VIS model on the YouTubeVIS-2019 validation split.Source: §7 · p. 12, Figure 8 · p. 13
  • Figure 9 presents quality-predictor results on the YouTubeVIS-2019 training split, with quality scores displayed at the center of each pseudo-label.Source: §7 · p. 12, Figure 9 · p. 14
Loading 2512.06864v1…