Source-linked AI summary
Self-Correction for Human Parsing
Peike Li, Yunqiu Xu, Yunchao Wei, Yi Yang
TL;DR
Fine-grained human-parsing annotations are vulnerable to noise from ambiguous semantic-part boundaries. SCHP progressively refines labels while aggregating models online, and achieves state-of-the-art performance on two benchmarks, including 59.36 mIoU on LIP, though its soft-pseudo-label basis limits direct applicability to regression tasks.
Problem
Ambiguous boundaries and similar semantic parts make pixel-level human-parsing labels noisy, complicating reliable model training.
Method
SCHP cyclically aggregates models online and refines ground-truth labels into soft pseudo-masks that mutually improve models and labels.
Results
SCHP achieves new state-of-the-art performance on two human-parsing benchmarks; on LIP, it reaches 59.36 mIoU, 6.2 points above the previous closest approach.
Takeaways & Limitations
The self-correction strategy can be incorporated into other training frameworks for further performance improvement and can benefit relatively clean data through model ensembling.
Takeaways & Limitations
Because SCHP relies on soft pseudo-labels, the authors state that it may not apply to regression tasks such as detection.
Abstract
from arXiv · showhide
Labeling pixel-level masks for fine-grained semantic segmentation tasks, e.g. human parsing, remains a challenging task. The ambiguous boundary between different semantic parts and those categories with similar appearance usually are confusing, leading to unexpected noises in ground truth masks. To tackle the problem of learning with label noises, this work introduces a purification strategy, called Self-Correction for Human Parsing (SCHP), to progressively promote the reliability of the supervised labels as well as the learned models. In particular, starting from a model trained with inaccurate annotations as initialization, we design a cyclically learning scheduler to infer more reliable pseudo-masks by iteratively aggregating the current learned model with the former optimal one in an online manner. Besides, those correspondingly corrected labels can in turn to further boost the model performance. In this way, the models and the labels will reciprocally become more robust and accurate during the self-correction learning cycles. Benefiting from the superiority of SCHP, we achieve the best performance on two popular single-person human parsing benchmarks, including LIP and Pascal-Person-Part datasets. Our overall system ranks 1st in CVPR2019 LIP Challenge. Code is available at https://github.com/PeikeLi/Self-Correction-Human-Parsing.
1. Introduction
Human parsing requires fine-grained pixel labels that are difficult to annotate reliably, especially around ambiguous semantic-part boundaries. The paper addresses resulting label noise with SCHP and A-CE2P, reporting state-of-the-art benchmark performance.
- Human parsing assigns each human-image pixel to detailed semantic categories needed for image editing, human analysis, virtual try-on, and virtual reality.
- Ambiguous boundaries and similar semantic parts make pixel-level human-parsing annotations difficult and introduce multiple types of label noise.
- SCHP cyclically aggregates the current model with a former optimal model and refines labels into pseudo-masks that mutually improve model reliability and label accuracy.
- A-CE2P combines parsing and edge information implicitly through feature fusion and explicitly through a consistency constraint.
- 59.36 mIoU on LIP outperforms the previous closest approach by 6.2 points, while SCHP achieves new state-of-the-art results on two human-parsing benchmarks.
2. Related Work
Prior human-parsing work commonly assumes clean annotations, whereas SCHP treats noisy labels through cyclic online aggregation of models and predictions. The approach differs from conventional pseudo-labeling and self-ensembling by jointly optimizing labels and models during supervised training.
- Prior human-parsing methods often assume ground-truth labels are correct, despite unavoidable annotation noise and impractical pixel-level label cleaning.
- Pseudo-Labeling: SCHP uses soft pseudo-labels and cyclically aggregates the current model with the former optimal model to infer more reliable masks in supervised learning.
- Self-Ensembling: Unlike standard self-ensembling, SCHP averages model weights and predictions online so models and labels mutually promote one another during cyclical training.
3. Methodology
A-CE2P augments CE2P with direct mIoU optimization and explicit parsing–edge consistency, while SCHP iteratively aggregates models and refines labels. The resulting cycle is initialized after the model reaches sufficient quality and alternates model aggregation, pseudo-mask generation, and label refinement.
- Augmented-CE2P: A-CE2P extends CE2P with an mIoU loss and an explicit constraint matching parsing predictions to predicted boundaries.The overall objective jointly weights edge, parsing, and consistency losses.
- Self-Correction: SCHP uses A-CE2P as its base framework and performs online mutual improvement between model weights and training labels.The method is designed as a model- and label-aggregating process during training.
- Training Strategy: SCHP begins after a good initialization because inaccurate intermediate predictions may harm the iterative correction process.The training strategy relies on the initial model performance before self-correction starts.
- Self-Correction: Model aggregation combines the current model with the previous cycle’s model, then re-estimates batch-normalization statistics before continuing.The successive aggregation cycles are intended to broaden model optima and improve generalization.
- Self-Correction: Label refinement averages the previous training labels with pseudo-masks generated by the aggregated model for supervision in the next cycle.The pseudo-masks are described as smoother, less ambiguous, and capable of carrying relational information between fine-grained categories.
- Training Strategy: The training loop updates the learning rate, optimizes with the current refined labels, aggregates the model, recalculates batch-normalization parameters, and refines pseudo-labels.These operations are repeated across self-correction cycles.
4. Experiments
Experiments evaluate SCHP on LIP and PASCAL-Person-Part, compare it with state-of-the-art methods, and analyze its architectural, loss, and self-correction components. SCHP improves parsing performance, with gains associated with model aggregation, label refinement, and cyclic training.
- Comparison with state-of-the-arts: The baseline A-CE2P framework outperforms other state-of-the-art methods, while SCHP adds a 6.26% mIoU improvement through model aggregation and label refinement.
- Comparison with state-of-the-arts: On PASCAL-Person-Part, SCHP outperforms previous approaches, including DPC, while using ResNet-101 and ImageNet pre-training instead of stronger backbones and additional MS COCO data.
- Ablation Study: SCHP improves a MobileNet-V2 baseline from 52.06 to 53.13 mIoU, and deeper backbones generally perform better.
- Ablation Study: Adding edge information improves mIoU by about 0.4, while IoU loss increases mean accuracy by 2.8 points and mIoU by 2.2 points.
- Ablation Study: Self-correction cycles alleviate inaccurate boundaries, confused fine-grained and mirror categories, and multi-person occlusion in generated pseudo-masks.
- Ablation Study: Performance consistently improves across self-correction cycles, with the largest gain after the first cycle and saturation toward the end.
5. Conclusion and Future Work
The paper concludes that SCHP is an effective strategy for handling label noise in human parsing and can be incorporated into other frameworks. Future work extends it to multiple-person and video multiple-person parsing.
- SCHP achieves state-of-the-art human parsing performance with a large margin gain.
- The self-correction mechanism is presented as a general training strategy that can be incorporated into other frameworks for further performance improvement.
- Future work will extend SCHP to multiple-person human parsing and video multiple-person human parsing.