PaperlayerView exact revision on arXiv

Source-linked AI summary

Instance-level Human Parsing via Part Grouping Network

Ke Gong, Xiaodan Liang, Yicheng Li, Yimin Chen, Ming Yang, Liang Lin

arXiv:1808.00157v1cs.CV

TL;DR

Instance-level human parsing must recognize fine-grained parts and assign them to individual people, but prior approaches rely on separate detection and parsing stages and lack large-scale data. The paper introduces a detection-free Part Grouping Network that jointly learns and mutually refines semantic part segmentation with instance-aware edge detection. PGN surpasses previous methods on semantic part segmentation and edge detection and achieves state-of-the-art instance-level parsing on PASCAL-Person-Part and CIHP, a 38,280-image benchmark.

  • Problem

    Instance-level human parsing is under-explored because parsing multiple people in one image requires both fine-grained part segmentation and instance assignment, while available data resources are insufficient.

  • Method

    PGN is a detection-free unified network that jointly learns semantic part segmentation and instance-aware edge detection, with mutual refinement and simple instance partitioning at inference.

  • Results

    PGN surpasses previous methods on semantic part segmentation and edge detection and achieves state-of-the-art instance-level human parsing on PASCAL-Person-Part and CIHP.

  • Takeaways & Limitations

    The work establishes a unified formulation and a large-scale benchmark for evaluating multi-person human parsing in real-world images.

Abstract

from arXiv · show

Instance-level human parsing towards real-world human analysis scenarios is still under-explored due to the absence of sufficient data resources and technical difficulty in parsing multiple instances in a single pass. Several related works all follow the "parsing-by-detection" pipeline that heavily relies on separately trained detection models to localize instances and then performs human parsing for each instance sequentially. Nonetheless, two discrepant optimization targets of detection and parsing lead to suboptimal representation learning and error accumulation for final results. In this work, we make the first attempt to explore a detection-free Part Grouping Network (PGN) for efficiently parsing multiple people in an image in a single pass. Our PGN reformulates instance-level human parsing as two twinned sub-tasks that can be jointly learned and mutually refined via a unified network: 1) semantic part segmentation for assigning each pixel as a human part (e.g., face, arms); 2) instance-aware edge detection to group semantic parts into distinct person instances. Thus the shared intermediate representation would be endowed with capabilities in both characterizing fine-grained parts and inferring instance belongings of each part. Finally, a simple instance partition process is employed to get final results during inference. We conducted experiments on PASCAL-Person-Part dataset and our PGN outperforms all state-of-the-art methods. Furthermore, we show its superiority on a newly collected multi-person parsing dataset (CIHP) including 38,280 diverse images, which is the largest dataset so far and can facilitate more advanced human analysis. The CIHP benchmark and our source code are available at http://sysu-hcp.net/lip/.

1 Introduction

Instance-level human parsing extends human-part segmentation to assigning each part to a distinct person in complex multi-person images. The paper introduces PGN, a detection-free unified network that jointly learns semantic part segmentation and instance-aware edge detection, supported by the CIHP dataset and benchmark.

  • Motivation: Previous human-parsing methods mainly address single-person images, limiting analysis in real-world multi-person scenarios.These methods often assume upright poses or simplified daily images and disregard instance belongings.
  • Motivation: Instance-level parsing must segment body parts and associate each part with one person despite varying instance counts, appearances, viewpoints, and occlusions.Fixed prediction spaces from conventional single-person pipelines cannot naturally handle widely varying numbers of people.
  • Motivation: Parsing-by-detection separates instance localization and per-instance parsing, creating independent optimization stages that can produce inconsistent localization and segmentation.For example, segmentation may predict parts outside detection boxes.
  • Method: PGN reformulates parsing as part-level pixel grouping and instance-level part grouping through semantic part segmentation and instance-aware edge detection.Semantic segmentation assigns each pixel a part label, while instance-aware edges separate parts belonging to different people.
  • Method: PGN shares representations across parallel segmentation and edge branches and adds refinement so complementary contextual information can correct errors between the tasks.The refinement scheme can recover small parts missed by segmentation and correct mistakenly labeled background edges.
  • Resources: The paper introduces CIHP, a 38,280-image dataset with pixel-wise annotations for 19 semantic parts in instance-level multi-person parsing.It also provides a public benchmark server for fair comparison.

2 Related Work

Prior human-parsing datasets and methods largely emphasize part segmentation or proposal-based instance segmentation rather than fine-grained multi-person parsing. The paper positions PGN as a unified, mutually refining alternative and CIHP as a comprehensive benchmark for this task.

  • Human Parsing: Earlier human-parsing datasets contain few person instances and evaluate part segmentation without representing instance belongings.CIHP is presented as the first and most comprehensive dataset focused on instance-level human parsing.
  • Instance-level Object Segmentation: Proposal-based instance-segmentation methods commonly optimize detection and segmentation sequentially, while proposal-free methods use multiple subtasks or complex partitioning.These approaches provide context for PGN’s unified formulation.
  • Positioning: The paper combines semantic part segmentation and instance-aware object segmentation into a more challenging fine-grained instance-level human-parsing task.This task requires both semantic part recognition and instance separation.
  • Positioning: PGN unifies and mutually refines semantic part segmentation and instance-aware edge detection end to end, avoiding the expensive CRF refinement used in prior work.Final results are obtained through a simple instance partition process.

3 Crowd Instance-level Human Parsing Dataset

CIHP is designed as a large-scale benchmark for multi-person human parsing in unconstrained images. It provides diverse, densely annotated data, controlled dataset splits, and statistics covering person counts and semantic-part labels.

  • Dataset Properties: CIHP contains 38,280 diverse human images and is described as the largest multi-person human-parsing dataset to date.Its images come from real-world scenarios and include rich person-level information.
  • Statistics: CIHP covers challenging poses and viewpoints, heavy occlusions, varied appearances, and a wide range of image resolutions.Figure 3 summarizes the number of persons per image and the distribution of 19 semantic-part labels.
  • Collection and Annotation: CIHP images were collected from unconstrained web resources using diverse multi-person search keywords and underwent professional annotation with quality control and a second-round check.Low-quality or unusable images were removed.
  • Splits: The dataset split contains 28,280 training images, 5,000 validation images, and 5,000 test images with withheld annotations.The hidden test annotations support the benchmarking setup.
  • Statistics: Every CIHP image contains at least two people, with an average of 3.4 person instances per image.The dataset also reports distributions over semantic-part labels.

4 Part Grouping Network

PGN uses shared features with parallel semantic-part and instance-aware edge branches, then mutually refines their predictions. A simple edge-guided partition process groups predicted parts into person instances without detection proposals.

  • Network architecture: PGN jointly trains semantic part segmentation and instance-aware edge detection from shared intermediate representations.The segmentation branch assigns pixels to part labels, while edge predictions determine which parts belong to distinct people.
  • Backbone sub-network: ResNet-101 with atrous convolution provides shared feature maps, while single-scale processing and pyramid pooling add contextual information efficiently.The design avoids the original Deeplab-v2 multi-scale input scheme, which is memory-intensive and time-consuming.
  • Prediction branches: Parallel segmentation and edge branches produce part and boundary score maps, with deep supervision and multi-scale context used for edge prediction.The edge branch attaches side outputs to the final three ResNet-101 blocks and uses ASPP with dilation rates of 2, 4, 8, and 16.
  • Refinement branch: A refinement branch maps segmentation and edge predictions back into feature space and uses pyramid pooling to mutually improve both outputs.The remapped predictions are combined with branch features before the refined segmentation and edge results are generated.
  • Learning objective: The learning objective combines segmentation and edge losses for initial, refined, and side-output predictions, balanced by weights α and β.Segmentation uses per-pixel multinomial cross-entropy, while each edge output uses per-pixel sigmoid binary cross-entropy.
  • Instance partition: During inference, horizontal and vertical line segments are grouped as connected components, then small boundary regions are merged with larger neighboring regions.The resulting instance maps are associated with part segmentation maps to produce instance-level parsing results directly.

5 Experiments

Experiments evaluate PGN on PASCAL-Person-Part and CIHP using semantic segmentation, edge detection, and instance-level parsing metrics. Ablations show that unified learning, refinement, and grouping improve the reported results, while CIHP remains more challenging because of its diverse multi-person content.

  • PASCAL-Person-Part results: PGN achieves a 2.1% improvement in average IoU over the closest competitor on PASCAL-Person-Part semantic part segmentation.The reported advantage is attributed to incorporating object-boundary information into pixel-wise prediction.
  • PASCAL-Person-Part results: PGN gives a substantial boost in ODS and OIS for instance-aware edge detection on PASCAL-Person-Part.The authors report that semantic part segmentation benefits edge detection within the unified network.
  • PASCAL-Person-Part results: PGN achieves state-of-the-art instance-level human parsing on PASCAL-Person-Part, particularly at high IoU thresholds.The authors associate this result with smoother segmentation boundaries refined by edge context.
  • CIHP results: On CIHP, part segmentation and instance-level parsing perform worse than on PASCAL-Person-Part, whereas edge detection performs better.CIHP contains more instances, diverse poses and appearances, and more occlusions, while its higher-resolution images support edge detection.
  • Ablation studies: Joint training outperforms single-task PGN variants, and the refinement branch lets segmentation and instance edges exploit complementary contextual information.The grouping algorithm further refines instance-level parsing and can generalize to standard instance segmentation without predicted part labels.
  • Qualitative results: Qualitative comparisons show more precise PGN predictions than Holistic because predicted edges suppress background interference from objects such as flags and dogs.The visualized results cover both PASCAL-Person-Part and CIHP.

6 Conclusion

The paper presents a detection-free Part Grouping Network that jointly optimizes semantic part segmentation and instance-aware edge detection for instance-level human parsing. It also introduces a large-scale benchmark, and experiments show superiority over previous methods on both evaluated datasets.

  • The detection-free Part Grouping Network jointly optimizes semantic part segmentation and instance-aware edge detection end to end.The two correlated tasks mutually refine each other within one network.
  • The authors introduce a large-scale benchmark for instance-level human parsing with 38,280 images and pixel-wise annotations on 19 semantic part labels.
  • Experiments on PASCAL-Person-Part and CIHP show superiority over previous methods for semantic part segmentation and edge detection, achieving state-of-the-art instance-level human parsing.
Loading 1808.00157v1…