Macro-Micro Adversarial Network for Human Parsing

1 Introduction

• Human parsing segments an image into semantic parts and requires understanding both local regions and the global human configuration.The task has applications including behavior analysis, pose estimation, and fashion synthesis.Source: §1 · p. 1
• Pixel-wise classification can produce local errors such as holes and blur, as well as semantic errors such as unreasonable poses and incorrect body-part relationships.The cited figure illustrates a hole on an arm and an unreasonable pose.Source: §1 · p. 2, Figure 1 · p. 2
• A single adversarial discriminator cannot separately target errors associated with top and deeper layers, and high-resolution inputs can leave training unbalanced.The poor-convergence example has discriminator losses approaching 0 and generator loss approaching 1.Source: §1 · p. 3, Figure 2 · p. 3
• MMAN uses a dual-output generator and separate Macro and Micro discriminators for semantic consistency and local consistency, respectively.Macro D evaluates low-resolution maps; Micro D evaluates high-resolution maps.Source: §1 · pp. 3–4
• Small fields of view let the discriminators supervise global and local consistency without using large fields of view on high-resolution maps.The introduction identifies avoiding poor convergence on high-resolution images as a benefit of this design.Source: §1 · p. 4
• MMAN reports competitive parsing performance on LIP and PASCAL-Person-Part and good generalization on the relatively small PPSS dataset.The supplied introduction gives no metric values for these results.Source: §1 · p. 5

2 Related works

• CNN-based human-parsing progress is linked to large-scale datasets and end-to-end learning, with prior work adding pose cues, human-related contexts, or grammar,,.The reviewed works use pose cues, integrate human-related contexts, or jointly infer pose and parts with grammar.Source: §2 · p. 5
• Pairwise CRFs handle low-level inconsistency in a limited scope, while higher-order potentials can enforce semantic validity but are difficult to design.The review describes context utilization in CNNs as an open problem.Source: §2 · p. 5
• Adversarial learning has been applied to segmentation, but high-resolution training can become unbalanced when the discriminator readily recognizes fake images.The generator and discriminator may become stuck in a local minimum.Source: §2 · p. 5
• MMAN explicitly separates macro and micro adversarial subtasks; the two are complementary to a single adversarial loss in parsing accuracy and reduce training-unbalance risk.Macro supervision targets semantic consistency, while Micro supervision targets local consistency.Source: §2 · pp. 5–6, Figure 4 · p. 6

3 Macro-Micro Adversarial Network

• MMAN’s generator produces low- and high-resolution label maps, supervised over the whole low-resolution map by Macro D and over high-resolution patches by Micro D.For a 3×256×256 input, the label maps have sizes C×16×16 and C×256×256, respectively.Source: §3 · p. 6

3.1 Training Objectives

• The training objective combines pixel-wise multi-class cross-entropy with adversarial loss to enforce spatial consistency.The cross-entropy compares predicted class probabilities with the ground-truth class labels at each pixel.Source: §3.1 · p. 7
• MMAN applies cross-entropy and adversarial losses to both generator outputs, with λ1, λ2, and λ3 controlling the relative weights of the four losses.The low-resolution cross-entropy uses a small-sized target map, while the high-resolution cross-entropy uses the original ground-truth map; Macro and Micro adversarial losses operate on the low- and high-resolution maps, respectively.Source: §3.1 · p. 7

G max DMa,DMa LMMAN(G, DMa, DMi). (5)

• Training alternates optimization of G, DMa, and DMi until the MMAN objective converges.The procedure is specified for Eq. 5.Source: G max DMa,DMa LMMAN(G, DMa, DMi). (5) · p. 7

3.2 Dual-output Generator

• Do-DeepLab-ASPP produces full-resolution and 1/16-resolution label maps, supervised by corresponding ground-truth maps and adversarial losses.The shrunken output emphasizes coarse human structure, while the full-size output retains visual detail.Source: §3.2 · p. 8

3.3 Macro Discriminator

• Macro D globally scores a low-resolution label map for semantic consistency, including plausible poses and spatial relations between body parts.It is attached to the generator’s bottom layer and downsamples its input to a 1×1 score.Source: §3.3 · p. 8

3.4 Micro Discriminator

• Micro D averages real-or-fake responses over 22×22 patches of the high-resolution label map to target local inconsistency.It uses a shallow three-convolution-layer structure.Source: §3.4 · p. 8

3.5 Discussions

• Macro D’s deep, low-resolution supervision targets semantic structure; Micro D’s top-layer supervision adds fine-grained visual detail to coarse features.The complementary pattern is presented as a hypothesis in the discussion.Source: §3.5 · p. 9
• Both discriminators avoid large fields of view, reducing high-resolution convergence risk; MMAN also uses two shallower, lower-parameter discriminators than a single adversarial network.The paper evaluates convergence and efficiency in the variant study.Source: §3.5 · p. 9

4.1 Dataset

• LIP has 50,462 images and 19 part labels; PASCAL-Person-Part has 1,716 training and 1,817 test images; PPSS has 3,673 annotated samples.The datasets represent different scales and annotation schemes.Source: §4.1 · pp. 9–10

4.2 Implementation Details

• Training uses 288-pixel shorter-side resizing, random 256×256 crops, Adam with batch size 1, and multi-scale test-score averaging.The stated testing scales are 0.8, 1, and 1.2 times the original image size.Source: §4.2 · p. 10

4.3 Comparison with the State-of-the-Art Methods

• 46.65% mIoU on LIP exceeds five compared methods; on the same baseline, MMAN gains +1.40% over ASN and +0.62% over SSL.The reported improvements are in mIoU.Source: §4.3 · p. 10, Table 1 · p. 11
• 59.91% mIoU: Attention+MMAN exceeds the PASCAL-Person-Part state of the art by +0.55%; unaugmented MMAN reaches 58.45%.The attention module is added to MMAN for the final result.Source: Table 2 · p. 11, §4.3 · p. 11
• 52.11% mIoU on PPSS without fine-tuning beats DL, DDN, and ASN by +16.9%, +4.9%, and +1.4%, respectively.The model was trained on LIP and evaluated on PPSS.Source: §4.3 · pp. 11–12, Table 3 · p. 12

4.4 Ablation Study

• On LIP, removing Macro D or Micro D lowers full-MMAN mIoU by 1.21% or 1.29%; each discriminator alone improves baseline mIoU.The reported standalone gains are +0.88% for Macro D and +0.80% for Micro D.Source: §4.4 · p. 13
• Macro D is better at improving low-resolution mIoU, while Micro D is better at reducing isolated-pixel rate (IPR).The paper uses low-resolution mIoU to assess high-level structure and IPR to assess local inconsistency.Source: §4.4 · p. 13

4.5 Variant Study

• Single AN reaches 45.23% mIoU and has poor convergence; MMAN beats Double AN by 0.82%, while a third discriminator adds only 0.16% with more parameters.The third discriminator makes the architecture more complex.Source: §4.5 · pp. 13–14

• MMAN achieves parsing accuracy comparable to state-of-the-art methods on two challenge human parsing datasets.Source: §5 · p. 14
• MMAN reduces semantic inconsistency, such as misplaced human parts, and local inconsistency, such as blur and holes.Source: §5 · p. 14
• The two adversarial losses are complementary and outperform previous methods using a single adversarial loss.Source: §5 · p. 14
• MMAN uses small receptive fields for global and local supervision, avoiding the poor convergence problem of adversarial networks on high-resolution images.Source: §5 · p. 14