UniHCP: A Unified Model for Human-Centric Perceptions

1. Introduction

• Human-centric tasks have distinct semantic focuses but share human-body structure, while dataset diversity and differing output structures make a unified model difficult to scale with separate heads.The paper notes that separate heads for each task or dataset do not scale when their number is large.Source: §1 · pp. 1–2
• UniHCP unifies pose estimation, semantic part segmentation, pedestrian detection, ReID, and person attribute recognition with a shared plain-ViT encoder-decoder, task-specific queries, and a task-guided interpreter.The design avoids task-specific output heads, minimizing task-specific parameters and making backbone features reusable across tasks.Source: §1 · p. 2
• Pretrained on 33 human-centric datasets, UniHCP handles the pretraining tasks with competitive performance against specialized baselines and achieves new SOTAs on several adapted benchmarks.The introduction also describes the model as competitive with task-specialized models.Source: §1 · p. 2
• 99.97% of UniHCP parameters are shared across tasks.The introduction lists maximum weight sharing as a central property of the model.Source: §1 · p. 2

2.1. Human-Centric Perceptions

• Pose estimation, pedestrian detection, parsing, attribute recognition, and ReID use different output-decoding approaches, and task-specific transformer designs do not integrate seamlessly.UniHCP builds on these transformer approaches by using a plain vision transformer architecture for human-centric tasks.Source: §2.1 · p. 2

2.2. Unified Models

• UniHead and UViM unify several vision tasks but are trained and evaluated in a single-task manner; other multitask models use independent decoder heads or autoregressive modeling.The cited multitask models do not focus on human-centric vision tasks.Source: §2.2 · pp. 2–3
• UniHCP uses task-specific queries and a task-guided interpreter to produce predictions in parallel through an encoder-decoder transformer.This is the shared-model formulation highlighted for handling a large collection of human-centric tasks uniformly.Source: Figure 2 · p. 3
• Earlier human-centric work co-trains task pairs, whereas UniHCP is presented as the first large-scale multitask model designed, trained, and evaluated across representative human-centric tasks.The paper identifies the absence of a general unified model handling all representative human-centric tasks as the gap.Source: §2.2 · p. 3

3. UniHCP

• 99.97% of UniHCP’s parameters are shared across tasks; only the decoder queries are task-specific.The model combines a task-agnostic encoder, a transformer decoder, and an interpreter that maps query outputs to units such as features, probability maps, probabilities, and boxes.Source: §3 · p. 3

3.1. Task-agnostic Transformer Encoder

• The encoder uses a plain ViT and interpolates a shared learnable positional embedding to accommodate different input resolutions.The embedding has size 84×84 before interpolation.Source: §3.1 · p. 4

3.2. Decoder with Task-specific Queries

• Task-specific queries guide decoder attention to task-relevant features; pedestrian detection uses learnable anchor points projected into positional embeddings.Other tasks use query counts corresponding to attributes, joints, parsing classes, or desired ReID feature length.Source: §3.2 · p. 4

3.3. Task-guided Interpreter

• The task-guided interpreter independently converts each query into task-requested output units, allowing parallel outputs without task-specific heads.Queries carry class or instance information, while outputs of the same modality use shared units.Source: §3.3 · pp. 4–5

3.4. Objective Functions

• Task objectives supervise the output units appropriate to each task, including global and local predictions for parsing and pose, and matched query predictions for detection.The overall objective weights losses across datasets; ReID uses identity and triplet losses, while attribute recognition uses weighted binary cross-entropy.Source: §3.4 · pp. 5–6

4.1. Implementation details

• Pretraining uses training splits from 33 public datasets, with task-specific input resolutions and a ViT-B encoder initialized from MAE weights.The reported main training run uses 105k iterations on 88 V100 GPUs.Source: §4.1 · p. 6

4.2. Main Results

• Evaluation covers 13 in-pretrain datasets, five datasets excluded from pretraining, and transfer with only a few training images.The in-pretrain results include direct evaluation and fine-tuning on target training splits.Source: §4.2 · p. 7, §4.2.1 · p. 7

4.2.1 In-pretrain Dataset Results

• After fine-tuning, UniHCP reaches new SOTAs on 9 of 12 datasets and matches the other three; direct evaluation is strongest on parsing and pose but trails SOTA on ReID.The authors attribute the ReID gap to its disparity from other tasks and suggest quick fine-tuning may remedy it.Source: §4.2.1 · p. 7

4.2.2 Cross-datasets Transfer Results

• On five datasets excluded from pretraining, UniHCP outperforms existing SOTAs on four, with transfer results reported across five human-centric task benchmarks.SenseReID is evaluated directly; UniHCP is fine-tuned on the other new datasets.Source: §4.2.2 · p. 8, Table 8 · p. 8

4.2.3 Data-Efficient Transferring

• With one image per class, prompt tuning reaches 93.65% pACC on ATR and 83.8% PCKh on MPII, close to full-data fine-tuning and better than one-shot whole-model fine-tuning.The reported prompt-tuning results also have lower standard deviations than one-shot fine-tuning for parsing and pose estimation.Source: §4.2.3 · p. 8, Table 9 · p. 8

4.3. Ablation Study on Weight Sharing

• Sharing task-guided interpreters across tasks matches task-specific heads on average while reducing parameters by about 30%; task-specific decoders or interpreters lower average performance by 6.8% or 2.4%.The comparison reports average direct-evaluation scores on in-pretrain datasets.Source: §4.3 · p. 8, Table 10 · p. 8

• UniHCP handles multiple distinctly defined human-centric tasks simultaneously using a simple query-based formulation.The conclusion reports competitive pretrained performance against task-specific models, SOTA results after task adaptation, and parameter- and data-efficient transfer capabilities.Source: §5 · p. 9

A. One-shot Transfer Results

• One-shot experiments sample ten image sets with different random seeds and tune iterations and learning rates; sampling accounts for absent classes in parsing and pose labels.The table note states that pose estimation needs only one sample when all keypoint joints are present.Source: A. One-shot Transfer Results · p. 14, Table 11 · p. 14

A.1. Human Parsing

• The human parsing one-shot results on ATR are reported using pACC, comparing fine-tuning with prompt tuning.The passage identifies the result table and tuning labels but provides no scores.Source: §A.1 · p. 14, Table 13 · p. 14

A.2. Pose Estimation

• The pose estimation one-shot results on MPII are reported using mAP, comparing fine-tuning with prompt tuning.The passage identifies the result table and tuning labels but provides no scores.Source: §A.2 · p. 14, Table 14 · p. 14

• On PETA, prompt tuning achieves better few-shot attribute-recognition performance with a smaller standard deviation; the balanced sampling constraint requires 68–75 images.Sampling covers ten domains and 35 attributes, ensuring both present and absent cases for each attribute.Source: B. Few-shot Transfer Results for… · p. 14, Table 15 · p. 15

C. Full Ablation Results on Weight Sharing

• UniHCP achieves comparable performance while sharing most parameters among human-centric tasks.Source: C. Full Ablation Results on Weight… · p. 15
• The reported comparison uses task-specific interpreters as the alternative to parameter sharing.Source: C. Full Ablation Results on Weight… · p. 15
• The ablation concerns different parameter-sharing methods.Source: C. Full Ablation Results on Weight… · p. 15

D.1. Task-guided Interpreter

• Each query token independently yields feature vectors, global probabilities, local probability maps, or bounding boxes through the task-guided interpreter.Feature vectors pass through unchanged; the other units use task-specific projections or feature-map operations.Source: §D.1 · p. 15

D.2. Positional Embedding for Encoder

• The encoder shares positional embeddings across tasks, interpolating them for input size; training images reach 1333 × 800 and use 7056 learned positional tokens.Images are padded to 1344 × 800 before patch projection, producing a maximum post-projection dimension of 84.Source: §D.2 · p. 15

D.3. Decoder Positional Embedding Projector

• The decoder positional projector sine-cosine encodes coordinates and then maps them through a two-layer MLP.Source: §D.3 · p. 15

D.4. Auxiliary Loss:

• Intermediate decoder queries receive auxiliary losses for pose estimation, human parsing, and pedestrian detection.The losses follow prior practices.Source: §D.4 · p. 15

D.5. Pose Estimation

• For pose estimation, inference multiplies the global probability prediction into confidence scores and lowers the visibility threshold to 0.05 when confidence filtering or NMS is used.The passage gives mAP as an example of a metric requiring confidence scores.Source: §D.5 · p. 15

E. Additional Training Details

• Dataset loss weights combine allocated batch size and task-type sample weight, then normalize to control only relative dataset weight.Samples within a task type receive equal importance; the authors tune newly added task types in a 10k-iteration joint-training setup.Source: E. Additional Training Details · p. 15
• The joint-training setup and detailed dataset-wise configurations are presented in Table 17.Downstream datasets include ATR, SenseReID, Caltech, MPII, and PETA.Source: E. Additional Training Details · p. 15, Table 17 · p. 16

F. Additional Finetuning Details

• Major finetuning configurations are provided in Table 18, with other settings matching the training configuration.Source: F. Additional Finetuning Details · p. 16, Table 18 · p. 17

G. Ethics

• Potential identity-information leakage without consent if misused leads the authors to release the pretrained model case by case for research use and exclude pretrained ReID query tokens.Source: G. Ethics · p. 16