PaperlayerView exact revision on arXiv

Source-linked AI summary

Auditing Quality Filters for Long-Tail Human Data Curation

Rishav Agarwal, Nirshal Chandra Sekar, Anirudh Vemula

arXiv:2609.31896v1cs.CVcs.AI

TL;DR

Low poses are rare in curated construction data, and their representation reflects both their occurrence in footage and their survival through filtering. The paper audits filtering with alternative pose clues and tests person detection in controlled simulations. Recall gaps persist after bounding-box-height matching, while the practical recommendation is to retain valid person boxes even when pose estimates are uncertain.

  • Problem

    Low poses are rare in curated data, and their scarcity combines how often they occur in footage with how often examples survive filtering.

    Source: §I · p. 1

  • Method

    The paper audits the pose filter with four alternative low-pose clues and separately tests a person detector on rendered scenes with controlled appearance, pose, and box-height changes.

    Source: §I · p. 1, §III · p. 2

  • Results

    About 18 points when standing figures are shrunk to posed size and about 25 points when posed figures are enlarged to standing size; both recall-gap intervals exclude zero.

    Source: §IV · p. 3

  • Takeaways & Limitations

    Person-detection training should retain valid person boxes even when pose estimates fail the quality filter, while pose labels can use a stricter criterion.

    Source: §VI · p. 4

    The proposed separation of box validity from pose-estimate quality remains to be evaluated end to end.

    Source: §VI · p. 4

Abstract

from arXiv · show

Robots on construction sites must detect workers who are kneeling or bending, which we call low poses. These workers can be lost from training datasets during automatic labeling. We study a pipeline that detects people, estimates their body joints using NLF, and groups similar poses. Low poses account for only about 2 percent of the retained examples. This low share may partly reflect the pipeline's quality filter, which rejects examples with low detection confidence or uncertain joint estimates. We examine this filtering using four alternative pose clues: bounding-box shape, vertical body span, pose grouping aligned to the scene's vertical direction, and image appearance. All four suggest that low poses are rejected by the filter more often. Separately, controlled simulated scenes show that a person detector fine-tuned on a public construction dataset misses more workers in these poses even when we correct their bounding box height is matched to that of standing workers. These findings suggest that low poses are scarce and hard to find, and we cannot rely on bounding boxes or poses for long-tail human data curation.

I. INTRODUCTION

Low poses—kneeling, squatting, or deeply bent-over postures—are part of the dataset’s long tail. The paper examines how pose-quality filtering and person detection affect their representation and retention.

I. INTRODUCTION

  • Low poses means kneeling, squatting, or deeply bent-over postures, regardless of visibility or occlusion.A worker may be fully visible or partially occluded.Source: §I · p. 1
  • Low poses are rare in curated data, and their scarcity reflects both how often they occur in footage and how often they survive filtering.The paper calls this the selection–scarcity confound.Source: §I · p. 1
  • The paper separately examines which detected examples survive annotation and which poses a detector finds in rendered scenes.It also compares alternative clues for identifying low poses and tests changes in appearance, pose, and bounding-box height in simulation.Source: §I · p. 1

II. POSE MINING AND OBSERVED SCARCITY

The mined library contains few low poses, and their observed share varies by image source. Pose grouping aligns estimated bodies to scene vertical before clustering, while the filter audit is limited to people already detected.

II. POSE MINING AND OBSERVED SCARCITY

  • The kneeling/squatting and bent-double groups account for about 2% of the retained library.The fourteen groups were clustered from 3,762 detections; the two low-pose groups contain 41 and 43 instances.Source: §II · p. 2, Figure 2 · p. 2
  • Pose grouping aligns estimated bodies to scene vertical, normalizes joint coordinates by body extent, and rotates them to a common hip orientation.This is intended to group body arrangements rather than position, scale, or facing direction.Source: §II · pp. 1–2
  • Low poses make up a larger share of retained examples in external construction images than in deployment camera footage.The comparison does not identify the cause or establish how common low poses are across construction sites.Source: §II · p. 2, Table I · p. 2
  • The filter audit includes only people the detector already found, so it cannot account for people the detector missed.Confidence intervals resample whole recordings for video and treat external still images as separate units.Source: §II · p. 2

III. AUDITING POSE-DEPENDENT FILTERING

Four alternative checks indicate that low poses pass the quality filter less often, though pose-label errors and occlusion complicate the audit. Joint uncertainty appears to drive much of the unequal filtering, while body-based vertical alignment can also fail on low poses.

  • The four alternative audits use box shape, normalized vertical body span, scene-based alignment, and image appearance to identify low poses.They avoid some, but not all, weaknesses of the original pose labels.Source: §III · p. 2
  • All four checks suggest more frequent rejection of low poses, but they can misidentify poses and the image classifier remains unverified on rejected examples.The pre-filter frequency of low poses is also uncertain because checks can mislabel examples and removing repeated poses changes counts.Source: §III · p. 2
  • People with wider boxes relative to their height are rejected more often, even after accounting for box height, image-boundary contact, and image source.Box shape may reflect partial occlusion as well as pose, so pose alone may not explain the difference.Source: §III · p. 2
  • Allowing less certain joint estimates reduces the rejection gap, while changing required detection confidence has much less effect.The original pose-group audit is inconclusive because the uncertain joints also determine group assignment, and similar low-pose labels arise when comparable noise is added to trusted poses.Source: §III · p. 2
  • Body-based vertical alignment assumes the ankle-to-pelvis direction points upward, but this can fail for kneeling or bent workers; scene-based estimates disagree more in low-pose frames.The comparison shows disagreement, not exact alignment error, because neither estimate is ground truth; depth-based vertical estimation remains untested.Source: §III · pp. 2–3
  • Pass-rate ratios were 0.26 for box shape, 0.43–0.47 for vertical body span, 0.66 for scene-based alignment, and 0.50 for image appearance, all below 1.A ratio of 1 indicates equal pass rates for examples identified as low poses and other examples.Source: §III · p. 3, Table II

IV. TESTING THE DETECTOR IN SIMULATION

The simulation varies human models and poses in fixed scenes to test their effects on person detection. Mined poses reduce recall, and matching bounding-box height does not eliminate the gap.

IV. TESTING THE DETECTOR IN SIMULATION

  • The study renders 200 construction scenes with original models, new upright models, and the same new models in mined poses, holding locations, cameras, lighting, and scenes fixed.The detector was fine-tuned on a public construction dataset and evaluated on 968 people matched across five rendering conditions.Source: §IV · p. 3, Figure 4 · p. 3
  • Applying mined poses to the same new human models reduced recall by about 17 points, while changing models alone raised it by about 17 percentage points.The pose comparison holds the new models fixed; changing pose also changes box height, body outline, and visibility.Source: §IV · p. 3
  • Matching box height reduces but does not eliminate the recall gap, so smaller box height alone cannot explain the detector’s poorer performance on mined poses.The two height adjustments change scale in opposite directions, and the gap persists in both comparisons.Source: §IV · pp. 3–4
  • Within this subset, matching bounding-box height reduced but did not eliminate the recall gap: from about 29 points at original scale to about 18 when standing figures were shrunk and about 25 when posed figures were enlarged; both intervals excluded zero.Matching box height does not match silhouette or visibility.Source: §IV · p. 3
  • The detector and pose-quality filter need separate checks because the same postures are harder to detect and less likely to survive annotation.The simulation tests detection, while the field filter audit concerns annotation survival.Source: §IV · p. 4
  • The simulation tests one detector, and its paired pose comparisons cannot isolate pose from changes in body outline and visibility.The study also notes that simulated examples’ effect on detection in real construction footage remains untested.Source: §IV · p. 4

V. RELATED WORK

Related work situates this study among rare-case mining, selection-bias audits, pose-level subgroup analysis, and synthetic evaluation. Its focus is whether pose labels reliably audit rejected examples in a construction-robotics pipeline.

Rare-case mining in autonomous driving

  • The study examines whether a pose-annotation filter selectively removes a within-person pose group and whether labels used to audit that removal are reliable.This contrasts with autonomous-driving work on corner-case benchmarks and integrated data curation and model updating [10], [11].Source: §V · p. 4

Selection bias and latent subgroups

  • The construction-robotics case study addresses quality filtering’s effect on group representation, using alternative measurements and a noise control for auditing selective rejection.Related work includes class-balanced self-training [12] and a DataComp audit of group representation under quality filtering [13].Source: §V · p. 4
  • Pose-level analysis is motivated by rare-pose learning and hidden-subgroup methods, but recovering a pose group from model outputs does not validate its frequency among rejected annotations.The cited work includes Hwang et al., hidden stratification, and GEORGE.Source: §V · p. 4

Synthetic evaluation and application context

  • The study applies synthetic evaluation to construction poses and bounding-box height, focusing on the reliability of pose labels used to audit rejected examples.Prior work uses synthetic human datasets – and rendering for controlled model analysis –.Source: §V · p. 4

VI. IMPLICATIONS AND LIMITATIONS

The implications separate person-box quality from pose-estimate quality and call for audits of both filtering and detection. The authors identify unverified proxy labels and limited evaluation settings as unresolved boundaries.

VI. IMPLICATIONS AND LIMITATIONS

  • Keep valid person boxes for detection training even when pose estimates fail, while retaining a stricter criterion for pose labels.This does not mean accepting every detector prediction; false positives and localization errors still need quality control, and the separation remains to be evaluated end to end.Source: §VI · p. 4
  • Report the population behind each frequency and validate proxy pose labels on samples from both sides of the filter.The appropriate operating point for the coverage–fidelity trade-off is not established.Source: §VI · p. 4
  • Report detector recall by important pose group with example counts and confidence intervals, alongside precision at the chosen threshold.Pose-wise precision requires a rule for assigning false detections to pose groups; these measures do not establish robot coverage.Source: §VI · p. 4
  • The filter audit covers only people already detected, and alternative pose checks have not been independently verified on rejected examples.The study also tests one pose estimator; its vertical-direction estimate may be wrong, and the restricted simulation comparison may not represent crowded scenes.Source: §VI · p. 4
  • The practical lesson is to check both which examples filters discard and which people detectors miss, reporting results separately for important pose groups.The authors identify manual labeling of accepted and rejected examples, another pose estimator, and testing on real images not used in training as next steps.Source: §VI · p. 4

Related papers

Loading 2609.31896v1…