PaperlayerView exact revision on arXiv

Source-linked AI summary

HandAnthro: Automated Hand Anthropometry from a Single Image

Fan Zhou, Shuairan Chen, Mengying Zhang, Yulin Wu, Sadegh Jafari, Sixing Yu, Rui Li, Ali Jannesari, Guowen Song

arXiv:2609.37855v1cs.CVcs.AI

TL;DR

Existing hand measurement can require trained operators, specialized hardware, or manual landmarking, limiting practical collection of large, current samples. HandAnthro estimates projected dimensions from smartphone photographs using paper rectification and refined landmarks. In controlled testing, it achieved 97.8% completion and 3.80 mm caliper-referenced MAE, while its evaluation scope remains bounded.

  • Problem

    Hand-anthropometry methods often require trained operators, specialized hardware, or manual landmarking, while large, current samples are valuable for glove and hand-tool design.

    Source: §1 · p. 1

  • Method

    HandAnthro processes a smartphone image of a palm-up hand on letter-size paper, reconstructing occluded paper boundaries and refining YOLO-predicted landmarks with image-specific geometry and contours.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    97.8% of controlled captures yielded complete outputs, with 3.80 mm caliper-referenced MAE; regional MAE was 2.48 mm for non-thumb fingers, 6.04 mm for thumbs, and 6.17 mm for palm and wrist.

    Source: §6 · p. 6

  • Takeaways & Limitations

    The smartphone-and-paper protocol targets reduced equipment requirements and automated processing for larger, geographically distributed occupational hand-anthropometry studies.

    Source: §1 · p. 1

    Evaluation is limited to 45 controlled participants and a researcher-assisted field cohort without matched caliper measurements; generalization to unassisted capture, other devices, sites, and occupations remains unevaluated.

    Source: §6 · p. 6

Abstract

from arXiv · show

Hand anthropometry supports protective-glove design, but existing measurement methods often require trained operators, specialized hardware, or manual landmarking. We present HandAnthro, which estimates 44 projected hand dimensions from a smartphone photograph of a palm-up hand on US letter-size paper. The pipeline reconstructs wrist-occluded paper boundaries for rectification, whitens non-hand pixels, and refines 41 anthropometry-specific landmarks from a fine-tuned You Only Look Once (YOLO) pose model using image-specific geometry and contours. Controlled evaluation comprised 720 captures from 45 held-out participants, each contributing 16 images across two smartphones, two backgrounds, two angles, and two nominal illumination settings. HandAnthro produced complete outputs for 704 captures (97.8%); among these, mean absolute error (MAE) was 3.80 mm per dimension against two trained operators' caliper measurements. Regional MAEs were 2.48 mm for non-thumb fingers, 6.04 mm for thumbs, and 6.17 mm for palm and wrist. In a researcher-assisted mobile-app pilot, automated batch processing returned all 44 dimensions for 260 of 268 retained, researcher-screened firefighter images (97.0%). A descriptive, unpaired comparison with an independent national firefighter reference yielded a mean absolute difference of 2.40 mm across 28 sex-by-dimension group-mean contrasts. These results characterize controlled measurement performance and researcher-assisted field feasibility for future distributed hand-anthropometry studies.

1 Introduction

The section motivates larger, current hand-anthropometry samples for glove and tool design and presents HandAnthro as a smartphone-based approach to acquisition and localization barriers.

1 Introduction

  • Large, current anthropometric samples matter for glove and hand-tool design, while calipers, manual photogrammetry, and 3D systems impose staffing, per-image work, or hardware requirements.Poor glove fit compounds losses in dexterity, tactile sensitivity, and grip strength and reduces user acceptance.Source: §1 · p. 1
  • HandAnthro uses a smartphone photograph of a palm-up, fingerspread hand on letter-size paper to recover scale and estimate 44 projected dimensions.Its design targets reduced on-site equipment requirements and automated processing for geographically distributed studies.Source: §1 · p. 1
  • The pipeline reconstructs wrist-occluded paper boundaries, whitens non-hand pixels, and refines YOLO-predicted landmarks using image-specific geometry and contours.The stages produce protocol-defined measurements without participant-specific calibration.Source: §1 · p. 1
  • Evaluation combines a controlled repeated-capture study of completion, caliper-referenced accuracy, and capture-condition sensitivity with a researcher-assisted field pilot.The independent-cohort comparison assesses population-level plausibility, not individual-level measurement accuracy.Source: §1 · p. 1

2 Related Work

Prior hand-anthropometry methods span direct measurement and digital imaging, but their acquisition requirements, automation, and measurement coverage vary.

2 Related Work

  • Classical anthropometry measures standardized dimensions with calipers or rulers between predefined landmarks, creating staffing and quality-control burdens for distributed collection (Gordon et al. 1989; Yu et al. 2013).These burdens do not preclude large centrally organized surveys.Source: §2 · p. 2
  • Earlier controlled 2D studies used fixed camera geometry and interactive calibration or measurement lines, while 3D approaches commonly require scanners or calibrated multi-view systems (Habibi, Soury, and Zadeh 2013; Patel et al. 2018; Griffin et al. 2018b; Yang et al. 2021).Deep learning has automated extraction from a single 3D hand scan, but still depends on 3D sensing hardware.Source: §2 · p. 2
  • Automated 2D systems range from a scale-free two-finger ratio to 18 dimensions from one palmar scan and 24 parameters from 12 images in a light-controlled box (Sandnes 2014; Han and Park 2016; Nguyen, Le, and La 2025).Their portability, capture count, environmental control, and output breadth differ.Source: §2 · p. 2
  • Table 1 organizes selected methods by automated extraction, equipment, study sample, and source-reported hand-dimension MAE.Its caption cautions that MAEs are not directly comparable across studies.Source: §2 · p. 2

3 System Design and Methodology

HandAnthro converts a hand-on-paper smartphone image into 44 projected dimensions through paper rectification, background whitening, landmark prediction, and geometry-constrained refinement.

3 System Design and Methodology

  • The workflow rectifies the paper to correct projective distortion and establish metric scale, whitens non-hand pixels, predicts 41 landmarks, refines them, and visualizes 44 dimensions.The mobile app acquires the hand-on-paper image.Source: §3 · p. 2
  • Because the wrist interrupts a paper boundary, the perspective-rectification stage reconstructs the paper mask before detecting its quadrilateral.The recovered paper geometry defines the target plane for projective correction and metric scaling.Source: §3 · p. 2, Figure 2 · p. 3
  • Image-adaptive SAM-HQ prompts use MediaPipe’s 21 hand-pose landmarks as coarse location and scale cues; a dilated hand mask defines the region LaMa inpaints in the paper mask.The completed mask supports paper-quadrilateral detection and homography-based rectification.Source: §3 · p. 2
  • Background whitening retains pixels inside the warped hand mask and turns the rest white, removing non-hand pixels including shadows without a second segmentation inference.The whitened image is resized to a canonical frame for the YOLO predictor.Source: §3 · pp. 2–3
  • The fine-tuned YOLOv11x-pose model predicts 41 annotated anthropometry-specific landmarks, while post-processing uses finger geometry and contour evidence to refine endpoint placement.For each finger, the predicted root stays fixed as root-to-tip angle and length differences determine group rotation and scaling.Source: §3 · p. 3
  • After refinement, 44 fixed endpoint pairs define dimensions; the rectified US Letter sheet maps its 11-inch height to 932 pixels for conversion to millimeters.The scale is 932/11 pixels per inch, and each dimension is computed from its final endpoint distance.Source: §3 · p. 3

4 Experiments and Evaluation

The evaluation uses held-out participants, repeated capture conditions, caliper references, component comparisons, and a researcher-assisted firefighter field pilot.

4 Experiments and Evaluation

  • Model development used one annotated image from each of 42 participants, split into 33 training and 9 validation images, with the 45 controlled-test participants held out.Ablations compared four pipeline configurations across two pose backbones and two augmentation settings, yielding 16 experiments.Source: §4 · p. 4
  • The controlled test comprised 720 captures from 45 participants across two smartphones, two backgrounds, two angles, and two nominal illumination settings.The illumination settings were pooled because automatic exposure and image processing produced little visible brightness difference.Source: §4 · p. 4
  • Two trained operators’ mean caliper measurements served as the reference, and automated accuracy was evaluated on the 704 complete outputs using dimension-level and regional MAEs.Regions were thumb, four non-thumb fingers, and palm and wrist.Source: §4 · p. 4
  • Capture-condition accuracy was averaged across dimensions, while stability used within-participant prediction variation and differences between predictions at each factor’s two levels.The tested factors were smartphone, background, and angle.Source: §4 · p. 4
  • Landmark evaluation manually annotated 41 points on one image per participant; stage baselines compared rectification completion and error, alternative whitening methods, and a researcher-assisted pilot of 268 firefighters.The field pilot used one retained dominant-hand image per firefighter; sex-specific means for 14 dimensions were compared with an independent cohort as 28 population-level plausibility contrasts.Source: §4 · p. 4

5 Results

HandAnthro completed most controlled captures with millimeter-scale caliper error, and the field pilot returned complete measurements for most retained images. Errors and failures varied with anatomical region and capture conditions.

5.1 Component Selection, Ablations, and Stage-Level Baselines

  • On 406 shared captures, HandAnthro MAE was 3.66 mm versus 10.68 mm for CH-PR; it completed 704/720 captures versus 412/720 for CH-PR.The comparison evaluates perspective rectification against the CH-PR pipeline.Source: §5.1 · p. 4, Figure 4 · p. 5
  • HandAnthro’s background whitening had 3.80 mm MAE versus 4.26–4.71 mm for alternatives, with 17 ms stage time versus 93–379 ms.All four methods processed the same 704 rectified images and returned all 44 dimensions.Source: §5.1 · p. 5
  • Geometry-constrained refinement reduced mean 2D landmark error from 11.18 to 7.83 px (29.9%) but increased caliper-referenced MAE by 2.62%.The final framework retained refinement for anatomical landmark placement and user visual inspection.Source: §5.1 · p. 5

5.2 Controlled Repeated-Capture Evaluation

  • 704/720 controlled captures (97.8%) completed; overall caliper-referenced MAE was 3.80 mm, with regional MAEs of 2.48 mm non-thumb, 6.04 mm thumb, and 6.17 mm palm-and-wrist.MAE was calculated across 44 dimension-specific errors for completed outputs.Source: §5.2 · p. 5
  • Within-person prediction shifts were largest for angle and smallest for smartphone; top-down captures had higher MAE than oblique captures.MAE was similar between the tested smartphones and backgrounds; within-person SD had a 1.25 mm median and 3.99 mm 95th percentile.Source: Figure 5 · p. 5, §5.2 · p. 5

5.3 Researcher-Assisted Mobile-App Field Pilot

  • 260/268 retained images (97.0%) returned all 44 dimensions; the independent reference comparison differed by 2.40 mm across 28 sex-by-dimension group-mean contrasts.The comparison was descriptive and unpaired.Source: §5.3 · p. 5

5.4 Failure Analysis

  • All 16 controlled and 8 field failures occurred during PR; controlled failures involved wrist placement at or beyond a paper edge.Field failures also involved adaptive prompts outside the sheet, incomplete framing, hands extending beyond the sheet, or covered corners.Source: §5.4 · pp. 5–6

6 Discussion

HandAnthro’s overall controlled performance combines different regional errors and a trade-off between landmark localization and dimension MAE. The discussion also notes practical processing speed, capture-condition sensitivity, and limits on generalization.

Principal Findings

  • HandAnthro achieved 97.8% controlled completion and 3.80 mm MAE; PR reduced validation pose loss by 77–94%, and PP reduced landmark error by 29.9%.PP increased caliper-referenced MAE by 2.62%; the retained PR-stage mask for BW yielded lower MAE and shorter runtime than the three tested alternatives.Source: §6 · p. 6

Interpreting Measurement Agreement

  • Regional MAEs were 2.48 mm for 28 non-thumb finger dimensions, 6.04 mm for five thumb dimensions, and 6.17 mm for 11 palm-and-wrist dimensions.The overall 3.80 mm MAE therefore does not represent equal agreement across anatomical regions.Source: §6 · p. 6
  • HandAnthro’s 3.80 mm MAE is on the same broad several-millimeter scale as Kaashki et al. (2022)’s 4.5 mm, but study differences prevent direct accuracy ranking.The reported comparison differs in dimension definitions, samples, and reference measurements.Source: §6 · p. 6
  • Manual-landmark dimensions still had 3.38 mm caliper-referenced MAE; possible contributors include endpoint definitions, projection, and annotation, rectification, or caliper uncertainty.Manual annotations did not eliminate discrepancies between image-derived dimensions and caliper measurements.Source: §6 · p. 6

Practical Implications

  • Across tested settings, within-person prediction changes were larger for angle than smartphone or background, motivating clearer angle guidance and testing its effect on consistency.The discussion proposes evaluating whether such guidance improves measurement consistency.Source: §6 · p. 6
  • After capture, processing all 44 dimensions took about 4 seconds on an NVIDIA RTX 4000 Ada GPU or 26 seconds on an Apple M3, versus about 15 minutes for caliper measurement.The processing times exclude image collection; simple capture requirements and short processing times offer potential for larger-population data collection.Source: §6 · p. 6

Limitations and Path to Deployment

  • Evaluation covered 45 controlled participants and a researcher-assisted field cohort without matched calipers; unassisted use and other devices, sites, and occupations remain unevaluated.Planned work includes evaluating unassisted app use and calibrations for dimension-dependent biases using paired image-derived and caliper measurements.Source: §6 · p. 6

Ethics Statement and Resource Availability

The supplementary material documents HandAnthro’s capture, rectification, landmark-refinement, and comparison procedures, alongside measurement-accuracy results and study resource restrictions.

Ethics Statement and Resource Availability

  • Individual participant images and data are not publicly released; access requires prior IRB modification review, while code, a runnable sample, schemas, and aggregate results are provided.Source: Ethics Statement and Resource… · p. 6

D Adaptive SAM-HQ Prompts

  • Adaptive SAM-HQ prompts use MediaPipe hand landmarks and middle-finger length to generate separate paper and hand masks.The middle-finger root-to-tip distance sets the scale of the prompt offsets.Source: D Adaptive SAM-HQ Prompts · p. 8

E Quadrilateral Detection and Homography

  • The PR stage reconstructs the wrist-occluded paper mask, detects its quadrilateral, and maps its corners to a letter-ratio rectangle for perspective rectification.Quadrilateral candidates are selected from the five largest contours after fixed smoothing, edge-detection, and morphological-closing steps.Source: E Quadrilateral Detection and Homography · pp. 8, 10

F Background Whitening

  • Background whitening preserves rectified RGB pixels within the undilated hand mask and sets all other pixels to white.Source: F Background Whitening · p. 11

H Geometry-Constrained Post-Processing

  • Geometry-constrained post-processing uses MediaPipe landmarks and hand-contour searches to refine YOLO landmarks, including finger axes, fingertips, and measurement endpoints.Boundary snapping and special-point refinement are followed by recomputation of midpoint landmarks and metric dimensions.Source: H Geometry-Constrained Post-Processing · pp. 11–12

I.1 Canny–Hough Perspective Rectification (CH-PR)

  • CH-PR produced warpable quadrilaterals for 412 of 720 images (57.2%); on 406 shared completions, its MAE was 10.68 mm versus 3.66 mm for HandAnthro.Of 308 non-warpable cases, 266 lacked two orthogonal line groups, 32 had non-convex intersections, and 10 had an unrecoverable missing side.Source: §I.1 · p. 12

I.2 Background Removal Baselines (REMBG, BackgroundRemover, and CarveKit)

  • 1213 ms per image was required when HandAnthro’s BW ran from scratch, compared with 93 ms for REMBG; the production advantage comes from reusing the PR-stage mask.Source: §I.2 · p. 12

J Per-Dimension Accuracy Breakdown

K Per-Keypoint Pixel Error by Anatomical Group

The appendix evaluates post-processing (PP) through pixel-level keypoint comparisons and a paired millimeter-space ablation on common completed captures.

K Per-Keypoint Pixel Error by Anatomical Group

L Paired Millimeter-Space PP Ablation

M Full Ablation Configurations

The appendix reports training and validation pose losses across the full ablation configurations and descriptive accuracy by capture factor.

M Full Ablation Configurations

N Capture-Condition Sensitivity and Within-Person Variability

  • Capture-factor MAEs are descriptive rather than equivalence tests, and nominal illumination settings were pooled because captured brightness differed little.Table S11 reports accuracy by factor among complete outputs; its image counts are not counts of independent participants.Source: N Capture-Condition Sensitivity and… · p. 17, Table S11 · p. 17

N.1 Within-Person Capture Variability

Across completed captures, within-person prediction variability was modest overall, while condition-level differences were largest for capture angle. Angle-specific errors varied by dimension, and the reported runtime workloads are not directly comparable.

N.1 Within-Person Capture Variability

  • Median/P95 within-person SD was 1.25/3.99 mm and median/P95 CV was 3.41/8.12% across 1,980 participant–dimension combinations.These summaries are conditional on completion and describe capture variability, not caliper accuracy.Source: §N.1 · p. 17
  • Across participants and equally across dimensions, mean absolute differences between predictions at the two levels of each factor were 2.36 mm for angle, 0.46 mm for smartphone, and 0.88 mm for background.Summaries use the 704 completed captures.Source: §N.1 · p. 17, Table S12 · p. 18

O Dimension-Specific Capture-Angle Patterns

P Runtime Analysis

  • Runtime measurements used different CPU and GPU workloads, so they do not constitute a matched hardware comparison.The BW runtime includes cached-mask composition only and excludes mask I/O.Source: P Runtime Analysis · p. 19, Table S14 · p. 19

Q Annotation-to-Caliper Discrepancy Analysis

The annotation-to-caliper diagnostic averaged 3.38 mm, while trained operators showed small median absolute differences but broad agreement limits. The independent firefighter comparison describes group-level differences, not individual agreement.

Q Annotation-to-Caliper Discrepancy Analysis

  • Across 1,980 annotation-to-caliper errors, the mean discrepancy was 3.38 mm and the sample SD was 3.50 mm.The diagnostic compared all 44 dimensions derived from one manually annotated rectified image per test participant with the two-operator mean caliper reference; it combines endpoint, projection, annotation, rectification, and caliper discrepancies, and is not an irreducible lower bound on automated error.Source: Q Annotation-to-Caliper Discrepancy… · p. 19

R Inter-Operator Reliability of Manual Caliper Measurements

  • Mean relative bias between operators was 0.44%, with 95% limits of agreement from −9.80% to +10.69%.Median absolute difference was 0.90 mm (IQR 1.50 mm; P95 = 5.21 mm), and median APD was 2.86% (IQR 4.18%; P95 = 11.00%). Figure S11 plots signed relative differences against pair means.Source: R Inter-Operator Reliability of Manual… · p. 19, Figure S11 · p. 19

S Unpaired Population-Level Comparison with the National Firefighter Reference

  • Mean absolute difference was 2.40 mm across 28 sex-by-dimension group-mean contrasts, with 20/28 within 5% of the reference.The comparison is unpaired and descriptive; differing cohorts, laterality, and measurement protocols, plus a palm-breadth proxy, prevent separating measurement effects from cohort differences.Source: S Unpaired Population-Level Comparison… · pp. 20–21, Table S16 · p. 20

T Pipeline Failure-Mode Analysis

All 16 controlled pipeline failures occurred during paper rectification, and all eight incomplete field cases failed at that stage. The literature comparison defines automation narrowly as extraction after acquisition without per-hand manual landmarking or digitization.

T Pipeline Failure-Mode Analysis

  • PR and end-to-end completion were both 704/720 (97.8%), with no post-rectification failures among completed outputs.Accuracy was evaluated on those 704 complete captures, with no errors imputed for the 16 failures.Source: T Pipeline Failure-Mode Analysis · p. 21
  • All 16 controlled PR failures involved wrist placement at or beyond a paper edge, causing lower prompts to fall outside the sheet and preventing quadrilateral recovery.The complete sheet remained visible in every failed frame.Source: T Pipeline Failure-Mode Analysis · p. 21
  • All eight incomplete field captures failed during PR, including cases with misplaced hands, incomplete framing, a hand beyond the paper, or a covered corner.Source: T Pipeline Failure-Mode Analysis · p. 21

U Notes on the Literature Comparison

  • Here, automation means dimension extraction after acquisition without per-hand manual landmark annotation or digitization.Equipment lists capture hardware; computing and software are additionally required.Source: U Notes on the Literature Comparison · p. 21

Related papers

Loading 2609.37855v1…