Aggregate, Don't Adapt: Subject-Level Posterior Aggregation and Transductive Calibration for Cross-Site Parkinsonian Gait Severity

Junlong Shen

Source-linked AI summary

Aggregate, Don't Adapt: Subject-Level Posterior Aggregation and Transductive Calibration for Cross-Site Parkinsonian Gait Severity

Junlong Shen

arXiv:2608.20587v1Aug 20, 2026cs.CVcs.AI

TL;DR

Cross-site Parkinsonian gait severity prediction is difficult when clinical test sites are unseen during training. The paper evaluates a frozen-encoder system whose main gains come from exact benchmark recipe reproduction, subject-level posterior aggregation, and label-free transductive calibration. It wins the hidden-test challenge while exposing cross-validation failure, aggregation-induced limits, and seed instability.

Problem
The paper addresses cross-site prediction of MDS-UPDRS gait severity from canonicalized SMPL motion at clinical sites absent from training.
Method
The system uses a frozen public motion encoder, a linear head, subject-level posterior aggregation, and label-free transductive corrections based on unlabeled test statistics.
Results
0.69447 macro-F1 ranked the system 1 of 58, with subject-level aggregation worth +0.143 macro-F1 and cross-validation anti-correlated with hidden-test performance.
Takeaways & Limitations
The benchmark’s decisive leverage lies in its supplied subject grouping and exact evaluation recipe rather than in changing the motion representation.
Takeaways & Limitations
Subject aggregation imposes a 69.5% hard walk-accuracy ceiling because 30.5% of walks are unreachable under one-label-per-subject predictions, while seed results span 0.694 to 0.558 macro-F1.

Abstract

from arXiv · show

We describe the winning entry to the MoCha 2026 Benchmark and Challenge on Parkinsonian Gait, which predicts MDS-UPDRS gait severity from canonicalized SMPL motion recorded at clinical sites unseen during training. The system reaches 0.6945 macro-F1 on the hidden test and ranked first of 58 entries, ahead of the runner-up at 0.5807 and the organizers' baseline at 0.4289, on a frozen public motion encoder with a single $4\times512$ linear layer. Nearly all of the margin comes from three stages usually treated as bookkeeping: reproducing the reference benchmark's exact head recipe, averaging per-walk posteriors within the subject grouping the organizers ship, and a label-free transductive calibration of the feature mean and the decision operating point. Fine-tuning the encoder lost in four distinct forms, and ten alternative encoders were worse. Every ablation number is a paid read on the hidden test, because our own leave-two-cohort-out cross-validation proved anti-correlated with the deciding score over eleven configurations. We give the negative record in full, and identify our largest gain, subject-level aggregation, as the binding ceiling on this benchmark.

1 Introduction

The paper studies cross-site Parkinsonian gait severity prediction and finds that benchmark-aware pipeline choices, rather than motion representation changes, produced the decisive gains.

Introduction: The task predicts MDS-UPDRS gait severity from canonicalized SMPL motion at clinical sites absent from training.Entries are evaluated by macro-F1 on hidden test motion.
Introduction: Subject-level posterior aggregation contributed +0.143 macro-F1, exceeding all representation changes combined.The aggregation uses the subject grouping supplied in the input, and its ablation values were paid hidden-test reads.
Introduction: The initial representation-gap hypothesis failed, while exact reproduction of the reference recipe, subject aggregation, and label-free transductive corrections improved the score.The four measured contributions are presented together in Fig. 1.
Introduction: 0.69447 macro-F1 ranked the system 1 of 58, using 14 KB of trained parameters on a frozen public encoder.The result was +0.114 over the runner-up and +0.266 over the released baseline.

2 Task, data and metric

The benchmark predicts walk-level severity from subject-grouped SMPL sequences, but subjects can have mixed labels and the metric specification leaves a key averaging detail unresolved.

2.1 What goes in and what comes out: The evaluation unit is a walk, and the server receives all hidden-test walks already grouped by subject.The prediction interface returns one integer severity class per walk.
2.1 What goes in and what comes out: Fig. 2 shows that one subject’s walks can carry different ground-truth severity classes, while aggregation over that grouping improves macro-F1 by +0.143.The example subject has eleven walks, including walks labeled classes 1 and 2.
2.2 Label space: The labeled CARE-PD data contains four UPDRS-labeled cohorts, 110 subjects, and 2952 walks, with rare class 3 and sharply differing cohort frequencies.Class 3 appears in 44 walks and is absent from two cohorts; 62 of 110 subjects have multiple labels across their walks.
2.2 Label space: The remaining five CARE-PD cohorts lack severity labels and were used only as unlabeled data in a negative semi-supervised arm.No cohort outside CARE-PD entered the system.
2.2 Label space: Table 1 counts walks by severity class for the four labeled cohorts and counts subjects whose walks have mixed labels.Its caption highlights rare, cohort-missing class 3 and the prevalence of mixed subjects.
2.3 Metric: Entries are ranked by macro-F1, but the provided artifacts do not specify which class set the average covers.The ambiguity matters because severity 3 is rare.

3 Method

The method freezes a public motion encoder, trains a linear probe, and applies transductive calibration, subject-level pooling, and a fixed operating point without using hidden-test labels.

3 Method: The system uses a frozen encoder with a linear probe and three post-hoc stages, with no encoder fine-tuning.The method therefore concentrates adaptation after feature extraction.
3 Method: Motion is converted from SMPL pose and translation into normalized side-view clips, then MotionAGFormer-S produces one 512-dimensional feature vector per walk.The encoder is used unmodified, with averaging over valid frames, clips, and joint tokens.
3 Method: Test-feature centering subtracts 0.90 times the estimated test feature mean before a 4×512 focal-loss linear head produces per-walk posteriors.The centering operation is the only distribution-alignment operation that transferred.
3 Method: Per-walk posteriors are uniformly averaged within supplied subject ids at λ = 1.0, then blended toward five nearest subjects with cosine-similarity weight λ_nbr = 0.40.The subject-mean aggregation gives every walk of a subject the same label before subject-kNN pooling.
3 Method: The system is transductive but label-free: its test-time stages use only unlabeled feature means and predicted class marginals.No hidden-test label is used, inferred, or reconstructed, and only the anonymized supplied subject ids are used.
3 Method: Training follows the reference protocol on four labeled CARE-PD cohorts, using transferred augmentations and a two-phase epoch-selection and refitting procedure.The data comprises 110 subjects and 2952 walks, expanded to 44280 augmented rows.

4 How we measured

The authors found that local cross-validation reversed the hidden-test ordering, especially for subject aggregation, so final decisions used paid reads on the fixed deterministic scorer.

Cross-validation calibration: Spearman ρ = −0.373 and Pearson r = −0.760 measured the association between local cross-validation and hidden-test scores across eleven configurations.The server’s best configuration scored close to the lowest locally, while the server’s worst scored highest.
Cross-validation calibration: Hidden-test performance preferred full subject aggregation by +0.143, whereas cross-validation preferred no aggregation by −0.042.The inversion was concentrated on the aggregation axis, which was also the most consequential axis.
Cross-validation calibration: Shuffling training labels reduced the same estimator from 0.4416 to 0.2234, indicating that cross-validation measured real signal despite ranking transfer performance incorrectly.The authors attribute the inversion to folds rewarding cohort-specific details that subject aggregation discards.
Selection procedure: The authors retired cross-validation for aggregation and operating-point choices and selected configurations using paid hidden-test reads.They considered this defensible because evaluation used one fixed test set and a deterministic scorer without a private resplit.
Selection procedure: Each submission consumed one of three daily evaluation slots, so local probes and prediction-dictionary diffs screened out configurations producing identical outputs.This screening eliminated a k-sweep, an evidence-weighting family, and four operating-point retune cells.

5 Results

The results show that reproducing the benchmark recipe and exploiting released subject groupings mattered more than changing the encoder, with subject-level aggregation providing the largest measured gain.

5.2 Which stage bought the margin: 0.524 macro-F1 resulted from reproducing the CARE-PD reference head recipe, compared with 0.467 for the initial logistic-regression head.The recipe used focal loss with α = 1, γ = 1, AdamW, and z-scored features on the same frozen features.
5.2 Which stage bought the margin: +0.143 was the gain from subject-level posterior aggregation, exceeding every model change combined.Averaging per-walk posteriors within each released subject grouping produced one label per subject.
5.2 Which stage bought the margin: 0.68371 was obtained at λ = 1.00, as the subject-aggregation sweep increased from 0.5407 at λ = 0.00.Uniform arithmetic pooling beat confidence-weighted, trimmed-25%, and logit-space geometric alternatives.
5.2 Which stage bought the margin: 0.69447 was achieved by local nearest-subject blending at λnbr = 0.40, while global test-mean blending scored 0.67517.The nearest-subject sweep had a sharp interior optimum, and k ∈ {3, 8, 12} matched k = 5 at λnbr = 0.40.
5.2 Which stage bought the margin: 0.68699 was reached at centering shrinkage c = 0.90, while the operating-point sweep peaked at 0.68371 for τ = 0.50 = τ = 0.52.Both continuous knobs were saturated and piecewise-constant after pooling was added.

6 What did not work

The evaluated alternatives did not improve the already aggregated pipeline, while averaging itself supplied the key variance reduction. Negative results were re-measured on the final system to isolate their effects.

0.04 macro-F1 was lost by eight-head same-recipe averaging on the pre-aggregation base, scoring 0.5007 against 0.5407.On the final system, two- and five-head averaging reached 0.69106, while a decorrelated-head pair reached 0.68764 against 0.69447 for the single head.
Ensembling was roughly neutral after subject-mean and kNN pooling, at −0.003 rather than a collapse.The authors attribute this to aggregation already providing the variance reduction an ensemble would supply.
A single-head no-op control reproduced deployed predictions exactly, isolating the measured effect of averaging.
Selecting voters by solo score outperformed selecting them by decorrelation, contrary to the usual guidance.

7 Where the ceiling is

Subject-level aggregation produces the largest gain but also imposes a hard ceiling because many subjects have mixed walk labels. Performance is additionally constrained by seed instability, site-dependent severity distributions, possible rater effects, and evaluation on one benchmark.

7.1 Our best stage is also our ceiling: 69.5% is the hard walk-accuracy ceiling imposed by one-label-per-subject aggregation, while the deployed system realizes 65.59%.Mixed labels affect 62 of 110 subjects, making 901 of 2952 walks unreachable; the λ = 1.0 family is about 95% saturated.
7.2 The head is unstable to its random initialization: 0.14 is the hidden-test macro-F1 spread across five retraining seeds, with scores ranging from 0.694 to 0.558.The shipped head is the best draw, and seed ensembling does not remove the variance.
7.3 A single operating point is wrong for every site at once: Class 3 is absent from two of four cohorts, while class 2 ranges from 0.15 to 0.44, limiting a single operating point across sites.An oracle-site-label arm gains +0.0221 in cross-validation, but every deployable version lost.
7.3 A single operating point is wrong for every site at once: Rater- and protocol-specific UPDRS differences may leave part of unseen-site generalization unrecoverable from source labels, but rater metadata was unavailable for testing.
The claims are limited to one benchmark, one metric, and one hidden test; the tested negative families do not exhaust possible methods.

8 Reproducibility and data availability

The report releases the ranked entry’s inference code, trained head, verification script, and runtime checks. Training and evaluation use the official CARE-PD release, with a frozen public encoder and specified labeled cohorts.

The public MIT-licensed code includes the ranked entry’s inference code, 14 KB trained head, three-step chain, and verification script.The script assembles the runtime layout, checks binary md5 values, and runs predict() on a released-cohort probe without hidden data.
CARE-PD supplies all training and evaluation data under CC BY-NC 4.0 from the official release.The four labeled cohorts are 3DGait, BMCLab, T-SDU-PD, and PD-GaM.
MotionAGFormer-S is used as released, pretrained on Human3.6M and not further trained.

9 Conclusion

The benchmark-winning margin came from exact recipe reproduction, released subject grouping, and label-free transductive corrections rather than a better motion representation. The authors conclude that subject-level scoring or rater metadata would sharpen the challenge, while local cross-validation should be calibrated against the deterministic objective.

The winning margin came from the exact reference recipe, subject grouping, and two label-free transductive corrections on a frozen encoder with 14 KB of trained parameters.
The practical ordering is to read the input format before exploring model alternatives and treat the exact training recipe as part of the benchmark.
Subject-level scoring or released rater metadata could make the challenge measure its intended target more sharply.The ceiling analysis suggests the current benchmark is measuring subject-level severity rather than walk-level severity.