Filter-Aware Fine-Tuning for Safe Humanoid Whole-Body Tracking

I. INTRODUCTION

• Reference-tracking policies translate motion sources into dynamically consistent joint commands, but their references may omit constraints that are absent during training or change at runtime.The motion source may be a teleoperator, recorded demonstration, or kinematic motion generator.Source: §I · p. 1
• Runtime filters can enforce constraints absent during training, whereas training safety into a policy generally requires retraining for new constraints and does not itself provide a runtime guarantee.CBFs modify proposed commands only as much as needed to satisfy their input conditions [5].Source: §I · p. 1
• CoFiT adapts pretrained trackers to an existing CBF filter through filter-active fine-tuning, compact constraint information, temporal feedback, and constraint-aware penalties.The paper also analyzes dynamics, objective, and information mismatches at the policy–filter interface.Source: §I · pp. 1–2

A. Learning with safety filters

• Prior filter-training methods do not address adapting a pretrained whole-body tracker to an already deployed filter or which constraint representations and feedback signals are useful.Related work also spans learning with filters and safety constraints across the control hierarchy.Source: A. Learning with safety filters · p. 2, §II · p. 2

B. Constraints across the control hierarchy

• Safety constraints can be incorporated in planning, reference generation, or whole-body command execution, but this work focuses on adapting the tracker to an existing whole-body filter.Prior approaches include filters above learned locomotion controllers and command-level formulations that account for dynamics and contact [13], [14], [15], [16].Source: B. Constraints across the control… · p. 2

A. Relative Degree Two Control Barrier Functions

• Position constraints under torque control have relative degree two because the input does not enter through the position constraint’s first input derivative.The section introduces a relative-degree-two CBF formulation for these constraints.Source: A. Relative Degree Two Control Barrier… · p. 2
• The intermediate barrier combines constraint value with approach rate, regulating both proximity to the boundary and how quickly the system approaches it.Source: A. Relative Degree Two Control Barrier… · p. 2

B. Whole-Body Constrained Tracking

• The filter chooses corrections that limit disruption to contact support and the motion task, using a task metric over contact forces and task and posture accelerations.The paper uses the ConstrainedMimic framework to construct its safety filters [7].Source: B. Whole-Body Constrained Tracking · pp. 2–3
• For spherical body proxies and static spherical obstacles, the distance barrier uses their centers and radii plus a desired safety buffer; other shapes use corresponding signed-distance functions.Source: B. Whole-Body Constrained Tracking · p. 2
• The quadratic program stacks CBF constraints with actuator bounds, penalizes constraint relaxation, and returns filtered torque and slack before torque is mapped back to joint targets.The filter weights corrective effects on contact force and task/posture accelerations.Source: B. Whole-Body Constrained Tracking · p. 3
• The objective is better constraint adherence and smaller corrections without sacrificing tracking accuracy or smooth, coordinated motion, but execution can still violate constraints.Zero-order hold, model uncertainty, and QP relaxation can produce violations; the tracker is not expected to replan around fully blocking obstacles.Source: B. Whole-Body Constrained Tracking · p. 3

IV. FAILURE MODES OF INDEPENDENTLY DESIGNED POLICIES AND SAFETY FILTERS

• Applying runtime filters to trackers trained with an unconstrained actuator model introduces a distribution shift affecting tracking and stability.The paper distinguishes dynamics, objective, and information mismatches in the composed policy-filter loop.Source: §IV · p. 3

A. Dynamics Mismatch

• For humanoid tracking, enforcing clearance must preserve both progress toward the reference and whole-body stabilization.Source: A. Dynamics Mismatch · p. 3
• Filter interventions can disturb the tracker’s subsequent whole-body response even when the filter limits instantaneous intervention to preserve contact and task behavior.In the torso-clearance example, a sharp forward bend and raised knee follow the CBF interaction, potentially increasing contact-mode uncertainty and disturbing balance.Source: A. Dynamics Mismatch · p. 3
• When another constraint intervenes before coordination recovers, the recovery motion is corrected too, leaving the robot displaced from its reference and motivating filter-active fine-tuning.The paper also connects correction and motion histories to accounting for compounding interventions.Source: A. Dynamics Mismatch · p. 3

B. Objective Mismatch

• Persistent opposition between reference pursuit and safety filtering produces violent oscillations and can displace the ball in the forward-reach example.Even without violations, the passage reports pronounced jerky motion and says the tracker must accommodate the blocked reach without repeatedly disrupting the rest of the motion.Source: B. Objective Mismatch · p. 3
• Training on filtered transitions alone may not reward lower intervention, because different proposals can yield the same executed command and execution-based reward.A filter-blocked proposal that reaches farther into a restriction can be indistinguishable in that reward from one requiring less correction.Source: B. Objective Mismatch · p. 3

C. Information Mismatch

• With identical state, reference, and input history, the tracker cannot distinguish constraint placements that require opposite lateral adjustments before either triggers correction.The passage motivates providing constraint state, approach rate, action-direction information, and a history of constraint information.Source: C. Information Mismatch · p. 4

V. FILTER-AWARE POLICY DESIGN

• CoFiT retains the base tracker’s MLP and native tracking rewards while keeping the ConstrainedMimic whole-body filter active during training and deployment [7].Source: §V · p. 4

A. Constraint-Context Observations

• Constraint context expresses CBF inequalities in policy-action coordinates, where the constraint normal points toward safer action changes and the margin gives the constraint status at zero action.The observation also includes barrier value and rate, and the method encodes the constraint with the smallest barrier value; the filter still enforces all constraints.Source: A. Constraint-Context Observations · p. 4

B. Temporal Feedback and Policy Integration

• Constraint history tracks evolving local restrictions, while correction history records how the filter changed the tracker’s proposals.The passage distinguishes a proposed target before correction from the returned command after correction.Source: B. Temporal Feedback and Policy… · p. 4
• The policy uses filtered previous-action feedback and the corresponding action-rate reference, so both refer to command feedback after filtering.The action-rate reference is scaled elementwise by the action scale.Source: B. Temporal Feedback and Policy… · p. 4
• CoFiT appends the latest correction and retains 10 preceding frames of observations and encoded constraint context, resetting both histories at episode boundaries.Stored context embeddings retain their collection-time values without gradient flow.Source: B. Temporal Feedback and Policy… · p. 4
• New weights are initialized to zero, making the augmented network behave identically to the base policy before fine-tuning; Fig. 4 curves average three seeds over 100-update windows with sample-SD bands.Source: B. Temporal Feedback and Policy… · p. 4, Figure 4 · p. 5

C. Policy–Filter Alignment Rewards

• CoFiT adds three filter-related penalties: adaptations of correction and raw-condition rewards, plus a penalty on the largest solved slack.The correction and raw-condition reward adaptations follow CBF-RL and are modified for a whole-body task-space filter metric [9].Source: C. Policy–Filter Alignment Rewards · p. 5
• The solved-slack penalty addresses QP constraint relaxation when the problem would otherwise be infeasible.Source: C. Policy–Filter Alignment Rewards · p. 5
• The reward retains native terms while adding the penalties; σ sets correction scale, V_t contains CBF rows, and δ*_i,t is solved slack.Source: C. Policy–Filter Alignment Rewards · p. 5

VI. EXPERIMENTS

• The evaluation measures realized safety, filter cooperation, motion quality, and reference tracking, progressing from isolated design choices to realistic scenes, another tracker, and hardware.Source: §VI · p. 5
• Fixed world-frame references cannot replan, so scenes target near-misses or mild interference requiring local adjustment rather than rerouting.The passage notes that lateral-position or heading drift may turn these cases into planning-level problems.Source: §VI · p. 5
• Training uses three seeds except for the pretrained baseline (n=1), with 4,096 initial episodes per seed evaluated through first termination.Violation time is the percent of recorded steps with any violation; correction RMS is the joint-command correction norm.Source: §VI · p. 5
• Metrics include violation time, correction RMS, residual acceleration, command total variation, and world-frame MPJPE; Table I covers runtime-filter validation on sphere scenes and reports means with sample SD.Lower is better for the Table I metrics.Source: §VI · p. 5, Table I · p. 5

A. Controlled Design Study on Sphere Scenes

• The controlled study compares 29 configurations across three seeds, using training violation-rate curves as the primary selection tool under exploration noise and domain randomization.Those training violation values are higher than, and not directly comparable to, deterministic validation values.Source: A. Controlled Design Study on Sphere… · p. 5

What constraint information does the policy need? (Fig. 4a)

• The complete constraint-observation package yields the lowest training violations; neither the condition margin nor action-space normal helps alone, and the package beats oracle world-frame obstacle knowledge.Barrier value and rate help individually but do not specify how candidate actions affect the constraint.Source: What constraint information does the… · p. 6

How should constraint information be integrated? (Fig. 4b)

• Attention and softmax produce the highest training violation rates, while the other integration methods are essentially tied; top-1 is chosen for slightly safer validation and simplicity.The comparison holds rewards fixed and uses no constraint history.Source: How should constraint information be… · p. 6

What historical information is useful? (Fig. 4c)

• Adding constraint history and then correction history reduces violation time at each step.The trackers already receive the filtered previous action and one step of correction; the ablation holds nearest-constraint description and rewards fixed.Source: What historical information is useful?… · p. 6

How should the reward be modified? (Fig. 4d)

• Penalizing task-space correction captures most of the available gain, and adding a penalty for pre-correction violating proposals helps further.Further penalties are incremental; the recipe uses the solver-slack penalty, which measures solver behavior rather than robot state.Source: How should the reward be modified?… · p. 6

The CoFiT recipe

• Against continued unfiltered training, the chosen CoFiT recipe cuts violation time by 89% and filter correction by more than half, with 0.9 cm more world-frame tracking error.The recipe combines the nearest constraint’s complete description, histories of that description and filter corrections, and correction, raw-condition, and slack penalties.Source: The CoFiT recipe · p. 6

B. Constrained Tracking Across Scenes and Policies

• Violation time falls 91% for TWIST2 and 21% for SONIC versus filter-only fine-tuning, while correction RMS falls 59% and 29%, respectively.CoFiT also has the lowest mean command variation in both trackers; Table II summarizes constrained tracking across generated scenes.Source: B. Constrained Tracking Across Scenes… · p. 6, Table II · p. 7
• Across both trackers, observation-only training improves adherence without higher mean world-frame tracking error, while adding penalties further reduces violation and corrective effort at a tracking-accuracy cost.For SONIC, world-frame MPJPE is 7.11 cm after continued unfiltered training, 12.63 cm after filter-only fine-tuning, and 13.07 cm with CoFiT.Source: B. Constrained Tracking Across Scenes… · pp. 6–7

C. Real-Robot Validation

• The authors hypothesize that SONIC’s lack of world-frame feedback contributes to its tracking loss, but do not validate this experimentally.TWIST2 receives additional global-position feedback; SONIC does not.Source: C. Real-Robot Validation · p. 7
• TWIST2 CoFiT reduces violation time by 83% relative to continued training; residual acceleration falls from 65.6 to 31.9 rad/s2, and all ten trials finish without an operator stop.Five of ten baseline trials required an e-stop for violent thrashing.Source: C. Real-Robot Validation · p. 8
• For SONIC, mean violation time falls from 7.64% to 5.24%, alongside lower correction RMS, residual acceleration, and reaching-arm velocity error, but world-frame MPJPE rises from 5.45 to 12.17 cm.Command variation is slightly higher with CoFiT; all ten trials in both conditions finish without an operator stop.Source: C. Real-Robot Validation · p. 8
• Every CoFiT hardware trial across both trackers completes without an operator stop.This includes the SONIC-CoFiT trial excluded from continuous-metric statistics because its logged data were unusable.Source: C. Real-Robot Validation · pp. 7–8

• CoFiT fine-tunes humanoid trackers to improve constraint adherence and cooperation with runtime filters.Source: §VII · p. 8
• A compact constraint description already computed by the filter improves cooperation even under native rewards.Source: §VII · p. 8
• Correction feedback and penalties provide further gains, and the recipe applies across both tested trackers and several obstacle geometries.Future work will integrate CoFiT with safety-aware planners for more complex tasks.Source: §VII · p. 8