PaperlayerView exact revision on arXiv

Source-linked AI summary

Tool-Policy Co-Design for Powder Weighing in Laboratory Automation

Nikola Radulov, Xin Yang, Kevin S. Luck, Gabriella Pizzuto

arXiv:2609.39797v2cs.RO

TL;DR

Powder manipulation is challenging because material flow varies and fixed human-centric tools constrain robot control. The paper jointly optimises a dispensing tool and its policy, using morphology search and similarity-based warm-starting. Co-designed tools achieve lower overall errors than baselines across the combined material set, although the simulation is limited for cohesive powders.

  • Problem

    Powder manipulation is challenging because material dynamics vary, while human-centric tool geometry bounds the precision achievable through policy control.

    Source: §I · p. 1

  • Method

    A bi-level framework jointly optimises tool morphology and its reinforcement-learning policy across powder flowabilities, warm-starting training from structurally similar designs.

    Source: §I · pp. 1–2

  • Results

    Co-designed Config. B achieves lower overall errors than baselines across the combined material set, though policy reliability decreases as material dynamics drift from training.

    Source: E. Discussion · p. 8

  • Takeaways & Limitations

    Although specialised for its target powder-dynamics distribution, the co-designed tool remains more effective overall than standard human tools.

    Source: E. Discussion · p. 8

    The simulation cannot accurately reproduce cohesive powders.

    Source: §V · p. 8

Abstract

from arXiv · show

Autonomous powder weighing is one of many bottlenecks in laboratory automation due to the complex, non-linear dynamics of heterogeneous materials. Robot chemists performing this task utilise standard tools shaped for the dexterity of human hands, whose fixed geometry sets the dynamics that the control policy needs to regulate. This work introduces a tool-policy co-design framework that concurrently optimises the morphology of a dispensing tool and its control policy for use by robots in chemistry laboratories, formulated as a bi-level optimisation that minimises dispensing error over a target distribution of powder flowabilities. The outer loop varies tool-design parameters such as tool depth, width and rim spike topology using Bayesian optimisation and hyperband, while an inner loop optimises a control policy for each candidate morphology. We also introduce a geometric similarity metric that warm-starts policy training from cached policies of structurally similar designs, exploring 28% more configurations under the same compute budget. The proposed framework is evaluated on a robotic powder weighing task across seven materials with distinct physical dynamics in a flowability-informed robot-material simulation framework. Experimental results demonstrate that our co-designed tool morphology reduces real-world weighing errors by 45% relative to a standard tool, including on previously unseen materials. These results demonstrate our method can adapt both the control policy and the physical tool to the dynamics of the target material, bringing a new paradigm for material manipulation to the field of laboratory automation.

I. INTRODUCTION

The introduction frames powder manipulation as a challenge because material dynamics vary and standard human-centric tools constrain robot control. It presents joint tool-and-policy optimisation, similarity-based warm-starting, and evaluation against standard and manually designed tools.

I. INTRODUCTION

  • Human-centric tools constrain robot-material manipulation because tool geometry sets the action-to-mass transfer function and bounds achievable precision.Prior approaches adapt policies around fixed tools.Source: §I · p. 1
  • The framework jointly optimises dispensing-tool morphology and a reinforcement-learning policy to minimise error across powder flowabilities.It formulates the task as a bi-level optimisation.Source: §I · p. 1
  • The design space combines continuous bowl scaling with discrete rim-spike configurations derived from standard tool geometries.The outer loop varies depth, width, and spike topology.Source: §I · pp. 1–2
  • A geometric similarity metric warm-starts policy training from structurally related designs, accelerating search without compromising discovered-tool performance.The framework pairs morphology search with a trained policy for each candidate.Source: §I · pp. 1–2
  • The authors report that co-designed tools outperform standard tools and grid-search baselines across diverse powder behaviours.They evaluate in simulation and on a real robotic manipulator.Source: §I · p. 2

A. Robot Skill Learning for Laboratory Automation

Prior laboratory-robotics work adapts skills to material dynamics, while prior tool co-design approaches leave joint optimisation under material variation open. This paper formulates powder weighing as a bi-level optimisation of tool morphology and its associated policy across material flowabilities.

A. Robot Skill Learning for Laboratory Automation

  • Robot skill-learning work addresses material manipulation with deep reinforcement learning, domain randomisation, or material properties in the learning process.Sample grinding, powder weighing, and scooping are identified as challenging because of nonlinear, unpredictable material dynamics.Source: A. Robot Skill Learning for Laboratory… · p. 2

B. Robotic Tool Co-Design

  • Prior co-design spans morphology-policy search and handcrafted laboratory end-effectors, but does not jointly optimise tool geometry and policy across material-flow variation.Li et al. vary object pose while holding policy fixed; laboratory tools are manually designed.Source: B. Robotic Tool Co-Design · p. 2

III. METHODOLOGY

  • The method casts powder weighing as a bi-level joint optimisation that minimises expected dispensing error over materials with diverse physical dynamics.The morphology and policy are treated as mutually dependent design choices.Source: §III · p. 2, A. Problem Formulation · p. 2

A. Problem Formulation

  • Material flowability is represented by angle of repose, with larger angles corresponding to lower flowability, and sampled across a representative range.The manipulator dispenses into a container on an analytical balance that provides weight feedback at each control step.Source: A. Problem Formulation · p. 2
  • Because geometry determines powder displaced per action and dynamics vary with material and remaining powder, morphology and policy cannot be selected independently.The formulation optimises morphology and policy using separate morphology and policy-learning objectives.Source: A. Problem Formulation · p. 2

B. Outer Loop: Tool Morphology Parametrisation

The design space parameterises a spoon-like tool through bowl dimensions and rim spikes, while keeping bowl length fixed. BOHB searches these morphologies by allocating progressively larger training budgets to stronger candidates.

B. Outer Loop: Tool Morphology Parametrisation

  • The tool design space uses an ellipsoidal spoon bowl with length, depth, width, and independently varied rim-spike heights.Bowl length is held constant so the tool centre point remains identical across candidates.Source: B. Outer Loop: Tool Morphology… · pp. 2–3, Figure 2 · p. 2
  • Depth controls bowl vertical extent, width controls lateral scale, and rim spikes provide finer control over material outflow.Greater width broadens surface area and enables capture of larger volumes per scoop; spike distribution is coupled to depth.Source: B. Outer Loop: Tool Morphology… · pp. 2–3

C. Outer Loop: Tool Design Generation

  • BOHB proposes morphologies and uses successive halving to discard weaker candidates at small training budgets before promoting survivors.A candidate receives a training-episode budget for its associated policy; multiple random seeds are evaluated because policy optimisation is stochastic.Source: C. Outer Loop: Tool Design Generation · p. 3
  • The outer-loop objective is expected weighing error across the flowability range, comparing target weight with final dispensed weight for each candidate and policy.Performance is estimated over multiple seeds for stability.Source: C. Outer Loop: Tool Design Generation · p. 3

D. Inner Loop: Policy Optimisation

The inner loop trains a control policy to dispense powder using tool-pitch adjustments and shaking actions, with weight feedback. Its reward is designed so cumulative return reflects final dispensing precision.

D. Inner Loop: Policy Optimisation

1) Reward:

  • The revised reward penalises step-wise changes in absolute weight error, making cumulative return depend only on final dispensing error.This aligns the policy objective with the outer loop’s final-precision cost and avoids the previous reward’s incentive for speed that could encourage overshooting.Source: 1) Reward · p. 3

2) Sampling of the Flowability Range:

Uniformly sampling flowability keeps morphology scores comparable across BOHB evaluation budgets. A similarity metric enables policy warm-starting from previously trained, structurally similar designs.

2) Sampling of the Flowability Range:

  • Uniform random flowability sampling makes evaluations at different BOHB budgets comparable by estimating the same target-range score.A curriculum would expose low-budget policies only to easier materials, potentially shifting morphology rankings as candidates advance.Source: 2) Sampling of the Flowability Range · p. 3

E. Similarity-based Evaluation

  • The similarity metric compares tool designs by total volume, width-to-depth ratio, and rim-spike topology.Volume and ratio similarities use scale-invariant min–max ratios; spike similarity captures changes between adjacent spikes weighted by arc ratio.Source: E. Similarity-based Evaluation · pp. 3–4
  • When a sufficiently similar design trained at the target budget is available, its cached policy warm-starts the new policy and reduces the training budget.Otherwise, the new policy receives standard training for the full budget.Source: E. Similarity-based Evaluation · p. 4

IV. EXPERIMENTAL EVALUATION

The evaluation asks whether simulation predicts real-world performance and morphology rankings, whether BOHB finds tools that beat baselines, and whether similarity-based search accelerates optimisation without degrading performance.

IV. EXPERIMENTAL EVALUATION

  • The evaluation tests simulation-to-reality prediction, real-world performance against commercial-tool and grid-search baselines, and similarity-based search acceleration.Source: §IV · p. 4

1) Simulation Setup:

  • Powder is simulated with Isaac Sim’s position-based dynamics solver, prioritising computational efficiency over physical fidelity for reinforcement learning.The setup uses high-fidelity and low-fidelity environments; the latter removes the balance and target receptacle and uses a coarser timestep.Source: 1) Simulation Setup · p. 4
  • Episodes begin with powder already in the tool, last 10 steps, and use 14 rim spikes with four possible discrete height levels.Tool width and depth scale continuously from 0.7 to 1.5 relative to the standard tool, alongside 414 discrete spike configurations.Source: 1) Simulation Setup · pp. 4–5

2) Materials:

The simulated target distribution covers four angle-of-repose values, while highly cohesive materials such as flour are excluded.

2) Materials:

  • The simulated flowability range spans 28°–41° angle of repose, sampled at 28°, 32°, 36°, and 41°; highly cohesive materials such as flour are excluded.Capturing clumping dynamics for those materials would require techniques considered prohibitively expensive for this framework.Source: 2) Materials · p. 5

3) Real World Setup:

The physical evaluation compares simulation predictions with real-world powder-weighing performance across tool morphologies. The manually designed spike configuration performed best in both simulation environments, while prediction gaps widened for the full test set that included highly cohesive out-of-distribution powders.

3) Real World Setup:

  • The real-world setup used a Franka Research 3 robot, an analytical balance, and 3D-printed tools; grid-search policies were trained with SAC for 3,000 episodes across three seeds.Scooping parameters were manually tuned to keep initial acquired mass within the simulated training distribution.Source: 3) Real World Setup · p. 5, B. Sim-to-Real Transferability and… · p. 5

B. Sim-to-Real Transferability and Simulation Fidelity

  • 1.27 mg and 1.54 mg were the mean errors for the manually designed spike configuration in the low- and high-fidelity environments, respectively, the lowest in both.All-spikes-enabled tools had the highest mean errors: 2.81 mg in low fidelity and 4.69 mg in high fidelity.Source: B. Sim-to-Real Transferability and… · p. 5, Figure 3 · p. 5
  • Config. 2 performed best on the IID and full material sets, while Config. 4 had lower IID error than the standard tool and performed best on salt.Config. 2 had the lowest IID error for powders other than salt.Source: B. Sim-to-Real Transferability and… · pp. 5–6
  • On IID materials, mean simulation-to-real prediction gaps were 1.78 ± 1.94 mg for high fidelity and 2.63 ± 2.83 mg for low fidelity.For the full test set including OOD powders, the gaps widened to 4.18 ± 1.78 mg and 5.10 ± 2.34 mg, respectively.Source: B. Sim-to-Real Transferability and… · p. 6
  • Highly cohesive materials were excluded from simulation because modelling their complex dynamics was prohibitively costly, so designs were not optimised for clumping or compressible behaviour.The real-world transfer evaluation included IID and OOD materials.Source: B. Sim-to-Real Transferability and… · pp. 5–6
  • Only three tools changed rank from simulation to real-world IID evaluation; the best- and worst-performing configurations retained their ranks.One Rank 1 configuration fell to Rank 2, and two Rank 3 configurations rose to Rank 2.Source: B. Sim-to-Real Transferability and… · p. 6, Figure 4 · p. 6

C. Tool Optimisation via BOHB

BOHB-selected tool designs were evaluated in simulation and on a physical robot. Two of the three designs improved on the earlier best tool in real-world trials, although the simulator predicted a different design would be best.

C. Tool Optimisation via BOHB

  • BOHB used a random-forest surrogate and expected improvement, assigning 330 to 3,000 training episodes through successive halving.The 200,000-episode search took approximately 7.85 days across four parallel workstations.Source: C. Tool Optimisation via BOHB · p. 6
  • The three BOHB designs were selected by high-fidelity simulation error at the maximum budget, then evaluated on the robot using their best-performing simulated policies.Real-world error was reported only for IID materials because transfer to poorly modelled OOD, highly cohesive powders was unreliable.Source: C. Tool Optimisation via BOHB · p. 6, Table II · p. 7
  • Config. b improved real-world weighing error by 0.26 mg across IID powders, while Config. c achieved the lowest absolute error on semolina.Config. b performed better than Config. c in physical trials, despite the optimiser predicting Config. c would be best overall.Source: C. Tool Optimisation via BOHB · pp. 6–7, Table II · p. 7
  • All three BOHB designs were predicted to outperform the earlier best tool, but only Config. b and Config. c achieved lower real-world errors.The prediction gaps were 0.25 mg for Config. c and 0.03 mg for Config. b; the authors attribute the rank inversion to unmodelled dynamics and 3D-printing variations.Source: C. Tool Optimisation via BOHB · pp. 6–7

D. Accelerated Co-design via Morphological Similarity

The similarity-based warm-start strategy increased the number of unique configurations explored under a fixed compute budget. The discovered designs also retained real-world performance, with one geometry recurring across the standard and similarity-augmented searches.

D. Accelerated Co-design via Morphological Similarity

  • Morphologies clustered by the similarity metric also showed comparable simulated task performance, supporting its use for warm-starting policy training.Figure 6 maps configurations in two dimensions, with proximity indicating greater morphological similarity and point colour indicating simulated task error.Source: D. Accelerated Co-design via… · p. 7, Figure 6 · p. 7
  • 136 configurations were explored with similarity-based BOHB versus 106 with standard BOHB under a 200,000-episode budget.The two methods had similar initial exploration rates, while similarity-based BOHB accelerated as more configurations were sampled.Source: D. Accelerated Co-design via… · p. 7, Figure 7 · p. 8
  • All three similarity-augmented designs outperformed the grid-search tool in simulation and physical experiments; Config. B improved real-world error by 0.10 mg.Config. B was morphologically identical to Config. b from the standard BOHB search, and the authors report that warm-starting did not degrade final tool quality.Source: D. Accelerated Co-design via… · pp. 7–8, Table III · p. 8

E. Discussion

Config. B achieves lower overall errors than the baselines across the combined material set, although the policy becomes less reliable as materials move outside the training distribution. The conclusion reports real-world gains across materials and identifies cohesive-powder simulation as a limitation.

E. Discussion

  • Config. B achieves lower overall errors than the baselines across the combined set, despite declining policy reliability as OOD material dynamics widen the sim-to-real gap.The tool is specialized for its target powder-dynamics distribution but remains more effective overall than standard human tools.Source: Table IV · p. 8, E. Discussion · p. 8

V. CONCLUSION

  • The framework jointly optimizes tool morphology and reinforcement-learning control, using geometric similarity to warm-start search from structurally related candidates.Source: §V · p. 8
  • The simulation cannot accurately reproduce cohesive powders, motivating higher-fidelity granular simulation that remains tractable for search.Source: §V · p. 8
  • Real-world experiments show co-designed tools outperform both a standard tool and grid-search baselines across materials.Source: §V · p. 8
Loading 2609.39797v2…