Model-Free Adaptive Parameter Tuning for Efficient Multi-Robot Warehouse Operations

Pratap Tokekar; Mouhacine Benosman; Rahul Chandan; Alexandre Ormiga Galvao Barbosa; Michael Caldara; Joseph W. Durham

Source-linked AI summary

Model-Free Adaptive Parameter Tuning for Efficient Multi-Robot Warehouse Operations

Pratap Tokekar, Mouhacine Benosman, Rahul Chandan, Alexandre Ormiga Galvao Barbosa, Michael Caldara, Joseph W. Durham

arXiv:2608.21533v1Aug 21, 2026cs.ROeess.SY

TL;DR

Dense warehouse digout planning has facility- and condition-dependent parameters, making fixed offline tuning difficult. The paper applies model-free ESC to adapt planner parameters from throughput feedback, improving throughput by 5.0% across map and fleet variations and 8.4% under dynamic conditions.

Problem
Planner parameters trade off lane congestion against extraction time, while the throughput-optimal setting varies with facility configuration and changing demand or congestion.
Method
ESC tunes planner parameters online by applying sinusoidal dither and correlating perturbations with measured throughput to estimate gradients without a closed-form model.
Results
5.0% average throughput improvement occurs across map and robot-fleet variations, and 8.4% under dynamic operating conditions.
Takeaways & Limitations
The framework provides a self-tuning approach that eliminates manual parameter provisioning and adapts planner behavior to changing fulfillment-center conditions.

Abstract

from arXiv · show

Robotic Fulfillment Centers (FCs) store inventory on shelves (pods) arranged in dense blocks. Retrieving a target pod that is buried deep in a block requires moving obstructing pods out of the way (i.e., digout). Multi-robot planners use parameterized cost functions to control digout behavior, producing a spectrum of strategies: at one extreme, obstructing pods are sent to other blocks (using more robots in travel lanes); at the other, pods are shuffled within the block (avoiding lane congestion but increasing extraction time). Each point on this spectrum has different downstream consequences for floor congestion and throughput. The optimal operating point depends on the specific facility configuration and shifts with operational conditions such as varying station demand and congestion patterns, making offline tuning impractical. We present an adaptive parameter tuning framework based on Extremum Seeking Control (ESC) that continuously adjusts planner parameters in response to measured throughput. ESC performs model-free optimization by perturbing parameters with sinusoidal dither signals and correlating perturbations with performance changes to estimate gradients, making it robust to the multi-minute delayed effects and credit assignment challenges inherent in large FC operations. Simulation studies demonstrate that the adaptive policy improves upon fixed policies across several conditions. We observe an improvement in throughput by an average of 5.0% across map and robot fleet size variations, and by 8.4% under dynamic operating conditions. This work eliminates manual parameter provisioning and enables real-time adaptation, providing a self-tuning paradigm for FC storage operations.

I. INTRODUCTION

Dense fulfillment-center storage creates a throughput-sensitive digout trade-off whose best planner parameters vary across facilities and operating conditions. The paper proposes ESC-based online tuning and reports improved throughput over fixed policies.

Target pods buried in dense storage blocks require obstructing pods to be moved before retrieval.Robots transport pods between storage blocks and stations for item picking or stowing.
Sending obstructing pods to other blocks increases lane traffic and peripheral storage contention, whereas in-block shuffling avoids lane congestion but lengthens extraction.Intermediate parameter values produce hybrid digout strategies.
The throughput-maximizing parameter configuration depends on map layout, robot count, storage density, demand, and congestion, making static offline tuning brittle.Simulation-to-reality discrepancies further limit direct transfer of tuned parameters.
ESC continuously adapts planner parameters from measured throughput using dither perturbations and correlation-based gradient estimates without a closed-form system model.The approach targets automatic provisioning and dynamic adaptation under noise and delayed effects.
An average 5.0% throughput improvement occurs across map and robot-count transfers, while dynamic conditions yield 8.4% improvement overall and 14.3% post-disturbance.These results come from high-fidelity discrete-event simulation, including a 50% station logout disturbance.

II. RELATED WORK

The paper positions ESC as an underexplored adaptive-control approach for multi-robot warehouse planning, where combinatorial planner parameters influence delayed system-level throughput. The formulation separates local digout planning from the global objective of maximizing cumulative retrieval throughput.

Extremum Seeking Control: Prior ESC work establishes adaptive-control foundations, but applications to multi-robot warehouse systems remain limited.This paper extends ESC to combinatorial planning parameters mediated by a large discrete-event system.
Black-Box Optimization for Robotics: Bayesian optimization offers sample efficiency but typically assumes stationary objectives and may be less suitable for simple industrial deployment under compute constraints.Contextual bandits address adaptive decisions but commonly assume immediate rewards.
Multi-Robot Coordination: Existing multi-robot coordination work addresses block rearrangement, while real-time planner-parameter tuning from throughput feedback remains underexplored.The paper frames parameter selection as an online feedback-gain tuning problem.
Planning Model: The planner represents storage blocks as graphs and uses atomic slide, lift, lower, and dig actions to construct pod-extraction plans.A* plans one pod extraction at a time, potentially involving several robots.
Parameterized Planning Cost: Candidate digout plans are scored by a weighted action-count cost, with γ controlling the trade-off between external digouts and in-block shuffling.Low γ favors sending pods to other blocks; high γ favors slide-only plans that avoid lane congestion but may take longer.
Online Objective: The study tunes γ and station queue size ρ while fixing α and β, seeking long-term throughput rather than merely minimizing an individual plan’s local cost.Throughput is defined as pod retrievals per unit time, and the optimizer observes noisy scalar measurements without gradients.

IV. PROPOSED METHOD

The proposed method uses dither-based Extremum Seeking Control to optimize multi-robot digout-planner cost parameters, focusing on scalar γ while holding α and β constant.

Dither-based ESC tunes planner cost parameters for multi-robot digout planning and extends naturally from scalar γ to vector parameters θ ∈ R^d.The scalar presentation holds α and β constant while tuning γ.

A. Continuous-Time Dither-Based ESC

Continuous-time ESC estimates throughput gradients without explicit differentiation by perturbing the parameter sinusoidally, filtering and demodulating performance, then integrating the resulting signal.

ESC adds a sinusoidal dither to the parameter estimate and measures the resulting performance signal.The perturbation has amplitude a > 0 and frequency ω > 0.
A high-pass filter removes the throughput signal’s DC component before demodulation.This isolates variations associated with the dither frequency.
The filtered performance is multiplied by the dither and passed through a low-pass filter to produce the demodulated signal.The low-pass stage uses cutoff frequency ω_l > 0.
Under standard singular-perturbation and averaging assumptions, the demodulated signal approximates (a/2)∇J and drives gradient ascent on throughput.The update therefore estimates gradients without an analytical performance model.

B. Discrete-Time Implementation

The discrete ESC implementation filters measured throughput, demodulates dither-related variation, and updates the parameter estimate at fixed intervals while accounting for system delay.

The discrete-time ESC law uses θk as the parameter during each interval [tk, tk+1), with update period ∆t and K = T/∆t intervals.
The high-pass filter removes the throughput signal’s DC component while preserving variations at the dither frequency.
The demodulated signal is low-pass filtered before the discrete integration step updates the parameter estimate.
The applied parameter adds a sinusoidal dither to the estimate and is projected onto the feasible set Θ.
The demodulation signal can be phase-shifted to account for the delay between parameter changes and throughput response.

C. Multi-Parameter Extension

The framework extends ESC to multiple planner parameters by assigning separated dither frequencies and demodulating each parameter’s response independently. Its convergence statement is local and depends on smoothness, boundedness, small dither amplitude, and frequency separation assumptions.

For d-dimensional parameters, each component receives a sinusoidal dither with its own mutually distinct frequency.
Corresponding-frequency demodulation makes crosscoupling between parameter updates average to zero over time.
The convergence analysis assumes a convex compact feasible set, smooth bounded performance, a locally strict maximizer, bounded episode lengths, and bounded measurement variance.
The analysis requires dither frequency much larger than the low-pass cutoff and sufficiently small amplitude so higher-order and filtering remainders are negligible.
Under these assumptions and suitable initialization, ESC trajectories converge locally in practice toward a local maximizer as amplitude decreases and frequency increases.
The paper notes that its convergence discussion recalls continuous ESC theory, while discretization and multi-variable extensions rely on established results.

V. EXPERIMENTAL VALIDATION

The evaluation uses a discrete-event simulator with robot, pod, station, and request dynamics, across small and large warehouse maps and controlled ESC initialization conditions.

The simulator models robot kinematics, pod handling, station request patterns, individual robot states, pod locations, and station queues.
Experiments use MAP1 and MAP2, representing a smaller floor with hundreds of robots and a larger floor with more than a thousand robots.
Each simulation runs for 8 hours, excluding the first 10 minutes from analysis to allow steady-state operation.
The ESC is initialized with θ̂0 = θ0, yhp,0 = 0, and ξlp,0 = 0.
The experiments map research questions RQ1–RQ4 to tests of observability, phase delay, adaptive tuning, and transfer or dynamic performance.
The dither frequency and filter settings were selected after preliminary analysis to satisfy timescale separation, including a frequency slower than the measured phase delay.

B. RQ1: Does Dither in Control Input Manifest in Throughput?

RQ1 tests whether parameter perturbations produce observable throughput changes that ESC can use for gradient estimation. PSD analysis finds a throughput response at the dither frequency, while cross-correlation characterizes the response delay.

PSD analysis examines both control and throughput signals to test whether parameter perturbations produce observable performance changes.
The throughput PSD has a local peak at least 15 dB above the noise floor at the same frequency as the dig-cost signal.
The matched-frequency throughput oscillation demonstrates that dig-cost dither propagates through the system and provides a signal for gradient estimation.
The throughput signal also contains a strong DC component and high-frequency noise, which the high-pass and low-pass filters respectively remove or attenuate.

C. RQ2: What is the Phase Delay Between Cost and Throughput?

Throughput changes lag dig-cost changes by approximately 4 minutes, so ESC uses a 5-minute update period to reduce transient and attribution errors. Under the evaluated conditions, ESC adapts parameters and improves throughput over transferred fixed policies.

Phase delay: Approximately 4 minutes separates dig-cost changes from their measurable throughput response.Cross-correlation peaks at this time because changes propagate through planning, robot allocation, and pod movements.
Phase delay: A 5-minute ESC update period gives the system time to settle before gradient estimation.More frequent updates risk measuring transient effects or assigning throughput changes to outdated parameter values.
Nominal performance: ESC consistently outperforms fixed policies in final-simulation throughput comparisons, with improvement varying by operating regime, initialization, and system scale.Figure 4 uses the final 2 hours of each 8-hour simulation after ESC adaptation.
Parameter convergence: ESC converges toward queue-size values near ρ = 1.6 −2.0 across robot counts and initializations.The reported convergence supports automatic discovery of a narrow operating range rather than dependence on the starting value.
Initialization effects: 24-42% throughput improvement occurs when ESC starts at ρ = 1, where the initial parameter is far from optimal.Improvements are more modest near ρ = 2, while dither can temporarily reduce performance near an optimum.
Transfer performance: Across 25 test seeds, ESC improves throughput by an average of 5.0% over transferred fixed policies.The 95% confidence interval is [2.6%, 7.4%], with observed losses and gains ranging from −3.0% to +17.3%.

E. RQ4: Performance Under Dynamic Conditions

Under a 50% station reduction, adaptive ESC tuning adjusts ρ and recovers throughput, while the fixed policy maintains its parameter and suffers a decline. Across paired experiments, ESC’s advantage grows after the disturbance.

Dynamic station reduction: After 50% of stations log off, ESC increases ρ from approximately 3.3 to 4.3 while the fixed policy remains at ρ = 3.The disturbance reduces station capacity, prompting online adaptation primarily through ρ; γ does not change significantly.
Dynamic station reduction: 14.3%: ESC’s post-logout throughput advantage increases as it adapts to reduced station capacity.This result comes from five paired experiments with randomized logout times between 150 and 250 minutes.
Dynamic station reduction: 8.4%: ESC improves throughput over the full 8-hour simulation under dynamic station logouts.The averaged result compares adaptive ESC with a fixed policy across the full simulation horizon.
Dynamic station reduction: In a representative MAP2 run with 1600 robots, fixed-policy throughput drops after the station reduction, whereas ESC recovers higher throughput by increasing ρ.The 50% station reduction occurs at t = 150 minutes.

VI. CONCLUSIONS

The paper concludes that ESC provides self-tuning for dense fulfillment-center operations by adapting planner parameters to measured throughput without an analytical model. The framework is presented as extensible beyond the two parameters and planning system studied.

Conclusions: ESC eliminates manual parameter sweeps and enables real-time adaptation of multi-robot planner parameters in dense fulfillment centers.It uses sinusoidal perturbations and throughput correlations to estimate gradients despite delayed effects and credit-assignment challenges.
Conclusions: Parameter perturbations produce observable throughput changes, supporting the feasibility of gradient-based adaptation in fulfillment-center operations.The conclusion frames this as evidence from realistic FC simulations.
Conclusions: The ESC framework generalizes beyond the two planning and allocation parameters studied to other parameters or planning systems with complex, time-varying performance relationships.The proposed extension is stated as a potential application scope rather than an evaluated result.