PaperlayerView exact revision on arXiv

Source-linked AI summary

A Dynamic Inventory Rationing Model for Last-Mile Delivery of Medication in Liberia's Community Health Program

Baris Ata, Yufan Chen, Robert Montgomery

arXiv:2610.32051v1math.OCcs.LGcs.SYeess.SYmath.AP

TL;DR

The paper asks how a health facility and its CHAs should share scarce medication when stockouts can trigger costly referrals and allocation fairness matters. It approximates the dynamic allocation problem with reflected Brownian motion and deep splitting, then tests the policy against benchmarks. The policy lowers operating costs by about 6% versus the fairness-only benchmark while retaining approximately 98% of its fairness utility.

  • Problem

    How should a health facility and its CHAs share scarce medication when local stockouts can lead to referrals and untreated patients?

    Source: §1 · p. 2

  • Method

    The paper formulates a finite-horizon stochastic control problem and adapts deep splitting to a continuous-time reflected Brownian approximation.

    Source: §1 · p. 3, §2 · p. 7, §3 · p. 13

  • Results

    About 6% lower operating costs than the fairness-only policy, the strongest benchmark, while retaining approximately 98% of its fairness utility.

    Source: §7 · p. 29, §8 · p. 36

  • Takeaways & Limitations

    Dynamic allocation can improve scarce medication use by accounting for current inventory, uncertain demand, referral costs, and fairness.

    Source: §8 · p. 36

    Demand is approximated by a Gaussian process, and the initial model sets off-diagonal covariance terms to zero.

    Source: §3 · p. 13, §6.1 · p. 25

Abstract

from arXiv · show

We study the dynamic allocation of scarce medications in Liberia's community health program. A central warehouse supplies a health facility and the Community Health Assistants (CHAs) it supervises. When a CHA runs out of medication, patients are referred to the facility, incurring travel costs if they complete the referral and remain untreated otherwise. Allocating inventory to CHAs allows more patients to receive treatment without traveling, while retaining inventory centrally preserves flexibility to respond to uncertain demand. The allocation problem must balance these considerations while accounting for fairness across locations. We formulate this problem as a finite-horizon stochastic control problem whose objective combines the costs of referrals and untreated patients with a demand-weighted fairness utility. Because standard dynamic programming methods are computationally intractable in this setting, we develop a continuous-time approximation that involves controlling the drift rate of a reflected Brownian motion. We then adapt the deep splitting method, using deep neural networks to approximate the value function and its gradient and compute a dynamic allocation policy. We calibrate the model using data on malaria cases among children in the Fangonda region of Lofa County, Liberia, and evaluate the proposed policy in the original discrete-time model. In the baseline setting, our policy reduces operating costs by about 6% relative to the strongest benchmark, the fairness-only policy, while retaining approximately 98% of its fairness utility. Additional experiments show that higher referral costs shift inventory toward CHAs and reduce local unmet demand, while greater demand variability increases our policy's performance over the fairness-only benchmark. Our model and computational approach provide a framework for studying similar last-mile allocation problems.

1 Introduction

The paper frames medication allocation between a health facility and its CHAs as a dynamic rationing problem: more local stock can prevent stockout-driven referrals, but central reserves preserve flexibility and demand differences matter. It develops a Brownian-control approximation and deep-learning policy, reporting lower operating costs than the strongest benchmark while retaining most of its fairness utility.

1 Introduction

  • The policy dynamically allocates scarce medication by balancing central inventory flexibility against local treatment access and differences in CHA demand.The model combines referral and untreated-patient costs with a demand-weighted α-fair utility.Source: §1 · pp. 2–3
  • Standard dynamic programming is computationally intractable, so the paper approximates the problem as drift control of a reflected Brownian motion and uses deep neural networks to estimate value and gradient.The approach adapts deep splitting to compute a dynamic allocation policy.Source: §1 · p. 3
  • About 6% lower operating costs than the fairness-only policy, the strongest benchmark, while retaining approximately 98% of its fairness utility.The policy is calibrated with Fangonda-region data and evaluated in the original discrete-time model.Source: §1 · p. 3
  • Higher referral costs shift inventory toward CHAs and reduce local unmet demand, while greater demand variability increases the policy’s performance over the fairness-only benchmark.The experiments also vary fairness weight, spatial demand correlation, HF demand, and initial inventory.Source: §1 · p. 3
  • The study’s broader conclusion is that dynamic allocation can improve scarce-inventory use by accounting for inventory states, uncertain demand, referral costs, and fairness.The model and computational approach are offered as a framework for similar last-mile allocation problems.Source: §1 · p. 4

2 Literature Review

The paper situates its model at the intersection of inventory rationing, healthcare logistics, fairness, and stochastic control. Its distinct focus is stockout-induced patient referrals, addressed through a continuous-time Brownian approximation and an adapted deep-splitting method.

Inventory rationing and distribution systems

  • The paper frames the allocation question as sharing limited medication between a health facility and its community health assistants.Central stock preserves flexibility, while decentralized stock brings treatment closer to patients.Source: §2 · p. 4
  • Unlike conventional rationing, shortages here move some patients to the facility, linking CHA availability to facility demand and patients’ travel burdens.Referred patients may reach the facility or remain untreated, and successful referrals still depend on facility stock.Source: §2 · p. 5

Fairness

  • The objective adds demand-weighted α-fair allocation utility to stockout-related costs.The paper situates this choice alongside fairness formulations and healthcare allocation applications.Source: §2 · pp. 5–6

Stochastic control

  • The discrete-time model is computationally intractable, prompting a moment-matched Gaussian demand approximation and continuous-time controlled reflected Brownian motion.The authors describe this construction as a demand-based approximation, not a heavy-traffic limit theorem.Source: §2 · p. 7

Computational method

  • The deep-splitting adaptation fits current value and allocation-relevant marginal values in successive regression problems, rather than jointly training across all time steps.The method is applied to a nine-dimensional Lofa County control problem and presented as a proof of concept.Source: §2 · p. 7

Literature positioning

  • The contribution combines dynamic medication rationing, stockout-induced referrals, unequal access costs, and explicit fairness preferences under limited supply.The model examines how central reserve retention and local treatment access should be balanced over time.Source: §2 · p. 8

3 Model

The model allocates a finite warehouse supply over time among one health facility and multiple CHAs, whose inventory serves local patients and, at the facility, successful referrals. It represents uncertain demand and unmet need explicitly, combining stockout costs with demand-weighted fairness in a stochastic control objective and a continuous-time approximation.

System structure

  • One warehouse supplies a health facility and K CHAs over a finite horizon without replenishment; the facility treats both local patients and referred CHA patients who reach it.The CHAs serve separate regions, while the health facility is indexed as location 0.Source: §3 · p. 8

Demand and assumptions

  • Location-period demand is modeled as independent binomial demand, with populations represented by the HF catchment and equally sized CHA assignments.The model assumes each CHA serves roughly the same number of families and uses a malaria-treatment probability for each individual.Source: §3 · p. 8

Allocation decisions

  • At each period’s start, the manager observes the state and allocates available stock before demand occurs; CHA shipment rates are bounded by supervisor carrying capacity.The total allocated inventory cannot exceed the warehouse balance.Source: §3 · pp. 9–10

Patient flows and inventory dynamics

  • When CHA demand exceeds local stock, unmet demand is tracked as cumulative referrals, while the HF faces local demand plus successful referrals.The model approximates successful referrals using their expected number.Source: §3 · p. 11

Costs and fairness

  • A CHA stockout incurs referral travel cost for patients who complete referral and untreated-patient cost for those who do not; facility stockouts also leave patients untreated.A referred patient who reaches the HF but cannot be treated incurs both referral and untreated costs.Source: §3 · pp. 9, 11
  • The objective combines stockout costs with demand-weighted α-fair utility, with larger α penalizing allocation imbalances more strongly.The model makes allocation dynamically over time and weights fairness utility by λ.Source: §3 · p. 12

Continuous-time approximation

  • Because standard MDP methods are computationally prohibitive, the paper permits continuous allocation and approximates cumulative demand with a Gaussian process.The approximation matches the mean and variance of each Brownian demand increment to discrete binomial demand over a review period.Source: §3 · pp. 12–13

4 HJB Equation and the Proposed Policy

The paper characterizes optimal allocation through an HJB equation and derives allocation rates from marginal value. It then uses neural-network approximations of those marginal values to implement a discrete-period policy subject to inventory and shipment constraints.

HJB equation

  • The HJB equation characterizes the value function, and its minimization term yields the optimal allocation-rate policy.The control is subject to boundary and terminal conditions, including zero cost assigned to remaining inventory at the horizon.Source: §4 · p. 16

Proposed policy

  • The allocation-rate rule uses the marginal opportunity cost of moving one unit from central inventory to a location.With no fairness weight, the policy is bang-bang; with positive fairness weight, low allocation levels can justify allocation despite positive marginal opportunity cost.Source: §4 · p. 17
  • The computational method approximates the value gradient on a refined time grid and substitutes it into the allocation-rate rule.The resulting period allocation aggregates decisions from a deterministic virtual system, then scales tentative shipments if they exceed available warehouse inventory.Source: §4 · pp. 18–19

5 Computational Method

The method trains a dynamic allocation policy by approximating the value function and its relevant gradient using simulated state paths. It uses a continuous-time approximation and solves backward across time intervals.

5 Computational Method

  • Reference-policy paths sample states the system may occupy under an optimal policy, providing training data for the computational method.At each interval, states are sampled from the reference-path state set and advanced with an Euler–Maruyama discretization.Source: §5 · pp. 19–20
  • The method evaluates a stochastic identity along reference processes and uses it to construct the interval-wise approximation.The identity is based on the HJB equation and includes the reference policy and auxiliary function F.Source: §5 · pp. 19–20
  • Deep splitting linearizes the nonlinear term over successive time intervals, yielding a Feynman–Kac representation to approximate the original PDE solution.Training proceeds backward from the terminal condition, minimizing empirical losses over sampled state pairs.Source: §5 · pp. 21–23
  • Neural networks approximate the value function V and the gradient difference DV, rather than V and its full gradient.The method uses separate networks for the value function and DV at each time step.Source: §5 · pp. 19–21

6 Data, Test Problems and Benchmark Policies

The paper calibrates a malaria-demand test problem for the Fangonda cluster, then compares a dynamic allocation policy with five benchmark policies. Its test setting uses a 22-week planning horizon and explicitly models resource scarcity.

6 Data, Test Problems and Benchmark Policies

  • The calibration uses Fangonda data supplemented with secondary sources, with the analysis focused on malaria cases among children under five.The stated aim is to represent cost tradeoffs in Fangonda and similar settings while acknowledging data limitations.Source: §6 · p. 23

6.1 Data Description and Test Problem

  • Demand is calibrated from CHA case records over a 22-week horizon, with exposure sizes of 28 households per CHA and 219 for the health facility.The 22 weeks span the observed active period from early February to mid-July; demand incidence is pooled across CHA observations.Source: §6.1 · p. 24
  • The test problem begins with warehouse inventory of 142 against expected demand of 109.5 health-facility cases and 98 CHA cases, covering about 68.4% of demand.Referral compliance is set to 0.69, and costs are normalized in units of one untreated under-five malaria patient.Source: §6.1 · pp. 25–26

6.2 Benchmark Policies

  • The 80/20 allocation assigns 80% of initial inventory to the HF and divides the remaining 20% equally among CHAs; subsequent shipments fulfill these fixed quotas at the maximum rate permitted per review period.The static policy follows mean demand and ignores realized inventory states.Source: §6.2 · pp. 27–28
  • When shipment bounds do not bind, the fairness-only policy gives every location the same fraction of expected demand in each period, maximizing fairness utility under those conditions.The fluid-optimal policy instead replaces uncertain demand with its mean and produces an open-loop shipment schedule.Source: §6.2 · p. 28

7 Computational Results

In the baseline comparison, the dynamic policy reduces operating costs while retaining nearly all fairness utility of the strongest benchmark. Experiments show how its allocations and cost–fairness trade-offs respond to fairness weight, travel costs, demand variability and correlation, HF demand, and initial inventory.

Baseline policy comparison

  • About 6% lower operating costs and approximately 98% of the fairness utility of the fairness-only policy, the strongest benchmark.The static policy has slightly lower operating costs but substantially lower fairness utility.Source: §7 · p. 29, Table 2 · p. 29

Baseline shipments and inventory

  • The policy releases 45.0% of initial warehouse inventory by week 8, ahead of the week 9 demand peak, and leaves a mean 2.34 units at CHAs at week 22.Aggregate warehouse shipments vary less across demand paths than inventories at individual locations.Source: §7 · p. 29, Figure 3 · p. 30, Figure 4 · p. 31

Fairness weight

  • Increasing λ from 0.25 to 0.5 yields a substantial increase in fairness utility for only a slight increase in operational costs.The results suggest λ = 0.5 may offer the best balance.Source: §7 · p. 32, Figure 5 · p. 32

Travel cost

  • Higher travel costs lead to larger CHA inventory allocations, reducing unmet local CHA demand, referrals, and both the frequency and magnitude of local mismatches.Overall inventory allocation over time does not increase because higher CHA allocations are accompanied by lower HF allocations.Source: §7 · p. 32, Table 3 · p. 32

Demand variability

  • As demand variance increases, operating costs rise and demand coverage declines, but the policy’s improvement over the fairness-only benchmark also increases.Mismatches occur less frequently but their magnitudes increase substantially.Source: §7 · p. 33, Table 4 · p. 33

Spatial demand correlation

  • Across the tested spatial correlations, the policy outperforms the benchmarks; as correlation rises, the CHA share of allocations increases and mismatch frequency and magnitude decline.Source: §7 · pp. 33–34, Table 5 · p. 33

HF demand

  • As HF demand increases, the policy shifts inventory toward the HF, while coverage and facility-specific service measures decline; its improvement over fairness-only also decreases.Source: §7 · p. 34, Table 6 · p. 34

Initial inventory

  • Increasing initial inventory improves service and shifts the policy’s relative performance against fairness-only from unfavorable at low inventory to increasingly favorable at higher levels.Additional inventory is allocated mainly to the HF; at low inventory, the policy commits a larger fraction of available stock early and depletes it sooner.Source: §7 · pp. 34–35, Table 7 · p. 34

8 Conclusion

The paper formulates medication allocation between a health facility and its CHAs as a finite-horizon stochastic control problem balancing uncertain demand, referrals, untreated patients, and geographical fairness. It develops a continuous-time reflected Brownian motion approximation and deep-splitting solution, then evaluates the policy in Liberia’s discrete-review system.

8 Conclusion

  • The deep-splitting approach approximates the value function and marginal values in a reflected Brownian motion model to determine dynamic allocations.It addresses the computational intractability of standard dynamic programming for this problem.Source: §8 · p. 36
  • About 6% lower operating costs while retaining approximately 98% of the fairness-only policy’s fairness utility, the strongest benchmark, in the discrete-review evaluation.The model is calibrated with Fangonda region data from Lofa County, Liberia.Source: §8 · p. 36
  • Higher referral costs shift inventory toward CHAs and reduce local unmet demand and supply–demand mismatches, while greater demand variability increases performance over the fairness-only benchmark.Increasing fairness weight raises fairness utility at the expense of somewhat higher operating costs.Source: §8 · p. 36
  • The results suggest dynamic allocation can improve scarce-medication use by accounting for inventory states, uncertain demand, referral costs, and fairness across locations.The model and computational approach offer a framework for similar last-mile allocation problems.Source: §8 · p. 36

A Details of Benchmark Policies

The appendix describes two benchmark policies: a fairness-only allocation based on expected demand shares and a fluid-optimal policy that replaces random demand with its mean. The latter solves a deterministic finite-horizon allocation problem, represented as a linear or constrained quadratic program.

A.1 Fairness-only Policy

  • When initial stock is below mean total demand and shipment bounds are nonbinding, the fairness-only policy allocates the same fraction of each location’s expected demand in every period.That fraction is initial stock divided by mean total demand.Source: §A.1 · p. 43

A.2 Fluid-optimal Policy

  • The fluid-optimal policy replaces random demand with calibrated means and optimizes allocations subject to warehouse and local inventory balances and unmet-demand terms.Auxiliary variables express these balances and terms linearly.Source: §A.2 · pp. 43–44
  • With fairness weight λ = 0, the fluid-optimal formulation is a linear program; with λ > 0, it is equivalent to a constrained quadratic program.The implementation solves both cases using Gurobi.Source: §A.2 · p. 44

B Evaluation Metrics

The appendix defines evaluation metrics computed over simulated review-period paths and averaged across paths. It measures service, objective and cost components, shipment allocation, inventory use, and cross-CHA stock–unmet-demand mismatches.

B Evaluation Metrics

  • 10,000 demand paths are used to evaluate the policy and benchmarks in each setting, with Monte Carlo means and standard errors reported for pathwise metrics.Metrics are calculated over review periods and then averaged across paths.Source: B Evaluation Metrics · pp. 44–45
  • The pathwise objective is decomposed into operational cost and fairness utility, with operational cost split into untreated and referral costs.The objective is also compared with that of the fairness-only policy.Source: B Evaluation Metrics · p. 45
  • Service measures distinguish overall treatment, CHA-origin demand treated locally before referral, and HF load treated at the facility, including referrals.A zero pathwise demand denominator is set to zero before averaging.Source: B Evaluation Metrics · pp. 45–46
  • Allocation metrics track total shipments, destination shares, and cumulative inventory released over time; utilization measures the fraction of initial stock consumed.The shipment shares describe the facility–CHA split, while cumulative release does not identify the destination.Source: B Evaluation Metrics · p. 46
  • Mismatch severity pairs CHA stock with local unmet demand across locations; it represents hypothetical same-week pairing, not feasible transfers or avoidable untreated demand.The appendix reports mismatch frequency and mean severity across path-weeks, plus severity conditional on a mismatch.Source: B Evaluation Metrics · pp. 46–47

C Derivation of the Deep Splitting Algorithm

The derivation turns the HJB equation into a regression target for the deep-splitting algorithm. It partitions time, approximates the equation over short intervals, and uses reflected Brownian motion dynamics and conditional expectations to obtain the update used by the algorithm.

C Derivation of the Deep Splitting Algorithm

  • The derivation freezes the nonlinear term on each fine interval and conditions on the reference state to obtain the regression target.This is the stated purpose of the appendix derivation.Source: C Derivation of the Deep Splitting… · p. 47
  • A sufficiently fine time partition gives a linear approximation of the HJB equation on each interval, leading to the displayed differential relation.The derivation assumes a sufficiently regular value function and suitable integrability conditions.Source: C Derivation of the Deep Splitting… · pp. 47–49
  • Applying Itô’s formula to the reflected Brownian motion, then integrating and taking conditional expectations, yields the regression relation identified as Equation (5.5).The boundary condition contributes the reflection term in the derivation.Source: C Derivation of the Deep Splitting… · pp. 48–49

D Implementation Details

The appendix records the implementation used for the Section 7 computational results. The baseline uses a fine time grid and reference-path state pools to train the policy.

D.1 Computational Configuration

  • The baseline uses 1,100 computational intervals over 22 weeks, with weekly calibrated demand inputs and costs measured in untreated-patient units.The model has K = 7 CHAs and initial warehouse inventory Q0 = 142.Source: §D.1 · p. 50
  • Separate feed-forward networks approximate the value function and its gradient difference at each fine time interval.For Fangonda, both networks use 9-dimensional inputs and four 512-neuron hidden layers; the value network has scalar output and the gradient-difference network has 8-dimensional output.Source: §D.1 · p. 50

D.2 Reference-Path State Sampling

  • The 32,768-path training pool combines paths from fairness-only, static, and fluid-optimal policies to cover states the proposed policy may visit.The pool includes 16,384 fairness-only paths and 8,192 paths from each of the other two policies.Source: §D.2 · pp. 50–51
  • Training samples complete states reached by reference paths, then advance the reflected Brownian process using a feasible allocation rate and a fresh Brownian increment.Initial states mix the physical starting state with perturbed states; state components are not sampled independently.Source: §D.2 · pp. 50–51

D.3 Two-stage Backward Training

The implementation trains value and gradient-difference networks backward on coarse and fine grids, with coarse networks initializing fine-grid blocks. It calibrates demand inputs and evaluates policies in the original discrete-time model.

D.3 Two-stage Backward Training

  • A 110-interval coarse model initializes blocks of ten intervals on the 1,100-interval fine grid, and training proceeds backward on both grids.Each preceding network pair uses the trained pair at the next interval; fine-grid pairs are warm-started from their trained neighbor within each block.Source: §D.3 · pp. 52–53

D.4 Demand Generation

  • Demand-variability experiments increase CHA demand variance by multipliers of 2 or 5 while preserving its mean and leaving health-facility demand unchanged.Shipment bounds and demand means remain unchanged.Source: §D.4 · pp. 53–54
  • The demand construction preserves baseline binomial marginals, but matrix adjustment can shift target correlations and the Brownian covariance generally differs from binomial-demand covariance.Pairwise correlations are calibrated using a Gaussian copula and adjusted to produce a positive-definite correlation matrix.Source: §D.4 · pp. 55–56

D.5 Policy Evaluation

  • Policies are evaluated in the original discrete-time model on 10,000 common demand paths, with trained networks held fixed.Reported tables give Monte Carlo means; cost units equal one untreated under-five malaria patient.Source: §D.5 · p. 56
Loading 2610.32051v1…