APGEM: Adaptive Policy-Guided Error Mitigation for Quantum Reinforcement Learning on a Real-World CVRP Case Study

I. INTRODUCTION

• Noise lowers QRL fidelity and destabilizes learning, while conventional mitigation is typically selected once rather than adapted to execution conditions.The introduction notes that mitigation is rarely coupled to the reinforcement signal driving policy optimization.Source: §I · p. 1
• The paper addresses this gap with Adaptive Policy-Guided Error Mitigation (APGEM), presenting the method as its response to the gap.The Vehicle Routing Problem (VRP) is a canonical NP-hard combinatorial optimization task underpinning transportation and last-mile logistics –.Source: §I · pp. 1–2
• APGEM combines online selection among ZNE, PEC, CDR, and REM with a fidelity-, entropy-, and cost-aware utility and an adaptive learning rate.The evaluation uses a Delhi last-mile delivery scenario with authentic geographic structure and simulated NISQ noise.Source: §I · p. 2

II. RELATED WORK

• Prior quantum logistics work largely targets optimization formulations, while QRL routing studies typically do not make mitigation part of the learning loop.Existing error-mitigation methods are usually fixed, pre-selected stages; APGEM instead makes technique choice an online, reward-coupled decision.Source: §II · p. 2

A. CVRP as a Reinforcement Learning Problem

• CVRP is cast as an episodic Markov decision process whose state tracks vehicle positions, loads, route lengths, and customer visit and demand status.Actions select an unserved customer within capacity or return to the depot; rewards combine distance, visit bonuses, and terminal penalties or rewards.Source: A. CVRP as a Reinforcement Learning… · p. 2

B. Quantum Reinforcement Learning Formulation

• A variational quantum circuit maps the state to measurement probabilities, and classical post-processing maps bitstrings to feasible actions.The circuit has trainable parameters, and training uses classical optimization to update them.Source: B. Quantum Reinforcement Learning… · p. 2
• Training maximizes expected return, which is equivalent to minimizing routing cost because the reward encodes negative travel cost.The formulation describes stochastic gradient ascent for updating policy parameters.Source: B. Quantum Reinforcement Learning… · p. 2

D. Noise Modeling

• Device execution is modeled by a completely positive trace-preserving map applied to the circuit-prepared state.The passage expresses the channel in Kraus form.Source: C. Noise Modeling · p. 2
• The simulation includes depolarizing, dephasing, and amplitude-damping channels, plus readout noise represented by a confusion matrix.The passage specifies channel parameters for depolarizing, dephasing, and amplitude damping.Source: C. Noise Modeling · p. 2
• Two-qubit gate noise is applied as depolarizing noise on entangling gates.Source: C. Noise Modeling · p. 2

D. Error Mitigation Techniques

• ZNE extrapolates expectation values to the zero-noise limit by scaling effective noise, reducing bias at the cost of added variance.Richardson extrapolation is given as an example.Source: D. Error Mitigation Techniques · p. 3
• PEC decomposes target gates into quasi-probability mixtures and reweights outcomes, yielding an unbiased estimate with variance overhead that scales as Γ^2.Source: D. Error Mitigation Techniques · p. 3
• CDR regresses noisy against ideal expectation values using near-Clifford proxy circuits and benefits from richer calibration data.Source: D. Error Mitigation Techniques · p. 3
• REM corrects measurement statistics by inverting the readout confusion matrix, and no method dominates across noise regimes or training.Source: D. Error Mitigation Techniques · p. 3

• APGEM scores candidate mitigation methods using fidelity, policy entropy, and computational cost, with weights setting the trade-offs.Source: §IV · p. 3
• The selected method supplies corrected statistics for gradient estimation, so policy gradients use adaptively mitigated measurements.Source: §IV · p. 3
• Each method’s Q-score is updated from its utility, and ε-greedy selection chooses a method for the current state.Source: §IV · p. 3
• APGEM modulates the policy learning rate using fidelity and entropy diagnostics during optimization.Source: §IV · p. 3

• Quantum reliability is assessed with noisy-to-ideal state fidelity and normalized von Neumann entropy, where lower entropy indicates purer execution.Source: §V · p. 4
• Optimization is measured by cumulative reward, total routing cost, and approximation ratio, with ratios near one indicating parity with the classical baseline.Source: §V · p. 4
• Robustness is assessed through mitigation gain, mitigation-method selection frequency, and across-seed variance in reward, cost, and fidelity.Source: §V · p. 4

• Instances use a Connaught Place depot, real Delhi landmarks as customers, and geodesic great-circle distances as inter-node costs.Demands are sampled with zero depot demand, and vehicle capacity is set as a function of total demand to ensure feasibility.Source: §VI · p. 4
• AerSimulator provides a directly accessible noiseless reference, enabling exact fidelity computation at negligible cost; hardware-side fidelity estimation is left to future work.For hardware, the passage describes using fidelity proxies or entropy and estimator-variance diagnostics instead.Source: §VI · p. 4
• The simulated grid covers depolarizing, amplitude-damping, phase-damping, two-qubit-gate, and readout noise at severity levels 0.01, 0.05, 0.08, and 0.10.Source: §VI · p. 4
• Classical comparisons include Nearest Neighbor, Savings, Genetic Algorithm, Simulated Annealing, and Tabu Search, over 100- and 500-episode horizons.Source: §VI · p. 4
• The CVRP has one depot, 8 customer nodes, and 3 vehicles; its 4-qubit policy has 12 trainable parameters across three rotation layers.Parameters are initialized uniformly in [0, 2π), and the episodic discount factor is γ = 1.Source: §VI · p. 4
• Metrics are averaged over multiple random seeds, with the same seeds shared across noise families and mitigation settings for paired comparison.Source: §VI · p. 4
• The policy ansatz encodes state features with Ry rotations, entangles qubits with a controlled-Z layer, and uses stacked trainable Ry layers before measurement drives action selection.Source: Figure 2 · p. 4, §VI · p. 4

• Against Nearest Neighbor, the approximation ratio averages 0.743 with success above 92%; ratios against metaheuristics range from 0.83 to 0.91.At 500 episodes, ratios improve uniformly, including Genetic Algorithm from 0.909 to 0.915 and Tabu Search from 0.890 to 0.896.Source: §VII · p. 4, Table I · p. 5
• Under the short horizon, PEC is strongest at 0.922; under the long horizon, CDR overtakes PEC, 0.918 vs. 0.913.All techniques improve on the unmitigated baseline.Source: §VII · pp. 4–5, Table II · p. 5
• At high noise, mitigation restores approximation ratios from 0.84–0.87 to 0.92–0.94 across five noise families.Depolarizing, amplitude damping, and readout noise are mitigated most effectively; dephasing and gate noise remain partially challenging.Source: §VII · p. 5, Table III · p. 5, Figure 3 · p. 5
• Under the short horizon, APGEM favors CDR with 8,332 selections; under the long horizon, usage balances across all four techniques as success rates converge.Long-horizon selections are ZNE 24,067, PEC 24,485, CDR 19,305, and REM 20,692, indicating regime-dependent selection.Source: §VII · p. 5, Table IV · p. 5, Figure 4 · p. 5

VIII. DISCUSSION

• PEC leads under short horizons, while CDR leads under long horizons; APGEM shifts from CDR-dominated selection to balanced deployment.The discussion associates PEC’s lead with the value of unbiased correction and CDR’s with accumulated calibration data sharpening its regression.Source: §VIII · p. 5
• At the highest noise severities, mitigation separates sub-parity performance from approximation ratios near 0.94.Source: §VIII · p. 5
• The evaluation is a focused single-instance Delhi last-mile case study; cross-instance benchmarking on standard suites is future work.Source: §VIII · pp. 5–6

IX. CONCLUSION

• APGEM selects among ZNE, PEC, CDR, and REM online, restoring approximation ratios under high noise and shifting from CDR-dominated short-horizon to balanced long-horizon deployment.Source: §IX · p. 6
• The Delhi last-mile case study evaluates the controller under the spatial structure of an operational routing task.Source: §IX · p. 6
• The conclusion identifies standard CVRPLIB benchmarking, richer VRP variants, qubit-efficient encodings, and real-hardware deployment as extensions.Source: §IX · p. 6