Source-linked AI summary
Context-Aware Error Mitigation Orchestration for Hybrid Quantum Reinforcement Learning on NISQ Systems
Bisma Majid, Shabir Ahmed Sofi, Mir Mohammad Yousuf
TL;DR
Noise can degrade QRL policies and destabilize learning, while fixed mitigation choices do not adapt to changing noise or training conditions. This paper integrates APGEM into QRL training for CVRP and finds that it outperforms fixed strategies while reaching 94.2% of an expected-utility oracle. The authors frame the quantum routing component as a feasibility-and-learning demonstration, not a claim of quantum advantage.
Problem
NISQ noise degrades QRL policy quality and learning stability, while existing mitigation strategies are generally fixed despite changing noise and training conditions.
Source: §1 · p. 2Method
The framework integrates online APGEM selection among mitigation techniques into a hybrid QRL loop for CVRP, alongside a feasibility-guaranteed decoder and REINFORCE training.
Source: §1 · p. 2, §8 · p. 16Results
APGEM outperforms fixed strategies and attains 94.2% of an expected-utility oracle while significantly outperforming four of five fixed techniques.
Source: §7 · p. 11, §8 · p. 17Takeaways & Limitations
The evidence supports APGEM as an adaptive mitigation approach for more robust and stable QRL under the evaluated noise conditions.
Source: §1 · p. 2The compact policy circuit’s representational capacity is capped on the largest instances, so quantum routing is presented as a feasibility-and-learning demonstration rather than evidence of quantum advantage.
Source: §7.4 · p. 13
Abstract
from arXiv · showhide
Quantum Reinforcement Learning (QRL) integrates reinforcement learning with parameterized quantum circuits and is a promising approach to combinatorial optimization. On Noisy Intermediate-Scale Quantum (NISQ) devices, however, decoherence, gate imperfections, and measurement errors reduce policy quality and make learning less reliable. Existing error mitigation techniques are generally applied as fixed corrections that do not adapt to changing noise conditions or to the evolving state of training. This work presents Adaptive Policy-Guided Error Mitigation (APGEM) as a context-aware orchestration layer of the hybrid quantum-classical training loop that dynamically selects the most suitable mitigation strategy during QRL training. APGEM evaluates Zero-Noise Extrapolation (ZNE), Probabilistic Error Cancellation (PEC), Clifford Data Regression (CDR), and Readout Error Mitigation (REM) using policy-level indicators, including quantum-state fidelity, policy entropy, cumulative reward, and approximation ratio, and integrates the selected strategy directly into the reinforcement learning loop. The framework is evaluated on the Capacitated Vehicle Routing Problem (CVRP), a representative NP-hard problem in urban logistics, under a range of NISQ noise models and noise levels. APGEM consistently outperforms conventional static mitigation methods, reaches approximately 94% of the utility of an oracle strategy, maintains higher quantum-state fidelity as noise increases, and produces more stable learning behaviour throughout training. Ablation studies show that the framework learns context-aware mitigation policies that adapt to different noise environments and circuit execution conditions. These findings demonstrate that integrating adaptive error mitigation into the learning process substantially improves the robustness and reliability of QRL on NISQ hardware.
1. Introduction
The paper addresses the mismatch between noise-sensitive QRL and mitigation methods that are usually fixed despite changing noise and training conditions. It extends APGEM to CVRP QRL and reports that adaptive mitigation outperforms static strategies across noise settings.
1. Introduction
- NISQ noise degrades QRL policy fidelity and learning stability, while fixed ZNE, PEC, CDR, or REM choices do not adapt to changing circuits or noise.The paper identifies adaptive selection from real-time performance signals as an open challenge for QRL.Source: §1 · p. 2
- The work extends APGEM to QRL for capacitated VRP and places adaptive mitigation within a closed feedback loop combining reinforcement learning and variational quantum circuits.Its contributions include a VRP learning environment and a controller that selects among ZNE, PEC, CDR, and REM.Source: §1 · p. 2
- Across multiple noise families and levels, APGEM consistently outperforms static mitigation, approaches oracle-level performance, preserves higher fidelity, and improves learning stability.The evaluation includes depolarizing, amplitude damping, dephasing, and readout noise.Source: §1 · p. 2
2. Related Work
Prior logistics research covers quantum optimization, quantum-inspired heuristics, and end-to-end systems, but leaves noise adaptation within learning insufficiently addressed. This work positions APGEM-QRL as an adaptive mitigation framework evaluated on CVRPLIB instances.
2. Related Work
- The paper contrasts its approach with quantum-inspired classical logistics methods, which it says lack dynamic error-management mechanisms during classical execution.APGEM-QRL instead uses noisy quantum circuits and adapts mitigation in real time.Source: §2 · p. 3
- Prior quantum logistics studies generally use fixed noise assumptions or static mitigation, whereas this approach adapts noise handling to policy progress and circuit depth.The related work describes direct quantum logistics algorithms mapped to Ising or QUBO formulations.Source: §2 · p. 3
- Earlier APGEM work addressed QRL for TSP and a real-world CVRP case study using an ε-greedy selector [31]; this study adds LinUCB, a feasibility-guaranteed decoder, and statistical validation.The evaluation uses thirty CVRPLIB instances.Source: §2 · p. 3
- Prior logistics work leaves gaps in adaptive noise treatment, integration of mitigation with learning, real-world urban validation, and breadth of performance metrics; APGEM-QRL addresses these on CVRPLIB benchmarks.The paper reports approximation ratios, convergence, and robustness under varying noise conditions.Source: §2 · p. 3
3. Problem Formulation
The formulation casts CVRP routing as a learning problem, represents the policy with a variational quantum circuit, and applies adaptive mitigation to circuit execution under noise. The setup links feasible route construction and reward-based learning to noise-aware selection among mitigation methods.
3.1. Capacitated Vehicle Routing Problem (CVRP)
- CVRP minimizes fleet travel cost while satisfying customer coverage and vehicle-capacity constraints; the classical MIP is intractable at large scale and serves as a reference rather than the agent’s solver.The framework trains RL/QRL policies to approximate the same cost-minimization objective.Source: §3.1 · p. 4
3.2. Reinforcement Learning Formulation of the CVRP
- The RL formulation represents routing as sequential decisions over customers and depot return, with vehicle capacity and served-customer status restricting feasible actions.The environment updates vehicle location and load after each action.Source: §3.2 · p. 5
- The RL reward penalizes travel cost and capacity violations while rewarding valid visits, so maximizing cumulative reward aligns with minimizing routing cost.An additional terminal penalty applies for unvisited customers.Source: §3.2 · pp. 5–6
3.3. Quantum Reinforcement Learning (QRL) Formulation of the CVRP
- QRL realizes the policy with a parameterized quantum circuit, maps measurement bitstrings to feasible actions, and updates circuit parameters using parameter-shift gradients.The quantum policy retains the classical CVRP objective of maximizing expected return.Source: §3.3 · p. 6
3.4. Noise Model and Error Mitigation Techniques
- The framework executes circuits under depolarizing, dephasing, amplitude-damping, and readout noise, and considers ZNE, PEC, CDR, and REM as distinct corrections.The methods address different error sources and trade off statistical variance against systematic bias.Source: §3.4 · p. 6
3.5. Adaptive Policy-Guided Error Mitigation (APGEM)
- Mitigation effectiveness varies with noise, circuit structure, and execution conditions, so one fixed technique can become suboptimal across operating regimes.The passages describe no mitigation as often sufficient at very low noise, ZNE as advantageous at moderate noise, and CDR as potentially beneficial under severe noise.Source: §3.5 · pp. 6–7
- APGEM treats mitigation selection as an online contextual decision, monitoring learning state and execution context to select a technique and define the corrected policy.The context, cost-aware utility, LinUCB controller, and oracle reference are specified in Section 4.7.Source: §3.5 · p. 7
4. Methodology
The methodology combines a feasible CVRP environment, a variational quantum policy, noise modeling, and APGEM in a closed training loop. Mitigated measurements shape action selection, while rewards update both the quantum policy and mitigation controller.
4.1. Proposed Framework
- The framework links a hard-feasible CVRP environment, variational quantum policy, noise model, and APGEM controller in a closed loop updated by reward.The reward drives both the policy-gradient update and the online update of the mitigation controller.Source: §4.1 · p. 7, §4 · p. 7, Figure 2 · p. 7
4.2. CVRP as a Reinforcement Learning Environment
- The CVRP state tracks the current node, residual capacity, visited customers, and vehicles used; an action mask excludes capacity-infeasible customers.Shaped rewards combine progress and distance with a terminal penalty for unserved customers; feasibility-guaranteed decoding serves every customer exactly once while respecting capacity.Source: §4.2 · pp. 7–8
4.3. Quantum Policy Representation
- A VQC encodes compact state features, and its measurement outcomes are grouped into contiguous blocks that define probabilities over feasible routing actions.Feasible actions are ordered by distance, with the depot last, so lower-index outcomes correspond to nearer customers.Source: §4.3 · p. 8
4.4. Policy Optimization
- The quantum policy is trained with REINFORCE and exact parameter-shift gradients, using standardized returns and a mean baseline for variance reduction.The policy samples actions during training and is evaluated greedily.Source: §4.4 · p. 8
4.5. Noise Modeling
The methodology models multiple NISQ noise sources and chooses among mitigation strategies online. APGEM uses context features and a utility that balances fidelity against sampling cost and estimator variance.
4.5. Noise Modeling
- The noise model includes depolarizing, dephasing, amplitude damping, two-qubit gate, and readout errors, with depolarizing and readout levels varied independently across operating regimes.Readout error acts through a confusion matrix on measured probabilities.Source: §4.5 · p. 8
4.6. Error Mitigation Pool
- The mitigation pool includes no mitigation, ZNE, PEC, CDR, and REM; preferred strategies vary by noise regime, from no mitigation at very low noise to CDR at severe noise.PEC and ZNE lead at low-to-moderate depolarizing noise, while REM is preferable under readout-dominated noise.Source: §4.7 · p. 8, §4.6 · p. 8
4.7. Adaptive Policy-Guided Error Mitigation
- APGEM's utility rewards fidelity relative to a fixed usability threshold and penalizes normalized sampling cost and estimator variance.A fidelity gain near F0 = 0.80 is valued more than a gain in an already high-fidelity region.Source: §4.7 · p. 9
- APGEM uses LinUCB to select mitigation from context features including noise, fidelity and its degradation, entropy, circuit depth, shot budget, approximation ratio, and recent reward trend.Radial-basis features augment the context because the best strategy can vary non-monotonically with noise level.Source: §4.7 · pp. 8–9
- The selected strategy's mitigated distribution defines the action policy, making mitigation part of the learning loop rather than static preprocessing.After application, the observed reward updates the chosen arm only.Source: §4.7 · p. 9
4.8. Oracle Reference and Regret
The paper defines an expected-utility oracle as an upper-bound reference for mitigation selection at each context. Regret is measured against this reference, separately from the residual stochastic gap.
4.8. Oracle Reference and Regret
- At each context, the oracle selects the strategy with the highest expected utility, estimated by averaging repeated executions.This avoids treating the highest single stochastic realization as the achievable reference.Source: §4.8 · p. 9
- Regret is reported against the expected-utility oracle, with the residual stochastic gap reported separately to distinguish policy suboptimality from sampling variance.The framework uses context-dependent mitigation strategies within its feasible routing policy.Source: §4.8 · p. 9
5. Performance Metrics
The paper evaluates execution quality, optimization and learning, adaptive mitigation, and robustness using measured circuit output distributions.
5. Performance Metrics
- Performance assessment covers quantum execution quality, optimization and learning performance, adaptive mitigation effectiveness, and robustness and statistical reliability.The framework operates on measurement statistics rather than reconstructed density matrices.Source: §5 · pp. 9–10
- Metrics are computed from the measured output probability distributions of executed circuits.The approach uses measurement statistics rather than reconstructed density matrices.Source: §5 · pp. 9–10
- Table 2 summarizes the metrics and their definitions.The passage identifies the table as the location of those definitions.Source: §5 · p. 10
6. Experimental Setup
The experimental setup evaluates APGEM-QRL on standard CVRPLIB instances, using classical solver baselines, a parameterized quantum policy, varied noise conditions, and a multi-metric evaluation protocol.
6. Experimental Setup
- The study uses CVRPLIB instances spanning 15–100 customers, with published best-known solutions as approximation-ratio references.Instances are drawn from Augerat Set A and B, E, F, M, and P families.Source: §6 · p. 10
6.1. Classical Baselines
- Classical comparisons include Nearest Neighbour, Clarke–Wright Savings, Genetic Algorithm, Simulated Annealing, Tabu Search, and Google OR-Tools.Stochastic methods report median results over random seeds; capacity feasibility is enforced during route construction and perturbation.Source: §6.1 · p. 10
6.2. Quantum Policy and Circuit Design
- The quantum policy uses nq = ⌈log2(n + 1)⌉ qubits, capped for NISQ scale, with feature encoding, CZ entanglement, and trainable Ry layers.The manually specified ansatz balances expressive power against circuit depth and is simulated with Qiskit AerSimulator.Source: §6.2 · p. 10, Figure 3 · p. 10
6.3. Reinforcement Learning Environment
- CVRP episodes encode service, vehicle load, distance, and budget state; an action mask enforces capacity feasibility, while route construction and repair ensure complete feasible solutions.Mitigated measurement probabilities form the categorical policy, trained with REINFORCE and exact parameter-shift gradients.Source: §6.3 · p. 10
6.4. Noise Modeling
- Noise experiments cover five noise families at severity levels 0.01, 0.05, 0.08, and 0.10, with wider depolarizing and independent readout sweeps for adaptive mitigation.The adaptive depolarizing sweep ranges from 0.01 to 0.18.Source: §6.4 · p. 10
6.5. Adaptive Error Mitigation
- APGEM selects among NoMitigation, ZNE, PEC, CDR, and REM using a LinUCB contextual bandit informed by noise, fidelity, entropy, circuit depth, shot budget, approximation ratio, and reward trend.Its cost-aware utility rewards mitigated fidelity relative to a usability threshold while penalizing sampling cost and estimator variance; the controller updates online.Source: §6.5 · p. 11
6.6. Evaluation Protocol
- Evaluation tracks route cost, approximation ratio, feasibility, reward, distribution fidelity, entropy, and runtime across solvers, episodes, noise conditions, and mitigation techniques.Differences are tested with Friedman, Nemenyi post-hoc, and Wilcoxon signed-rank tests using instances as blocks, with bootstrap confidence intervals for stability.Source: §6.6 · p. 11
7. Results and Discussion
Across the CVRPLIB evaluation, APGEM adaptively selects mitigation strategies and approaches an expected-utility oracle, while the quantum routing policy is feasible and learns but trails classical solvers. The results also characterize mitigation overhead, noise robustness, and training behavior.
7.2. Classical Baseline Benchmark and Statistical Validation
- OR-Tools achieves mean approximation ratio 1.043 and Clarke–Wright 1.048; they are statistically indistinguishable and both outperform the metaheuristics.Figure 5 presents mean ratios with bootstrap 95% confidence intervals, and Figure 6 presents Friedman ranks with the Nemenyi critical difference.Source: Figure 5 · p. 12, Figure 6 · p. 12, §7.2 · p. 12
7.3. Feasibility and Policy Learning
- The decoded solutions are feasible in every run, and REINFORCE reduces mean route cost from 629.3 for the frozen policy to 610.8, a 2.08% improvement.The learning comparison uses P-n16-k8 over three seeds.Source: §7.3 · p. 12, Figure 7 · p. 13, Table 5 · p. 13
7.4. Quantum Routing Performance and Scalability
- On A-n32-k5, the quantum policy’s best cost is 1894 versus the best-known 784 (approximation ratio 2.42), while OR-Tools reaches 784.Across 15 to 100 customers, the approximation ratio degrades from 1.16 to 3.35; the work presents this as a feasibility-and-learning demonstration, not a classical advantage claim.Source: Figure 8 · p. 13, §7.4 · p. 13, Figure 9 · p. 14
7.5. Per-Technique Mitigation Characterization
- PEC and ZNE deliver the highest fidelity, while static sampling overhead ranges from 2× for PEC to 9× for CDR and adaptive controllers use about 3.8–4.0×.Figure 10 reports fidelity and normalized entropy; Figure 11 reports overhead relative to unmitigated execution.Source: §7.5 · p. 13, Figure 10 · p. 14, Figure 11 · p. 14
7.6. Adaptive Mitigation: Context-Dependent Selection
- APGEM’s mitigation choice varies by operating regime: no mitigation dominates at noise 0.01 (65%), PEC at 0.05 (74%), ZNE at 0.08 (73%), and CDR reaches 18% at 0.18.Source: §7.6 · pp. 13–14
7.8. Regret Analysis
- Mean expected regret is 0.0151 against an irreducible per-realization oracle noise floor of 0.0171; contextual and Bayesian controllers match fidelity at equal or lower sampling cost than epsilon-greedy.The regret decomposition attributes roughly 53% of raw regret to stochastic sampling noise.Source: §7.8 · p. 15, §7.9 · p. 15
7.10. Noise Robustness; 7.11. Training Dynamics Across Horizons
- At noise level 0.10, APGEM maintains fidelity of 0.794 versus 0.684 without mitigation; across 100, 500, and 1000 episodes, approximation ratios are 1.234, 1.215, and 1.221.The best solution improves from 1.200 to 1.162 as training length increases, while sampling cost grows from 4.7 million to 46.1 million shots.Source: §7.10 · p. 15, Table 12 · p. 15, §7.11 · pp. 15–16
7.7. APGEM versus Static Strategies
- APGEM reaches 94.2% of oracle reward (0.639), with mean cost-aware reward 0.602, exceeding every fixed technique and matching ZNE statistically.Across 60 held-out conditions, the head-to-head APGEM–ZNE Wilcoxon test gives p = 0.57.Source: §7.7 · p. 15, Table 10 · p. 15, Figure 15 · p. 16
8. Conclusion
The framework combines a variational quantum routing policy with online APGEM mitigation and was evaluated on thirty CVRPLIB instances with formal statistical testing. APGEM performs near its expected-utility oracle, while quantum routing remains a feasibility-and-learning demonstration rather than a claim of quantum advantage.
8. Conclusion
- The framework combines a variational quantum policy trained with REINFORCE and parameter-shift gradients, a feasibility-guaranteed decoder, explicit NISQ noise modeling, and online APGEM selection.Evaluation used a thirty-instance CVRPLIB benchmark with formal statistical testing.Source: §8 · p. 16
- OR-Tools and Clarke–Wright Savings are the strongest classical references, with mean approximation ratios of 1.043 and 1.048 to best-known, and both significantly outperform metaheuristics.The two strongest references are statistically indistinguishable; the classical comparison uses instance as the blocking factor across thirty instances.Source: §8 · p. 16
- Corrected regret analysis places residual policy suboptimality at 0.0151 and the irreducible stochastic noise floor at 0.0171, indicating operation close to the achievable limit for the controller's context and pool.Figure 16 separates expected regret from the noise floor and decomposes raw cumulative regret into those components.Source: Figure 16 · p. 17, §8 · p. 17
- 94.2% of the expected-utility oracle: APGEM significantly outperforms four of five fixed techniques, matches the strongest single generalist, and uses all five candidates across distinct regimes.The corrected residual policy suboptimality is 0.0151, comparable in order to the 0.0171 irreducible stochastic noise floor.Source: Figure 17 · p. 17, §8 · p. 17
- APGEM sustains higher distribution fidelity than unmitigated execution as noise grows.Figure 18 presents this noise-robustness comparison.Source: Figure 18 · p. 17
- The quantum routing policy is feasible on every instance and reduces mean P-n16-k8 route cost from 629.3 to 610.8, but its solution quality trails mature classical heuristics and worsens with instance size.Its approximation ratio rises from roughly 1.16 at 15 customers to over 3.3 at 100 customers.Source: §8 · pp. 16–17
9. Future Scope
Future work targets the policy circuit's representational limits at larger instances, validation beyond simulated noise, and improvements to controller design, benchmarks, and training.
9. Future Scope
- The dominant limitation is the compact policy circuit's representational capacity at larger instances; proposed responses include more expressive encodings, hybrid classical local-search refinement, and alternative training objectives.The suggested objectives include actor-critic and advantage-based estimators to strengthen the learning signal beyond REINFORCE.Source: §9 · p. 18
- Because results use simulated noise, the framework still needs evaluation with calibrated device-noise models and ultimately on real quantum processors to test transfer of regime-dependent mitigation behavior.Proposed models include thermal relaxation, coherent and crosstalk errors, and measured readout confusion matrices.Source: §9 · p. 18
- The residual regret suggests limited headroom for the linear contextual-bandit controller; proposed alternatives include nonlinear selectors, finer noise discretization, and a broader mitigation pool.The controller currently under-reacts in the severe-noise regime.Source: §9 · p. 18
- Evaluation could expand to richer routing formulations and benchmark families, with more independent seeds per instance to strengthen statistical power.Suggested routing settings include time windows, pickup-and-delivery, and multi-depot problems; proposed benchmark expansion includes the B, E, F, M, P, and X families and larger Xseries instances.Source: §9 · p. 18
- The routing policy and mitigation controller are trained as coupled but distinct components; future work proposes joint co-adaptation and regret guarantees for the cost-aware contextual bandit.The proposed theoretical analysis concerns the controller's empirical near-oracle behavior.Source: §9 · p. 18
Funding
The paper reports that it did not receive any specific grant from funding agencies.
Funding
- The research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.Source: Funding · p. 18