PaperlayerView exact revision on arXiv

Source-linked AI summary

QEMScore: How Much Does the Measurement Add to Learned Quantum Error Mitigation?

Yue Zhao, Huayue Gu, Yushun Dong, Xiyang Hu

arXiv:2609.17896v1quant-phcs.LG

TL;DR

Accuracy alone cannot show how much learned quantum error mitigation benefits from measurement inputs, because circuit descriptors can also predict ideal values. QEMScore compares a mitigator with a capacity-matched descriptor-only control and analyzes both simulated circuits and released Q-LEAR and QRAFT data. The control reproduces most of the mitigator’s gain over an affine fit in the primary campaign, while released-data analyses find predictive gains from measurement inputs.

  • Problem

    An accuracy table does not isolate how much a learned mitigator benefits from noisy measurements when circuit structure can also predict ideal values.

    Source: §1 · p. 1

  • Method

    QEMScore compares a learned mitigator using descriptors and measurements with a capacity-matched descriptor-only control, using exact simulated targets and released-data reanalyses.

    Source: §1 · p. 2, §6 · p. 12

  • Results

    In all six primary evaluations, descriptor-only controls reproduced 87.7 to 100.5 percent of the gain over an affine control, while measurement-derived inputs carried predictive gains in released Q-LEAR and QRAFT data.

    Source: §1 · p. 2, §6 · p. 12

  • Takeaways & Limitations

    The paper recommends reporting the full learner alongside descriptor-only controls, same-unit errors, uncertainty, circuit overlap, and each method’s measurement spend.

    Source: §6 · p. 12

    The comparisons are representation- and protocol-specific and do not isolate a cause of differences between the controlled campaign and released-data analyses.

    Source: §5 · p. 12

Abstract

from arXiv · show

How much does the noisy measurement add to learned quantum error mitigation? An accuracy table cannot say, because a model handed circuit structure can score well without reading the measurement at all. QEMScore adds the comparison that can. Each simulated circuit carries an exact ideal answer. The learned mitigator is scored beside a capacity-matched control, a model just as flexible that reads the same circuit description but never the measurement. Each method's measurement spend is accounted and not equalized. We run a controlled campaign on simulated circuits and reanalyze two published learned mitigators, Q-LEAR and QRAFT, from their released hardware data. Three findings stand out. First, under familiar within-family conditions (S0) evaluated across two spin-chain families and three seeds, continuous couplings identify the target, and the control that never reads the measurement matches 87.7 to 100.5 percent of the mitigator's gain over an affine fit to the circuit description. A plain polynomial in the coupling parameters, fitted after the campaign, beats the mitigator on all six evaluations, reflecting the selected learners' capacity. Second, for these selected learners, matching most of the gain is not matching the accuracy: on five of six evaluations the mitigator removes 19.5 to 74.5 percent of the error the capacity-matched control leaves, a learner-specific gap rather than a measurement requirement. Third, on released hardware data where descriptors only partially identify queries, the findings differ: flexible models of the descriptors show negligible mean gain over affine fits in Q-LEAR, and measurement inputs carry predictive gains in both Q-LEAR and QRAFT. These comparisons reflect representation- and protocol-specific behavior rather than an isolated cross-regime difference. A learned mitigator's accuracy should therefore be reported beside such controls.

1 Introduction

QEMScore tests what measurement inputs add by comparing a learned mitigator with an equally flexible descriptor-only control. In the primary campaign, that control reproduces nearly all of the mitigator’s gain over an affine fit, while released-data reanalyses show predictive gains from measurement inputs in both Q-LEAR and QRAFT.

1 Introduction

  • QEMScore compares a learned method reading circuit descriptors and a noisy estimate with a capacity-matched control reading descriptors alone, scoring both against exact ideal values.The accounting records each method’s measurement spend at its own declared total without equalizing cost.Source: §1 · pp. 1–2, Figure 1 · p. 2

What QEMScore Contains

  • The completed campaign covers twelve exact-label datasets in S0: two spin-chain families, one noise family, two strengths, and three seeds.The artifact specifies five circuit families, six noise families, six settings, and eight registered methods; the other circuit families, noise families, and changed settings remain future work.Source: §1 · p. 2

Contributions: Three Questions and Their Answers

  • 87.7 to 100.5 percent of the gain over an affine feature-only control was reproduced by the capacity-matched descriptor-only control in all six primary evaluations.The full method reduced the control’s remaining error by 19.5 to 74.5 percent on five rows; on the sixth, the reduction was −12.6 percent and the gap’s interval spanned zero.Source: §1 · p. 2
  • Measurement-derived inputs carried predictive gains in both released Q-LEAR and QRAFT data, while only Q-LEAR allowed a capacity contrast.Under Q-LEAR’s released protocol, the nonlinear descriptor control’s ten-fit mean hardware error reduction over the affine control was +0.000055; QRAFT’s descriptor-only arm was about six times worse than the arm adding forward observed probability.Source: §1 · p. 2
  • On hardware, base execution added 1,024 evaluations for a 0.1969 gap; depth cut added 3,072 for 0.0222, a 26.61 ratio.The design had no shot-count sweep or reallocating arm, so it does not establish what one further evaluation would buy.Source: §1 · p. 3
  • A secondary study supplements the three main questions with three screened systems: ML-QEM, Synergy CNN, and Q-Cluster.The study is reported in Section 4.7 and Appendix O.Source: §1 · p. 3

Scope

  • The scope excludes Clifford data regression and its variable-noise variant, and the restricted-feature arm is not a reproduction of Liao et al. (2024).There is no observable-shift setting, and the four strength levels are not calibrated across noise families.Source: §1 · p. 3
  • The controlled conclusions are conditional on tested simulator regimes and budgets; field conclusions are conditional on released archived data and protocols.The paper reports no routing result, quantum advantage, or hardware speedup.Source: §1 · p. 3

2 Background and Related Work

Learned mitigation maps noisy estimates and circuit or observable features to ideal-value predictions. QEMScore is positioned as a complementary effort focused on what an evaluation score can establish, rather than on standardizing training and scoring data.

2.1 From Noisy Measurements to Learned Mitigation

  • Learned mitigation trains a supervised model to predict ideal expectation values from noisy measurements and circuit features.The paper situates it alongside zero-noise extrapolation, Clifford data regression, and variable-noise CDR, citing Strikis et al. (2021) and Liao et al. (2024) for learned mitigation.Source: §2.1 · p. 3
  • The paper registers ZNE but does not evaluate CDR or vnCDR, whose target-specific training-circuit construction rule it leaves unsettled.Mitiq is used only for a continuous-integration cross-check of the benchmark’s ZNE folding and Richardson extrapolation; Table 15 reports untouched-test ZNE errors.Source: §2.1 · p. 3

2.2 Positioning and Controlled Shift

  • QEMScore complements Bao et al. (2025): that work standardizes what learned mitigators are trained and scored on, while QEMScore standardizes what a score is allowed to mean.The paper relates its controlled evaluation pattern to WILDS, ADBench, and Bowles et al. (2024).Source: §2.2 · p. 3

3 QEMScore: Benchmark Design

QEMScore defines a benchmark around circuit-level evaluation items, exact simulated targets, and comparisons between learned methods and feature controls. Its completed campaign evaluates the familiar setting only, with results scored against exact labels and measurement costs accounted separately at each method’s declared total.

3.1 Task Definition and Testbed Construction

  • Each testbed item joins a circuit, observable, noise configuration, and shot count; noisy execution and noiseless simulation use the same sampled circuit to produce r and exact target y.The two paths converge into one verified item.Source: §3 · p. 3, §3.1 · p. 4

3 QEMScore: Benchmark Design

  • The artifact defines six evaluation settings, but the completed campaign evaluates only familiar setting S0; the five changed settings remain future work.Table 1 frames prior efforts as differing across labels, declared shifts, and feature controls rather than along one axis.Source: §3 · p. 4, Table 1 · p. 4

3.2 Benchmark Composition: Circuit Families and Noise Models

  • The five circuit families span structured to unstructured workloads, but the campaign evaluates only transverse-field Ising and Heisenberg chains under depolarizing plus readout noise.The campaign uses two strengths; the four strength levels are not calibrated across noise families, so an unseen-noise setting changes channel and effective severity together.Source: §3.2 · pp. 4–5

3.3 Artifact Capabilities: Six Evaluation Settings

  • Table 2 defines six conditional stress tests, but the campaign runs S0 alone and has no observable-transfer setting.The generator refuses unseen-observable splits because the random-forest candidate fits a separate forest per physical observable and requires prediction observables to appear in training.Source: Table 2 · p. 5, §3.3 · p. 5

3.4 Methods, Budgets, and Scoring

  • The registered roster includes raw estimates, ZNE, two learned arms, and four surrogate controls; the restricted-feature arm is not a reproduction of Liao et al. (2024).The feature vector has thirty columns and omits native-gate counts, angle-bin histograms, and sparse Pauli encoding.Source: §3.4 · p. 5
  • CDR and vnCDR are absent from the executable method roster, although the release includes a closed-form cost model and training-circuit viability probe.No executable mitigator is provided for either method.Source: §3.4 · p. 5
  • Budgets are counted in circuit evaluations, and each method’s spend is recorded at its own declared total rather than equalized to a shared budget.The campaign uses the highest feasibility tier, 250,000,000 evaluations; the audit does not independently verify declared per-arm totals as executed counts and reports no empirical campaign cost comparison.Source: §3.4 · pp. 5–6
  • Scoring uses macro mean absolute error against exact labels, averaging within evaluation cells and then weighting cells equally.Intervals bootstrap whole physical circuits, keeping observables that share a circuit together; the campaign reports no harm endpoint.Source: §3.4 · p. 6

4 Experiments

The controlled campaign finds that a capacity-matched model without measurements reproduces most of the restricted-feature ablation’s gain, while released-data reanalyses show measurement-associated predictive gains in Q-LEAR and QRAFT. These results are bounded by the selected learners, data, and evaluation protocols.

4 Experiments

  • The six primary evaluations cover two spin-chain families across three seeds, at shipped step sizes and 640 training circuits per family.The campaign subset uses depolarizing-plus-readout noise, two strengths, and middle-site and middle-pair observables.Source: §4 · p. 6

Which learned arms pass the source-validation gate?

  • The restricted-feature ablation passes the source-validation gate on all twelve campaign datasets, while ridge passes only on the six large-step datasets.At shipped step sizes, affine-control error is 1.0004 to 1.0145 times ridge’s, within the 1.05 margin.Source: Which learned arms pass the… · p. 6

4.2 RQ1: How Much Improvement Can Circuit Features Alone Reproduce?

  • 87.7 to 100.5 percent of the ablation’s improvement over the affine control is reproduced in all six primary evaluations; on five rows the ablation reduces the control’s remaining error by 19.5 to 74.5 percent.On seed 101 Heisenberg, the reduction is −12.6 percent and the gap interval spans zero.Source: §4.2 · pp. 7–8, Figure 3 · p. 8

4.4 RQ2 and RQ3: Measurement Gains and Costs in Q-LEAR

  • On Q-LEAR hardware, descriptor-model capacity adds +0.000055 over the affine control, while measurement inputs improve the full method over the nonlinear descriptor control by 0.2191.Every per-fit interval for the capacity contrast spans zero; the measurement-associated gap’s intervals lie above zero on all ten fits.Source: §4.4 · p. 9, Figure 4 · p. 10

RQ3: what collection requirements accompany these ordered gains?

  • Q-LEAR’s base-statistics step opens 0.1969 of the hardware gap and its depth-cut step 0.0222; the steps add 1,024 and 3,072 shot-level evaluations, respectively.The depth-cut interval is above zero on nine of ten hardware fits and three of ten simulator fits.Source: RQ3: what collection requirements… · p. 9, Figure 4 · p. 10

4.5 RQ2: How Much Do Measurements Improve QRAFT?

  • QRAFT descriptors alone score 12.4838 against 2.1074 with forward observed probability in released predictions, and 13.5903 against 2.2685 under matched retraining.Adding reverse-execution statistics and machine identity improves matched retraining by about six percent but worsens released-prediction error.Source: §4.5 · pp. 9–10

4.6 Scope of the Q-LEAR and QRAFT Findings

  • The Q-LEAR and QRAFT reanalyses do not establish prevalence: they use two feature designs from one release, and QRAFT has no affine arm or reproduced-share estimate.Both reanalyses split rows rather than circuits, so they do not assess unseen application circuits.Source: §4.3 · p. 9, §4.6 · p. 10

What does a descriptor control’s reproduced share reveal in screened published systems?

  • Screened systems show mixed descriptor-control patterns: ML-QEM reads present, Synergy CNN absent, and Q-Cluster’s present share comes from intermediate regression rather than final mitigation accuracy.Synergy CNN’s controls use reduced data, so that reading does not hold at the published budget.Source: What does a descriptor control’s… · pp. 11–12

5 Discussion and Limitations

The discussion distinguishes controlled simulator findings from released-data comparisons and limits both to their tested representations, protocols, and budgets. Descriptor-only controls diagnose the value of measurement inputs; they are not execution-free estimators.

5 Discussion and Limitations

  • Released field descriptors only partially identify queries, and the Q-LEAR and QRAFT comparisons remain specific to their archives and protocols.The two case studies do not isolate a cause of cross-regime differences.Source: §5 · p. 12
  • In the controlled campaign, descriptor-only controls learn from exact simulated targets and do not supply targets for arbitrary circuits.In the released archives, descriptor-only arms still condition on output support obtained by executing the circuit.Source: §5 · p. 12
  • Conclusions are conditional on the tested simulator regimes, circuit families, budgets, eight IBM backends, simulator companion, and released protocols.Each screened-system reading also depends on its released material, evaluation unit, and split.Source: §5 · p. 12
  • The roster omits executable Clifford data regression comparators, native-gate counts, angle-bin histograms, sparse Pauli encodings, and observable-shift evaluation.The restricted-feature arm is not a reproduction of Liao et al. (2024)’s published method, and the per-observable forest cannot predict an unseen observable.Source: §5 · p. 12

6 Conclusion

QEMScore compares learned mitigators with capacity-matched controls that omit measurement inputs, using exact simulated targets, released-data reanalyses, and cost accounting. The results motivate reporting these controls alongside each learner, while the conclusions remain specific to the tested protocols.

6 Conclusion

  • QEMScore compares each learner with a capacity-matched control using the same circuit descriptors but omitting the measurement.Exact simulated targets, released-data reanalyses, and measurement-cost accounting make the comparisons inspectable alongside prediction error.Source: §6 · p. 12
  • Across six primary within-family evaluations, descriptor-only controls reproduce 87.7 to 100.5 percent of the gain over an affine control, while the full method removes 19.5 to 74.5 percent of remaining control error on five evaluations.The sixth gap’s interval spans zero; released Q-LEAR and QRAFT archives show predictive gains from measurement-derived inputs.Source: §6 · p. 12
  • Report the full learner beside descriptor-only controls with same-unit errors, uncertainty, circuit overlap, and each method’s measurement spend.A reproduced share needs its affine reference and remaining-error reduction to be interpretable.Source: §6 · p. 12

6 Conclusion; Broader Impact Statement

  • Making measurement-input contribution and collection cost visible gives subsequent evaluations a clearer basis for judging progress in learned quantum error mitigation.The study identifies no direct dual-use concern and reports no aggregate-comparison evidence about individual harmful corrections.Source: §6 · p. 13, Broader Impact Statement · p. 13

Reproducibility Statement

The reproducibility and protocol materials describe how the campaign is generated, audited, scored, and costed, while distinguishing completed evaluations from proposed future transfer studies. They specify important boundaries, including uncalibrated cross-family noise severity and conditional statistical intervals.

Reproducibility Statement

  • Controlled datasets regenerate from recorded seeds, and validation rebuilds circuits, recomputes exact labels, checks hashes, and verifies ledger conservation.The campaign archive and per-file checksums are released; the paper describes a stratified campaign audit rather than an independent replay of every execution.Source: Reproducibility Statement · p. 16, A Formal Protocol and Code Crosswalk · p. 17

A Formal Protocol and Code Crosswalk

  • The primary decomposition reports affine, capacity-matched, and full-method errors separately for every family-and-seed row, with pointwise bootstrap intervals conditional on selected models.Evolution-step and training-size contrasts are secondary; the primary setting and decomposition were declared after inspecting a rehearsal whose scores are not reported.Source: A Formal Protocol and Code Crosswalk · pp. 16–17

B Measurement Bill: Formulas and Tiers

  • The measurement ledger charges circuit evaluations across training, mitigation-specific test circuits, and base prediction; exact simulated labels are logged separately.Compatible observables from one circuit, noise, and shot configuration share a measurement charged once.Source: B Measurement Bill: Formulas and Tiers · p. 17
  • Budget tiers cap spending but do not equalize realized totals or reallocate a cheaper method’s unused allowance to extra base-measurement shots.ZNE’s folded executions can make its realized total larger than its nominal total.Source: B Measurement Bill: Formulas and Tiers · p. 17

C Noise Families and Strength Grids

  • The campaign evaluates depolarizing plus readout at two strengths, but shipped strength grids are uncalibrated across noise families.Unseen-noise transfer therefore changes channel and effective severity together, leaving cross-family severity comparisons out of scope.Source: C Noise Families and Strength Grids · p. 18

D Statistical Protocol

  • Accuracy is mean absolute error against exact labels on the raw expectation scale, while aggregation gives predeclared cells equal weight and retains cell-level signed biases.Intervals are pointwise and conditional on fitted and selected models, so they do not include training or model-selection uncertainty.Source: D Statistical Protocol · p. 18

E Field Statistical Protocol

The field reanalysis uses separately specified Q-LEAR and QRAFT protocols, with distinct scores, input arms, and aggregation choices. Its comparisons are conditional on released data and row-level splits, and the QRAFT reanalysis neither reproduces nor evaluates the original method under its chosen objective.

E Field Statistical Protocol

  • The field-half specification has no pre-fit freeze file, although pre-review checksums show inputs did not change afterwards.Those checksums cannot establish when the specification was frozen.Source: E Field Statistical Protocol · p. 19
  • Q-LEAR’s affine and nonlinear descriptor controls differ only in learner capacity, while additional arms add state-weight, base-run statistics, and depth-cut features.Its score is Hellinger distance on released observed-state support, with clipped nonnegative predictions normalized before scoring; field results average backends within applications and then applications equally.Source: E Field Statistical Protocol · p. 19
  • Field intervals resample applications and condition on one fit, eight backends, and released labels; they exclude refitting and model-selection uncertainty.No QRAFT panel interval is reported because its archive does not support resampling that respects within-circuit correlation, and five machine clusters are too few for a stable 95% interval.Source: E Field Statistical Protocol · pp. 19–20
  • Released CSVs lack circuit identifiers, and the validation and evaluation panels draw from the same six-application directory, precluding reconstructed circuit-disjoint splits.These results are conditional predictive comparisons under the released protocol, not evidence about unseen application circuits.Source: E Field Statistical Protocol · pp. 19–20
  • The QRAFT panel uses a separate mean-absolute-error score in probability points, weighted equally within machines and across five machines.Its released archive lacks circuit groupings needed to compute Hellinger distance.Source: E Field Statistical Protocol · p. 20
  • QRAFT is read through shipped predictions and a ten-seed retraining panel with three descriptor, forward-probability, and reverse-statistics arms.The retrained panel uses histogram gradient boosting rather than QRAFT’s MATLAB classification ensembles, changes the objective, and reports absolute errors without weights; only arm ordering is interpreted.Source: E Field Statistical Protocol · p. 20

F Field Execution Accounting

The accounting reports how feature-collection steps use circuit evaluations without equalizing methods’ costs. It also specifies how the campaign separates strata and selects and diagnoses models.

F Field Execution Accounting

  • On hardware, base statistics return 26.61 times more error reduction per added evaluation than depth-cut circuits, descriptively and without an interval on the ratio.The efficiency ratio is descriptive for this collection and retraining protocol; the simulator panel has no efficiency multiple because depth-cut intervals often cross zero.Source: Table 5 · p. 21, F Field Execution Accounting · p. 21
  • The descriptor control already uses support from the first base batch, while base statistics use both batches; depth-cut collection costs three times the base step within each panel.Base statistics require 1,024 hardware evaluations and 44,843 simulator evaluations; the unread full inverse adds the same respective counts as the base-statistics step.Source: F Field Execution Accounting · pp. 20–21
  • Because the design has no shot-count sweep or fixed-evaluation comparison, it cannot establish what one further evaluation would buy or how evaluations should be allocated.Counts are source-implied intended counts, not evidence about historical billing, retries, or wall-clock time.Source: F Field Execution Accounting · pp. 20, 22

G The Clifford Control Stratum

  • Clifford circuits are reported as a separate stratum because their ideal values are discrete, and their cells are excluded from continuous-regression headline statistics and costs.The runner refuses mixed-stratum datasets and the report renders Clifford results separately.Source: G The Clifford Control Stratum · p. 22

H Model Selection and Feature Diagnostics

  • Model selection uses a one-standard-error rule on mean circuit-block validation error, with ties broken by signed validation improvement, then lower cost, then model simplicity.For changed settings, validation data remain in the familiar domain.Source: H Model Selection and Feature… · p. 23
  • The feature specification includes the noisy estimate and sampled circuit parameters but omits noise-family and strength inputs, enabling controls to test prediction from the specified representation.The sampled parameters are why a simulator surrogate is possible; every learned arm consumes the same ordered vector.Source: H Model Selection and Feature… · p. 23
  • A competitive feature-only score or accuracy surviving a shuffled-measurement diagnostic motivates direct gain estimates, but neither observation alone establishes that a model ignores measurements.The automatic alarm compares feature-only and ridge test errors without uncertainty; the source-validation diagnostic uses paired circuit-blocked intervals and is not an equivalence test.Source: H Model Selection and Feature… · p. 23

I Reproducibility Details

The reproducibility design links generated datasets and result manifests through recorded seeds, hashes, versions, and validation checks. Reproducing a table requires several commands rather than one.

I Reproducibility Details

  • Dataset artifacts are write-once, and each item records seeds, structure, noise, shot counts, noisy estimates, exact labels, and label methods.Manifests record package and backend versions, seed derivation, feature specification, active severity grids, and dataset hashes.Source: I Reproducibility Details · p. 23
  • The validator checks schema, hashes, roles, exact labels, and measurement-bill conservation, while the runner checks item-ID overlap and a separate campaign check verifies physical-circuit separation.Methods use a phased fit-select-predict interface that is tested but not declared stable across releases.Source: I Reproducibility Details · p. 23
  • Reproducing a table takes a sequence of commands, and each report’s numerical trace identifies the source runs and cells behind its printed values.Source: I Reproducibility Details · p. 24

J Detailed Tables Behind the Result Figures

The detailed tables record the campaign’s primary contrasts and the released-hardware reanalyses. Their results are bounded by the evaluation protocol and the contents of the released data.

J Detailed Tables Behind the Result Figures

  • A measurement-free capacity-matched control reproduces 87.7 to 100.5 percent of the affine control’s improvement across six evaluations, while the control–ablation gap excludes zero on five.Seed 101 with Heisenberg is the exception; the five shares with intervals entirely below one do not reduce to a single share.Source: Table 7 · p. 24
  • The capacity-matched-control gap excludes zero on five evaluations and is a small fraction of the ablation’s total improvement on every row.The pointwise circuit-bootstrap intervals condition on fitted and selected models, omitting retraining and model-selection variation.Source: Table 8 · p. 24
  • Under the released Q-LEAR protocol, the estimated macro capacity gain is small and every per-fit interval spans zero; the base-statistics step is positive on both panels in every fit.The table contrasts capacity at fixed descriptors first, then adds measurement-derived feature groups; the panels disagree on the depth-cut step.Source: Table 9 · p. 25
  • Across hardware applications, the near-zero nonlinear-descriptor average partly reflects cancellation, while the full method’s gap over that control stays positive on every application.Two pricing applications gain and four lose against the affine control.Source: Table 10 · p. 25
  • The three admitted systems’ readings disagree, and each is qualified by its release-specific evaluation unit and intervals conditional on fitted and selected models.The table does not average or pool shares across systems.Source: Table 11 · p. 26

K.1 From Noisy Measurements to Learned Mitigation

  • The paper registers ZNE and reports its untouched-test errors, but does not evaluate CDR or variable-noise CDR because their per-target training-set construction rule is unsettled here.It ships a probe and cost formulas for those methods, without fit-and-predict paths.Source: §K.1 · p. 27

K.2 What Existing Learned-QEM Studies Already Cover

QEMScore distinguishes its measurement-contribution comparison from prior learned-QEM benchmarks and sets boundaries on what its protocols and data can establish. The artifact supports exact-label controls and controlled evaluation settings, while some shifts and transfer questions remain unevaluated.

K.2 What Existing Learned-QEM Studies Already Cover

  • QEM-Bench standardizes training and scoring datasets and generalization settings, while QEMScore separately standardizes what a score may mean through measurement accounting, validated exact-label datasets, and surrogate controls.The two benchmarks are described as complementary rather than competing.Source: §K.2 · p. 27
  • QEMScore has no unseen-observable setting because each tested observable must appear in training, and its restricted-feature Liao-style arm omits encodings used by the published method.The restricted-feature comparison therefore does not establish how the published method would perform.Source: §K.2 · pp. 27–28

K.3 Reliability Benchmarks and Controlled Shift

  • QEMScore’s controlled shift recipes isolate one declared item-tuple component at a time, but the first campaign runs only the familiar setting.The five shifted-setting recipes specify zero-shot transfer; their degradation results remain future work.Source: §K.2 · p. 27, L Benchmark Design in Full · p. 30

K.2 What Existing Learned-QEM Studies Already Cover

  • Q-LEAR and QRAFT already group inputs by execution requirements; QEMScore adds exact-label, capacity-matched controls and applies ordered feature-group comparisons to their released data.Those reanalyses remain conditional on executed support, and the QRAFT readout is not a reproduction of its published objective.Source: §K.2 · pp. 28–29
  • QEMScore does not evaluate selective mitigation because its released artifacts cannot verify that reject decisions were made without access to test labels.A label-free selective layer would require fitting before labels are seen or a verifiable producer signature.Source: §K.2 · p. 29

L Benchmark Design in Full

  • Exact simulator labels let the benchmark measure correction error against the ideal value and estimate noisy-input predictive benefit for specified learners and features.A competitive feature-only control indicates that the representation supports accurate prediction within that model class.Source: L Benchmark Design in Full · p. 29, §L.1 · p. 31

L.1 Task Definition and Testbed Construction

  • Each testbed joins a noisy estimate and an exact ideal label from the same sampled circuit, then verifies the assembled dataset against its canonical descriptors and recomputed labels.The validator does not re-execute noisy sampling; stored counts are bound to rows by content hash.Source: §L.1 · pp. 30–31

L.2 Benchmark Composition: Circuit Families and Noise Models

  • The family set spans mitigation workloads and structural controls with exact ideal labels, while cross-family unseen-noise transfer remains confounded by uncalibrated severity differences.The first campaign uses depolarizing plus readout alone; its noise-severity grids are not calibrated to a common physical scale across families.Source: §L.2 · pp. 31–33

L.3 Artifact Capabilities: Six Evaluation Settings

This section distinguishes the artifact’s six evaluation settings from the scope of the campaign: the first campaign uses only the familiar setting, while the field half reanalyzes Q-LEAR and QRAFT through ordered feature comparisons. The shift-resolved reliability-map protocol is described as future work, not as part of the campaign.

L.3 Artifact Capabilities: Six Evaluation Settings

  • Observable transfer is absent by construction because the restricted-feature forest cannot predict an observable unseen in training.The generator enforces observable coverage, and class-name splits can leave the physical Pauli string unchanged; observable transfer requires a model that predicts unseen observables and a suitable generator.Source: §L.3 · p. 33
  • The transfer settings are not directly rankable: unseen noise couples mechanism and severity, while strength indices are uncalibrated across noise families.The held-out noise families are generally milder than the depolarizing anchor, and the stronger-noise recipe does not ensure severity extrapolation beyond training. The benchmark does not isolate causal effects when a setting changes coupled properties.Source: §L.3 · pp. 34–35
  • Only the familiar setting was run in the first campaign; five single-component transfer settings are specified for future experiments.The settings cover unseen or stronger noise, unseen circuit families, greater depth, and fewer shots. Circuit-width transfer and compound shifts are outside the grammar.Source: §L.3 · pp. 33–35

L.4 Methods, Budgets, and Scoring

  • The roster includes raw and ZNE estimates, ridge, a restricted-feature ablation of the published learned mitigator, and four surrogate controls.The learned arms share a feature vector and measurement charge; the ablation omits features from the published construction, so its result does not reproduce or establish performance of that method.Source: §L.4 · p. 35
  • Each method faces its own feasibility cap rather than an equalized spend; the campaign uses tier H, a cap of 2.5 × 10^8 circuit evaluations.Realized totals differ by design, and this release reports no per-arm campaign ledger or empirical comparison of realized costs.Source: §L.4 · p. 36, Table 13 · p. 36

L.5 The Field Half: Reanalyzing Two Published Systems

  • The field half orders feature comparisons for Q-LEAR and QRAFT, using Hellinger distance and per-row mean absolute error, respectively.Q-LEAR compares descriptor-only, nonlinear descriptor, base-statistics, and full-feature rungs; QRAFT compares descriptor-only, forward-measurement, and forward-and-reverse feature sets. The contribution is a matched retraining comparison with execution counts, not a shared score across systems.Source: §L.5 · pp. 37–39, Table 14 · p. 38
  • Field-half interpretation is limited by absent exact labels and circuit-disjoint splits, and the Q-LEAR replay gate had no prespecified tolerance.QRAFT is scored with an objective different from its own, and its released rows cannot be reassembled into circuits; neither system’s panel establishes performance on unseen application circuits.Source: §L.5 · pp. 37, 39

M Campaign Results in Full

  • Campaign claims use untouched-test estimates and direct contrasts, with continuous-regression results separated from the discrete random-Clifford stratum.Results are grouped by evaluation setting, and the default within-stratum aggregate is distinct from the split-specific sections used for setting-specific claims.Source: M Campaign Results in Full · p. 39
  • A future shift-resolved reliability map would require declared win, tie, loss, and harm rules plus frozen paired comparisons with Holm correction.This protocol is future work in Appendix D; the campaign instead follows the primary decomposition specified in Section 4.2 and Appendix A.Source: M Campaign Results in Full · p. 40

M.1 Do the Controls Leave Room for Mitigation?

The campaign separates capacity from measurement access with matched controls, then reports primary within-family results alongside secondary diagnostics and two released-data reanalyses. Its findings are conditional on the tested learners, settings, and scoring protocols.

M.1 Do the Controls Leave Room for Mitigation?

  • The source-validation gate passes for the restricted-feature ablation but fails for ridge on all six shipped-step datasets; it is a selection-row diagnostic, not an attribution decision.Across the shipped-step rows, the affine control is within 1.0004 to 1.0145 times ridge’s error, inside the 1.05 margin.Source: §M.1 · p. 40
  • ZNE reduces raw error to 0.46–0.47 on Heisenberg and 0.16–0.17 on transverse-field Ising, yet remains less accurate than feature-based estimators; shuffled-noisy matches the affine control within 0.0001.Ridge reverses against the affine control on two transverse-field Ising seeds, so these six points support no ranking of ridge against it.Source: §M.1 · p. 41, Table 15 · p. 41

M.2 Secondary: Measurement Benefit across Evolution-Step Regimes

  • For the larger-step contrast, three of six intervals lie below zero, two above, and one spans zero; for training size, ten of twelve point differences are positive, but only five intervals lie above zero.Both positive larger-step contrasts belong to seed 211; the counts do not establish general trends.Source: §M.2 · p. 42, Table 16 · p. 42, Table 17 · p. 42

M.3 RQ1: The Capacity-Matched Feature-Only Control

  • The primary decomposition compares a full nonlinear model with a capacity-matched feature-only model that deletes the noisy-estimate input, scoring both against exact ideal labels.The campaign covers two spin-chain families and three seeds at shipped steps and 640 training circuits per family.Source: §M.1 · p. 40, §M.3 · p. 43
  • The capacity-matched control reproduces 87.7 to 100.5 percent of the affine-control improvement in all six evaluations, while the full method reduces its remaining error by 19.5 to 74.5 percent on five.The sixth reduction is −12.6 percent and its interval spans zero; the two ratios order the evaluations oppositely.Source: §M.3 · pp. 43–44, Figure 6 · p. 44

M.4 Controlled Diagnostics: Signal and Polynomial Baselines

  • Signal attenuation varies by family, severity, and observable: transverse-field Ising slopes range from 0.79 to 0.86 at L1 and 0.23 to 0.40 at L3, while Heisenberg L3 slopes are 0.0692 and 0.0064.These descriptive statistics identify severe attenuation in part of the testbed, not a uniform absence of signal.Source: §M.4 · p. 44, Table 18 · p. 45
  • The recorded degree-five polynomial using coupling descriptors alone has lower error than the full method on all six primary evaluations, but this post-hoc comparison has no paired interval.The comparison challenges the evaluated learners’ adequacy on the smooth, low-dimensional coupling parameterization, not an information-theoretic measurement requirement.Source: Table 19 · p. 45, §M.4 · p. 45

N.1 Does the Replay Reproduce the Published Numbers?

  • All six published Q-LEAR application values fall inside the ten-fit range, with a largest application mean deviation of 0.0134, but this is not an equivalence test.At backend resolution, the Lagos published value is 0.44, below the retrained range of 0.4446 to 0.4595.Source: Table 20 · p. 46, §N.1 · p. 46

N.2 Does the Descriptor Control Carry the Improvement on Q-LEAR?

The Q-LEAR and QRAFT reanalyses compare ordered feature groups under their released-data protocols. Measurement-derived inputs carry predictive gains in both panels, while the descriptor-capacity comparison and the QRAFT feature-group ordering remain protocol- and score-specific.

N.2 Does the Descriptor Control Carry the Improvement on Q-LEAR?

  • The nonlinear descriptor control’s mean hardware gain over the affine control is only +0.000055, and every per-fit interval spans zero; this does not establish equivalence.The average partly reflects cancellation: both pricing applications improve on all ten fits, while the other four applications worsen on all ten.Source: §N.2 · p. 47
  • On Q-LEAR hardware, the full method improves on the nonlinear descriptor control by a ten-fit macro mean of 0.2191, with all ten per-fit intervals above zero.The ordered split assigns 0.1969 to base statistics and 0.0222 to three depth-cut circuits; the latter interval is above zero on nine of ten fits, and the simulator pattern differs.Source: §N.2 · p. 47
  • Q-LEAR’s four circuit descriptors are constant across output states within a circuit, but the state weight separates states of unequal ideal probability in some released files.The descriptors therefore carry partial query information rather than none.Source: §N.2 · p. 47

N.3 Does a Richer Descriptor Set Carry It on QRAFT?

  • On QRAFT, descriptors alone score roughly six times worse than descriptors plus forward observed probability in both released predictions and matched retraining: 12.4838 against 2.1074 and 13.5903 against 2.2685.The descriptor-only arm loses on every machine and every retraining seed, despite the released row split allowing a circuit’s output states into both train and test.Source: Table 21 · p. 48, §N.3 · p. 48
  • Adding reverse-execution statistics and machine identity improves matched retraining from 2.2685 to 2.1246 but worsens released predictions from 2.1074 to 2.4467.The panels use different learners, and QRAFT’s weighted classification objective is not evaluated by this mean-absolute-error readout.Source: §N.3 · p. 48

N.4 What Two Instances Establish

  • The two reanalyses do not establish prevalence: QRAFT has no affine arm or intervals, and its data come from the same application circuits and release as Q-LEAR.Both panels estimate retained numerical-feature contributions conditional on executed-circuit support, not execution-free prediction.Source: §N.4 · pp. 48–49

O An Additional Question: Three Screened Published Systems

The screened-system reanalysis gives different, protocol-bounded readings rather than a pooled result: Q-Cluster and ML-QEM read present, while Synergy CNN reads absent under reduced-data conditions. The readings carry limits from circuit overlap, model reconstruction, and the systems’ released materials.

O An Additional Question: Three Screened Published Systems

  • The scoring plan defines S = (A − C)/(A − F) and compares each system’s own-loss readings with 0.5; QRAFT has no affine arm or share.The three systems are not pooled, and Figure 7 shows their twelve declared panels with row-specific intervals.Source: O An Additional Question: Three… · p. 49, Figure 7 · p. 50
  • ML-QEM reads present at S = 0.5985 [0.5808, 0.6172], with 277 of 2,500 test rows sharing a descriptor block with training.A sensitivity using ridge as the affine control and the separately released saved model reads 0.4721 [0.4529, 0.4928].Source: O An Additional Question: Three… · pp. 49, 51
  • Synergy CNN reads absent across six panels, but its reduced-data controls use 90 and 72 rows per fold against a released model trained on roughly 600,000 circuits.One panel uses a reconstructed one-dimensional architecture, so applicability at the original published budget is not established.Source: Figure 8a · p. 50, O An Additional Question: Three… · p. 51
  • Q-Cluster reads present at S = 0.8795 [0.7723, 0.9658] with calibration available and 0.9211 [0.8480, 0.9716] with it withheld, but scores an intermediate regression stage.Its released split puts every circuit in both training and test, favoring the descriptor control; unseen-circuit robustness is not established.Source: O An Additional Question: Three… · p. 51
  • The descriptor-feasibility audit is incomplete: it supplies no controlled-reference saturation classification, and its diagnostics do not bound descriptor prediction universally.Synergy CNN’s descriptors determine its target exactly, with a state-vector simulator agreeing to 1.46 × 10−13 across 300 rows.Source: O An Additional Question: Three… · p. 51
  • On ML-QEM, S = 0.5985 coexists with (R − C)/(R − F) = −0.2552 because both controls have greater error than the raw estimate.A = 0.44188, C = 0.23757, R = 0.20971, and F = 0.10052 on mean per-row distance from the predicted four-vector to the ideal one.Source: O An Additional Question: Three… · p. 51
  • The screen’s criteria were informed by prior knowledge, adjudication remains incomplete, and the three admitted systems do not exhaust eligible candidates.The criteria included properties already known for ML-QEM, which had been named in the project before screening.Source: O An Additional Question: Three… · p. 52
  • Three screened systems cannot estimate prevalence, and none establishes a clean occurrence of the controlled pattern in final mitigation accuracy or its absence.The readings are conditional on released materials, declared evaluation units, and released splits.Source: O An Additional Question: Three… · p. 52

P Extended Discussion

The discussion clarifies that the campaign measures noisy-input gains for specified models and representations, not a universal need for measurement. It also identifies evaluation priorities and describes the release’s reproducibility checks and practical limits.

P Extended Discussion

  • Four surrogate controls accompany campaign methods, including a circuit-and-observable predictor that never sees the noisy measurement.The feature contract includes sampled couplings, angles, and evolution step size, so circuit parameters can support ideal-target prediction.Source: P Extended Discussion · p. 52
  • The restricted-feature ablation uses QEMScore’s feature vector and Liao et al.’s architectures, but omits native-gate counts, angle-bin histograms, and sparse Pauli encoding.Its outcome describes one restricted representation under one protocol, not the published method.Source: P Extended Discussion · p. 52
  • A common-severity calibration is needed before unseen-noise results can isolate a change of mechanism, and executable CDR and vnCDR comparators remain future work.The discussion also proposes an allocator for equal-spend comparisons and an unseen-observable model.Source: P Extended Discussion · p. 52
  • The campaign has not evaluated settings S1–S4 and S6; prevalence assessment would require representative releases, final mitigation accuracy, full training data, and circuit-disjoint splits.Controlled comparisons on physical processors are also identified as future work.Source: P Extended Discussion · p. 53

Q Reproducibility and Release in Full

  • Dataset generation is deterministic, derives recoverable per-item seeds from a master seed, and writes content-addressed artifacts that are not edited by runs.Generation refuses to write into an existing path; reports are views over result manifests.Source: Q Reproducibility and Release in Full · p. 53
  • The validator rebuilds split-artifact circuits and recomputes exact labels while checking hashes, roles, counts, and ledger limits, but legacy fixtures lack sidecar hashes and validation status.A separate entry point checks run identities and ledger conservation.Source: Q Reproducibility and Release in Full · p. 53
  • The command interface generates and runs named presets, but arbitrary specification files are not read and additional-method registration is not stable across releases.Reporting uses a separate Python module entry point.Source: Q Reproducibility and Release in Full · p. 53
  • Mitiq cross-checks ZNE as the suite’s only external reference implementation; the release also archives campaign outputs with per-file checksums.Two benchmark dataset hashes regenerate under the recorded dependency lock, although no single command reproduces a table or figure.Source: Q Reproducibility and Release in Full · p. 53

R Limitations in Full

The paper’s claims are conditional on the tested simulator regimes, released field data, and method roster. It also specifies limits on spending, setting coverage, audit verification, and transfer beyond those data.

R Limitations in Full

  • Controlled conclusions are conditional on tested simulator regimes, circuit families, and budgets, and do not establish hardware transfer.Field reanalysis uses archived data from eight IBM backends and sampled ideal-simulator supervision, without new quantum executions or claims of quantum advantage or hardware speedup.Source: R Limitations in Full · p. 54
  • The method roster omits executable CDR and vnCDR comparators, and Liao et al.’s method appears only as a restricted-feature ablation, not a reproduction.Controlled comparisons concern this roster and testbed; field comparisons use separate released-data protocols.Source: R Limitations in Full · p. 54
  • Equal budget tiers do not equalize method spending, and no allocator uses a cheaper method’s remaining allowance for additional shots.The paper reports no per-method bucket totals; micro-preset totals illustrate different realized costs within one tier.Source: R Limitations in Full · p. 54
  • The completed campaign evaluates only the familiar setting; uncalibrated noise strengths prevent isolating noise-family transfer, and the fixed forest cannot predict unseen observables.Circuit width and compound multi-axis changes are also omitted.Source: R Limitations in Full · p. 54
  • Validation checks stored labels and structure fields but does not replay sampled counts or recompute transpilation features, so consistent but incorrect records could pass.The stratified audit covered all twelve campaign datasets, but was not an independent replay of every execution.Source: R Limitations in Full · pp. 54–55

Related papers

Loading 2609.17896v1…