PaperlayerView exact revision on arXiv

Source-linked AI summary

RACE-AIMC: Selective Inference for Heterogeneous Analog In-Memory Accelerators at the Edge

Osama Yousuf, Martin Lueker-Boden

arXiv:2609.03149v1cs.ETcs.ARcs.LG

TL;DR

Imperfect, chip-specific AIMC hardware creates a choice between running every accelerator and trusting one without an error guarantee. RACE-AIMC selects and certifies one accelerator offline, then accepts or defers predictions online. Across five runs, its policies certified accepted error below the 10% target while matching clean digital accuracy and reducing modeled energy relative to an always-on ensemble.

  • Problem

    AIMC device imperfections make accelerator errors chip-specific, while running every chip wastes energy and blindly trusting one provides no application-level error guarantee.

    Source: §I · p. 1

  • Method

    RACE-AIMC characterizes a pool offline, selects one accelerator and threshold, and certifies accepted-prediction error on untouched data before online accept-or-defer decisions.

    Source: §I · p. 1, D. Threshold fitting, selection, and… · p. 4

  • Results

    Across five independent runs, all frozen policies certified accepted-error rates below 10%, answered roughly 71% of queries directly, matched clean digital accuracy overall, and cut modeled energy by 69% versus a naive always-on heterogeneous ensemble.

    Source: §VII · p. 5

  • Takeaways & Limitations

    A lightweight edge controller can use a statistically certified low-cost analog path for accepted queries and reserve the more expensive fallback for others.

    Source: §VII · p. 5, §VI · p. 5

    The certificate assumes exchangeability between certificate and deployment data and must be recomputed after distribution shifts, temperature changes, conductance drift, or policy changes.

    Source: §VI · p. 5

Abstract

from arXiv · show

Analog in-memory computing (AIMC) speeds up neural-network inference by doing the arithmetic directly inside a memory array, instead of shuttling weights back and forth between memory and a processor. This saves energy, but the physical devices that store the weights are imperfect: programming errors, electrical noise, limited-resolution converters, and outright broken cells all distort the computation, and every physical chip is distorted in its own way. A designer with several such chips available faces an uncomfortable choice: run all of them and combine the answers (safe, but wasteful of energy), or trust a single chip blindly (cheap, but with no guarantee on how often it is wrong). This paper introduces RACE-AIMC (Risk-Aware Certified Ensemble for AIMC), a framework that resolves this choice with statistics rather than guesswork. Offline, RACE-AIMC studies a pool of physical accelerators, picks the single best one for a given energy budget, and computes a mathematically exact upper bound on how often that accelerator will be wrong when it chooses to answer. Online, only that one accelerator is switched on; a lightweight check decides whether to accept its answer or defer to a fallback. In our simulations using a noisy weight mapping and multiple independent test runs, every certified bound stayed under a 10% error target (mean bound 7.83% +- 0.89%, with 70.88% +- 0.98% of inputs answered directly). The resulting system matches the accuracy of a clean digital baseline while cutting modeled energy use by 69.02% relative to always running every accelerator in the pool.

I. INTRODUCTION

AIMC reduces memory–processor data movement, but device imperfections make each accelerator’s errors chip-specific. RACE-AIMC selects one accelerator offline, certifies its accepted-error rate, and lets an online controller accept or defer each prediction.

I. INTRODUCTION

  • Programming errors, read noise, converter quantization, and stuck faults make mapped networks differ from their digital versions, with mistakes varying across physical chips.The variation arises because each chip has its own manufacturing imperfections.Source: §I · p. 1
  • RACE-AIMC addresses the trade-off between wasteful all-chip ensembles and uncertified single-chip predictions by selecting one accelerator and deferring low-confidence inputs.The offline process characterizes the pool and certifies accepted error; online, only the selected accelerator runs before the controller accepts or defers its answer.Source: §I · p. 1, Figure 1 · p. 2
  • The contributions are an energy-aware accelerator-selection rule, a per-accelerator stopping rule, an exact certificate on accepted-answer error using untuned data, and simulation experiments.The supplied contribution passage ends before describing the experiments’ findings.Source: §I · p. 1

II. BACKGROUND AND RELATED WORK

The background explains how crossbar arrays perform neural-network operations and situates RACE-AIMC alongside selective classification and conformal prediction. The paper focuses on deciding whether to trust a characterized chip for a particular input.

A. Computations in crossbar arrays

  • A crossbar stores weights as device conductances and computes a layer’s matrix-vector product through applied voltages and resulting currents.Accuracy depends on weight encoding and programming, read noise, converter resolution, and peripheral circuit non-idealities.Source: A. Computations in crossbar arrays · p. 2
  • The experiments use transparent multilevel weight encoding to measure the proposed policy rather than an artifact of limited bits per weight.The passage contrasts this choice with binary/sign-only encoding.Source: A. Computations in crossbar arrays · p. 2

B. Selective classification and conformal prediction

  • Selective classification trades coverage against risk by accepting sufficiently confident predictions and rejecting the rest; prior systems generally assume stable digital confidence signals.RACE-AIMC instead asks whether an already-characterized chip should handle a particular input, and targets the accepted predictions’ Bernoulli error rate with an exact one-sided binomial interval.Source: B. Selective classification and… · p. 2, A. Computations in crossbar arrays · p. 2

III. RACE-AIMC

RACE-AIMC models a heterogeneous pool whose accelerators share a trained classifier and digital feature front end. It separates calibration data by role and balances direct-answer coverage against the selective-risk budget.

A. System model and data separation

  • Each accelerator produces class logits for inputs from a shared digital feature front end, and the highest logit determines its predicted label.Each accelerator has an activation energy and communication cost; its confidence score is used in the acceptance decision.Source: A. System model and data separation · pp. 2–3
  • Each accelerator accepts an input only when its confidence score clears that accelerator’s threshold.The acceptance rule is a yes/no decision to accept or defer.Source: A. System model and data separation · p. 3
  • Selective risk is the wrong-answer fraction among accepted inputs, while coverage is the fraction of inputs the accelerator accepts.The design goal is to maximize coverage while keeping selective risk below the application-defined budget.Source: A. System model and data separation · p. 3
  • Calibration data are divided into non-overlapping hardware, policy, and certificate splits, each assigned a distinct purpose.The hardware split estimates per-accelerator correction; the policy split fits thresholds and selects the deployed accelerator; the certificate split evaluates the frozen policy.Source: A. System model and data separation · p. 3

B. Multilevel differential weight mapping

The multilevel differential mapping quantizes weight magnitudes and represents weight signs with pairs of physical conductances. It also applies a shared layer scale to weights and inputs and audits mapping quality.

B. Multilevel differential weight mapping

  • Weights are normalized by each layer’s maximum magnitude and rounded onto a 256-level grid.The mapping uses Q = 255 for the 8-bit conductance levels.Source: B. Multilevel differential weight… · p. 3
  • Each weight’s sign is encoded with two devices because analog conductances cannot be negative.For positive weights, the positive branch carries the encoded magnitude while the negative branch remains at its floor; the roles swap for negative weights.Source: B. Multilevel differential weight… · p. 3
  • The same per-layer scale is applied to the inputs during the physical multiply to avoid mismatched weight and input scales.Each mapped layer is audited for distinct reconstructed values, clipping, and standard mapping-error metrics.Source: B. Multilevel differential weight… · p. 3

C. Hardware correction and confidence

RACE-AIMC estimates each accelerator’s persistent logit bias against clean digital logits, then uses the top-two bias-corrected logit margin as its confidence score.

C. Hardware correction and confidence

D. Threshold fitting, selection, and certification

RACE-AIMC fits per-accelerator thresholds on a policy split, chooses and freezes one accelerator, then certifies its selective risk on an untouched split.

D. Threshold fitting, selection, and certification

  • For each accelerator, policy-split examples are ordered by confidence and acceptance is built from progressively larger prefixes, with the Clopper-Pearson bound set to 1 when n = 0 or e = n.Source: D. Threshold fitting, selection, and… · p. 3
  • The largest prefix whose Clopper-Pearson upper bound meets the stricter design-risk target is kept, creating slack before the separate certificate-split check.The passage gives rd = 8% versus deployment target r = 10% as an example.Source: D. Threshold fitting, selection, and… · p. 3
  • After fitting each accelerator’s threshold and measuring its empirical coverage, the method selects a single front-line accelerator using coverage per unit energy.Source: D. Threshold fitting, selection, and… · p. 3
  • The winning accelerator and its threshold are frozen, with no further tuning.Source: D. Threshold fitting, selection, and… · p. 3
  • On the untouched certificate split, the frozen policy is certified when its exact one-sided binomial upper bound is at most r; this is a finite-sample risk statement under exchangeability.The check uses nC accepted examples and eC errors.Source: D. Threshold fitting, selection, and… · p. 4

IV. EXPERIMENTAL METHODOLOGY

The experiments evaluate a convolutional CIFAR-10 classifier, train multiple independently seeded realizations, and reserve the official test set for descriptive checks.

IV. EXPERIMENTAL METHODOLOGY

  • Experiments use CIFAR-10 with a three-stage convolutional feature extractor and a two-layer classifier head.The feature path ends with adaptive pooling to 4 × 4; the classifier maps 2048 features through 256 units to 10 classes.Source: A. Model, data, and the analog/digital… · p. 4
  • Each independently seeded realization trains on 40,000 examples for 30 epochs, using batch size 128, AdamW, cosine annealing, cross-entropy, and random-crop/flip augmentation.Source: A. Model, data, and the analog/digital… · p. 4
  • The remaining 10,000 training examples are split into H = 3000, P = 2000, and C = 5000, while the official 10,000-example test set is used only descriptively.The test set sanity-checks behavior and does not compute the certified guarantee.Source: A. Model, data, and the analog/digital… · p. 4

B. Heterogeneous accelerator population

The study uses six simulated 8-bit AIMC profiles as a reproducible stress envelope and evaluates the frozen protocol against four systems with a component-level modeled energy comparison.

B. Heterogeneous accelerator population

  • Six simulated profiles share 8-bit conductance devices and converters but vary in noise, fault rate, and energy/latency, from mild, cheap A0 to harsh, expensive A5.The profiles use distinct deterministic simulator seeds as a reproducible stress envelope rather than a fit to one real chip.Source: B. Heterogeneous accelerator population · p. 4, Table I · p. 4

C. Frozen protocol, baselines, and energy model

  • Five confirmatory random seeds were fixed before the runs, making the reported results out-of-sample confirmations rather than post hoc selections.Source: C. Frozen protocol, baselines, and… · p. 4
  • The comparison includes clean digital inference, forced execution on the selected analog accelerator, selective inference with digital fallback, and an always-on six-accelerator ensemble.Source: C. Frozen protocol, baselines, and… · p. 4
  • Energy is a component-level modeled estimate that excludes the shared digital convolutional path, rather than a full-platform hardware measurement.The model includes analog activation, communication, controller, fused-class, and fallback costs.Source: C. Frozen protocol, baselines, and… · p. 4

V. RESULTS

RACE-AIMC’s selected accelerator outperforms blind six-chip averaging, while its selective hybrid matches clean digital accuracy. Its frozen policies also certify below the 10% risk target.

A. Selection beats blind redundancy

  • 81.00% ± 0.55% forced accuracy for the selected accelerator exceeds the six-chip ensemble’s 75.32% ± 1.74%, despite using far less energy.The selected accelerator is 5.68 accuracy points better on average than blind always-on redundancy.Source: A. Selection beats blind redundancy · p. 4, Table II · p. 4

B. Independent risk certificates and accuracy

C. Risk, availability, and confidence-score ablations

The ablations examine risk-target choice, accelerator availability, and confidence-score choice, while the energy analysis tests savings across fallback costs.

C. Risk, availability, and confidence-score ablations

  • At 5% design risk, mean test coverage is 58.35%, versus 73.40% at 9%, with all five certificate replays passing; at 10%, only 3 of 5 pass.The stricter design target provides a safety margin because thresholds are fitted before the independent certificate is computed.Source: C. Risk, availability, and… · p. 5
  • When A0, then A0–A1, then A0–A2 are unavailable, mean certified coverage falls to 59.27%, 39.24%, and 6.47%, with 5/5, 5/5, and 4/5 certificates passing, respectively.The system degrades gracefully under moderate hardware loss, but the harshest accelerators alone offer limited useful coverage.Source: C. Risk, availability, and… · p. 5
  • Four alternative confidence scores all certify 5/5 policies, with mean test coverage of 70.53–71.69% and mean accepted risk of 7.29–7.53%.The alternatives are maximum softmax probability, probability margin, log-odds, and negative entropy.Source: C. Risk, availability, and… · p. 5

D. Energy sensitivity

  • 69.02% ± 0.53% lower modeled energy: the selective system uses 2.00 ± 0.03 µJ per query versus 6.45 µJ for the always-on six-accelerator baseline.The selective system remains cheaper until fallback cost reaches a mean break-even point of 19.19 ± 0.56 µJ, compared with the modeled 4 µJ operating point.Source: D. Energy sensitivity · p. 5

VI. DISCUSSION

The discussion pairs offline hardware characterization and certification with a lightweight online decision path. Across five independent runs, the frozen policies met the stated risk target while achieving substantial direct coverage and modeled energy savings, within a simulation-based proof of concept.

VI. DISCUSSION

  • The online path uses one analog inference, a top-two logit comparison, subtraction, threshold check, and routing decision; hardware characterization, accelerator selection, and certification happen offline.The offline work is performed once ahead of deployment.Source: §VI · p. 5
  • Deliberate heterogeneity is not required: nominally identical chips can differ through programmed states, aging, faults, and operating temperatures.Heterogeneity enlarges the selection opportunity, according to the discussion.Source: §VI · p. 5
  • The six profiles are a simulator-generated stress envelope, energy figures are modeled component-level costs, and CIFAR-10 with a compact CNN is a controlled proof of concept.The certificate also assumes exchangeability with future deployment data and must be recomputed after distribution, temperature, drift, or frozen-policy changes.Source: §VI · p. 5
  • Future work includes validation on measured fabricated-accelerator traces, drift-aware recalibration, sequential risk control, multi-stage fallback chains, and joint latency-energy objectives.Source: §VI · p. 5

VII. CONCLUSION

  • Across five independent runs, frozen policies certified accepted-error below a 10% target, answered roughly 71% of queries directly, matched clean digital accuracy overall, and cut modeled energy 69% versus an always-on heterogeneous ensemble.The conclusion describes a single cheap analog accelerator for direct answers, reserving fallback for other queries.Source: §VII · p. 5

Related papers

Loading 2609.03149v1…