PaperlayerView exact revision on arXiv

Source-linked AI summary

Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer

Haoran Jin, Kangqi Zhang, Jirong Yang, Barry Lyu, Qiuyi Ding, Ruijie Gao, Nathan Bleier

arXiv:2609.32922v1cs.AI

TL;DR

Matrix multiplication dominates inference cost, but conventional accelerators expose only a small set of fixed-width formats for mixed precision. This paper builds a GPU substrate for stochastic computing and a per-row stream-length policy; across evaluated vision tasks, SC remains competitive with fixed-format quantization, while mixed precision helps preserve accuracy at lower budgets.

  • Problem

    Matrix multiplications dominate modern transformer inference, while hardware-supported fixed-width formats constrain the precision choices available to mixed-precision methods.

    Source: §1 · p. 1

  • Method

    The paper builds a GPU library for SC matrix multiplication and assigns stream lengths per row using lightweight operator-specific saliency metrics.

    Source: §1 · p. 2, §4.4 · p. 7

  • Results

    SC retains near-lossless accuracy across evaluated transformer-based vision tasks, and per-row mixed precision improves matched-budget results, including classification and detection settings.

    Source: §5.1 · p. 7, §5.2 · p. 8, §5.4 · p. 9

  • Takeaways & Limitations

    The results provide software-level feasibility evidence for exploring dense, temporally controlled SC precision in future accelerator design.

    Source: §6 · p. 9

    Evaluation covers representative backbones and few precision budgets rather than broad sweeps, and hardware benefits are modeled rather than measured on end-to-end silicon.

    Source: §6 · p. 9

Abstract

from arXiv · show

Matrix multiplications dominate the inference cost of modern transformer-based vision models, yet existing efficiency techniques such as post-training quantization and mixed-precision inference are largely limited to the small set of fixed-width formats (INT4, INT8, BF16, and FP16) supported by conventional accelerators. We revisit stochastic computing (SC) as a way to lift this constraint: viewed as a dense adaptive quantizer, SC controls precision by bit-stream length L rather than a fixed datapath, while each multiplication reduces to a single AND/XNOR gate. We build a GPU library that emulates SC matrix multiplication at scale, exposes stream lengths as first-class kernel arguments, and evaluates SC end-to-end on image classification, object detection and instance segmentation, class-conditional image generation, and visual world-model planning. On top of this substrate, we develop a dynamic per-row mixed-precision policy that assigns stream length per token or group at matched average budget, requires no retraining, and uses the same SC hardware across schedules. Across tasks, SC remains competitive with fixed-format INT quantization at matched bit budgets, while per-row mixed precision helps maintain accuracy at lower average stream lengths. These results provide software-level feasibility evidence that SC can serve as a dense-precision substrate for fine-grained mixed-precision inference on modern vision transformers.

1 Introduction

Matrix multiplication dominates modern neural-network inference, while conventional mixed precision is constrained to a sparse set of hardware-supported formats. This work treats stochastic computing (SC) as a dense adaptive quantizer and evaluates it across modern vision workloads.

1 Introduction

  • Matrix multiplications dominate inference cost across modern workloads, complicating deployment in latency- and energy-constrained settings.The passage names edge devices, real-time control systems, and power-limited datacenters as examples.Source: §1 · p. 1
  • Conventional mixed precision is limited by hardware to a small set of formats, including INT4, INT8, BF16, and FP16, constraining precision granularity.The passage frames this as a constraint on mixed-precision inference.Source: §1 · pp. 1–2
  • SC represents values by bit-stream density and uses a single AND or XNOR gate for multiplication, with precision set by stream length L.Unlike fixed word sizes, L can be any positive integer, and the same hardware supports different precision choices.Source: §1 · p. 2
  • Prior SC deep-learning studies mostly tested small CNNs or simplified transformers; the paper identifies a gap in end-to-end modern transformer evaluation and fine-grained mixed-precision evidence.The authors qualify the gap with “to our knowledge.”Source: §1 · p. 2
  • The work introduces a GPU library for large-scale SC matrix multiplication, evaluates it across vision tasks, and assigns stream lengths per row using operator-specific saliency metrics.The authors describe this as the first end-to-end study of SC across modern transformer-based vision workloads.Source: §1 · p. 2

2 Related Works

Related work covers stochastic computing substrates for neural networks and methods that quantize pretrained models or assign mixed precision. The paper situates its work against prior SC scaling efforts and sensitivity-aware precision assignment.

2.1 Stochastic Computing

  • SC represents values through probabilistic bit streams and reduces multiplication to an AND or XNOR gate; prior work scales it to medium-sized CNNs and develops accelerator-oriented substrates.Examples include SC-DCNN [Ren et al., 2017], uSystolic [Wu and San Miguel, 2022a], and Cambricon-U.Source: §2.1 · p. 2

2.2 Post-training Quantization and Mixed Precision

  • PTQ maps pretrained weights and activations to low-bit formats, while mixed-precision methods assign different precisions across layers, operators, or tensors.The passage describes sensitivity-aware bit-width searches and formats specialized for accelerators and generative workloads.Source: §2.2 · p. 3

3 Preliminaries and Problem Formulations

The preliminaries define SC encoding and matrix multiplication, then motivate stream length as a precision and runtime control. They also assess neural-network tolerance to SC-like perturbations as a basis for adaptive precision.

3.1 Stochastic Computing Basics

  • Bipolar SC encodes signed values in [-1,1] through the probability of 1s in a bit stream; this paper uses bipolar encoding for signed neural-network activations and weights.For bipolar encoding, Pr[zℓ=1]=(v+1)/2.Source: §3.1 · p. 3
  • An integer-comparator SNG emits each bit by comparing the input against an RNG draw, and decoding estimates the value from the count of 1s in the stream.The decoded estimate is (2c−L)/L.Source: §3.1 · p. 3
  • C-BSG gates the second stream’s RNG with the first stream so its conditional distribution matches its marginal, enforcing SCC=0 and making the bipolar XNOR multiplier bias-free at finite L.The figure also depicts stochastic-number generation and unipolar multiplication with an AND gate.Source: Figure 1 · p. 3, §3.1 · p. 4
  • SC matrix multiplication assumes inputs are already quantized by a standard PTQ pipeline, then replaces each inner-dimension integer multiplier with an SC primitive.Per-row symmetric scales produce signed integer matrices that are normalized to bipolar values before stream encoding.Source: §3.1 · pp. 3–4
  • The SC dot-product estimator sums XNOR-based products over the inner dimension and rescales by per-row scales; C-BSG makes the estimator unbiased at finite L.The paper states E[Ĉij]=Cij.Source: §3.1 · p. 4
  • Stream length L sets precision: under pairwise independence, the per-pair XNOR estimator variance is bounded by 1/L, and early termination trades quality for proportional time and energy savings.L can be any positive integer; for an INT8 operand, L=192 corresponds to about 7.58 effective bits.Source: §3.1 · p. 4

3.2 Error Tolerance in Neural Networks

  • The neural workload has a usable SC fluctuation margin: L=128/96 remain near the quality range used in prior quantized-diffusion evaluations [Chen et al., 2025a, Li et al., 2023a], while quality drops at L=64 and degrades substantially for L≤32.Source: §3.2 · p. 4

4 Methodology

The methodology models SC precision through stream length, implements its matrix multiplication on GPUs, measures where SC error concentrates, and uses offline calibration to assign per-row precision. Its policy targets longer streams to sensitive rows and shorter streams to less sensitive rows.

4.1 Hardware Model

  • At k=8, bipolar XNOR remains 1.6–1.8× lower-energy than its signed binary counterpart after the L-cycle penalty.The comparison is among multipliers synthesized at 400 MHz.Source: Table 1 · p. 5, §4.1 · p. 5

4.2 GPU Kernel for SC Matrix Multiplication

  • The GPU path fuses bit-stream generation, AND-and-popcount multiplication, and accumulation, while runtime row-length vectors vary precision without recompilation.The cumulative-indicator table is built once per layer at Lmax = 28 and reused for shorter streams.Source: §4.2 · pp. 5–6

4.3 Sensitivity Analysis

  • At L = 64, late-FC2 tiles account for 19 of the 20 most sensitive tiles in the ViT-L/14 ImageNet probe.FC2 dominates the high-sensitivity region across tested stream lengths, motivating longer streams for sensitive operators.Source: §4.3 · p. 6, Figure 3 · p. 6

4.4 Per-Row Mixed-Precision SC

  • Offline calibration replays selected matmuls with uniform SC at candidate stream lengths and records row-wise deviation for each target average length.Inference evaluates lightweight row metrics, looks up calibrated thresholds, and passes the resulting length vector to the SC kernel.Source: §4.4 · pp. 6–7
  • Rows with larger normalized metrics receive longer streams; budgeted Lagrangian assignment uses measured row-wise cosine error and binary search on λ to match the target budget.Metrics use query magnitude for QK, peak attention probability for AV, and activation magnitude for projection and MLP operators.Source: §4.4 · p. 7

5 Experiments

Across four vision tasks, the experiments compare stochastic computing (SC) with fixed-format quantization or full precision and test whether mixed precision (MP) helps at matched average budgets. MP recovers accuracy or planning headroom in lower-budget settings, while the reported image-generation comparison emphasizes sample quality at matched stream length.

5.1 Classification

  • At matched budgets, classification SC with compensation tracks QwT, and per-row MP beats uniform stream length by allocating longer streams to MLP_FC2 and shorter ones to cheap projections.Compensation lifts SC by +0.13 pt at L=256 up to +3.93 pt at L=128.Source: §5.1 · p. 7

5.2 Object Detection and Instance Segmentation

  • At 7/7, MP recovers +1.61/+2.08 AP over uniform SC, while at 8/8 uniform SC is essentially FP-equivalent on boxes and outperforms QwT 8/8.At 8/8, box AP is 61.98 for uniform SC, 62.06 for FP, and 61.20 for QwT.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 Image Generation

  • The image-generation comparison tests whether schedules preserving recognition accuracy also maintain sample quality at matched average stream length; the authors describe matching or slightly improving FP on FID as reasonable in this setting.Table 4 reports IS, sFID, KID, and FID for DiT-XL/2 class-conditional ImageNet generation.Source: §5.3 · p. 8, Table 4 · p. 9

5.4 Visual World-Model Planning

  • At the 7-bit tier, uniform SC reaches 0.800 SR while MP at the same average budget restores FP-equivalent 0.833 SR; at the ∼8-bit tier, all SC configurations reach the 0.833 plateau.The reported 0.933 SR for MP at log2192 is qualified by sampling uncertainty of ±2 episodes per seed.Source: §5.4 · p. 9, Table 5 · p. 9

6 Discussion

The evaluation establishes feasibility only within a limited set of representative configurations, and hardware benefits remain modeled rather than measured on silicon. The authors frame software-level evidence as useful for assessing SC as a possible future accelerator direction.

Limitations

  • The evaluation covers representative backbones and few precision budgets, not exhaustive sweeps across models, seeds, or more aggressive low-bit settings.The authors therefore say robustness across a broader configuration space remains unshown.Source: §6 · p. 9
  • Hardware benefits are estimated with analytical and gate-level models rather than measured end-to-end silicon.Source: §6 · p. 9

Broader impacts

  • Software-level feasibility evidence can help decide which arithmetic paradigms warrant the design effort of specialized hardware.The paper presents dense, temporally controlled SC precision as a promising direction for future accelerator design.Source: §6 · p. 9

7 Conclusion

The conclusion presents stochastic computing as a dense adaptive-precision substrate for transformer inference. It pairs a GPU matrix-multiplication library with per-row mixed precision and reports competitive accuracy across several vision tasks.

7 Conclusion

  • SC is framed as a dense adaptive quantizer whose precision is controlled by stream length rather than a small set of fixed word sizes.Source: §7 · p. 10
  • The work contributes a fused GPU library for SC matrix multiplication on modern vision transformers and a per-row mixed-precision algorithm that uses no retraining.The algorithm turns row-level sensitivity into a calibrated stream-length budget.Source: §7 · p. 10
  • Across classification, detection and segmentation, class-conditional generation, and visual world-model planning, SC retains near-lossless accuracy.Source: §7 · p. 10

A.1 Correlation and the C-BSG Construction

Stochastic cross-correlation is a source of bias in SC multiplication because XNOR agreement depends on the joint distribution of its input streams. C-BSG enforces zero correlation by construction, making the finite-length XNOR multiplier bias-free.

A.1 Correlation and the C-BSG Construction

  • Non-zero stochastic cross-correlation biases the SC estimator because bitwise XNOR agreement depends on joint probabilities, not only input marginals.The independence assumption behind the estimator is identified as the principal practical source of SC error.Source: §A.1 · p. 14
  • SCC normalizes ad − bc by its maximum attainable magnitude given the two stream marginals, reaching ±1 for perfect correlation or anti-correlation.The bit-pair counts a, b, c, and d correspond to the four possible values of the two streams.Source: §A.1 · p. 14
  • Scrambled Sobol sequences reduce |SCC| but do not eliminate finite-length correlation, which can still bias the estimator.Their stated discrepancy rate is O(L^-1 log^d L), compared with O(L^-1/2) for i.i.d. samples.Source: §A.1 · p. 14
  • C-BSG gates the second stream’s RNG update on the first stream so its conditional distribution matches its marginal, enforcing zero correlation and a bias-free finite-L XNOR multiplier.Substituting the resulting conditional probabilities yields ad − bc = 0 and SCC = 0.Source: §A.1 · p. 14

A.2 Residual Compensation in the SC Datapath

The paper adapts per-block residual compensation to SC, fitting corrections against deterministic SC error and implementing those corrections within the SC datapath. The authors present this as a feasibility recipe, while leaving its stability and operating boundaries unresolved.

A.2 Residual Compensation in the SC Datapath

  • The SC recipe uses 1,024 calibration images, ridge regularization, per-head fits, and a disjoint cross-seed cosine gate for admitting Wb.The regularization is λ=10^-4; the gate addresses fits that may capture calibration noise and hurt held-out accuracy.Source: §A.2 · p. 15
  • The compensated network uses the same per-block residual structure as QwT, with SC-specific fitting and admission modifications.QwT attaches a lightweight residual compensation module to each block of an already-quantized network.Source: §A.2 · pp. 14–15
  • SC compensation remains inside the SC datapath: each compensated block adds an auxiliary SC matmul at fixed Lcomp=256 rather than an FP correction.The fitted FP-valued Wb is materialized as an SC linear layer; the added cost is one auxiliary d×d SC matmul.Source: §A.2 · p. 15
  • Fixed Sobol configurations make SC error deterministic across forward passes, so compensation fits against the same residual encountered at inference.The approach uses the same Sobol configuration during calibration and inference, allowing Wb to absorb a fixed perturbation.Source: §A.2 · p. 15
  • The authors frame compensation as a feasibility demonstration, not a formal analysis of its stability or optimality.They do not establish uniqueness or stability across Sobol configurations, optimality of the cosine gate, or the profitable range for Lcomp<256.Source: §A.2 · pp. 15–16

A.3 Image Generation Samples

Figure 4 presents class-conditional image-generation samples from full precision and SC at several uniform stream lengths.

A.3 Image Generation Samples

  • Figure 4 compares full-precision samples with SC samples generated at uniform L ∈ {32, 48, 64, 96, 128}.The prompts are “golden retriever”, “otter”, “lesser panda”, “geyser”, “macaw”, “valley”, “balloon”, and “arctic fox”.Source: Figure 4 · p. 17, §A.3 · p. 17
  • The eight sample prompts span animals, a geyser, a valley, and a balloon.They are golden retriever, otter, lesser panda, geyser, macaw, valley, balloon, and arctic fox.Source: Figure 4 · p. 17
  • The figure’s comparison is across full precision and SC stream lengths, not across different prompt sets.Each listed prompt is part of the sample set shown in Figure 4.Source: Figure 4 · p. 17, §A.3 · p. 17

Related papers

Loading 2609.32922v1…