PaperlayerView exact revision on arXiv

Source-linked AI summary

IMC-CLINIC: Coupled Loss-Informed Newton Iterations for Clipping in Analog In-Memory Computing

Yung-Chin Chen, Chia-Yu Chen, Naveen Verma

arXiv:2609.35586v1cs.AI

TL;DR

Analog IMC can make LLM inference more energy-efficient, but ADC quantization adds output error and higher ADC resolution comes at rapidly increasing energy cost. IMC-CLINIC combines an analytical output-error surrogate with safeguarded Newton optimization to calibrate clipping. Across models and datasets, it improves average zero-shot accuracy over grid search while reducing calibration time.

  • Problem

    ADC quantization adds output-side error in analog IMC, while increasing ADC resolution to mitigate it comes at rapidly increasing energy cost.

    Source: §1 · p. 2

  • Method

    IMC-CLINIC couples an analytical surrogate for IMC MatMul output error with safeguarded Newton-type optimization of clipping factors.

    Source: §1 · p. 2, §4 · p. 6

  • Results

    Across models and datasets, IMC-CLINIC improves average zero-shot accuracy by 6.5–11.5 percentage points over grid search while reducing calibration time by 10.0×–12.1×.

    Source: §1 · p. 2

  • Takeaways & Limitations

    IMC-CLINIC provides an efficient and accurate clipping-calibration framework for LLM inference on analog IMC.

    Source: §6 · p. 9

    The approximate Hessian omits density-sensitive second-order terms because boundary-density estimation can be costly and noisy near distribution tails.

    Source: §4 · p. 6

Abstract

from arXiv · show

Analog in-memory computing (IMC) offers a promising path toward energy-efficient large language model (LLM) inference by executing matrix multiplications (MatMul) directly within memory arrays in the analog domain. Its efficiency, however, comes with an additional source of error: limited-precision analog-to-digital converters (ADCs) quantize accumulated analog partial sums, introducing output-side error distinct from conventional activation and weight quantization at the MatMul inputs. Clipping can mitigate both operand and ADC quantization errors, but the optimal clipping factors must jointly balance activation rounding and clipping, weight rounding and clipping, and ADC quantization. Existing clipping methods, designed for digital quantization, do not explicitly optimize these coupled sources of IMC error and often rely on costly search-based calibration. We introduce IMC-CLINIC (Coupled Loss-Informed Newton Iterations for Clipping), a clipping calibration framework based on an analytical surrogate for IMC MatMul output error. The surrogate jointly models operand quantization, accumulated clipping-induced bias, and ADC quantization, enabling efficient evaluation of its gradient and approximate curvature from a small calibration set. IMC-CLINIC jointly optimizes activation and weight clipping factors using a safeguarded Newton-type method. Across multiple models and datasets, it improves average zero-shot accuracy by 6.5-11.5 percentage points over the grid search baseline while reducing calibration time by factors of 10.0-12.1. Its analytical surrogate closely tracks empirical IMC output error, and its optimizer is certified within 1% of the global optimum under the loss objective across all projections on two representative models.

1 INTRODUCTION

Analog IMC reduces energy-intensive matrix multiplication costs but introduces ADC quantization error that is difficult to address without sacrificing efficiency. IMC-CLINIC jointly models the coupled clipping errors and optimizes clipping factors, improving accuracy while shortening calibration.

1 INTRODUCTION

  • ADC quantization of accumulated analog partial sums adds output-side error, while increasing ADC resolution raises energy cost and diminishes IMC’s efficiency advantage.This motivates algorithmic accuracy improvements at fixed ADC resolution.Source: §1 · pp. 1–2
  • Clipping must jointly balance activation and weight rounding and clipping errors with ADC quantization, which existing digital-oriented methods do not explicitly optimize.Common approaches also often rely on costly search-based calibration.Source: §1 · p. 2
  • IMC-CLINIC combines an analytical MatMul output-error surrogate with safeguarded Newton optimization of activation and weight clipping factors.The surrogate models operand errors, accumulated clipping-induced bias, and ADC error while enabling efficient gradient and curvature evaluation.Source: §1 · p. 2
  • 6.5–11.5 percentage points higher average zero-shot accuracy than W/A grid search, with calibration accelerated 10.0×–12.1×.Reported for a 9-bit ADC IMC accelerator across multiple models and datasets.Source: §1 · p. 2

2 BACKGROUND

Analog IMC performs most inner-product computation in memory, but ADC conversion of partial sums limits precision. Clipping trades rounding error against saturation and, in IMC, also changes ADC error through the operand scales.

2.1 ANALOG IMC

  • Analog IMC accumulates products within memory tiles, digitizes tile-level partial sums, and digitally accumulates the tile contributions.The inner-product reduction is performed largely within memory; the passage gives 128 INT8-equivalent TOPS/W versus 5 TOPS/W for digital INT8 MAC at the same 0.8 V supply.Source: §2.1 · p. 3
  • ADC conversion of analog partial sums is the accuracy bottleneck because a fixed-resolution ADC must resolve a diminishing share of its range as accumulation grows.Under the stated first-order model, signal swing occupies an O(1/√h) fraction of the full range, motivating dynamic-range reduction without increasing ADC resolution.Source: §2.1 · p. 3

2.2 CLIPPING FOR LLM QUANTIZATION

  • Narrowing a fixed-bit quantizer’s clipping interval reduces in-range rounding error but increases saturation and clipping distortion.The optimal threshold balances these opposing effects.Source: §2.2 · p. 3
  • Activation thresholds use bounded retention factors on dynamic positive and negative ranges, while symmetric weight clipping uses a factor on the fixed weight range.The activation factors remain fixed while the observed per-sample ranges vary; weight clipping is static.Source: §2.2 · p. 3
  • In IMC, clipping also changes output-referred ADC error through operand scales, so activation and weight thresholds must be jointly calibrated with operand errors.Prior work typically treats operand clipping separately or modifies the ADC range directly.Source: §2.2 · p. 4

3 COUPLED ANALYTICAL OUTPUT-ERROR MODEL

The output-error model estimates clipping-dependent operand errors from calibration samples, propagates them through the MatMul, and combines them with ADC error. It retains coherent clipping bias while using approximations to make the coupled objective tractable.

3 COUPLED ANALYTICAL OUTPUT-ERROR MODEL

  • Operand clipping MSE is modeled from distribution tails, with required statistics estimated from calibration activations and pretrained weights.The paper rewrites tail integrals as sample-estimable indicator-function expectations and computes rounding error using the Δ²/12 model.Source: §3 · p. 4
  • The model propagates per-coordinate quantization errors through the MatMul, assuming centered MAC errors are approximately pairwise uncorrelated to avoid cross-coordinate covariance estimation.A second-moment separability approximation retains leading signal–noise contributions while omitting mixed and higher-order interactions.Source: §3 · pp. 4–5
  • The surrogate retains the signed mean clipping error because its bias can accumulate coherently across the MAC reduction.Expressing operand errors in output space also permits direct combination with output-referred ADC error.Source: §3 · p. 5
  • The complete objective combines operand-induced output error with ADC quantization error mapped through operand scales and summed across independently digitized row-tile partial sums.It assumes operand-induced and ADC errors are approximately uncorrelated; activation, weight, and signed-error terms depend on their corresponding clipping factors.Source: §3 · p. 5
  • The resulting surrogate captures coupled activation and weight rounding, clipping, and ADC effects while remaining structured for efficient optimization.Source: §3 · p. 5

4 CLIPPING CALIBRATION WITH SAFEGUARDED NEWTON ITERATIONS

The calibration method evaluates analytical gradients and approximate curvature from calibration statistics, then uses safeguarded Newton updates. Its safeguards require descent and sufficient decrease, while the approximate Hessian omits density-sensitive terms.

4 CLIPPING CALIBRATION WITH SAFEGUARDED NEWTON ITERATIONS

  • Across evaluated projections, the initialization, accepted trajectory, and calibrated solution lie within one connected positive-semidefinite Hessian region.The paper reports global ε-optimality under the calibration objective across evaluated projections.Source: §4 · pp. 6–7
  • Analytical gradients of the coupled clipping objective can be evaluated directly from calibration statistics without dense clipping search or repeated MatMul reconstruction.Indicator-based error statistics permit differentiation with respect to the clipping factors.Source: §4 · p. 6
  • The approximate Hessian omits density-sensitive second-order terms because boundary-density estimation is costly and can be noisy near distribution tails.The surrogate loss and analytical gradient remain unchanged; the approximation affects only the Newton direction.Source: §4 · p. 6
  • Accepted projected Newton updates must be descent steps that satisfy Armijo sufficient decrease and preserve positive curvature, so each accepted step decreases the surrogate objective.Initialization selects a low-loss grid point with positive minimum Hessian eigenvalue, and backtracking reduces step size when safeguards fail.Source: §4 · pp. 6–7

5 EXPERIMENTS

The experiments test IMC-CLINIC under a shared analog IMC setup, examining how it balances output-error sources, improves model accuracy and calibration time, and performs as a surrogate and optimizer. Across the evaluated models, it outperforms grid search while the surrogate closely tracks empirical error and the optimizer is certified near-optimal under its objective.

5.1 IMC CONFIGURATION AND BASELINES

  • The common setup uses 8-bit quantized operands represented with 4-bit slices and a fixed 9-bit ADC; the baselines are no clipping and W/A grid search.MatMuls map to 512-row arrays, with larger reductions partitioned across arrays; all methods use the same ADC full-scale range.Source: §5.1 · p. 7, §5 · p. 7

5.2 OUTPUT-ERROR ANALYSIS ACROSS ERROR SOURCES

  • IMC-CLINIC achieves the lowest total output MSE across projection types by trading somewhat larger operand quantization errors for reduced ADC error.Without clipping, ADC quantization dominates total output error; optimized clipping factors typically fall in the 0.5–0.7 range.Source: §5.2 · p. 7, Figure 2 · p. 8

5.3 MODEL ACCURACY AND CALIBRATION TIME

  • 15.9–57.9% lower perplexity and 6.5–11.5 percentage points higher average zero-shot accuracy than W/A grid search across four models; calibration is also 10.0×–12.1× faster.Table 1 reports WikiText-2 PPL, mean seven-task accuracy, and calibration time.Source: Table 1 · p. 8, §5.3 · p. 8

5.4 FIDELITY OF SURROGATE LOSS

  • IMC-CLINIC’s surrogate has a better WikiText-2 perplexity–calibration-time trade-off than direct ADC-aware alternating search, with median mismatch to empirical output MSE of about 2–4%.The surrogate retains an analytical objective instead of repeatedly evaluating empirical MSE.Source: §5.4 · pp. 8–9

5.5 OPTIMIZATION EFFICIENCY AND SOLUTION OPTIMALITY

  • The safeguarded Newton optimizer reaches 95% of post-initialization loss improvement in a median 72 ms, versus 1.10 s for projected gradient descent.Branch-and-bound validation certifies its solutions as globally 1%-optimal under the surrogate objective across evaluated projections in LLaMA-3.2-3B and Qwen3-4B.Source: §5.5 · p. 9

6 CONCLUSION

The conclusion presents IMC-CLINIC as a clipping-calibration framework that jointly accounts for operand and ADC quantization effects. It reports accuracy gains, faster calibration, a close match between the surrogate and empirical error, and near-optimal solutions under the calibration objective.

6 CONCLUSION

  • IMC-CLINIC combines an analytical MatMul output-error surrogate with safeguarded Newton-type optimization to jointly account for activation, weight, and ADC quantization.Across four LLMs, it improves accuracy over W/A grid search while reducing calibration time by 10.0×–12.1×.Source: §6 · p. 9

AI USE STATEMENT

  • Generative AI tools supported manuscript polishing, prior-work discovery, research discussions, code and experiment scripts, results collection, and drafting that authors reviewed and revised.The statement also reports tool use in research ideation and execution.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • Supplementary code includes IMC-CLINIC, the simulator, baselines, evaluation, experiments, and optimality validation; evaluated models and datasets are publicly available.The paper says the code contains configurations and scripts used to reproduce reported tables and figures.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 ANALOG IMC ARCHITECTURES

The appendix describes analog IMC architectures and their relevance to LLM workloads, then compares the energy efficiency of switched-capacitor SRAM IMC with digital MACs. It identifies ADC quantization as the primary precision limitation in the studied high-SNR regime.

A.1 ANALOG IMC ARCHITECTURES

  • Analog IMC spans RRAM, MRAM, PCM, eDRAM, and SRAM, with differing density, programmability, and efficiency tradeoffs; analog variation, nonlinearity, and noise can limit MatMul precision.The paper focuses on switched-capacitor SRAM IMC, where ADC quantization rather than in-array analog compute noise is the primary precision limitation.Source: §A.1 · p. 17

A.2 LLM INFERENCE WORKLOADS

  • IMC’s compute-energy benefit is most direct in compute-bound prefill, while emerging workloads and accelerator data-movement optimization can increase the relevance of efficient MatMul in decode and across inference.On-chip array capacity can require weights to be streamed, but the compute-energy reduction remains beneficial even when full weight localization is infeasible.Source: §A.2 · p. 17, §A.1 · p. 17

A.3 ENERGY-EFFICIENCY COMPARISON

  • Under the paper’s normalization, switched-capacitor SRAM IMC reaches approximately 24.9× higher energy efficiency than an 8×8-bit digital MAC.The comparison uses nominal 28-nm technologies at 0.8 V and converts the reported 1-bit-normalized IMC metric to an INT8-equivalent figure.Source: §A.3 · pp. 17–18

B EXISTING CLIPPING METHODS FOR LLM AND ANALOG IMC QUANTIZATION

Prior clipping methods calibrate operand ranges or ADC ranges; the C.1 formulation derives operand-error statistics for asymmetric activations and symmetric weights.

B.1 CLIPPING CALIBRATION FOR LLM QUANTIZATION

  • Prior methods that optimize both operands generally do not directly optimize their coupled contribution to MatMul output error, unlike IMC-CLINIC.Approaches include learned thresholds, independent per-tensor optimization, alternating range optimization, and reconstruction-based search.Source: §B.1 · p. 18, B EXISTING CLIPPING METHODS FOR LLM… · p. 18

B.2 CLIPPING AND RANGE OPTIMIZATION FOR ANALOG IMC

  • Hardware-aware retraining models ADC/DAC quantization and PCM nonidealities, but requires iterative forward and backward optimization and incurs high calibration cost.Source: §B.2 · p. 19
  • ADC-input clipping methods narrow partial-sum ranges, but algorithm-level optimization is constrained by circuit reference voltages and noise-dependent modeling needs.Narrower ADC steps make analog noise a larger fraction of each quantization step.Source: §B.2 · p. 19

C.1 QUANTIZATION-ERROR DECOMPOSITION

  • For asymmetric activations, averaging the fractional zero-point offset over U(−1/2, 1/2) smooths the rounding model despite displaced boundary reconstruction levels.The resulting activation rounding contribution covers the full distribution, including values mapped to clipping boundaries.Source: §C.1 · p. 19
  • Symmetric weight quantization aligns clipping thresholds with extreme reconstruction levels, so rounding applies only to weights within the clipping range.The weight-rounding term is weighted by retained probability mass.Source: §C.1 · p. 20
  • Clipping can introduce signed error that accumulates coherently across a MatMul reduction, so its mean and second moments are retained in the operand statistics.Those statistics are estimated from calibration samples and feed the output-space surrogate.Source: §C.1 · p. 20

C.2 MATMUL OUTPUT-ERROR SURROGATE

This section validates the analytical MatMul output-error surrogate and derives the gradients and curvature used for calibration, then describes the safeguarded Newton procedure and evaluation protocol for assessing IMC-CLINIC.

C.2 MATMUL OUTPUT-ERROR SURROGATE

  • O(d^2) cross-coordinate joint statistics are replaced by per-coordinate moments and reductions under the pairwise-decorrelation approximation, enabling efficient surrogate and derivative evaluation.Centered MAC-error correlations were small for the representative layer-13 q projection (median |r| = 0.017; 95th-percentile |r| = 0.071), and the approximation remained close to exact operand MSE across projections and layers.Source: §C.2 · p. 21
  • 95.5% of the exact second moment is captured by the two retained terms in the representative layer-13 q projection; across 28 layers, median retained-to-exact ratios are approximately 93–96%.The remaining four terms involve mixed activation–weight error products or higher-order moments; the roughly 5% mismatch is presented as a trade-off for a simpler surrogate with a more benign optimization landscape.Source: §C.2 · p. 22, Figure 5 · p. 22
  • 45–50% lower predicted operand-output MSE without Lbias, especially for q and k projections, supports retaining the signed-bias term in the surrogate.Structured outliers can cause repeated same-direction clipping at a few MAC coordinates, whose signed errors accumulate across the MatMul reduction.Source: §C.2 · p. 22, Figure 6 · p. 23
  • r = −0.001 for operand-induced and additional ADC error in the representative layer-13 q projection, with correlations near zero across 28 layers, supporting additive MSE contributions.The ADC error derives from four independently digitized 4-bit slice products; their reconstructed variances add across row tiles.Source: §C.2 · pp. 22–23, Figure 7 · p. 24

C.3 GRADIENT AND HESSIAN DERIVATIONS

  • Analytical derivatives use clipping-threshold tail probabilities and boundary densities, while mixed activation–weight curvature enters through the signed-bias and ADC terms.Calibration retains the full analytical gradient but uses an approximate Hessian that omits density-sensitive second-order terms while preserving the remaining curvature.Source: §C.3 · pp. 24, 26, 27, 28

C.4 EMPIRICAL VALIDATION OF THE PSD REGION

  • Every evaluated projection of LLaMA-3.2-3B and Qwen3-4B has a single connected PSD component containing initialization, the optimization trajectory, and the calibrated solution.Representative LLaMA-3.2-3B decoder-block-13 landscapes show the λmin = 0 boundary separating the PSD region from indefinite-curvature configurations.Source: §C.4 · p. 28, Table 2 · p. 28, Figure 8 · p. 28

D SAFEGUARDED NEWTON OPTIMIZATION

  • IMC-CLINIC combines the analytical gradient and approximate Hessian in projected Newton updates with backtracking line search, initialized from coarse candidate points.The procedure includes an initialization criterion, curvature safeguards, projected-step acceptance conditions, and a stopping criterion.Source: D SAFEGUARDED NEWTON OPTIMIZATION · p. 29

D.1 INITIALIZATION AND CURVATURE SAFEGUARDS

  • A 16-candidate initialization bank uses equal activation clipping factors and selects the lowest-loss candidate with positive-definite approximate curvature.The safeguard maintains positive-definite local curvature along accepted iterates; projected backtracking and sufficient decrease address the fact that a full Newton step may not reduce the constrained nonlinear objective.Source: §D.1 · p. 29

D.2 PROJECTED UPDATES, BACKTRACKING, AND STOPPING

  • Projected Newton updates are clipped to box constraints and accepted only when they provide descent, satisfy Armijo decrease, and preserve positive curvature; failed trials trigger geometric backtracking.Convergence checks both relative loss change and accepted parameter displacement after warmup, avoiding false stopping when projection or backtracking repeatedly reduces the raw step.Source: §D.2 · p. 30

E.1 LLM EVALUATION SETUP

  • Experiments cover four LLaMA and Qwen models, WikiText-2 perplexity, and seven zero-shot benchmarks; calibration uses eight 2048-token WikiText-2 training windows shared across clipping methods.All quantized comparisons apply the same fixed Hadamard rotation configuration.Source: §E.1 · p. 31

E.2 ERROR-SOURCE ANALYSIS SETUP

  • Error-source analysis evaluates all clipping methods on real LLaMA-3.2-3B activations across attention and MLP projections, using shared full-precision inputs and rotated weights.Eight non-overlapping WikiText-2 validation sequences of length 2048 are evaluated across all 28 decoder layers.Source: §E.2 · p. 31
  • Output-space normalized MSE compares activation, weight, ADC, and total IMC errors on a common scale, while total error includes interactions and is not their additive sum.Errors are accumulated across eight validation sequences and output channels; Figure 2 reports median normalized MSE across 28 decoder layers with 25th–75th percentile shading.Source: §E.2 · pp. 31–32

E.3 QUANTIZATION AND IMC CONFIGURATION

  • The IMC setup uses A8 activations, W8 weights, a 9-bit ADC, and 512-row arrays; larger reductions are split across arrays and accumulated digitally.Each 8-bit operand is decomposed into two 4-bit slices, yielding four 4-bit-by-4-bit partial products; the ADC range is fixed across clipping methods.Source: §E.3 · p. 32

E.4 GRID-SEARCH BASELINE IMPLEMENTATION

  • The W/A grid-search baseline uses PrefixQuant-style clipping searches but evaluates decoder-block reconstruction loss through the analog IMC emulator, making it partially ADC-aware without explicitly optimizing coupled errors.Activation upper and lower clipping factors use a 9×9 grid; weight scales use the activation-aware group-wise search.Source: §E.4 · pp. 32–33

E.5 ADC-AWARE ALTERNATING SEARCH BENCHMARK IMPLEMENTATION

  • The ADC-aware alternating-search benchmark directly minimizes empirical IMC output MSE, alternating activation and weight clipping searches over a finite grid for five iterations.Candidates are evaluated through the complete ADC-enabled IMC path, trading search resolution against calibration cost.Source: §E.5 · p. 33

E.6 SURROGATE FIDELITY EVALUATION

The analytical surrogate closely tracks measured IMC output error, and IMC-CLINIC’s safeguarded Newton updates reach the target faster than projected gradient descent. Across additional models and hardware settings, its clipping balances operand and ADC error, with performance varying by ADC precision and array dimension.

E.6 SURROGATE FIDELITY EVALUATION

  • Approximately 2–4% median mismatch across projection types separates the complete surrogate from empirical IMC output MSE.Most approximation error comes from clipping; rounding and ADC terms are modeled particularly closely.Source: §E.6 · p. 33, Figure 9 · p. 34

E.7 PROJECTED GRADIENT DESCENT IMPLEMENTATION

  • 72 ms median time lets IMC-CLINIC reach the target on 112/112 problems, versus 1.10 s among projected GD’s 87/112 successful runs.The comparison holds the surrogate objective, initialization, feasible domain, and calibration data fixed.Source: §E.7 · p. 34

F.1 ERROR-SOURCE ANALYSIS ACROSS MODELS

  • Across three additional models, IMC-CLINIC accepts somewhat larger operand quantization error to reduce ADC error, yielding lower total output error than clipping baselines.Figure 10 reports normalized MSE components and total output error across projection types.Source: §F.1 · p. 35, Figure 10 · p. 36

F.2 OPTIMIZED CLIPPING FACTORS

  • Optimized activation and weight clipping factors are typically approximately 0.5–0.7, retaining roughly 50–70% of operand ranges.This corresponds to reducing retained ranges by roughly 30–50% relative to no clipping.Source: §F.2 · p. 35, Figure 11 · p. 37

F.3 FULL TASK-LEVEL ACCURACY RESULTS

  • Table 3 reports WikiText-2 perplexity, seven-task zero-shot accuracy, and calibration time; its W8A8 no-IMC reference isolates ADC-quantization degradation.Average accuracy is the unweighted mean across seven tasks; the digital reference stays close to FP16 across the tested models and configurations.Source: §F.3 · p. 35, Table 3 · p. 35

F.4 ADC PRECISION

  • At 8 and 9 bits, clipping improves inference accuracy over No Clipping and W/A Grid Search; at 10 bits, the gap narrows, but IMC-CLINIC still has the best perplexity and accuracy.The benefit is largest at lower ADC precision, where ADC quantization is more severe.Source: §F.4 · p. 35, Table 4 · p. 37

F.5 IMC ARRAY DIMENSIONS

  • Across row counts from 128 to 2048, IMC-CLINIC outperforms clipping baselines, with a larger advantage at larger reduction dimensions where ADC quantization is more severe.The surrogate captures the effect of row count on accumulated partial-sum range and ADC error.Source: §F.5 · p. 37
  • Changing the number of array columns produces nearly identical results because columns affect parallel output channels, not per-column reduction or ADC quantization.Table 6 reports results across column counts.Source: §F.5 · p. 37

F.6 ROBUSTNESS TO ADC ANALOG NOISE

The robustness study adds zero-mean Gaussian noise before ADC quantization and tests several noise magnitudes. IMC-CLINIC continues to outperform baseline clipping methods across the evaluated levels.

F.6 ROBUSTNESS TO ADC ANALOG NOISE

  • IMC-CLINIC consistently outperforms baseline clipping methods across evaluated ADC analog-noise levels, although increasing noise reduces accuracy for all clipping methods.Noise is injected independently into each of four slice-level ADC conversions before quantization.Source: §F.6 · pp. 37–38

F.7 CALIBRATION ROBUSTNESS

Calibration remains stable across the tested sample sizes, sequence lengths, and corpora, while taking less time than grid search. The safeguarded Newton optimizer also reaches near-final loss values within a small number of accepted iterations.

F.7 CALIBRATION ROBUSTNESS

  • Across 8–128 calibration samples, IMC-CLINIC maintains perplexity between 12.21 and 12.31 and average zero-shot accuracy between 0.565 and 0.579, while calibration takes 3.4–10.7 minutes.W/A Grid Search takes 32.9 minutes at 8 samples and exceeds the one-hour limit for larger sets.Source: §F.7 · p. 38, Table 8 · p. 39
  • Across sequence lengths of 512–2048, IMC-CLINIC maintains average zero-shot accuracy between 0.571 and 0.584 and takes 3.0–3.4 minutes to calibrate.W/A Grid Search time increases from 10.9 to 32.9 minutes as sequence length grows.Source: §F.7 · p. 39
  • Across WikiText-2, C4, and Pile, IMC-CLINIC produces perplexity between 12.27 and 12.32 and average zero-shot accuracy between 0.575 and 0.581, outperforming W/A Grid Search.The results indicate that calibrated clipping factors are not sensitive to the calibration corpus.Source: §F.7 · p. 39

F.8 OPTIMIZATION CONVERGENCE

  • Across 28 decoder blocks, the median accepted Newton iterations to optimizer return ranges from approximately 6 to 14, with normalized surrogate loss dropping sharply in the first few iterations.The loss quickly reaches a plateau across projection groups.Source: §F.8 · p. 39

F.9 METHOD ABLATIONS

The ablations support ADC-aware calibration, signed-bias modeling, and joint activation–weight optimization, while finer-grained weight clipping offers modest gains. The optimality validation uses a numerical 1% certificate rather than an exact global-optimality proof, relying on dependency-preserving bounds and adaptive branch-and-bound.

F.9 METHOD ABLATIONS

  • Removing the ADC term or signed-bias term degrades perplexity, while independently optimizing activation and weight clipping raises PPL to 15.80.These ablations support ADC-aware calibration, signed-bias modeling, and joint activation–weight optimization.Source: §F.9 · pp. 39–40, Table 11 · p. 41

F.10 PER-CHANNEL WEIGHT CLIPPING

  • Per-channel weight clipping modestly improves both perplexity and average downstream accuracy relative to the shared scalar weight factor.Each output channel receives its own weight-clipping factor while activation factors remain shared.Source: §F.10 · p. 40, Table 12 · p. 42

F.11 EMPIRICAL-FISHER WEIGHTING FOR SHARED-INPUT PROJECTIONS

  • Empirical-Fisher weighting improves WikiText-2 perplexity from 12.27 to 11.88 but slightly lowers average zero-shot accuracy from 0.575 to 0.568.Because the gain does not consistently transfer across downstream tasks, uniform weighting remains the default.Source: §F.11 · p. 40, Table 13 · p. 42

G.1 OPTIMALITY CRITERION AND CERTIFICATION STRATEGY

  • Exact convexity or verified-root certification is difficult because full-Hessian enclosures depend on activation and weight densities at moving clipping boundaries.The authors therefore construct a numerical ϵ-optimality certificate by lower-bounding the surrogate over the feasible domain.Source: §G.1 · p. 42
  • The validation checks each projection independently over three clipping variables, making certification less costly while imposing a stricter per-projection requirement.If shared calibrated factors are within 1% of optimal for every constituent projection, the summed grouped objective is also within 1%.Source: §G.2 · p. 42, §G.1 · p. 42

G.2 DEPENDENCY-PRESERVING LOWER BOUNDS

  • The box lower bounds preserve shared clipping-factor dependencies while bounding operand, accumulated-bias, and ADC loss terms before combining them.Affine bounds are combined and minimized at a box corner to obtain a guaranteed lower bound for branch-and-bound.Source: §G.2 · pp. 42–44

G.3 ADAPTIVE BRANCH-AND-BOUND CERTIFICATION

  • Branch-and-bound prunes boxes whose lower bound is at least 0.99 times the candidate objective and bisects unresolved boxes until a counterexample is found or none remain.When the feasible domain is fully covered by pruned boxes, the candidate is certified 1%-optimal.Source: §G.3 · p. 44

G.4 CERTIFICATE VALIDATION AND RESULTS

All evaluated projections in LLaMA-3.2-3B and Qwen3-4B are certified within 1% of the calibration objective’s global minimum. Independent checks validate the certificates, and the certification is an offline check rather than part of calibration.

G.4 CERTIFICATE VALIDATION AND RESULTS

  • All evaluated projections in LLaMA-3.2-3B and Qwen3-4B are certified within 1% of the global minimum, with no counterexamples or unresolved regions.For LLaMA-3.2-3B, this covers 196/196 projection-level problems across 28 decoder blocks and seven projection types.Source: Table 14 · p. 45, §G.4 · p. 45
  • Certificate checks verify complete, non-overlapping domain coverage, recomputed box bounds, reconstructed certificates, and sampled objective values against conservative lower bounds.These checks validate domain bookkeeping and the numerical implementation of the lower-bound procedure.Source: §G.4 · p. 45
  • The reported validation time is incurred only for offline optimality checking, not during clipping calibration or deployment.Numerical safeguards conservatively adjust interval endpoints and candidate bounds, with margins audited using extended-precision recomputation.Source: §G.4 · pp. 44–45, Table 14 · p. 45
Loading 2609.35586v1…