FORGE: Forward-Only Test-Time Adaptation for Integer-Only Vision Models on Microcontrollers

Muhammad Rehan; Haider Ali; Muhammad Ali Munir; Moaz Amjad

Source-linked AI summary

FORGE: Forward-Only Test-Time Adaptation for Integer-Only Vision Models on Microcontrollers

Muhammad Rehan, Haider Ali, Muhammad Ali Munir, Moaz Amjad

arXiv:2609.01683v1Sep 1, 2026cs.CVcs.ARcs.LG

TL;DR

Microcontroller deployment folds batch normalization and quantizes convolutions, removing the statistics and runtime machinery needed by existing adaptation methods. Forge restores forward-only adaptation through per-channel recalibration around deployed int8 convolutions, recovering most of TENT’s gain and costing 6.8% of inference energy on an ESP32-S3.

Problem
Microcontroller vision models lack usable batch-normalization statistics and backpropagation machinery for adapting to field distribution shifts in inference-only integer runtimes.
Method
Forge applies forward-only per-channel recalibration to folded convolution outputs, targeting clean training statistics with forward-pass estimates while retaining int8 convolutions.
Results
+20.9 points recovered versus +24.9 points for gradient-based TENT, while 3 of 21 layers recover 93% of the benefit and deployment costs 6.8% of inference energy.
Takeaways & Limitations
Forward-only test-time adaptation is practical on deployed microcontroller models, including single-sample streaming and real ESP32-S3 integer convolution execution.
Takeaways & Limitations
Forge targets folded Conv→BN sites and uses lightweight fp32 recalibration, so it is not an end-to-end integer-only adaptation path; on-device measurement covers ResNet-20 while the second architecture is simulated.

Abstract

from arXiv · show

Vision models deployed on microcontrollers (MCUs) are quantized to integer-only arithmetic and run in inference-only runtimes that do not carry the machinery backpropagation needs: the standard tool for adapting a model to the distribution shift (sensor noise, blur, lighting) it meets in the field. Existing forward-only test-time adaptation (TTA) methods either run only on server- or edge-GPU-class models (not true microcontroller integer execution), or require the batch-normalization (BN) layers that integer deployment fuses away. We present a forward-only TTA method that operates on deployed, BN-folded, integer-only convolutional networks. The key observation is that fusing BN into the preceding convolution, a mandatory step for integer inference, destroys the statistics that normalization-based adaptation relies on. We restore adaptation by re-normalizing each folded convolution's per-channel output to its clean training statistics, using only forward-pass estimates. The method (i) recovers most of gradient-based TENT's accuracy gain (+20.9 vs. +24.9 points) and matches forward-only BN adaptation, while being the only method that runs on a folded integer-only model; (ii) needs to adapt only 3 of 21 layers (selected without seeing the test corruptions) to recover 93% of the benefit; (iii) survives single-sample streaming with a batch-size-scaled momentum; and (iv) generalizes across three datasets (up to 200 classes) and two architectures. We validate bit-exact int8 convolution execution and deploy on an ESP32-S3, where, measured with a Nordic PPK2 power profiler, the forward-only adaptation (a lightweight fp32 recalibration around the int8 convolutions) costs only 8.3 mJ (6.8% of inference energy) and 21.9 ms on the deployed SIMD-optimized model: forward-only adaptation is cheap on a real microcontroller.

1 Introduction

Microcontroller deployment folds away the normalization machinery that test-time adaptation needs, while Forge restores forward-only adaptation on the deployed integer model with low measured cost. It recovers substantial accuracy using few layers and supports streaming and cross-dataset evaluation.

Deployment gap: Folding batch normalization into int8 convolutions removes the statistics needed for forward-only adaptation, leaving deployed models unable to adapt out of the box.This deployment step is mandatory for efficient integer inference but creates an adaptation gap.
Forge: Forge restores adaptation through forward-only per-channel recalibration of folded convolution outputs to clean training statistics, without gradients or learnable parameters.The method uses running per-channel statistics and retains the clean target values from folding.
Selective adaptation: 3 of 21 layers recover 93% of the benefit when selected using held-out corruptions without accessing the test stream.The selection is performed offline before deployment.
Robustness and scope: Forge supports single-sample streaming with batch-size-scaled momentum and generalizes across three datasets and two architectures.The datasets span up to 200 classes.
Measured deployment: +20.9 accuracy points are recovered on an ESP32-S3, with 8.3 mJ additional energy, equal to 6.8% of inference energy, and 21.9 ms latency.The deployment uses bit-exact int8 convolutions with lightweight fp32 recalibration on ESP-NN SIMD kernels.

2 Related Work

Prior test-time adaptation methods either require gradients, live normalization layers, or hardware beyond true MCU integer execution. Forge instead operates on the deployed folded int8 model and reports measured MCU energy.

Gradient-based TTA: Gradient-based TTA methods require backpropagation machinery that inference-only microcontroller runtimes do not provide.On-device training can restore backpropagation, but with substantial memory and compute costs.
Forward-only normalization adaptation: Forward-only normalization methods depend on live BN statistics or cached feature representations that integer deployment removes.Folding deletes addressable BN statistics, while feature banks are not shipped with the deployed model.
Quantized-model TTA: FOA, ZOA, and PACE target GPU or edge-GPU hardware, while PEA does not address quantization.Their demonstrated settings therefore do not establish forward-only adaptation under MCU integer execution.
MCU TTA: TinyTTA is validated on an MCU but remains gradient-based and adapts before BN folding, so it does not run on the deployed folded integer model as-is.Its approach reduces backpropagation memory rather than eliminating gradients.
Positioning: Forge is the only compared method that is forward-only, runs on a deployed folded int8-convolution model on a microcontroller, and reports measured energy.The comparison distinguishes deployment compatibility from merely reducing training or adaptation memory.

3 Method

Forge restores forward-only adaptation after BN folding removes the statistics that BN-based recalibration needs, using per-channel forward-pass estimates around folded convolutions. It preserves deployability on int8-convolution models while keeping recalibration lightweight and selective.

Folded integer-only deployment: BN folding removes the running statistics needed by normalization adaptation, reducing forward-only BN recalibration from +20.1 mean accuracy points to +0.0 on CIFAR-10-C.The corruption error is identical, and the BN-preserving model is not deployable in integer-only form.
Forge recalibration: Forge restores adaptation by reintroducing a gradient-free per-channel correction at each former BN site.It retains the clean per-channel targets βc and |γc| recorded when folding, without requiring live BN layers or a feature bank.
Forge recalibration: Forward-pass recalibration maintains exponential moving averages of shifted activation means and variances, then renormalizes each channel to its clean target.The algorithm uses no gradients or learnable parameters and adds two reductions plus an affine operation per activation.
Implementation: Forge dequantizes int8 convolution outputs to fp32 for recalibration, then requantizes them to int8 for the next convolution.The only added state is 2C fp32 scalars per site, so the method is mixed-precision rather than strictly integer arithmetic throughout.
Selective and streaming adaptation: Adapting only the most important layers is cheaper and more accurate than adapting all layers, because per-layer recovery ranges from +0.4 to +17 points.Single-sample streaming uses momentum scaled with batch size to preserve the effective averaging window.
Mechanism on real activations: On a real deployed int8 channel, Forge reduces corruption displacement from 1.8σ off the clean mean to within 0.06σ while recovering the spread.The measurement uses layer2.0.bn1 under CIFAR-10-C Gaussian noise.

4 Experiments

Experiments show Forge recovers accuracy across corruptions, datasets, architectures, precisions, and streaming regimes while remaining deployable on folded int8 microcontroller models. Selective layers, scaled momentum, and lightweight on-device execution preserve effectiveness with limited overhead.

4.1 Comparison to baselines: +20.9 versus +24.9 points: Forge recovers most of TENT’s gain, matches forward-only BN adaptation, and uniquely runs on the deployed folded integer-only model.The comparison uses recovery on CIFAR-10-C; TENT and BN-adapt cannot run on the folded integer-only model as deployed.
4.2 Selective-layer recalibration: 93% of the full benefit is recovered by adapting the top 3 of 21 layers selected on held-out corruptions, while the best subset exceeds adapting all layers.The held-out ranking transfers to unseen corruptions; eight selected layers peak at +19.6 versus +18.4 for all 21.
4.3 Streaming: Batch-size-scaled momentum preserves recovery at single-sample streaming, whereas fixed momentum collapses at batch size 1.Window-matched momentum uses m = bs/640; fixed momentum remains effective only down to batch 4.
4.4 Generalization: Forge generalizes across ResNet-20 and MobileNetV2 and across CIFAR-10-C, CIFAR-100-C, and Tiny-ImageNet-C, recovering +11.0 points on the 200-class benchmark.The method applies to depthwise and pointwise layers and improves all 15 Tiny-ImageNet-C corruptions.
4.5 Ablations: Mean recovery changes by under 0.8 points as EMA momentum sweeps from 0.01 to 1.0, peaking at m=0.1.This supports using one default momentum without per-deployment tuning.
4.5 Ablations: Forge continues recovering across reduced precision, including +23.3 with 4-bit weights and +13.3 at 4/4-bit weights and activations.At 4/4 precision, quantization alone reduces clean accuracy to 60.6%, while the recalibration mechanism still operates.
4.6 When to adapt: a safety gate: The safety gate degrades no corruption across three benchmarks but becomes increasingly conservative as class count grows, firing on 0 of 15 Tiny-ImageNet-C streams.On CIFAR-100-C, gated recovery falls from +14.4 to +5.9; on Tiny-ImageNet-C, it forfeits the ungated +11.0 recovery.
4.7 On-device deployment: On ESP32-S3, adaptation adds 8.3 mJ, or 6.8% of inference energy, and 21.9 ms around SIMD-optimized int8 convolution.The recalibration runs on physical hardware with measured fp32 statistics and bit-exact int8 convolution execution.

5 Limitations

Forge’s recalibration is intentionally close to BN-statistic adaptation rather than introducing a new adaptation principle. Its scope is bounded by mixed precision, BN-folded convolutional sites, and limited on-device validation.

Forge’s recalibration mechanism is close to BN-statistic adaptation; its contribution is deployment on folded integer-only models and on-device cost measurement.
The adaptation is not end-to-end integer-only because int8 convolutions surround fp32 recalibration on the microcontroller’s FPU.
On-device measurement is reported for ResNet-20, while the second architecture is evaluated in simulation.
Forge targets folded Conv→BN sites in convolutional backbones and does not directly cover transformer backbones with unfused LayerNorm.

6 Conclusion

BN folding and integer quantization silently remove a deployed microcontroller model’s ability to adapt to field distribution shifts. Forge restores adaptation with forward-only per-channel recalibration and reports practical performance, streaming, generalization, and hardware-cost results.

Forge restores adaptation on deployed int8-convolution models with lightweight fp32 per-channel correction around the integer convolutions.
Forge matches gradient-based adaptation in accuracy, needs only a few layers, survives single-sample streaming, and generalizes across datasets and architectures.
6.8% of inference energy is the measured cost of forward-only adaptation on real microcontroller hardware.

A Why recalibration cancels the corruption, and the variance it omits

Forge restores clean per-channel output statistics after folded Conv→BN sites by estimating shifted moments forward-only and rescaling activations to fold-time targets. Its cancellation is exact under an affine channel shift, while batch variance omission creates a batch-size-dependent error controlled by momentum.

Setup: At a folded Conv→BN site, Forge records clean output mean βc and standard deviation |γc| for each channel.The clean normalized pre-activation has zero mean and unit variance on the training distribution.
Exact cancellation under an affine channel shift: Under an affine corruption ỹc = acyc + dc, shifted moments become acβc + dc and a_c^2|γc|^2, which Forge estimates from forward passes.The running estimates converge to the shifted mean and variance on a stationary stream.
Exact cancellation under an affine channel shift: Forge’s standardization removes the corruption scale and offset, recovering the clean output without knowing the corruption coefficients.The method needs only the fold-time targets (βc, |γc|).
The variance the EMA omits: The variance tracked by Forge’s EMA omits the between-batch variance term from the law of total variance.The omitted component is Var(E[ỹc | Batch]).
The variance the EMA omits: 1/√(1−f)−1 ≈ f/2 is the residual scaling error when the omitted variance is fraction f of total variance.
The variance the EMA omits: At B=64, the omitted term is ≈0.1% and the scaling error ≈0.05%; at single-sample streaming, they reach ≈8% and ≈4.7% on Gaussian noise.The omitted term grows as 1/B, motivating window-matched momentum m = bs/640.