When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference

Ismail Erbas; Xavier Intes; Vikas Pandey

Source-linked AI summary

When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference

Ismail Erbas, Xavier Intes, Vikas Pandey

arXiv:2609.04490v1Sep 3, 2026cs.AIcs.LGphysics.opticsq-bio.QM

TL;DR

The paper asks how recurrent-state storage interfaces affect low-precision recurrent inference when a trained model is deployed with altered state precision. It isolates write-back in GRU and LSTM fluorescence-lifetime models, finding that suppressed updates can cause major degradation while temporal memory and interface-compatible training recover accuracy.

Problem
Low-precision recurrent inference lacks a complete account of how changing the state-storage interface after training affects the temporal dynamics executed by the model.
Method
The study holds trained networks fixed while replacing recurrent-state write-back, then compares temporal-memory operators, precision changes, matched training, and an independently trained LSTM.
Results
Coarse write-back suppresses persistent sub-threshold updates and degrades recurrent inference, while error feedback, residual memory, and direction memory recover substantial accuracy; LSTM tests reproduce the failure and rescue.
Takeaways & Limitations
Recurrent-state write-back is part of the temporal computation, so low-precision recurrent inference should evaluate both stored-state accuracy and the dynamics produced when states are repeatedly returned.
Takeaways & Limitations
Temporal memory is not intrinsically better than deterministic storage: the native 4-bit model is already accurate under its trained interface, and alternatives do not improve both lifetime parameters.

Abstract

from arXiv · show

Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized state is stored and returned at the next time step, so the rule used to store that state can alter subsequent computations. Here, we introduce recurrent-state write-back to denote this rule and isolate its effect in a compact GRU encoder--decoder for fluorescence lifetime imaging, a molecular imaging modality used in quantitative biological imaging. A central task is estimating two lifetime parameters, the short-lived component τ1 and the long-lived component τ2, from high-noise time-resolved fluorescence signals. Holding the trained model fixed, replacing continuous state propagation with deterministic 4-bit state storage increases estimation errors for τ1 and τ2 by approximately 70x and 300x, respectively. Failure occurs when repeated small updates remain below the write threshold, leaving the stored state nearly fixed while the network continues to propose change. Error feedback, residual memory, and direction memory carry information from these suppressed updates across time and recover accuracy without retraining. Precision sweeps show that increasing state precision can worsen a fixed recurrent solution, while matched training shows that compatibility with the state interface can be learned. To test whether this behavior extends beyond the GRU, we repeat the post-training intervention in an independently trained LSTM, where coarse write-back reproduces the failure, error feedback restores accuracy, and state-specific interventions reveal greater sensitivity of the cell state than the hidden state. Our results establish recurrent-state write-back as a key determinant of low-precision recurrent dynamics and identify the state-storage interface as a central design consideration for quantized recurrent inference.

1 Introduction

The paper examines recurrent-state write-back as a distinct interface in low-precision recurrent inference, focusing on how stored states shape future computation. It studies this issue in fluorescence lifetime imaging and uses controlled interventions to separate storage effects from learning.

Problem and concept: Low-precision state storage matters because recurrent states serve both as sequence memory and inputs to future computation.Changing their representation can change the sequence of states visited over time.
Problem and concept: Quantization-aware training can support accurate reduced-precision recurrent networks, but deployment may impose a coarser state interface on a trained model without retraining.The paper identifies this mismatch between learning and deployment as an incomplete remedy for low-precision degradation.
Problem and concept: Recurrent-state write-back maps each computed recurrent state to the value stored and returned at the next step.Deterministic low-precision write-back can leave sub-threshold updates absent from the recurrence-visible state.
Application: The study targets short- and long-fluorescence lifetimes, τ1 and τ2, from high-noise time-resolved fluorescence signals in fluorescence lifetime imaging.These estimates convert temporal emission profiles into quantitative molecular information.
Approach: The experiments replace only the recurrent-state write-back rule while keeping trained weights, recurrent updates, readouts, inputs, and evaluation fixed.This design isolates storage from learning.
Approach: The study presents write-back as a design variable and uses error feedback, residual memory, and direction memory to preserve suppressed updates across time.These mechanisms restore accurate inference in fixed networks without retraining.

2 Methods

The methods combine a compact GRU fluorescence-lifetime model with controlled post-training write-back interventions and matched precision analyses. Diagnostics relate recurrent updates to quantizer boundaries, while temporal-memory operators test how discarded information can persist across steps.

Data and model: The study uses 1,600,000 high-noise fluorescence signals with 135 temporal bins and a fixed 80/10/10 train-validation-test partition.The same partition is used across GRU training, interventions, precision analyses, and matched-training comparisons.
Data and model: Seq2SeqLite is a single-layer 32-unit GRU encoder-decoder with 6,627 trainable parameters and a linear readout.Lifetime estimates are derived from predicted temporal outputs and evaluated with τ1 and τ2 RMSE.
Experimental overview: Figure 1 summarizes the GRU encoder-decoder, deterministic 4-bit write-back, temporal-memory mechanisms, and lifetime-RMSE precision sweeps.The figure compares short lifetime τ1 and long lifetime τ2 RMSE in nanoseconds on logarithmic axes.
Quantization-aware training: QMem introduces 4-bit quantization sequentially across kernels, gates, biases, activations, and finally the recurrent state.P2F is the primary checkpoint for testing whether changing write-back alone produces failure, while P3 continues training on the hardened interface.
Interventions: Post-training intervention changes only the state-storage rule while keeping learned parameters, recurrent calculations, inputs, readout, and evaluation unchanged.Continuous propagation, deterministic storage, and other write-back rules are compared without retraining.
Diagnostics: The recurrent write margin compares each proposed state change with the quantizer half-step to identify the write deadband.For interior grid levels, M_t,j < 1 means deterministic nearest-level write-back retains the same stored value.
Interventions: Temporal-memory interventions retain discarded information through error feedback, quantized residual memory, or direction memory while the recurrence-visible state remains 4-bit.Direction memory accumulates repeated same-direction sub-threshold changes in a counter and triggers a later one-level transition.

3 Results

Post-training coarse write-back can disrupt a fixed recurrent solution by suppressing state changes, while temporal-memory mechanisms restore useful information without retraining. The effect extends across GRU and LSTM models, but depends on the learned state interface and recurrent variable.

Post-training intervention: 25.37/106.59 ns lifetime RMSE under deterministic 4-bit write-back versus 0.36/0.35 ns with continuous propagation isolates state storage as the changing operation.The trained parameters, recurrent update, readout, inputs, and evaluation procedure remain identical.
Recurrent-state write activity: 99.59% of decoder updates fall inside the 4-bit write boundary, leaving only 0.25% of decoder state elements changing stored level between steps.The network continues proposing new states, but most proposals do not enter the carried state.
Temporal organization: 132-step median same-direction runs under post-training 4-bit write-back contrast with 2-step medians for native 4-bit and P3 interfaces.The post-training 4-bit interface reduces the decoder state-change fraction to 0.60%, whereas recurrence-visible changes repeatedly interrupt runs in native 4-bit and P3.
Temporal-memory interventions: Error feedback, residual memory, and direction memory restore the fixed 8-bit GRU to 0.34/0.46, 0.34/0.45, and 0.49/0.58 ns τ1/τ2 RMSE, respectively.These mechanisms preserve suppressed update information while the returned state remains on the 4-bit grid.
Boundary of the rescue: Temporal memory is not intrinsically better than deterministic storage: it helps when the storage rule removes information required by the recurrent solution.The native 4-bit model is already accurate at 0.35/0.40 ns, and alternatives do not improve both lifetime parameters.
Precision and interface adaptation: Increasing state precision can worsen a fixed recurrent solution, while matched training changes the ranking by learning compatibility with the state interface.For the 4-bit-state reference, increasing state precision to 8-bit worsens errors from 0.35/0.40 ns to 0.43/0.57 ns; matched training favors direction memory.
Cross-architecture replication: Forcing both LSTM states from 8-bit to 4-bit raises RMSE from 0.239/0.254 ns to 3.857/1.327 ns, while error feedback restores 0.263/0.332 ns.Cell-state-only quantization is far more damaging than hidden-state-only quantization: 8.158/1.292 versus 0.296/0.354 ns.

4 Discussion

The discussion identifies recurrent-state write-back as a dynamical interface: suppressed updates can alter the trajectory of a fixed recurrent model, while temporal memory and training compatibility can restore performance. The LSTM replication indicates that sensitivity depends on the functional role of the stored state, not precision alone.

Write-back as a dynamical variable: 0.36/0.35 ns to 25.37/106.59 ns: changing only state-storage rules sharply increases τ1 and τ2 lifetime RMSE at the P2F checkpoint.Trained parameters and all other operations remain unchanged.
Write-back as a dynamical variable: Persistent same-direction proposals become harmful when coarse write-back leaves the stored state unchanged for nearly the complete decoder sequence.The recurrence repeatedly receives a state missing a persistent component of the required temporal evolution.
Temporal memory: Error feedback, residual memory, and direction memory recover substantial accuracy by preserving discarded information across later write opportunities.Direction memory helps despite retaining no update magnitude.
Learned compatibility: Post-training precision changes can reduce compatibility with a learned trajectory, whereas an interface present during optimization lets the recurrent solution adapt to its allowed transitions.Thus, increasing precision after training changes the interface rather than guaranteeing improvement.
Cross-architecture evidence: An independently trained LSTM reproduces coarse-write-back failure and error-feedback rescue, with cell-state write-back more damaging than hidden-state write-back.Because c and h remain coupled, single-state interventions are not additive decompositions.
Implications: Low-precision recurrent inference must evaluate both stored-state numerical accuracy and the dynamics produced when that state is repeatedly returned to the network.Bit width alone does not specify which proposed changes are written or how long discarded updates remain absent.

Supplementary material

Table S1 documents the staged QMem hardening schedule, culminating in recurrent-state quantization through a 15-epoch state blend.

QMem hardening schedule: P2F introduces recurrent-state quantization through a 15-epoch state blend in the QMem hardening schedule.The table uses “On” to indicate when each 4-bit quantizer is active.

Supplementary Results

The supplementary results provide location, metric, cross-architecture, precision, diagnostic, and memory-allocation controls for recurrent-state write-back. Together, they compare where write-back is applied, how state usage changes, and whether auxiliary memory or matched training alters the outcome.

Location controls: 7.79/8.36 ns versus 2.55/1.12 ns: decoder-only 4-bit write-back produces larger τ1/τ2 RMSE than encoder-only write-back at the frozen P2E checkpoint.Applying 4-bit write-back to both regions yields 39.04/24.58 ns.
Evaluation metrics: Sequence MAE 0.09 can coexist with substantially larger lifetime errors, so sequence reconstruction and lifetime RMSE need not rank write-back conditions identically.Lifetime RMSE is treated as the primary task-level metric because lifetime depends on predicted decay structure.
LSTM controls: The independently trained LSTM uses frozen whole-state and state-type interventions, retaining native 8-bit write-back for the non-targeted state.Complete task metrics and targeted-state diagnostics are reported separately.
QMem trajectory: P2E predictions broaden and begin clustering at discrete output levels, P2F further concentrates them at discrete lifetime levels, and P3 fine-tuning restores prediction–ground-truth correspondence.The phase-resolved distributions use 160,000 held-out test samples and separate τ1 and τ2 density maps.
LSTM controls: The LSTM intervention table fixes the checkpoint, compares native 8-bit, deterministic 4-bit, error-feedback, and continuous propagation, and reports lifetime RMSE with lifetime correlations.All metrics use the complete 160,000-sample held-out test partition.
Precision sweeps: Performance across the post-training precision sweep is not monotonically ordered by state bit width for fixed trained models.The 8-bit-state reference reaches 0.20 and 0.22 ns natively, while the independently trained 4-bit model reaches 0.35 and 0.40 ns at native 4-bit.
Precision sweeps: 12 to 176.5 occupied decoder levels: changing a frozen 4-bit-state GRU to 8-bit write-back expands state occupancy while worsening lifetime RMSE.The occupancy increase appears across all 32 decoder units.
State usage: Deterministic P2F write-back most strongly suppresses recurrence-visible decoder writes and restricts state-level usage, whereas error feedback and stochastic rounding increase both.Figure S2 reports write probability, effective level count, and distinct stored levels across 32 encoder and decoder units.

Supplementary Methods for Reproducibility

The supplementary methods define the fluorescence simulation, model architectures, dataset partition, distillation setup, and optimization procedure used for reproducible evaluation.

Experimental setup: The study retains the published simulation pipeline, fixed data partition, teacher architecture, and Seq2SeqLite student configuration across analyses.QMem progressively applies low-precision constraints to Seq2SeqLite while optimization continues.
Signal generation: Signals use mono- and bi-exponential fluorescence kinetics, photon statistics, detector-dependent noise, and measured pixelwise instrument response functions.The resulting inputs are high-noise time-resolved fluorescence signals whose temporal structure reflects kinetics and instrument response.
Dataset: The dataset contains 1,600,000 signals represented by 135 temporal bins, split into fixed training, validation, and held-out partitions.The held-out partition is excluded from training, checkpoint selection, learning-rate adaptation, and distillation-target generation.
Architectures: The teacher is a two-layer 128-unit GRU encoder–decoder, while Seq2SeqLite is a single-layer 32-unit GRU student with 6,627 trainable parameters.The teacher encoder initializes the decoder, which receives a zero-valued input sequence before dense readout.
Training objective: Student optimization combines supervised sequence error with teacher–student sequence error using fixed predictions from the frozen teacher.The distillation objective uses direct mean-squared error between student and teacher sequence outputs with α = 0.6.
Optimization: Optimization uses Adam with batch size 1024, phase-specific learning-rate multipliers, warm-up in Phase 1, and validation-triggered rate reductions.The base learning rate is 10^-4, with later phases assigned lower initial rates and a minimum rate of 10^-6.

Quantized GRU formulation

The quantized GRU formulation defines staged 4-bit hardening, recurrent-state write-back, diagnostic margins, and auxiliary mechanisms that preserve suppressed state changes.

QMem progressive quantization procedure: QMem applies 4-bit constraints to kernels, gates, biases, candidate activation, and recurrent state through staged hardening.The recurrent state is introduced last, after which the hardening trajectory is completed and repacked into the deployment graph.
QMem progressive quantization procedure: P2F progressively introduces recurrent-state quantization over 15 epochs, reaching full 4-bit state write-back at the fifteenth epoch.Each phase starts from the preceding phase’s best validation checkpoint without transferring optimizer state.
Post-training write-back interventions and diagnostics: Post-training interventions reconstruct saved checkpoints and replace only the state-storage rule after verifying native inference reproduction.Weights, recurrent updates, readout, inputs, and evaluation procedures remain fixed during intervention analysis.
Post-training write-back interventions and diagnostics: The recurrence-visible state follows qt−1 → ht → qt: the stored state enters step t, the GRU computes ht, and write-back produces qt for step t + 1.The implemented recurrence uses the write-back output as the next recurrent input.
Quantized GRU formulation: Continuous propagation stores qt = ht, whereas deterministic B-bit write-back applies nearest-level quantization on a grid with spacing ∆B.The recurrent write margin identifies whether a proposed update lies inside the half-step boundary and remains on the same stored level.
Post-training write-back interventions and diagnostics: Error-feedback write-back stores a quantized state augmented by an auxiliary residual that carries the quantization error forward.The residual is an auxiliary write-back state rather than a trained network variable; evaluated widths are k = 2, 3, 4.
Post-training write-back interventions and diagnostics: Stochastic rounding selects adjacent state levels probabilistically, with five independent inference realizations evaluated for each stochastic condition.This contrasts with deterministic nearest-level write-back.
Post-training write-back interventions and diagnostics: Direction memory accumulates signed sub-threshold votes and advances the recurrence-visible state after a trigger, resetting the counter at the transition.The counter uses 2^k−1 active states, while transitions are clipped at representable grid rails and small changes may leave the state unchanged.

Representative-trajectory selection

The representative trajectory is selected by fixed rules, while decoder diagnostics quantify write activity, persistence, occupancy, handoff error, and direction-memory behavior under frozen models.

Representative selection: The representative held-out sequence is chosen by its proximity to the median deterministic 8-bit-to-4-bit decoder state-change fraction.Within that sequence, the decoder unit is selected by the greatest number of recurrence-visible direction-memory triggers, with ties resolved by test-partition position.
Trajectory analysis: The trajectory analysis compares P3, 4-bit-state, and 8-bit-state reference GRUs under post-training 4-bit write-back with fixed parameters.Direction memory uses the fixed voting region ∆B/8 < |δt| < ∆B/2.
Direction-memory events: Only decoder steps capable of updating recurrence-visible state contribute votes; the terminal decoder output is excluded from write statistics.Eligible votes are assigned by the sign of the proposed state change.
Persistence metrics: The same-sign fraction counts positive-to-positive and negative-to-negative transitions among four adjacent vote-transition types.This statistic summarizes one-step sign persistence.
Persistence metrics: Same-sign run length counts consecutive eligible votes with one sign and ends at reversals, ineligible events, ordinary writes, direction-memory triggers, or sequence termination.The run definition is distinct from the direction-memory counter state.
Aggregation and handoff: All sign-transition and run-length summaries pool decoder events over the complete 160,000-sample test partition without retraining.Handoff error is measured between the encoder final state and decoder initial recurrence-visible state after write-back.
State diagnostics: Write diagnostics report zero-write probability, changing hidden units, conditional sub-threshold writes, visited levels, occupancy concentration, and encoder-to-decoder handoff error.Effective occupancy is not interpreted as information capacity or effective network bit width.

LSTM cross-architecture replication and state-type dissection

An independently trained LSTM extends the write-back intervention beyond the GRU and separates the effects of quantizing the cell state from those of quantizing the hidden state.

LSTM intervention design: The cross-architecture analysis uses an independently trained 32-unit LSTM checkpoint with 8-bit native state and parameter representations.Raw recurrent proposals and recurrence-visible stored states are defined separately for the cell and hidden variables.
State-specific write-back: Deterministic interventions evaluate native 8-bit, whole-state 4-bit, cell-only 4-bit, hidden-only 4-bit, and identity propagation conditions.The native condition uses Bc = Bh = 8, while targeted conditions quantize only one state variable to 4 bits.
Error-feedback intervention: Targeted error feedback is applied only to the selected 4-bit LSTM state while the non-targeted state retains deterministic 8-bit write-back.The same condition is applied in encoder and decoder recurrence without retraining or fine-tuning.
State-specific diagnostics: Write statistics are accumulated separately for cell and hidden states across 135 encoder and 134 decoder live write opportunities per sequence.The analysis reports deadband, state-change, conditional sub-threshold, same-sign, and completed same-sign-run measures for each state.
Reference validation: Native LSTM reconstruction matches native inference with maximum and mean absolute differences of 0.0 on both 2,048 samples and the 160,000-sample test set.The comparisons use a predefined tolerance of 5 × 10^-5.

Matched recurrent-memory training campaign

The campaign trains four recurrent-memory interfaces under matched optimization conditions, varying state precision and auxiliary memory while keeping other quantized components fixed. Residual and direction memory preserve a 4-bit visible state while adding two stored auxiliary bits per unit.

Memory allocations: Four conditions compare 4-bit state-only, 6-bit state-only, 4-bit state plus 2-bit residual memory, and 4-bit state plus 2-bit direction memory.The latter three conditions use six stored recurrent-memory bits per unit.
Memory allocations: The 6-bit-state condition changes only recurrence-visible state precision, whereas residual and direction conditions allocate the extra two bits to auxiliary memory.Input kernels, recurrent kernels, biases, and candidate activation remain 4-bit in every condition.
Matched training: All conditions share the teacher, data partition, distillation objective, optimizer, schedule, batches, stopping rule, and matched initial parameters.Three matched training runs are evaluated for each condition.
Training interface: Residual and direction memory use hard-discrete forward visible states during training with straight-through treatment, but their auxiliary states do not receive ordinary recurrent gradients.Residual memory retains a local amplitude-dependent surrogate through compensated visible-state writes, whereas direction memory has no gradient through vote accumulation or triggers.

S9 Lifetime estimation and reconstruction fidelity

This section defines lifetime estimation from the model’s temporal outputs and reports both aggregate agreement metrics and predicted-versus-ground-truth distributions. The distributional view is intended to expose quantization effects that a single error statistic can miss.

Lifetime reconstruction: The model predicts three temporal output parameters, while analysis focuses on τ1 and τ2 because they carry fluorescence lifetime decay information.The physical time vector samples 135 gates at 0.09 ns intervals.
Lifetime reconstruction: Lifetime estimates for each component are obtained by trapezoidal integration normalized to the first-gate amplitude.The implementation returns zero when the initial signal amplitude is at or below 10^-6.
Evaluation metrics: Each sample’s estimated lifetime vector is represented as (τ̂1,i, τ̂2,i), with ground-truth values taken directly from the stored scalar label array.The reported correlations use Pearson correlation across samples between estimated and ground-truth lifetime values.
Evaluation metrics: Predicted-versus-ground-truth distributions accompany aggregate metrics to reveal discrete-level clustering, dynamic-range compression, or reduced input dependence.These effects may remain hidden by a single error statistic.

S10 Paired storage and lifetime analyses

The analyses compare stored recurrent-memory designs and quantify uncertainty, state occupancy, write-back behavior, and lifetime performance on a common held-out partition. They distinguish stored-memory comparisons from broader hardware or model-memory claims.

Uncertainty analysis: Paired uncertainty evaluates two frozen write-back conditions on the same resampled held-out sequences, using paired differences between their metrics.Two thousand paired bootstrap replicates define the reported 95% interval from the 2.5th and 97.5th percentiles.
Write-back behavior: Aggregate write-back statistics include element-wise zero-write probability, the probability that no hidden unit changes level, and the average number of changing hidden units per step.These quantities are reported in Table S7 for principal native and intervention conditions.
Write-back behavior: Occupancy analysis counts visited state levels per hidden unit and summarizes occupancy entropy with Neff.Table S8 reports median per-unit occupancy and write-back statistics for selected conditions.
Intervention analysis: Post-training interventions use a common 160,000-sample held-out test partition, with stochastic-rounding RMSE averaged across five inference realizations and other evaluations deterministic.Psub→write measures sub-threshold proposed changes that still produce visible state transitions and is undefined for identity propagation.
Lifetime-conditioned analysis: Lifetime-conditioned analyses sort held-out samples separately by ground-truth τ1 and τ2 into ten equal-count bins of 16,000 samples.Paired sequence-level bootstrap comparisons within bins depend only on ground-truth lifetime, not prediction error.
Training and reproducibility: The matched training campaign reports lifetime RMSE in nanoseconds across three runs per recurrent-memory condition, while paired differences are interpreted consistently across all three runs.The native-4-bit and native-8-bit GRUs are independently trained reference solutions, whereas QMem hardening represents one training trajectory.