PaperlayerView exact revision on arXiv

Source-linked AI summary

HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Training

Xinrui Chen, Mengyang Li, Ou Wu, Ji Zhang

arXiv:2609.33570v1cs.LG

TL;DR

GRPO activation recovery is a memory–computation bottleneck, and existing methods do not explicitly use GRPO’s analytic update structure to set state fidelity. HiLoRe combines update-conditioned risk with measured recovery utility to allocate storage, compression, or recomputation. Across five model–task settings, it improves actor-update throughput within the stated memory budget while keeping paired downstream scores close to GC.

  • Problem

    Existing activation-recovery policies set state fidelity using execution structure, tensor properties, compression error, or generic sensitivity, without explicitly connecting GRPO’s analytic update structure to state-level fidelity allocation.

    Source: §1 · p. 1

  • Method

    HiLoRe combines GRPO policy-update exposure with measured recovery utility to allocate graph-attributed units to high-precision storage, low-precision recovery, or deterministic recomputation under memory and calibrated-risk constraints.

    Source: §1 · pp. 1–2

  • Results

    With 2K responses and memory ceiling 1.10BGC, actor-update throughput improves up to 13.47% over GC and 7.87% over the fastest evaluated baseline, with downstream quality close to paired GC.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The findings support allocating recovery precision according to active-update sensitivity alongside memory and recomputation costs.

    Source: §5 · p. 10

    Evaluations primarily cover rank-16 LoRA with responses up to 4K on 48-GB GPUs; full-parameter evidence is limited to single-seed learning curves and single-A100 efficiency measurements.

    Source: LIMITATIONS · p. 33

Abstract

from arXiv · show

Group-relative policy optimization (GRPO) makes learner-side activations a major memory-computation bottleneck: gradient checkpointing reduces activation memory through recomputation, but fixed schedules can leave roughly 18 GB unused on a 48-GB GPU despite substantial recomputation overhead. Existing activation-management methods set state fidelity from execution cost, tensor properties, or generic compression sensitivity, without explicitly incorporating GRPO's analytic update structure into state-fidelity allocation. We formalize this dependence as policy-update exposure, linking the current GRPO loss coefficients to state-level approximation sensitivity. These coefficients are available before backward without an additional backward pass. We introduce HiLoRe, which allocates graph-attributed recovery units among high-precision storage, low-precision compression, and deterministic recomputation using measured recovery utility and update-conditioned approximation risk. It combines high-precision storage and deterministic recomputation with low-precision recovery under a calibrated risk budget. Across five model-task settings with 2K responses and memory < 1.10 times GC's per-GPU actor-update peak, HiLoRe's actor-update throughput gains reach 13.5% over GC and 7.9% over the fastest evaluated baseline, with paired mean downstream-score differences below 0.6 percentage points.

1 INTRODUCTION

GRPO’s analytic update signals provide a way to make recovery fidelity depend on the active update, which existing recovery policies do not explicitly do. HiLoRe uses that signal to allocate learner-state recovery and improves throughput within the evaluated memory budget.

1 INTRODUCTION

  • GRPO’s exact analytic per-token update coefficients are available before backward without another backward pass, and the top 10% by |ωt| carry 52.4–66.8% of coefficient mass in diagnostic replays.The coefficients combine rollout advantages, policy ratios, clipping states, update masks, and KL regularization.Source: §1 · p. 1
  • HiLoRe allocates graph-attributed recovery units among high-precision storage, low-precision recovery, and deterministic recomputation using realized utility and update-conditioned approximation risk.It selects among these actions under memory and fidelity constraints.Source: §1 · p. 2
  • With 2K responses and B = 1.10BGC, actor-update throughput improves up to 13.47% over GC and 7.87% over the fastest evaluated baseline, with paired mean downstream-score differences below 0.6 percentage points.These results span five model–task settings.Source: §1 · p. 2

2 RELATED WORK

Prior work reduces training-memory costs through checkpointing, compression, hybrid recovery, learning-signal prioritization, and broader LLM reinforcement-learning system optimizations. HiLoRe’s distinction is to use GRPO update signals to set learner-state fidelity for a fixed workload.

2 RELATED WORK

  • Training-memory methods use checkpointing, compression, and hybrid compression–recomputation, while HiLoRe allocates backward-state fidelity using GRPO signals while preserving forward computation.Prior methods use execution structure, representations, or sensitivity; HiLoRe targets learner-state recovery.Source: §2 · p. 2
  • Learning signals have been used to prioritize tokens, activations, and trajectories, while HiLoRe uses GRPO update signals to set learner-state fidelity rather than select workloads or forward-pass activations.The related work also describes token and rollout selection in LLM RL.Source: §2 · pp. 2–3
  • LLM RL systems reduce pipeline costs through execution and rollout optimizations, selection methods, and KV-state management; HiLoRe focuses on update-conditioned learner-state recovery within a fixed GRPO workload.Examples include distributed rollout and training coordination, sparse rollout, and long-context scheduling.Source: §2 · p. 3

3 HILORE

HiLoRe uses GRPO’s active update to guide fidelity choices for learner-state recovery. It combines graph-attributed recovery units, measured utility, and update-conditioned risk within memory and fidelity constraints.

3 HILORE

  • HiLoRe conditions learner-state fidelity on the active GRPO update and allocates graph-attributed H/L/R actions using update exposure, measured utility, and memory and fidelity constraints.The actions are high-precision storage, low-precision recovery, and recomputation.Source: §3 · p. 3

3.1 PROBLEM SETUP

  • Recovery selection chooses actions for verified learner-graph units to minimize actor-update time subject to peak-memory and update-distortion budgets.The candidate storage plan is fixed before the microbatch forward pass, and available actions depend on the retained representation.Source: §3.1 · pp. 3–4

3.2 POLICY-UPDATE EXPOSURE

  • Policy-update exposure connects GRPO token coefficients to state-level approximation sensitivity; the coefficients are available from rollout data and forward-computed signals.The exposure-based analysis motivates a bound on gradient deviation under low-precision recovery.Source: §3.2 · pp. 4–5

3.3 GRAPH-ATTRIBUTED RECOVERY UNITS

  • Graph-valid recovery units couple forward subgraphs with complete backward state bundles, accounting for graph dependencies so recovery choices remain executable.Candidate units are profiled to confirm that materializing their bundles removes attributed recomputation paths.Source: §3.1 · p. 3, §3.3 · p. 5

3.4 RECOVERY UTILITY AND APPROXIMATION RISK

  • Initialization calibration estimates low-precision fidelity cost from matched passes, combining update exposure, reconstruction error, and calibrated susceptibility into a risk surrogate.Current-update exposure and calibrated statistics are used to refresh risk scores during training.Source: §3.4 · p. 5

3.5 JOINT H/L/R ALLOCATION

  • HiLoRe selects per-unit recovery actions using additive utility, memory, and risk estimates, then validates complete capture-and-recovery memory and gradient error.The discretized two-constraint multiple-choice knapsack is solved by dynamic programming; plans and calibration statistics are reused while coefficients and risk scores refresh each microbatch.Source: §3.5 · p. 6

4 EXPERIMENTS

The experiments evaluate whether HiLoRe improves GRPO update efficiency while maintaining gradient fidelity and downstream quality, and whether its gains extend across budgets, workloads, response lengths, and models. They also test policy-update exposure as a predictor of approximation-sensitive states.

4.1 EXPERIMENTAL SETUP

  • The setup spans three models and three training datasets, compares checkpointing, rematerialization, compression, and hybrid baselines, and uses matched replays to measure efficiency and gradient fidelity.Defaults use rank-16 LoRA on four L40S 48-GB GPUs; configuration selection favors throughput subject to memory, gradient-error, and validation-score constraints.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • On Qwen2.5-3B, HiLoRe exceeds GC and Adacc throughput by 10.16% and 4.89%; recomputation time falls from 207.38 to 118.64 seconds (42.79%).Mean held-out relative gradient error is 0.0104, below the 0.015 threshold.Source: §4.2 · p. 7
  • At B = 1.10BGC, HiLoRe leads evaluated configurations on TACO-Verified and Logic-RL K&K by 9.00% and 8.64% over GC, with downstream-score differences within 0.10 percentage points across four benchmarks.Across response lengths, Qwen/Phi throughput gains span 6.13–13.57%/4.73–12.64%.Source: §4.2 · p. 7
  • In full-parameter GRPO on Llama-3.2-3B-Instruct, HiLoRe reaches 48.40% MATH500 accuracy, while GC improves from 9.60% to 47.80%.Across evaluated checkpoints, HiLoRe–GC differences span −0.20 to +0.60 percentage points.Source: §4.2 · p. 8

4.3 MEMORY–THROUGHPUT FRONTIER

  • Across tested budgets, HiLoRe leads mean throughput; at 1.20BGC, its gain reaches 13.97% while maximum observed gradient error remains 0.0141.As H rises from 0% to 25.0% and R falls from 91.7% to 61.1%, mean held-out gradient error rises from 0.0068 to 0.0130.Source: §4.3 · p. 8
  • On Qwen, adaptive recovery gains 13.97% using 10,892 MiB less than GC with 10240-token packing; on Llama, HiLoRe gains 13.47% at 1.096BGC without increasing packing.GC's 7168-token packing triggers OOM on Llama.Source: §4.3 · p. 8

4.4 RISK PREDICTION IN POST-UPDATE REPLAYS

  • Post-update replays show the top 10% of response slots by |ωt| carry 52.4–66.8% of coefficient mass; exposure with susceptibility and reconstruction distortion reaches 0.80 Top-10% recall and 0.88 violation AUROC.Across three datasets, Spearman correlations are 0.46 for advantage-only exposure, 0.68 for complete exposure, and 0.77 for ψu.Source: §4.4 · pp. 8–9

4.5 ABLATIONS, SENSITIVITY, AND OPERATIONAL ANALYSIS

  • Recovery ablations show 10.16% throughput gain for full HiLoRe versus 8.31% for static-risk H/L/R; diagnostic replays allocate H toward high-advantage regions and L toward clipped regions.H allocation is 14.7% versus 4.8%, while L allocation is 21.3% versus 11.2%; isolated L errors average 0.0051 and 0.0115 for clipped and active-unclipped groups.Source: §4.5 · p. 9

5 CONCLUSION

HiLoRe conditions learner-state fidelity on the active GRPO update and combines policy-update exposure with measured recovery utility to allocate H/L/R actions under memory and calibrated-risk budgets. With 2K responses and a memory ceiling of 1.10BGC, actor-update throughput improves up to 13.47% over GC and 7.87% over the fastest evaluated baseline, with downstream quality close to paired GC.

5 CONCLUSION

  • 13.47% over GC and 7.87% over the fastest evaluated baseline: HiLoRe improves actor-update throughput with 2K responses and a 1.10BGC memory ceiling.Downstream quality remains close to paired GC; benefits span mathematics, code, and logical reasoning across response lengths and architectures.Source: §5 · p. 10

AI USE STATEMENT

  • The paper reports AI assistance for mathematical checking and language polishing, while stating that authors reviewed the work and take responsibility for its contents.The work uses existing models and public datasets; responsible application requires respecting licenses and assessing downstream safety and misuse risks.Source: AI USE STATEMENT · p. 11, ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The appendices document graph attribution, profiling, calibration, scheduling, theoretical assumptions and proofs, and experimental protocols and analyses.The reproducibility statement identifies Appendix D.1 as specifying data splits, execution settings, baselines, configuration selection, timing, diagnostics, and on-policy evaluation.Source: REPRODUCIBILITY STATEMENT · p. 11, Appendix Contents · p. 17

A.1 GRAPH ATTRIBUTION AND RECOMPUTATION VERIFICATION

  • A recovery unit couples a forward subgraph with the complete state bundle required by its backward operators, and verified units admit feasible H, L, or R actions.H preserves the complete bundle, L reconstructs from a supported low-precision component, and R recomputes from exact checkpoint inputs.Source: §A.1 · p. 18

A.2 MEASURED RECOVERY UTILITY

  • Conditional recovery utility compares actions under the same candidate-storage plan, while complete actor-update timing includes capture costs even for candidates later discarded.Profiling uses paired updates with matched parameters, inputs, microbatches, precision, RNG states, and execution settings.Source: §A.2 · p. 18

A.3 NON-ADDITIVE RECOVERY INTERACTIONS

  • Overlapping recomputation footprints identify dependent units, and schedule validation removes actions with non-positive leave-one-out runtime utility before reevaluation.This measures runtime contribution under joint execution, including overlapping recomputation and allocator effects.Source: §A.3 · p. 18

B.1 EXPOSURE AND LOW-PRECISION CALIBRATION

  • For evaluated MLP units, activation-based token weights are normalized to mean one within each unit and microbatch, and masked positions receive zero update coefficients.Calibration compares matched exact and low-precision recovery passes; the activation-based weights are an empirical proxy calibrated against measured single-unit gradient error.Source: §B.1 · pp. 18–19

B.2 JOINT ALLOCATION AND SCHEDULE VALIDATION

The allocator chooses recovery actions subject to residual-memory and calibrated-risk constraints, then validates complete schedules against measured utility, memory, and gradient deviation. Online scheduling reuses validated storage plans and computes current-update exposure and risk after the forward pass.

B.2 JOINT ALLOCATION AND SCHEDULE VALIDATION

  • The allocator selects feasible assignments under residual-memory and risk budgets, maximizing utility over the discretized costs with dynamic programming.Upward rounding of memory and risk costs preserves feasibility for the additive surrogate; terminal ties favor lower risk, then memory.Source: §B.2 · pp. 19–20
  • Schedule validation removes non-R actions with non-positive leave-one-out utility and checks accumulated gradient error against a 0.015 development-set mean tolerance.Configurations remain fixed for held-out evaluation, and selection also enforces a terminal validation criterion.Source: §B.2 · p. 20

B.3 ONLINE SCHEDULING AND COMPLEXITY

  • For each microbatch, a validated candidate-storage plan is selected before forward; unsupported shapes use all-R capture.Plans are selected using complete actor-update timing and reserve memory for candidates, checkpoints, encoding, and backward workspaces.Source: §B.3 · p. 20
  • After forward, HiLoRe forms current-update exposure and risk scores, then allocates recovery actions using available representations, measured utilities, and memory costs.Timing includes input preparation, transfers, synchronization, dynamic programming, backtracking, capture, exposure collection, and recovery.Source: §B.3 · p. 20
  • Allocation takes O(UKBKδ) time and uses O(KBKδ) rolling-table storage, with predecessor actions requiring O(UKBKδ) storage for backtracking.Given activation weights, coefficient and score refresh takes O(|T| + I + U) per microbatch.Source: §B.3 · p. 21

C THEORETICAL ANALYSIS

The theoretical analysis derives GRPO’s token-level loss coefficients and uses them to relate low-precision state perturbations to parameter-gradient deviation. It also gives exact-recovery conditions for high-precision storage and deterministic recomputation and motivates a calibrated joint-risk surrogate.

C.1 UPDATE SEMANTICS AND EXACT RECOVERY

  • GRPO’s token coefficients are derived by the chain rule from the loss dependence on token log-probabilities, with coefficients available from rollout data and the exact forward pass in linear time.Recovery changes saved states while preserving forward log-probabilities and loss-side quantities, so the coefficients remain fixed.Source: §C.1 · p. 21
  • Under matched deterministic execution and accumulation, high-precision storage and recomputation produce equal losses and parameter gradients when recomputation reproduces the complete saved states.The assumptions include identical parameters, inputs, positional and RNG states, operator and precision settings, and backward execution.Source: §C.1 · p. 21

C.2 POLICY-UPDATE EXPOSURE AND CALIBRATED RISK

  • The gradient-deviation bound separates reconstruction magnitude from current-update sensitivity, while realized deviation also depends on perturbation direction and cancellation.The theorem assumes smoothness of the backward map and a Lipschitz Jacobian along the state-reconstruction segment.Source: §C.2 · p. 22
  • If every coefficient on a unit’s response-position support is zero, perturbing that unit’s state causes no gradient change under the fixed backward map.This identity does not require a Taylor approximation; with KL regularization, unmasked positions can retain a KL coefficient even when the policy-loss coefficient is zero.Source: §C.2 · p. 22
  • HiLoRe’s calibrated surrogate sums unit-level risk costs, motivated by the first-order bound and second-order interactions, and checks fidelity after joint recovery and gradient accumulation.The calibration uses exact update coefficients, structural support, and an empirical activation-based proxy for token weights.Source: §C.2 · p. 23

D.1.1 TRAINING CONFIGURATION AND DATA

The experimental configuration uses shared learner settings and data splits across recovery methods within each workload, with the specified model, task, and optimization choices. Validation selects configurations, while validation and test data are excluded from parameter updates and calibration.

D.1.1 TRAINING CONFIGURATION AND DATA

  • Unless otherwise specified, training uses LoRA with rank r = 16, scaling alpha/r with alpha = 32, zero dropout, and only adapters trainable.Source: §D.1.1 · p. 23
  • LoRA uses rank 16, scaling α/r with α = 32, and zero dropout; only adapters are trainable, with model-specific adapter targets.Default prompt and response limits are 1024 and 2048 tokens, and rollout temperature is 0.6.Source: §D.1.1 · p. 23
  • Shared learner execution settings are reported in Table 6, with LoRA defaults applying unless specified.Within each workload, recovery methods share training and batching settings except for specified packing or length variations.Source: Table 6 · p. 24, §D.1.1 · p. 24
  • Training and calibration data supply on-policy rollouts while validation supports configuration selection; validation and test data are excluded from updates and calibration.Problem lists are fixed across methods, training seeds, models, and lengths using split seed 2026 after deduplication and benchmark-overlap removal.Source: §D.1.1 · p. 24

D.1.2 RECOVERY-METHOD IMPLEMENTATIONS

This appendix specifies the recovery baselines and evaluation procedures, then reports throughput, fidelity, and transfer results for HiLoRe under shared memory and quality constraints.

D.1.2 RECOVERY-METHOD IMPLEMENTATIONS

  • Baselines span checkpointing, selective recomputation, compression, and hybrid recovery, with implementation-specific tensor coverage and tuning choices documented for comparison.Rockmate’s profiled blockwise schedules yield lower throughput than decoder-layer GC at the reported operating points when evaluated with full actor-update timing.Source: §D.1.2 · p. 25

D.1.3 CONFIGURATION SELECTION

  • Final configurations are selected from feasible candidates using memory, mean gradient error, and terminal validation score constraints, then throughput with validation-score tie-breaking.The default mean gradient-error tolerance is 0.015, and quality must be within one absolute percentage point of GC.Source: §D.1.3 · p. 26
  • For transfer, candidates are fixed before testing and DSR-sub freezes recovery hyperparameters while renewing profiling and calibration without task-specific search.Final test scores do not enter configuration selection.Source: §D.1.3 · p. 27

D.1.4 EFFICIENCY AND MEMORY MEASUREMENTS

  • Actor-update throughput includes forward, capture and compression, recovery, backward, communication, optimizer work, and online adaptation; rollout generation is outside the timed region.Memory peaks and budget checks use maxima across ranks, measured updates, and repetitions.Source: §D.1.4 · p. 27

D.1.6 DOWNSTREAM TRAINING AND EVALUATION; D.5.3 TRAINING QUALITY AND FULL-PARAMETER TRANSFER

  • Final LoRA quality comparisons use 60 updates across seeds 1234, 2025, and 3407, with results reported as across-seed means, sample SDs, and paired method–GC differences.Source: §D.1.6 · p. 28

D.2 ADDITIONAL BASELINES; D.4 CANONICAL MEMORY–THROUGHPUT FRONTIER

  • Under identical memory ceilings, HiLoRe gains 10.16–13.47% throughput over GC across three models, exceeding all added activation-compression and selective-recomputation baselines, with mean gradient errors of 0.0091–0.0104.Source: §D.2 · p. 28

D.3 POLICY-UPDATE EXPOSURE DETAILS

  • Diagnostic replays show exposure and calibrated risk predict approximation-induced gradient distortion: complete exposure improves Spearman by 0.21–0.22 over advantage-only weighting, and susceptibility plus reconstruction distortion add 0.09.The resulting correlation is ρ = 0.75–0.78 across workloads.Source: §D.3 · p. 29

D.5.1 CROSS-TASK TRANSFER

  • 9.00% on TACO-Verified and 8.64% on Logic-RL K&K: HiLoRe leads evaluated configurations on both tasks at 1.10BGC, with paired mean downstream-score differences within 0.10 percentage points across four benchmarks.These comparisons use Qwen2.5-3B and 2K responses.Source: §D.5.1 · p. 30

D.5.2 RESPONSE-LENGTH AND MODEL TRANSFER

HiLoRe’s throughput gains persist across the tested response lengths and model architectures. Recovery allocations shift toward retention at longer responses, while the reported gradient errors and memory increases remain within the stated bounds.

D.5.2 RESPONSE-LENGTH AND MODEL TRANSFER

  • 4.73–13.57%: all six tested response-length settings improve throughput over paired GC, with mean relative gradient errors at or below 1.21%.Qwen’s gain rises from 6.13% at 1K responses to 13.57% at 4K; Phi’s largest gain occurs at 2K.Source: §D.5.2 · p. 30, Figure 8 · p. 31
  • From 1K to 4K responses, recomputation’s recovery share falls from 86.2% to 67.0% on Qwen and from 88.8% to 75.1% on Phi as high- and low-precision retention increase.The selected allocations vary with response length while maintaining the reported gradient fidelity.Source: §D.5.2 · p. 30, Figure 8 · p. 31
  • 8.39–10.18%: throughput improves across three GQA/MHA models under the tighter 1.05BGC budget, with mean relative gradient errors at or below 1.00%.Additional peak memory is 4.8% for Qwen, 4.7% for Phi, and 4.9% for Llama, below the shared 5% allowance.Source: §D.5.2 · p. 31, Figure 8 · p. 31

D.5.3 TRAINING QUALITY AND FULL-PARAMETER TRANSFER

HiLoRe maintains downstream quality near GC in the reported training settings and improves full-parameter actor-update throughput over both fixed and tuned GC as memory budgets rise.

D.5.3 TRAINING QUALITY AND FULL-PARAMETER TRANSFER

  • Paired downstream-score differences from GC remain within 0.53 percentage points across settings; in the single-seed full-parameter experiment, HiLoRe reaches 48.40% MATH500 accuracy.GC improves from 9.60% to 47.80% MATH500 accuracy in that experiment.Source: §D.5.3 · p. 31, Table 14 · p. 31

D.5.4 FULL-PARAMETER MEMORY–THROUGHPUT TRADE-OFFS

  • 5.04–12.52% over fixed GC and 2.88–6.56% over tuned GC: HiLoRe’s throughput improvement at budgets from 1.05–1.20BGC in the full-parameter setting.The comparison uses Llama-3.2-3B-Instruct / DSR-sub; tuned GC adjusts microbatch size while retaining decoder-layer checkpointing.Source: §D.5.4 · pp. 31–32

D.6 SENSITIVITY TO THE SHARED GRADIENT-ERROR TOLERANCE

HiLoRe’s throughput advantage remains stable across the evaluated shared gradient-error tolerances.

D.6 SENSITIVITY TO THE SHARED GRADIENT-ERROR TOLERANCE

  • 8.76–11.28%: HiLoRe’s throughput gain over GC across five tolerances, exceeding the strongest baseline by 1.58–5.14 percentage points.The 1.25% and 1.50% settings select identical configurations, retaining a 10.16% gain with 1.04% mean gradient error.Source: §D.6 · p. 32, Table 15 · p. 32

D.7 ABLATION, SENSITIVITY, AND RUNTIME DETAILS

The section specifies the replay and sensitivity protocols, reports how risk budgets and refresh schedules affect gain and error, and bounds the evidence by the evaluation settings and gradient-distortion proxy.

D.7 ABLATION, SENSITIVITY, AND RUNTIME DETAILS

  • The ablation protocol uses pre-update replays for Qwen2.5-3B / DeepMath10K, with 16 calibration microbatches and per-microbatch exposure/risk refresh.Block recovery uses Transformer blocks; size-based and uniform H replace utility, HiLoRe-HR uses H/R, and δ = 0 disables L while retaining capture costs.Source: §D.7 · p. 32
  • Sensitivity sweeps vary one factor while holding replay inputs, candidate plans, units, utilities, budgets, and held-out updates fixed.Calibration uses nested 32-microbatch subsets; dynamic scores refresh every 1, 4, or 8 microbatches or on cache misses, while static risk uses initialization-period mean risk.Source: §D.7 · p. 32
  • Raising δ from zero to δ⋆ increases gain from 6.14% to 10.16% at mean error 0.0104; both larger tested budgets exceed ϵg = 0.015.Per-microbatch refresh raises gain over static risk from 8.31% to 10.16% and lowers maximum error from 0.0161 to 0.0114 at 0.21% overhead; a four-microbatch interval retains 10.04% gain at 0.12% overhead.Source: §D.7 · pp. 32–33, Table 17 · p. 33
  • Recomputation saves 88.74 s against 15.57 s of encoding, reconstruction, refresh, and allocation costs; the profiled total falls 9.22% versus GC.The total also saves 23.48 s beyond H/R alone; measurements use five warmed-up repetitions, and actor-update timing includes online collection, refresh, and allocation.Source: §D.7 · p. 33, Table 18 · p. 33

LIMITATIONS

  • Gradient distortion is estimated with calibrated exposure proxies and additive risk scores motivated by first-order perturbation analysis.Evaluations primarily cover rank-16 LoRA with responses up to 4K on 48-GB GPUs; full-parameter evidence is limited to single-seed learning curves and single-A100 efficiency measurements.Source: LIMITATIONS · p. 33
Loading 2609.33570v1…