Source-linked AI summary
Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation
Gautam Kishore
TL;DR
Language models make factual, reasoning, and suggestion errors, while adapting them can trade targeted correction against general capabilities. This paper tests CRN v2, a frozen-base logit-correction approach, and reports correction without measurable degradation on tested benchmarks.
Problem
The paper asks whether adapting a language model to fix target-domain errors can avoid degrading capabilities it already has.
Source: §1 · p. 1Method
CRN v2 keeps the base frozen and learns additive logit adjustments while using KL preservation to stay close to base behavior.
Source: §1 · p. 2Results
Takeaways & Limitations
For deployment, the authors argue that preserving capabilities can matter more than correcting more errors.
Source: §5 · p. 7CRN v2 leaves 46.7% of exam errors unfixed, which the authors say is insufficient for high-stakes use.
Source: §6 · p. 7
Abstract
from arXiv · showhide
We study a practical question: can a small correction module fix errors in a frozen language model's outputs without degrading its base capabilities? We propose CRN v2, a lightweight logit-level correction module (~34M trainable parameters, 0.73% of the 4.65B text module) that sits atop a fully frozen Gemma 4 E2B model. The base model is never updated; only the correction module learns, via supervised fine-tuning followed by reference-free DPO on 83,400 error-correction pairs. On a 60-question domain exam (CEHRI: Certified Human-Robot Intelligence, covering facts, arithmetic, and implicit-goal reasoning), CRN v2 corrects 53.3% of base-model errors (reworded variant: 43.3%) while showing no degradation on tested capability benchmarks (MMLU/BoolQ N=200; car-wash N=8). A LoRA baseline at the matched CRN v1 budget (6.6M params, rank 19) achieves 83.3% correction but suffers 30-75% capability loss on the same benchmarks -- the correction-capability tradeoff. An ablation shows that the KL preservation term (lambda=0.1) is critical: lowering it to 0.01 degrades correction to 35.0%. A hidden-state injection variant at earlier layers (1.6M params, SFT-only) reaches 50.0%/55.8% but does not exceed logit correction; shallower injection (layer 4) drops to 30.0%/28.3%; multi-depth logit correction (~35M) reaches only 40%; and longer training (5,000 SFT + 2,000 DPO) stays at 53.3% -- none of the alternative configurations we tested exceeded the rank-128 logit result, consistent with a best-achieved result of ~53% rather than a floor. This is a study of a design principle (frozen base + logit correction + KL anchoring), not a claim of architectural novelty. All code, main-result weights, and evaluation scripts are released (deep variant as code only -- no trained deep checkpoints).
1 Introduction
The paper asks whether a frozen-base correction module can fix language-model errors without degrading existing capabilities, and proposes CRN v2 as a logit-level approach. Its reported results show correction with no measurable degradation on tested benchmarks, alongside a correction-capability tradeoff for LoRA.
Design principle
- CRN v2 is a lightweight module that reads the frozen base’s final hidden state and produces an additive logit correction; the base model receives no gradients.The design aims to adjust logits where the base is wrong while staying close to its behavior elsewhere.Source: §1 · p. 2
Empirical finding
- CRN v2 corrects 53.3% of errors with no measurable degradation on MMLU/BoolQ (N=200) and car-wash (N=8), while LoRA corrects 83.3% but degrades 30–75% on those benchmarks.The authors describe this as a correction-capability tradeoff.Source: §1 · p. 2
Ablation
- Lowering the KL preservation weight from λ=0.1 to 0.01 reduces correction from 53.3% to 35.0%.The authors identify the anchoring term as load-bearing.Source: §1 · p. 2
Exploratory variants
- The tested alternatives did not exceed rank-128 logit correction: layer-7 injection reached 50.0%/55.8% SFT-only, layer-4 injection reached 30.0%/28.3%, and multi-depth logit correction reached 40%.Longer training stayed at 53.3%; the variant results are single-run and session-observed except for the independently log-verified layer-7 SFT-only row.Source: §1 · p. 2
Scope and transparency
- The paper presents a design-principle study rather than an architectural-novelty claim, notes a small evaluation suite and exploratory deep-variant evidence, and releases code, main-result weights, and evaluation scripts.The deep variant is released as code only.Source: §1 · p. 2
2 Related Work
The related work distinguishes CRN v2 from adaptation methods that place trainable capacity inside the model, weight-editing methods, and inference-time correction methods.
Parameter-efficient adaptation
- Adapters, LoRA, and prefix tuning inject trainable capacity inside the base model’s computation graph, whereas CRN v2 reads the final hidden state and corrects logits as a side network.The exploratory deep variant injects within the graph without updating base parameters.Source: §2 · p. 2
Model editing
- ROME and MEMIT rewrite transformer weights to edit factual associations, while CRN v2 uses additive logit adjustment and leaves the base intact.The paper characterizes the latter as a lighter-weight alternative to weight surgery.Source: §2 · p. 2
Error correction in LLMs
- Self-refinement and chain-of-verification use multiple inference-time forward passes, while CRN v2 is a trained correction module that produces corrected logits in a single forward pass.Autoregressive generation still requires one base forward pass per generated token.Source: §2 · p. 3
Forgetting and regularization
- The paper contrasts weight-protection regularization with CRN v2’s frozen base, which it says sidesteps catastrophic forgetting because the base is not updated.The cited related work describes catastrophic forgetting during fine-tuning and regularization penalties for protecting important weights.Source: Capability preservation · p. 3
3 Method
CRN v2 reads the frozen model’s final hidden state to produce logit corrections and trains the correction module with supervised fine-tuning and reference-free preference optimization. The method section also describes the data, inference procedure, matched-budget LoRA baseline, and exploratory hidden-state injection variant.
3.1 Architecture
- The architecture applies the correction module to the frozen base’s final hidden state, and its 34,013,185 trainable parameters are approximately 0.73% of the 4.65B text module.The base model produces hidden states and logits; the next-token distribution is computed from corrected logits.Source: §3.1 · p. 3
3.2 Training
- Training uses answer-only, anchor-weighted supervised fine-tuning with a KL preservation term, followed by reference-free DPO that prefers the correct answer over the frozen base’s wrong answer.The KL term pushes the correction toward base behavior where the base handles inputs correctly, while allowing divergence where it is wrong.Source: §3.2 · p. 3
3.3 Data
- The training file contains 83,400 correction pairs across facts, arithmetic, and implicit-goal reasoning, and the CEHRI exam prompts appear verbatim in that training distribution.The 120-question reworded exam consists of near-duplicate paraphrases, so neither exam tests out-of-distribution generalization.Source: §3.3 · p. 4
3.4 Inference
- At inference, CRN v2 greedily decodes from corrected logits after a base-model forward pass over the full generated sequence at each token.The stated implementation uses no KV cache and takes approximately 1 second per token on Apple M4 (MPS).Source: §3.4 · p. 4
3.5 Baseline: LoRA at the CRN v1 budget
- The LoRA baseline matches the CRN v1 parameter budget with rank 19 across eight depths, uses the same data and DPO objective, and receives more SFT and DPO steps than CRN v2.CRN v2 has approximately five times as many parameters as this LoRA baseline.Source: §3.5 · p. 4
3.6 Variant: deep hidden-state injection
- The deep variant injects a low-rank residual at a chosen hidden-state layer so that it propagates through frozen downstream layers before reaching the output head.The tested single-depth injection locations are layers 7 and 4; the base parameters receive no gradients.Source: §3.6 · pp. 4–5
4 Results
CRN v2 corrects fewer CEHRI errors than LoRA but preserves the frozen base’s scores on the tested capability benchmarks; the KL ablation and exploratory variants qualify this result.
4.1 Error correction
- CRN v2 corrects 53.3% of base-model errors on the original CEHRI exam and 43.3% on its reworded variant.The frozen base answers 7/60 correctly; CRN v2 adds 25 correct answers on the original exam under substring matching.Source: Table 1 · p. 5, §4.1 · p. 5
4.2 Capability preservation
- CRN v2 matches the frozen base on MMLU, BoolQ, and car-wash, while LoRA degrades 17–75 pp across those benchmarks; CEHRI correction is 53.3% versus LoRA’s 83.3%.Table 2 uses N=200 per task except car-wash (N=8).Source: Table 2 · p. 5, §4.2 · p. 5, §4.1 · p. 5
4.3 KL preservation ablation
- Lowering λ from 0.1 to 0.01 reduces correction from 53.3% to 35.0%, but the ablation also changes rank and DPO steps.The authors caution that these confounds prevent attributing the drop to λ alone.Source: Table 3 · p. 6, §4.3 · p. 6
4.4 Injection-depth sweep
- In the exploratory sweep, layer-7 injection reaches 50.0%/55.8% and layer 4 reaches 30.0%/28.3%; no tested alternative exceeds logit correction’s 53.3%.Multi-depth logit correction reaches 40%, while longer training remains at 53.3%; alternative configurations require independent verification.Source: Table 4 · p. 6, §4.4 · pp. 6–7
4.5 Training time
- On an Apple M4 with 16 GB memory, SFT for 2,000 steps plus DPO for 500 steps takes about 25 minutes end-to-end.Deep-variant SFT-only takes about 55–60 minutes for 2,000 steps on the same device.Source: §4.5 · p. 7
5 Discussion
CRN v2 corrects fewer errors than LoRA but preserves tested capabilities, while LoRA’s higher correction rate comes with capability losses. The authors describe ∼53% as the best result across tested configurations, not a proven ceiling, and limit broader claims about generalization and preservation.
5 Discussion
- 83.3% versus 53.3% correction: LoRA degrades capabilities by 17–75 pp, while CRN v2 preserves every tested capability.The comparison supports a correction–capability tradeoff within the tested benchmarks.Source: §5 · p. 7
- ∼53% is the best result across tested configurations, not a proven ceiling; the alternatives lack independent verification.The proposed mechanism is that LoRA changes weights throughout the network, whereas frozen-base methods add to activations or logits processed by unchanged layers.Source: §5 · p. 7
- Frozen-base adjustment with KL anchoring is presented as a design principle that can extend beyond Gemma 4 E2B.The released code is model-agnostic up to tokenizer and hidden-dimension constants.Source: §5 · p. 7
- The paper does not claim architectural novelty, out-of-distribution generalization, or universal capability preservation.All exam prompts appear verbatim in training, and capability claims are limited to three benchmarks tested at N=200/8.Source: §5 · p. 7
6 Limitations
The reported evidence is bounded by incomplete correction, a single domain exam with near-duplicate rewordings, small capability samples, and unresolved training, runtime, and exploratory-variant constraints.
6 Limitations
- 53.3% correction leaves 46.7% of exam errors unfixed, which the authors deem insufficient for high-stakes use.Source: §6 · p. 7
- The reworded exam uses low-diversity templates with 71.6% character overlap, rather than genuine paraphrases.Source: §6 · p. 7
- Capability evaluation uses N=200 for MMLU/BoolQ and N=8 for car-wash, limiting detection of finer differences.Source: §6 · p. 8
- The KL ablation confounds rank, λ, and DPO steps, so it does not isolate their individual effects.The authors call for a single-variable sweep.Source: §6 · p. 8
- Autoregressive generation requires a full base forward pass per token without KV cache, taking ∼1 s/token on M4.Source: §6 · p. 8
- The 83,400-row training set comes from 17,810 unique prompts, and its scaling behavior is unknown.Source: §6 · p. 8
- HellaSwag scores are unusable because the parser expects a single A–D letter while both models generate full-sentence completions.Source: §6 · p. 8
- Deep injection remains exploratory: only depth-7 SFT-only results are log-verified, while other rows are session-observed and capability probes are small.Depth-7 DPO destroyed capabilities in one run without a hyperparameter sweep.Source: §6 · p. 8
7 Conclusion
The conclusion presents CRN v2 as a small correction module trained over a frozen base, with logit adjustment and KL preservation. It reports error correction without measurable degradation on tested benchmarks, while acknowledging that tested alternatives do not establish a ceiling.
7 Conclusion
- CRN v2 applies a 34M-parameter correction module, 0.73% of the base, to a frozen model to fix errors without degrading base capabilities.Source: §7 · p. 8
- 53.3% correction with no measurable degradation contrasts with matched-budget LoRA’s 83.3% correction and 17–75 pp capability loss.Source: §7 · p. 8
- Tested alternatives did not exceed the rank-128 logit result, but the single-run exploratory results do not prove a ceiling.The conclusion identifies frozen-base logit correction with KL anchoring as the design principle and reports release of code, main-result weights, and evaluation scripts.Source: §7 · p. 8
Reproducibility
The paper reports committed scripts and fixed checkpoints for main results, with evaluation artifacts for correction and capability benchmarks. Deep-injection training code is released without trained deep checkpoints, and its exploratory rows have weaker verification.
Reproducibility
- Main-result numbers use committed scripts and fixed checkpoints, with no exam-threshold tuning; deep-variant rows are excepted.Source: Reproducibility · p. 8
- Recorded capability artifacts use identical prompts and report matching base and CRN v2 scores: MMLU 125/200, BoolQ 144/200, and car-wash 6/8.Source: Reproducibility · pp. 8–9
- Deep injection is released as code only, without trained deep checkpoints; only depth-7 SFT-only results are verified against retained logs.Source: Reproducibility · p. 9
- The released HuggingFace bundle contains CRN v2 weights and evaluation scripts; reported correction scores are 32/60 original and 52/120 reworded.Source: Reproducibility · pp. 8–9