PaperlayerView exact revision on arXiv

Source-linked AI summary

EdgeRock: Strong Reasoning at Constant Memory

Huishan Ji, Weiming Zhang, Qirui Ye, Jinchao Xu, Yike Guo, Hua Xu

arXiv:2610.10118v2cs.LGcs.CL

TL;DR

Reasoning histories that grow with every token consume accelerator memory, while fixed-state approaches can lose distant details that later reasoning needs. EdgeRock stores learned key–value records in a fixed-size bank, making retained details directly readable. EdgeRock-4B reaches 82.9% mean pass@1 on AIME 2024–2026, versus 18.8% for the strongest bounded-state comparator evaluated.

  • Problem

    Growing KV caches consume accelerator memory and constrain deployment, while bounded-state models may not preserve distant details needed for later reasoning.

    Source: §1 · p. 2

  • Method

    Rock Memory Anchors learn which details to store as key–value records in a fixed-size bank that later queries can read directly.

    Source: §1 · p. 2

  • Results

    82.9% mean pass@1 on AIME 2024–2026 for EdgeRock-4B, versus 18.8% for RWKV-7 G1j 13.3B, the strongest bounded-state AIME comparator.

    Source: §6.1 · p. 7

  • Takeaways & Limitations

    With fixed history, memory can be sized once at design time and relied on for as long as the model reasons.

    Source: §9 · p. 13

    Whether constant memory keeps pace with stronger reasoning remains to be tested by scaling RMA to larger backbones.

    Source: §9 · p. 13

Abstract

from arXiv · show

We present EdgeRock, a family of multimodal models that deliver strong reasoning at constant memory. Most models keep reasoning history in a key-value (KV) cache that grows with every token, consuming accelerator memory, limiting concurrent requests, and competing with perception and control on edge devices. With fixed history, memory is allocated once and never runs out, however long the model reasons. Constant memory alone is not new: bounded-state models (recurrent, state-space, and sliding-window) already achieve it, but often at the cost of reasoning, because they cannot keep the distant details that later reasoning needs. We therefore introduce Rock Memory Anchors (RMA): the model learns what is worth remembering and stores it as KV records in a fixed-size bank, directly readable however far back the source lies. EdgeRock-4B, our first open-source release, supports text and image reasoning and reaches 82.9% mean pass@1 on AIME 2024-2026, while the bounded-state models we evaluate reach at most 18.8%, even at 14B scale. This result is on par with leading compact models whose memory grows with reasoning. Compared with its source model, EdgeRock-4B uses up to 99.1% less KV cache and delivers up to 6.42$\times$ higher end-to-end batch throughput.

1 Introduction

EdgeRock targets reasoning systems that need a fixed history budget without losing access to important distant details. It uses Rock Memory Anchors to retain selected records in a fixed-size bank, and its first open release demonstrates compatibility between constant memory and strong reasoning.

1 Introduction

  • Under full attention [1], each generated token adds to the KV cache, consuming HBM and limiting concurrent requests; cache-compression methods still leave it growing with tokens.The paper distinguishes reducing cache growth from eliminating it.Source: §1 · p. 2
  • A fixed history budget lets edge deployments reserve memory predictably for sequences of different lengths instead of reserving for the longest trace or cutting reasoning short.The paper frames predictability, not only efficiency, as the need for robots, vehicles, and dedicated chips.Source: §1 · p. 2
  • Across 16 bounded-state models up to 14B parameters, none exceeds 18.8% on AIME 2024–2026, while distant details can blur in recurrent summaries or leave a sliding window.The paper identifies definitions, problem constraints, and intermediate results as examples of details later reasoning may need.Source: §1 · p. 2
  • EdgeRock stores selected learned key–value records in a fixed-size anchor bank, preserving direct access to valuable details beyond the 1,024-token local window.The recurrent state summarizes evolving context; anchors remain separate and unchanged until displaced by admission.Source: §1 · p. 2
  • EdgeRock-4B, the first open release, shows that constant memory and strong reasoning are compatible (Table 1).The supplied table passage identifies Table 1 as an at-a-glance summary.Source: §1 · p. 2, Table 1 · p. 2

2 How is EdgeRock built?

EdgeRock combines recurrent processing, local attention, and Rock Memory Anchors in a shared text-and-image decoder. Its persistent history has fixed capacity and the model is converted through staged adaptation and post-training.

2.1 Decoder layout

  • The decoder retains 24 Gated DeltaNet layers and replaces eight full-attention layers with a 1,024-position sliding window and an independent RMA module.Text embeddings and image features from the visual encoder and merger enter a shared decoder sequence.Source: §2.1 · p. 3

2.2 Fixed capacity, adaptive content

  • EdgeRock keeps persistent history fixed in size while updating its contents, tracking recurrent states, local KV, committed anchor banks, pending candidates, and progress metadata.The candidate block contains 256 causal positions.Source: §2.2 · p. 3

2.3 Integration into the reasoning computation

  • Each local-plus-RMA layer integrates retrieved information alongside local attention into the residual stream, with RMA using independent memory and projection components.The residual carries local and retrieved information to later layers.Source: §2.3 · p. 4

2.4 Staged conversion

  • Conversion proceeds through RMA adaptation, joint adaptation with the recurrent and local-attention backbone, supervised post-training, and reinforcement learning with verifiable rewards.The stages teach retention and retrieval, train downstream layers to use retrieved information, develop responses, and refine them using answer-correctness and task-completion feedback.Source: §2.4 · p. 4

3 How does RMA preserve distant information?

RMA writes candidate records, admits selected records to a bounded bank, and lets later queries retrieve retained details directly. Blockwise updates preserve causal visibility while query-conditioned retrieval and gating determine memory's contribution.

3 How does RMA preserve distant information?

  • At each token, RMA reads the committed bank and produces a candidate; at block boundaries, selected records become readable in the next block.Local attention covers recent tokens while candidates accumulate.Source: §3 · p. 4

3.1 Independent memory writer

  • An independent writer constructs storage representations from normalized decoder states, and RMA uses those writer outputs for its key and value projections.Query projections instead use the normalized current representation.Source: §3.1 · p. 4

3.2 Causal admission in four independent KV groups

  • A small network scores retention priority for each KV group, which independently keeps its highest-scoring records.The projections compute FP32 scores from current causal context.Source: §3.2 · p. 4

3.3 Blockwise commits and causal visibility

  • Every 256 tokens, each KV group selects among its committed bank and new candidates for 1,024 slots; selected records become visible in the next block.Each update inspects at most 1,280 records per group, and capacity limits record count rather than retention age.Source: §3.3 · pp. 4–5

3.4 Query-dependent retrieval from the anchor bank

  • A later query can address any retained record beyond the local window; query heads share each group's bank but compute distinct attention weights.The query reads the current representation while keys and values store writer representations; an empty bank returns exactly zero.Source: §3.4 · p. 5

3.5 Query-conditioned memory output

  • A signed, query-conditioned gate controls each retrieved head's contribution to the residual stream, while admission chooses records and retrieval determines content and attention mass.The gate's sign comes from a learned scale and its query-conditioned sigmoid modulates magnitude.Source: §3.5 · p. 5

Relation to other retained-memory designs

  • Unlike AnLLM [29], HOLA [30], and LTE [31], RMA uses independent writer records, causal per-group top-K commits, and a separately normalized, gated read.AnLLM anchors accumulate with sequence length; HOLA uses fixed parameter-free priorities, while LTE learns per-head retention scores.Source: §3.5 · p. 6

4 What stays constant as context grows?

EdgeRock keeps history storage and per-token history access bounded as context grows. Its fixed-size recurrent state, local window, anchor banks, and candidate storage support that bound, while the runtime keeps each request’s history in its own slot.

4.1 Memory that does not grow with sequence length

  • The complete persistent history allocation is O(1) in prompt plus generated length because every stored component has bounded capacity.The allocation includes recurrent state, the local ring, committed anchors, pending candidates, scores, and indices.Source: §4.1 · p. 6
  • RMA, recurrent state, and a 1,024-token local window use 72 MiB of attention KV per sequence, about 99.1% less than an 8 GiB 256K-token full-attention cache.Table 3’s persistent-history entries are fixed in size independent of context length.Source: §4.1 · p. 6, Table 3 · p. 6

4.2 Work per generated token

  • Each generated token updates fixed recurrent states and reads at most W local positions and K anchors per group; block commits select from at most K + C records.The stated conservative sorting bound for selection accompanies the per-token work analysis.Source: §4.2 · p. 6
  • EdgeRock’s streamed, bounded-chunk prefill gives O(P + N) total stream-processing work, while full attention has O(PN + N^2) attention work during decode and O(P + N) history storage.The passage says compression and sparse or hybrid attention lower constants but leave history storage linear in P + N.Source: §4.2 · p. 6

4.3 Inference cache and request concurrency

  • History storage grows with the number of concurrent requests, not their length, and each request keeps its history in its own slot.Requests that repeat a prompt can share prefill, but each continuation keeps its own state.Source: §4.3 · p. 7

5 Evaluation design

The evaluation compares EdgeRock with bounded-state models, its source model, and compact growing-history models, and examines memory, throughput, and visual reasoning. It reports benchmark aggregation rules and uses separate execution profiles for memory and visual evaluations versus runtime experiments.

5 Evaluation design

  • The evaluation covers 28 models on 29 text benchmarks, plus visual reasoning, distant memory, and long-sequence inference.Complete text scores are reported in Appendices B and C.Source: §5 · p. 7

5.1 Comparison objectives

  • Five comparisons test bounded-state reasoning, conversion cost against Qwen3.5-4B, standing among compact growing-history models, dependence on RMA for distant answers, and runtime performance.The runtime comparison covers decode rate, batch throughput, and GPU memory.Source: §5.1 · p. 7

5.2 Metrics and execution profiles

  • The overall mean weights all 29 primary metrics equally, AIME averages the 2024–2026 editions equally, and paired Qwen3.5-4B comparisons report retention.Most tasks report accuracy; DROP reports F1.Source: §5.2 · p. 7
  • Memory and visual evaluations use reference arithmetic, while runtime experiments use the optimized profiles in Section 8.Appendix A specifies sampling, scoring, and source configurations.Source: §5.2 · p. 7

6 Does strong reasoning survive constant memory?

EdgeRock-4B brings strong reasoning to bounded-state models, leading the evaluated class while retaining much of its source model’s performance. It also performs in the range of compact growing-history models and extends constant-memory reasoning to images.

6.1 A clear lead over 16 constant-memory models

  • EdgeRock-4B leads 17 evaluated bounded-state models in the 29-benchmark mean and AIME 2024–2026 summaries.The 29-benchmark mean is 23.8 percentage points above AHN-GDN Qwen2.5 14B, the strongest baseline on that aggregate.Source: §6.1 · p. 7, Figure 3 · p. 8
  • 82.9% mean pass@1 on AIME 2024–2026 compares with 18.8% for RWKV-7 G1j 13.3B, the strongest bounded-state AIME comparator.Source: §6.1 · p. 7, Figure 3 · p. 8

6.2 Retaining 96% of the source model’s reasoning

  • EdgeRock-4B retains 96.3% of Qwen3.5-4B’s equal-weight AIME 2024–2026 mean pass@1 after conversion.It also reaches 97.5% on MATH-500.Source: §6.2 · p. 7, Table 4 · p. 8
  • EdgeRock-4B exceeds its Qwen3.5-4B source on eight of 29 text benchmarks, including MMLU, MMLU-Pro, MMLU-Redux, SuperGPQA, HumanEval, and MBPP.Figure 4 shows benchmark deltas in percentage points and aligned absolute scores.Source: §6.2 · p. 8, Figure 4 · p. 9

6.3 In the range of compact models with growing memory

  • On AIME 2024–2026, EdgeRock-4B’s 82.9% lies between Granite4.2-3B (82.2%) and Qwen3.5-4B (86.1%), and its score is in the range of leading compact growing-history models.It also exceeds Nemotron-3-Nano-4B, LFM2.5-2.6B, Gemma4-E4B, and Phi-4-mini-reasoning on AIME.Source: §6.3 · p. 8, Table 5 · p. 9

6.4 Vision retained at constant memory

  • Across six visual benchmarks, EdgeRock-4B reaches 84.9% equal-weight mean answer-level accuracy versus Qwen3.5-4B’s 89.1%, retaining 95.3%.The benchmarks cover perception, chart and diagram understanding, and visual question answering.Source: §6.4 · p. 10, Table 6 · p. 10

7 Do distant answers depend on RMA?

Tests of distant-memory reasoning find that EdgeRock answers questions about evidence beyond its local window when RMA is enabled. Disabling RMA sharply reduces query accuracy and complete-context success, while task and paired-evidence tests probe what the model retrieves and uses.

7 Do distant answers depend on RMA?

  • With RMA enabled, EdgeRock answers 94.1% of queries and completes 80.1% of contexts across four distant-memory tasks.Query accuracy rises from 79.4% after RMA adaptation to 96.1% after joint adaptation, then is 94.1% in the final model.Source: §7 · p. 10, Figure 5 · p. 10

7.1 Without RMA, distant recall collapses

  • With RMA disabled at inference, correct answers fall from 964 to 4 of 1,024 queries, and complete-context success falls from 205 to 0 of 256 contexts.Recurrence and local attention remain active, supporting the conclusion that distant answers in the trained model flow through RMA.Source: §7.1 · p. 10

7.2 RMA across memory tasks

  • Complete-context success is 90.6% on exact retrieval, 87.5% on state-update interpretation, and 89.1% on cross-document combination.Source: §7.2 · p. 10, Table 7 · p. 11
  • EdgeRock answers both versions correctly in 89.8% of pairs requiring a changed answer and 96.5% of pairs where the answer should stay the same.Among the latter pairs, the answer never changed.Source: §7.2 · p. 11, Table 7 · p. 11

8 What does constant memory buy in deployment?

On one Hopper 80GB GPU, EdgeRock’s fixed history supports high decode rates, concurrent long generations, and bounded memory use. Its reported throughput advantages include single-request decoding and batch workloads against compact comparison models.

8 What does constant memory buy in deployment?

  • EdgeRock-4B averages 4.94× Qwen3.5-4B’s end-to-end throughput and runs 512 long-generation sequences at once on one Hopper 80GB GPU.The paper attributes the gains to fixed history, which keeps per-token work from growing with context and allows more sequences to run concurrently.Source: §8 · p. 11

8.1 Steady decoding and higher throughput

  • EdgeRock sustains 233.7 tokens/s through 128K output after 128K input, 1.54× Qwen3.5-4B’s rate at batch size 1.Across 4K–32K inputs with 32K output, it averages 4.94× Qwen3.5-4B’s end-to-end throughput; at 32K input and output, its history is 97.76 MiB per sequence versus Qwen3.5-4B’s 2 GiB attention KV.Source: §8.1 · pp. 11–12, Figure 6 · p. 11

8.2 Concurrency scales to 512 sequences on one GPU

  • At batch size 512, weighted decoding reaches 12,497 tokens/s, or 24.4 tokens/s per active sequence, across 16 input/output combinations.Source: §8.2 · p. 12, Figure 7 · p. 12

8.3 Efficient decoding at short contexts

  • At batch size 1, EdgeRock-4B reaches 229.6 tokens/s for 4K input and 2K output, excluding prefill.At batch size 512, the same workload reaches 12,481 tokens/s.Source: §8.3 · p. 12

8.4 Memory known before reasoning starts

  • Each sequence carries 97.76 MiB of history from 4K through 256K reasoning, and 512 concurrent sequences need at most 61.26 GiB across the tested workloads.A single 128K-input/128K-output request peaks at 16.02 GiB on one Hopper 80GB GPU.Source: §8.4 · p. 12, Figure 8 · p. 12

9 Conclusion

EdgeRock argues that strong reasoning can use fixed history, making memory depend on the model rather than input length. This fixed budget has potential applications in robotics, dedicated chips, agents, and serving systems; scaling and extending RMA remain future directions.

9 Conclusion

  • Fixed history makes memory a property of the model rather than input length, allowing it to be sized once and relied on for as long as reasoning continues.The conclusion states that EdgeRock demonstrates strong reasoning without memory growing with every token.Source: §9 · p. 13
  • A fixed budget could let robots run perception and reasoning continuously, fit a model’s history on-chip, prevent agent memory from growing with tool calls, and let serving capacity be planned by request count.These are applications the paper proposes, not evaluated outcomes in this passage.Source: §9 · p. 13
  • Scaling RMA to larger backbones and other hybrid architectures, extending anchors to image, audio, and sensor streams, and inspecting remembered source positions are future directions.The authors say additional EdgeRock sizes are in preparation and that adapting other hybrids could avoid pretraining from scratch.Source: §9 · p. 13

A Evaluation and verification protocol

The evaluation combines text benchmarks, visual benchmarks, and long-memory diagnostics, with task-specific scoring and sampling protocols. Visual results use six benchmarks, while memory tests cover four kinds of context and include paired interventions.

Text evaluation and sampling

  • Text evaluation uses fixed prompts, reference answers, and executable tests across 29 benchmarks.Binary task scores use mean pass@1 with benchmark-prescribed aggregation, and AIME weights the 2024–2026 editions equally.Source: A Evaluation and verification protocol · p. 16

Task-specific verification and aggregation

  • Task-specific scoring uses answer extraction and executable tests for code, while IFEval and IFBench use strict constraint checkers.Conflicting or unresolved answers receive no credit; code uses relaxed program extraction followed by executable tests.Source: A Evaluation and verification protocol · p. 16
  • Benchmark means weight benchmarks equally while retaining each benchmark’s designated primary metric; BBH, BBEH, CMMLU, MuSR, DROP, and LogiQA2 retain their specified aggregation or metrics.MMLU-Redux uses corrected choices; MuSR averages its three task accuracies, and DROP reports token F1 and exact match.Source: A Evaluation and verification protocol · p. 16

Visual and long-memory evaluation

  • The six visual benchmarks are AI2D, ChartQA, MMBench EN/CN, MME, and POPE, and their visual mean weights them equally.The reported metrics include 1,600.4 MME Perception points, 728.2 Cognition points, 2,328.7 total points, 81.6% MMBench EN circular accuracy, and 85.5% six-benchmark answer-weighted accuracy.Source: A Evaluation and verification protocol · p. 16
  • The memory suite contains 256 contexts and 1,024 queries across exact retrieval, state-update interpretation, cross-document combination, and variable binding.It uses 64 contexts per type and four queries per context; complete-context success requires all four answers to be correct.Source: A Evaluation and verification protocol · p. 16
  • Paired interventions change distant evidence while holding surrounding structure fixed, and paired accuracy requires both answers to be correct.The tests distinguish answers that should change from those that should remain stable; the branch-disabled intervention retains the trained backbone and local window.Source: A Evaluation and verification protocol · p. 16

B Complete bounded-state benchmark results

The appendix specifies benchmark coverage, workload and scoring protocols, runtime configurations, memory accounting, and request-state handling. It describes bounded-state and growing-history comparison tables but the supplied passages do not report their cell values.

B Complete bounded-state benchmark results

  • The bounded-state results cover EdgeRock-4B and 16 comparators across 29 benchmarks, with values using each task’s primary metric on a 0–100 scale.The benchmark tables span mathematics, knowledge and science, commonsense and reasoning, coding and instruction following, and reading and multi-step reasoning.Source: B Complete bounded-state benchmark… · p. 17, Table 8 · p. 17, Table 9 · p. 17

C Complete growing-history benchmark results

  • The growing-history results compare EdgeRock-4B with eleven models across 29 benchmarks, with bold marking column maxima including ties.The supplied passages identify tables covering mathematics, knowledge and science, commonsense and reasoning, coding and instruction following, and reading and multi-step reasoning.Source: C Complete growing-history benchmark… · p. 19, Table 13 · p. 19, Table 14 · p. 20

D.1 Workloads and hardware

  • Workloads use one Hopper 80GB GPU with BF16 weights and KV, including 128K-input/128K-output single requests and throughput sweeps over multiple input, output, and batch sizes.The full EdgeRock sweep combines four input lengths with four output lengths and batch sizes 32–512; the short sweep covers batch sizes 1–512.Source: §D.1 · p. 21

D.2 Timed intervals and aggregation

  • Decode rate excludes the first token, while end-to-end output-token throughput includes prefill time; mean throughput ratios weight four 32K-output workloads equally.The weighted batch throughput aggregates subsequent output tokens and decode time across 16 length pairs and three runs.Source: §D.2 · p. 21

D.3 Runtime configuration and prompt reuse

  • The runtime tests different batch grids across model families, while EdgeRock can share repeated prompts across eight continuations that keep independent state.The tested batch sizes differ by model; vLLM peers use CUDA Graphs, chunked prefill, and a 16K prefill-token budget.Source: §D.3 · p. 21

D.4 History and device allocation

  • Each EdgeRock sequence has 97.76 MiB of history, including recurrent and other state, while its configuration-derived attention KV bound is 72 MiB.At batch size 512, shared storage for original and packed BF16 projections avoids duplicate weights while retaining full history capacity.Source: §D.4 · p. 22

D.5 Measured throughput and memory

  • The measured-throughput tables report output-token throughput for 32K output, weighted decoding and peak memory across 16 input/output pairs, and a separate 4K-input/2K-output test.The captions describe measurements on one Hopper 80GB GPU.Source: Table 19 · p. 22, Table 20 · p. 22, Table 21 · p. 23

E.1–E.4 Request state and forward procedure

  • A continuing request retains recurrent, local, anchor, candidate, and progress state; committed banks are read during each 256-token block, with selected candidates becoming available at its boundary.Local attention covers recent tokens, and commits are independent of prefill chunk boundaries.Source: §E.1 · p. 23, §E.2 · p. 24, §E.3 · p. 24

Related papers

Loading 2610.10118v2…