PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentOCR: Reimagining Agent History via Optical Self-Compression

Lang Feng, Fuchao Yang, Feng Chen, Xin Cheng, Haiyang Xu, Zhenglin Wan, Ming Yan, Bo An

arXiv:2601.04786v3cs.LGcs.AI

TL;DR

Long multi-turn agent histories strain context limits and increase computational costs. AgentOCR renders history as images, adds optical caching and adaptive self-compression, and preserves over 95% of text-based agent performance while reducing token consumption by over 50%.

  • Problem

    Growing multi-turn histories strain LLM context limits and raise computational costs, motivating more efficient history representation.

    Source: §1 · p. 1, §4 · p. 3

  • Method

    AgentOCR represents interaction history as a compact image, reuses rendered segments through optical caching, and trains agents to adapt visual compression.

    Source: §1 · pp. 1–2

  • Results

    Over 95% of strong text-based agent pipeline task performance is preserved while token consumption falls by over 50% across ALFWorld and search-based QA.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results suggest visual history can offer agents a resource-efficient alternative to text-only processing while retaining most decision-making capability.

    Source: §6 · p. 9

    The supplied Limitations passages signal that limitations warrant discussion but do not specify a particular limitation or scope boundary.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Recent advances in large language models (LLMs) enable agentic systems trained with reinforcement learning (RL) over multi-turn interaction, but practical deployment is bottlenecked by rapidly growing textual histories that inflate token and memory costs. We introduce AgentOCR, a framework that exploits visual tokens' superior information density by representing the accumulated observation-action history as a compact rendered image. To make multi-turn rollouts scalable, AgentOCR proposes segment optical caching. By decomposing history into hashable segments and maintaining a visual cache, this mechanism eliminates redundant re-rendering. Beyond fixed rendering, AgentOCR introduces agentic self-compression, where the agent actively emits a compression rate and is trained with compression-aware reward to adaptively balance task success and token efficiency. We conduct extensive experiments on challenging agentic benchmarks, ALFWorld and search-based QA. Remarkably, AgentOCR preserves over 95\% of text-based agent performance while substantially reducing token consumption (>50\%), yielding consistent token and memory efficiency. Further analysis validates a 20$\times$ rendering speedup from optical caching and effective self-compression balancing. Our code is available at https://github.com/langfengQ/AgentOCR.

1 Introduction

AgentOCR addresses the growing costs of multi-turn agent histories by representing accumulated observations and actions as images, with caching and learned self-compression to improve rollout efficiency.

1 Introduction

  • Long multi-turn histories exhaust finite token budgets and raise inference latency and compute costs; rendering text as images can reduce token footprint by approximately 10×.The introduction motivates visual information density as a way to reduce the burden of long-context processing.Source: §1 · p. 1
  • AgentOCR renders accumulated observation-action history as a compact image and conditions the policy on this visual history for multi-turn decisions.This replaces a growing text history with a dynamic sequence of images.Source: §1 · p. 1
  • Segment optical caching divides history into segments and uses a hash-based cache to reuse rendered content, avoiding redundant processing as history expands.The paper reports over 20× faster rendering with this mechanism.Source: §1 · p. 2
  • Agentic self-compression lets the agent choose a compression factor at each step through RL, balancing task success against token cost.The analysis finds that self-compression effectively optimizes the trade-off between information density and cost.Source: §1 · p. 2
  • On ALFWorld and search-based QA, AgentOCR preserves over 95% of strong text-based agent pipeline task performance while reducing token consumption by over 50% (up to 80% in peak tokens).Segment optical caching accelerates rendering by over 20×.Source: §1 · p. 2

2 Related Work

Related work covers reinforcement learning for LLM agents, OCR’s development toward vision-text compression, and approaches to long-horizon agent memory.

2.1 Reinforcement Learning for LLM Agents

  • RL is used to align LLM agents with human preferences and improve behavior, with deployments spanning games, GUI control, embodied tasks, web, and tool environments (Sheng et al., 2024).The cited environments are described as dynamic and open-ended.Source: §2.1 · p. 2

2.2 Optical Character Recognition

  • OCR has progressed from text extraction and document parsing to end-to-end image-to-text modeling, and is newly being explored for vision-text compression, a direction still in its early stages (Wei et al., 2025; Xing et al., 2025; Cheng et al., 2025).The passage identifies document digitization, image text extraction, and document parsing as established OCR applications.Source: §2.2 · pp. 2–3

2.3 Agent Memory

  • Agent-memory research addresses context limits through long-context optimization, retrieval, explicit memory modules, and learning-based memory control (Yan et al., 2025; Zhou et al., 2025; Yu et al., 2025a).Methods include sparse or hierarchical attention, recency-biased positional schemes, prompt compression, external stores, and structured or hierarchical representations; learning-based control adapts what agents write, retain, and retrieve.Source: §2.3 · p. 3

3 Preliminaries

The paper models an LLM agent as a stochastic policy interacting with an environment over a finite horizon, with actions conditioned on the instruction and accumulated history. It uses GRPO as a representative agentic RL algorithm and motivates history compression by the cost of processing long trajectories.

3.1 Problem Setup

  • At each step, an LLM policy receives an observation and samples a textual action conditioned on the task instruction and accumulated observation-action history, which can grow extensively.Actions may include intermediate reasoning or tool invocations; after execution, the environment returns a scalar reward and the next observation.Source: §3.1 · p. 3

3.2 Agentic Reinforcement Learning

  • GRPO is the representative, though not exclusive, agentic RL algorithm, and its clipped objective uses group-normalized reward advantages while requiring gradients over the cumulative history.Realistic trajectories can exceed 10k tokens in multi-turn search tasks, making processing prohibitively expensive as computation and memory scale with token count.Source: §3.2 · p. 3

4 AgentOCR

AgentOCR renders accumulated interaction histories as compact images to reduce textual token costs while retaining access to historical details. It adds segment-level caching to avoid redundant rendering and agent-controlled compression to balance information density against token cost.

4.1 Optical Memory Encoding

  • AgentOCR serializes buffered interaction records into history text, deterministically renders that text as an RGB image, and supplies the image with the observation to a vision-language policy.The buffer stores observation-action pairs or task-specific equivalents, and the policy's sampled action updates the environment and buffer.Source: §4.1 · p. 4

4.2 Segment Optical Caching

  • Segment caching renders only unseen segments, reuses content-keyed images within an episode, and stacks them in order, reducing per-step rendering to O(Ut) misses.Unlike appending only new context, segment caching can reuse recurring content; it stores one image per unique segment per episode rather than duplicating full-history images across timesteps.Source: §4.2 · pp. 4–5

4.3 Agentic Self-Compression

  • The agent invokes rendering with a compression factor ct ≥1, spatially downsampling history images to reduce visual tokens while adapting fidelity to task needs.RL compression rewards apply only to successful episodes and use a logarithmic formulation; injecting them every K training iterations mitigates overly greedy compression while preserving task-completion pressure.Source: §4.3 · pp. 5–6

5 Experiment

The experiments evaluate AgentOCR’s performance and token efficiency against text agents, vision-text compression, segment optical caching, and agentic self-compression.

5.1 Experimental Setup

  • Experiments compare embodied ALFWorld (Shridhar et al., 2021) with tool-using, text-dense search QA (Jin et al., 2025) against text-only, optical-history, and Text+GRPO baselines.Text+GRPO applies RL to raw textual context, whereas AgentOCR applies GRPO to the optical-memory agent.Source: §5.1 · p. 6

5.2 Main Results

  • AgentOCR scored 78.2%/81.2% on ALFWorld (3B/7B) and 40.1% vs. 41.9% for 7B Search Text+GRPO, retaining over 95% of search performance while cutting tokens over 50%.Average token reductions were approximately 55% on ALFWorld and 70% on search tasks, reaching 80.9% in peak contexts; optical history alone reduced tokens but initially hurt performance.Source: §5.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7

5.4 Analysis of Cache

  • Segment optical caching averages 168.77 ms latency and saves 26.82% peak memory versus naive caching, while no cache averages 3509.39 ms latency.Naive caching achieves 17.28× speedup over no cache but uses 151.41 MB peak memory; segment caching has −1.23 ms/step time growth and uses 110.80 MB by reusing newline-level segments.Source: §5.4 · pp. 7–8, Table 3 · p. 8

5.3 Vision-Text Compression Analysis

  • At ct=1.2, ∼55% token savings preserves over 95% of text-based performance (99.5% ALFWorld; 95.0% Search), but ∼67% savings at ct=2.0 sharply degrades it.At ct=2.0, ALFWorld retains 87.2% of performance, while Search falls to 66.8%; the findings motivate dynamic compression that adjusts fidelity to task sensitivity.Source: §5.3 · p. 7, Figure 3 · p. 8

5.5 Analysis of Self-Compression

  • With RL at K=5, self-compression reduced average visual tokens from 458.1 to 381.7 while achieving 78.2% success, comparable to the 78.5% uncompressed baseline.Without RL, compression barely changes token usage; with dense reward at K=1, the agent raises the compression factor to 4.91 and success falls to 45.3%.Source: Table 4 · p. 8, §5.5 · p. 8

6 Conclusions

AgentOCR explores visual tokens as a compact history medium for multi-turn agents, combining optical caching and agentic self-compression. Results on ALFWorld and search-based QA suggest it retains most decision-making capability while reducing token consumption.

6 Conclusions

  • AgentOCR combines segment optical caching to reduce rendering overhead with agentic self-compression to adaptively balance cost and fidelity.The framework explores visual tokens as a compact history medium that complements textual history.Source: §6 · p. 8
  • On ALFWorld and search-based QA, the optical approach retains the majority of agents’ decision-making capabilities while significantly reducing token consumption.The conclusion presents it as a resource-efficient alternative to text-only processing.Source: §6 · pp. 8–9
  • Future research may explore hybrid storage architectures and unified multimodal interfaces to advance versatile and efficient information processing.The conclusion frames this direction as moving closer to information processing found in biological systems.Source: §6 · p. 9

Limitations

AgentOCR’s limitations concern its evaluated VLM range, fixed rendering configuration, and focus on text-centric histories. The authors identify broader model, rendering, and multimodal-history evaluation as future directions.

Limitations

  • AgentOCR uses Qwen2.5-VL models and is not evaluated across broader VLM architectures or visual tokenization strategies, which may affect performance and compression efficiency.The authors cite DeepSeek-OCR (Wei et al., 2025) as an example of an architecture with different visual tokenization strategies or patch resolutions.Source: Limitations · p. 9
  • AgentOCR uses a deterministic renderer with fixed hyperparameters, and the effects of alternative rendering choices on legibility, layout cues, and reasoning remain untested.The default configuration shows stable performance, but suboptimal settings may reduce text legibility or distort layout cues.Source: Limitations · p. 9
  • AgentOCR assumes histories are primarily text, leaving compression of visual elements such as GUI screenshots, plots, diagrams, and structured tables for future investigation.The authors suggest multimodal-history compression could expand the framework beyond text-centric domains.Source: Limitations · p. 9

Ethical Considerations · A Pseudo Code

The ethical discussion notes that experiments use public benchmarks and synthesized environments without sensitive personal information, and that real-world deployment is not yet involved. The appendix provides pseudocode describing cache/rendering mechanics and policy control of visual resolution.

Ethical Considerations

  • Experiments use established public benchmarks and synthesized environments, without sensitive personal information or data privacy concerns.Source: Ethical Considerations · p. 9
  • The work does not currently involve real-world deployment, so its immediate social impact is limited.Source: Ethical Considerations · p. 9
  • In the long term, reduced token overhead may support energy-efficient “Green AI” and deployment on resource-constrained devices.Source: Ethical Considerations · p. 9
  • LLMs were used only to refine writing through grammar correction, style improvement, and wording adjustments for clarity and conciseness.Source: Ethical Considerations · p. 9

A Pseudo Code

  • Detailed AgentOCR pseudocode is provided to facilitate reproduction.Source: A Pseudo Code · p. 14
  • Algorithm 1 maintains the segment cache, runs rendering at the requested compression ct, and computes an efficiency-aware reward.Source: A Pseudo Code · p. 14
  • Algorithm 2 describes an interaction loop in which the policy selects ct to control the resolution of the next visual observation.Source: A Pseudo Code · p. 14

B Experiments · B.1 Details of Benchmarks

The experiments evaluate embodied household-task completion in ALFWorld and single- and multi-hop question answering with iterative retrieval in search-based QA.

B.1 Details of Benchmarks

  • ALFWorld comprises 3,827 publicly available tasks for non-commercial research spanning six household-task categories.The categories are Pick & Place, Examine in Light (Look), Clean & Place (Clean), Heat & Place (Heat), Cool & Place (Cool), and Pick Two & Place.Source: §B.1 · p. 14
  • In ALFWorld, agents choose actions from observations and interaction history, then receive environment feedback to verify task completion.The benchmark is an embodied environment (Shridhar et al., 2021).Source: §B.1 · p. 14
  • Search-based QA uses the Search-R1 dataset (Jin et al., 2025), comprising single-hop NQ, TriviaQA, and PopQA benchmarks.The cited datasets are NQ, TriviaQA (Joshi et al., 2017), and PopQA (Mallen et al., 2022).Source: §B.1 · p. 15
  • Its multi-hop benchmarks are HotpotQA, 2WikiMultiHopQA, MuSiQque, and Bamboogle.The associated references are Yang et al. (2018), Ho et al. (2020), Trivedi et al. (2023), and Press et al. (2023), respectively.Source: §B.1 · p. 15
  • For search-based QA, agents generate queries during step-by-step reasoning and use the E5 retriever (Wang et al., 2022) to fetch the top 3 passages per query.The passages are retrieved from a knowledge base.Source: §B.1 · p. 15
  • Agents iteratively analyze retrieved evidence and refine queries until they produce a final answer.Source: §B.1 · p. 15

B.2 Details of Training … C Case Study

The appendix specifies training, computing, rendering, and prompting details for both benchmarks, then illustrates how AgentOCR accumulates search results and adapts compression during a successful HotpotQA interaction.

B.2 Details of Training

  • ALFWorld uses 5120-token text prompts versus 2048-token optical prompts, with 512-token responses, up to 50 steps, and a 1e-6 learning rate.Training uses 16 samples per iteration and 8 trajectories per sample, reward 10 for successful actions and 0 otherwise, rollout temperature 1.0, validation temperature 0.4, batch size 256, and no KL-divergence loss.Source: §B.2 · p. 15
  • Search-based QA uses 14000-token text prompts versus 4096-token optical prompts, with 512-token responses, up to 4 steps, and a 1e-6 learning rate.Training uses 128 samples per iteration and 8 trajectories per sample, reward 1 for correct answers and 0 otherwise, rollout temperature 1.0, greedy validation, batch size 256, and no KL-divergence loss.Source: §B.2 · p. 15

B.3 Computing Details

  • Qwen2.5-VL-3B-Instruct experiments ran on 2×H100 GPUs, Qwen2.5-VL-7B-Instruct required 4×H100 GPUs, and training lasted 150 iterations.Source: §B.3 · p. 15

B.4 Optical Rendering Details

  • Optical memory is generated by a deterministic renderer, with typographic settings and semantic color codes intended to aid VLM parsing; rendering hyperparameters are specified for both benchmarks.The hyperparameters are listed in Table 5.Source: §B.4 · p. 15, Table 5 · p. 16

B.5 Prompts

  • Prompts use dynamic semantic slots and control tags for reasoning, actions, search queries, retrieved evidence, answers, and AgentOCR compression ratios.Figures 4 and 6 show text-only templates, while Figures 5 and 7 show optical-history templates.Source: §B.5 · pp. 15–16

C Case Study

  • AgentOCR progressively accumulates search results in optical memory, adjusts compression factors across steps, and reaches the correct HotpotQA answer while using visual history compression.The complete multi-turn trajectory is presented in Figures 8 and 9.Source: C Case Study · p. 16, Figure 8 · p. 19, Figure 9 · p. 20

Related papers

Loading 2601.04786v3…