PaperlayerView exact revision on arXiv

Source-linked AI summary

PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents

Kun Li, Zexuan Qiu, Tianhua Zhang, Irwin King, Helen Meng

arXiv:2609.06702v1cs.CL

TL;DR

Long-document reasoning is difficult when sequential memory makes accuracy depend on evidence placement and latency depend on document length. PARSER decouples reading from reasoning with parallel frozen chunk readers and an iteratively querying lead agent trained with reinforcement learning. Across 7K–896K-token multi-hop QA contexts, it improves over sequential memory and remains robust to evidence perturbations while reducing latency.

  • Problem

    Sequential memory couples document traversal to reasoning depth, making long-document QA sensitive to evidence placement and linearly increasing inference latency with document length.

  • Method

    PARSER assigns one frozen subagent to each chunk for parallel reading while a lead agent iteratively scatters queries, gathers evidence, and reasons across rounds.

  • Results

    Across 7K–896K-token multi-hop QA contexts, PARSER with 4B and 9B backbones outperforms the strongest sequential memory baselines by 5.7 and 6.7 points on average.

  • Takeaways & Limitations

    PARSER remains robust to evidence position, order, and distance perturbations while reducing inference latency by up to 11×.

  • Takeaways & Limitations

    The evaluation excludes ReMemR1 from the latency analysis because its extra retrieval module theoretically introduces higher inference latency than MemAgent.

Abstract

from arXiv · show

Sequential memory agents process long documents by reading chunks one after another while maintaining a compact memory state, coupling document traversal to reasoning depth. This coupling introduces sensitivity to evidence placement and ties inference latency linearly to document length. We introduce PARSER, which decouples reading from reasoning. A bank of lightweight subagents each bound to a single chunk read the entire document in parallel, while a lead agent reasons in depth through iterative scatter--gather rounds: at each round it broadcasts a query to all subagents, aggregates the returned evidence, and formulates a deeper follow-up query conditioned on what has been found so far. This decoupled design concentrates all learnable behavior in the lead agent, which is optimized with reinforcement learning, while the subagents remain frozen off-the-shelf models. On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens. Scaling to a 9B backbone, PARSER surpasses DeepSeek-V4-Pro by 6.3 points. Controlled experiments confirm that PARSER is robust to perturbations in evidence position, order, and distance, conditions that cause large accuracy swings in sequential methods, while reducing inference latency by up to 11x.

1 Introduction

Long-document QA struggles when evidence is scattered because sequential memory couples document traversal to reasoning depth, making accuracy position-sensitive and latency grow with document length. ParSer decouples reading from reasoning through parallel chunk readers and iterative query-driven aggregation.

  • Sequential memory is sensitive to evidence position, order, and distance because each chunk is compressed before later evidence is available.This information deficit is especially damaging for multi-hop reasoning over scattered evidence.
  • ParSer separates document reading from question reasoning by querying all chunk-level subagents in parallel and refining queries across rounds.The lead agent gathers evidence and formulates follow-up queries, while subagents remain dedicated to individual chunks.
  • ParSer evaluates long-context QA from 7K to 896K tokens and reduces inference latency by up to 11× while improving accuracy over sequential memory.At 896K tokens, the reported latency is 876s versus 78s per sample relative to MemAgent.
  • With a 4B backbone, ParSer outperforms the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens.The 9B version surpasses DeepSeek-V4-Pro by 6.3 points on average.

2 Related Work

Prior long-context approaches extend windows, reduce attention cost, or compress documents, but parallel chunk aggregation is often single-shot or coordinated by fixed protocols. ParSer belongs to a learned, multi-round orchestrator–worker alternative for adaptive multi-hop reading.

  • Long-context research combines positional extrapolation, sparse attention, and memory compression, yet longer windows alone do not ensure effective evidence use.Accuracy can degrade with context length even when the nominal context window is far from full.
  • Single-shot map-reduce methods send fixed queries to chunks, limiting their ability to discover later-hop evidence in multi-hop questions.Other multi-round systems use hand-specified coordination, while attention-level methods are query-agnostic and require architectural changes.
  • ParSer extends orchestrator–worker parallelism with question-conditioned, learned coordination across repeated reading rounds.This contrasts with prior systems that use fixed protocols or single-round aggregation.

3 Method

ParSer assigns one subagent to each document chunk and uses a lead agent to conduct iterative scatter–gather reasoning. Parallel readers provide local evidence, while the lead composes cross-chunk dependencies and is the only component trained with reinforcement learning.

  • Workflow: Parallel Reading, Sequential Reasoning: Each subagent reads only its assigned chunk and can abstain, while the lead agent receives gathered findings without directly seeing document tokens.All chunks are queried symmetrically and can be revisited under newly formulated questions.
  • Workflow: Parallel Reading, Sequential Reasoning: Cross-chunk dependencies are resolved across reasoning rounds as findings from one round condition the lead agent’s next queries.The lead decomposes multi-hop questions into chunk-local queries and composes their results through successive rounds.
  • Efficiency through sparsity: Sparse communication improves efficiency because most subagents abstain and their responses are dropped, unlike sequential methods that update memory after every chunk.The resulting critical path scales with reasoning complexity rather than document length.
  • Workflow: Parallel Reading, Sequential Reasoning: ParSer converts document length from sequential dependency depth into parallel width by running one reader per chunk in repeated scatter–gather rounds.Each round queries all chunks concurrently, so increasing the number of chunks adds parallel readers rather than dependent steps.
  • Optimization: Agentic Reinforcement Learning: Only the lead agent is trained with reinforcement learning, while frozen subagents perform query-conditioned evidence location in short chunks.The reward is a binary exact-match score, and policy optimization uses GRPO.

4 Experiments

Experiments evaluate ParSer on multi-hop QA across in- and out-of-distribution benchmarks and contexts from 7K to 896K tokens. ParSer achieves strong accuracy consistency across lengths and retains favorable out-of-distribution performance, with training and inference configured around lightweight frozen readers and a trained lead agent.

  • Implementation: Training uses Qwen3.5-4B and Qwen3.5-9B backbones with capped reasoning turns, chunk sizes, and asynchronous reinforcement-learning infrastructure.Inference increases the reasoning-turn cap and chunk size relative to training.
  • Evaluation Setup: Evaluation uses HotpotQA and 2WikiMultiHopQA with documents ranging from 7K to 896K tokens.The training data comprise 32,768 synthetic HotpotQA samples with approximately 28K-token contexts.
  • 4.3 Main Results: ParSer consistently outperforms baselines across HotpotQA subsets and long-document 2WikiMultiHopQA subsets while maintaining nearly zero performance variation across document lengths.Full-context baselines degrade rapidly as length increases, whereas ParSer achieves the best reported performance consistency.
  • 4.3 Main Results: ParSer maintains favorable out-of-distribution performance, unlike MemAgent and ReMemR1, which perform far worse out of distribution than in distribution.The authors attribute this robustness to training only the lead reasoning policy while keeping chunk readers frozen.

5 Analysis

The analysis examines why ParSer is robust and efficient, testing evidence-distribution sensitivity, inference latency, subagent size, chunking, and alternative subagent implementations.

  • 5.1 Why PARSER Outperforms Sequential Memory Agents: ParSer remains robust when evidence position, logical order, and distance are perturbed, unlike sequential memory agents whose document-ordered compression is sensitive to these factors.The controls vary absolute position, dependency order, and intervening distractors; the authors attribute ParSer’s robustness to concurrent, query-conditioned rereading across rounds.
  • 5.2 Inference Latency: Full-context methods are fastest on short documents, but their latency and memory demands worsen with length, while chunked MemAgent and ParSer remain usable on longer inputs.At high concurrency, full-context methods may fail to finish 896K-token inputs because of GPU memory limits.
  • 5.2 Inference Latency: ParSer reduces sequential inference steps because chunks are processed in parallel, whereas MemAgent’s step count grows with the number of document chunks.Figure 3 compares step counts across document lengths and links ParSer’s steps to the reasoning hops required by each question.
  • 5.3 Effect of Subagent: A 4B subagent improves over a 2B subagent, while a 9B subagent provides no further improvement, indicating that focused chunk queries can be handled by a lightweight model.The lead agent remains fixed in the subagent-size comparison.
  • 5.3 Effect of Subagent: Removing chunking causes a notable performance drop, supporting the contribution of chunk-level decomposition alongside lead-agent guidance.The ablation uses a single subagent receiving the full document rather than separate chunk-level inputs.
  • 5.4 Compatibility with Alternative Subagent Implementations: ParSer also improves over corresponding standalone baselines when using thinking or DCI subagents, suggesting compatibility with alternative subagent implementations.DCI denotes Direct Corpus Interaction, an agentic search paradigm using composable shell tools for corpus querying and evidence collection.

6 Conclusion

ParSer decouples long-document reading from reasoning by combining parallel chunk readers with an iterative lead-agent scatter–gather process. Frozen subagents locate local evidence while reinforcement learning optimizes the lead agent’s reasoning policy.

  • ParSer distributes chunks to lightweight subagents for parallel reading while a lead agent iteratively gathers evidence and refines queries.Only the lead agent’s reasoning policy is optimized with reinforcement learning; subagents remain frozen.
  • Across 7K–896K-token contexts, ParSer with 4B and 9B backbones beats sequential memory baselines by 5.7 and 6.7 points on average.At 896K tokens, the gaps widen to 12.0 and 9.9 points, while inference latency falls by up to 11×.
  • ParSer’s stable accuracy and robustness to evidence position, order, and distance support dynamically refining queries across the full text.The conclusion contrasts this approach with capacity-limited sequential memory processing.

A Time Complexity Analysis

The complexity analysis contrasts full-context, sequential-memory, and ParSer decoding, showing how chunk parallelism reduces ParSer’s wall-clock contribution while retaining sequential lead-agent work. Its advantage depends on retaining chunk KV caches.

  • Full-context prefill scales as O(n^2), while chunk-level encoding lets MemAgent and ParSer process c independent chunks of n/c tokens.The analysis attributes the full-context cost to Transformer attention and the chunked alternatives to independent chunk encoding.
  • ParSer generates subagent outputs over K rounds, with total subagent tokens r_S = Kcr′_S, while the lead agent generates r_L tokens.Each round independently decodes over the c chunks before the lead agent processes the gathered observation.
  • Parallel execution reduces subagent decoding’s wall-clock contribution by a factor of c, leaving the lead-agent terms sequential.The analysis states the corresponding latency reduction while retaining sequential lead-agent computation.
  • ParSer’s lower decoding computation mainly reflects shorter sparse subagent responses, whereas MemAgent produces roughly 1K memory tokens for every chunk.Most ParSer subagents emit only a short finding or “Unknown” response.
  • If chunk KV caches are evicted, ParSer’s prefill complexity rises from O(n^2/c) to O(Kn^2/c), weakening its latency advantage at high concurrency.With K = 4, this becomes four times MemAgent’s prefill computation, while MemAgent does not revisit processed chunks.

B Additional Experiments

Additional experiments show that reinforcement learning substantially improves ParSer accuracy, while the underlying parallel workflow already outperforms sequential memory agents and remains stable as documents lengthen.

  • RL raises HotpotQA accuracy from 74.42 to 84.57 for Qwen3.5-4B and from 76.24 to 86.79 for Qwen3.5-9B.On 2WikiMultiHopQA, the corresponding increases are 82.85→87.04 and 84.86→88.48.
  • Without RL, ParSer already beats MemAgent and ReMemR1 by a wide margin and stays essentially flat as document length grows.The sequential memory agents degrade sharply over the same length increase.
  • The authors attribute this baseline advantage to ParSer matching the pretrained model’s native multi-turn tool-calling format rather than imposing an unfamiliar recurrent memory-update interface.

C.1 Training Data Construction

The training set follows MemAgent’s Stage I recipe, using HotpotQA questions with gold supporting articles and distractor-packed contexts. MemAgent’s Stage II data is not used.

  • The training set follows Stage I of MemAgent and does not use MemAgent’s Stage II data.ReMemR1 adopts the same Stage I recipe.
  • Each sample retains gold supporting Wikipedia articles, adds distractors until reaching 200 paragraphs or approximately 28K tokens, and shuffles paragraph order with a fixed seed.

C.2 Evaluation Data Construction

The evaluation uses in-distribution HotpotQA and out-of-distribution 2WikiMultiHopQA across eight context lengths, with shared questions and padded or packed distractor contexts.

  • The in-distribution HotpotQA evaluation reuses 128 validation questions across all eight paragraph-count settings.The settings range from 50 to 6400 paragraphs.
  • The out-of-distribution 2WikiMultiHopQA evaluation retains supporting facts, adds random distractor paragraphs, and shuffles contexts at each length setting.The construction uses 128 samples per length setting.
  • Evaluation contexts span 50–6400 paragraphs, approximately 7K–896K tokens, using the same 128 questions across eight length settings.The training contexts contain 200 paragraphs, approximately 28K tokens.
  • The notation 128 × 8 denotes 128 shared questions evaluated under eight paragraph-count settings.

D Baseline Implementation Details

The baselines evaluate full-context models, tool-using single agents, and sequential-memory systems under specified multi-turn protocols for long-document QA.

  • Full-context baselines provide the entire document and question in one call, evaluating Qwen3.5 across its supported window and DeepSeek-V4-Pro at 1M-token context.Longer Qwen3.5 inputs use YaRN, while DeepSeek-V4-Pro uses maximum reasoning effort.
  • The full-context protocol separates reasoning from answer presentation with a second turn requesting only a concise final answer.DeepSeek-V4-Pro violated the required answer tags in 9.47% of preliminary in-distribution cases under think-max.
  • The DCI baseline keeps the corpus outside the conversation and lets a single agent inspect it through local read and bash tools.Parallel tool calls are allowed within one model turn, with sandboxed read-only shell access and truncated observations.
  • DCI inference uses a multi-turn ReAct loop with thinking enabled, temperature 0, a 48-turn limit, and continuation handling for truncated generations.A tool-free finalize prompt is appended if the turn budget ends before a usable answer appears.
  • ParSer’s baseline prompt assigns each subagent one document chunk and requires the lead agent to broadcast, revise, and repeat queries before producing an answer.Subagents return evidence or Unknown, while the lead agent synthesizes findings through repeated query_agents calls.

E.4 Training Setup

ParSer training uses asynchronous reinforcement learning for a lead agent that iteratively issues scatter–gather queries to fixed-chunk subagents.

  • ParSer uses fully asynchronous reinforcement-learning training, allowing rollout and policy-update stages to run continuously on separate GPUs.Subagents continue serving rollout requests while the lead policy is updated.
  • Each lead-agent turn can issue queries whose subagent calls run in parallel across all document chunks, after which findings are gathered into the reasoning history.The workflow repeats until an answer is extracted or the maximum number of steps is reached.
  • Subagents remain persistently assigned to fixed chunks, so later queries reuse an identical instruction-and-chunk prefix and enable KV-cache reuse.Cache-aware routing preserves locality across lead-agent turns.
  • The lead agent is trained with GRPO using five rollouts per prompt and at most nine turns, while subagents read 512-token chunks and generate up to 512 tokens per query.
  • The training configuration includes a 4B and 9B setup summarized in Table 9, with offloading enabled for parameters, gradients, and the optimizer.Training uses asynchronous rollout and policy-update GPU groups.

E.5 Training Dynamics

Training improves lead-agent reward and interaction behavior while maintaining reliable formatting, but a failure case shows that ambiguous local evidence can still mislead the lead agent.

  • Reward rises from 44.9% to 83.2% for 4B and from 49.8% to 83.3% for 9B across the first and last ten training steps.The 4B model increases its mean turns, whereas 9B decreases them while both produce longer responses.
  • The lead agent learns to place independent queries in the same turn for parallel execution, rather than executing them sequentially across turns.For 9B, subagent queries per turn increase from 1.06 to 1.62.
  • Tool-call format errors are 0.22% for 4B and 0.66% for 9B over the first ten steps, then approach zero without a format reward.The rollout–actor log-perplexity difference remains at most 1.41 × 10^-3.
  • Evaluation increases chunk size from 512 to 4,096 tokens and raises the lead-agent turn budget from 9 to 12 compared with training.The prefill complexity O(n^2/c) decreases as the number of chunks grows, motivating finer training chunking.
  • A misleading local finding can cause the lead agent to answer incorrectly when ambiguous identities are not cross-validated against contradictory evidence.The failure arises because subagents lack the reasoning trajectory and the lead agent cannot directly verify their source references.

F.3 Comparison against MemAgent on a Reverse-evidence Case

The reverse-evidence case exposes how document order can cause MemAgent to discard information needed for a multi-hop comparison. ParSer instead issues parallel, question-conditioned queries across the document and remains insensitive to evidence order.

  • The evaluation follows the bidirectional Sub_EM definition implemented by MemAgent and ReMemR1’s open-source code.
  • MemAgent fails the comparison because it drops directors’ death dates before encountering the later film–director mappings.The final memory retains the director names but not their death dates, preventing completion of the comparison.
  • ParSer remains insensitive to evidence order by issuing multi-round queries that execute in parallel over the full document.This directly addresses the reverse logical ordering of death dates and film–director mappings.
  • ParSer-4B solves the reverse-evidence example by broadcasting parallel director queries, then parallel death-date queries, and selecting the earlier death.
Loading 2609.06702v1…