PaperlayerView exact revision on arXiv

Source-linked AI summary

AWM: Answerable Working Memory for Long-Document VQA Agents

Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, Rui Lu, Yuxiao Dong, Jie Tang, Evgeny Kharlamov

arXiv:2608.25618v1cs.CL

TL;DR

Long-document VQA evaluation can miss whether terminal working memory preserves evidence needed to answer without page context. The paper introduces memory-only answerability and AWM-GRPO, which incorporates that signal into GRPO while preserving final-answer priority. AWM-GRPO improves accuracy over RAG Top-3 on both benchmarks and reduces memory-missing correct cases relative to answer-only GRPO.

  • Problem

    Existing evaluation checks final-answer correctness and evidence-page access, but not whether terminal working memory remains answer-supporting.

  • Method

    AWM-GRPO uses a frozen reader and judge to incorporate memory-only answerability into the GRPO reward alongside final-answer correctness.

  • Results

    AWM-GRPO exceeds RAG Top-3 by 8.1 points on MMLONGBENCH-DOC and 11.9 points on LONGDOCURL, while reducing memory-missing correct rate by 2.7 points over answer-only GRPO.

  • Takeaways & Limitations

    Memory-only answerability exposes whether correct VQA trajectories leave terminal working memory that can support answering independently.

  • Takeaways & Limitations

    The evaluation covers two benchmarks and one model size, and memory-only answerability may depend on the frozen reader; robustness with another reader was not tested.

Abstract

from arXiv · show

Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final-answer correctness and evidence-page access. This creates a memory-quality blind spot: an agent may reach the right page and answer correctly while leaving behind memory too generic or incomplete to support answering once page context is removed. We introduce \emph{memory-only answerability}, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone. Building on this diagnostic, \emph{Answerable Working Memory} (AWM) treats terminal working memory as an answerable evidence artifact, and AWM-GRPO incorporates this signal into the GRPO reward while preserving final-answer priority. Under GRPO, this reward assigns higher advantages to answer-correct trajectories whose terminal working memory remains answerable. On \textsc{MMLongBench-Doc}, even when gold evidence pages are provided, 42.5\% of correct answers still cannot be answered from terminal working memory alone. AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on \textsc{MMLongBench-Doc} and \textsc{LongDocURL} and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.

1 Introduction

Long-document VQA agents retrieve pages, inspect images, maintain working memory, and synthesize answers, but existing evaluation does not test whether terminal memory preserves answer-supporting evidence. The paper introduces memory-only answerability and AWM-GRPO to optimize this memory artifact alongside final-answer correctness.

  • Motivation: Current evaluation checks final-answer correctness and evidence-page access, not whether terminal working memory remains answer-supporting.An agent can answer correctly from page context while leaving memory too generic to support the answer independently.
  • Memory-quality gap: 42.5% of correctly answered MMLONGBENCH-DOC examples fail memory-only answerability even when gold evidence pages are supplied.Memory-only answerability gives a reader only the question and terminal working memory, without page images or trajectory context.
  • Method: AWM-GRPO rewards trajectories using both final-answer correctness and memory answerability, preferring correct answers with answerable terminal working memory.A frozen reader answers from the question and terminal memory alone, and a judge scores both outputs.
  • Results: 8.1 and 11.9 points are AWM-GRPO’s improvements over RAG Top-3 on MMLONGBENCH-DOC and LONGDOCURL, respectively.The controlled training pipeline varies post-training and reward while sharing the policy, retrieval harness, tools, memory schema, prompts, and optimizer.
  • Contributions: The paper identifies a memory-quality gap, proposes AWM-GRPO, and evaluates it against direct-input, RAG, SFT, and answer-only GRPO baselines.The evaluation reports improvements in final-answer accuracy and memory-quality diagnostics.

2 Agent Setup and Memory-Quality Motivation

The paper formalizes an agent-memory interaction in which retrieved page images yield source-linked findings and terminal memory, then diagnoses whether that memory supports answering independently. Controlled comparisons isolate terminal-memory quality from page access and other evaluation components.

  • Agent-memory framework: Long-document VQA instances pair a document, question, and answer while the policy retrieves evidence pages, updates memory, and outputs a final answer with terminal memory.Memory entries are source-linked findings intended to preserve question-relevant evidence.
  • Memory-only answerability: Memory-only answerability gives a frozen reader only the question and terminal memory, then evaluates its answer against the gold answer.The final-answer and memory-only scores partition instances into four outcome cases, including memory-missing correct.
  • Current memory quality: 42.5% of correct EP-given trajectories leave terminal memory unable to support answering independently.The EP-given setting replaces retrieval with gold evidence pages, controlling page access as a confound.
  • Controlled comparison: The controlled comparison holds gold-page access, the answer generator, reader, and judge fixed while substituting empty, original, or improved terminal memories.Improved memory is generated from the same gold pages by a stronger model.
  • Controlled results: 44.4 and 44.8 are the final-answer and memory-only binary rates for GPT-4o-improved memory, versus 36.2 and 30.2 for the original agent memory.With empty memory, both binary rates are near zero; the answer generator, reader, and judge remain fixed.
  • Takeaway: Improving terminal memory alone improves both final-answer and memory-only readouts when the answer generator, reader, and judge are held fixed.This comparison isolates the contribution of terminal-memory content to the two evaluation readouts.

3 Optimizing Answerable Working Memory with GRPO

AWM-GRPO makes terminal-memory answerability visible to GRPO by rewarding trajectories according to both final-answer correctness and memory-only answerability. Its four-cell reward preserves final-answer priority while refining preferences among answer-correct trajectories, and simulations show these distinctions affect normalized advantages as group composition changes.

  • AWM-GRPO overview: AWM-GRPO evaluates each trajectory’s final answer and terminal working memory, then uses both scores to define a GRPO reward.A frozen reader answers from terminal working memory alone, while a scorer evaluates both answers.
  • Desired advantage behavior: AWM enforces memory refinement, final-answer priority, and graded treatment of useful memory preserved after an incorrect answer.Answer-correct, memory-answerable trajectories outrank answer-correct, memory-imperfect ones; answer-correct trajectories still outrank answer-wrong trajectories.
  • Desired advantage behavior: GRPO assigns positive advantage only when a trajectory’s reward exceeds the group mean, so the same outcome can change status with group composition.Answer-correct but memory-imperfect trajectories can be tolerated when most samples fail but become disfavored as memory-supported-correct trajectories become common.
  • AWM reward instantiation: The reward distinguishes four outcome cells defined by final-answer correctness and memory-only answerability.The variables are s_ans and s_mem, with rewards r11, r10, r01, and r00 assigned to their combinations.
  • 3.3 Advantage distributions under synthetic outcome mixtures: Answer-only reward merges answer-correct trajectories regardless of memory quality, whereas AWM assigns them distinct rewards and normalized advantages.The simulation draws groups of size G=8; rows increase p11, the probability of memory-supported correct trajectories.
  • 3.3 Advantage distributions under synthetic outcome mixtures: As p11 increases, the answer-correct, memory-imperfect curve moves below zero while the memory-supported-correct curve remains positive.This is the key simulated comparison between the two answer-correct cells under AWM.
  • Reward scope: AWM-GRPO uses memory-only answerability as a training signal rather than full source-grounding verification and does not score memory style, length, or formatting.The reward requires no gold terminal working memory and does not allow memory answerability to override final-answer correctness.

4 Experiments

The experiments evaluate AWM-GRPO on two long-document VQA benchmarks, comparing controlled Qwen3-VL-4B variants on answer accuracy and terminal-memory quality. AWM-GRPO improves final-answer accuracy over baselines and improves memory-only diagnostics, including when gold evidence pages control retrieval.

  • Experimental setup: AWM-GRPO is evaluated on LONGDOCURL and MMLONGBENCH-DOC using final-answer accuracy, memory-only accuracy, and Pmmc.Both answer types are scored by the official benchmark judge; the controlled agent uses Qwen3-VL-4B and RAG Top-3 retrieval.
  • Experimental setup: The controlled AWM-Agent variants share the model, tools, memory schema, retrieval harness, prompts, and optimizer; only post-training or reward differs.The comparisons include SFT, answer-only GRPO, and AWM-GRPO, alongside direct-input and RAG Top-3 baselines.
  • Main results: 53.9 final-answer accuracy on MMLONGBENCH-DOC places AWM-GRPO 8.1 points above RAG Top-3 and 2.3 points above answer-only GRPO.It also exceeds direct input and SFT by 10.4 and 4.7 points, respectively.
  • Main results: 11.9 points separate AWM-GRPO from RAG Top-3 on LONGDOCURL, with additional gains of 4.4 over SFT and 2.7 over answer-only GRPO.The passage reports these as final-answer accuracy improvements.
  • Memory-quality analysis: AWM-GRPO improves memory-only accuracy from 42.5 to 44.5 and lowers Pmmc from 19.9% to 17.2% versus answer-only GRPO.It also improves final-answer accuracy and keeps Pmmc below SFT’s 17.7%.
  • Memory-quality analysis: With gold evidence pages, AWM-GRPO raises final-answer accuracy from 45.4 to 48.0 and memory-only accuracy from 41.2 to 43.5 versus answer-only GRPO.Pmmc falls from 19.1% to 16.4%, indicating benefit when retrieval is replaced by gold evidence pages.
  • Training dynamics: Memory-only accuracy increases from 38.8 at step 40 to 43.5 at step 280 during AWM-GRPO training on MMLONGBENCH-DOC.The reported training curve is monotonic.
  • Evidence-source analysis: AWM-GRPO lowers Pmmc across several evidence-source categories but has higher Pmmc for table and layout questions.The source analysis covers mixed visual+text, text-multi, figure, chart, and none categories.

5 Related Work

Related work studies long-document VQA agents, intermediate-state supervision, and memory evaluation. AWM focuses on whether an externalized terminal memory remains sufficient for answering after page and trajectory context are removed.

  • Long-document VQA agents: Long-document VQA benchmarks involve lengthy PDFs with textual, tabular, and figure-grounded questions, while agents retrieve pages and record findings in working memory.MMLONGBENCH-DOC and LONGDOCURL average tens to hundreds of pages.
  • Intermediate-state supervision: Prior intermediate-state methods supervise answer-critical tables, answerable captions, question-conditioned visual descriptions, or visual content in reasoning responses.These approaches target structured intermediate representations or visual reasoning states.
  • Memory evaluation: Evidence-citation and related RAG evaluations test whether answers are supported by cited or incomplete retrieved knowledge, rather than whether terminal memory alone suffices.These methods examine answer support or argument structure from external evidence.
  • Memory evaluation: Memory-only answerability evaluates whether a frozen reader can answer from the question and source-linked terminal record without trajectory or page images.Unlike interpretability memory probes, it tests an externalized record rather than internal activations.

6 Conclusion

The conclusion presents memory-only answerability as a diagnostic for correct answers whose terminal memory cannot independently support them, and reports AWM-GRPO gains over answer-only GRPO.

  • 6 Conclusion: Memory-only answerability identifies correct answers whose terminal working memory cannot support the answer independently.The diagnostic isolates a failure not captured by final-answer correctness alone.
  • 6 Conclusion: AWM-GRPO improves accuracy over answer-only GRPO on both benchmarks and lowers Pmmc in both settings.The conclusion summarizes gains in final-answer performance and memory-missing-correct diagnostics.
  • 6 Conclusion: The evidence remains limited to two benchmarks and one model size, leaving grounding unverified.The conclusion explicitly identifies these boundaries.

Limitations

The study’s controlled AWM comparisons use one 4B model on two benchmarks, and several evaluation and grounding limitations remain.

  • Limitations: AWM-GRPO is evaluated only with Qwen3-VL-4B on LONGDOCURL and MMLONGBENCH-DOC, leaving larger models and other document distributions untested.The open settings include scientific papers, financial filings, and slide decks.
  • Limitations: Memory-only answerability may depend on the frozen Qwen3-14B reader, and robustness with a second reader was not tested.The fixed judge’s answer-extraction errors may also affect reported metrics.
  • Limitations: The AWM reward measures answerability rather than full source-grounding verification for individual memory claims.The paper identifies source-conditioned checks as future work.
  • Limitations: Memory-only answerability requires extra reader and extraction passes during offline evaluation, though neither is needed after deployment.A 100-example audit found six disagreements without a consistent error direction; larger audits are needed.

Ethics Statement

The study uses public benchmarks and an open-weight VLM, collects no new data or human-subject data, and does not target deployment.

  • The work uses publicly released benchmarks, a public open-weight Qwen3-VL-4B model, and a fixed local Qwen3-14B model.
  • The study collects no new data, does not work with human subjects, and does not target deployment.

A Dataset Details

Dataset evaluation uses the complete released examples from MMLONGBENCH-DOC and LONGDOCURL, with full-size MMLONGBENCH-DOC page images in the main-results table.

  • Benchmark sources: MMLONGBENCH-DOC uses all 1,082 examples from the original samples.json.
  • Benchmark sources: LONGDOCURL uses all 2,325 examples from the original LongDocURL_public.jsonl.
  • Benchmark sources: The main-results table uses the original full-size MMLONGBENCH-DOC page images.
  • EP-given evaluation: EP-given diagnostics replace retrieval with annotated gold evidence pages on fixed MMLONGBENCH-DOC subsets.No LONGDOCURL EP-given diagnostic is reported in the main experiments.
  • Routing verification: All reported runs pass a check pairing each question with its source document.

B Evaluation Protocol

The evaluation uses official benchmark judges for final-answer and memory-only metrics, supplements them with a manual audit, and defines diagnostics that isolate memory failures among correct answers.

  • Official benchmark scoring: Official scoring extracts concise answers with GPT-4o, then applies benchmark-specific deterministic rule scoring.
  • Type-aware rule scoring: The scorers normalize answers by type, using exact integer matching, tolerance-based floating-point matching, ANLS-based string matching, and benchmark-specific list handling.
  • Final-answer and memory-only metrics: Final-answer and memory-only accuracy use the same official judge, with the latter reading only the question and terminal working memory.The frozen Qwen3-14B reader receives no trajectory, page images, or tool instructions.
  • Manual audit: A manual audit of 100 scored examples found six disagreements, mainly involving aliases, list formatting, and partial-credit edge cases.The disagreements were not concentrated on one side of the comparison.
  • Memory diagnostics: The memory-missing-correct rate Pmmc measures correct final answers whose terminal memory remains unanswerable, avoiding cancellation from answering errors.The signed marginal gap can be near zero even when many correct answers have unanswerable memory.
  • Memory updates: Mandatory free-form memory updates do not guarantee memory-only accuracy because agents can write generic descriptions while retaining needed evidence in page context.Each update is limited to 40 words and requires a source page.

C.3 Reward variants we considered

The appendix compares alternative memory rewards, analyzes AWM reward discrimination, and reports controlled intervention results while documenting training and deployment overheads.

  • Reward variants: Per-finding source-image grounding was rejected because it requires one scoring call per finding and sacrifices schema-agnosticism.The authors identify higher compute and scoring noise as additional costs.
  • Reward variants: Self-consistency over multiple reader rollouts was not evaluated because reducing single-rollout variance would multiply scoring cost by K.
  • Reward variants: A length penalty on terminal memory was rejected because it could suppress legitimate findings containing the answer value.Answer-echo detection remains a separate audit.
  • Reward discrimination: The AWM reward strictly orders memory-supported correct, memory-missing correct, answering error, and unresolved error, unlike answer-only reward.Default values preserve final-answer priority because r10 > r01.
  • Reward discrimination: AWM distinguishes policies with equal final-answer accuracy but different memory answerability, creating different GRPO advantages for memory-supported and memory-missing correct trajectories.
  • Simulation: The Figure 3 simulation samples 15,000 groups of size 8 and computes normalized group-relative advantages under AWM and answer-only rewards.It uses ε = 10^-6 in the normalization.
  • Improved-memory intervention: The controlled intervention uses 500 EP-given MMLONGBENCH-DOC questions, comparing empty, original-agent, and GPT-4o-improved terminal memory with fixed answer generation and judging.Improved memories are not used as training data.
  • Improved-memory intervention: 44.4% final-answer binary correctness with improved memory exceeded 36.2% with original agent memory in the controlled comparison.With empty memory, 14 of 500 final answers received a positive official score; these are not full-benchmark accuracy claims.

D.3 Memory-only results by original outcome

The intervention improves memory-only answerability overall, while audits and controlled analyses show that source-page validity, answer-string copying, and single-seed gains do not establish fully grounded or generalizable memories.

  • 73 more examples become memory-answerable, increasing the total from 151 to 224 out of 500.The replacement improves answerability overall but does not dominate the original memory in every outcome cell.
  • All 500 improved memories cite a page within the corresponding gold evidence_pages set.This verifies eligible source-page selection, not entailment of every statement or completeness of the evidence.
  • 32.2% of the 304 examples remaining after exact answer-string filtering receive a positive memory-only reader score.Because filtering selects a different subset, this rate is not a like-for-like estimate of the full-set effect and cannot rule out paraphrased leakage or establish claim-level grounding.
  • Memory-only answerability and grounding are distinct: answerable memories may contain unsupported guesses, while grounded memories may omit needed facts.The audits check source-page validity and exact answer copying but do not establish claim-level grounding.
  • The intervention is a single-seed upper-bound analysis using one fixed subset, reader, judge, and improved-memory model.GPT-4o writes improved memories from gold evidence pages, and repeated runs plus claim-level audits are needed.
Loading 2608.25618v1…