PaperlayerView exact revision on arXiv

Source-linked AI summary

AGM: Achievement-Grounded Memory for Closed-Loop Agents with Frozen VLA Policies

Hongbo Gao, Zeyu Ni, Xin Wen, Siyu Xu, Ruifeng Li

arXiv:2608.29537v1cs.ROcs.AI

TL;DR

Frozen VLA policies lack persistent progress tracking, and attempt-based memory writes can turn local execution failures into persistent task-state errors. AGM uses verified physical evidence to update a compact subgoal pointer while keeping the policy frozen. It achieves strong results across RoboMME Counting tasks and on a physical robot, with the framework’s verification vocabulary bounded to gripper interactions.

  • Problem

    Frozen VLA policies execute open-loop action chunks without representing task progress, while unverified memory updates can turn failed attempts into persistent state errors.

  • Method

    AGM uses gripper events to trigger verification and frozen foundation models to determine whether physical subgoals were achieved before updating a compact progress pointer.

  • Results

    AGM attains the best RoboMME Counting average success rate, reaches 100.00% on PickXTimes and 84.00% on BinFill, and yields gains on a physical robot.

  • Takeaways & Limitations

    Reliable embodied memory depends on disciplined, achievement-grounded state updates rather than memory capacity alone.

  • Takeaways & Limitations

    AGM’s verification vocabulary remains anchored to gripper interactions, and the physical evaluation should be widened.

Abstract

from arXiv · show

Frozen vision-language-action (VLA) policies offer broad manipulation skills but execute open-loop action chunks without tracking task progress, so the agent cannot reliably decide whether to continue, retry, or terminate. External memory is a natural remedy, yet it can be harmful when attempted actions are treated as completed progress, turning local execution errors into persistent task-state errors. We propose Achievement-Grounded Memory (AGM), a lightweight closed-loop framework for frozen VLA policies that represents a task as a subgoal sequence with a progress pointer and advances this memory only after the current subgoal is verified by physical evidence. Proprioceptive interaction cues decide when to verify, while coherent point tracking and language-conditioned cross-view comparison, sourced from frozen foundation models through a single 2.43M-parameter verification head, decide what was achieved. AGM thereby converts open-loop execution into a closed loop of execution, verification, and progress, keeping the policy frozen without test-time large-model inference. On the RoboMME Counting benchmark, AGM reaches on PickXTimes and on BinFill, surpassing the strongest memory-augmented baseline by points on average, and the framework yields equally decisive gains on a physical robot. Reliable embodied memory thus depends more on disciplined state updates than on memory capacity.

Introduction

Frozen VLA policies lack persistent task state, while unverified memory updates can convert manipulation attempts into lasting errors. AGM closes this loop by verifying physical achievement before updating a compact progress memory, improving benchmark and robot performance.

  • Frozen VLA policies do not represent persistent task progress, leaving agents unable to reliably continue, retry, roll back, or terminate.
  • Attempt-based memory updates can turn transient manipulation failures into persistent task-state errors and perform worse than using no progress memory.
  • AGM uses gripper-load events to trigger verification and frozen foundation models to assess kinematic and semantic achievement.
  • 55.96% average success on RoboMME Counting surpasses the strongest memory-augmented baseline by 7.1 points.
  • 100.00% on PickXTimes and 84.00% on BinFill establish new state-of-the-art results, with validation on a physical robot.
  • AGM grounds a compact progress pointer in observable execution evidence, enabling advancement, retention, and recovery-aware rollback.

Related Work

Prior embodied-agent methods expand stored memories, retrieve experience, or monitor execution, often using finetuned or large models. AGM instead focuses on when memory writes are justified during repetitive manipulation.

  • Prior embodied-agent systems use memory streams, episodic reflections, skill libraries, spatial memory, or retrieved task-relevant keyframes.
  • For repetitive manipulation, AGM argues that a compact subgoal pointer is sufficient and unverified writes are the dominant failure source rather than limited memory capacity.
  • Execution verification has used success detectors, multimodal failure reasoning, runtime monitors, and environment feedback in planning loops.
  • AGM triggers verification at proprioceptive gripper-load transitions rather than querying a large model at fixed intervals or analyzing failures post hoc.

Method

AGM closes the loop around a frozen VLA by maintaining a typed subgoal sequence and updating a progress pointer only after compatible interaction events are visually verified. It combines proprioceptive event localization with distinct grasp and placement evidence, while retaining rollback and retry behavior for different subgoal types.

  • Task representation: AGM expands each repetitive-manipulation instruction into an ordered typed subgoal sequence for the frozen VLA.Type labels determine compatible verification events and recovery behavior; they are not additional policy outputs.
  • Task representation: The compact memory M_t = (G, p_t) stores the fixed subgoal sequence and a single persistent progress pointer updated online.The pointer preserves task-stage information that current observations alone cannot reveal, disambiguating visually similar repetitions without full interaction history.
  • System design: AGM grounds task-state updates in observable outcomes while keeping the VLA, point tracker, and vision-language encoder frozen.Only the lightweight verification head is trained, with 2.43M trainable parameters; bounded timeout and retry rules address unavailable evidence or deadlock.
  • Event-triggered verification: AGM triggers verification only for interaction events compatible with the current subgoal, separating interaction occurrence from achievement.Empty releases and incompatible event–subgoal pairs are ignored because events localize potential transitions but do not establish success.
  • Achievement verification: Grasp verification uses coherent upward motion of tracked object points during lifting rather than commanded gripper motion.A frozen tracker supplies trajectories and visibility; high-response selection suppresses background tracks and median aggregation reduces isolated tracking errors.
  • Achievement verification: Placement verification compares language-conditioned pre- and post-release evidence across front and wrist views to assess the requested object–target relation.The event representation combines cross-view appearance, event-induced change, proprioceptive context, and the normalized achievement query.
  • Memory transitions: A verified achievement advances the pointer, a rejected grasp retains the current state, and a failed recoverable placement rolls back to its associated grasp subgoal.Container placement is non-recoverable after release, so AGM uses a more permissive threshold and bounded rejection handling; recoverable planar placement uses stricter acceptance.

Experiments

Experiments evaluate AGM on RoboMME Counting, controlled ablations, and a physical robot. Verified progress updates improve repetitive-task reliability while preserving the frozen policy on tasks without gripper-anchored subgoals.

  • Experimental design: The evaluation tests task success, progress tracking, verification components, and physical-robot transfer across RoboMME Counting tasks.The suite includes PickXTimes, BinFill, SwingXTimes, and StopCube, with official test protocols and baselines.
  • Main results: 55.96% average success surpasses MemER’s 48.83% by 7.1 points and nearly doubles frozen π0.5’s 28.78%.AGM results are averaged over two independent runs on the 50-episode-per-task test split.
  • Repetition depth: AGM maintains 100.0% on PickXTimes across every instructed count, while BinFill declines to 66.7% at N=5 as later physical repetitions become harder.The frozen base policy collapses to 0.0% on both tasks by N=5, despite possessing the underlying manipulation skills.
  • Error correction: A failed grasp can be rejected while retaining the progress pointer, enabling a successful retry instead of recording false progress.The simulation rollout illustrates how attempt-based memory would finish one cube short.
  • Ablations: Proprioception-only verification reaches 98.0% on PickXTimes and 78.0% on BinFill, while flattening reversibility thresholds lowers BinFill from 84.0% to 74.0%.The coherent-motion test adds residual margin, and strict thresholds falsely reject some irreversible drops.
  • Physical validation: On the physical robot, AGM keeps PickXTimes at 100.0% for every repetition count, while BinFill remains 100.0% through N=2 and reaches 60.0% at N=5.Bare policies fall to zero on both tasks by N=4, even though they execute individual grasps and placements competently.

Conclusion

AGM closes the loop around frozen VLA policies by grounding progress-memory updates in observable execution evidence. It achieves strong benchmark and physical-robot results with a small trainable verification head, while remaining scoped to gripper-centered verification and broader physical evaluation as future work.

  • Conclusion: AGM updates a compact task memory only when gripper events trigger verification and frozen models provide evidence of the achieved physical transition.The verdict can advance, retain, or roll back the progress pointer.
  • Conclusion: AGM trains only a 2.43M-parameter head while achieving the best four-task average and new state-of-the-art results on PickXTimes and BinFill.The framework transfers across backbones without retuning and delivers decisive gains on a physical robot under an unchanged framework.
  • Limitations and next steps: The verification vocabulary remains anchored to gripper interactions, and widening physical evaluation is identified as a next step.These boundaries define the current scope of the framework.

A. Implementation Details

AGM uses a fixed subgoal sequence and progress pointer, with verification triggered by gripper events and evidence gathered from motion tracking, visual comparison, and proprioception. Its shared configuration handles all tasks while adapting thresholds and recovery to placement reversibility.

  • Task representation: The instruction expands into T = 2N+1 alternating grasp and placement subgoals, followed by a terminal action, with repetition indexed on grasp subgoals.
  • Task representation: Placement verification normalizes language queries across repetitions, so visual evidence checks the object–target relation while the pointer tracks sequence position.
  • Event triggering: The hysteretic gripper state machine emits only confirmed transitions, suppressing chatter through separated thresholds and five-frame persistence.
  • Grasp verification: Grasp clips end after a 0.03 m lift, and coherent upward motion is aggregated from surviving tracks to determine grasp achievement.
  • Verification design: The verification head combines frozen visual and language features with proprioception, while one shared hyperparameter configuration serves all four tasks and episodes.
  • Recovery and termination: AGM uses stricter recoverable-placement acceptance, permissive irreversible-placement acceptance, rollback where possible, bounded retries, and a pre-termination occupancy recheck.

B.1 Event Dataset

The verification dataset contains 788 execution events from benchmark training episodes, combining verified and attempt-based controllers to provide both positive and negative outcomes. Labels are derived from physical achievement predicates, with disjoint test seeds reserved for evaluation.

  • Dataset construction: The verification head is trained on 788 execution events collected from the official RoboMME training split.
  • Dataset construction: Failures constitute 25.1% of the event corpus, providing negative examples for achievement verification.
  • Collection controllers: Attempt-based collection contributes 72 failed grasps and 57 failed placements, complementing the nearly all-positive traces from verified collection.
  • Train–test separation: Training uses 100 episodes each from PickXTimes and BinFill, while evaluation uses disjoint test-split seeds with 50 episodes per task.
  • Labeling: Offline labels mark grasps, recoverable placements, and irreversible placements using object motion, target proximity, release, and occupancy predicates.

B.2 Training Configuration

AGM trains one small shared verification head while keeping the tracker and foundation encoder frozen. Deployment refits the head on all training events after a separate held-out measurement split evaluates it.

  • Optimization: The verification head uses a Linear(4626, 512)–ReLU–Dropout(0.3)–Linear(512, 128)–ReLU–Linear(128, 2) architecture.
  • Optimization: Training uses AdamW with learning rate 10^-3, weight decay 10^-4, 300 full-batch steps, and inverse-frequency class weighting.
  • Shared supervision: Grasp and placement events train jointly through one feature map, although deployment decides grasps by point tracking rather than this head.
  • Measurement and deployment: The measurement model trains on 650 events and is evaluated on 138 held-out events, whereas deployment refits the head on all 788 events.

B.3 Held-Out Verification Performance

Held-out evaluation shows exact recoverable-placement verification but weak discrimination for irreversible container placement, motivating asymmetric thresholds and recovery behavior. Matched event timing and verified memory writing are important for performance, while instruction-only plans remain competitive but slightly lower.

  • Held-out verification: All 41 held-out recoverable planar-placement events are classified correctly at the 0.5 deployment threshold.
  • Held-out verification: Among 28 held-out irreversible container-placement events, none of four failures is caught at argmax, so this branch is not a reliable discriminator.
  • Threshold asymmetry: Strictly judging container placement lowers BinFill from 84.0% to 74.0%, because correct drops can be rejected and the pointer stalled.
  • Event timing: An earlier mismatched post-release offset reached 66.0% on BinFill versus 86.0% and 82.0% with matched offsets, although the comparison also changed the training corpus.
  • Evaluation protocol: Evaluation uses 50 test episodes per task, exact-count termination, a 1,300-step budget, and two-run averages for AGM entries.
  • Plan specification: Instruction-only parsing achieves 98.0% on PickXTimes and 82.0% on BinFill, compared with 100.0% and 86.0% using benchmark decomposition.

C.2 What Verified Writing Contributes

The comparison isolates verified writing as the source of AGM’s benefit: identical subgoal decomposition reaches 100.0% on PickXTimes with evidence-based updates but only 32.0% with attempt-based updates. The task-specific gap is wide when failed actions remain recoverable and narrow when releases are nearly self-evidencing.

  • Verified writing versus attempt-based writing: 100.0% versus 32.0% on PickXTimes separates verified pointer updates from attempt-based updates under the same policy, subgoal sequence, and pointer.Unverified writing converts transient grasp or placement failures into permanent counting errors.
  • Task-dependent mechanism: On BinFill, the attempt-based controller nearly matches AGM because released cubes leave the scene, making attempts and achievements nearly coincide.Verification has less to separate when container placement is structurally self-evidencing.
  • Task-dependent mechanism: Verified writing contributes most when failed actions leave the world recoverable, whereas irreversible self-evidencing actions narrow the verification gap.The two tasks locate the mechanism rather than merely confirming an aggregate performance difference.
  • Rollout interpretation: At N=5, simulation rollouts expose the pointer transitions, active subgoals, and state vectors that govern complete episodes.PickXTimes scenes can remain visually identical across repetitions, so the pointer-supplied subgoal distinguishes otherwise aliased states.
  • Physical transfer: The framework was also transferred to a physical platform, where the runtime mechanism remained unchanged while hardware interfaces and calibration were adapted.The robot deployment used a 6-DoF arm, a parallel gripper, dual cameras, and distributed inference services.

D.2 Task Suite

The task suite evaluates long-horizon counting through BinFill and PickXTimes, each expanded into an ordered sequence of pick, place, and termination subgoals. Evaluation spans repetition counts through N=5, including BinFill at a count absent from its demonstrations.

  • Task definitions: BinFill requires placing N instructed-color cubes into a bin before pressing the stop button.The physical setup uses colored cubes, a bin, and a terminal button action.
  • Task definitions: PickXTimes requires repeatedly picking and placing the same instructed-color cube N times before pressing the stop button.The benchmark returns the cube to the workspace between repetitions, preserving the counting challenge.
  • Task structure: Each task expands into G = [pick, place]×N + [button], yielding T = 2N+1 subgoals and up to eleven subgoals at N=5.The subgoal wording follows the benchmark templates for train–deployment consistency.
  • Repetition depth: Evaluation covers N = 1,...,5 for both tasks, while demonstrations cover N ≤4 for BinFill and N ≤5 for PickXTimes.Thus BinFill at N=5 is outside its demonstration counts, although its generated subgoal strings remain in vocabulary.
  • Data and annotation: The corpus contains 194 episodes and 237,517 frames, with automatic segmentation based on gripper signals and measured-width confirmation rather than manual or vision-language-model annotation.An independent audit correctly labeled all 194 episodes, with pick boundaries within 4 frames for PickXTimes and 7 frames for BinFill.

D.4 Policy Training and Language Conditioning

The training and deployment interface conditions frozen policies on the full task plus the active subgoal, while AGM’s physical verification components are calibrated around gripper events and placement outcomes. The resulting interface supports repetition beyond demonstrations and transfers to real-robot rollouts.

  • Policy conditioning: The AGM-compatible checkpoints train with the full task instruction and current subgoal together, while merged checkpoints receive only the bare task instruction.This yields four policy configurations across π0 and π0.5.
  • Policy conditioning: Deployment rebuilds the conditioning string from the same template used for annotation and checks that every emitted string appears in the training vocabulary.The parity check is intended to prevent silent conditioning drift.
  • Hardware adaptation: AGM retains the state-machine structure on hardware, recalibrating thresholds and interface quantities rather than changing the core mechanism.The physical transfer includes recalibration for the gripper, camera geometry, and service interface.
  • Verification interface: A pick advances only after verification, while a release cancels a pending pick, preventing an empty closure from advancing the pointer.Grasp verification combines a proprioceptive lift-and-load cue with coherent point tracking; placement uses learned visual evidence and task-specific gates.
  • Repetition depth: AGM reaches 60.0% on undemonstrated BinFill N=5 versus 70.0% at demonstrated N=4, while π0.5+AGM reaches 100.0% across in-distribution PickXTimes counts.The new repetition depth is externalized into the pointer, whereas PickXTimes N=5 is already represented in demonstrations.
  • Physical evaluation: All three reported N=5 physical episodes succeed using the AGM configuration, including BinFill with color-selective counting and PickXTimes with repeated visually aliased scenes.The physical rollouts preserve the framework configuration while adapting the hardware interface.

E. Extended Limitations

AGM’s scope is bounded by its event vocabulary, linear pointer memory, rejection safeguards, verifier quality, and narrow physical evaluation. These constraints define where its guarantees and transfer claims apply.

  • Event coverage: AGM directly covers prehensile subgoals but does not provide compatible achievement events for pushing, sliding, articulated-object manipulation, or tool use under a persistent grasp.Broader coverage requires additional interaction detectors rather than a different pointer memory.
  • Memory structure: The single-pointer sequence cannot express branching, partially ordered, or out-of-order-satisfiable subgoals.Rollback reaches only the associated grasp of the current repetition.
  • Recoverability: Unexpected object unreachability may trigger retries until the rejection bound forces an advance, because recoverability is fixed when the instruction is expanded.The resulting behavior depends on the predefined subgoal type and rejection policy.
  • Deadlock handling: Bounded rejection handling can advance on an unverified subgoal, trading a potential progress error for avoiding a stalled episode.AGM guarantees that unverified writes are rare and bounded, not impossible.
  • Verification quality: Verifier errors remain performance limits: false acceptance records nonexistent progress, while false rejection repeats a completed subgoal.Threshold asymmetry manages these costs but does not eliminate the trade-off.
  • Physical scope: The physical evaluation covers two task families on one 6-DoF platform, one gripper geometry, and one object class, without claiming threshold transfer across platforms.Cross-platform use requires the documented recalibration procedure.
Loading 2608.29537v1…