PaperlayerView exact revision on arXiv

Source-linked AI summary

ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context

Jianshu Zhang, Keliang Wu, Chengxuan Qian, Xiyuan Yang, Ce Zhang, Ariel Tian, Anbang Liu, Haoran Lu, Han Liu

arXiv:2609.36684v1cs.CL

TL;DR

Progress estimation remains underexplored when a task’s progress depends on history rather than the current frame alone. The paper builds a benchmark, diagnoses PRMs with and without the needed context, and introduces ProgressCompass to supply context to a frozen PRM; the same PRM’s error falls by 63% and rank agreement rises by 76%.

  • Problem

    Existing progress-estimation benchmarks mostly cover short tasks whose progress is visible in the current observation, leaving context-dependent progress estimation underexplored.

    Source: §1 · p. 2

  • Method

    The paper builds ContextProgress-Bench, diagnoses five PRMs with and without the right context, and proposes ProgressCompass, which uses general-purpose VLMs to supply context to a frozen PRM.

    Source: §1 · p. 2

  • Results

    63% lower progress error and 76% higher rank agreement: ProgressCompass improves the same frozen PRM, while supplying the right context cuts each of five PRMs’ error by 77% to 82%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results indicate that embodied PRMs can estimate progress more accurately when given the right context, rather than being incapable of progress estimation.

    Source: §1 · p. 2, §7 · p. 9

Abstract

from arXiv · show

Embodied agents now take on ever longer tasks. For long tasks, knowing only whether a task finally succeeds or fails says little; the steps along the way matter. Progress Reward Models (PRMs) score how far a task has come at every step, and serve as dense rewards, verifiers and monitors. Yet in long tasks the current frame alone often cannot tell how far the task has come, because progress depends on what happened before. We call this problem context-dependent progress estimation. Existing benchmarks on progress estimation mostly focus on short tasks whose progress can be read from the current observation, and whether PRMs can estimate progress when context is needed remains underexplored. We therefore build ContextProgress-Bench, with 24 manipulation tasks for 120 episodes. The benchmark covers three settings: (i) State Recall, where information needed for progress appeared earlier but is not in the current frame; (ii) Sequence Tracking, where steps follow a fixed order, so progress requires knowing which steps are done and which comes next; and (iii) Recurrence Disambiguation, where look-alike frames sit at very different progress. We then run a paired diagnosis: each PRM keeps the same input format in both runs, and in one run its instruction integrates the right context. Even PRMs that read the entire history get lost in estimating progress, yet with the right context the same five models cut their progress error by 77-82%. Embodied PRMs are thus not incapable of progress estimation, but lost without the right context. We therefore propose ProgressCompass, an autonomous agentic loop that reorients an existing PRM and uses current general-purpose VLMs to supply the context the PRM needs. Wrapped in the loop, the same frozen PRM cuts its progress error by 63% and raises its rank agreement by 76%. With such a compass, PRMs estimate progress far better on longer, more complex tasks.

1 INTRODUCTION

The paper frames context-dependent progress estimation as a gap in current PRMs and benchmarks, then diagnoses how supplying the right context changes model performance. It introduces ProgressCompass to provide that context to a frozen PRM.

1 INTRODUCTION

  • Current-frame progress estimates fail when history is needed to recall a hidden state, track ordered steps, or distinguish visually similar moments.The paper calls this problem context-dependent progress estimation.Source: §1 · p. 2
  • ContextProgress-Bench addresses an underexplored setting with 24 hand-selected tasks, 120 episodes, and 552 annotated subtask intervals that require context.Each interval is annotated with the context it requires.Source: §1 · p. 2
  • 77% to 82% lower MAE with the right context: all five PRMs improve, while their error without it is 15.6 to 31.0 on a 0 to 100 scale.The gain holds on 93.3% of paired intervals; even models reading or retrieving the full history get lost without the right context.Source: §1 · p. 2
  • ProgressCompass uses general-purpose VLMs to supply a frozen PRM with context through an autonomous agentic loop.Its Orienter identifies the current step, the PRM scores it, the Verifier checks completion, and a text-only Navigator runs the loop.Source: §1 · p. 2
  • 63% lower progress error and 76% higher rank agreement: ProgressCompass improves the same frozen PRM, closing 78% of the gap to ground-truth context.The loop remains robust when executions stop early, include extra steps, or follow an unrelated task.Source: §1 · p. 2

2 CONTEXT-DEPENDENT PROGRESS ESTIMATION

Context-dependent progress estimation requires interpreting history in light of the task instruction and current observation, not merely providing the raw history. The section defines three forms of missing context and describes the benchmark built to study them.

2.1 TASK FORMULATION

  • Progress is context-dependent when the current observation alone does not determine it; the history must be interpreted for the instruction and current moment.The context mapping uses the instruction to identify what matters and the observation to identify what is already visible; the resulting gap sets an error floor for estimators using only the instruction and observation.Source: §2.1 · p. 3

2.2 THREE FORMS OF CONTEXT DEPENDENCE

  • The three forms are State Recall of hidden earlier facts, Sequence Tracking of the active position in an ordered plan, and Recurrence Disambiguation of look-alike repeated events.For recurrence, context includes the number of completed occurrences and whether the current occurrence is underway.Source: §2.2 · p. 4

2.3 CONTROLLED BENCHMARK CONSTRUCTION

  • ContextProgress-Bench selects context-dependent robot-manipulation tasks from RMBench, RoboDojo, and LIBERO-Mem, with hand-annotated subtask intervals and required context.It contains 24 tasks with five randomly sampled episodes each, for 120 videos and 71,708 execution frames.Source: §2.3 · p. 4

3 WHERE PROGRESS REWARD MODELS GET LOST

The paired evaluation finds that all five PRMs estimate progress poorly without context, even when they read episode history; supplying the active subtask context improves error and ordering across models. Their failures take three forms: state amnesia, phase drift, and occurrence confusion.

3.1 DO PROGRESS REWARD MODELS LOSE THE TASK WITHOUT CONTEXT?

  • Table 1 compares no-context, oracle-context, and self-chained-context estimates, with paired interval MAEs plotted against the diagonal.Dots below the diagonal indicate intervals where context helped.Source: Table 1 · p. 5, §3.1 · p. 5

3 WHERE PROGRESS REWARD MODELS GET LOST

  • The paired runs use five PRMs and differ only in context: the no-context instruction is the episode instruction, while the other is the active subtask instruction.Context is supplied by hand from annotations, and subtask boundaries are set in advance.Source: §3 · p. 5

3.1 DO PROGRESS REWARD MODELS LOSE THE TASK WITHOUT CONTEXT?

  • Supplying context turns around progress error and ordering for all five PRMs; nearly every paired interval improves, rather than gains coming only from easy episodes.The comparison pairs runs with the same input format, differing only in whether the instruction integrates the annotated context.Source: §3.1 · p. 5, §3 · p. 5, Table 1 · p. 5

3.2 HOW THE THREE FORMS FAIL

  • Without context, PRMs make three distinct errors: forgetting hidden state changes, drifting across ordered phases, and assigning repeated actions nearly identical progress.Examples are returning red blocks, inserting tubes in order, and pressing the same button eight times.Source: Figure 3 · p. 5, §3.2 · p. 6

4 PR O G R E S SCO M P A S S: REORIENTING PROGRESS REWARD MODELS

ProgressCompass keeps a PRM frozen and uses VLMs to interpret history, track task position, and verify step completion in a repeated loop. Its error advantage over current-frame estimators is conditional on the combined position and local errors being sufficiently small.

4 PR O G R E S SCO M P A S S: REORIENTING PROGRESS REWARD MODELS

  • A VLM Orienter interprets history for the instruction and current frame, while the PRM scores progress once the active step is known.The PRM remains frozen; the VLM supplies context and task-step understanding rather than a calibrated progress estimate.Source: §4 · p. 6
  • ProgressCompass loops through orienting, estimating, verifying, and orienting again: the Orienter supplies context and an expected transition, the frozen PRM proposes completion, and the Verifier checks it.The Navigator coordinates the components and carries the plan and confirmed outcomes between steps.Source: §4 · p. 6, Figure 4 · p. 6
  • The Verifier checks a stated expected transition, limiting premature step advances to its false-accept rate; missed completions delay position instead of corrupting later steps.The Navigator records an outcome only after acceptance.Source: §4 · pp. 6–7
  • ProgressCompass is more accurate than current-frame estimators only when (ηPC + λ)/K < δS + δQ + δR, a condition rather than a guarantee.The context floor is removed when context is supplied, while position and local errors remain in the bound.Source: §4 · pp. 7–8

5 RESULTS

ProgressCompass improves a frozen PRM’s progress estimates and remains robust when instructions and executions diverge. Ablations show the expected transition matters for verification, while scheduling independent episodes reduces runtime without changing measured accuracy.

5.1 OVERALL PERFORMANCE

  • 25.4 to 9.3 progress error and 0.53 to 0.93 rank agreement: frozen RoboMeter-4B becomes best on every context form with ProgressCompass.The comparison includes five PRMs reading the full episode and instruction, plus R2VLM.Source: §5.1 · p. 8, §5 · p. 8

5.2 WHEN INSTRUCTION AND EXECUTION DO NOT MATCH

  • Negative cases show deviations of 7.3 versus 32.7/50.3 on Early stop, 15.8 versus 24.0–40.2 on Extra steps, and never above zero on Mismatch.Lower deviation is better; ProgressCompass alone reaches the correct terminal value of 100 on Extra steps.Source: Figure 7 · p. 8, §5.2 · p. 8

5.3 WHAT THE EXPECTED TRANSITION BUYS

  • 8.6 end-of-subtask error rises to 17.2 without τt at V and 18.8 without it at O and V; progress MAE rises to 17.5 and 18.3.O proposes the same number of steps in all runs, while episodes stalled with an unverified subtask rise from 7 to 40 and 45.Source: Figure 8 · p. 9, §5.3 · p. 9

5.4 PARALLELIZING THE LOOP ACROSS EPISODES

  • Parallel scheduling cuts per-episode time from 114.7 to 39.5 s, with MAE and ρ unchanged within paired bootstrap intervals.The scheduler runs up to five episodes concurrently while preserving each episode’s internal order.Source: §5.4 · p. 9

6 RELATED WORK

Prior embodied progress models use several supervision and modeling strategies, but their evaluations mostly concern tasks readable from the current observation. The paper connects this gap to agentic context management and applies that idea to frozen PRMs.

6 RELATED WORK

7 CONCLUSION

The paper introduces a benchmark for context-dependent progress estimation and finds that PRMs improve when given the right context. ProgressCompass supplies context to a frozen PRM and improves its error while remaining robust across instruction-execution mismatches.

7 CONCLUSION

  • ProgressCompass supplies context to a frozen PRM with general-purpose VLMs, without annotation or training, more than halves backbone error, and stays robust across three instruction-execution mismatches.The paired diagnosis finds that each tested PRM cuts error by 77% to 82% when given the right context, even after reading the full history.Source: §7 · p. 9

A.1 SETUP

The step-form setup expresses overall progress using the active step’s position and local progress. It also states the conditions under which earlier history can be compressed into sufficient context.

A.1 SETUP

  • For K ordered steps, active position is one plus the number of completed steps; local progress within the active step lies in [0,1] and is 0 once all steps are done.Source: §A.1 · p. 14
  • A locally identifiable step has a self-contained description that, with observations since it began, determines progress within that step.The description includes the object, source, target, and completion condition.Source: §A.1 · p. 14
  • Under that condition, earlier history affects progress only through a constant-size sufficient context; differing progress definitions shift the theorem bounds by at most their mean absolute difference.Two observations count as the same when they show the same scene and differ only in details that carry no progress.Source: §A.1 · p. 14

A.2 CONVENTIONS OF THE SECOND BOUND

The section defines context-related error bounds and lemmas for progress estimation. It specifies how context floors, history gaps, and verification affect those bounds.

A.2 CONVENTIONS OF THE SECOND BOUND

  • The second bound defines α, β, and γ as rates of premature proposal, acceptance of a premature proposal, and rejection of a true completion.The analysis assumes no component is accurate by default; every term is defined rather than assumed small.Source: §A.2 · p. 14

A.3 CONTEXT FLOORS AND LEMMAS

  • The context floor δX averages the minimum achievable error on frames labelled by context form X, and any estimator using only the current scene faces a floor of δS + δQ + δR.For equally frequent same-scene frames whose progress differs by δ, the minimum mean error is δ/2; omitting unlabelled frames makes the floors conservative.Source: §A.3 · p. 14
  • Sampling B frames uniformly from a history of length t can miss a distinguishing window of L frames with probability at least 1 − BL/t.When every sample misses the window, the two executions provide identical inputs to the estimator.Source: §A.3 · p. 14
  • Without verification, premature step advances can inflate progress until the actual step completes and corrupt later context derived from the record.With independent premature advances at rate α, the probability of at least one across K steps is 1 − (1 − α)^K.Source: §A.3 · p. 15
  • Verification limits an early pass to cases where the Navigator proposes completion early and the Verifier accepts, reducing the probability from αk to at most αkβk.Rejecting a true completion delays the position by one step and leaves the record unchanged; after a premature advance, position can realign when the step truly completes unless another false acceptance occurs first.Source: §A.3 · p. 15

A.4 PROOF OF THEOREM 1

The theorem bounds the error of current-frame estimators below and ProgressCompass's error above, while stating a verification bound and the scope of the result.

A.4 PROOF OF THEOREM 1

  • Every estimator depending only on the instruction, current observation, and independent randomness has error at least δS + δQ + δR.The ProgressCompass upper bound is εPC ≤ (ηPC + λ)/K, where ηPC measures position error and λ is local error when the plan matches and position is correct.Source: §A.4 · p. 15
  • A step passes early with probability at most αβ when ProgressCompass verifies completion, compared with α without verification.A rejected true completion only delays the position.Source: §A.4 · p. 15
  • With step weights summing to one, the theorem's 1/K terms become maxℓwℓ; the paper uses uniform weights wℓ = 1/K.The first bound covers current-frame estimators and fixed-budget sampled-history estimators, but not models that read the whole history.Source: §A.4 · p. 15
  • The theorem gives a condition under which ProgressCompass is more accurate, not a guarantee.Its first bound does not cover a model that reads the whole history.Source: §A.4 · p. 15

B PR O G R E S SCO M P A S S IMPLEMENTATION

ProgressCompass combines a frozen PRM with an Orienter, Navigator, and Verifier in a turn-by-turn loop. The section also specifies how the compared PRMs are run with and without context.

B PR O G R E S SCO M P A S S IMPLEMENTATION

  • The implementation uses RoboMeter-4B as the frozen PRM, Qwen3.5-27B as Orienter, and Qwen3.5-9B as both Verifier and Navigator.All three VLM components decode at temperature 0 with thinking disabled; episodes are sampled every 10 frames, up to 128 frames.Source: B PR O G R E S SCO M P A S S… · p. 15
  • The Orienter builds and updates an ordered plan from the instruction and current frame, while the Navigator tracks verified state and requests the next step.Satisfied steps stay fixed; each plan step must be required by the instruction, and the frame grounds its objects without adding steps.Source: B PR O G R E S SCO M P A S S… · pp. 15–16
  • The PRM scores frames since the active step began, and the Verifier checks a candidate completion against three ordered frames before the Navigator advances.Uncertain evidence is rejected; accepted candidate descriptions enter verified memory, with at most eight verifications per step.Source: B PR O G R E S SCO M P A S S… · p. 16

C INFERENCE DETAILS OF THE COMPARED MODELS

  • Compared models retain their own inference procedures, while the context run scores annotated subtask clips using that subtask's instruction instead of the episode instruction.ProgressLM uses five demonstrations; Robo-Dopamine averages its incremental, forward, and backward curves; RoboMeter stores prefix rewards on a 0-to-100 scale.Source: C INFERENCE DETAILS OF THE COMPARED… · p. 16
  • TOPReward min-max normalizes sampled-prefix instruction rewards, while VLAC runs in critic mode and appends the terminal frame when sampling omits it.Both models use their specified context-dependent inference settings.Source: C INFERENCE DETAILS OF THE COMPARED… · p. 16
Loading 2609.36684v1…