No Free Checker: A Survey of Verifiers for Robot Policies

Introduction

• Reliable robot verification is difficult because success is inferred from imperfect sensing, rollouts consume time, errors may need detection before task completion, and performance is continuous.A binary success label can miss differences in progress, safety, and execution quality.Source: §1 · p. 3
• The survey covers roughly 150 works and compares verifiers by availability—the cost, timing, and frequency of verdicts—and credibility: how much a high score tells us about the task.A verdict may be boolean, scalar, vector-valued, or a distribution; the candidate may range from an action chunk to a full policy or trajectory.Source: §1 · p. 3
• The survey groups verifiers by judgment source into human, rule-based and formal, learned and pretrained, and model-intrinsic families.Human judgments are costly but refer directly to task intent; rules are repeatable but depend on accurate criteria and assumptions; learned judgments are cheap and dense but depend on accuracy and generalization; intrinsic signals are easy to obtain but relate to task performance indirectly.Source: §1 · pp. 3–4
• Across verifier families, cheaper and denser judgments tend to be less direct measures of task performance, making high availability and high credibility costly to obtain together.The authors call this trade-off “no free checker.”Source: §1 · p. 4
• The paper examines three ways to measure verifier error: agreement with a fixed reference, the policy trained using the verifier, and behavior when searching for inputs where its score is wrong.It closes with reporting metrics intended to make verifier claims checkable by others.Source: §1 · p. 4

2 Human Verifiers

• Human verdicts are costly but task-general and close to intended goals, and their labels supervise cheaper verifiers.The four roles differ in what a person sees and returns: comparisons, time-step scores, interventions, or judgments of generated rollouts.Source: §2 · p. 4, §2.4 · p. 7

2.1 Comparing Trajectories

• Pairwise human preferences provide relative judgments: RoboArena ranks fixed policies, while preference-learning methods train reward models from comparisons.PEBBLE reduces human labels with unsupervised pre-training and relabeling stored experience; B-Pref uses simulated teachers to test algorithm robustness.Source: §2.1 · p. 5

2.2 Scoring Trajectories

• Reward sketching records a 0-to-1 progress curve over a single trajectory, and a small set of curves trains a reward model to score a larger corpus frame by frame.The annotator draws the curve while viewing the corresponding video frame.Source: §2.2 · pp. 5–6

2.3 Intervening During Execution

• Human-gated supervisors take over and supply corrective actions that can prevent failure and train policies, while robot-gated verifiers automate requests for help.ThriftyDAgger uses an intervention budget, AIM learns a proxy Q-function for requesting assistance, and Sirius-Fleet predicts failures ahead of time; requests must be early enough to prevent failure but selective enough to limit effort.Source: §2.3 · pp. 6–7

2.4 Validating Generated Rollouts

• For generated rollouts, humans judge outcome preservation against real execution, combining task success and video fidelity or scoring success directly; failed preservation can make policies look better than reality.GigaWorld-1 asks annotators for a four-level score, while VeoEval has people score success across nominal, out-of-distribution, and red-teaming conditions.Source: Figure 2 · p. 6, §2.4 · p. 7

3.1.1 Temporal-Logic Scoring

• Temporal-logic scoring expresses required behavior and timing as a formal specification, then assigns a satisfaction margin whose sign indicates satisfaction or violation and whose magnitude indicates distance from the boundary.The same margin applies to recorded executions, simulated rollouts, and predicted trajectories; it can also serve as a dense optimization reward when exact states are available.Source: §3.1.1 · pp. 7–9

3 Rule-Based and Formal Verifiers

• These verifiers use predefined criteria on full trajectories, final states, or physical constraints, as grouped in Figure 3.Full-trajectory methods include temporal-logic scoring, trajectory geometry, and model-written code; terminal-state methods check goal satisfaction, while physical-constraint methods check feasibility or safety.Source: §3 · p. 7, Figure 3 · p. 8

3.1.1 Temporal-Logic Scoring

• Conformal calibration can bound predictive-verification errors under exchangeability, but policy training against the verifier breaks that condition; extensions address distribution shift and spatio-temporal specifications.Calibrated prediction can warn of an impending specification violation, while a related world-model method calibrates dynamics error and a learned safety score for closed-loop constraints.Source: §3.1.1 · pp. 8–9

3.1.2 Geometry-Based Scoring

• Geometry-based scoring needs demonstrations but no hand-written task specification; RINSE uses smoothness and self-similarity scores to filter training data, not to establish task success.RINSE trains on its highest-scoring subset and performs better than training on the full set (58); smooth trajectories may still fail, while jerky ones may succeed.Source: §3.1.2 · pp. 9–10

3.1.3 Model-Written Rules

• Model-written rules serve as training objectives or runtime monitors: Eureka selects reward code using the environment’s task metric, whereas Code-as-Monitor and ReKep do not check generated constraints against task requirements.Eureka’s converged reward outperforms human-written rewards on most tasks in a large benchmark; DrEureka adds safety instructions and domain-randomization ranges for hardware transfer.Source: §3.1.3 · p. 10

3.2 Terminal-State Verification

• A goal predicate is a cheap binary test on exact final simulator state, used for benchmark success rates and sparse RL rewards, but it may miss task failure and can be exploited by a policy.MimicGen uses the predicate to generate and retain successful trials automatically; SimpleVLA-RL trains from the simulator’s success bit alone, while the same pose test on hardware requires a different verifier.Source: §3.2 · pp. 10–11

3.3 Physical-Constraint Verification

• Control barrier functions can certify forward invariance and filter unsafe policy inputs, but guarantees for neural approximations require a separate verification step that some experiments omit.So et al. (28) construct a filter from a nominal policy’s value function; PNCBF supplies the verification step skipped in those experiments and extends the guarantee to an 𝜖-optimal approximate value function.Source: §3.3 · pp. 11–12
• Symbolic feasibility checks test whether a plan’s action preconditions and affordances can be satisfied before execution, while Text2Motion’s coverage is limited by its hand-maintained symbolic abstraction.Task and motion planning returns a symbolic plan for revision when no motion satisfies it (115); Text2Motion applies feasibility checks to language-model-proposed skill sequences (86).Source: §3.3 · p. 12

4 Learned and Pretrained Verifiers

• Learned and pretrained verifiers serve three roles: evaluating behavior, selecting candidates at inference time, and feeding scores into policy or data decisions.Task-specific verifiers can be compared with real robot outcomes; pretrained-only verifiers require transfer across embodiments and task distributions before such comparison is possible.Source: §4 · p. 12, Figure 4 · p. 13

4.1 Evaluating Behavior and Policies

• Evaluation verifiers provide localized process-level scores or categorical judgments, or a holistic verdict for a completed rollout.Policy evaluations and rankings aggregate outcomes across repeated rollouts; methods include dense frame-level rewards, localized success judgments, and terminal rollout classifiers.Source: §4.1 · pp. 13–14

4.2 Inference-Time Verification

• At inference time, verifiers score candidate actions, skills, or plans so the robot can execute the highest-scoring option.Direct-selection methods include sampling policy outputs and adding external test-time verification; calibrated prediction sets can instead trigger human assistance when several actions remain plausible.Source: §4.2 · p. 14
• World-model methods either score predicted consequences of candidate actions or monitor the predicted consequences of an action already chosen.AHEAD estimates uncertainty across five sampled rollouts, while Foresight monitors a committed action block and conformally calibrates predictions using trajectory-level labels.Source: §4.2 · pp. 14–15

4.3 Feeding Verifier Feedback into Policy Learning

• Verifier feedback updates policy parameters through reward, advantage, or preference objectives, and can also filter new rollouts for further training.RECAP more than doubles throughput and roughly halves failure rate on the most challenging household tasks (40).Source: §4.3 · p. 15
• Demonstration curation can score an example alone, in its corpus context, or by its contribution to downstream policy performance.DataMIL estimates downstream contribution without a new robot rollout, while that contribution is the most expensive curation signal because it is defined through a training run. Re-Mix optimizes the worst-performing domain and outperforms uniform mixtures and hand-specified compositions (9); SCIZOR combines progress estimation with deduplication, with ablations showing the progress estimator contributes more (120).Source: §4.3 · pp. 15–16

5.1 Policy-Based Self-Verification

• Policy-based verifiers detect runtime deviation or select candidate actions from the policy’s own signals, but a familiar-looking failure may still score highly.SAFE uses a latent-representation predictor and calibrated, time-varying threshold; its false-positive guarantee assumes matched calibration and test distributions and may fail under cross-task shift (26).Source: §5.1 · p. 17

5.2 Prediction-Based Outcome Verification

• VIPER’s expert-video likelihood reward supports expert-level control and transfer, while likelihood-based verification may mistake high generative likelihood for semantically correct behavior under distribution shift (71).Garrido et al. use predicted-versus-observed latent discrepancy to detect physical implausibility zero-shot; latent-space prediction substantially outperforms pixel-space prediction and multimodal language models on this task (145).Source: §5.2 · pp. 17–18
• A 570K-parameter detector, tested on a diffusion policy and a real-world bimanual cable-manipulation dataset, beats the next-best learned baseline by 3.8 points in failure-detection rate.It needs no failure examples of its own.Source: §5.2 · p. 18
• Reachability values use predicted dynamics to identify states where failure cannot be avoided, enabling a safety filter to reject unsafe actions before failure occurs (42, 147).Follow-up work addresses partial-observation errors and varies the filter through adjustable latent constraints, uncertainty penalties, learned control barrier functions, or simulated adversarial interactions (148, 75, 149, 150, 151).Source: §5.2 · p. 18

5.3 Beyond Action Verification: Task Selection

• ACCEL selects and edits training environments where the policy’s value function most underestimates its returns, using value loss as a single-policy proxy for regret (152, 153).Neither ACCEL nor PAIRED was evaluated on a robot, and their scores choose the next environment rather than judging whether a completed behavior succeeded.Source: §5.3 · pp. 18–19

5 Model-Intrinsic Verifiers

• Because model-intrinsic scores come from models already running, they are the survey’s cheapest verifiers, but can score highly a failure the model does not recognize.They can assess actions, outcomes, or whole environments rather than relying on a separate verifier.Source: §5 · p. 16, §5.3 · p. 19

6 Validating the Verifier

• Verifier errors can affect demonstrations, policy or action rankings, training rewards, and runtime actions; errors in searched-against rewards or gates can be exploited, unlike errors averaged over unsought candidates.Curation and ranking select among candidates produced without access to the score, whereas training rewards and runtime gates apply to candidates searched for score errors.Source: §6 · p. 19

6.1 Agreement on a Fixed Set of Rollouts

• A fixed-rollout agreement rate omits error attribution, uncertainty, and input type; separate safety evaluations also find that policies completing more tasks are not safer.Agreement depends on the rollouts tested, and GE-Sim 2.0 reports an eight-point accuracy gap between generated rollouts and recorded video of the same tasks.Source: §6.1 · pp. 19–20

6.2 Measuring a Verifier by the Policy It Trains

• Agreement and downstream policy gain can diverge because training drives policies toward rollouts where the verifier may be wrong; measuring policy gain is expensive and rare.The corresponding measurement for robot verifiers has yet to be run, and agreement and downstream gain are reported separately and rarely for the same verifier.Source: §6.2 · pp. 20–21

6.3 Measuring a Verifier Under Optimization

• Optimization can drive policies toward wrong-scoring inputs, making reported reward gains overestimates and invalidating released calibration.Filtering by verifier scores changes the rollout distribution, so conformal coverage may fail; continuous online recalibration provides asymptotic rather than finite-sample guarantees.Source: §6.3 · pp. 21–22

6.4 Metrics for Validating a Verifier

• The section proposes nine metrics to make verifier claims comparable, including proxy gain transfer: Δpanel/Δproxy, reported with both underlying differences because the ratio is unstable when Δproxy is small.A ratio of one means the panel confirms the whole gain, zero means it confirms none, and a negative value means the panel scores the policy lower while the training reward rises.Source: §6.4 · p. 22

Conclusion

• The four verifier families are human, rule-based and formal, learned and pretrained, and model-intrinsic, with the latter two having credibility least often established.Learned and pretrained verifiers form the survey’s largest group, while model-intrinsic verifiers cost the least to obtain.Source: §7 · pp. 22–23
• As judgment moves closer to the evaluated model, verdicts become cheaper, earlier, and more frequent, while evidence of actual task performance becomes harder to assemble.Human judgments are costly but directly reflect task intent; rule-based and formal checks can be repeatable and strongly guaranteed when predicates, state estimates, and dynamics assumptions fit the task. Learned verifiers are inexpensive and dense but depend on their training and validation data, while model-intrinsic scores are cheapest and indirectly related to task performance.Source: §7 · p. 23
• Verifier credibility can be assessed through comparisons ending in human labels or benchmarks reporting the verifier’s own error rate, but label error is rarely reported.Curation and ranking can tolerate average error, whereas training rewards and runtime gates demand more because search can find inputs where the score is wrong; collected agreement rates and downstream gains used non-adversarial candidates.Source: §7 · p. 23
• The survey identifies achieving high availability and high credibility together as the main future work in robot-learning verification.Table 7 presents nine metrics for making verifier claims comparable: most use measurements authors already have, while the lower block requires searching against the verifier, which robotics has yet to do.Source: §7 · p. 23, Table 7 · p. 23

• The authors used a large language model to draft parts of the survey and checked every claim, citation, and count against the cited papers.Source: Use of Large Language Models · p. 24