PaperlayerView exact revision on arXiv

Source-linked AI summary

SafeInferCom: Safe Inference-Time Compute via Verifier-Guided Mid-Generation Intervention for Robotic Task Planning

Weizhe Xu, Jialiang Fan, Mengyu Liu, Fanxin Kong

arXiv:2610.11223v1cs.ROcs.AIcs.CLcs.LO

TL;DR

Continued LRLM reasoning can overwrite valid intermediate plans or leave violations unresolved, and post-hoc verification may come only after a candidate is complete. SafeInferCom verifies intermediate plans to guide correction and preserve valid candidates during generation. It improves planning success over one-shot inference, while its combination with iterative refinement achieves higher success with fewer output tokens than refinement alone.

  • Problem

    LRLM reasoning can overwrite valid intermediate plans or leave constraint violations unresolved, while existing verification typically occurs only after a candidate is fully generated.

    Source: §I · p. 1

  • Method

    SafeInferCom monitors and verifies intermediate plans, then uses verifier-guided mid-generation actions to preserve valid candidates, correct violations, terminate, or replan.

    Source: §I · pp. 1–2

  • Results

    SafeInferCom improves planning success over one-shot inference, and S+IR achieves higher success with fewer output tokens than IR across all three model scales.

    Source: 1) Final Success Rate and Inference Cost · p. 7

  • Takeaways & Limitations

    VirtualHome and robotic-arm studies demonstrate applicability beyond symbolic benchmarks.

    Source: §VIII · p. 8

Abstract

from arXiv · show

Large Reasoning Language Models (LRLMs) enable multi-step reasoning for robotic task planning, but continued reasoning can overwrite valid intermediate plans or leave constraint violations unresolved, reducing planning reliability and wasting inference-time computation. We develop an inference-time monitor that exposes and verifies intermediate plans without disrupting the original decoding trajectory. Building on this monitor, we propose SafeInferCom, a formal verifier-guided framework that preserves valid intermediate plans and directs error correction during generation. Experiments across multiple LRLMs and planning domains reveal reasoning-response inconsistency and limited self-correction under one-shot inference. SafeInferCom improves planning success and accelerates error correction relative to one-shot inference. When combined with iterative refinement, it further improves success while reducing token usage compared with refinement alone. We additionally evaluate SafeInferCom in VirtualHome and provide a real-world robotic-arm demonstration.

I. INTRODUCTION

Robotic planning methods trade off formal guarantees, modeling demands, and computational cost, while LRLMs deliberate over plans but can still produce unreliable results. SafeInferCom combines intermediate-plan monitoring with verifier-guided intervention during generation.

A. Classical, Learned, and Language-Based Task Planning

  • Classical planners offer principled, auditable validation but require engineered symbolic models and can become computationally expensive; learned planners depend on training data and generally lack explicit safety guarantees.Language-based planners generate action sequences from natural-language instructions and contextual descriptions.Source: §I · p. 1, A. Classical, Learned, and… · p. 2

I. INTRODUCTION

  • LRLMs can overwrite safe intermediate plans or carry constraint violations forward, while post-hoc verification arrives too late to prevent these reasoning failures.The paper describes this reasoning–response inconsistency as reducing reliability and wasting inference-time computation.Source: §I · p. 1
  • SafeInferCom uses verifier feedback to continue, correct, terminate, or replan during generation, preserving valid candidates and addressing violations before a full failed trajectory is produced.The monitor first exposes intermediate plans without altering primary decoding; the controller then intervenes using verification outcomes.Source: §I · pp. 1–2
  • The paper evaluates intermediate-plan fidelity, self-correction, and success–compute trade-offs across LRLMs and domains, with additional VirtualHome and real-robot studies.The introduction frames these studies as evaluating SafeInferCom against one-shot inference and post-hoc refinement.Source: §I · p. 2

B. Language Models with Symbolic Planning and Verification

Prior verification and reasoning-evaluation approaches do not directly regulate intermediate robotic plans during a single inference pass. SafeInferCom uses formal verification of intermediate candidates to guide correction and compute regulation.

B. Language Models with Symbolic Planning and Verification

  • Unlike post-hoc plan checking, SafeInferCom verifies candidates induced from intermediate reasoning prefixes and uses the feedback for mid-generation control.Prior methods typically verify completed plans or check between successive generation rounds.Source: B. Language Models with Symbolic… · p. 2

C. Evaluation of Language-Model Reasoning

  • Existing reasoning evaluations focus on language or logical quality, not whether intermediate reasoning implies an action sequence valid under an explicit robotic task specification.A coherent trace can still imply a plan that violates action preconditions, safety constraints, or task goals.Source: C. Evaluation of Language-Model… · p. 2

D. Inference-Time Compute Scaling and Regulation

  • SafeInferCom uses external formal verification of intermediate plans, rather than predicted quality or confidence, as its control signal for corrective intervention and verified early termination.The framework also reuses prefix decoding state to reduce monitoring overhead.Source: D. Inference-Time Compute Scaling and… · p. 2

III. PRELIMINARIES AND PROBLEM STATEMENT

The paper studies LRLM planning where an external verifier can assess candidate plans against an explicit task specification. It aims to monitor reasoning without altering one-shot inference and use intermediate verification to guide generation.

A. Robotic Task Planning with Safety Constraints

  • A valid plan must be executable under the domain’s action semantics, satisfy safety constraints throughout execution, and reach a goal state.The planning setting specifies a domain, initial state, goal condition, and optional safety constraints.Source: A. Robotic Task Planning with Safety… · pp. 2–3
  • The external verifier labels candidates UNSAFE, SAFE-BUT-NOT-GOAL, or VALID and returns the first failing step and diagnostic information for invalid plans.Validity guarantees are relative to the supplied specification and a correctly functioning verifier; they do not cover task-grounding errors, symbolic-model omissions, perception, or physical execution.Source: A. Robotic Task Planning with Safety… · p. 3

B. LRLM Inference-Time Reasoning Setting

  • An LRLM generates an intermediate reasoning trace followed by a final plan, and its reasoning may disagree with the final plan’s validity.A model may lose a valid plan through continued deliberation or reach a valid final plan without showing a valid intermediate candidate.Source: B. LRLM Inference-Time Reasoning Setting · p. 3

C. Problem Statement

  • The monitoring objective evaluates reasoning without changing one-shot inference, while the control objective uses intermediate feedback to guide generation and avoid needless reasoning after a valid candidate appears.In standard one-shot inference, the LRLM completes reasoning and its final response without verifier input during generation.Source: C. Problem Statement · p. 3

IV. INFERENCE-TIME REASONING MONITOR

The monitor converts selected LRLM reasoning prefixes into candidate plans for an external verifier while leaving primary decoding intact. It uses reasoning transitions to choose checkpoints and an auxiliary branch to induce plans.

IV. INFERENCE-TIME REASONING MONITOR

  • The monitor exposes intermediate planning states by converting selected reasoning prefixes into verifier-checkable candidate plans without altering standard one-shot inference.Verifier outcomes are later used by SafeInferCom for mid-generation control.Source: §IV · p. 3

A. Reasoning Transition Monitoring

  • Checkpoints use natural-language transitions such as “wait,” “alternatively,” and “hmm” as markers of points where the model examines its planning decisions.These transitions are not explicitly requested in the prompt.Source: A. Reasoning Transition Monitoring · p. 3

B. Intermediate Plan Induction

  • At a checkpoint, the monitor saves the primary decoding state, then uses an auxiliary branch to append an induction prompt to the preceding reasoning prefix and produce a verifier-format plan.After verification, the branch can be discarded and primary decoding resumed; cached state reuse limits the additional decoding to the prompt and induced continuation.Source: B. Intermediate Plan Induction · p. 4, A. Reasoning Transition Monitoring · p. 4
  • Induction exposes the decisions already reflected in the reasoning prefix rather than repairing or completing them, preserving errors and revisions while excluding explicitly rejected alternatives.The induced candidate plan is evaluated by the external verifier.Source: B. Intermediate Plan Induction · p. 4

C. Verification and Reasoning Continuation

The monitor verifies intermediate plans while preserving the primary decoding trajectory. SafeInferCom uses those outcomes to guide continuation, correction, preservation of valid plans, or replanning.

C. Verification and Reasoning Continuation

  • The monitor formally verifies induced candidate plans and returns failure-step diagnostics that provide a control signal for SafeInferCom.After verification, the auxiliary branch is discarded and the saved primary decoding state is restored, so monitor-only operation preserves the original one-shot trajectory under the same decoding configuration.Source: C. Verification and Reasoning… · p. 4

V. SAFEINFERCOM: FORMAL VERIFIER-GUIDED MID-GENERATION INTERVENTION

  • SafeInferCom uses intermediate verification outcomes to decide whether reasoning continues, is corrected, terminates with a verified plan, or escalates to replanning.Unlike monitoring alone, intervention can prevent detected violations from persisting or valid candidates from being overwritten.Source: §V · p. 4

A. Intervention Principles

  • Its principles preserve formally valid plans, continue safe but incomplete candidates while targeting detected violations, and escalate recurring errors to broader replanning.Local corrective feedback may fail to resolve an underlying error, leading similar violations to recur across checkpoints.Source: A. Intervention Principles · p. 4

B. Mid-Generation Verification and Intervention

At reasoning checkpoints, SafeInferCom verifies candidate plans and maps each outcome to a decoding action. These interventions occur during generation, before a complete failed trajectory is produced.

B. Mid-Generation Verification and Intervention

  • At each checkpoint, the monitor induces a candidate plan, verifies it, and maps the outcome to a decoding action; the end-of-reasoning marker is also checked.The end marker is treated as an additional checkpoint before reasoning can terminate.Source: B. Mid-Generation Verification and… · p. 4
  • A VALID candidate triggers early termination and is returned as the final plan, preventing further decoding from overwriting it.This action preserves a verified candidate rather than continuing generation.Source: B. Mid-Generation Verification and… · p. 4
  • SAFE-BUT-NOT-GOAL candidates continue without feedback at transition checkpoints, while the same outcome at the end marker prompts further reasoning toward a goal-achieving plan.The controller avoids interrupting safe reasoning that may naturally converge to a correct plan.Source: B. Mid-Generation Verification and… · p. 4
  • For UNSAFE candidates, verifier diagnostics guide correction; recurring violation patterns trigger a stronger replanning prompt within the active context.Replanning follows the same violation pattern persisting for N consecutive checkpoints.Source: B. Mid-Generation Verification and… · p. 5
  • SafeInferCom integrates formal verification into active reasoning to intervene before a complete failed trajectory is generated, and can be combined with post-hoc refinement.Fig. 2 identifies the method workflow.Source: B. Mid-Generation Verification and… · p. 5, Figure 2 · p. 5

VI. EXPERIMENTS

  • The evaluation compares baseline inference strategies on PDDL tasks and includes additional deployment studies in VirtualHome and on a physical robotic arm.The experiments are introduced as evaluating SafeInferCom against baseline inference strategies.Source: §VI · p. 5

A. Experimental Setup

The experiments use five PDDL planning domains and assess final-plan outcomes alongside reasoning-level inconsistency and self-correction. Metrics cover validity, token use, candidate revisions, and recovery from unsafe plans.

A. Experimental Setup

  • Experiments use 500 generated instances across Blocksworld, Logistics, Depots, Gripper, and Miconic, with induced and final plans verified using VAL.The setup uses 100 instances per domain and an NVIDIA A100-SXM4-80GB.Source: A. Experimental Setup · p. 5

B. Evaluation Metrics

  • Success rate measures the fraction of instances with verifier-valid final plans, while token usage counts generated output tokens.Token accounting excludes monitoring overhead in one-shot reasoning analyses but includes induction and intervention outputs across SafeInferCom refinement rounds.Source: B. Evaluation Metrics · p. 5
  • The two inconsistency metrics capture final failure after an observed valid intermediate candidate and final success without any observed valid intermediate candidate.The metrics contrast observed intermediate-plan validity with final-plan validity.Source: B. Evaluation Metrics · p. 5
  • Revision of an action sequence serves as an observable indicator of error detection, with TPR and TNR measuring revisions after unsafe candidates and retention or extension of safe candidates.A revision changes the prior action sequence rather than merely extending it.Source: B. Evaluation Metrics · pp. 5–6
  • Rcorr measures the average subsequent checkpoints needed for an unsafe candidate to become SAFE-BUT-NOT-GOAL or VALID; lower Rcorr indicates faster recovery.It evaluates correction efficiency starting from an unsafe checkpoint.Source: B. Evaluation Metrics · p. 6

C. Baselines

The baselines contrast one-shot generation, post-hoc refinement, mid-generation intervention, and symbolic planning. The evaluation also includes a reasoning-evaluation table spanning LRLMs and planning domains.

C. Baselines

  • One-shot inference has no external feedback during decoding; iterative refinement checks completed outputs and can append verifier diagnostics for up to three rounds.SafeInferCom instead uses intermediate verification to continue, correct, terminate, or replan during generation.Source: C. Baselines · p. 6
  • Early Stopping retains intermediate-plan monitoring and early termination but disables corrective feedback and replanning, isolating valid-plan preservation.SafeInferCom plus refinement runs SafeInferCom within each refinement round and uses the same three-round maximum as iterative refinement.Source: C. Baselines · p. 6
  • Fast Downward achieves 100% success on these benchmarks and serves as a symbolic-planning reference; SafeInferCom targets language-model planning with external formal verification.The distinction is relevant to VirtualHome, where a validator is available but an equivalent off-the-shelf symbolic planner is not directly provided.Source: C. Baselines · p. 6

D. Validation of Intermediate Plan Induction

  • Table II covers reasoning evaluation across LRLMs and planning domains.The supplied table passage identifies the table's scope but gives no cell values.Source: Table II · p. 6

E. Reasoning Behavior Analysis

One-shot reasoning can reach high success while still overwriting valid plans and correcting unsafe candidates inefficiently. SafeInferCom improves success and correction efficiency, while combining it with iterative refinement improves the success–compute trade-off over refinement alone.

1) Outcome Performance and Inference Cost:

2) Reasoning–Response Inconsistency:

  • 84.4% of failed 32B Miconic outputs follow an observed valid intermediate candidate, showing that continued reasoning can overwrite a correct solution.Some successful outputs also occur without a valid candidate at monitored checkpoints.Source: 2) Reasoning–Response Inconsistency · p. 6

3) Limited Self-Correction Capability:

  • Average TPR remains below 37%, while TNR exceeds 91%, indicating many unsafe candidates go unrevised while safe candidates are usually retained or extended.Even when correction occurs, recovery typically takes multiple subsequent checkpoints.Source: 3) Limited Self-Correction Capability · p. 7

1) Final Success Rate and Inference Cost:

  • 54.8% to 78.2%: 14B success rises from one-shot inference to standalone SafeInferCom.Standalone SafeInferCom also consistently outperforms Early across the reported comparisons.Source: 1) Final Success Rate and Inference Cost · p. 7
  • Standalone SafeInferCom trades additional inference-time computation for higher planning success relative to one-shot inference.Early termination can avoid continued reasoning after a valid plan, but monitoring, correction, and replanning add computation when no valid plan has been reached.Source: 1) Final Success Rate and Inference Cost · p. 7
  • S+IR achieves higher success while using fewer output tokens than IR across all three model scales.The comparison supports a better success–compute trade-off from combining mid-generation control with post-hoc refinement.Source: 1) Final Success Rate and Inference Cost · p. 7
  • SafeInferCom does not increase mean runtime over one-shot inference for the 32B model, while S+IR has the lowest mean and median latency among compared methods.S has higher 95th-percentile latency, indicating long-tail overhead for trajectories needing repeated intervention.Source: 1) Final Success Rate and Inference Cost · p. 7
  • 22.9% reduction: reasoning-prefix KV-cache reuse lowers average end-to-end latency from 156.43 s to 120.65 s on 100 Blocksworld instances with the 32B model.P95 latency falls from 495.43 s to 342.47 s.Source: 1) Final Success Rate and Inference Cost · p. 7

2) Correction Efficiency:

Verifier-guided feedback reduces the number of checkpoints needed for correction, and SafeInferCom improves success across additional LRLM families and in VirtualHome.

2) Correction Efficiency:

  • Rcorr falls from 4.03 to 2.73 for 7B, 3.72 to 2.70 for 14B, and 3.88 to 1.99 for 32B with SafeInferCom versus one-shot inference.The authors report this as evidence that verifier-guided feedback accelerates error correction during reasoning.Source: 2) Correction Efficiency · p. 7

3) Generalization Across LRLM Families:

  • SafeInferCom improves success for R1-14B, Qwen3-4B-Thinking, and Phi-4-mini-reasoning in Logistics, while token-use effects depend on the model.S+IR further improves success across all three families, also with model-dependent token overhead.Source: 3) Generalization Across LRLM Families · p. 7

4) VirtualHome Evaluation:

  • In VirtualHome, 14B success rises from 20.0% with one-shot inference to 59.0% with SafeInferCom and 69.0% with S+IR.Success improves across all three model scales, with larger gains for 14B and 32B; the gains involve substantially higher output-token usage.Source: 4) VirtualHome Evaluation · p. 7

5) Robotic-Arm Demonstration:

The conclusion reports improved planning reliability and correction efficiency, with further gains from iterative refinement, and the arm study demonstrates execution of a verified plan from visual input through physical action. The discussion bounds the approach by plan quality, checkpoint verification cost, and reliance on explicit task specifications and external formal verification.

5) Robotic-Arm Demonstration:

  • The Blocksworld demonstration links a scene image and target state to a PDDL problem, then executes a verified SafeInferCom plan with a six-degree-of-freedom arm.A vision-language model derives the symbolic initial state; a third-person camera records the demonstration.Source: 5) Robotic-Arm Demonstration · p. 7

VII. DISCUSSION

  • Intermediate-plan induction may omit or alter prefix actions, making verifier outcomes diverge from the plan implied by the reasoning.Source: §VII · p. 7
  • SafeInferCom targets validity and inference efficiency, not optimality, so verified plans may contain unnecessary actions or cost more to execute.Source: §VII · p. 8
  • Although SafeInferCom reuses the LRLM prefix KV cache, it independently verifies each checkpoint; incremental reuse is proposed for future work.Source: §VII · p. 8
  • SafeInferCom assumes an explicit task specification and external formal verifier; natural-language extensions may weaken guarantees through translation or learned-verifier errors.Source: §VII · p. 8

VIII. CONCLUSION

  • SafeInferCom improves planning reliability and correction efficiency, with further gains when combined with iterative refinement.VirtualHome and robotic-arm studies demonstrate applicability beyond symbolic benchmarks.Source: §VIII · p. 8

Related papers

Loading 2610.11223v1…