Source-linked AI summary
From Natural Language Policies to Executable Obligations: A Verification Harness for Dependable In-Car LLM Agents
Radouane Bouchekir, Damir Safin, Tomas Bueno Momcilovic
TL;DR
In-car LLM agents must follow operating policies despite ambiguity, unavailable capabilities, and safety-relevant actions. AgentGuardUtil compiles policies into executable rules and places deterministic verification around a fallible proposing model. It beats the baseline on all three reported scores, including a 13.3-point Pass@3 gain, while remaining slower and more expensive.
Problem
In-car agents face incomplete or ambiguous requests, unavailable resources, and domain policies governing preconditions, side effects, confirmations, and disclosures.
Method
AgentGuardUtil uses a student–teacher architecture in which compiled policy rules, deterministic gates, and an obligation engine verify drafts against observed data before bounded revision and execution.
Results
AgentGuardUtil beats the baseline on all three scores, with gains of 13.3 points on Pass@3 and 3.3 points on Pass3.
Takeaways & Limitations
Deterministic checks block hallucinated identifiers and enforce executable policy side-effects, while ambiguous requests remain dependent on the LLM critic.
Takeaways & Limitations
The harness costs 362k tokens and 19.2 s per turn versus 82k and 3.2 s for the baseline, and its checks are silent when no compiled rule covers a task.
Abstract
from arXiv · showhide
Large Language Models (LLMs) agents deployed in vehicles must satisfy a written operating policy on every turn: a single hallucinated identifier, omitted mandatory side-effect, or premature completion claim fails the task. We present AgentGuardUtil, our entry to CAR-bench Track~1, which treats the AI planer (LLM) as a fallible proposer inside a grounded verify-and-revise loop. Its core novelty is a runtime policy compiler: the natural-language policy shipped with each conversation is compiled, once per policy, into typed machine-checkable rules, a subset of which receive an executable form. A deterministic obligation engine interprets these rules against live tool results and the simulated post-write state of the draft itself, emitting the exact remedial calls with computed arguments rather than natural-language reminders. Around this engine, 25 deterministic gates (identifier provenance, schema and enum validity, gather-before-act, confirmation and future-time protocols) and an LLM critic produce tiered findings that drive a bounded revision loop tuned for the pass k metric.
1 Introduction
AgentGuardUtil addresses the risks of deploying fallible LLM agents in vehicles by separating action proposal from policy-compliant approval. It compiles each conversation’s policy into executable checks and uses deterministic verification with bounded revision to prevent unsupported or incomplete actions.
- Motivation: In-car agents act on safety-relevant systems under policies governing ambiguous requests, unavailable capabilities, preconditions, side effects, confirmations, and disclosures.CAR-bench evaluates these deployment challenges through conjunctive criteria covering environment state, tool execution, policy compliance, and task termination.
- Approach: AgentGuardUtil assigns proposal to a fallible LLM student and approval to a verification harness that rechecks every draft before execution.The harness carries policy compliance and grounding guarantees independently of the student’s behavior.
- Approach: The runtime compiler converts the conversation’s natural-language policy into typed machine-checkable rules, caching the result so later turns skip compilation.A subset of the compiled rules receives an executable form grounded in real tool signatures.
- Verification: Deterministic gates and an LLM critic produce tiered findings that send flawed drafts back for revision, while the critic can only filter advisory candidates and cannot override hard findings.The design combines chain-of-verification prompting and self-consistency while moving decisive checks into code.
2 The AgentGuardUtil
AgentGuardUtil compiles policies into executable rules, checks proposed turns with deterministic gates and an LLM critic, and revises drafts before release. Its obligation engine projects post-action state and emits concrete missing or excess actions, while tiered findings bound critic-induced variance.
- Runtime Policy Compilation: At a conversation’s first turn, the runtime policy compiler converts policy text into typed rules, caches them by hash, and grounds triggers against the tool inventory.Compound clauses become atomic rules with one condition and one remedy.
- Architecture: AgentGuardUtil separates proposal from approval: the LLM drafts actions, while a verification harness returns findings and re-verifies revisions before release.The harness also sanitizes the spoken reply and releases verified tool calls to the environment.
- Runtime Policy Compilation: Executable obligations bind policy-matched entities to validated tool arguments; invalid argument bindings degrade the rule to advisory rather than forcing an invalid call.The air-conditioning example closes exactly the driver and passenger windows whose observed positions exceed 20%.
- Executable Obligations: The obligation engine estimates post-action state, activates rules whose trigger tools appear in history or the draft, and computes actions still owed after removing completed actions.It also checks expected granularity, including ALL-style collective actions and excess actions on exempt entities.
- Tiered Verification and Variance Control: 25 deterministic gates inspect each draft, history, and compiled rule set, while hard findings always trigger revision and advisory findings reach the student only after critic confirmation.The critic cannot add findings or override hard findings; an oscillation valve prevents repeated conflicting revisions within the round budget.
3 Evaluation
AgentGuardUtil outperformed the baseline across all three CAR-bench scores, with the largest gain on Pass@3 and a smaller gain on Pass3. Results also show that deterministic checks are strongest for hallucination-related errors, while disambiguation remains limited by the LLM critic and execution is costly.
- Overall results: AgentGuardUtil beat the baseline on all three CAR-bench scores, gaining 13.3 points on Pass@3 and 3.3 points on Pass3.The paper attributes the larger Pass@3 gain to removing mistakes before they reach the driver, without making repeated student drafts more consistent.
- Evaluation protocol: Table 2 reports results in total and across three categories of 10 tasks each, using Pass1, Pass@3, and Pass3 as distinct success-rate measures.Pass1 counts single-try success, Pass@3 counts tasks with at least one successful try, and Pass3 requires success in all three trials.
- Category results: Hallucination was the strongest category because identifier provenance and schema checks reduce to deterministic lookups, whereas disambiguation was weakest because deciding whether clarification was needed remained with the imperfect critic.The benchmark reports three categories, but the supplied results passage identifies the clearest contrast between hallucination and disambiguation.
- Overall results: Pass@3 reached 80.0 while Pass3 reached 53.3, leaving eight tasks successful in some trials but not all three.The gap indicates that the checks are stable while the student remains variable across attempts.
- Cost and limitations: The harness required 362k tokens and 19.2 seconds per turn, compared with 82k tokens and 3.2 seconds for the baseline.The paper also reports that spending more tokens across the field did not correspond to a better score, with r=0.435.
4 Conclusion
AgentGuardUtil combines an LLM student with a deterministic teacher that verifies each draft before it takes effect. Policy compilation, grounded state checking, and executable remediation target hallucinated identifiers, ambiguous requests, and omitted policy side effects.
- Conclusion: AgentGuardUtil pairs an LLM student with a deterministic teacher that checks each draft before execution and revises it until no findings remain.The teacher evaluates observed state and the state the environment would reach after the draft’s own actions.
- Conclusion: The system blocks unobserved identifiers, requires confirmation for ambiguous requests, and returns forgotten policy side effects as executable actions.These checks directly address the three challenge classes described for the in-car setting.