PaperlayerView exact revision on arXiv

Source-linked AI summary

LAST-V2X: Language-Driven Adaptive Scenario Synthesis and Testing for Communication-Induced Failures in V2X Safety Applications

Keshu Wu, Hao Zhang, Rui Gan, Pei Li, Jiwan Jiang, Yang Zhou

arXiv:2610.30979v1cs.ROcs.AI

TL;DR

Testing V2X safety failures is difficult because traffic, communication conditions, and application responses must be coordinated, and observed failures must be attributed to communication rather than traffic. LAST-V2X translates natural-language intents into executable scenarios, tests them with paired information conditions, and adaptively refines ineffective configurations. Experiments across CARLA environments and V2X applications report reliable scenario generation, reduced spurious attribution, and improved failure discovery through refinement.

  • Problem

    V2X safety tests must jointly configure traffic interactions, communication conditions, and application responses, while determining whether communication rather than traffic produced a failure.

    Source: §1 · p. 2

  • Method

    LAST-V2X translates natural-language intents into grounded scenarios, compares matched degraded-information and ground-truth executions, and refines tests through parameter search and LLM self-reflection.

    Source: §1 · p. 3, §2.4 · p. 8

  • Results

    Across CARLA environments and V2X applications, LAST-V2X reliably generated executable scenarios, avoided spurious failure attribution under healthy communication, and improved communication-induced failure discovery through adaptive refinement.

    Source: §4 · p. 16

  • Takeaways & Limitations

    Paired execution distinguishes information discrepancies from failures that actually affect safety outcomes, while simulation-guided refinement improves failure discovery in the evaluated settings.

    Source: §4 · p. 16

    Communication parameters are representative rather than deployment-measured, and large-scale adaptive search was tested with a simplified backend rather than fully integrated simulation.

    Source: §4 · p. 16

Abstract

from arXiv · show

Vehicle-to-everything (V2X) communication enables connected vehicles to make safety-critical decisions, but delayed, missing, or incorrect information can compromise driving safety. Systematically testing such failures is challenging because traffic interactions, communication conditions, and application responses must be jointly configured to produce meaningful scenarios. This paper presents LAST-V2X, a language-driven, adaptive framework for automated scenario synthesis and safety testing of V2X applications. LAST-V2X leverages large language models (LLMs) to translate natural-language test descriptions into executable scenarios grounded in roadway geometry, V2X communication semantics, and vehicle behavior. The framework integrates physical traffic simulation with spatially aware communication modeling and message-level fault injection to evaluate how information degradation affects vehicle decisions. To establish whether an observed safety failure is caused by communication rather than the underlying traffic conditions, LAST-V2X employs matched executions under degraded and ground-truth information. Furthermore, an execution-guided refinement mechanism combines parameter search and LLM self-reflection to identify ineffective test configurations, improve scenario effectiveness, and produce minimal reproducible failure cases. Experiments across diverse CARLA environments and V2X safety applications demonstrate the framework's ability to generate reliable scenarios, distinguish communication-induced failures from unrelated safety violations, and substantially improve failure discovery through adaptive refinement. These results establish LAST-V2X as a systematic approach to language-driven, causally verified, and adaptive safety testing for connected vehicle systems.

1 Introduction

V2X safety testing must coordinate traffic, communication, and application behavior, then verify that information faults—not underlying traffic—produce unsafe outcomes. LAST-V2X uses language-driven scenario generation, paired execution, and adaptive refinement to address this challenge.

1 Introduction

  • LLMs can translate engineers’ high-level safety intents into executable V2X tests by connecting application needs with transportation, communication, and simulation knowledge.This can reduce scenario-engineering and simulator-programming burden.Source: §1 · p. 2
  • V2X failures arise from mismatches between the physical scene and the information available to the vehicle, not simply from unsafe physical traffic.Examples include stale green SPaT, missing BSMs, or MAP messages associating the ego with the wrong movement.Source: §1 · p. 2
  • The workflow verifies intent, grounds it on a feasible junction, runs faulted-message and ground-truth egos in CARLA, and admits only message-driven failures.Otherwise, the scenario is refined.Source: Figure 1 · p. 3
  • Existing approaches leave scenario construction, communication simulation, refinement, and causal validation disconnected, motivating an integrated framework for communication-induced failures.A collision after fault injection alone cannot establish that communication caused it.Source: §1 · pp. 2–3
  • LAST-V2X converts natural-language intents into grounded scenarios, runs matched degraded and non-degraded executions, and refines ineffective tests using execution evidence.Parameter search and LLM self-reflection address quantitative and semantic deficiencies; successful failures are reduced to minimal reproducible cases.Source: §1 · p. 3

2 Methodology

LAST-V2X turns natural-language prompts into executable V2X scenarios and uses paired causal testing with execution-guided search, reflection, and minimization to identify and reproduce communication-induced safety failures.

2.1 Problem Description

  • LAST-V2X admits a communication-induced failure only when the message-driven ego fails while its matched ground-truth counterpart stays safe, then seeks minimal witnesses across applications and test conditions.The framework distinguishes stale-but-detectable, fresh-but-wrong, and silent-by-absence information failures; the final witness retains only the smallest fault configuration needed to reproduce the failure.Source: §2.1 · pp. 4–5

2.2 Module I: LLM Agent Analysis

  • An LLM parses and repairs schema-constrained specifications, while a deterministic verifier checks whether requested communication faults can meaningfully affect the target application before simulation.Temporal faults are checked against the application's information-freshness budget; semantic faults are checked for consumption, decision-window overlap, and the possibility of corrupting reconstructed information.Source: §2.2 · pp. 5–6

2.3 Module II: Retrieval-Augmented Scenario Grounding

  • Retrieval grounds specifications in signal-semantic road maps, V2X knowledge, and parameterized driver behavior, then validates lane assignments and assembles typed programs for a fixed runtime.The LLM may map semantic roles to retrieved approaches but cannot invent roadway geometry; the resulting program is not LLM-generated simulator source code.Source: §2.3 · pp. 6–7

2.4 Module III: Automated V2X Scenario Synthesis

  • The simulator synchronously executes traffic, signals, V2X messaging, reconstructed information, and application decisions, with communication quality varying by vehicle position and urban blockage.Protocol-level faults alter delivery, timestamps, or message content, and a shadow ground-truth ego shares the message-driven ego's scene and conditions while receiving ground-truth information.Source: §2.4 · pp. 7–8

2.5 Module IV: Scenario Refinement

  • Scenario Refinement combines bounded specification-space search with prompt-space reflection, using execution evidence to diagnose divergence and improve ineffective tests.Hazard slack links information divergence to the next application decision, while inner-loop mutations adjust fault properties, vehicle conditions, and V2X penetration without changing high-level intent.Source: §2.5 · p. 8
  • Feasible mutants are executed and ranked by failure margin, with candidates closer to failure more likely to produce descendants; near misses guide search without replacing causal admission.Source: §2.5 · p. 9
  • Admitted scenarios are cataloged by application, fault family, V2X penetration, and conflict geometry to discourage repeated discovery of equivalent failures.Source: §2.5 · p. 9
  • Delta debugging weakens faults one at a time and retains changes that preserve admission, yielding a replayable 1-minimal scenario with no single fault removable or weakenable without losing the failure.It tries removing a fault, halving its magnitude, or shortening its window, stopping when no single weakening preserves admission under the paired causal test.Source: §2.5 · p. 9
  • Reflection uses paired outcomes, divergence episodes, hazard slack, decision-epoch AoI, failure margins, and verifier diagnostics to construct an evidence digest for prompt revision.Source: §2.5 · p. 9
  • Prompt reflection diagnoses why the intended mechanism failed to affect the application and revises the description, for example by aligning a fault window with the decision or changing geometry to enable communication exposure.A fault such as a SPaT freeze activated after the safety-critical decision cannot be made causal simply by increasing its magnitude.Source: §2.5 · p. 9
  • Every reflected prompt re-enters parsing, analytic verification, grounding, and execution under the same causal admission criterion, so reflection cannot directly manipulate simulator state or force a violation.Source: §2.5 · p. 9

3 Experimental Results

The experiments assess language-driven scenario generation, causal attribution of communication faults, adaptive refinement, and component contributions. They find reliable broad scenario generation, measurable benefits from communication cooperation, and more admitted failures after refinement.

3.1 Experiment Setup

  • The evaluation tests language-driven generation, causal verification, and adaptive refinement through a 100-scene benchmark, communication-fault suite, 50-scene refinement study, and ablations.The normal-traffic prompts leave the application and town to the pipeline to infer or assign, and cover all six applications and eight CARLA towns.Source: §3 · p. 9, §3.1 · p. 10

3.2 Scenario Generation at Scale

  • The 100-scene normal-traffic benchmark spans six V2X applications and eight CARLA towns; all scenes compile and execute, with zero admitted communication-induced failures under healthy communication.All 100 scenes compile within at most two parse attempts, and 253 of 254 vehicles remain on route.Source: Figure 2 · p. 10, §3.2 · pp. 11–12

3.3 Corridor Traversal along an RSU Chain

  • At 50 m radio range on an empty corridor, the RSU relay reduces GLOSA stops from 1.9 to 1.0 versus no V2X, while direct SPaT changes little.The relay gives knowledge of 2.6 junctions ahead versus 0.5 with direct coverage; travel time remains signal-bound, so the benefit is smoother control rather than higher speed.Source: §3.3 · pp. 12–13, Figure 4 · p. 12

3.4 Cooperative Scenarios with Multiple Connected Vehicles

  • In the EEBL relay chain, V2V rebroadcast yields no collisions in 10 seeds, while 7 of 10 runs collide without the relay and 9 of 10 with CV2 unequipped.In IMA mutual yield, both-equipped vehicles achieve a 12.7 m closest approach and 2.04 s post-encroachment time; with one or neither equipped, contact occurs in 2 of 10 seeds.Source: Figure 5 · p. 13, §3.4 · p. 14

3.5 Scenario Refinement at Scale

  • Across the 50-scene fault suite, refinement raises admitted failures from 7 to 32, while search and LLM self-reflection address different ineffective-test cases.Search finds failures in 23 of 43 refinement targets; self-reflection rescues 8, including two beyond the bounded search space. Delta debugging reduces admitted cases to 1-minimal witnesses with a single fault.Source: §3.5 · p. 14, Figure 6 · p. 14

3.6 Ablation and Sensitivity Studies

  • The paired-rollout ablation identifies one communication-induced failure among 112 live scenes with divergent estimated states, compared with 60 alarms from divergence-only monitoring.The ablation also finds building blockage dominant in channel sensitivity: reducing blockage loss from 5 to 1 dB/m raises in-range road coverage from 47.7% to 95.8%.Source: §3.6 · pp. 15–16
  • Without the feasibility gate, IMA admissions fall from 29.0 to 15.3 per 150-execution ablation budget, while RLVW is largely unaffected because its mutations remain feasible.The scene planner matters most at multi-approach intersections; prompt reflection rescues eight prompts, including two beyond parameter search.Source: §3.6 · pp. 15–16, Table 2 · p. 16

4 Conclusion

LAST-V2X combines language-driven scenario synthesis, causal verification, and execution-guided refinement to test V2X safety applications. Experiments report reliable scenario generation and improved discovery of communication-induced failures, alongside limitations in realism and evaluation scope.

4 Conclusion

  • LAST-V2X integrates natural-language scenario synthesis, paired execution for causal verification, and execution-guided parameter search and LLM reflection to refine ineffective tests.Successful failures are reduced to minimal reproducible cases.Source: §4 · p. 16
  • Experiments across CARLA environments and V2X applications report reliable scenario generation, avoidance of spurious attribution under healthy communication, and improved failure discovery through adaptive refinement.The paper also reports that information availability, communication coverage, and vehicle cooperation influence safety and driving behavior.Source: §4 · p. 16
  • The evaluation uses representative communication and fault parameters, simplified deterministic application logic, and a simplified backend for large-scale adaptive search.The effectiveness of adaptive search in fully integrated simulation remains to be established; the framework also assumes fixed-time signals and does not distinguish communication errors from adaptive signal-timing changes.Source: §4 · p. 16
  • Future work targets field-measured communication, more advanced application logic, fully integrated CARLA search, complex traffic, actuated signals, and broader uncertainties.The paper also proposes using generated failure evidence to inform communication infrastructure design and V2X safety improvements.Source: §4 · p. 17

Appendix A Verifier Check Families

The analytic verifier screens candidate specifications before simulation using specification fields and closed-form AoI relations. Findings block, warn about attribution or informativeness, or record scenario properties.

Appendix A Verifier Check Families

  • Before simulation, the verifier checks candidate specifications using specification fields and closed-form AoI relations, returning the check, severity, responsible fields, and a minimal repair.Repairs can include a breaching drop count or a minimum episode duration.Source: Appendix A · p. 21
  • Error findings block execution and trigger parser revision or gate a search mutant, while warnings flag low-attribution or uninformative scenarios and notes record scenario properties.A warning example is a pristine channel already exceeding an age budget; a note example is a deliberate silent-miss probe with unequipped conflict agents.Source: Appendix A · p. 21
  • The verifier defines eleven check families; conflict timing uses an 8 s arrival-overlap window, and penetration warnings trigger below a rate of 0.2.Table 3 groups the check families by severity.Source: Appendix A · p. 21, Table 3 · p. 21

Appendix B Channel Model Details

The spatial channel models each V2I and V2V message using transmitter–receiver distance and signal strength, with blockage, latency, and delivery determining message receipt. Messages beyond the communication range receive the noise floor.

Appendix B Channel Model Details

  • Each V2I and V2V message passes independently through a spatial channel that evaluates received signal strength for the transmitter–receiver pair.Links beyond range R receive the noise floor, and Table 4 lists default channel parameters with corridor-study range values of 50, 100, and 150 m.Source: Appendix B · p. 21, Table 4 · p. 22
  • Building blockage loss accrues only where a building rises above the line of sight, increases with its height above that line until saturation, and is capped at BNLOS.The ray height falls linearly between the RSU and receiver antenna heights; the cap represents residual energy reaching the receiver despite deep obstruction.Source: Appendix B · p. 21
  • Latency grows linearly below an RSSI knee, while delivery probability is logistic in RSSI; a message is lost at the noise floor or when a Bernoulli delivery draw fails.Otherwise, it is delivered after latency L, and semantic faults act on top of the channel.Source: Appendix B · p. 22

Appendix C Fault Operators

Fault operators specify the message stream and field to corrupt, the activation time window, and the magnitude. Their timing can be anchored to scenario, signal, or ego events.

Appendix C Fault Operators

  • Each typed fault specifies the target stream and field, activation timing, and corruption magnitude, with optional sender restriction and recorded magnitude provenance.Activation windows are anchored to scenario, signal-phase, or ego events; unset duration keeps a fault active through the scenario.Source: Appendix C · p. 22
  • A freeze captures the payload when its window opens, so a feed intended to remain green must be anchored before amber onset.A window beginning at red onset captures yellow instead and typically stops the scenario safely.Source: Appendix C · p. 22

Appendix D Safety Application Logic

Appendix D specifies shared braking thresholds and common violation adjudication, alongside the decision rules for six safety applications and criteria for identifying communication-relevant divergences.

Appendix D Safety Application Logic

  • Both rollouts use the same application function; signal decisions occur at the last point where stopping remains comfortable, using a 3.5 m/s² deceleration and 1.0 s reaction time.The decision epoch satisfies d ≤ ds(v), where d is distance to the stop bar.Source: Appendix D · p. 22
  • Violations are adjudicated identically across rollouts: red entry requires crossing under the ego group’s true red signal, while a collision requires close center-to-center distance with a vehicle moving above 0.5 m/s.For FCW and EEBL, only vehicles ahead count and the ego must be the striking vehicle; CARLA conflicts exclude traffic more than 2.5 m from the ego route.Source: Appendix D · pp. 22–23
  • Table 6 lists decision rules and violation criteria for six safety applications, while Table 5 specifies fault operators, target streams, and explored parameter ranges.The divergence monitor flags signal-phase errors or countdown errors above 1.0 s, displaced neighbor tracks beyond the age-adjusted position budget, and nonexistent tracks after 2.0 s.Source: Table 6 · p. 23, Table 5 · p. 23, Appendix D · p. 23

Appendix E Failure Margin and Search Settings

The failure margin quantifies how close an executed scenario came to failing, with distinct definitions for signal and conflict applications. Search prioritizes near misses while preserving test intent through bounded mutations and verification.

Appendix E Failure Margin and Search Settings

  • For signal applications, the failure margin is the smallest non-negative hazard slack across divergence episodes; negative slack means a decision occurred while the belief was wrong.A small positive slack means the wrong belief ended just before a decision.Source: Appendix E · p. 23
  • For conflict applications, the margin is the minimum distance between the message-driven ego and an agent over the run, counting only vehicles ahead for FCW and EEBL.If no qualifying episode or conflict occurs, the margin is set to a large constant; runs with only negative-slack episodes also retain that constant.Source: Appendix E · p. 23
  • Specification-space search keeps an elite set of eight scenarios, samples parents by priority, and tests up to 120 candidates per scenario, including gated mutants.Mutations preserve the application, ego maneuver, agent roster, and map query; each mutant is verified before execution, and admitted scenarios are minimized by delta debugging.Source: Appendix E · p. 24, Table 7 · p. 24
  • Search uses a fixed random seed on the stub backend and permits up to five LLM-reflection rounds per scenario.For conflict applications, operator weights favor interaction timing; for signal applications, they favor fault operators.Source: Appendix E · p. 24

Appendix F LLM Prompts

The appendix documents the three LLM roles’ prompt and backend configuration, including fixed sampling and shared domain notes for reflection.

Appendix F LLM Prompts

  • LAST-V2X uses three LLM roles with GPT-4o at temperature 0.2 and a fixed sampling seed; each role returns a validated JSON object.The reflection role receives identical V2X knowledge-base notes for every scenario, rather than notes tuned to individual prompts.Source: Appendix F · p. 24

Appendix G Reproducibility

The reproducibility setup fixes scenario-level randomness for deterministic replay and archives each admitted failure with its minimal witness and supporting execution artifacts. Experiments use synchronous live CARLA except search runs, while headline mechanisms are validated in live CARLA.

Appendix G Reproducibility

  • All runs are deterministic given a scenario seed, which fixes equipage, bus jitter, fault stochasticity, RSU selection, vehicle blueprints, and retrieval tie-breaking.Admitted failures are stored with their minimal-witness specifications and re-verified against the paired admission test on replay.Source: Appendix G · p. 24
  • Table 8 indicates that experiments use GPT-4o and synchronous live CARLA, except search runs on the stub backend; headline mechanisms were validated in live CARLA.The archive includes compiled specifications, scene plans, grounding records, execution traces, monitor reports, camera streams, and communication maps.Source: Table 8 · p. 24, Appendix G · p. 24
  • Figure 8 presents condensed system prompts for the three LAST-V2X LLM roles.The figure caption identifies the prompt material but does not describe its contents.Source: Figure 8 · p. 25

Related papers

Loading 2610.30979v1…