PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM-Based Hierarchical Coordinated Control with Continuation-Aware Policy Learning

Changhong He, Jinda Gao, Xinkuan Liu, Le Zhang, Xizi Luo, Yu Mei

arXiv:2608.15041v1cs.AI

TL;DR

Coordinating interacting engineering units is difficult when strategies depend on heterogeneous context, complex interactions, and constrained physical actions. This paper uses an LLM for system-level coordination, task-specific controllers for executable actions, and Continuation-Aware GRPO to evaluate delayed effects, outperforming baselines across traffic control and VPP management.

  • Problem

    System-level coordination is difficult when appropriate strategies depend on temporal, spatial, uncertain, urgent, and externally driven information that fixed rules cannot readily encode.

  • Method

    An LLM maps heterogeneous operational context to coordination decisions, task-specific controllers execute constraint-aware actions, and Continuation-Aware GRPO evaluates subsequent closed-loop evolution.

  • Results

    The method consistently outperforms direct-control and hierarchical-coordination baselines across multi-ramp traffic control and VPP energy management, including 16625.4 ± 84.5 traffic throughput.

  • Takeaways & Limitations

    The framework remains effective without adaptation in simulators with different dynamics and modeling assumptions, while ablations support continuation-aware return construction.

  • Takeaways & Limitations

    The framework assumes executable actions satisfy operational constraints represented in the control model.

Abstract

from arXiv · show

Coordinating multiple interacting units in complex engineering systems is challenging when system interactions are difficult to model, operational information is heterogeneous, and low-level actions must satisfy strict constraints. We propose an LLM-based hierarchical framework in which the LLM coordinates interacting units based on heterogeneous operational context, while task-specific controllers or optimizers generate executable and constraint-aware actions. We further introduce Continuation-Aware GRPO to capture the consequences of coordination decisions over subsequent control intervals. Rather than judging a decision only by its immediate outcome, the method also evaluates how the system evolves afterward under the current policy. We validate the framework on multi-ramp traffic control and virtual power plant (VPP) energy management, using simplified system models for training and more realistic simulators for evaluation. Across both tasks, the proposed method consistently outperforms direct task-specific control and optimization, end-to-end reinforcement learning, rule-based and RL-based hierarchical coordination, and prompting-only LLM coordinators, demonstrating the value of heterogeneous-context reasoning, hierarchical execution, and continuation-aware policy learning.

Introduction

The introduction identifies the difficulty of coordinating interacting engineering units under heterogeneous information, coupled dynamics, and strict constraints. It proposes an LLM-based hierarchical framework with Continuation-Aware GRPO to separate coordination from executable control and address short-horizon bias.

  • Motivation: Interacting engineering systems require adaptive system-level coordination because local decisions are coupled through shared dynamics, resource constraints, and global objectives.Examples include transportation networks, energy systems, industrial processes, and distributed infrastructures.
  • Limitations of Existing Methods: Centralized optimization depends on accurate models and can become computationally expensive, while decentralized control often relies on fixed parameters or manually designed switching rules.These limitations make coordination difficult when appropriate strategies depend on trends, interactions, uncertainty, urgency, priorities, and external events.
  • Limitations of Existing Methods: Directly learning joint low-level actions faces rapidly growing action spaces, difficult constraint guarantees, and overfitting to training-environment dynamics.The introduction also notes related limitations for prompting-only LLM control.
  • Proposed Framework: The proposed hierarchy lets the LLM integrate heterogeneous operational context for system-level coordination, while task-specific controllers or optimizers produce precise, executable, constraint-aware actions.The LLM may determine relative aggressiveness or conservativeness across units and prioritize objectives; execution handles physical and operational constraints.
  • Continuation-Aware Learning: Continuation-Aware GRPO evaluates each coordination decision through immediate execution followed by longer-horizon evolution under a frozen copy of the current policy.The accumulated return captures immediate and subsequent outcomes without an additional value model or changes to the standard GRPO objective.
  • Evaluation: The framework is trained on simplified models and evaluated in more realistic environments across multi-ramp traffic control and virtual power plant energy management.These case studies differ in dynamics, information structure, temporal coupling, constraints, and low-level execution mechanisms, enabling evaluation across diverse operating patterns and simulator dynamics.

Related Work

Existing engineering control methods trade off efficiency, model-based coordination, and learning flexibility, while hierarchical and LLM-based approaches remain task-specific. Reinforcement fine-tuning has begun addressing stateful interaction, but engineering control still faces temporal effects across control intervals.

  • Feedback control is efficient but often uses fixed coordination logic, whereas centralized optimization captures couplings and constraints at the cost of accurate models, forecasts, and scalability.
  • Reinforcement learning avoids predefined rules, but direct low-level policies must jointly learn coordination, numerical actions, and constraint satisfaction.Hierarchical methods separate high-level decisions from domain-specific controllers or optimizers, but their representations, coordination variables, and execution interfaces remain task- and model-specific.
  • LLMs have been applied as action selectors, supervisors, and interfaces to numerical controllers, including traffic signal-phase selection from textual states.CoLLMLight additionally incorporates neighboring states and historical evolution, while other approaches connect LLMs to model predictive control, reinforcement learning, or optimization through high-level decisions, predictions, parameter adjustment, or formulation generation.
  • Existing reinforcement fine-tuning methods primarily evaluate complete outputs in static domains, while newer methods learn from trajectories, turn-level interactions, or simulator feedback.Engineering control introduces a temporal mismatch because decisions executed during one interval change physical states and may affect performance over subsequent intervals.

Problem Formulation

The formulation models a complex engineering system as N interacting control units whose high-level coordination decisions use physical and heterogeneous operational context. A task-specific low-level controller converts those decisions into executable actions satisfying operational constraints while the high-level policy is optimized for system performance.

  • System and constraints: The system contains N interacting control units, with physical state s_t capturing observable system variables and executable actions subject to operational constraints.Exogenous disturbances w_t include time-varying demand, renewable generation, weather, and market conditions.
  • Decision context: Coordination decisions depend on recent state history and heterogeneous information, including forecasts, uncertainty estimates, resource urgency, priorities, external events, and system rules.
  • Hierarchical execution: The high-level policy produces a structured joint coordination decision, which a task-specific low-level controller or optimizer maps to an executable physical action for each unit.
  • Hierarchical execution: The LLM parameterizes the high-level coordination policy, while the low-level controller remains responsible for physical execution under the system constraints.
  • Learning objective: The learning objective optimizes the high-level policy’s discounted system-level trajectory performance, with trajectories induced by coordination, low-level control, system dynamics, and exogenous disturbances.The discount factor satisfies γ ∈ (0, 1].

Method

The method combines structured-context LLM coordination with low-level controllers that execute constraint-aware actions, and trains the high-level policy using continuation-aware closed-loop returns. Continuation-Aware GRPO evaluates each initial decision over a longer trajectory while retaining group-relative policy updates.

  • Hierarchical coordination architecture: The heterogeneous context ξt is converted into a structured natural-language prompt describing system conditions, trends, uncertainty, urgency, priorities, external events, and applicable rules.
  • Hierarchical coordination architecture: The LLM generates a structured joint decision whose components map to task-specific controller parameters, including feedback thresholds, objective weights, or resource priorities.
  • Hierarchical coordination architecture: Low-level controllers compute executable actions using feedback control, model predictive control, or mathematical programming while explicitly handling physical and operational constraints.
  • Continuation-Aware GRPO: Continuation-Aware GRPO applies each sampled high-level decision for one interval of duration ∆, then continues the rollout to horizon H with H > ∆ under the same frozen policy.
  • Continuation-Aware GRPO: Matched group evaluation holds the physical state and future exogenous trajectory fixed across decisions, attributing return differences primarily to high-level coordination choices.
  • Continuation-Aware GRPO: The initial decision’s return captures both its immediate outcome and subsequent closed-loop evolution, without introducing a terminal-value model or auxiliary reward; when H = ∆, evaluation reduces to short rollouts.

Experiments

Experiments evaluate the framework on multi-ramp traffic control and virtual power plant energy management under seen, unseen, and higher-uncertainty scenarios, including frozen-policy transfer to more realistic simulators. The method achieves the strongest overall performance and robustness, while continuation-aware return construction improves both tasks, especially under high uncertainty.

  • Evaluation tasks: The framework is tested on two structurally distinct systems: multi-ramp traffic control and virtual power plant energy management.Traffic uses hierarchical ramp-mode coordination with low-level controllers, while the VPP aggregates photovoltaic generation, storage, electric vehicles, and HVAC.
  • Evaluation protocol: The evaluation uses 300 frozen-policy test scenarios per task, equally split across seen-pattern, unseen-pattern, and higher-uncertainty subsets without target-environment adaptation.Each task uses 150 training and 50 validation scenarios, with complete-scenario splits preventing trajectory leakage.
  • Overall results: 16625.4 ± 84.5 traffic throughput improves over Feedback Control by 6.62% and Hierarchical RL by 3.06%, while 32679.2±223.6 VPP operating cost is 4.37% below MPC-MILP and 2.49% below the comparison baseline.The method attains the highest mean throughput and lowest mean operating cost in every test subset.
  • Robustness: Traffic throughput degrades by 4.86% from seen-pattern to higher-uncertainty scenarios, versus 8.14%–11.87% for baselines; VPP operating cost rises by 4.94%, versus 8.52%–16.74%.The method leads in mean performance on unseen-pattern and higher-uncertainty scenarios.
  • Cross-simulator transfer: 3.82 ± 0.33% and 5.62 ± 0.54% are the smallest cross-simulator gaps for traffic and VPP, compared with 10.09 ± 1.07% and 9.23 ± 0.95% for the strongest fast-simulator alternatives.The method attains the highest mean SUMO throughput and lowest mean OpenDSS cost, indicating the most robust transfer across both simulator pairs.
  • Continuation-aware ablation: Continuation-aware GRPO yields a 1.88% traffic-throughput gain and a 2.69% VPP-cost reduction overall, with High-U gains of 2.84% and 3.81%, respectively.The comparison is against current-interval GRPO and supports evaluating consequences beyond the initial control interval.

Conclusion · Appendix

The paper proposes an LLM-based hierarchical coordination framework that maps heterogeneous operational context to joint modes while task-specific controllers or optimizers generate low-level actions. Continuation-Aware GRPO evaluates delayed coordination effects through subsequent closed-loop trajectories, and experiments cover traffic control and virtual power plant energy management.

  • Conclusion: The framework uses an LLM to map heterogeneous operational context to joint modes for interacting units.This establishes the LLM as the coordination layer within the hierarchy.
  • Conclusion: Task-specific controllers or optimizers generate the low-level actions executed by interacting units.The framework separates coordination decisions from executable action generation.
  • Conclusion: Continuation-Aware GRPO evaluates the delayed effects of coordination decisions through subsequent closed-loop trajectories.The method assesses consequences beyond the immediate decision outcome.
  • Conclusion: The proposed framework was evaluated on multi-ramp traffic control and virtual power plant energy management.These experiments span transportation control and energy-management applications.
  • Conclusion: The experiments used hierarchical coordination together with task-specific low-level control or optimization.This combines context-aware coordination with specialized action-generation mechanisms.
  • Conclusion: Across the evaluated applications, the method outperformed baseline approaches.The supplied conclusion states an overall advantage over baseline methods without specifying their individual identities or numerical margins.

A Task-Specific System Models and Low-Level Controllers … B.3 Baseline Implementations

The paper pairs task-specific system models with executable low-level controllers, then exposes their mode parameters to an LLM coordinator through structured operational prompts. Continuation-Aware GRPO evaluates decisions through closed-loop continuation, while baselines isolate the contributions of hierarchical coordination and learned policy selection.

  • A.1 Multi-Ramp Traffic Control: The corridor training model uses 27 directed links over 14.8 km, with eight of nine on-ramps controlled and mainline merge priority χ = 0.8.The remaining on-ramp is permanently open, and the corridor includes 14 mainline, 9 on-ramp, and 4 off-ramp segments.
  • A.1 Multi-Ramp Traffic Control: Traffic dynamics integrate at h = 5 s while the low-level gate updates every ∆low = 60 s, preserving nonzero delays for the corridor’s shortest 170 m link.At 60 s, links shorter than 1333 m would otherwise receive zero free-flow delay; the 5 s integration step avoids that distortion.
  • A.1 Multi-Ramp Traffic Control: Each controlled ramp uses an occupancy-feedback binary hysteresis gate, and inter-ramp spatial coordination is carried entirely by the high-level mode assignment.The low-level rule has no optimizer, queue override, or explicit inter-ramp coupling; the two-threshold band reduces chattering.
  • A.1 Multi-Ramp Traffic Control: The corridor coordinator selects one of four modes for each of eight ramps, yielding |Z| = 48 = 65,536 joint assignments, while Always-Open pins its gate open.Sensitive, Standard, and Sluggish select hysteresis bandwidths; calibrated centers differ across ramps and simulators, but mode semantics remain aligned.
  • A.2 Virtual Power Plant Energy Management: The VPP low-level layer re-solves a MILP every ∆low = 5 min over Hmpc = 60 min, enforcing physical resource and state limits exactly while penalizing operational-target slacks.The program uses rolling-horizon optimization; import, comfort, EV shortfall, and terminal-reserve targets may trade off through penalties while physical bounds remain hard.
  • A.2 Virtual Power Plant Energy Management: The VPP coordinator selects one mode per resource from {Conservative, Balanced, Proactive}^4, giving |Z| = 81, with each component modifying only its resource’s optimizer coefficients.Mode effects include PV de-rating, terminal SoC reserve, discharge limits, and objective weights; all other coefficients are shared.
  • B.1 Operational Context, Prompt Templates, and Output Schemas: At each decision step, heterogeneous operational context is serialized into a structured prompt containing task, observation, trend, forecast, and output-format blocks.Prompts explicitly define mode mechanics and may include domain guidance, while outputs use machine-checkable corridor vectors or VPP JSON objects validated before execution.
  • B.2 Continuation-Aware GRPO and Reward Construction: Continuation-Aware GRPO credits each initial decision for consequences generated by independently sampled continuation actions from the frozen behavior policy over H = 4∆.The reward uses realized task-level outcomes without auxiliary shaping for valid responses, and training uses a precomputed, bit-exact snapshot pool that is not refreshed as the policy improves.

B.4 Computational Cost and Training Overhead · C Simulators, Scenario Construction, and Evaluation Protocol · C.1 Dual-Fidelity Simulation Environments and Interface Correspondence

The framework separates affordable closed-loop training from higher-fidelity evaluation, while aligning simulator interfaces and enforcing executable constraints. Its computational overhead remains compatible with deployment intervals, but transfer is limited to aligned, data-informed simulator changes rather than zero-shot transfer.

  • B.4 Computational Cost and Training Overhead: Training uses four A100 80 GB GPUs; 1,200 steps issue roughly 3.1 × 10^5 generations, with task-specific prompt and response lengths.One snapshot requires 32 decisions, and an optimizer step over eight snapshots issues 256 generations.
  • B.4 Computational Cost and Training Overhead: Deployment decisions cost about 1.6 s for the corridor and 0.4 s for the VPP, remaining within their 3 min and 30 min control intervals.The coordination layer is therefore not on the critical path; prompting-only API baselines instead depend on network latency.
  • B.4 Computational Cost and Training Overhead: 3.1× wall-clock growth results when increasing the horizon from H = ∆ to H = 4∆, because batched generation amortizes while simulator advances remain serial per candidate.The added generations amortize across candidates, but simulator advancement does not.
  • C.1 Dual-Fidelity Simulation Environments and Interface Correspondence: Each task uses a fast simulator for affordable closed-loop training and a higher-fidelity simulator for reported evaluation, requiring consistent operational-context, joint-mode, and parameter interfaces.Many short rollouts make detailed simulation in the training loop impractical.
  • C.1 Dual-Fidelity Simulation Environments and Interface Correspondence: Traffic evaluation uses SUMO 1.21 on the Changchun Western Expressway, with detector observables from induction-loop and 100 m lane-area detectors sampled every 60 s.The traffic setup uses the Krauss car-following model and LC2013 lane-change model.
  • C.1 Dual-Fidelity Simulation Environments and Interface Correspondence: The corridor result demonstrates transfer across changed dynamics only after interface alignment, not zero-shot transfer, because calibration consumes SUMO data.The required information includes capacity scale, inflow scale, and eight hysteresis centres; the VPP pair needs no such step.
  • C.1 Dual-Fidelity Simulation Environments and Interface Correspondence: Traffic hysteresis centres are calibrated per simulator by quantile alignment, using SUMO as reference and mapping each ramp’s congested-period occupancy quantile to the LTM distribution.This accounts for occupancy being measured differently by macroscopic and microscopic simulators.
  • C.1 Dual-Fidelity Simulation Environments and Interface Correspondence: VPP evaluation uses the EPRI Ckt5 feeder in OpenDSS, and infeasible voltage or line-loading commands are curtailed to the largest feasible magnitude.A scalar bisection scales controllable injections toward zero and reaches one-percent precision in six to seven power-flow solutions.

C.2 Scenario Construction and Data Splits

Scenarios combine measured exogenous-series shapes with installation-specific amplitude mapping and structured operating-pattern families. Each task uses fixed train/validation/test splits spanning familiar, withheld-combination, and higher-uncertainty conditions.

  • Data provenance: VPP irradiance, outdoor temperature, park load, and EV-availability series use measured records that are resampled and rescaled without reshaping.Measured shapes are normalized and mapped to the modelled installation’s capacity range.
  • Operating-pattern families: Operating-pattern families define the binding-constraint regime while allowing quantitative realizations to vary, preventing trajectory memorization.For the corridor, families fix spatial loading regimes and are crossed with temporal profiles.
  • Corridor demand: Corridor demand varies by weak, medium, or strong origin levels and by temporal profiles with randomized rise onset and 20–40 min rise and decline durations.Profiles begin with a warm-up at 0.50 of amplitude, followed by rise, plateau, and decline phases.
  • Splits and test subsets: Each task uses 150 training, 50 validation, and 300 test scenarios, with the test set split equally into three disjoint subsets of 100.The subsets represent increasing departures from the training distribution, including familiar patterns, withheld descriptor combinations, and higher uncertainty.
  • Splits and test subsets: Higher-uncertainty scenarios increase multiplicative demand and PV noise from 5% to 15% and forecast mean absolute error from 8% to 20%.These changes use fixed factors rather than retuning the scenario-generation process.
  • Splits and test subsets: Splits are fixed at whole-scenario granularity, and descriptor-key deduplication prevents trajectories from crossing splits or test scenarios from duplicating training scenarios.For VPP scenarios, the key includes season, weather, load scale and peak, import-cap ratio, urgent share, and EV energy target.

C.3 Evaluation Protocol and Metrics · D Additional Quantitative Results

The evaluation defines corridor throughput and VPP operating cost as episode-level quantities aligned with the training objectives, using controlled warm-ups and frozen deterministic policies. Results aggregate equally across three 100-scenario subsets and report mean and standard deviation over five runs.

  • C.3 Evaluation Protocol and Metrics: Corridor throughput counts vehicles exiting during the 3-hour evaluation window, excluding a standardized 15-minute warm-up.All methods use the same all-Standard warm-up, producing bit-identical starting states and preventing unscored carry-over differences.
  • C.3 Evaluation Protocol and Metrics: Reported corridor values are vehicle counts per episode, with observed values of roughly 14,000–17,000 vehicles.Exit counting ensures each vehicle is counted once regardless of its distance traveled within the corridor.
  • C.3 Evaluation Protocol and Metrics: VPP operating cost equals actually purchased energy priced by tariff plus penalties actually incurred by the operator.The cost composition includes scenario-specific charges rather than applying an average rate to an average quantity.
  • C.3 Evaluation Protocol and Metrics: Both reported metrics accumulate the same quantities used for policy training over the full episode, avoiding proxy-objective gaps.The planner objective additionally uses battery wear and terminal-reserve hinge terms that shape planning but are not charged in reported cost.
  • C.3 Evaluation Protocol and Metrics: The policy is frozen on the target simulator, with only method-independent interface calibration performed using training and validation scenarios.LLM policies decode greedily at temperature zero, so run-to-run variation comes from environment stochasticity rather than sampling.
  • C.3 Evaluation Protocol and Metrics: MPC-MILP implies an average shortfall rate of 16.4 CNY/kWh versus 14.8 CNY/kWh for the proposed policy.The difference reflects shortfalls occurring in different urgent-share conditions; residual shortfalls for the proposed policy preferentially occur in low-ω scenarios.
  • C.3 Evaluation Protocol and Metrics: Each run averages 100 scenarios within each subset, then equally weights the three subset means; tables report mean and standard deviation across five run-level scores.Because all three subsets contain 100 scenarios, the equal-weight aggregate is numerically identical to pooling 300 scenarios.
  • C.3 Evaluation Protocol and Metrics: Scenario definitions, derived series, and split-building scripts are included in the code release, enabling exact reconstruction of the evaluation splits.The reported datasets and simulators are used under their stated open or source-publication licensing terms.

D.1 Additional Metrics Beyond the Reported Objective · D.2 Cross-Simulator Transfer for All Methods

Additional metrics expose whether gains arise from desirable system behavior or hidden constraint and fairness costs, while cross-simulator results show that non-learning controllers transfer better than most trained or prompted methods. Our policy achieves the strongest reported constraint-side outcomes and the smallest transfer gap.

  • D.1 Additional Metrics Beyond the Reported Objective: Overall scores are equal-weight means across three test subsets, averaged over five matched runs, and separate throughput or cost from their underlying tradeoffs.The additional metrics are reported for the same runs and aggregation as the main tables.
  • D.1 Additional Metrics Beyond the Reported Objective: Our policy attains the highest bottleneck speed, shortest congestion duration, and shortest ramp queue among the reported corridor outcomes.Congestion duration uses intervals below 40% of free-flow speed, while spillovers occur when ramp queues reach storage length.
  • D.1 Additional Metrics Beyond the Reported Objective: End-to-End RL improves throughput but has a 52% longer ramp queue and more than twice as many spillovers than Feedback Control.This indicates that its throughput gain buys mainline flow with ramp delay.
  • D.1 Additional Metrics Beyond the Reported Objective: Maximum uninterrupted ramp closure averages 22 min for our policy, versus 25 for Feedback Control, 26 for Fixed-Mode Hierarchy, 31 for Hierarchical RL, and 43 for End-to-End RL.This fairness metric is maximised over the eight ramps after averaging over runs.
  • D.1 Additional Metrics Beyond the Reported Objective: Our policy simultaneously has the fewest violation steps, least curtailment, least undelivered EV energy, and smallest comfort excursion in the virtual power plant.Violation steps concern commanded dispatch before curtailment, while comfort excursion integrates temperature deviation outside [22.0, 26.5] °C.
  • D.1 Additional Metrics Beyond the Reported Objective: Hierarchical RL beats MPC-MILP on cost but curtails more, indicating that its margin comes from operating closer to the constraint boundary.The passage links this behavior to Hierarchical RL’s larger transfer gap in Appendix D.2.
  • D.1 Additional Metrics Beyond the Reported Objective: LLM decision violations are schema-rejected outputs, and recovery retains the previous joint decision for one interval, costing a decision opportunity rather than producing invalid action.Validity is reported only for LLM policies.
  • D.2 Cross-Simulator Transfer for All Methods: Feedback Control has a 4.31% transfer gap and MPC-MILP 6.94%, the two smallest gaps after ours; every other trained or prompted method has a larger gap.The gap measures degradation from the fast simulator to the detailed simulator, using task-specific formulas.

D.3 Continuation-Horizon Sensitivity

Continuation-horizon sensitivity shows that most available benefit appears by 2∆, while 4∆captures the robust gains, especially under higher uncertainty. The sweep also motivates a compute-matched control because longer continuations increase both simulated intervals and generation calls.

  • Horizon sweep: The H ∈ {∆, 2∆, 4∆, 8∆} sweep shows that the curve saturates, with most available gain realised by 2∆.Each horizon was retrained from the same base policy while other Table 10 settings remained unchanged.
  • Horizon sweep: At 8∆, run-to-run spread widens, and its residual improvement is smaller than that spread, so it is not treated as a real difference from 4∆.The widening spread is consistent with returns absorbing disturbances the initial decision did not cause.
  • Uncertainty dependence: The higher-uncertainty subset has the largest gains, reaching 2.84% and 3.81% at 4∆.Under low uncertainty, the immediate outcome already approximates the eventual outcome; under high uncertainty, continuation can reveal decisions that must be reversed.
  • Compute control: Longer continuation multiplies simulated time and generation calls, so the sweep alone cannot distinguish return construction from computational budget.The sensitivity experiment therefore does not by itself establish whether gains arise from continuation-aware returns or extra computation.
  • Compute control: The control keeps H = ∆ while quadrupling snapshots per optimiser step, matching H = 4∆ in generation calls and simulated intervals.The two settings differ only in whether those intervals are used as continuation, according to the supplied passage.

D.4 Operational-Context Ablation … E.1 Mode-Selection Distributions

Operational-context ablations show that predictive and reliability information materially improves coordination, while bootstrap intervals support the main method comparisons without resolving prompting-only coordinator rankings. Mode-selection analyses further indicate broad, context-responsive behavior rather than fixed or arbitrary assignments.

  • D.4 Operational-Context Ablation: Removing any operational-context category reduces performance, with recent trends most valuable for the corridor and forecasts most valuable for the VPP.Recent-trend removal costs 1.91% of throughput, while forecast removal costs 1.65% of cost.
  • D.4 Operational-Context Ablation: The two predictive blocks rank first and second on both tasks, while uncertainty estimates rank third at 0.93% and 1.00%.Trends and forecasts swap order across tasks; uncertainty estimates cost 0.93% and 1.00%.
  • D.4 Operational-Context Ablation: Cumulative context stripping causes full losses of 3.86% of throughput and 4.09% of cost, roughly twice the largest single-category effect.The cumulative ablation removes categories down to current numerical measurements alone, confirming joint contributions beyond individual removals.
  • D.4 Operational-Context Ablation: With only current measurements, the stripped policy still beats direct control by 2.51% on throughput and 0.45% on cost, and beats Fixed-Mode Hierarchy by 1.05% and 0.76%.This supports independent contributions from hierarchy and training, since state-dependent mode selection outperforms static assignment even without richer context.
  • D.5 Uncertainty of the Reported Differences: All paired scenario-level bootstrap intervals exclude zero, with the narrowest margins over Hierarchical RL at [+408, +574] vehicles and [+688, +1002] CNY.The intervals compare methods on the same 300 scenarios and disturbance realisations; they do not resample training seeds.
  • E.1 Mode-Selection Distributions: Our policy selects modes broadly, with entropy 0.86 versus 0.66 for Qwen3-8B in the corridor and 0.95 versus 0.76 across VPP resources.Fixed-Mode Hierarchy is beaten on both tasks, so constant assignment does not explain the results; pooled entropy alone does not establish adaptivity.
  • E.1 Mode-Selection Distributions: Conditional mode shares respond to context: Sensitive reaches 44.2% on rising occupancy and 38.6% on falling versus 19.7% when flat, while proactive-battery share rises from 21.4% to 68.7% across VPP urgency tiers.Across occupancy-trend bins, Sensitive varies by 24.5 percentage points for the proposed policy versus 6.3 for Qwen3-8B; proactive battery varies by 47.3 versus 16.0 across tariff tiers.

E.2 A Representative Episode of Each Task

Representative episodes show the hierarchy adapting coordination roles across traffic phases and VPP tariff tiers. In traffic, assignments use spatial propagation and hysteresis, while VPP control prepares resources before a price spike.

  • Traffic control: Table 29 follows one balanced-heavy corridor scenario through five phases, with the mainline near capacity and three of eight ramps carrying strong demand.The scenario is selected to exercise coordination rather than to favor the policy.
  • Traffic control: During the rise, the policy assigns Sensitive to three upstream ramps near the failing merge rather than to the ramp with the longest queue.This reflects spatial-propagation reasoning.
  • Traffic control: During the plateau, the policy switches most ramps to Sluggish to prevent gate oscillation when occupancy is stable near the centres.The passage identifies this phase as where the hysteresis design pays.
  • VPP energy management: The VPP policy follows tariff tiers, keeping the battery conservative and EV balanced through the shoulder before switching both to proactive entering the critical-peak tier.This builds battery state of charge and residual EV energy before spending them during the price spike.

E.3 Where the Method Helps and Where It Does Not

The method helps most when coordination decisions materially affect shared capacity or resource allocation, but offers little advantage when the system is saturated or has little meaningful choice. Its benefits also depend on low-level flexibility and a low-dimensional, semantically labelled high-level decision.

  • Where the Method Helps: 8.5% and 9.8% are the largest corridor gains, occurring in ramp-dominant and balanced-heavy scenarios where ramps compete for downstream capacity.The choice of which ramp to tighten first has a lasting effect.
  • Where the Method Helps: 5 to 6% are the VPP gains across PV-, EV-, and grid-limited families, where different resources relieve each limiting pressure.The joint mode vector is central to these scenarios.
  • Where the Method Does Not: 0.3% is the gain for comfort-limited VPP scenarios, with a bootstrap interval of [−148, +361] CNY around +97 CNY that contains zero.The passage states that this interval supports no claim of improvement.
  • Where the Method Does Not: No detectable improvement occurs for HVAC power, where the MILP has almost no feasible slack and physics fixes the outcome; MPC-MILP is appropriate.The interval admits both a gain and a loss, so equivalence is not claimed either.
  • Design Limits: The advantage is largest when the low-level layer has genuine freedom, while a hierarchy over a saturated controller inherits that saturation.The approach also presumes a low-dimensional, semantically labelled delegated decision; the studied tasks have four roles per ramp and three modes per resource.
Loading 2608.15041v1…