LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systems

A. RL-Based EVs Charging Scheduling

• Existing RL-based charging studies typically optimize only one or two perspectives; a survey of 111 papers found none simultaneously integrated EV users, charging stations, and the grid.The introduction notes that optimizing EV satisfaction and station revenue alone may exacerbate peak grid load.Source: §I · pp. 1–2

I. INTRODUCTION

• The proposed LLM-enhanced MARL framework combines interpretable feature selection and adaptive balancing of stakeholder objectives to address complexity, interpretability, and adaptation challenges.LLMs score features from text prompts and dynamically assign weights to conflicting objectives using real-time state descriptions.Source: §I · p. 2

B. Feature Selection

• LLM-based feature selection uses semantic reasoning about feature relevance to provide interpretable justifications, addressing computational burden and limited interpretability in traditional methods.The paper extends this approach to transportation and power systems and integrates it with MARL for EV charging scheduling.Source: §I · p. 2, B. Feature Selection · p. 3

I. INTRODUCTION

• The integrated framework combines the LLM modules with MADDPG and is reported to outperform feature-selection-based and MORL-based baselines across user satisfaction, station profits, grid load pressure, and overall market efficiency.The introduction also reports reduced policy training cost, without specifying a magnitude in these passages.Source: §I · p. 2

C. Multi-Objective Reinforcement Learning

• The paper proposes semantic, state-responsive multi-objective balancing as an alternative to MORL approaches that face computational complexity, limited interpretability, and inflexibility.Its LLM-based module translates objectives and environmental states into semantic representations and provides natural-language justifications.Source: C. Multi-Objective Reinforcement… · p. 3

A. System Model

• The public charging scenario includes EVs, a station with photovoltaic panels and energy storage, and the grid, with grid purchases supplementing renewable generation to meet charging demand.The framework depicts station pricing, EV charging power decisions, and grid transmission power control.Source: A. System Model · p. 3, Figure 1 · p. 4
• An M/M/s queue models limited charging infrastructure with Markovian EV arrivals and service times across s charging piles.The EV arrival rate follows a homogeneous Poisson process, and service rate depends on available piles and charging power.Source: A. System Model · p. 3
• The queueing model defines occupancy rate ρ = λ/µ and a steady-state distribution Γ for the charging process.The passage identifies k as queue length and Γ(0) as the probability of an empty queue.Source: A. System Model · pp. 3–4
• The study focuses on charging operations and excludes vehicle-to-grid scenarios to maximize service availability for EV users.Source: A. System Model · p. 4

B. Problem Formulation

• The unified optimization framework covers EV users, the charging station, and the grid over a discretized horizon of T intervals.It formulates each stakeholder’s problem before presenting the unified framework.Source: B. Problem Formulation · p. 4

1) EV User Perspective:

• The EV-user objective minimizes charging cost, waiting-time penalties, and dissatisfaction with state of charge at departure.Source: 1) EV User Perspective · p. 4
• EV charging constraints represent state-of-charge evolution, battery boundaries, physical charging-power limits, and the arrival-to-departure charging window.The state-of-charge dynamics incorporate charging efficiency and battery capacity.Source: 1) EV User Perspective · p. 4

2) Charging Station Perspective:

• The charging station’s objective maximizes service profits while minimizing energy-storage degradation and station operating costs.Source: 2) Charging Station Perspective · p. 4
• Station operating costs include fixed and throughput-dependent components, while pricing boundaries are set to support commercial viability and market competitiveness.The degradation-cost model depends on battery cost, labor cost, cycle life, and storage capacity.Source: 2) Charging Station Perspective · p. 4

3) Grid Perspective:

• The grid objective minimizes load pressure and the station’s power-purchase cost, with a quadratic load-variation term encouraging smoother grid operation.The purchasing cost is included on the grid side because transmission power appears in grid constraints and directly affects grid load burden.Source: 3) Grid Perspective · p. 5
• Grid constraints limit instantaneous power draw, maintain supply-demand balance with photovoltaic generation and EV demand, enforce ramping limits, and bound station storage energy.Source: 3) Grid Perspective · p. 5

4) Unified Optimization Framework:

• The unified optimization problem balances the interests of EV users, the charging station, and the grid using stakeholder-specific objective weights.The paper identifies privacy restrictions on obtaining accurate user objectives and uncertainty in real-world model parameters as challenges for implementing the theoretical model.Source: 4) Unified Optimization Framework · p. 5

IV. MARKOV GAME MODEL

• The proposed LLM-enhanced MARL approach develops data-driven charging strategies to address privacy-limited user-objective formulation and uncertain real-world model parameters.The paper first formulates the scheduling problem as a Markov Game and analyzes its components.Source: §IV · p. 5

A. Markov Game Model Definition

• The charging market is modeled as a Markov Game with EVs, a charging station, and a grid operator as decision-making agents.Its state represents charging-market features, while the transition function describes state evolution given the current state and actions.Source: A. Markov Game Model Definition · p. 5
• The model’s state and action spaces include EV charging efficiency and power, station and grid metrics, and charging decisions by EVs.Station features include charging price, photovoltaic generation, demand, and current and predicted queues; grid features include base load and price.Source: A. Markov Game Model Definition · p. 5

• High-dimensional states and interdependent stakeholder rewards are the two primary challenges for efficient MARL policy development.Reward weights must adapt to changing demand, grid conditions, station utilization, and renewable-energy availability.Source: B. State and Reward Analysis · p. 6
• A station with 20 charging piles has 118 state dimensions, which the paper says increases computational complexity and slows MARL convergence.The state spans EV operations, station management, and grid conditions.Source: B. State and Reward Analysis · p. 6
• The approach proposes interpretable feature extraction to reduce state dimensionality and adaptive reward weighting to respond to time-varying conditions.The stated goal is to preserve critical decision-making information while making MARL implementation more efficient.Source: B. State and Reward Analysis · p. 6

V. OUR APPROACH

• The framework combines interpretable LLM-based feature selection, adaptive LLM-based objective balancing, and MADDPG to learn a charging-scheduling policy.The two LLM modules identify and rank relevant state features and adjust objective trade-offs based on the real-time environment.Source: §V · p. 6

A. LLM for Feature Selection

• Feature selection gives the LLM natural-language descriptions of each subproblem and candidate features, then scores feature relevance separately for each stakeholder objective.The LLM scores features from 0 to 1 and provides reasoning for each score; the example gives desired EV state of charge a score of 1.0 for its direct effect on energy demand.Source: A. LLM for Feature Selection · pp. 6–7
• The method ranks features by score and selects the top K for each subproblem, producing a concise state representation.Out-of-range scores trigger a repeat evaluation with an error-notifying prompt.Source: A. LLM for Feature Selection · p. 7
• The authors describe the selection process as using LLM domain knowledge to interpret feature significance, with prompting instead of complex statistical computation and extensive parameter tuning.The method also provides interpretable, semantic justifications for selected features.Source: A. LLM for Feature Selection · p. 7

B. LLM for Multi-Objective Balancing

• The balancing module assigns objective weights adaptively from the current environmental state, using descriptions of the objectives and real-time selected features.It gives weights on a 0-to-3 scale, requires them to sum to 3, and provides reasoning for the assignments.Source: B. LLM for Multi-Objective Balancing · p. 7
• The module is triggered every G training episodes when one stakeholder’s cumulative reward exceeds the combined rewards of the other two; otherwise, previous weights are retained.This imbalance-identification mechanism avoids asking the LLM to redetermine weights every episode.Source: B. LLM for Multi-Objective Balancing · p. 8
• The authors characterize the method as providing adaptive prioritization from real-time states, interpretable weight assignments, and implementation without complex multi-objective balancing mechanisms.Source: B. LLM for Multi-Objective Balancing · p. 8

C. LLM-Enhanced MARL Method

• Training uses centralized multi-agent policy learning with continuous and discrete actions, and the overall method separates policy training from execution.Agents update actor and critic networks based on weighted rewards.Source: 1) Training Phase · p. 8, C. LLM-Enhanced MARL Method · p. 8

1) Training Phase:

• The LLM selects the top-K features from prompted descriptions, then agents train a charging policy using those features and weighted rewards.The algorithm uses LLM-derived feature scores and explanations for selection, and periodically obtains objective weights when the system is imbalanced.Source: 1) Training Phase · p. 8
• LLM-MOB is triggered every G episodes when system status is imbalanced; agents use newly generated weights then and previously determined weights otherwise.The imbalance condition and multi-objective prompt guide weight generation.Source: 1) Training Phase · p. 8
• MADDPG uses actor and critic networks per agent, with the critic evaluating global state and joint actions while actors map selected features to actions.Exploration adds Gaussian noise with episode-dependent variance; target actor and critic networks are slowly updated.Source: 1) Training Phase · pp. 8–9

2) Execution Phase:

• Each agent generates actions from local observations using its trained actor network, without exploration noise, enabling decentralized execution.The paper describes this arrangement as preserving centralized-training benefits while supporting scalable, responsive real-time decision-making.Source: 2) Execution Phase · p. 9
• The complete process has agents interact with the environment during training, then use trained actor networks for decentralized decisions in execution.The LLM-MOB module adjusts objective weights during training when the system is identified as imbalanced.Source: 2) Execution Phase · p. 9

1) Environmental Settings:

• Hourly PJM electricity prices and solar data from April 2023 to March 2024 support training on each month’s first 20 days and evaluation on its remaining days.The simulation uses specifications from the 10 best-selling 2023 U.S. EVs and Poisson arrivals, averaging over 200 EVs daily.Source: 1) Environmental Settings · p. 9

2) Algorithm Benchmark:

• Benchmarks include five feature-selection methods paired with MADDPG and/or LLM-MOB, three MORL methods with full or LLM-selected features, and vanilla MADDPG.Feature-selection methods are LassoNet, RFE, MI, mRMR, and Transformer; MORL methods are CAPQL, GPI-PD, and PGMORL.Source: 2) Algorithm Benchmark · p. 9

3) Algorithm Settings:

• GPT-4o supplies both LLM modules, and each training and execution episode has 24 time steps representing one day.RL parameters are standardized across benchmarks.Source: 3) Algorithm Settings · p. 9

4) Evaluation Metrics:

• Execution metrics cover station profit, EV dissatisfaction, queue length, charging cost, grid load pressure, and overall charging-market efficiency.The efficiency index combines station and grid perspectives with equal coefficients and gives each of three EV-user metrics a coefficient of one third.Source: 4) Evaluation Metrics · pp. 9–10
• The charging market efficiency index uses weights of 1 for station and grid perspectives and 1/3 for each EV-user perspective.The index is presented as a comprehensive metric based on the five evaluation metrics.Source: 4) Evaluation Metrics · p. 10

B. Comparison with Feature Selection-Based Approaches

• 642 episodes to convergence compares with over 1,200 for MI+MARL and vanilla MARL; training takes 3.85 seconds per episode and 41.16 minutes total.The paper attributes the training-cost advantage to selecting features once before training rather than extracting them each episode.Source: B. Comparison with Feature… · p. 10, Table IV · p. 11

C. Comparison with MORL-Based Approaches

• Across MORL methods, the framework shows more stable, higher episodic rewards and a more diverse Pareto front balancing EV, grid, and station rewards.CAPQL stabilizes around 700 reward units, PGMORL has high variance, and GPI-PD struggles to exceed 610.Source: Figure 6 · p. 10, C. Comparison with MORL-Based Approaches · pp. 10–11, Figure 7 · p. 11

B. Comparison with Feature Selection-Based Approaches

• 2.195 ± 0.192 CMEI is the highest among feature-selection approaches, alongside superior AQL (2.958 ± 1.241), ECC (104.2 ± 35.1), and LPG (0.411 ± 0.021).MI+LLM-MOB and Transformer+MARL are marginally better on DEG and DPS, respectively.Source: B. Comparison with Feature… · p. 10, Table III · p. 11

D. Interpretability Analysis

• The interpretability examples show the LLM ranking five EV-perspective features and assigning the EV perspective a weight of 1.5 when 18 EVs are charging, several near departure.The feature-selection rationale refers to physical significance and relevance to EV charging optimization; the weight example considers driver sensitivity to charging duration and cost.Source: D. Interpretability Analysis · pp. 11–12, Figure 8 · p. 12

C. Comparison with MORL-Based Approaches

• 2.195 ± 0.192 CMEI is highest among the MORL comparisons, exceeding next-best CAPQL+LLM-FS by 9.1%.Some baselines lead on individual metrics, but the paper reports that they do not optimize all objectives simultaneously.Source: C. Comparison with MORL-Based Approaches · p. 11, Table V · p. 12
• Adding LLM-FS to MORL approaches reduces per-episode and total training time, while MORL baselines require more episodes and longer training than the proposed approach.Table VI summarizes episodes to convergence, seconds per episode, and total training time.Source: C. Comparison with MORL-Based Approaches · p. 11, Table VI · p. 12

E. Extension to Larger Scenarios

• For multi-station scalability tests, EVs choose stations by price, queue length, and distance on a randomized 100 × 100 grid, with β2 and β3 sampled from specified intervals.β1 is fixed at 1; β2 is sampled from [0, 0.4] and β3 from [0, 0.04].Source: E. Extension to Larger Scenarios · p. 12
• CMEI > 1.9 across tested scales; adding EVs with fixed stations worsens DEG and AQL, while adding stations improves DEG and AQL but increases DPS and ECC.The reported patterns reflect intensified competition for limited resources as EV numbers rise and increased charging-resource availability as station capacity expands.Source: E. Extension to Larger Scenarios · p. 12, Table VII · p. 13

1) Component Analysis:

• The complete model leads on five of six metrics; removing LLM-FS raises LPG from 0.411 to 6.428 and lowers CMEI from 2.195 to 1.573.Without LLM-MOB, DPS is higher, but AQL and ECC suffer.Source: 1) Component Analysis · p. 13, Table VIII · p. 13

2) Performance with different LLMs:

• Across tested LLMs, CMEI ranges from 1.82 to 2.22; GPT-4o leads DEG (0.037) and AQL (2.958), o3-mini DPS (279.4) and CMEI (2.215), Gemini-1.5 ECC (90.3), and LLaMA3-70B LPG (0.410).The authors report robust overall performance across LLM implementations despite metric-specific variation.Source: 2) Performance with different LLMs · p. 13, Table IX · p. 13

VII. CONCLUSION

• The conclusion attributes unified EV-station-grid scheduling to LLM-FS feature extraction and LLM-MOB adaptive objective balancing, reporting interpretable natural-language explanations and better performance across multiple metrics.The reported comparison is against state-of-the-art feature-selection-based and multi-objective-optimization-based approaches.Source: §VII · p. 13