Source-linked AI summary
Collective Autonomous Vehicle Routing Needs Social Awareness for Coexistence with Human Drivers
Anastasia Psarou, Mikolaj Rams, Blazej Torbus, Rafal Kucharski
TL;DR
In mixed-autonomy traffic, selfish AV routing can increase travel times for human drivers, raising the question of how to reduce that harm while retaining AV benefits. The paper adds a counterfactual marginal-cost component to MARL routing rewards and tests applying it to selected impactful AVs. The reported results show improved travel times with social rewards and comparable outcomes from selective rewards with fewer simulations.
Problem
Selfish AV route choices can increase travel times for other road users, while prior work has given little attention to objectives that mitigate these externalities in mixed-autonomy traffic.
Source: §1 · p. 1, §1.1 · pp. 2–3Method
The authors use MARL routing rewards that combine an AV’s travel time with marginal cost, measured by comparing system travel times with the AV present and removed.
Source: §3.1 · p. 4, §5 · p. 8Results
40% of the most impactful AVs receiving marginal-cost rewards achieved AV, HDV, and system travel times comparable to rewarding all AVs, while reducing additional simulation runs by 40%.
Source: §4 · p. 5Takeaways & Limitations
The findings show that AV routing policies can affect how travel-time costs are distributed, supporting attention to AV impact on shared transportation systems.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Collective Autonomous Vehicle (AV) route choices in real-world traffic networks can create externalities for Human-Driven Vehicles (HDVs), as routes that benefit AVs may increase travel times for other road users. To address this, we introduce a social component, the marginal cost, to the selfish travel time AV reward. Marginal cost captures an AV's impact on other road users by comparing their travel times with those in a counterfactual simulation in which the AV is removed. We study AV route choice in mixed-autonomy traffic settings using Multi-Agent Reinforcement Learning (MARL), where AVs learn routing policies through repeated interactions with the traffic environment. Experiments on the real-world networks of Ingolstadt and Provins show that incorporating marginal cost into the AV reward not only limits the increase in HDV travel times and shortens AV travel times relative to selfish AV routing, but also improves overall system performance. In Ingolstadt, selfish routing of 40\% AVs increases the travel times of other vehicles by 1.7\%, and this drops to 1.1\% when the social reward is included. Interestingly, not only does overall system performance improve, but the AVs are also better off when acting socially.This effect is consistent across tested cities, penetration rates, and replications, yet when the weight of the social component in the reward increases, the system fails to reach the desired performance. For a more scalable approach to social AV routing in mixed-autonomy traffic, we compute the marginal cost only for the AVs whose route choices most affect the traffic system. This reduces the number of counterfactual simulations and achieves similar performance at a significantly lower computational cost. Our results suggest that AV routing should go beyond selfish rewards and, for the benefit of all traffic participants, include a social component in its objective.
1 Introduction
Selfish AV routing can raise human drivers’ travel times in shared urban networks, motivating rewards that account for each AV’s system impact. The paper introduces marginal-cost and selective marginal-cost rewards, addressing a research gap in reducing these externalities while retaining AV benefits.
1 Introduction
- Selfish AV policies that minimize individual travel times can increase HDV travel times relative to those before AVs enter the system.This is presented as a negative externality of selfish routing.Source: §1 · pp. 1–2
- Marginal-cost rewards can mitigate HDV travel-time increases while shortening AV travel times relative to selfish routing; higher weights may worsen AV and overall system performance.The paper also reports that greater marginal-cost weight can further improve HDV travel times.Source: §1 · pp. 1–2
- Selective marginal-cost reward shaping applies the reward only to the most impactful AVs to reduce counterfactual simulations while seeking comparable travel-time improvements.The paper identifies impactful AVs by how much alternative route choices change total system travel time.Source: §1 · p. 2, §1.1 · p. 3
1.1 Literature review
- Route choice is framed as a congestion game; the System Optimum minimizes total system travel time, potentially requiring some travelers to accept less favorable routes.The literature review contrasts this system-level objective with User Equilibrium, where no traveler can improve travel time by unilaterally switching routes.Source: §1.1 · p. 2
- The paper defines an AV’s marginal cost by comparing all vehicles’ travel times when that AV is present with a counterfactual in which it is removed.The approach follows a prior formulation and extends investigation to real-world networks beyond the previously studied toy network.Source: §1.1 · p. 3
- The stated research gap is limited attention to AV objectives that reduce AV travel times while mitigating effects on other road users in mixed-autonomy traffic.Prior difference-reward work is described as primarily focused on cooperative settings, unlike this use of counterfactuals for cross-population externalities.Source: §1.1 · p. 3
2 Preliminaries
The study simulates repeated urban trips with fixed routes and demand, first stabilizing human route choices before AVs learn policies. It formalizes daily route selection as a sequential multi-agent environment in which AVs learn from observations and rewards.
2.1 Urban routing problem
- The urban routing problem gives each vehicle one daily trip between an origin and destination, with route alternatives simulated using RouteRL integrated with SUMO.The environment begins with human drivers choosing routes through random utility theory; after their choices stabilize, some switch to AVs and human choices are sampled from the learned distribution.Source: §2.1 · p. 3
2.2 Problem formulation
- Daily route choices are modeled as a repeated congestion game and a one-cycle Agent Environment Cycle problem with AV and human agents selecting routes sequentially.The agent-selection mechanism orders actions by departure time; AVs learn routing policies over simulated days from their current observations and rewards.Source: §2.2 · pp. 3–4
3 Method
The method adds an AV-specific marginal-cost term to selfish travel-time rewards, using travel-time comparisons with counterfactual simulations that remove an AV. To reduce that computation, it prioritizes AVs whose route choices have the largest estimated impact on system travel times.
3.1 Reward with marginal cost
- The AV reward combines individual travel time with a weighted marginal cost based on travel-time differences when an AV is present versus removed.β controls the trade-off; β = 0 corresponds to selfish routing.Source: §3.1 · p. 4
3.2 Marginal cost table
- Computing marginal costs builds an impact matrix from separate counterfactual simulations that remove each AV, then sums each AV’s effects on other agents’ travel times.The counterfactuals are independent and can be run in parallel.Source: §3.2 · p. 4
3.3 Selective marginal cost
- Selective marginal cost rewards only a fraction of AVs, ranked by the standard deviation of average system travel times across their available routes.The score is estimated from sampled commute days and is intended to concentrate counterfactual computation on the AVs with the largest estimated system impact.Source: §3.3 · pp. 4–5
4 Results
Experiments on Ingolstadt and Provins show that marginal-cost rewards can reduce HDV travel-time increases while improving AV and system travel times relative to selfish routing. Applying the reward to the most impactful AVs can achieve comparable travel-time changes with fewer counterfactual simulations.
4.3 Performance of the selective marginal cost
- Rewarding the 40% most impactful AVs with selective marginal cost achieved travel-time changes comparable to rewarding all AVs, with about 40% fewer additional simulation runs.For the 40% setting, average HDV relative travel-time change was about 0.2 percentage points below the all-AV marginal-cost case.Source: §4 · p. 5, §4.3 · p. 8
4.1 Experimental setting
- The experiments used Ingolstadt and Provins networks, replaced a specified fraction of HDVs with MARL-trained AVs after 300 commute days, and tested fixed policies for 100 days after 10,000 training days.The networks contain 1,035 and 523 agents, respectively.Source: §4.1 · p. 5
4.2 Performance of the marginal cost
- In both networks, selfish AV routing increased HDV travel times across examined penetration rates, while marginal-cost rewards shortened HDV and AV travel times relative to selfish routing.The marginal-cost settings reported were β = 0.1 for Ingolstadt and Provins.Source: §4.2 · pp. 5–6
- As β increased, HDVs could gain relative to AVs, but sufficiently high weights worsened both AV and overall system travel times in Ingolstadt and Provins.The reported examples were β = 0.5 in Ingolstadt and β = 1 in Provins.Source: §4.2 · p. 6
- AV route-group choices shifted as the marginal-cost weight rose; in Ingolstadt, the share choosing group 2 fell by β = 0.5, while shares for groups 1, 3, and 4 rose.The groups refer to route options rather than physical routes.Source: §4.2 · pp. 6–7
4.3 Performance of the selective marginal cost
- In Ingolstadt, applying marginal cost to the 40% most impactful AVs improved HDV travel times by 0.4% versus selfish routing, about 0.2% below the all-AV reward.Table 1 reports results for Ingolstadt at 40% AV penetration.Source: Table 1 · p. 7, §4.3 · p. 7
- Historical impact scores varied across Ingolstadt vehicles, and applying marginal cost to the 10% most impactful AVs improved HDV relative travel-time change by 0.1% versus a random 10%.Most impact scores were concentrated around 30.Source: §4.3 · p. 7
5 Conclusion
In capacitated urban networks, AV route choices can redistribute travel-time costs among road users. The conclusion reports that marginal-cost rewards improve travel times for HDVs, AVs, and the system, while selective computation reduces the simulation burden.
5 Conclusion
- The approach adds each AV’s counterfactual impact on all vehicles’ travel times to MARL rewards, balancing its own travel time against its system impact.The setting involves road users competing for limited capacity in shared urban networks.Source: §5 · p. 8
- Marginal-cost rewards mitigate HDV travel-time increases while reducing AV travel times relative to selfish routing, but greater marginal-cost weight can worsen total system performance.The trade-off is illustrated at 40% AV penetration in Ingolstadt: increasing β from 0 to 0.1 improves HDV, AV, and system travel time relative to pre-AV levels, while increasing β to 0.2 further improves HDV travel time but worsens AV and system changes relative to β = 0.1.Source: §5 · p. 8
- At 40% AV penetration in Ingolstadt, increasing β from 0 to 0.1 improves HDV, AV, and system travel time relative to pre-AV levels by 0.6, 0.1, and 0.4 percentage points, respectively.Increasing β to 0.2 yields an additional 0.1-point HDV improvement but reduces the relative travel time change for AVs and the system by 1.3 and 0.5 percentage points, respectively, relative to β = 0.1.Source: §5 · p. 8
- Selective marginal-cost rewards for the 40% most impactful AVs reduce additional simulations by around 40% in Ingolstadt, with HDV relative travel time change around 0.2 percentage points lower than when all AVs receive the reward.This approach targets the computational burden of an additional counterfactual simulation for each AV.Source: §5 · p. 8
- AV routing policies that optimize only individual AV travel times may systematically disadvantage human drivers, so the conclusion emphasizes accounting for AV impacts on shared transportation systems.The paper frames this as affecting how travel-time costs are distributed across road users.Source: §5 · p. 8