PaperlayerView exact revision on arXiv

Source-linked AI summary

CityPlanner: A Sandbox Agent for Executable Urban Planning

Wentao Zhang, Jingyuan Wang, Zetong Zhou, Yifan Yang, Wenrui Wang

arXiv:2609.09578v1cs.AIcs.CL

TL;DR

Urban planning requires feasible decisions across heterogeneous action spaces and task-specific constraints, while existing methods often assume fixed formulations. CityPlanner provides a file-based sandbox with executable evaluation and decomposes planning into BuildPlan and ImprovePlan. On a real-world benchmark, it consistently improves planning quality over heuristic, task-specific RL, and LLM-agent baselines, with ablations supporting its main components.

  • Problem

    Urban planning involves heterogeneous tasks with different candidate types, constraints, evaluators, and feedback, while direct reinforcement learning over long sandbox trajectories faces noisy interactions and sparse rewards.

  • Method

    CityPlanner uses UrbanSandbox for file inspection, plan generation, executable evaluation, and feedback, while atomic-task reinforcement learning separates BuildPlan from ImprovePlan and composes them iteratively.

  • Results

    CityPlanner consistently improves planning quality over heuristic, task-specific RL, and LLM-agent baselines on a real-world benchmark, with ablations verifying UrbanSandbox, atomic-task RL, and iterative refinement.

  • Takeaways & Limitations

    Executable feedback and structured decomposition provide a common approach for training and deploying agents across heterogeneous urban planning tasks.

  • Takeaways & Limitations

    CityPlanner requires slower multi-turn sandbox interaction, provides no theoretical optimality guarantee, and remains dependent on model capability, reward design, and evaluator feedback.

Abstract

from arXiv · show

Urban planning is a real-world spatial optimization problem that requires selecting feasible actions from large candidate spaces under practical objectives such as cost and service quality. Existing optimization and reinforcement learning methods are effective for fixed formulations, but often depend on task-specific representations and constraint handling. We propose \emph{CityPlanner}, a sandbox-agent framework for executable urban planning. CityPlanner introduces \emph{UrbanSandbox}, a unified file-based environment where agents inspect task files, generate plans, run evaluators, and revise decisions based on executable feedback. To make learning tractable, we further propose atomic-task reinforcement learning, which decomposes long sandbox trajectories into \emph{BuildPlan} for initial construction and \emph{ImprovePlan} for feedback-based refinement. Experiments on a real-world benchmark show that CityPlanner consistently outperforms heuristic, task-specific RL, and general LLM-agent baselines. Ablations verify the contributions of UrbanSandbox, atomic-task RL, and iterative deployment. We release the code and dataset at https://anonymous.4open.science/r/co-agent-C1C8

1 Introduction

Urban planning combines large, constrained action spaces with context-dependent, competing objectives, while existing methods often require task-specific formulations. CityPlanner addresses this heterogeneity through an executable sandbox, atomic-task reinforcement learning, and iterative plan refinement.

  • Motivation: Urban planning requires decisions over land use, roads, and facilities under large combinatorial spaces, hard feasibility constraints, and competing objectives.Action values also depend on surrounding population, topology, and local demand patterns.
  • Motivation: Existing optimization, heuristic, and reinforcement learning methods work best when representations, actions, constraints, and objectives are fixed, limiting transfer across heterogeneous tasks.Different tasks may use distinct candidate types, constraint definitions, evaluators, and feedback forms.
  • Motivation: LLM agents can inspect files, execute tools, and revise solutions, but constrained planning still requires explicit feasibility checks and task-specific executable evaluation.Natural-language plausibility alone is insufficient for validating generated plans.
  • Approach: CityPlanner uses UrbanSandbox, a unified file-based environment where agents inspect inputs, construct plans, run evaluators, observe diagnostics, and revise decisions.The protocol preserves task-specific objectives and constraints while using a common interaction interface.
  • Approach: Atomic-task reinforcement learning separates initial construction from feedback-based refinement through BuildPlan and ImprovePlan, then composes them iteratively at inference time.This decomposition targets long, noisy trajectories and sparse rewards that make direct reinforcement learning difficult.
  • Evaluation: Across 5,670 OpenStreetMap-based planning instances and three tasks, CityPlanner improves planning quality over heuristic, task-specific RL, and LLM-agent baselines.Ablations also support contributions from UrbanSandbox, atomic-task reinforcement learning, and iterative refinement.

2 Related Work

Prior work applies LLM agents and reinforcement learning to interactive tasks, including urban analysis and planning. CityPlanner targets the long, noisy, sparsely rewarded trajectories that arise when these capabilities are used for executable urban planning.

  • LLM agents and executable environments: Recent LLM agents support interactive problem solving with tools, APIs, files, and executable environments, while LLMs have also been explored for urban analysis and planning assistance.Existing agent benchmarks commonly emphasize general tool use, web interaction, coding, or command-line tasks.
  • Reinforcement learning for LLM agents: Reinforcement learning has improved LLM agents in tool use, web navigation, coding, and interactive planning, but trajectories are often long, noisy, and rewarded only after completion.CityPlanner responds by separating construction and refinement into finite-context tasks with distinct task-level rewards.

3 Problem Statement

The paper models urban planning as selecting a constraint-satisfying subset of candidate spatial actions for a given urban region. The formulation covers land allocation, road construction, and station placement while sharing a common decision structure.

  • Planning tasks: Land allocation assigns land-use types to candidate parcels under land-use, spatial, ecological, and service-balance constraints.
  • Planning tasks: Road construction selects network segments under budget, quantity, and spatial-validity constraints.
  • Planning tasks: Station placement selects candidate charging-facility sites under demand, capacity, infrastructure, and cost constraints.
  • Shared formulation: Despite different spatial objectives, the tasks share a decision structure: selecting a constraint-satisfying subset from a finite candidate action set.
  • Formal representation: Each instance is represented by an attributed city graph containing spatial units, road-network edges, and their attributes.The candidate set is drawn from spatial units and road-network edges.
  • Formal representation: A plan P selects a subset of candidates, with Ψ_p(P, G_p) indicating feasibility and S_p(P) denoting the planning score.The formal objective and detailed definitions are provided beyond this passage.

4 Method

CityPlanner models urban planning as executable sandbox interaction and makes long-horizon learning tractable by decomposing it into atomic construction and improvement tasks. At inference, it iteratively evaluates and selectively accepts revised plans under validity-first ordering.

  • UrbanSandbox: UrbanSandbox exposes planning through a file-based environment where agents inspect inputs, execute commands, and receive observations from the sandbox.The environment includes task data, candidate sets, constraints, evaluators, writable workspaces, outputs, and execution logs.
  • Atomic-Task Reinforcement Learning: Direct flat RL faces sparse credit assignment because complete sandbox trajectories contain long sequences of reading, construction, revision, and evaluation.The formulation explicitly contrasts long-horizon trajectories with shorter atomic-task horizons and dense task-level rewards.
  • UrbanSandbox: Executable verification determines both plan validity and quality rather than language-level checking.Final plans are parsed from outputs/final_plan.json, checked against aggregate constraints, and evaluated for planning score.
  • Atomic-Task Reinforcement Learning: Atomic-task RL replaces long-horizon planning with BuildPlan for feasible initial construction and ImprovePlan for feedback-based revision.Each task has a shorter horizon and an independent task-level reward, while policy optimization uses GRPO rollouts from both tasks.
  • Iterative Inference: At inference, CityPlanner builds an initial plan, repeatedly proposes revisions using diagnostics, and evaluates each proposal before acceptance.The loop retains a proposal only when it is better under validity-first ordering: feasible plans outrank infeasible plans, and higher-scoring feasible plans are preferred.

5 Experiments

Experiments evaluate CityPlanner on a 5,670-instance OpenStreetMap benchmark across three tasks, comparing it with heuristic, task-specific RL, and general LLM-agent baselines. CityPlanner achieves the strongest overall performance, while ablations show benefits from executable scaffolding, atomic-task training, iterative refinement, and moderate runtime.

  • Experimental Setup: The benchmark contains 5,670 planning instances over 1,890 urban tiles across three tasks.All methods use the same instances, sandbox files, output format, and interaction budget; higher evaluator objective scores indicate better planning quality.
  • Overall Performance: CityPlanner obtains the best score in 8 out of 9 task-and-difficulty settings.This is reported as the strongest overall performance across three tasks and three difficulty levels.
  • Overall Performance: Heuristic methods remain competitive, especially on smaller or more regular instances, but require task-specific search moves, repair rules, and constraint handling.GRASP performs best on small Road Construction and second-best on medium and large Road Construction, while SA is competitive on small Station Placement.
  • Overall Performance: Task-specific RL baselines perform substantially worse across all tasks under limited training data.The reported interpretation is that neural policies fail to learn effective planning strategies in this setting.
  • Ablation Study: UrbanSandbox achieves the best feasibility and score on all tasks among frozen-planner test-time strategies.Text-only prompting has feasibility below 30% on every task, while EoH and FunSearch remain unstable, including 21.00% and 32.00% feasibility on Road Planning.
  • Ablation Study: ATRL improves feasibility over GRPO by 8.3 percentage points on average and improves objective scores by 20.7% on average.ATRL reaches 100.00% feasibility on all tasks; iterative refinement adds 4.9% over ATRL, and Qwen3-14B adds another 2.8% average gain.
  • Efficiency Analysis: CityPlanner achieves the best overall planning quality while maintaining moderate runtime.TSRL is fastest but has limited quality, heuristics require non-trivial search time, and general LLM agents are slower because of multi-turn reasoning and sandbox interaction.

6 Conclusion

CityPlanner formulates heterogeneous urban planning as sandbox interaction and combines a unified file-based environment with atomic-task reinforcement learning. Experiments show consistent planning-quality improvements over heuristic, task-specific RL, and LLM-agent baselines, with ablations supporting its major components.

  • Conclusion: CityPlanner provides UrbanSandbox, a unified file-based environment with task-specific constraints, evaluators, and feedback.It decomposes long planning trajectories into BuildPlan and ImprovePlan, then composes the learned skills through iterative refinement at inference time.
  • Conclusion: Experiments on an OpenStreetMap-based benchmark show that CityPlanner consistently improves planning quality over heuristic, task-specific RL, and LLM-agent baselines.Further ablations verify contributions from UrbanSandbox, atomic-task reinforcement learning, and iterative deployment.

Limitations

CityPlanner requires costly multi-turn sandbox interaction and does not provide theoretical optimality guarantees. Its final solution can depend on model capability, reward design, and evaluator-feedback informativeness.

  • Computational Cost: Multi-turn file inspection, script execution, and repeated evaluator calls make CityPlanner slower than direct policy inference or lightweight heuristics.This is identified as the method's computational-cost limitation.
  • Optimality Guarantee: Iterative refinement improves plans without providing theoretical optimality guarantees.The final solution may still depend on model capability, reward design, and the informativeness of evaluator feedback.

Ethical Considerations

The benchmark uses public OpenStreetMap elements and excludes sensitive user-level information. The accompanying text also identifies additional analyses of training dynamics, context evolution, interaction cost, and refinement behavior.

  • Data Sources: The dataset uses public roads, land parcels, facilities, and points of interest from OpenStreetMap.It excludes personal trajectories, demographic profiles, land-ownership records, and other sensitive user-level information.
  • Privacy: The dataset construction does not introduce additional privacy risks beyond the underlying public map data.This conclusion follows from restricting inputs to public map elements and excluding sensitive user-level information.
  • Additional Analyses: Additional analyses examine training dynamics, context evolution under atomic tasks, interaction cost, and refinement behavior.The passage identifies these analyses without reporting their findings.

A.1 Training Dynamics

Atomic-task reinforcement learning separates initial feasible construction from feedback-based refinement, while sandbox feedback and iterative deployment support effective planning without substantially longer interactions.

  • Atomic-task reinforcement learning: BuildPlan’s valid rate rises steadily, but its final score remains noisy, indicating emphasis on feasible initial construction over fine-grained optimization.The training dynamics distinguish validity learning from quality optimization.
  • Atomic-task reinforcement learning: ImprovePlan increases both validity and final score, showing that the model learns to revise plans using evaluator feedback.The two atomic tasks therefore acquire complementary construction and refinement skills.
  • Context evolution: Atomic-task planning resets context between calls, limiting context growth and reducing cumulative prefill-token workload relative to flat sandbox planning.Flat planning accumulates file contents, command outputs, failed attempts, and revisions in one trajectory.
  • Interaction efficiency: CityPlanner uses a comparable number of interaction turns to other strong LLM agents, so its gains do not come from substantially longer interactions.The reported comparison links effectiveness to sandbox feedback and iterative refinement rather than extra turns.
  • Refinement behavior: Iterative deployment quickly finds a valid intermediate road plan and improves it through evaluator feedback while preserving the best-so-far solution.On the representative episode, CityPlanner eventually exceeds the Greedy baseline after surpassing ALNS and SA.

C Dataset Construction

The benchmark is constructed from filtered OpenStreetMap data across 15 Chinese cities, producing 5,670 instances for land allocation, road construction, and station placement at three difficulty levels.

  • Data collection: The initial crawl contains 103,098 tiles from five major urban clusters covering 15 representative Chinese cities.Tiles cover approximately 0.03° × 0.03°, or about 3.3 km by 2.6 km depending on latitude.
  • Filtering and splits: Filtering dense, reliable urban regions yields 1,890 final tiles, split into 1,512 training tiles and 378 test tiles.The split follows an 80/20 allocation.
  • Benchmark composition: Each final tile instantiates land allocation, road construction, and station placement, producing 5,670 planning instances.The benchmark therefore contains three task instances per tile.
  • Difficulty levels: Instances are grouped into small, medium, and large difficulty levels according to candidate-action and demand-region counts.Road construction averages about 63, 102, and 143 candidates across the three levels; station placement averages 80, 80, and 180; land allocation averages 85, 119, and 199.
  • Candidate representation: Candidate actions encode task-specific decisions, including charger configurations, road segments, or land-use assignments with associated spatial and cost attributes.These candidate lists provide the selectable action space for each planning task.

D Sandbox Implementation

UrbanSandbox gives agents a common file-based terminal interface for inspecting city data, constructing and evaluating plans, and refining them through BuildPlan and ImprovePlan workflows.

  • Workspace interface: Each episode uses input/, work/, outputs/, and logs/ directories for read-only task data, scratch work, submitted plans, results, and execution traces.This directory structure standardizes workspace interaction across tasks.
  • File-based information access: The workspace exposes city state, candidate summaries, complete candidate lists, task exports, and evaluator scripts through input files.The prompt does not contain the full instance data; agents retrieve it by inspecting files and executing commands.
  • Plan submission and evaluation: Plans use the official candidate_ids schema in final_plan.json, and evaluation returns validity, diagnostics, task-specific scores, and parsed candidate ids.Evaluation history is appended to outputs/eval_history.jsonl, and hard errors must be repaired before completion.
  • Terminal protocol: Agents must emit JSON containing analysis, plan, commands, and task_complete, with completion allowed only after writing and evaluating the final plan.Commands execute through the terminal protocol and must be valid JSON actions.
  • Atomic-task workflows: BuildPlan creates a valid non-empty initial plan, whereas ImprovePlan edits a preloaded plan to increase its score while preserving validity and non-emptiness.BuildPlan and ImprovePlan share the same system prompt and workspace I/O contract.
  • Training and baselines: CityPlanner combines SFT with GRPO over BuildPlan and ImprovePlan, while inference retries BuildPlan and repeats ImprovePlan until a patience budget is reached.The implementation also compares heuristic, task-specific RL, and LLM-guided program-search baselines under the shared evaluator.
Loading 2609.09578v1…