PaperlayerView exact revision on arXiv

Source-linked AI summary

SimSkill: A Lifelong Learning AI Agent for Autonomous Mastery of Traffic Simulation

Qi Liu, Qinzheng Wang, Yiming Bie

arXiv:2609.03753v1cs.AIcs.MA

TL;DR

SimSkill addresses how AI agents can transform accumulated experience into durable, reusable competence. It uses autonomous curriculum generation, environment-grounded execution, verification, and episodic, procedural, and semantic memory for lifelong SUMO mastery, improving verified completion by up to 25 percentage points while remaining model- and budget-dependent.

  • Problem

    AI systems need ways to transform experience and accumulated knowledge into durable, reusable competence rather than solving each request from scratch.

  • Method

    SimSkill autonomously generates and solves SUMO tasks, verifies solutions, and consolidates outcomes into episodic, procedural, and semantic memory without updating the backbone LLM.

  • Results

    SimSkill improved verified completion by up to 25 percentage points across two held-out benchmarks, with complementary contributions from procedural and semantic memory.

  • Takeaways & Limitations

    SimSkill demonstrates that explicit natural-language memory can preserve and compose reusable capabilities alongside executable resources for precise, reproducible execution.

  • Takeaways & Limitations

    The observed retrieval strategy remained adequate at 150 skills and 153 pages, but larger libraries may require hierarchical indexing, learned retrieval, or domain-specific routing.

Abstract

from arXiv · show

As large language models (LLMs) become increasingly capable, the long-term value of AI systems depends not only on solving individual requests, but also on transforming experience and accumulated knowledge into durable, reusable competence. We introduce SimSkill, a self-evolving agent built around the Simulation of Urban MObility (SUMO) traffic simulator. SimSkill identifies capability gaps, generates and solves environment-grounded tasks, verifies solutions through an action--critic loop, and consolidates experience into episodic, procedural, and semantic memory without updating the backbone model. Through autonomous exploration, it builds a reusable library spanning the traffic-simulation workflow. We evaluate SimSkill on two held-out benchmarks with three backbone LLMs and independent artifact-based verification. SimSkill improves verified completion by up to 25 percentage points, while ablations show complementary contributions from procedural and semantic memory. Its benefits remain backbone- and budget-dependent: memory does not improve every model or uniformly reduce inference cost. More broadly, SimSkill illustrates a design paradigm in which natural language preserves and composes computational capabilities, while executable tools and code provide precise and reproducible execution. All code and experimental data are publicly available at https://github.com/qiliuchn/SimSkill-V1.

1 Introduction

SimSkill frames traffic-simulator mastery as a lifelong learning problem: an agent must retain and reuse experience rather than solve each SUMO task independently. It addresses this through autonomous task generation, verified execution, persistent memory, and a natural-language layer connecting reusable procedures with executable tools.

  • Motivation: Traffic-simulation systems largely automate individual user-specified studies, leaving procedures and lessons insufficiently retained for later compositional tasks.SUMO use also requires substantial transportation and software expertise.
  • Approach: SimSkill treats SUMO as an environment for autonomous exploration and lifelong mastery, rather than only as a tool for completing the current task.The framework targets autonomous cross-task capability acquisition without updating the backbone LLM.
  • Approach: Its tripartite memory preserves complete attempts episodically, reusable executable procedures procedurally, and structured domain knowledge semantically.The architecture includes bounded retrieval, ingestion, merging, and linting to keep accumulated artifacts reusable.
  • Broader principle: Natural language serves as an organizing medium for preserving, transmitting, selecting, adapting, and composing computational capabilities, while tools and code provide precise execution.This design principle is presented as relevant beyond traffic simulation.
  • Evaluation: SimSkill improves verified completion over vanilla Claude Code for DeepSeek-V4-Pro by +10 and +20 percentage points on V1 and V2, and for Qwen3.7-Max by +25 points on V1.On V2, Qwen3.7-Max completed 2 tasks versus 0 for vanilla Claude Code; GLM-5.2 did not improve.

2 Related Work

Prior work studies self-improvement, lifelong learning, external memory, reusable skills, and LLM-based traffic simulation, but SimSkill combines these directions around persistent cross-task learning in SUMO. Its emphasis is not merely automating one simulation study, but expanding capabilities over time.

  • Self-improvement: Self-evolving LLM research uses generated training data, interaction, exploration, feedback, and reflection to improve behavior with reduced human supervision.Examples include Self-Instruct, Autotelic, AppAgent, and Reflexion.
  • Lifelong learning: Lifelong LLM agents accumulate and transfer capabilities through parameter updates or external memory, with externalized approaches avoiding repeated training while raising curriculum and retrieval challenges.Voyager exemplifies automatic curriculum, execution feedback, and a growing executable-skill library in Minecraft.
  • Skill abstraction: Recent work favors abstracted, maintainable skill artifacts over indiscriminate transcript replay because irrelevant information and context limits constrain raw-experience reuse.SimSkill extends this direction with procedural and semantic memory for traffic simulation.
  • Agent memory: Long-term memory research increasingly emphasizes selecting, consolidating, structuring, and revising linked information rather than retaining a flat archive.Systems such as Mem0, MemInsight, and A-MEM illustrate these developments.
  • Traffic simulation: Transportation systems use LLMs for traffic data, analysis, optimization, control, mobility simulation, and natural-language interfaces to SUMO workflows.ChatSUMO and SUMO-MCP translate requests into scenario modifications or composable simulation-tool workflows.
  • Positioning: Unlike systems focused on automating the simulation study at hand, SimSkill focuses on autonomous cross-task learning that expands persistent capabilities over time.This positions SimSkill at the intersection of lifelong learning, external memory, reusable skills, and traffic simulation.

3 Method

SimSkill frames lifelong traffic-simulation mastery as cumulative system-level learning: an LLM agent proposes tasks, interacts with SUMO, verifies execution, and consolidates reusable experience into explicit external memory. Its design uses general interaction and memory operations rather than domain-specific workflows, preserving capabilities as inspectable artifacts without updating the backbone model.

  • Conceptual foundations: The framework uses task proposal, memory retrieval, environment interaction, criticism, consolidation, and maintenance instead of hard-coded workflows for each traffic task.Domain knowledge is accumulated through experience rather than embedded exhaustively in the architecture.
  • Conceptual foundations: SimSkill treats competence as performance across a broad task family, using curriculum design to favor novelty, diversity, practical value, gap coverage, and increasing difficulty.This objective is intended to avoid repeatedly optimizing a narrow family of easy tasks.
  • Memory architecture: SimSkill represents persistent state as episodic, procedural, and semantic memory that can be added, revised, linked, validated, and consolidated.Episodic memory records what happened, procedural memory records how to act, and semantic memory records what is known.
  • Memory architecture: Procedural skills combine natural-language instructions with executable resources, enabling reusable procedures to be retrieved, refined, composed, and shared without modifying the backbone model.Skill artifacts include validation checks and known failure modes, while executable scripts capture operations requiring exact repetition.
  • Memory architecture: Semantic-memory pages preserve declarative knowledge with retrieval metadata, provenance, related pages, and related procedural skills in a linked structure.The knowledge base is maintained as Markdown pages whose front matter supports retrieval and relationship tracking.
  • Autonomous self-evolution: Autonomous learning identifies capability gaps, proposes achievable extensions, executes tasks with concrete outputs, and uses action–critic verification before consolidating reusable outcomes.Learning-time verification is stringent because undetected errors may be written into memory and affect future tasks; user-directed inference can instead trade assurance against latency or cost.

4 Case Studies

SimSkill’s case studies show how autonomous curriculum generation, retrieval, execution, criticism, and memory maintenance produce reusable traffic-simulation capabilities. Across successful and unsuccessful tasks, the system updates skills and knowledge based on verified evidence rather than task success alone.

  • Overview: Approximately 80 hours of autonomous operation produced curriculum construction, retrieval, decomposition, revision, failure learning, composition, repair, and collection-wide memory maintenance.The case-study snapshot covers these behaviors across representative learning episodes.
  • Curriculum Construction: The curriculum agent searches for missing capabilities and decision variables, such as count-station placement and predictive signal control, rather than merely absent topic names.It identified gaps in fixed-location OD estimation and in controllers that optimize future switching decisions against predicted arrivals.
  • Retrieval and Decomposition: Retrieval made complex tasks cumulative by supplying eight procedural skills and two semantic pages for a multimodal traffic-simulation portfolio.The retrieved materials covered network construction, demand, transit, accessibility, equity, benefit–cost analysis, network design, project interactions, and simulation execution.
  • Learning from Failure: A failed shared-micromobility task still produced semantic knowledge about SUMO’s boundary and corrected existing pages after verification rejected the original limitation.The episode created no procedural skill because the motivating claim did not survive more discriminating tests.
  • Learning from Failure: The predictive-control episode created reusable procedures, validation rules, and semantic knowledge after critic feedback exposed insufficient verification and a confounded comparison.The study did not establish an advantage for predictive control, but the revised and independently validated episode still yielded memory updates.
  • Composition and Repair: GPS map matching composed existing procedures, repaired a reused script, and committed the new skill, dependency update, and semantic changes together.This coordinated update lets later agents receive consistent artifacts while using reuse failures to validate earlier memory.
  • Memory Maintenance: memory-lint performs collection-level quality control by checking changes, repairing inconsistencies, maintaining cross-references, and identifying redundant or obsolete content.The learned library contains 150 skills spanning the end-to-end traffic-simulation workflow, with skills counted once for summary purposes.

5 Experiments

The experiments test SimSkill across backbones, held-out task difficulty, memory components, verification quality, and resource use. SimSkill improves completion for some models and difficult tasks, but gains and costs depend on the backbone and operating budget.

  • Completion and backbone dependence: DeepSeek-V4-Pro improves from 34 to 38 verified V1 tasks and from 19 to 27 V2 tasks under SimSkill, while Qwen3.7-Max improves from 13 to 23 on V1 and from 0 to 2 on V2.The absolute gains are 10 and 20 percentage points for DeepSeek-V4-Pro; Qwen3.7-Max shows the largest V1 gain.
  • Completion and backbone dependence: GLM-5.2 shows no completion gain, with SimSkill trailing vanilla by one task on V1 and tying it on V2.The experiment does not isolate whether model–runtime factors explain this difference, so the proposed explanation remains plausible rather than established.
  • Generalization to difficult tasks: SimSkill also improves difficult-task performance: DeepSeek-V4-Pro gains six V2 Tier 4 tasks, while Qwen3.7-Max gains four V1 Tier 4 tasks and solves two V2 Tier 4 tasks.These results support generalization to new compositions and first-principles tasks rather than only direct recall of stored skills.
  • Accuracy–resource trade-off: Memory creates an accuracy–resource trade-off rather than uniformly lowering cost: it raises some completion endpoints and costs, while reducing GLM-5.2’s V1 median cost by approximately 31% and time by approximately 52%.For DeepSeek-V4-Pro on V2, SimSkill costs $3.93 versus $2.92 while achieving eight additional completions; budget-dependent curves therefore matter more than median cost alone.
  • Continuous quality scores: Under the GLM-5.2 pointwise judge, SimSkill raises mean V2 scores from 0.891 to 0.940 for DeepSeek-V4-Pro and from 0.674 to 0.728 for Qwen3.7-Max.For DeepSeek-V4-Pro, the lower median under SimSkill indicates that the improvement is concentrated in reducing the lower tail rather than uniformly shifting every task.
  • Ablation study: Procedural and semantic memory both contribute to V1 completion, with their combination reaching 38 tasks for DeepSeek-V4-Pro and 23 for Qwen3.7-Max.Procedural memory contributes slightly more than semantic memory for both backbones, while the aggregate endpoint effects are additive rather than super-additive.
  • Overall findings: Overall, SimSkill improves verified completion for two of three tested backbones and retains advantages on hard compositional and novel tasks, but memory overhead frequently raises total inference cost.The results support model-dependent benefits and a modestly larger procedural contribution.

6 Conclusion

SimSkill converts interaction with SUMO into persistent competence through autonomous curriculum generation, verified execution, and episodic, procedural, and semantic memory. Its benefits are measurable but remain model- and cost-dependent, motivating broader evaluation and refinement.

  • Core contribution: SimSkill couples autonomous curriculum generation with verified task execution and three memory forms without updating the backbone model.Episodic, procedural, and semantic memory support persistent competence from executable-environment interaction.
  • Core contribution: After approximately 80 hours, SimSkill accumulated 150 procedural skills and 153 semantic-memory pages.The artifacts span the major stages of traffic-simulation practice.
  • Results: Verified completion improved by 10 and 20 percentage points for DeepSeek-V4-Pro on V1 and V2, and by 25 points for Qwen3.7-Max on V1.Qwen3.7-Max also completed 2 V2 tasks versus 0 for vanilla Qwen3.7-Max.
  • Scope and future work: Benefits were not universal: GLM-5.2 showed no improvement, and greater task coverage did not consistently reduce inference cost.Future work targets repeated snapshot evaluation, curriculum and memory improvements, and transfer and forgetting across simulators and domains.
  • Broader implication: Natural language preserves and composes adaptable knowledge, while executable tools and code provide precise and reproducible execution.The paper presents this division as a broader design principle for self-evolving AI systems.

A.1 Retrieval and Task Decomposition for PP-T4-3-V2

For a multimodal portfolio task, bounded retrieval supplied heterogeneous procedures and semantic knowledge, after which the action agent generated a task-specific workflow. The workflow was composed rather than retrieved as a fixed end-to-end recipe.

  • Retrieval: Bounded retrieval supplied 8 procedural items and 2 semantic-memory items for the PP-T4-3-V2 task.The retrieved components included discrete network design and transport-equity knowledge.
  • Task decomposition: The task-specific decomposition covered grid construction, OD routing, D-optimal portfolio selection, simulation, surrogate modeling, Pareto analysis, stress testing, and equity evaluation.It also included benefit monetization and distributional reporting.
  • Composition: No single retrieved item specified the end-to-end workflow; the action agent composed it from retrieved procedures and knowledge pages.This composition connected heterogeneous capabilities rather than copying a fixed workflow.
  • Curriculum principle: The curriculum trace distinguishes missing memory coverage from simulator impossibility and requires adversarially establishing absence before proposing a replacement.The shared-micromobility episode tested candidate primitives through explicit refutations and verification.

A.3 Predictive-Control Skill Acquisition

The predictive-control episode used a curriculum gap to compare rolling-horizon methods under validated prediction and controls, then iteratively repaired implementation and interpretation errors through action–critic verification.

  • Curriculum: The curriculum targeted signal control that optimizes future decisions using predicted arrivals, comparing rolling-horizon DP with simulation-rollout MPC.The design included null and oracle controls, demand and arrival regimes, replication intervals, and a two-signal coordination arm.
  • Verification: The first outer attempt failed after a watchdog timeout and required checks covering state restoration, sequence optimization, prediction-null equivalence, controller constraints, and claim scope.The action agent consolidated 1,016 existing SUMO runs without re-simulating completed cells.
  • Verification: The second attempt succeeded after nine independent checks, including constraint sweeps, exhaustive DP comparison, predictor–optimizer separation, and repeated critical cells.The critic-driven process converted the failed attempt into a verified result.
  • Memory maintenance: The resulting memory update added predictive-control skills and semantic pages while retaining distinct artifacts after linting and link synchronization.Memory lint checked ten changed artifacts, parsed 3/3 new scripts, resolved links, synchronized summaries, and found no repair was needed.
  • Skill library: The learned library comprised 150 categorized procedural skills spanning scenario construction, infrastructure, demand, signals, operations, multimodal systems, calibration, and impact analysis.Categories indicate primary function, while cross-links and composition connect skills across boundaries.

C.1 Task and Memory Use

For a compositional congestion-and-safety task, retrieval provided complementary execution, visualization, safety-analysis, and validation artifacts. The action agent composed them into one workflow and required three critic attempts to reconcile implementation and reporting errors.

  • Task: The task required a 4 × 4-block signalized grid, 200 m spacing, 1,000 veh/h inbound demand, congestion heat maps, SSM conflict matching, and statistical comparison.Its components spanned network and demand generation, simulation, visualization, safety configuration, and analysis.
  • Memory use: Six retrieved procedural skills supplied network construction, demand generation, simulation, heat-map generation, SSM analysis, and output-integrity checks.Two semantic pages shaped analytical decisions rather than execution.
  • Composition: The action agent composed one common simulation, twelve interval-level congestion maps, directed-edge conflict assignments, and congestion–safety ranking comparisons.The workflow was not copied from a single skill.
  • Verification: The critic required three outer attempts: it rejected demand and parsing errors first, then corrected interpretation discrepancies and omitted limitations.The final attempt reconciled the report with output artifacts and corrected the hotspot description.

C.3 Verified Result and Output

The verified OA-T3 cross-check distinguishes congestion ranking from direct safety-hotspot analysis, while DG-T4-3-V2 demonstrates budgeted, uncertainty-aware OD inference. The outputs preserve reproducibility through scripts, ledgers, diagnostics, and independently checked artifacts.

  • Output: The final package included reproducible scripts, simulation and SSM files, rankings, reports, heat maps, ledgers, and diagnostic outputs.Table 9 is identified as the accepted OA-T3 congestion–safety cross-check, and Table 10 as the accepted DG-T4-3-V2 results.
  • OA-T3: Congestion heat-map rankings cannot substitute for direct safety-hotspot analysis in this scenario.The comparison used edge-data heat maps and conflicts parsed with a retrieved safety procedure.
  • DG-T4-3-V2: The accepted DG-T4-3-V2 task inferred posterior distributions for 240 nonnegative OD flows from sparse counts and Bluetooth travel times.The task used 24 screenline counts, sparse Bluetooth observations, holdouts, and a fixed budget of 15 SUMO executions.
  • DG-T4-3-V2: The workflow combined retrieved ODME and identifiability procedures with a task-owned likelihood-free ABC–SMC component.The agent excluded holdouts from fitting and stopping while using one route set for posterior and comparator analyses.
  • DG-T4-3-V2: Fifteen designated simulations supported surrogate training, posterior representatives, and the deterministic ODME comparator.Twelve runs trained the surrogate, while posterior inference used the cached surrogate without additional SUMO calls.
  • Verification: Independent criticism verified masks, budgets, comparator use, diagnostic requirements, and the absence of SUMO calls during posterior inference.Five discrepancies in reported numbers and stored artifacts were corrected without changing code or rerunning the experiment.

D.3 Verified Result and Output

The accepted DG-T4-3-V2 result met its simulation, convergence, and holdout requirements while retaining uncertainty about individual OD cells. Its artifacts expose both inferred demand and remaining non-identifiability.

  • Verified result: The accepted output satisfied the prescribed simulation budget, convergence checks, and holdout criterion.The result retained semantic-memory limitations concerning aggregate demand identification and the OD null space.
  • Limitations: Aggregate demand was better identified than individual OD cells, and good count fit did not eliminate OD non-identifiability or simulator–surrogate discrepancy.These limitations were explicitly retained in the accepted output.
  • Output: Figure 10 combines demand checks, posterior-predictive checks, parameter correlations, and non-identifiable contrasts.The figure distinguishes fit to observed aggregates from uncertainty in underlying OD cells.

E.1 Task and Selective Memory Use

MT-T4-4-V2 evaluates three shared-scooter rebalancing policies under operational, environmental, and equity constraints. SimSkill transferred general procedures while constructing missing micromobility logic in task-owned code.

  • Task: MT-T4-4-V2 compares no rebalancing, scheduled truck rebalancing, and incentive-based user rebalancing for a docked shared e-scooter system.The comparison includes operational, environmental, and equity constraints.
  • Selective memory use: No shared-scooter or micromobility semantic page was available, so the episode constructed missing domain-specific logic while transferring procedural memory.The retrieved procedures covered network construction, lanes, freight routing, emissions, and accessibility or equity reporting.
  • Results: The base-scenario results are summarized as averages over four paired perturbation seeds.Table 11 identifies the accepted MT-T4-4-V2 base-scenario results.
  • Selective memory use: The selected event-driven formulation required no TraCI state-transfer loop, so retrieved capabilities were adapted rather than applied mechanically.This reflects selective use of candidate procedures for the task’s implementation.

E.2 Memory-Guided Inference and Task-Specific Extension

For MT-T4-4-V2, the action agent assigned SUMO to network and route validation while implementing the scooter system in a task-owned discrete-event engine. The critic reran the complete workflow and accepted the first outer attempt.

  • Memory-guided inference: SUMO handled network construction and route-time validation, while task-owned code modeled inventory, batteries, walking, docking, trucks, and incentives.The division combined stored reliable components with newly constructed domain-specific mechanisms.
  • Execution: The workflow produced 18 policy–scenario–seed runs across four base perturbations and two rain perturbations.The inner loop corrected four local defects involving metrics, incentives, reporting, and route validation.
  • Verification: The critic reran the complete workflow, obtained byte-identical outputs, checked every explicit requirement, and accepted the first outer attempt.Non-blocking modeling qualifications remained in the accepted result.

E.3 Verified Result and Output

The evaluated MT-T4-4-V2 task produced a verified, reproducible traffic-simulation package and showed that truck rebalancing delivered the strongest service and equity outcomes, including under rain demand.

  • 95.8% served share under truck rebalancing exceeded 58.8% without rebalancing and 75.1% under incentives in the base scenario.Low-income-origin station service increased from 40.9% to 95.7% under truck rebalancing.
  • 98.9% served share under truck rebalancing remained highest in the rain sensitivity case, ahead of 97.8% for incentives and 87.6% without rebalancing.The action agent recommended truck rebalancing as the primary strategy and incentives as a complement without truck travel.
  • The final output included five task scripts, the SUMO network and specification, validation, machine-readable results, outcome files, a report, and two figures.Figure 11 combined the retrieved infrastructure procedures with policy, equity, and VKT outcomes; critic qualifications were retained in the report.
  • Across three cases, memory-guided inference composed procedural and semantic artifacts, constructed task-owned analyses, or selectively transferred relevant procedural skills.MT-T4-4-V2 had no matching semantic page and discarded unnecessary retrieved skills.
  • The benchmark tasks ran in test mode, so their interactions did not change memory available to later benchmark tasks.
Loading 2609.03753v1…