PaperlayerView exact revision on arXiv

Source-linked AI summary

TransitLM: A Large-Scale Dataset and Benchmark for Map-Free Transit Route Generation

Hanyu Guo, Jiedong Yang, Chao Chen, Longfei Xu, Kaikui Liu, Xiangxiang Chu

arXiv:2605.22355v1cs.CLcs.AIcs.LG

TL;DR

TransitLM addresses the lack of data for learning transit planning without maps or routing engines by combining a large route-planning corpus with a three-task benchmark. A model trained on it generates structurally valid routes and grounds GPS inputs to stations, although generalization beyond the covered Chinese networks and languages remains unverified.

  • Problem

    Existing transit planning depends on structured maps and routing engines, while available datasets do not support training models for direct map-free route generation.

  • Method

    TransitLM provides over 13 million records from four Chinese cities as a continual-pretraining corpus and benchmark fine-tuning data for three planning tasks.

  • Results

    Models produce connected routes, achieve station grounding and accurate numeric prediction, and a jointly trained model generalizes across all three benchmark tasks.

  • Takeaways & Limitations

    Transit route planning can be performed through end-to-end text generation using transit knowledge acquired entirely from data, without external maps or routing engines.

  • Takeaways & Limitations

    Generalization beyond four Chinese cities and Chinese-language networks remains unverified, while station-token scaling creates substantial vocabulary and computation costs.

Abstract

from arXiv · show

Public transit route planning traditionally depends on structured map infrastructure and complex routing engines, and no existing dataset supports training models to bypass this dependency. We present TransitLM, a large-scale dataset of over 13 million transit route planning records from four Chinese cities covering 120,845 stations and 13,666 lines, released as a continual pre-training corpus and benchmark data for three evaluation tasks with complementary metrics. Experiments show that an LLM trained on TransitLM produces structurally valid routes at high accuracy and implicitly grounds arbitrary GPS coordinates to appropriate stations without any explicit mapping. These results demonstrate that transit route planning can be learned entirely from data, enabling end-to-end, map-free route generation directly from origin-destination information. The dataset and benchmark are available at https://huggingface.co/datasets/GD-ML/TransitLM, with evaluation code at https://github.com/HotTricker/TransitLM.

1 Introduction

TransitLM addresses the lack of training data for map-free transit planning with a large dataset and benchmark. Experiments indicate that models trained on it can generate valid routes, ground GPS coordinates to stations, and support multiple planning objectives.

  • Conventional transit planning depends on structured maps and complex routing pipelines, while route logs encode stations, transfers, and traveler preferences.
  • General-purpose LLMs often produce hallucinated stations and broken connections, especially for less prominent origin-destination pairs.
  • TransitLM contains over 13 million route-planning records from four Chinese cities, covering 120,845 stations and 13,666 lines.
  • The released resources include a 13.9-million-record continual pre-training corpus and supervised data for optimal, preference-aware, and multi-route generation.
  • Trained models generate connected routes, implicitly ground GPS coordinates to stations, and generalize across three planning objectives with one jointly trained model.

2 Related Work

Classical transit routing achieves efficient multi-criteria planning through explicit graphs, maps, and schedules, while newer LLM approaches still retain infrastructure or operate on simplified settings. Existing datasets and benchmarks do not provide direct station-level transit route generation coverage.

  • Classical methods such as Dijkstra, A*, RAPTOR, Connection Scan, and Transfer Patterns require structured map infrastructure and schedule data.
  • LLM-assisted routing can supply heuristics but still requires graph input, while grid and pixel-map benchmarks do not establish end-to-end transit route generation.
  • Vehicle trajectories lack public-transit station structures, transfer logic, and line identifiers, whereas static network datasets lack user behavior and route preferences.
  • Travel-planning benchmarks focus on multi-day itineraries and high-level constraints, and MobilityBench evaluates map-API orchestration rather than direct route generation.
  • No existing benchmark assesses whether an LLM can directly produce structurally valid transit routes with station-level precision.

3 Dataset Construction

TransitLM is built from Amap route-planning logs and complementary static transit descriptions, then released as continual-pretraining and task-specific fine-tuning resources. Its corpus spans diverse cities, modalities, distances, and sequence lengths.

  • Amap logs from Beijing, Shanghai, Shenzhen, and Chengdu cover 120,845 stations and 13,666 lines, with over 12.9 million planning sessions.
  • The dataset releases a continual-pretraining corpus and benchmark supervised fine-tuning data for three route-generation tasks.
  • The corpus combines route sessions with station and line descriptions so models can learn network topology, spatial relationships, and user preference patterns.
  • TransitBench defines three tasks with standardized prompts and labels, including optimal, preference-aware, and multi-route generation.
  • Data Statistics and Analysis: Bus-only routes comprise 33.0%, subway-only routes 19.0%, bus+subway routes 16.8%, and mixed routes 30.5%.
  • Data Statistics and Analysis: 22.8% of routes are under 5 km, 47.4% span 5–20 km, and 29.7% exceed 20 km, with most travel times between 15 and 90 minutes.
  • Data Statistics and Analysis: The corpus averages 2,377 Chinese characters per record, and totals over 20 billion tokens for continual pre-training.

4 Benchmark Tasks

The benchmark evaluates map-free transit route generation through three tasks and complementary metrics covering route validity, access feasibility, structural overlap, numeric accuracy, preferences, and diversity.

  • Task Definitions: The benchmark defines optimal, preference-aware, and multi-route generation as three complementary transit planning tasks.The tasks generate structured routes from origin-destination information, with preference-aware planning adding explicit user constraints and multi-route generation producing three diverse routes.
  • Task Definitions: Optimal Route Generation requires a single structured-JSON route containing lines, station sequences, transfers, distance, time, fare, and access details.Its label is the top-ranked route also selected by the user.
  • Task Definitions: Preference-Aware Planning tests subway-first, bus-first, fewer-transfers, and shortest-time constraints while retaining the optimal-route format.Training examples come from sessions where users explicitly set one of the four preferences.
  • Task Definitions: Multi-Route Generation produces three routes with route-type tags, using candidate triples assembled by user selection, tag distinction, and route non-overlap.The task is designed to represent multiple valid alternatives in one JSON response.
  • Evaluation Metrics: Evaluation measures connectivity, access feasibility, route overlap, and numeric-field accuracy, with preference compliance and route diversity added for relevant tasks.Connectivity gates subsequent metrics; overlap uses LO, SSO, and REM, while numeric accuracy uses EA and MAPE on exact-match samples.

5 Experiments

Experiments train Qwen3 models with continual pre-training and supervised fine-tuning, then compare them with general-purpose and tool-augmented systems across benchmark and ablation settings. Transit-specific training yields strong map-free performance across tasks, while results improve with model capacity and data volume.

  • Experimental Setup: The experimental pipeline registers all 120,845 station IDs as single tokens and trains models with continual pre-training followed by task-specific supervised fine-tuning.Separate time periods for pre-training and fine-tuning prevent overlap, and a joint 4B variant combines all three task datasets.
  • Comparisons: General-purpose LLMs remain weak under a lenient evaluation, with Gemini-3.1-Pro reaching 75.5% connectivity and 40.2% Route Exact Match.These models predicted only boarding and alighting stations, whereas TransitLM models generated complete intermediate station sequences.
  • Benchmark Results: TransitLM’s Qwen3-4B achieves at least 93% connectivity, at least 96% station grounding, up to 71.0% Route Exact Match, estimation accuracy above 92%, and MAPE below 2.1%.Route Exact Match is 71.0% for Optimal Route Generation, 50.4% for Preference-Aware Planning, and 64.5% for Multi-Route Generation.
  • Benchmark Results: Performance scales monotonically with model capacity, with Qwen3-4B gaining +8.9pp Route Exact Match over Qwen3-0.6B on Optimal Route Generation.The smallest 0.6B model also surpasses the six general-purpose LLMs evaluated under more lenient conditions.
  • Joint Training: The joint 4B model matches or exceeds the single-task 4B model across all benchmarks, with Preference-Aware Planning improving by 2.1 percentage points in connectivity and 2.2 percentage points in Route Exact Match.The reported results show no negative transfer across the three tasks.
  • Data Scaling: At 6.25% of continual-pre-training session data, Qwen3-4B already achieves 94.0% connectivity and 49.9% Route Exact Match, while metrics improve with additional data.Static descriptions and supervised fine-tuning data remain unchanged across the data-fraction comparison.
  • GPS-Only Ablation: In GPS-only ablations, general-purpose LLMs mostly fall below 1% Route Exact Match, whereas TransitLM models show minimal degradation after textual cues are removed.This comparison tests whether performance depends on learned spatial knowledge rather than origin and destination names.

6 Conclusion

TransitLM establishes that map-free transit planning can be learned from data, while its current coverage remains limited to four cities and static route structures.

  • 6 Conclusion: TransitLM provides a three-task benchmark and standardized metrics for map-free transit route generation across four Chinese cities.The dataset covers 120,845 stations and 13,666 lines, while the benchmark evaluates optimal, preference-aware, and multi-route planning.
  • 6 Conclusion: End-to-end transit planning through pure text generation is feasible without external maps or routing engines.The experiments indicate that topology, spatial, and behavioral knowledge can be acquired entirely from data.
  • 6 Conclusion: The learned representations retain spatial structure under GPS-only input, with near-zero performance degradation.The conclusion contrasts this behavior with general-purpose LLMs, which collapse under the same input condition.
  • 6 Conclusion: The dataset currently covers four cities from one platform and captures only static route structures.The authors identify broader geography and real-time dynamics as natural extensions.
  • 6 Conclusion: The dataset reflects real-world demand through dense urban-core coverage and dispersion toward suburban areas.Figure 3 reports geographic origin distributions across the four cities rather than synthetic or uniform sampling.

B CPT Corpus Sample

The CPT corpus combines real-world route-planning sessions with static descriptions of transit lines and stations, representing queries, routes, topology, and spatial attributes in text.

  • B CPT Corpus Sample: The CPT corpus contains real-world session records pairing origin–destination requests with candidate routes, plus static line and station descriptions.The original corpus is Chinese, with English translations supplied for readability.
  • B CPT Corpus Sample: Each session specifies city, origin–destination GPS coordinates, POI names, and detailed candidate-route attributes.Route details include transport mode, line name, segment distance and time, fare, boarding and alighting stations, coordinates, and station IDs.
  • B CPT Corpus Sample: A representative Beijing session presents alternative subway and bus routes with walking access, transfers, times, distances, fares, and station sequences.The examples show both direct and transfer-based route options for one origin–destination query.
  • B CPT Corpus Sample: Static descriptions encode transit-line length, stop count, operating hours, fare policy, coordinates, and station connectivity.The examples include a line description and a station description with served lines and a connected route.

C Benchmark SFT Examples

The benchmark uses standardized prompt–label examples for optimal, preference-aware, and multi-route generation, with structured JSON outputs describing routes and their attributes.

  • C Benchmark SFT Examples: Each benchmark task pairs a standardized prompt containing a transit request with a structured JSON route label.Prompts use origin–destination coordinates and city information, while labels encode expected route fields.
  • C Benchmark SFT Examples: Optimal Route Generation produces one structured route from origin–destination coordinates.The output includes lines, stations and transfers, distance, time, fare, and first- and last-mile transfer details.
  • C Benchmark SFT Examples: Preference-Aware Planning adds a user preference that the generated route must satisfy while maintaining overall quality.The example specifies a bus-first preference for travel to Jingshan Park.
  • C Benchmark SFT Examples: The benchmark labels encode route sequences and numeric fields in JSON, including transfer markers and walking access information.The optimal-route example includes line and station sequences, total distance, time, fare, and start/end walking transfers.
  • C Benchmark SFT Examples: Multi-Route Generation produces three diverse routes for one origin–destination pair using different modes or line combinations.Each route is represented with tags, lines, stations, transfers, distance, time, fare, and access details.

D Evaluation Metrics

The evaluation framework measures structural validity, access plausibility, route similarity, numeric accuracy, preference compliance, and route diversity, with several metrics conditioned on prior success.

  • D Evaluation Metrics: The benchmark evaluates connectivity, access feasibility, route overlap, and numeric field accuracy, supplemented by task-specific preference and diversity metrics.These dimensions establish a multi-axis protocol for assessing generated transit routes.
  • D.1 Connectivity: Connectivity counts routes whose consecutive station pairs are reachable on a shared line or through a valid transfer.It is a prerequisite because most subsequent metrics are computed only on connected samples.
  • D.2 Access Feasibility: Access Feasibility checks whether first- and last-mile segments jointly make the origin-to-destination path connected and physically accessible.The plausibility rule requires predicted access distance to be at most three times the straight-line distance.
  • D.2 Access Feasibility: Station Grounding evaluates whether GPS coordinates are mapped implicitly to nearby boarding and alighting stations using mode-specific thresholds.The thresholds are 3 km for walking, 5 km for cycling, and 10 km for taxi.
  • D.3 Route Overlap: Route Overlap compares predicted and ground-truth route structure using Line Overlap and Station Sequence Overlap based on IoU.For multi-route generation, only the first predicted route is compared with the ground-truth route.
  • D.4 Numeric Field Accuracy: Numeric Field Accuracy evaluates distance, time, and fare only on samples achieving Route Exact Match.Estimation Accuracy uses tolerance-based pass/fail judgments, while MAPE measures continuous error magnitude.
  • Task-Specific Metrics: Preference Compliance applies hard rules for user preferences, while Route Diversity measures structural dissimilarity among generated alternatives.Shortest-time compliance allows a 1.1 tolerance factor, and preference compliance can have a theoretical upper bound below 100%.
  • Validation Results: The CPT loss falls from above 1.0 to approximately 0.1 within the first 2k steps across all three backbone models.The reported ordering is Qwen3-4B < Qwen3-1.7B < Qwen3-0.6B in loss and aligns with downstream performance.

F.3 Effect of Continual Pre-Training

Continual pre-training improves map-free transit planning, especially when textual cues are removed, and the gains generalize across preference-aware and multi-route tasks. Tool-augmented systems remain competitive, but TransitLM approaches or exceeds them on route-structure metrics without external routing access.

  • Optimal Route Generation: SFT-only reaches 74.9% Route Exact Match with standard text, but its GPS-only Route Exact Match drops by 8.8 percentage points.Its Estimation Accuracy also drops by 21.8 percentage points and MAPE rises from 1.35% to 4.96%.
  • Data Scaling: Preference Compliance rises from 87.3% to 89.8% as CPT data increases, while Multi-Route Generation diversity rises from 0.507 to 0.545.Both tasks improve monotonically across CPT data volumes and metrics.
  • GPS-only Ablation: The 4B-Joint model reaches 51.8% GPS-only Route Exact Match on Preference-Aware Planning and 66.1% on Multi-Route Generation.Larger domain-specific models consistently outperform smaller ones under GPS-only input.
  • Tool-Augmented Comparison: Tool-augmented LLMs reach up to 74.4% Route Exact Match, but TransitLM attains 0.835 Line Overlap and 0.847 Station Sequence Overlap without external tools.TransitLM surpasses the best tool-augmented model on Station Sequence Overlap despite generating complete intermediate station sequences.

G Limitations and Future Work

TransitLM is limited by geographic, linguistic, temporal, and evaluation scope. Its static, China-only data and platform-specific construction constrain claims about real-time operation, broader generalization, and user experience.

  • Scope and Representation: The dataset covers four Chinese cities and uses Chinese text, so generalization to other topologies, transfer conventions, and languages remains unverified.Representing each station as a vocabulary token also makes geographic expansion incur linear vocabulary growth.
  • Scope and Representation: Nationwide China coverage would require roughly 1.8 million station tokens, imposing substantial memory and computational overhead.The paper identifies vocabulary compression or hierarchical encoding as needed for practical scaling.
  • Temporal Scope: The static network snapshot cannot reflect real-time congestion, temporary route changes, service suspensions, or newly opened stations and lines.Incorporating new entities currently requires retraining on data that includes them.
  • Data Provenance: The data comes from a single navigation platform, whose route-ranking strategy may not generalize to other platforms.The released corpus also contains isolated requests sampled from one calendar day without timestamps.
  • Evaluation Scope: Evaluation compares structures with routing-engine outputs but does not include real-trip validation or user-satisfaction assessment.This leaves practical travel experience outside the reported evaluation protocol.

I Qualitative Examples

Qualitative examples show one model producing coherent optimal, preference-constrained, multi-route, and GPS-only outputs for the same Beijing origin-destination pair. The examples illustrate route structure, preference compliance, diversity, and spatial grounding.

  • Optimal Route Generation: The model generates a two-segment subway route with one transfer, including station sequences, distance, time, fare, and first/last-mile access details.The example covers 21.4 km in 1 h 17 min at ¥5 and identifies walking access.
  • Preference-Aware Planning: With a “bus first” constraint, the model switches to a bus-only route using Bus 405 and Bus 1 Express.The route avoids all subway segments, demonstrating preference-conditioned selection.
  • Multi-Route Generation: For multi-route generation, the model produces three alternatives spanning subway, subway with cycling, and bus.Only the subway-plus-cycling alternative is visualized because of space constraints.
  • GPS-only Generation: Removing the textual query yields a route nearly identical to the text-conditioned result, including the same subway lines and transfer station.The example supports spatial grounding from GPS coordinates without textual cues.
Loading 2605.22355v1…