PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Model-Assisted Preparation of Transportation Management Plans: A Case Study with WisDOT WisTMP System

Zihao Sheng, Pei Li, Zilin Huang, Yen-Jung Chen, Yuhao Luo, Zhengyang Wan, Steven T. Parker, David A. Noyce, Sikai Chen

arXiv:2610.10650v1cs.CL

TL;DR

TMP preparation is labor-intensive and depends on practitioner expertise, yet LLM use for generating TMP content remains underexplored. This paper fine-tunes open-source models on historical WisTMP data and evaluates their generation of operational sections. Fine-tuning improves performance across models, although strategy over-generation and cost-estimation problems remain.

  • Problem

    TMP preparation is labor-intensive and practitioner-dependent, while LLM-assisted generation of TMP content remains underexplored.

    Source: §1 · p. 2

  • Method

    The study converts historical WisTMP documents into structured QA pairs and fine-tunes open-source LLMs across model families and scales for TMP content generation.

    Source: §1 · p. 2, §5 · p. 14

  • Results

    Fine-tuning improves all models by approximately 35–43 points on BLEU-4 and ROUGE metrics; Qwen2.5-7B-Instruct-1M achieves the highest overall average score, 56.4.

    Source: §5 · p. 14

  • Takeaways & Limitations

    The results demonstrate the feasibility and potential of LLM-assisted TMP preparation as a foundation for decision-support tools for transportation planning.

    Source: §5 · p. 14

    Models add unsupported justifications, while generated costs are often $0 or inaccurate because Sections 1–3 lack sufficient cost-estimation information.

    Source: §4.7 · p. 14

Abstract

from arXiv · show

Work zones are critical yet hazardous components of transportation infrastructure, requiring carefully designed Transportation Management Plans (TMPs) to ensure safety and mobility. However, TMP preparation remains labor-intensive and heavily dependent on practitioner expertise. This paper proposes a Large Language Model (LLM)-assisted framework to automate TMP content generation, leveraging the WisDOT WisTMP system as the application context. The framework fine-tunes multiple open-source LLMs across different model scales and deploys them locally to ensure data security. To support model training, we construct a domain-specific dataset from historical WisTMP documents by converting PDF files into structured question-answer pairs in JSON format. Experimental results show that fine-tuning significantly improves performance across standard text generation metrics. Further section-wise and strategy-level analyses reveal that, while LLMs achieve strong overall performance, they tend to over-generate strategies and struggle to produce project-specific justifications and accurate cost estimates. In addition, scaling from 7B/8B to 14B yields limited gains. These findings demonstrate the potential of LLMs to improve TMP preparation efficiency while highlighting remaining challenges in LLM-assisted TMP development. The source code and demo videos will be publicly available at https://zihaosheng.github.io/TMP-LLM/.

1. Introduction

Work zones pose safety and mobility risks, while preparing their TMPs requires practitioners to synthesize project information and expertise. The paper proposes fine-tuned LLMs to generate operational TMP sections and reports improved text-generation performance, alongside persistent strategy-generation challenges.

Introduction

  • TMP preparation is labor-intensive: engineers synthesize diverse project factors, rely heavily on practitioner expertise, and may spend days or weeks in review and revision.Work zones also create risks for motorists and construction workers.Source: §1 · pp. 1–2
  • Fine-tuning improves all evaluated models on BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L, while Qwen2.5-7B-Instruct-1M scores highest overall and 14B scaling adds marginal gains.Strategy-level analysis finds reasonable recall but a tendency to over-generate.Source: §1 · p. 2
  • The paper addresses limited LLM research on TMP preparation with a framework that generates operational sections from foundational project information.It constructs a domain-specific dataset from historical WisTMP documents and evaluates open-source LLMs across model scales.Source: §1 · p. 2
  • Fine-tuned models are deployed locally in an interactive demonstration system so practitioners can generate TMP content in real time while protecting sensitive project information.The system takes project information as input.Source: §1 · p. 2

2. Related Works

Prior work supports work-zone strategy analysis and broader transportation applications of LLMs, but offers limited help drafting project-specific TMP sections. The paper situates TMP generation within emerging LLM-assisted document-generation research.

2.1. Work Zone Management and TMP Practices

  • Work-zone tools and studies support strategy analysis and workflow standardization more than document generation, leaving a practical gap in drafting project-specific TMP sections.TMP drafting still depends on practitioner experience.Source: §2.1 · pp. 2–3

2.2. LLMs in Transportation

  • Transportation LLM research spans traffic analysis, autonomous driving, safety, planning, and decision support, while structured regulation-compliant document generation—especially TMP drafting—has received limited attention.The cited applications include traffic prediction and control, driving tasks, crash analysis, and planning support.Source: §2.2 · p. 3

2.3. LLM-Assisted Document and Report Generation

  • LLMs have been used for document generation in medicine, law, and construction compliance; this work extends that research to drafting TMPs.Figure 1 depicts a framework combining historical WisTMP data, model selection, fine-tuning, and local deployment.Source: §2.3 · p. 3, Figure 1 · p. 4

3. Methodology

The proposed workflow builds on historical WisTMP documents, fine-tunes selected open-source LLMs, and deploys them locally for inference.

Methodology

  • The framework begins by constructing a dataset from historical WisTMP documents.This is the first of four workflow stages.Source: §3 · p. 4
  • The workflow selects open-source LLMs across three model families and two parameter scales, then fine-tunes them on the constructed dataset.Model selection and fine-tuning are the second and third stages.Source: §3 · p. 4
  • The final stage locally deploys fine-tuned models for inference.The framework overview describes local deployment.Source: §3 · p. 4

3.1. Dataset Construction

The dataset links foundational WisTMP project information to operational sections through section-specific training examples. Its construction includes document cleaning, QA-pair generation, and characterization of the resulting corpus.

3.1.1. Data Source

  • Each TMP provides project context in Sections 1–3 and operational content in Sections 4–9; the study excludes supplementary Section 10.Training samples pair an input with a target output, using project sections as context and a target operational section as output.Source: §3.1.1 · p. 4

3.1.2. Data Corpus Extraction and Cleaning

  • PDFs are converted to Markdown and cleaned, including merging appropriate split table fragments and retaining finalized plans while removing review content and repetitive elements.The processing pipeline is summarized in Fig. 2.Source: §3.1.2 · pp. 4–5, Figure 2 · p. 5

3.1.3. QA Pair Construction

  • Each training input combines a target-section instruction with Sections 1–3 context, and its output is one corresponding section from Sections 4–9.A single TMP can yield six QA pairs; the instances are formatted as instruction, input, and output fields to support context-aware section generation.Source: §3.1.3 · p. 5

3.1.4. Dataset Statistics

  • 843 processed TMP records yielded 5,048 instruction–input–output triples across Sections 4–9.Most records fall in 2021–2023; among valid type labels, Type 2 represents 645 (80.8%). Output lengths vary by section, and Sections 7–8 have concentrated strategy distributions.Source: §3.1.4 · pp. 5–6, Figure 3 · p. 6

3.2. Model Selection

The study evaluates open-source LLMs from the LLaMA, Qwen, and DeepSeek families at 7B/8B and 14B scales. This selection supports local deployment, scale and cross-family comparisons, and reproducibility.

3.2. Model Selection

  • The evaluation spans LLaMA, Qwen, and DeepSeek models at 7B/8B and 14B scales.Table 1 summarizes the models selected for fine-tuning.Source: §3.2 · p. 6, Table 1 · p. 7
  • LLaMA 3.1-8B-Instruct serves as a representative baseline from the widely adopted LLaMA ecosystem.It is instruction-tuned, pretrained on a large multilingual corpus, and fine-tuned with reinforcement learning from human feedback (RLHF).Source: §3.2 · p. 6
  • Qwen 2.5-7B/14B-Instruct-1M supports context lengths up to one million tokens, and both scales enable within-family scale comparisons.The long context is noted as advantageous for processing extensive project descriptions and tabular data in TMP documents.Source: §3.2 · p. 7
  • Qwen3-14B is included to compare model generations within the Qwen family at the 14B scale.The paper describes it as incorporating architectural improvements and expanded pre-training data relative to Qwen 2.5.Source: §3.2 · p. 7
  • DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Qwen-14B test whether distilled reasoning capabilities help structured TMP generation.They transfer capabilities from DeepSeek’s 671B-parameter R1 reasoning model into more compact architectures.Source: §3.2 · p. 7
  • The selection prioritizes freely available open-source models for reproducibility and local deployment while enabling scale and cross-architecture comparisons.The family diversity is intended to mitigate architecture-specific biases in the evaluation.Source: §3.2 · p. 7

3.3. Fine-Tuning with Low-Rank Adaptation

The paper uses LoRA to adapt large language models without updating their full pretrained weight matrices. Trainable low-rank adapters are applied across attention and feedforward projections, using causal language-modeling loss.

3.3. Fine-Tuning with Low-Rank Adaptation

  • LoRA freezes pretrained weights and represents fine-tuning updates with trainable low-rank matrices instead of updating the full weight matrix.The approach is motivated by the stated insight that fine-tuning weight updates exhibit low intrinsic rank.Source: §3.3 · p. 7
  • Only matrices A and B are updated, reducing trainable parameters from d × k to r × (d + k).The rank r is much smaller than d and k; the update is scaled by α/r, with α controlling adaptation magnitude and scaling intended to stabilize its contribution across ranks.Source: §3.3 · p. 7
  • LoRA adapters are applied to attention query, key, value, and output projections and to feedforward up- and down-projections.Training minimizes the negative log-likelihood of target tokens given the input context; θ denotes trainable LoRA parameters, and y<t denotes preceding output tokens.Source: §3.3 · p. 7

3.4. Local Deployment

The system deploys fine-tuned models locally to protect potentially sensitive TMP project information. Its demonstration interface accepts project inputs and target-section requests, with configurable generation parameters.

3.4. Local Deployment

  • Fine-tuned models run on institutional hardware rather than external cloud APIs to prevent leakage of potentially sensitive TMP project information.At inference, the merged base model and LoRA adapters process project information from Sections 1–3 and generate the requested operational section.Source: §3.4 · p. 8
  • The Gradio demonstration lets users provide project information, choose target sections, and configure maximum output tokens, temperature, and top-p sampling.It also supports image uploads for potential future extensions.Source: §3.4 · p. 8

4. Results and Evaluation

Fine-tuning substantially improves overall TMP-generation scores, but section-specific difficulty and strategy over-generation persist. Larger 14B models yield only marginal gains, and generated justifications and cost estimates can exceed what the project inputs support.

4.1. Experimental Setup

  • Evaluation uses four widely used text-generation metrics assessing overlap between generated TMP content and ground-truth references, plus strategy-level recall and precision for Sections 4, 7, and 8.Models were fine-tuned for 15 epochs using LoRA on a 9:1 training/test split.Source: §4.1 · p. 8

4.2. Overall Performance Comparison

  • Fine-tuning raises all models’ average scores by approximately 35–43 points; the top five post-tuning models fall within a 1.3-point range.Qwen2.5-7B-Instruct-1M scores 56.4, while its 14B counterpart scores 56.3; no 14B model surpasses the best 7B/8B model.Source: §4.2 · pp. 9–10, Table 2 · p. 9, Figure 4 · p. 9

4.3. Section-Level Performance Analysis

  • Section 8 gains over 35–49 BLEU-4 points after fine-tuning, while Section 4 remains lowest-scoring post-tuning and Section 5 reaches 62.3.Section 8 was the weakest before fine-tuning; the section-level ROUGE-L rankings largely corroborate the BLEU-4 patterns.Source: Table 3 · p. 10, Figure 5 · p. 10, §4.3 · p. 11

4.4. Strategy-Level Performance Analysis

  • Across models, strategy recall exceeds precision; Qwen3-14B leads with 51.75% recall and 47.51% precision, while Section 4 recall is generally below 41%.The results indicate coverage of many reference strategies alongside redundant generations; project-specific strategy diversity is a likely challenge.Source: §4.4 · p. 11, Table 4 · p. 11

4.5. Qualitative Case Study

  • In two Section 4 cases, the model recovered all three strategies in one project but recovered 14 of 16 and added five unmatched strategies in another.One unmatched prediction selected Reduced Shoulder Width where the reference specified Reduced Widths, illustrating confusion between closely related strategy types.Source: §4.5 · pp. 12–13, Table 5 · p. 12, Table 6 · p. 12

4.6. Inference Efficiency Analysis

  • All 7B/8B models generate Sections 4–9 in under one minute, whereas 14B models take 91–103 seconds; longer outputs generally take longer.The largest model still generates the operational sections in under two minutes, compared with a conventional manual process described as days or weeks.Source: §4.6 · p. 13, Table 7 · p. 13

4.7. Error Analysis and Limitations

  • The error case includes unsupported explanatory comments and inaccurate costs: generated estimates include $2,000 and $50,000 against reference costs of $500–$1,000.The authors attribute cost-estimation difficulty to insufficient project-specific input and note that external cost databases would likely be needed.Source: Table 8 · p. 13, §4.7 · p. 14

5. Conclusion and Future Work

The paper develops and evaluates a fine-tuned LLM framework for TMP preparation, finding substantial gains but persistent challenges in project-specific generation and cost estimation.

5. Conclusion and Future Work

  • The framework builds 5,048 structured question–answer pairs from 843 historical TMP documents and fine-tunes six open-source LLMs with LoRA.The evaluation covers LLaMA, Qwen, and DeepSeek models at 7B/8B and 14B scales.Source: §5 · p. 14
  • Fine-tuning improves average BLEU-4 and ROUGE scores by approximately 35–43 points, and Qwen2.5-7B-Instruct-1M achieves the highest overall average score, 56.4.The reported average-score gains span all evaluated models.Source: §5 · p. 14
  • Scaling from 7B/8B to 14B yields only marginal gains, while strategy generation achieves reasonable recall but tends to over-generate.Incident Management Strategies show the largest absolute gains from fine-tuning.Source: §5 · p. 14
  • The framework generates TMP sections independently and struggles with project-specific justifications and accurate cost estimates.The paper identifies human evaluation, external project and cost data, larger models, and multimodal inputs as directions for future work.Source: §5 · p. 14
  • The work demonstrates the feasibility and potential of LLM-assisted TMP preparation as a foundation for transportation-planning decision-support tools.The stated potential includes improving efficiency, consistency, and scalability.Source: §5 · p. 14

Appendix A. TMP Example for WisTMP

The appendix presents a representative WisTMP example across three pages, with sensitive and identifying information removed.

Appendix A. TMP Example for WisTMP

Appendix B. Local Deployment Interface

The appendix shows the locally deployed demonstration interface, an interactive environment for TMP content generation.

Appendix B. Local Deployment Interface

Related papers

Loading 2610.10650v1…