Source-linked AI summary
Crash Narrative-Guided Countermeasure Recommendation Using Large Language Models: A Retrieval-Augmented Generation Framework for Intersection Safety
Abu Saif Md Nasim Uddin, Mohamed Abdel-Aty, Zubayer Islam, Parvez Anowar, Chenzhu Wang
TL;DR
Intersection countermeasure selection is labor-intensive and expert-dependent, while crash narratives contain underused mechanism information. The paper develops a narrative-guided RAG framework that links extracted mechanisms to evidence-based treatments and retrieves similar intersections. On the evaluated Florida dataset, it achieved precision 0.82, recall 0.85, and F1-score 0.82, supporting its potential as a scalable decision-support tool.
Problem
Intersection countermeasure selection relies on labor-intensive expert judgment, while narrative descriptions of crash mechanisms remain underused in safety analysis.
Method
The framework extracts crash mechanisms, links them to evidence-based treatments, and combines similar-intersection retrieval, association rules, count statistics, and engineering reasoning.
Results
Precision was 0.82, recall was 0.85, and F1-score was 0.82 in the evaluated framework.
Takeaways & Limitations
The framework demonstrates potential as an interpretable and scalable tool for translating crash narratives into site-specific intersection countermeasure recommendations.
Takeaways & Limitations
Evaluation covered a limited number of intersections within two counties, so performance may vary in other settings.
Abstract
from arXiv · showhide
Improving safety at intersections requires identifying crash mechanisms and recommending appropriate countermeasures. However, this process traditionally relies on expert judgment, making it labor-intensive, difficult to scale, and dependent on the availability of experienced traffic safety engineers. Although crash narratives contain rich description of crash mechanisms, this unstructured information remains largely underutilized in safety analyses. This study presents a crash narrative-guided retrieval-augmented generation (RAG) framework that translates narrative-derived crash mechanisms into site-specific countermeasure recommendations. Key mechanism attributes including traffic control, signal indication, driver fault, vehicle movement, and travel direction were extracted from crash narratives and linked to evidence-based treatments from the FHWA Proven Safety Countermeasures and the CMF Clearinghouse. The framework integrates embedding-based retrieval of historically similar intersections, association-rule mining, statistical guidance on the expected number of relevant countermeasures, and an engineering reasoning guidance that directs LLM through a domain-consistent decision process before selecting countermeasures. Evaluated on 312 fatal and serious-injury crashes across 115 intersections in Lake and Sumter Counties, Florida, using five-fold cross-validation, the framework achieved a precision of 0.82, recall of 0.85, and F1-score of 0.82, while recommending an average of 3.91 countermeasures per location with 3.14 matching, closely matching the actual average (3.86). Overall, the proposed framework demonstrates the potential of retrieval-augmented LLMs as an interpretable and scalable decision-support tool for transportation agencies for translating crash narratives into countermeasure recommendations.
1 Introduction
Intersection safety analysis is constrained by complex crash interactions, labor-intensive expert review, and underuse of narrative information. This study addresses these gaps with a narrative-guided RAG framework for site-specific countermeasure recommendations.
- Motivation: 28% of U.S. traffic fatalities during 2018–2022 occurred at or were related to intersections, with 66% at unsignalized intersections.Intersections are complex because turning, through, pedestrian, and bicyclist movements interact.
- Problem: Traditional countermeasure selection depends on traffic safety engineers and is resource-intensive, difficult to scale, and dependent on qualified personnel.The process combines crash analysis, safety references, and professional judgment.
- Motivation: Crash narratives capture vehicle movements, driver actions, traffic control, and pre-crash circumstances that coded records often simplify or omit.Narratives complement coded data and crash diagrams by preserving richer crash-mechanism information.
- Problem: Manual narrative review is labor-intensive, while traditional NLP often struggles to capture the semantic and contextual meaning of complex crash descriptions.LLMs offer stronger contextual understanding but can generate unsupported or unfaithful outputs without grounding.
- Contributions: The framework extracts traffic control, signal indication, driver fault, vehicle movement, and travel direction, then links these mechanisms to evidence-based treatments.It builds a location-level knowledge base for 115 Florida intersections using FHWA Proven Safety Countermeasures and the CMF Clearinghouse.
- Contributions: The proposed RAG framework retrieves historically similar intersections to generate site-specific countermeasure recommendations.Its evaluation and organization cover framework development, performance across crash contexts, and implications for practice.
2 Literature Review
Prior research increasingly extracts structured information from crash narratives and applies LLMs across transportation tasks, but rarely translates narrative mechanisms into site-specific intersection treatments. This literature gap motivates countermeasure recommendation as an engineering decision-support application.
- Crash Narratives in Traffic Safety: Earlier crash-narrative studies were largely descriptive, using manual review or keyword-based approaches to identify recurring hazards and crash determinants.Examples include work-zone narratives and military vehicle crash reports.
- Crash Narratives in Traffic Safety: More recent studies use NLP methods such as keyword extraction and named entity recognition to identify vehicle types, directions, violations, hazardous actions, and collision positions.These approaches support structured information extraction from narrative text.
- Crash Narratives in Traffic Safety: No previous work in the reviewed literature leveraged narrative-derived crash mechanisms to recommend site-specific intersection countermeasures.Effective selection requires translating crash mechanisms into treatments intended to prevent similar crashes.
- Countermeasure Recommendation: Countermeasure research has more often evaluated treatment effectiveness than addressed the process of generating or recommending interventions.Existing decision-support efforts include multicriteria prioritization and knowledge-based links between crash causes and countermeasures.
- LLMs in Transportation Safety: LLMs are increasingly applied to transportation engineering, including travel behavior, autonomous driving, traffic management, planning, and crash analysis.Their use in safety research has expanded from crash analysis toward broader engineering decision support.
- LLMs in Transportation Safety: The literature presents LLMs as a promising foundation for intelligent safety decision-support systems beyond crash analysis.The specific extension to countermeasure recommendation remains underdeveloped in the reviewed work.
3 Methodology
The methodology converts narrative-described crash mechanisms into standardized, location-specific countermeasure evidence and uses retrieval-augmented generation to support recommendations. It combines narrative processing, a structured training knowledge base, historically similar intersections, association rules, and countermeasure-count guidance.
- Narrative Processing and Countermeasure Knowledge Base Development: 312 crashes across 115 intersections were organized into narrative-derived mechanisms, corrected structured attributes, and location-level safety interventions.The study extracted vehicle movements, travel directions, signal indications, traffic-control context, and contributing factors, then used narratives to verify or correct recorded fields.
- Narrative Processing and Countermeasure Knowledge Base Development: Crash narratives were interpreted to identify recurring conflict patterns and synthesize dominant crash mechanisms at each intersection before selecting candidate treatments.The process linked narrative findings to countermeasures from the CMF Clearinghouse and other evidence-based sources.
- Training Data Preparation: The 115 locations were split into 92 training and 23 test intersections to build retrieval, standardized countermeasure, and association-rule knowledge components.Training data included concatenated crash narratives and structured descriptors, while the master countermeasure list reduced wording inconsistencies.
- Retrieval-Augmented Generation (RAG): Embedding-based retrieval jointly represented narratives and structured descriptors so historically similar intersections could be identified despite differences in wording.Retrieved locations supplied empirical countermeasure packages and crash characteristics for later augmentation.
- Training Data Preparation: Association rules supplied feature-linked countermeasure evidence, while lift captured localized relationships complementary to globally similar-location retrieval.Lift used the overall countermeasure probability as its reference and higher lift indicated stronger-than-chance association.
- Retrieval-Augmented Generation (RAG): RAG augmented the crash query with retrieved locations, rule-based hints, and countermeasure-count statistics before generating recommendations grounded in external evidence.Count statistics guided the approximate package size rather than determining which countermeasures must be selected.
4 Results and Discussion
The framework’s full evaluation compares recommendation methods and language models, then examines how individual components affect performance. Results show strong matching coverage, with association rules and count guidance especially influential in balancing relevance and recommendation-set size.
- Framework Performance: The RAG framework identified at least one correct countermeasure for all 22 test locations and at least three for 15 of 17 eligible locations.It achieved two or more correct matches for 21 locations, four or more for 7 of 10 eligible locations, and at least five for 3 of 9 eligible locations.
- Framework Performance: 0.82 precision, 0.85 recall, and 0.82 F1-score were achieved by GPT-4o mini within the RAG framework.GPT-4o mini had the strongest overall performance across nearly all evaluation metrics.
- Framework Performance: Llama 3.3 70B achieved a 0.75 F1-score, while DeepSeek V4 Pro had comparable precision but lower recall and F1-score.All three models identified at least one correct countermeasure for every evaluated location; Llama slightly exceeded GPT-4o mini on locations with three or more correct matches.
- Ablation Study: Removing association rules reduced F1-score from 0.82 to 0.65, with precision and recall declining to 0.67 and 0.66, respectively.This was the largest degradation among the ablations, indicating that statistically mined associations supplied substantial recommendation guidance.
- Ablation Study: Removing the three-gate selection mechanism reduced F1-score to 0.71 and average matched countermeasures from 3.14 to 2.64.Reranking and few-shot examples produced more modest F1 reductions to 0.73 and 0.74, while removing engineering reasoning caused only a marginal predictive change.
- Ablation Study: Disabling countermeasure-count guidance increased recommendations from 3.91 to 5.45 and raised recall to 0.93, but reduced precision to 0.63.The result illustrates a precision–recall trade-off and the role of count guidance in constraining recommendations to a focused set.
5 Conclusion
The study presents a crash narrative-guided RAG framework that converts extracted crash mechanisms into site-specific intersection countermeasure recommendations. Evaluated across 115 Florida intersections, it achieved strong recommendation performance while producing countermeasure counts close to observed values, but its broader applicability remains geographically limited.
- 5 Conclusion: Precision was 0.82, recall was 0.85, and F1-score was 0.82 across 115 intersections evaluated with five-fold cross-validation.The evaluation used intersections in Lake and Sumter Counties, Florida.
- 5 Conclusion: The framework extracted crash mechanisms from narratives and linked them with established safety treatments through retrieval, association rules, and engineering-guided reasoning.Mechanisms included traffic control, signal indication, driver fault, vehicle movement, and travel direction.
- 5 Conclusion: It matched an average of 3.14 countermeasures per location while predicting 3.91, closely aligning with the actual average of 3.86.These results indicate close agreement between recommended countermeasure counts and observed counts.
- 5 Conclusion: The framework can support more systematic, scalable countermeasure selection and convert unstructured crash reports into structured engineering knowledge.Its recommendations are described as context-specific and consistent with accepted safety practices.
- 5 Conclusion: Performance may vary across regions because the evaluation covered a limited number of intersections within only two counties.The authors recommend expanding evaluation across diverse jurisdictions and adding contextual factors such as traffic volume and roadway geometry.