PaperlayerView exact revision on arXiv

Source-linked AI summary

Broadening Access to Transportation Safety Data with Generative AI: A Schema-Grounded Framework for Spatial Natural Language Queries

Mahdi Azhdari, Eric J. Gonzales

arXiv:2605.21712v1cs.CL

TL;DR

Transportation safety data analysis remains difficult to access for stakeholders without GIS and database expertise, while public-sector use of generative AI requires reproducible, trustworthy execution. The paper introduces a schema-grounded natural-language interface with deterministic validation and spatial execution; all 80 queries succeeded, and 29% required correction before execution.

  • Problem

    Transportation safety analysis depends on GIS, database, and spatial-analysis expertise, limiting direct access for many agencies and community stakeholders while requiring trustworthy, reproducible results.

  • Method

    The framework uses an LLM to translate queries into semantic frames, validates and repairs them against a domain schema, then executes typed spatial operations deterministically.

  • Results

    29% of queries required validation-layer correction, while all 80 queries executed successfully and validated frames matched ground truth after repair.

  • Takeaways & Limitations

    Separating language interpretation from execution offers a practical design principle for broadening access to transportation safety data while retaining schema-grounded, reproducible analysis.

  • Takeaways & Limitations

    Scaling to new jurisdictions requires adapting schemas and likely retuning validation, with the needed automation and domain expertise remaining open questions.

Abstract

from arXiv · show

Transportation safety analysis requires integrating crash records, roadway attributes, and geospatial data through GIS-based workflows, but access remains uneven across agencies and community stakeholders. Technical prerequisites create a gap between analytical tools central to safety planning and the practitioners able to use them. Local agencies, school committees, and residents may have safety concerns but limited capacity to retrieve, filter, map, and analyze relevant data. Generative AI offers a way to narrow this divide, but its public-sector use raises questions about reliability, reproducibility, and governance. This paper presents a schema-grounded natural language interface for transportation safety analysis, using a large language model (LLM) to interpret user intent while preserving deterministic, reviewable execution against an authoritative database. User queries are translated into structured semantic frames, validated by a rule-based layer, compiled into a typed directed acyclic graph of spatial operations, and executed against a PostGIS database. This bounded design separates language interpretation from deterministic execution, keeping results reproducible and schema-grounded while removing access barriers. The framework is evaluated using a statewide Massachusetts transportation safety database integrating crash records, roadway attributes, and geospatial layers including schools, bus stops, crosswalks, and municipal boundaries. All queries executed successfully; the validation layer corrects errors in 29% of evaluation queries, reflecting the gap between flexible natural language and strict schema-grounded requirements. The results suggest that combining natural language accessibility with deterministic execution is a practical direction for broadening access to transportation safety data, with implications for trustworthy AI in public-sector planning.

1. Introduction

Transportation safety analysis combines crash, roadway, infrastructure, and spatial data but remains difficult for users without GIS and query-language expertise. This paper presents a schema-grounded natural-language interface that interprets user intent while preserving validated, reproducible execution against an authoritative spatial database.

  • Problem: Transportation safety analyses support risk identification, condition assessment near schools and transit stops, jurisdictional comparisons, and safety-resource allocation.These workflows combine crash records, roadway and infrastructure data, and spatial methods.
  • Motivation: GIS and query-language requirements create an access barrier for agencies and stakeholders seeking to conduct transportation safety analysis.The supplied passage identifies technical familiarity with GIS platforms as a practical prerequisite.
  • Contribution: The framework enables plain-language analytical intent while maintaining schema-grounded, reproducible, and consistent safety-analysis workflows.The design addresses institutional requirements beyond merely making safety data queryable.
  • Contribution: The proposed interface translates natural-language queries into semantic frames, validates and corrects them against a domain-specific schema, and compiles them into a typed spatial-operation DAG.An LLM serves as a controlled interpretation layer, while execution occurs against an authoritative spatial database.
  • Method: The system is developed and evaluated with a statewide Massachusetts database integrating crash records, roadway attributes, schools, bus stops, crosswalks, and municipal boundaries.It produces interactive maps, ranked tables, and exportable datasets across multiple safety-analysis contexts.

2. Background and Related Work

Transportation safety analysis increasingly depends on integrated GIS workflows, yet technical, data-integration, and organizational barriers limit access for smaller agencies and community stakeholders. Existing LLM-enabled spatial systems broaden access but do not consistently address transportation safety’s specialized requirements or ensure reproducible, traceable execution.

  • Transportation safety analysis: Federal safety programs require agencies to identify crash problems, prioritize interventions, and evaluate outcomes using integrated crash, roadway, and geospatial data.Systemic safety approaches also examine elevated-risk roadway characteristics across broader networks, supported by hotspot, proximity, and infrastructure-linked GIS methods.
  • Access barriers: Persistent barriers in technical expertise, data integration, and organizational capacity restrict GIS-based safety analysis, especially for smaller agencies and local stakeholders.Community groups, neighborhood advocates, and residents also seek transportation safety information despite limited conventional GIS capacity.
  • Related LLM and GIS work: LLM-enabled text-to-SQL, spatial databases, and GIS systems increasingly use natural language to broaden access, reduce coding requirements, and automate parts of spatial workflows.Prior transportation applications have largely emphasized traffic operations, autonomous systems, prediction, and simulation.
  • Domain-specific limitations: General-purpose query systems are not typically designed to consistently handle transportation safety’s domain-specific entities, geographic conventions, proximity screening, infrastructure filters, and program-specific temporal analysis.These specialized requirements distinguish safety analysis from broader natural-language database and spatial-analysis applications.
  • Trustworthy public-sector AI: Direct code generation and agentic execution can introduce nondeterminism, reduce reproducibility, and propagate errors, making schema conformance, validation, and interpretable execution governance decisions.Outputs must be correct, traceable, verifiable, and consistent with institutional and user-reliant definitions, standards, and data practices.
  • Paper contribution: This paper addresses the gap with a bounded, domain-specific generative-AI interface for structured transportation safety analysis, serving users with safety questions but limited conventional GIS capacity.The framework is implemented on a statewide Massachusetts transportation safety database and evaluated across supported analytical operations.

3. System Architecture

The system converts natural-language transportation safety queries into validated semantic frames, typed spatial workflows, and reproducible PostGIS analyses. Its architecture separates LLM-based interpretation from rule-based governance and deterministic execution.

  • System workflow: The pipeline transforms natural-language queries into semantic frames, validates them, compiles typed spatial operations, and executes them against a database.This separates language interpretation from analytical execution.
  • Data implementation: The implementation uses a statewide Massachusetts PostGIS database integrating crash records, roadway attributes, and geospatial infrastructure layers.Crash attributes include severity, harmful-event type, time, date, junction type, and sidewalk status.
  • Language interpretation: The LLM exclusively interprets queries using a structured prompt that defines supported entities, fields, values, spatial relationships, operators, and roles.The resulting JSON semantic frame represents user intent through typed entities and constraints.
  • Validation and repair: The Validation and Repair Layer converts approximate semantic frames into schema-conformant representations through validation, normalization, anchor resolution, and structural correction.It canonicalizes natural-language values, resolves geographic references, and repairs incomplete or inconsistent analytical relationships.
  • Structured execution: Validated frames compile into typed directed acyclic graphs whose explicit dependencies provide a reproducible and auditable path from analytical intent to PostGIS output.Graph checks ensure referenced nodes exist, the graph is acyclic, and final nodes represent valid output operations.

4. Evaluation

Within its bounded Massachusetts transportation safety schema, the system evaluated 80 natural-language queries across supported retrieval, spatial, filtering, temporal, relationship, and ranking capabilities. All queries executed successfully, validated frames matched ground truth after repair, and 23 queries required correction before execution.

  • Evaluation design: 80 natural-language queries tested nine groups spanning entity retrieval, spatial scoping, attribute and temporal filters, spatial relationships, and ranking tasks.The groups represented different combinations of supported analytical capabilities.
  • Evaluation design: Ground truth specified expected entity roles, spatial relationships, attribute filters, temporal constraints, and ranking parameters, enabling intent-completeness and execution evaluation.Ground truth represented the validated semantic frame expected after interpretation, validation, and repair.
  • Overall results: All 80 queries executed successfully, and every validated semantic frame matched its ground truth after validation and repair.The results were interpreted within the system’s bounded operational scope and summarized by query group in Table 2.
  • Validation and repair: 23 of 80 queries (29%) required validation-layer correction before execution, with 22 of 25 individual repairs involving value normalization.The remaining three repairs were structural: removing a spurious anchor reference and consolidating a duplicate attribute constraint.
  • Runtime: LLM interpretation took approximately 2–3 seconds per query, while simple retrieval and filtering queries completed within 2–22 seconds.Remaining runtime was driven mainly by database computation, query structure, and spatial scope.
  • Runtime: Infrastructure ranking reached 53 seconds, town-level aggregation averaged 62 seconds, and road segment ranking reached 142 seconds for crash joins over municipal road networks.These runtimes reflect the cost of the underlying spatial operations.

5. Discussion

The framework supports both site-level safety investigations and broader comparative planning through schema-grounded spatial queries. Its bounded, deterministic design improves reproducibility while leaving institutional definitions, interpretation, schema expansion, and jurisdictional adaptation as continuing challenges.

  • Local analysis: Technical barriers limit smaller agencies’ ability to navigate safety datasets and document needs for planning, advocacy, and funding opportunities.Communities with greater safety needs may also be least equipped to compete for investment.
  • Local analysis: Site-level queries produce location-centered maps and filtered crash records for schools, transit stops, intersections, and other places of immediate community concern.Examples include pedestrian crashes within specified distances of Amherst Regional High School, Amherst Center, or a bus stop.
  • Comparative planning: Broader planning queries rank schools, bus stops, municipalities, and road segments using crash, spatial, infrastructure, roadway, temporal, and facility-type filters.Examples include ranking towns or schools by crashes within specified distances and time periods, or near bus stops with speed limits above 30.
  • Comparative planning: Ranking currently relies primarily on crash counts under user-specified constraints, but agencies can adapt the semantic structure to weighted severity or organization-specific prioritization criteria.Consistent schema definitions and deterministic execution preserve reproducibility across users and sessions.
  • Trustworthiness and governance: The bounded design addresses risks involving reproducibility, auditability, and hallucination propagation, while deployment still depends on how risk and priority are defined and outputs interpreted.Public-sector concerns also include over-reliance, bias, and data privacy.
  • Limitations and extensions: Extending the framework requires modular additions to its analytical vocabulary and schema adaptation, with reliability in new jurisdictions depending on underlying data-model quality and validation tuning.Potential additions include exposure-adjusted metrics, severity-weighted ranking, network accessibility measures, temporal aggregation, and additional data layers.

6. Conclusion

The framework aims to broaden access to transportation safety analysis by providing a natural-language interface for stakeholders who face barriers to GIS and database workflows. It separates language interpretation from deterministic analytical execution, with validation correcting 29% of evaluated queries before execution.

  • Access: The natural-language interface is designed to narrow access gaps for stakeholders with limited capacity to use GIS platforms, databases, and spatial analysis tools.The targeted stakeholders include local agencies, school committees, community organizations, and planners with clear safety questions.
  • Architecture: The system separates language interpretation from execution by translating queries into semantic frames, validating and correcting them rule-by-rule, and compiling them into deterministic spatial operations.Execution occurs against a statewide Massachusetts crash and infrastructure database.
  • Evaluation: 29% of evaluation queries required repair before execution could proceed, making correction the framework’s most concrete evaluation finding.The repair rate indicates that nearly a third of queries needed correction before execution.

Appendix A. Semantic Frame Schema and Example

The appendix defines a semantic frame as a structured JSON representation of a query’s complete analytical intent before execution. It illustrates the schema with a validated Boston school-ranking query encoding entities, spatial and attribute constraints, and ranking instructions.

  • Semantic frame schema: Each natural-language query is converted into a structured JSON semantic frame that captures its full analytical intent before execution.Table A.4 summarizes the frame’s six top-level schema components.
  • Validated example: The example query ranks the top five schools by pedestrian crashes within 500 meters in Boston.The validated frame uses School as the primary target, Crash as support, and Town as scope, with Boston specified as the reference town.
  • Validated example: A within_distance constraint sets the support-to-primary relationship at 500.0 meters.The frame assigns target_role support, reference_role primary, and distance_m 500.0.
  • Validated example: An attribute constraint filters support records where first_hrmf equals “Collision with pedestrian.”The constraint applies operator eq to the value “Collision with pedestrian.”
  • Validated example: The ranking specification counts crashes, orders results highest first, and limits output to the top 5.It sets metric crash_count, target_role primary, order highest, and top_n 5.

Appendix B. Supported Entities and Schema

The appendix defines the schema supported by the system, covering six entity types from the statewide Massachusetts transportation safety database. It also specifies canonical values for crash severity and first harmful events.

  • Supported entities: The system supports six entity types from the statewide Massachusetts transportation safety database, each defined by geometry type and key analytical fields.Table 1 summarizes the supported entities and their schema characteristics.
  • Crash attributes: Crash severity uses four canonical values: Property damage only, Non-fatal injury, Fatal injury, and Unknown.These values are defined as the supported crash-severity categories.
  • Crash attributes: The first harmful event field supports 30 canonical categories, including collisions with pedestrians, cyclists, motor vehicles, fixed objects, and animals.The categories are drawn directly from the Massachusetts crash reporting standard.

Appendix C. Validation and Repair: Normalization Examples

The Validation and Repair Layer normalizes free-form natural-language expressions into canonical schema values before execution. It then resolves geographic anchors to verified coordinates and requires user selection when place names produce multiple candidates.

  • Normalization: The Validation and Repair Layer maps free-form natural-language expressions to canonical schema values before execution.Representative corrections applied during evaluation are shown in Table C.5.
  • Geographic anchor resolution: School and place names are resolved to verified spatial coordinates through database lookup or geocoding before execution proceeds.Geographic anchor resolution follows normalization.
  • Geographic anchor resolution: When multiple candidate locations are found, the system presents the options for user selection rather than executing an ambiguous reference.The system does not proceed until the ambiguous place reference is resolved.

Appendix D. Query Groups and Representative Prompts

Appendix D organizes the evaluation into nine query groups representing combinations of analytical capabilities and provides representative prompts for each group.

  • Query Groups: Nine query groups represent the analytical capability combinations evaluated.Table D.6 summarizes these groups and the combinations each represents.
  • Capability Definitions: The appendix defines Ret. as entity retrieval, Attr. as attribute filtering, Temp. as temporal filtering, and Comp. as combined multi-constraint queries.These abbreviations identify the analytical capabilities used to characterize query groups.
  • Representative Prompts: Table D.7 provides representative evaluation prompts for each query group.The prompts illustrate the groups used in the evaluation.
Loading 2605.21712v1…