PRISM: An Agentic Multi-Model Architecture for Proactive Safety in Autonomous Transportation Systems

Joyjit Roy; Samaresh Kumar Singh; Sushanta Das

Source-linked AI summary

PRISM: An Agentic Multi-Model Architecture for Proactive Safety in Autonomous Transportation Systems

Joyjit Roy, Samaresh Kumar Singh, Sushanta Das

arXiv:2609.01623v1Jul 29, 2026cs.MAcs.CVcs.ETcs.LGphysics.soc-ph

TL;DR

Reactive ADAS and fixed thresholds provide limited context sensitivity in complex urban traffic involving vehicles and VRUs. PRISM addresses this gap with parallel risk models and an agentic reasoning layer, achieving cross-dataset safety assessment while retaining important deployment and evaluation limitations.

Problem
Reactive ADAS activates after hazards emerge and uses fixed thresholds that do not adapt to complex urban conditions involving vehicles, environments, and VRUs.
Method
PRISM combines environmental, trajectory-kinematic, and VRU-interaction models with reinforcement learning, memory, and SHAP explanations to produce continuous, graduated safety decisions.
Results
PRISM achieved a mean safety score of 68/100 across 1,296 scenarios from three datasets without dataset-specific retraining, with 77.6% classified as advisory.
Takeaways & Limitations
PRISM provides a unified, interpretable framework for proactive safety assessment, with VRU risk identified as the primary decision driver.
Takeaways & Limitations
The evaluation uses only ten nuScenes scenes for adverse conditions, lacks annotated near-miss ground truth, and has approximately 596 ms CPU latency that prevents hard real-time deployment.

Abstract

from arXiv · show

Autonomous and intelligent transportation systems operate in complex urban environments where safety depends on interactions among vehicle behavior, environmental conditions, and vulnerable road users (VRUs) such as pedestrians and cyclists. Most advanced driver assistance systems (ADAS) employ reactive mechanisms that activate only after hazards have emerged, a critical limitation underscored by rising VRU fatalities in the United States. This study introduces PRISM (Proactive Risk Intelligence and Safety Management), an agentic multi-model safety architecture that transitions from reactive crash avoidance to proactive, continuous risk management. PRISM employs inverse crash-probability modeling to convert binary crash classifiers into dynamic, interpretable safety scores. Three specialized models addressing trajectory kinematics, environmental risk, and VRU interaction operate concurrently, coordinated by a reasoning layer incorporating reinforcement learning, contextual memory, and feature-level attribution. The system provides graduated safety interventions across four tiers, from silent monitoring to emergency alerts. Unlike rule-based systems with static thresholds, PRISM dynamically adjusts safety parameters in real time. Validated across 1,296 scenarios from three naturalistic driving datasets without dataset-specific retraining, the system yielded a mean safety score of 68 out of 100, classified 77.6% of scenarios as advisory, and flagged a near-miss rate of 3.8%, with 11% of scenarios escalating to intervention or emergency response. Feature attribution consistently identified trajectory risk and VRU proximity as primary safety factors. PRISM provides a unified, interpretable framework for proactive transportation safety with emphasis on VRU risk reduction in dense urban environments.

PRACTICAL APPLICATIONS

PRISM addresses the limitations of reactive, fixed-threshold safety systems by continuously combining environmental, kinematic, and VRU-interaction risks into explainable, graduated interventions for complex urban driving.

Applications: PRISM supports manufacturers, fleet operators, and planners through calibrated warnings, trip-level risk aggregation, and pedestrian- and cyclist-relevant planning metrics.Interventions include explanations of their contributing factors.
Motivation: PRISM continuously assesses urban driving risk instead of waiting for hazards to materialize, addressing reactive ADAS limitations and nuisance alerts.Existing systems rely on fixed thresholds that do not adapt to context.
Contribution: The architecture unifies environmental, trajectory, and VRU-interaction risks into a continuous, interpretable score with four intervention tiers.The tiers range from silent monitoring to emergency alerts.
Related Work: Prior approaches typically address only selected safety dimensions and rarely integrate continuous scoring, explainability, and graduated intervention in one framework.The stated gap motivates PRISM’s unified architecture.

PRISM: SYSTEM ARCHITECTURE

PRISM uses a four-layer architecture that normalizes heterogeneous driving data, runs three risk models in parallel, and fuses their outputs into explainable intervention decisions.

PRISM: SYSTEM ARCHITECTURE: PRISM converts raw multi-agent motion data into a continuous, explainable safety score with graduated intervention outputs.The architecture contains four layers spanning ingestion, parallel modeling, reasoning, and downstream applications.
Layer 3: Agentic Reasoning: Layer 3 fuses the risk signals, applies reinforcement learning to select an intervention tier, and generates a SHAP-based explanation using short- and long-term memory.All three risk models operate on every cycle to reduce blind spots from a single failure mode.
Layer 1: Data Ingestion and Normalization: The ingestion layer converts incompatible dataset formats into a unified DrivingScene object with standardized agent tracks and scene-level night and rain flags.Agent types are normalized to vehicles, pedestrians, or cyclists, with pedestrians and cyclists grouped as VRUs.
Layer 2: Parallel Risk Models: A single ingestion pass supports all three Layer 2 risk models through parallel data transformation and shared normalized inputs.The environmental branch also preserves the frozen Phase 1 model’s original feature space without retraining.

Environmental Risk Scoring.

PRISM combines environmental context with trajectory and VRU-interaction signals to produce dynamic risk assessments that respond to motion and scene conditions.

Environmental Risk Scoring: The environmental model reuses a frozen random forest to produce an inverse-crash safety score and normalized environmental risk.This preserves the prior model while incorporating contextual information into PRISM.
Environmental Risk Scoring: Environmental context acts as a multiplier, damping dynamic risk in benign conditions and amplifying it in hostile conditions such as night or rain.The multiplier ranges from 0.5 to 1.5.
Trajectory Kinematic Analysis: The trajectory model aggregates normalized exceedances of braking, acceleration, swerving, and speeding thresholds across each agent’s motion.Scene-level trajectory risk is the highest risk observed among all agents.
VRU Interaction Analysis: The VRU model estimates ego-VRU conflict risk from closest approach distance and time-to-collision proxies using reaction-aware and recurrent trajectory prediction.Risk increases as distance and TTC fall below warning thresholds.

Risk Fusion.

PRISM fuses parallel risk signals into a dynamic score and uses reinforcement learning to select context-adaptive intervention tiers rather than relying solely on fixed thresholds.

Risk Fusion: Trajectory and VRU risks are blended with VRU-dominant weights of w_t=0.5 and w_v=1.0 because VRU conflicts are safety-critical.The three model signals are fused into a single dynamic risk.
Risk Fusion: The environmental multiplier scales the fused dynamic risk before it is mapped to a continuous safety score.This allows scene context to modify the combined trajectory and VRU assessment.
Risk Fusion: The PRISM heuristic baseline is defined by Equations (3)–(5), while the reinforcement-learning agent learns the final intervention policy over the same state.The learned policy operates on top of the fused risk formulation.
Reinforcement Learning Policy: A deep Q-network maps an eight-dimensional scenario state to silent, advisory, intervention, or emergency tiers by selecting the action with highest estimated value.Learning the policy lets PRISM weigh combinations of risk factors that fixed rules treat independently.

Explainability and Memory.

PRISM combines explainable intervention decisions with memory mechanisms and supports graduated safety responses across multiple datasets and application domains.

Explainability and Memory: SHAP attribution identifies which of eight state features influenced each selected intervention tier.Short-term memory preserves the current episode, while long-term memory stores representative past states and outcomes for recalling similar situations.
Explainability and Memory: Graduated outputs support ADAS feedback, fleet risk profiles, and infrastructure planning without retraining.Continuous scores can be aggregated by route and driver, while persistently low-score locations identify high-conflict sites.
Explainability and Memory: The reasoning layer maps an eight-dimensional scenario state to one of four intervention tiers using a deep Q-network.The state includes environmental, trajectory, VRU, proximity, imminence, night, and rain features.
Explainability and Memory: The four tiers range from silent monitoring to emergency alerts as the continuous safety score indicates increasing risk.The graduated bands are designed to reduce nuisance alerts while maintaining stronger responses for severe risk.
Explainability and Memory: The evaluation uses three motion datasets for validation without dataset-specific retraining and compares tier distributions and top-ranked SHAP features across datasets.The datasets include nuScenes, Argoverse 2, and Waymo Open Motion Dataset; NHTSA CRSS was used to train the Phase 1 model.

RESULTS AND DISCUSSION

Across 1,296 scenarios, PRISM produced broadly advisory safety assessments while maintaining distinct responses to adverse conditions. Argoverse 2 and Waymo showed similar score distributions, whereas nuScenes shifted toward higher-risk tiers under night, rain, and dense pedestrian conditions.

RESULTS AND DISCUSSION: 1,296 scenarios were evaluated across three datasets without dataset-specific retraining.The cross-dataset performance summary reports mean safety score, tier distribution, near-miss rate, and SHAP attribution.
RESULTS AND DISCUSSION: 68.4 was the weighted mean safety score across all 1,296 scenarios.Figure 2 displays score distributions relative to emergency, intervention, advisory, and silent tiers.
RESULTS AND DISCUSSION: Argoverse 2 and Waymo had nearly identical mean safety scores of 68.6 and 68.0, respectively, despite independent data collection, sensors, and geography.Both distributions were advisory-dominant, with scores concentrated in the 40–70 band.
RESULTS AND DISCUSSION: 59.8 was the mean nuScenes safety score, with scores shifting toward intervention and emergency bands under adverse-condition sampling.Night and rain scenes increased the environmental multiplier to m = 1.35 versus 0.97 for clear daytime conditions.
RESULTS AND DISCUSSION: Two nuScenes emergency-tier scenes involved high pedestrian density with either a confirmed near-miss or simultaneous night and rain.A silent-tier scene with three VRUs at 27.7 m received a score of 85.6, indicating no escalation in a genuinely low-risk condition.

Geographic Generalizability

PRISM showed geographic consistency across Argoverse 2 cities and comparable tier patterns across the larger evaluation datasets. Differences in emergency rates and trajectory risk reflected local traffic composition and environmental sampling while VRU-weighted fusion moderated kinematic effects.

Geographic Generalizability: 66.4–71.7 was the range of mean safety scores across six Argoverse 2 cities without city-specific retraining.Miami had the highest emergency rate at 7.3%, while Palo Alto reported none.
Geographic Generalizability: 85.5% was Washington D.C.’s advisory rate, indicating frequent low-level risk detection without escalation.The city-level comparison covers Argoverse 2’s 1,000 scenarios.
Geographic Generalizability: 70% of nuScenes, 76.7% of Argoverse 2, and 77.6% of Waymo scenarios were classified as advisory.The advisory tier consistently dominated across independently collected datasets without retraining.
Geographic Generalizability: 20% of nuScenes scenes were emergency-tier compared with 4.6% in Argoverse 2 and 4.5% in Waymo.The higher nuScenes emergency rate was attributed to adverse-condition sampling rather than model miscalibration.
Geographic Generalizability: The intervention and silent tiers were closely matched between Argoverse 2 and Waymo at 6.1% versus 6.3% and 12.6% versus 11.5%, respectively.These distributions reinforced calibration consistency across the four intervention tiers.
Geographic Generalizability: Waymo’s mean trajectory risk was 0.87 because scenes averaged 52.7 vehicles and 5.9 pedestrians, yet its mean safety score was 68.0.VRU-dominant fusion weights w_v = 1.0 and w_t = 0.5 reduced the impact of the elevated kinematic signal, matching Argoverse 2’s 68.6 mean.

VRU Proximity and Near-Miss Detection

PRISM identifies near-miss risk through ego–VRU proximity and adverse-condition context, with VRU risk consistently dominating feature attribution across datasets.

Near-Miss Detection: 11 of 286 Waymo scenarios triggered near-miss detections, with a mean VRU risk of 0.063.The dataset averaged 5.9 pedestrians per scene.
Near-Miss Detection: 18.5% of Argoverse 2 VRU-present scenarios fell below the 5 m near-miss threshold, and all received intervention or emergency classifications.This comprised 73 of 394 scenarios.
Adverse Conditions: Night conditions lowered mean safety scores by 10.5 points, from 66.2 to 55.7, as the environmental multiplier increased from 0.97 to 1.33.The night-and-rain scene scored 23.3 with 55 pedestrians at 3.1 m.
Feature Attribution: VRU risk was the dominant SHAP feature in both nuScenes and Argoverse 2, followed by imminence or trajectory risk.Mean |SHAP| for VRU risk was 2.10 in nuScenes and 1.75 in Argoverse 2.

Ablation Study: Feature Group Contribution

The ablation study shows that progressively adding trajectory and VRU signals increases intervention severity, while the RL agent is needed to synthesize compound risk into emergency decisions.

Environmental Model: With only the environmental model, all ten nuScenes scenes were classified as silent.The environmental model captured context but not dynamic trajectory or VRU signals.
Trajectory Model: Adding the trajectory model escalated 8 of 10 scenes to advisory, capturing kinematic risk from aggressive or high-speed maneuvers.This component alone did not reach the emergency tier.
VRU Model: Adding the VRU model independently escalated the two highest-proximity scenes directly to intervention.The Social Force Model identified high-risk ego–VRU encounters even when trajectory risk was moderate.
Full PRISM: Only full PRISM with the RL agent reached the emergency tier.The full system classified both high-risk scenes as emergencies while maintaining advisory for 7 kinematically active scenes.
Boundary Robustness: 8.8% to 12.6% was the range for combined emergency and intervention escalation under ±5-point boundary shifts.The less-than-four-point range indicates robustness of safety-critical tier assignments to minor calibration errors.

Computational Latency

PRISM’s CPU implementation requires approximately 596 ms end to end, making it suitable for offline post-processing rather than hard real-time control without optimization.

End-to-End Latency: 596 ms is PRISM’s approximate total end-to-end CPU latency.The reported timing is based on 20 samples.
Latency Breakdown: 245.3 ms for the trajectory LSTM and 163.8 ms for the environmental RF bridge make them the dominant computational costs.Together, these components account for 68% of total processing time.
Latency Breakdown: 0.3 ms is the RL decision-step latency, making its computational contribution negligible.SHAP attribution adds 90.3 ms when explanations are requested.
Deployment Boundary: The approximately 596 ms CPU latency prevents hard real-time deployment on vehicle hardware.The paper identifies GPU deployment or ONNX-optimized inference as required for on-vehicle integration.
Optimization Priorities: The trajectory LSTM and environmental RF bridge are the main targets for model compression and quantization.These components together account for 68% of total latency.

CONCLUSION

PRISM combines parallel environmental, trajectory, and VRU risk models with DQN-based reasoning and SHAP explanations to support proactive, graduated safety assessment. Across 1,296 scenarios without dataset-specific retraining, it achieved a mean safety score of 68/100, with advisory classifications predominating and VRU risk as the primary decision driver.

Architecture: PRISM fuses environmental, trajectory-kinematic, and VRU-interaction models through a DQN agent with SHAP explainability.The architecture advances static crash-probability scoring toward dynamic, scene-aware intervention decisions.
Results: 68/100 was the mean safety score, while 77.6% of scenarios were classified as advisory under normal urban driving conditions.Emergency-tier rates increased from 4.5–4.6% in structured environments to 20% under adverse conditions.
Interpretability: VRU risk was the primary SHAP decision driver, and all emergency-tier classifications involved two or more active risk dimensions.The result is consistent with the paper’s compound-risk finding.
Conclusion: PRISM provides a modular, interpretable, dataset-agnostic foundation for proactive safety assessment and graduated intervention.The stated scope is diverse autonomous driving environments and next-generation transportation infrastructure.