PaperlayerView exact revision on arXiv

Source-linked AI summary

TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis

Jiahao Wu, Zhenqun Yang, Chen Jason Zhang, Qing Li

arXiv:2609.02540v1cs.LG

TL;DR

Collective trajectory systems often lack both verifiable event details and low-latency monitoring. TrajMind separates fast screening from slow, source-verified diagnosis with role-specialized adapters, improving typing and localization while reducing monitoring latency.

  • Problem

    Existing methods do not jointly provide operationally verified collective anomaly details and efficient monitoring from trajectory windows.

  • Method

    TrajMind switches role-specific LoRA adapters over a frozen vision–language backbone for fast screening and slow, executable verification of structured diagnoses.

  • Results

    TrajMind_slow improves three-way balanced accuracy by 15.3–29.0% and participant F1 by 13.8–23.2%, while TrajMind_fast reduces latency by 41.1%.

  • Takeaways & Limitations

    TrajMind provides accurate, evidence-backed diagnoses across cities alongside efficient front-line monitoring.

  • Takeaways & Limitations

    A case study shows that executable verification can reject unsupported localization without repairing the proposed segment.

Abstract

from arXiv · show

Diagnosing collective anomalies from urban trajectories is increasingly important for traffic governance, as it reveals what happened, who was involved, and where and when the event occurred. Existing detectors efficiently produce scores or labels, whereas vision--language pipelines provide richer semantics; neither couples verifiable diagnosis with low-latency monitoring. The central challenge is to recognize collective patterns and recover exact event details from the source trajectories without running the full diagnostic pipeline for every monitored window. We therefore separate always-on screening from on-demand diagnosis: screening raises alerts, while diagnosis releases only source-verified what--who--where--when records. We present TrajMind, a fast-and-slow framework that switches three role-specialized LoRA adapters over one frozen vision--language backbone. Its slow path, \textit{TrajMind$_{\text{slow}}$}, chains canvas-based typing, type-conditioned localization over serialized trajectories, and executable verification, yielding structured, evidence-backed diagnoses. Additionally, the fast path, \textit{TrajMind$_{\text{fast}}$}, screens each window in a single text-only pass, delivering efficient structured alerts. Extensive experiments show that, TrajMind$_{\mathrm{slow}}$ outperforms the strongest baselines by at least $15.3$ percentage points in anomaly typing and $13.8$ percentage points in localization. These gains persist under cross-city transfer, and TrajMind$_{\mathrm{fast}}$ reduces latency by $41.1\%$ and maintains binary balanced accuracy of at least $93.5\%$. Together, TrajMind delivers accurate, evidence-backed diagnoses across cities and efficient front-line monitoring.

1 Introduction

TrajMind addresses the challenge of diagnosing collective trajectory anomalies with verifiable what–who–where–when evidence while maintaining efficient monitoring. It specializes a frozen VLM into role-specific LoRA adapters and combines slow evidence-backed diagnosis with fast text-only screening.

  • Motivation: TrajMind targets the fundamental challenge of producing accurate collective-anomaly diagnoses and efficient monitoring for urban traffic governance.Operational evidence requires more than assigning an anomaly score to an isolated trip.
  • Framework: TrajMind assigns scene understanding, precise grounding, and efficient monitoring to lightweight role-specific LoRA adapters over a common frozen VLM backbone.Holistic visual rendering supports scene understanding, lossless textual records support identities, coordinates, and timestamps, and text-only processing supports throughput.
  • Operating modes: TrajMind_slow chains map-aligned scene typing with lossless textual grounding to produce complete, evidence-checked diagnoses, whereas TrajMind_fast screens each window in one text-only pass.The fast path drops visual processing and staged reasoning for monitoring.
  • Evaluation protocol: The study defines structured what–who–where–when diagnoses, separates alerts from evidence-backed diagnoses, and evaluates them on held-out Chengdu, Xi’an, and Porto trajectories.Controlled collective anomalies provide type, participant, segment, and temporal ground truth.
  • Results: 41.1% latency reduction, from 9.026 to 5.315 seconds per window, retained binary balanced accuracy of .960/.949/.935; participant F1 improved by 13.8–23.2%.The slow path remained stronger in fine-grained typing and participant localization, while results on Xi’an and Porto were zero-shot.

2 Related Work

Prior trajectory-anomaly methods detect individual deviations using classical, deep, foundation-model, or language-model approaches, while vision–language models extend perception and reasoning to maps, traffic, trajectories, and video. However, existing methods do not jointly recover a collective anomaly’s type, participants, affected road segments, and temporal extent.

  • Trajectory Anomaly Detection: Existing trajectory-anomaly methods identify deviations from expected spatial, temporal, or behavioral patterns using partitioning, geometric similarity, density, frequency, or learned sequential representations.Self-supervised encoders and trajectory foundation models further target transfer across tasks and regions [29] [35].
  • Trajectory Anomaly Detection: Language-model trajectory approaches encode trajectories as tokens or multimodal inputs [16] [22], but remain focused on individual trajectories or task-specific predictions.Consequently, they do not jointly recover a collective anomaly’s type, complete participant set, affected road segments, and temporal extent.
  • Vision–Language Models: Vision–language models couple visual perception with language generation and reasoning, supporting map and traffic understanding, single-trajectory mining [16], video dialogue and temporal grounding [20] [25], and video anomaly understanding across temporal scales [21] [32].Qwen2.5-VL [2] is identified as an adaptable general-purpose backbone.

3 Methodology

TrajMind separates always-on screening from evidence-backed diagnosis by switching three role-specialized LoRA adapters over one frozen vision–language backbone. Its slow path types anomalies from diagnostic canvases, localizes exact participants, regions, and times from serialized trajectories, and verifies the resulting record through executable checks.

  • 3 Methodology: The formal output couples window-level classification with set-valued participant and segment recovery plus temporal localization, while normal predictions contain only the type and empty localization fields.Fast-path outputs are treated as alerts, whereas actionable diagnoses require recomputation from the source window.
  • 3 Methodology: TrajMind uses LoRA-C for one-pass text-only screening, while escalated windows switch to LoRA-A for canvas-based typing and LoRA-B for type-conditioned localization before verification.All adapters share a frozen backbone but retain separate parameters, with LoRA-C serving the fast monitor and LoRA-A/LoRA-B serving diagnosis.
  • 3 Methodology: The slow path assigns representations by role: rendered panels expose spatial overlap, delays, and temporal evolution for typing, whereas serialized records preserve exact agent and segment identifiers and relative times for localization.The shared time origin t0 makes predicted times invariant to absolute clock time while retaining ordering and duration signals.
  • slow: Type, Localize, and Verify: LoRA-A predicts only normal or an anomaly type, and for non-normal outputs LoRA-B predicts participants, affected segments, relative start and end times, and confidence without revising the type.If LoRA-A predicts normal, the slow path terminates without producing an event hypothesis; LoRA-B is trained only on verifier-confirmable localizations.
  • slow: Type, Localize, and Verify: The verifier treats the predicted type and localization as hypotheses, rejects invalid or non-co-present claims, recomputes type-specific trajectory evidence, and retains raw measurements alongside support, contradiction, and uncertainty.This makes the slow-path diagnosis source-verifiable rather than merely model-proposed.

4 Experiments · 4.1 Experimental Settings

The experiments address detection, typing, localization, robustness, efficiency, and the effects of role specialization and reinforcement-learning refinement. They use real urban trajectories with synthetically injected, exactly recorded anomalies and compare TrajMind across defined tasks, metrics, and baseline families.

  • 4 Experiments: Six research questions evaluate anomaly detection and typing, localization, cross-city and strength robustness, fast-path efficiency, role specialization, RL refinement, and frozen-VLM perception limits.The questions are labeled RQ1–RQ6 in the experimental agenda.
  • 4.1 Experimental Settings: Experiments introduce datasets, tasks, evaluation metrics, and baselines for the study.This subsection defines the experimental setting before reporting results.
  • 4.1 Experimental Settings: Trajectories come from Chengdu and Xi’an in DiDi GAIA and Porto in the ECML/PKDD-2015 challenge, with map-matched road segments retained for Chengdu and Xi’an.Dataset statistics are summarized in Table 1.
  • 4.1 Experimental Settings: Each retained window is a 900-s bucket containing at least 3 agents.This defines the basic monitoring unit used in the experiments.
  • 4.1 Experimental Settings: Anomalies are synthetically injected because public datasets lack real-world collective traffic-anomaly labels, while injection records provide exact type, participant, segment, and interval ground truth.Injected slowdown and bunching use specified pass-time and arrival-time transformations; held-out stronger, weaker, and structure-only protocols probe robustness.
  • 4.1 Experimental Settings: Evaluation covers three-way detection and typing, binary anomaly detection, and localization, using balanced accuracy with chance levels .333 and .500 and localization sets of N=179/189/347.Localization separately injects anomalies into every test window, with slowdown:bunching 1:1, for Chengdu/Xi’an/Porto respectively.
  • 4.1 Experimental Settings: Baselines span a training-free statistical temporal rule, symbolic and hybrid localizers, DSAB [11], and the trajectory-centric VLM Traj-MLLM [16].DSAB is reported only in the binary panel because it emits no anomaly type; Cascade uses a VLM where Derive abstains.

4.2 Anomaly Detection/Typing (RQ1, Table 2)

TrajMind_slow achieves the best three-way balanced accuracy in every evaluated city, while TrajMind_fast delivers strong binary detection but weaker fine-grained typing. Zero-shot performance depends strongly on input representation and model scale, with canvas prompting helping larger models but not closing the adapted specialists’ gap.

  • Anomaly Detection/Typing: Role-specialized adaptation consistently outperforms statistical, graph-based, trajectory-MLLM, text-only, and canvas-based zero-shot alternatives.The three-way label space is {normal, collective slowdown, bunching}, with chance .333; binary detection merges the anomaly types, with chance .500.
  • Anomaly Detection/Typing: .983/.980/.959 three-way balanced accuracy makes TrajMind_slow the best method across Chengdu, Xi’an, and Porto.With the 8B backbone, it exceeds the strongest non-TrajMind results by .313/.289/.185; the 2B specialist still reaches .952/.942/.914, above all baselines.
  • Anomaly Detection/Typing: .774 is the highest zero-shot canvas score, while text-only prompting stays near the .333 three-way chance level.Canvas prompting improves 7B and 8B models, but 2B and 3B models collapse to chance; the 235B API model reaches only .667/.670/.667.
  • Anomaly Detection/Typing: .960/.949/.935 binary balanced accuracy lets TrajMind_fast outperform DSAB by .280/.224/.383 across the three cities.Its corresponding three-way scores fall to .637/.626/.614, whereas TrajMind_slow remains near ceiling in both panels, establishing complementary screening and typing roles.

4.3 Fine-Grained Localization (RQ2, Table 3) · 4.4 Cross-City Localization (RQ3, Table 4)

TrajMind’s type-conditioned slow specialist delivers highly reliable participant and segment localization, including after transfer from Chengdu to Xi’an and Porto. The fast and slow paths are complementary across temporal and segment metrics, while temporal comparisons require accounting for response coverage.

  • 4.3 Fine-Grained Localization (RQ2, Table 3): Subject F1 reaches .993/.995/.941 for TrajMind_slow across Chengdu, Xi’an, and Porto, exceeding the strongest non-TrajMind baseline by .232/.201/.138 and achieving full coverage.The slow path receives the ground-truth anomaly type under this localization protocol.
  • 4.3 Fine-Grained Localization (RQ2, Table 3): TrajMind_fast leads tIoU on Chengdu and Porto at .791 and .717, is only .009 behind TrajMind_slow on Xi’an, and leads segment F1 on Chengdu at .609.The slow path is strongest for transferred segment localization, showing that accurate participants and segments do not guarantee the highest temporal overlap.
  • 4.3 Fine-Grained Localization (RQ2, Table 3): Frozen API VLMs respond on at least .959 of windows yet obtain segment F1 between .002 and .035, while Traj-MLLM covers .967–.992 and gets zero segment F1 in all cities.Within Qwen2.5-VL, scaling from 3B to 7B improves subject F1 in every city, whereas Qwen3-VL 8B is worse than its 2B counterpart and abstains on some windows.
  • 4.4 Cross-City Localization (RQ3, Table 4): Across unseen Xi’an and Porto, TrajMind_slow achieves subject F1 of .995 and .941, improving over TrajMind_fast by .023 and .219, respectively.The slow path uses Chengdu-trained specialists without target-city adaptation, with ground-truth anomaly type and forced response during evaluation.
  • 4.4 Cross-City Localization (RQ3, Table 4): Segment localization requires specialization: TrajMind_slow reaches .636 on Xi’an and .930 on Porto, improving over TrajMind_fast by .081 and .333 after transfer.On Porto, the slow path directly emits correct 100 m grid-cell identifiers rather than inferring locations from participant predictions.
  • 4.4 Cross-City Localization (RQ3, Table 4): On Porto, TrajMind_fast attains higher tIoU than TrajMind_slow, .717 versus .379, but answers only 73.0% of windows while the slow path returns intervals for all windows.On Xi’an, the paths obtain nearly identical tIoU, .787 versus .796; therefore Porto’s gap does not establish end-to-end temporal superiority for the fast path.

4.5 Analysis of Cross-Protocol Localization

TrajMind_slow transfers participant localization across unseen anomaly strengths and city shifts, consistently outperforming symbolic baselines. Its accuracy is less sensitive to anomaly-strength changes, though compound city–protocol shifts affect Xi’an more than Porto.

  • Cross-Protocol Localization: TrajMind_slow ranks first in all nine settings, averaging .958 Subject-F1 versus .764 for Cascade and .539 for Derive.Its lowest score, .914 on Xi’an under weak Protocol C, exceeds Cascade’s best baseline score, .842 on Xi’an under Protocol B, by .072.
  • Cross-Protocol Localization: Protocol-strength changes reduce symbolic-method localization more sharply: from Protocol B to C on Chengdu, Subject-F1 drops .272 for Derive and .191 for Cascade, versus .023 for TrajMind_slow.Derive and Cascade follow C < A < B in every city, while stronger perturbations make their fixed thresholds easier to trigger.
  • Cross-Protocol Localization: Under simultaneous city and protocol shifts, TrajMind_slow remains strong but city dependent: Porto stays at .941/.942/.941 across A/B/C, while Xi’an falls from .995 to .930 and .914.It retains a clear lead over Cascade in the latter case, with .842 and .708, supporting transfer across unseen anomaly strengths and cities.

4.6 Analysis of Cross-City/Protocol Typing · 4.7 Inference Time Comparison (RQ4, Table 7)

TrajMind’s Chengdu-trained typer generalizes across cities, anomaly strengths, and compound shifts while avoiding structure-only false positives. For deployment, the fast path cuts inference time by 41.1% for latency-sensitive screening, whereas the slow path preserves stronger fine-grained diagnosis.

  • 4.6 Analysis of Cross-City/Protocol Typing: .954/.927 cross-city typing by the Chengdu-trained 3B model substantially exceeds the statistical rule’s .409/.392.Table 6 reports the Chengdu-trained typer’s cross-city result against the statistical-rule baseline.
  • 4.6 Analysis of Cross-City/Protocol Typing: .991/.982 typing under anomaly-strength shifts and .940/.919 under compound shifts demonstrate robustness beyond cross-city transfer.These results are reported for the Chengdu-trained 3B typer in Table 6.
  • 4.6 Analysis of Cross-City/Protocol Typing: 0.0% false positives on structure-only controls versus 76.7% for the statistical rule indicates reliance on temporal evidence rather than co-routing.The contrast is reported for the Chengdu-trained 3B typer and the statistical-rule baseline.
  • 4.7 Inference Time Comparison (RQ4, Table 7): 41.1% lower latency: TrajMind_fast saves 3.711 s per window versus TrajMind_slow under the same batch-one generation budget.The fast path requires 5.315 s per window, compared with 9.026 s for the slow path.
  • 4.7 Inference Time Comparison (RQ4, Table 7): .960 versus .974 binary balanced accuracy on Chengdu shows that fast-path screening largely preserves in-domain performance relative to the slow path.The corresponding subject F1 scores are .950 and .993 for the fast and slow paths, respectively.
  • 4.7 Inference Time Comparison (RQ4, Table 7): The fast path is suited to latency-sensitive screening, whereas the slow path remains preferable for complete and fine-grained diagnosis.The slow path reaches 1.000 on the cited fine-grained result, while the fast path is notably weaker there.

4.8 Ablation on TrajMindfast (RQ5, Figure 2) · 4.9 Ablation on Specialized Roles (RQ5, Table 8)

The ablations show that GRPO refinement substantially improves TrajMind_fast across all three cities, especially participant and segment F1. Specialized role adapters outperform parameter-matched shared adaptation, whose increased capacity only partially closes the gap and is especially limiting for exact spatial grounding under cross-city shift.

  • 4.8 Ablation on TrajMindfast (RQ5, Figure 2): GRPO raises all four TrajMind_fast metrics across Chengdu, Xi’an, and Porto relative to SFT initialization.Three-way balanced accuracy increases by .038 and binary detection by .065 in every city.
  • 4.8 Ablation on TrajMindfast (RQ5, Figure 2): Participant F1 gains .191, .188, and .286 on Chengdu, Xi’an, and Porto, respectively, after GRPO refinement.
  • 4.8 Ablation on TrajMindfast (RQ5, Figure 2): Segment F1 gains .139, .127, and .257 on Chengdu, Xi’an, and Porto, respectively, after GRPO refinement.
  • 4.8 Ablation on TrajMindfast (RQ5, Figure 2): The larger binary-detection gain indicates that reinforcement learning particularly strengthens TrajMind_fast’s normal–anomaly screening role.
  • 4.9 Ablation on Specialized Roles (RQ5, Table 8): Increasing shared-adapter rank from 16 to 32 improves typing by .012–.051, subject F1 by .007–.030, and segment F1 by .013–.032.The largest gains occur on Porto, but higher shared capacity only partially closes the gap.
  • 4.9 Ablation on Specialized Roles (RQ5, Table 8): Parameter-matched shared adaptation remains inferior to TrajMind_slow, indicating that shared adaptation especially limits exact spatial grounding under cross-city shift.The shared baselines use one 3B backbone and one LoRA for canvas-typing and text-localization, whereas r=32 matches the aggregate budget of two r=16 role adapters.

4.10 Analysis of Zero-Shot Failure: Perception or Calibration? (RQ6, Table 9) · 4.11 Verifier Ablation (Table 10) · 4.12 Case Study: A Bunching Anomaly Instance

The analyses attribute zero-shot failure primarily to calibration, show that verification improves the quality of selectively released diagnoses, and use a bunching case study to expose stage-specific typing and localization errors.

  • 4.10 Analysis of Zero-Shot Failure: Perception or Calibration? (RQ6, Table 9): Zero-shot failure is primarily calibration: removing normal yields .983–1.000 two-way accuracy but only .522–.670 three-way balanced accuracy in capable frozen models.This gap places the zero-shot ceiling below the adapted model despite strong two-way performance without the normal class.
  • 4.11 Verifier Ablation (Table 10): Verification raises three-way typing accuracy from .960 to .990, participant F1 from .960 to .996, and segment-localization F1 from .480 to .530 (+10.4%).The verified configuration is evaluated over released diagnoses, so these gains must be interpreted together with released coverage.
  • 4.11 Verifier Ablation (Table 10): The verifier’s .030 coverage loss accompanies a .050 absolute gain in segment F1, indicating a trade-off that favors withholding spatially unreliable diagnoses.The verification threshold can control answer availability versus the cost of releasing unsupported localization.
  • 4.11 Verifier Ablation (Table 10): Verification selectively releases or withholds outputs rather than rewriting them, so its improvements should be interpreted as selective release rather than error correction.This distinction qualifies how the verifier ablation results should be understood operationally.
  • 4.12 Case Study: A Bunching Anomaly Instance: In a 900 s Chengdu window with 54 trajectories, all six injected vehicles enter ground-truth segment 869 within 8.5 s, supporting a bunching anomaly rather than a broad slowdown.The case study combines dense route interactions, shared-segment pass times, and candidate-group formation over time with the decisive temporal signature.
  • 4.12 Case Study: A Bunching Anomaly Instance: The fast path recovers all six injected identities but labels the event as a slowdown and selects segment 3089; the slow canvas corrects the type to anomaly_bunching with confidence 1.0 while retaining those participants.The slow text localizer nevertheless selects the same segment 3089, illustrating that correct typing and participant recovery do not guarantee correct localization.
  • 4.12 Case Study: A Bunching Anomaly Instance: Recomputing the slow localization against lossless trajectories shows that only three of six vehicles traverse segment 3089, whereas all six pass through segment 869.This verification step exposes the spatial mismatch behind the case-study localization error.

5 Conclusion

TrajMind is a fast-and-slow framework that specializes a shared frozen vision–language backbone with role-specific LoRA adapters for collective trajectory anomaly detection and diagnosis. Its slow path provides evidence-backed diagnoses across cities and anomaly-severity levels, while its fast path reduces latency while preserving detection and participant identification.

  • 5 Conclusion: TrajMind specializes a shared frozen vision–language backbone with role-specific LoRA adapters for collective trajectory anomaly detection and diagnosis.The framework separates diagnosis from screening through complementary slow and fast paths.
  • 5 Conclusion: The slow path combines canvas-based typing, lossless textual localization, and evidence verification to deliver accurate, evidence-backed diagnoses across cities and anomaly-severity levels.
  • 5 Conclusion: The fast path provides single-pass text-only screening, reducing latency while preserving detection and participant identification.

6 Ethical Considerations

The study notes that trajectory analysis can expose sensitive mobility patterns and enable misuse, surveillance, or re-identification. It therefore recommends privacy safeguards for deployment.

  • 6 Ethical Considerations: Trajectory data may enable pervasive surveillance, automated enforcement, or re-identification, motivating data minimization, pseudonymization, retention limits, access control, and auditing.The study concerns group-level traffic behavior rather than demographic profiling, but participant identifiers and fine-grained spatiotemporal records may still be sensitive when combined with external data.
Loading 2609.02540v1…