PaperlayerView exact revision on arXiv

Source-linked AI summary

Revolutionizing Turn-by-Turn Navigation with Cloud-Edge Deep Learning

Yiming Yang, Hao Fu, Fanxiang Zeng, Xikai Yang, Yue Liu, Ning Guo

arXiv:2608.29073v1cs.AI

TL;DR

Existing TBT audio policies struggle to balance guidance quality, timing, density, and driver cognitive load. The paper decomposes instructions into modular elements and uses sequence modeling, multi-task learning, and cloud-edge collaboration for real-time generation. Real-world experiments report significantly lower yaw rates than HMM-based policies, while ablations support sequence information and CrossNet components.

  • Problem

    Rule-based TBT audio policies struggle to balance accurate guidance, play timing, audio density, and driver cognitive load.

  • Method

    The framework decomposes audio into modular elements and uses sequence models with multi-task learning and cloud-edge collaboration for real-time, context-aware instruction generation.

  • Results

    Real-world A/B tests and blind evaluations demonstrated significantly reduced yaw rates compared with HMM-based policies.

  • Takeaways & Limitations

    The method represents the first large-scale application of deep learning in practical TBT navigation and an advancement in intelligent transportation technologies.

Abstract

from arXiv · show

Turn-by-turn (TBT) navigation systems are integral to modern driving experiences, providing real-time audio instructions to guide drivers safely to destinations. However, existing audio instruction policy often relies on rule-based approaches that struggle to balance informational content with cognitive load, potentially leading to driver confusion or missed turns in complex environments. To overcome these difficulties, we first model the generation of navigation instructions as a multi-task learning problem by decomposing the audio content into combinations of modular elements. Then, we propose a novel deep learning framework that leverages the powerful spatiotemporal information processing capabilities of Transformers and the strong multi-task learning abilities of Mixture of Experts (MoE) to generate real-time, context-aware audio instructions for TBT driving navigation. A cloud-edge collaborative architecture is implemented to handle the computational demands of the model, ensuring scalability and real-time performance for practical applications. Experimental results in the real world demonstrate that the proposed method significantly reduces the yaw rate (the proportion of vehicles deviating from navigation routes) compared to traditional methods, delivering clearer and more effective audio instructions. This is the first large-scale application of deep learning in driving audio navigation, marking a substantial advancement in intelligent transportation and driving assistance technologies.

I. INTRODUCTION

Existing TBT audio policies struggle to balance accurate, complete guidance with timing, density, and driver cognitive load. The paper proposes a deep-learning, multi-task, cloud-edge framework that generates real-time, context-aware instructions.

  • Motivation: Rule-based policies can produce audio that is too general or too verbose, creating a seesaw effect among yaw rate, play timing, and audio density.Overly complex instructions may increase cognitive load, while simplistic audio may omit critical guidance.
  • Approach: The proposed pipeline decomposes audio context into modular elements, predicts element recall, play timing, and order, then generates speech with text-to-speech.This decomposition makes instruction generation more manageable and supports adaptable content.
  • Approach: The framework uses sequence models to generate real-time, context-aware instructions and capture dependencies and variations in driving scenarios.The authors report that the approach handles complex intersections and reduces yaw rate.
  • Contributions: The paper presents deep-learning-based TBT navigation as a practical application using sequence models to capture spatiotemporal dependencies.The authors position this as the first such implementation for practical TBT navigation and as an approach to the yaw-rate, timing, and density trade-off.
  • Contributions: A cloud-edge collaborative architecture addresses real-time computational demands while supporting scalability and large-scale service deployment.The architecture is presented as part of the practical deployment design.
  • Contributions: The work introduces a data-driven paradigm for optimizing TBT navigation instead of relying on rule-based optimization.The authors describe this shift as enabling continuous performance improvements.

A. Preliminaries

TBT navigation provides step-by-step driving guidance through audio and visual cues organized around navigation points, segments, and route paths. Prior systems generally use rule-based triggering, whereas this work applies deep learning to schedule flexible, context-aware audio instructions.

  • TBT Navigation: TBT navigation uses real-time driving data, often including GPS, to provide step-by-step audio and visual guidance for turns, lane changes, and other operations.The system guides drivers from a starting location to a destination.
  • TBT Navigation: A navigation point is where the system expects steering, a segment connects neighboring navigation points, and the navigation path consists of all segments.The instruction policy selects content and timing to help prevent route deviation or traffic-rule violations.
  • Audio Representation: Instruction audio is organized into action elements requiring steering and information elements that provide guidance without requiring steering.Each instruction also includes play timing and play order within a segment.
  • Audio Representation: Play order determines an audio's position within a segment and affects connector selection, so identical elements can produce different content at different positions.The paper illustrates this with multiple “turn left” audios whose connectors vary by order.
  • Traditional Methods: Traditional systems use HMMs for map matching but predominantly rely on geometric rules to determine when to speak and what to say.These methods cannot capture long-range temporal dependencies and may struggle with dynamic timing conditions.
  • Related Work: Research on human-driving audio navigation has received less attention than related accessibility, indoor, pedestrian, and visual-navigation applications.Prior driving studies remain grounded in static, rule-based design principles.
  • This Work: This work applies deep learning to TBT audio policy, using sequence modeling and multi-task learning to capture spatiotemporal dependencies and generate flexible instructions.The paper frames audio policy as scheduling voice-instruction generation and timing for human drivers.

III. METHOD

The paper formalizes TBT audio-instruction generation as multi-task learning, decomposing instruction composition into content, timing, and ordering decisions. Segment-level feature sequences are mapped to multiple outputs using deep neural networks.

  • Problem formalization: The method decomposes navigation instruction generation into concurrent tasks for content, linguistic structure, and delivery timing.This formalization treats the multifaceted instruction problem as a multi-task learning task.
  • Problem formalization: Driving scenarios are modeled at segment granularity because instructions within each segment have strong spatiotemporal correlation.Features are sampled at 1-second intervals across the time steps in a segment.
  • Problem formalization: The input is a sequence of feature vectors Xt={x_t−n,...,x_t}, mapped to trigger timing, action elements, information elements, and voice order.The outputs compose the audio instruction from modular decisions.
  • Problem formalization: The model learns this input-to-output mapping with deep neural networks trained by maximum likelihood estimation on high-quality data.The objective is to fit the distribution of high-quality data and learn an audio instruction policy.

B. Sequence modeling

The sequence model combines feature encoding, domain-informed positional representations, Transformer decoding, and an MoE prediction layer. Cloud-edge deployment separates static feature embedding from real-time edge inference and assembles predicted elements into speech.

  • Deployment: The cloud embeds relatively static segment features, while the edge embeds real-time features and performs model inference.The edge requests cloud embeddings at each segment’s start and orchestrates inference and playback.
  • Deployment: Predicted timing, action elements, information elements, and voice order are assembled into a sentence and converted to speech by TTS.This process repeats throughout the route on the navigation terminal.
  • Architecture: The architecture contains a Feature Encoder, Deep CrossNet, Transformer Decoder, and MoE Prediction Layer.These four levels process sparse features, cross-feature combinations, sequential dependencies, and multi-task outputs.
  • Architecture: Domain-informed position embeddings combine temporal sequence encoding with spatial semantic encoding based on distance to the next navigation point.Features closer to the navigation point receive more attention during multi-head attention.
  • Architecture: The Transformer Decoder models long-range dependencies across time slices rather than relying only on adjacent observations.Self-attention adaptively captures semantic information in the sequence.
  • Prediction: The MoE Prediction Layer jointly predicts trigger time, action elements, information elements, and within-segment audio order.Shared features are learned through the MoE for the four instruction-generation subtasks.

C. Model Training

Training uses task-specific decoders and losses for timing, action selection, information selection, and voice order. A geometric-mean total loss balances the four subtasks during multi-task optimization.

  • Task losses: The model produces four sub-task outputs: trigger, action, info, and voice order.Each output has a loss tailored to its prediction task.
  • Task losses: The trigger decoder predicts normalized audio play timing with a scalar output in [0,1] and uses mean squared error.The target is the audio play timing label.
  • Task losses: The action decoder predicts which available action elements to include using a multi-label probability vector thresholded at 0.5.An element is included when its predicted probability exceeds 0.5.
  • Task losses: The info decoder predicts information-type elements with cross-entropy because an instruction can contain at most one information-type element.The target uses a one-hot label and includes a no-information class.
  • Task losses: The voice order decoder classifies five categories, where 0 means no playback and 1–4 specify the play order relative to the segment endpoint.Cross-entropy loss is used for this one-hot classification task.
  • Multi-task optimization: The total loss uses a geometric mean so that no single sub-task dominates training.This is intended to balance learning difficulty across trigger, action, info, and voice order tasks.

D. Cloud-edge collaboration

The cloud-edge architecture assigns time-sensitive inference to edge devices and computationally intensive static-feature processing to the cloud. This division supports real-time responsiveness, scalability, resource efficiency, and faster model updates.

  • Task allocation: Edge inference provides real-time responsiveness by processing navigation data locally and reducing latency for audio instructions.The paper connects immediate feedback with timely driving decisions.
  • Architecture: The cloud-edge framework combines cloud preprocessing and edge-side decisions about instruction generation during driving progress.At each segment start, the edge requests cloud feature embeddings and coordinates subsequent inference.
  • Scalability: Separating edge inference from cloud preprocessing reduces centralized cloud burden and supports a larger user base without proportional cloud-resource growth.The architecture is described as avoiding wasted centralized computation.
  • Task allocation: The cloud generates embeddings for static and complex features, including road-graph data, some element features, and personalized driver features.These computations use centralized storage and cloud processing power and can be periodically updated.
  • Online updates: Independent updates to edge inference and cloud embeddings reduce the model iteration cycle from monthly to weekly.The modularity enables faster deployment of improvements and new features.
  • Practical outcome: The architecture is reported to improve scalability and performance while providing timely, accurate, context-aware audio instructions.The conclusion frames the design as balancing efficiency, effectiveness, and cost.

IV. EXPERIMENTS

The experiments evaluate the proposed navigation instruction model through real-world deployment, offline ablations, and blind driver assessment against an HMM-based policy.

  • Experimental design: The study evaluates the model using an A/B test, offline component ablations, and a blind evaluation involving 100 drivers.The A/B test compares the deployed model with the previously deployed HMM-based policy.

A. Dataset and Model configurations

The experiments use real-world navigation logs and a cloud-edge training and deployment process designed for iterative model improvement, safety, and privacy.

  • Dataset: The dataset is built from real-world driver navigation logs collected over 8 days, from June 11 to June 18, 2023.
  • Dataset: An HMM-based policy provides both the online data-collection mechanism and the experimental comparison baseline.The policy models sequential driving states rather than relying solely on distance-triggered geometric rules.
  • Dataset: Filtering, augmentation, and synthetic samples are used to improve the quality and diversity of training trajectories.The transformations include voice-trigger shifts and element perturbations based on real-road testing experience.
  • Dataset: The final dataset contains approximately 1.56 billion sequence samples, including 1.1 billion training samples.The training set includes 10 million supreme-quality samples for supervised fine-tuning.
  • Model configurations: The model uses three experts and four output heads for trigger, action, information, and voice-order prediction.Its input features have 2139 dimensions, and the sequence length is set to 3.
  • Deployment: Offline optimization supports safety validation before online deployment, while float16 conversion reduces deployment cost.The cloud-edge architecture accelerates retraining from monthly HMM updates to a weekly schedule.

B. Real-world A/B Test

A real-world A/B test compares the sequence model with the deployed HMM policy across navigation modes using yaw rate and audio-content measures.

  • Results: The model performs across detail, concise, minimalist, and intelligent navigation modes, which provide different prompt frequencies for driver preferences.Detail and intelligent modes use more frequent prompts, whereas concise and minimalist modes use fewer.
  • Experimental setup: The online A/B test spans one week and includes approximately 600 million navigation segments.Data were collected from August 28, 2024, to September 3, 2024.
  • Metrics: The evaluation compares yaw rate, average words per segment, and element play density, where lower yaw rate indicates more accurate instruction.More elements with fewer average words are associated with better content organization.
  • Results: The sequence model significantly reduces yaw rate relative to the deployed HMM baseline in all modes except intelligent mode.The largest improvements occur in concise and minimalist modes.
  • Conclusion: The real-world A/B test validates real-time, context-aware instructions that reduce yaw rate across diverse navigation-detail preferences.The authors characterize the results as evidence of robustness and generalizability under real-world driving conditions.

C. Ablation Study

Offline ablations show that sequence information, positional encoding, MoE, CrossNet, and the Decoder-only Transformer each contribute to navigation prediction performance, with sequence length 3 balancing context and cost.

  • Baseline performance: The full model achieves 83.3% Trigger 10m, 96.3% Trigger 30m, 97.0% Action, 98.6% Info, and 90.7% VoiceOrder accuracy.
  • Architecture ablations: Replacing the Decoder-only Transformer with BERT-like Transformer reduces Trigger 10m and Trigger 30m accuracies by 2.0% and 0.8%, respectively.The Decoder-only Transformer is reported as better suited to the sequence dependencies in this task.
  • Architecture ablations: Removing position embeddings or MoE also degrades performance, affecting timing and order prediction and the model’s multi-task handling.
  • Architecture ablations: Removing CrossNet causes the largest overall degradation, decreasing Trigger 10m and Action accuracies by 3.0% and 4.1%.The result highlights the role of high-order feature interactions.
  • Sequence information: Removing sequential input features reduces Trigger 10m accuracy by 5.5% and lowers other metrics.This corresponds to reducing sequence length from 3 to 1.
  • Sequence length: Sequence lengths of 1 and 2 significantly reduce performance, while shorter sequences fail to capture sufficient temporal dependencies.The effect particularly harms prediction within the critical 10-meter range.
  • Sequence length: Sequence length 3 balances historical context and inference cost because lengths beyond 4 yield diminishing returns.Length 5 produces only a few non-substantial improvements.

D. Blind Evaluation

A blind evaluation with 100 drivers compared the proposed sequence model against the deployed HMM-based policy across six difficult driving scenarios. Drivers generally preferred the sequence model in complex scenarios, while the tunnel exposed a setting where the HMM was preferred.

  • Evaluation setup: 100 drivers blindly compared sequence-model and HMM-based instructions across six scenarios prone to navigational difficulties.The scenarios were near double bend, mix fork, roundabout, short segment, double traffic light, and tunnel.
  • User preferences: 54% vs. 18% for roundabouts, and 45% vs. 15% for short segments, drivers preferred the sequence model over the HMM policy.The preference was accompanied by shorter reaction times for the final action element.
  • User preferences: In near double bend, mix fork, and double traffic light scenarios, most drivers perceived no significant difference, although the sequence model showed a slight preference.Its last-audio start and end reaction times were marginally better in these scenarios.
  • Limitations: 18% vs. 11% in the tunnel scenario, more drivers preferred the HMM policy over the proposed model.The proposed model also had a longer reaction time, likely because of the signal-degraded environment.
  • Overall findings: Overall, the sequence model generally outperformed the HMM policy by providing timely, contextually appropriate instructions, particularly in complex driving conditions.The evaluation links this performance to balancing informational content with cognitive load and adapting to dynamic contexts.

V. CONCLUSION

The paper concludes that its deep-learning framework enables real-time, context-aware TBT audio instructions through multi-task learning and cloud-edge collaboration. Real-world evaluations reportedly reduced yaw rates relative to HMM-based policies, while future work targets complex scenarios and personalization.

  • Conclusion: The framework uses sequence models, multi-task learning, and cloud-edge collaboration to generate real-time, context-aware TBT audio instructions.It formalizes instruction generation while balancing informational content with driver cognitive load.
  • Conclusion: Real-world A/B tests and blind evaluations demonstrated significantly reduced yaw rates compared with HMM-based policies, while ablations confirmed each model component's contribution.The method is presented as a large-scale practical application of deep learning to TBT navigation.
  • Future work: Future work will optimize performance in complex scenarios and explore personalized navigation using individual driver preferences and behaviors.
Loading 2608.29073v1…