PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-View Vision-Aided Proactive BS Selection and Beam Prediction for mmWave V2I Communications

Zijiao Hu, Haiyao Yu, Gaoyang Pang, Guangchen Wang, Litianyi Zhang, Wanchun Liu, George C. Alexandropoulos, Branka Vucetic, Yonghui Li

arXiv:2609.00617v1eess.SP

TL;DR

The paper addresses proactive BS selection and beam prediction for fragile, rapidly changing mmWave V2I links. It estimates building height maps from panoramic street-view and satellite images, fuses them with historical mobility, and predicts next-slot link states, rates, and beams. In geographically unseen regions, the framework outperforms evaluated deployable baselines and achieves strong LoS, rate-prediction, and throughput results.

  • Problem

    mmWave V2I needs reliable proactive BS and beam decisions despite propagation-sensitive building blockage and rapidly changing vehicular links.

  • Method

    VBHE estimates building height maps from cross-view images, while MMST combines geometric height-map features with historical mobility to predict next-slot link states, rates, and beam rankings.

  • Results

    The framework achieves 91.4% LoS classification accuracy, 0.638 bps/Hz rate-prediction MAE, and higher throughput than evaluated baselines in geographically unseen regions.

  • Takeaways & Limitations

    Estimated height maps provide useful geometric information for proactive V2I communications in geographically unseen urban regions.

Abstract

from arXiv · show

This paper investigates environmental-sensing-aided proactive base station (BS) selection and beam prediction for millimeter-wave (mmWave) vehicle-to-infrastructure (V2I) wireless systems. We exploit onboard panoramic street-view images and a preloaded satellite map to predict communication-relevant environmental information around the vehicle, including nearby building footprints and heights. The predicted height map provides a compact environmental prior and is combined with historical mobility information to jointly predict the next-slot line-of-sight (LoS) state, transmission rate, and transmit and receive beam selections. On our dataset covering different real-world regions across New South Wales, Australia, the proposed framework achieves 91.4% LoS classification accuracy, 0.638 bps/Hz mean absolute error of data rate prediction, and more than 40% higher transmission rate than the conventional reactive baseline in geographically unseen regions, outperforming all evaluated deployable learning-based baselines. The dataset and code will be released at https://github.com/Huzijiao/Cross-view_V2I

I. INTRODUCTION

Urban mmWave V2I requires proactive beamforming and handover because high-frequency links are fragile, mobility is rapid, and conventional search or pilot-based methods incur substantial overhead. The proposed framework uses cross-view imagery to estimate building geometry, then combines it with mobility information for proactive multi-task link prediction and BS/beam decisions.

  • mmWave V2I links suffer severe path loss, attenuation, and weak diffraction, complicating reliable beamforming and handover in high-mobility urban scenarios.
  • Conventional beam sweeping and pilot-based methods become expensive in vehicular scenarios because search overhead grows with codebook size and candidate-BS count.
  • Surrounding buildings often dominate blockage for rooftop-mounted VUE antennas communicating with elevated BSs, yet existing works may omit building geometry or assume its availability.
  • The framework uses onboard street-view images and a preloaded satellite map to capture environmental information and predict next-slot beam selection and handover decisions.
  • VBHE estimates building footprints and heights, while MMST fuses candidate-specific geometric features from the height map with historical mobility to jointly predict LoS, rate, and beam indices.
  • The dataset spans 18 real-world urban regions in New South Wales and contains 17K visual samples plus 290K communication-channel samples for geographically unseen-region evaluation.

2) Transmission Rate under mmWave Channel:

The transmission-rate formulation evaluates beamformed mmWave links across subcarriers and optimizes effective data transmission after accounting for beam-search and handover durations. A one-hot association selects exactly one serving BS per slot, with handover indicated by an association change.

  • The mmWave link uses transmit and receive analog beam codebooks, and the received signal is modeled for each OFDM subcarrier.
  • The transmission rate is averaged over K subcarriers with equal power allocation, P_k = P_tot/K.
  • Each time slot allocates duration to beam searching, handover, and data transmission, leaving the remainder for payload transfer.
  • The association vector is one-hot, so exactly one candidate BS serves the VUE in each time slot.
  • A handover indicator equals one when the selected BS changes from the previous slot and zero otherwise.
  • The objective jointly chooses the current BS and transmit–receive beam pair to maximize effective data transmission after search and switching costs.

2) Challenges for Solving the Rate Maximization Problem:

Directly solving rate maximization is difficult because BS association and beam indices are coupled, while exhaustive search, channel estimation, mobility, and handover introduce competing costs. The proposed proactive learner replaces unavailable next-slot channel quantities with predictions and reduces online beam-search complexity.

  • The association variable and transmit/receive beam indices are coupled, making the conventional optimization difficult to solve.
  • Exhaustive BS-beam evaluation requires channel estimation or sweeping many beam pairs, increasing beam-search duration as candidate and codebook sizes grow.
  • Rapid VUE movement changes blockage, dominant paths, and optimal beams, so pilot-based channel estimates or searched indices can become outdated.
  • Changing the serving BS incurs handover duration, so selecting the highest-rate BS can be suboptimal and may cause frequent switching.
  • The proactive approach predicts next-slot link state, rate, and beam rankings from pre-slot sensing and mobility information for each candidate BS.
  • Predicted rates select the BS, while ordered beam testing replaces exhaustive BS-beam exploration to target the effective-rate objective.

B. Learning-based Framework

The proposed learning framework uses VBHE to estimate a building height map from street-view and satellite images, then feeds this environmental representation into MMST for proactive V2I prediction.

  • B. Learning-based Framework: The framework contains two main components: VBHE for environmental extraction and MMST for next-slot BS selection and beam prediction.VBHE estimates building footprints and heights, while MMST uses the resulting representation for communication decisions.
  • B. Learning-based Framework: VBHE combines a panoramic street-view image with a satellite image to produce a building height-map representation around the VUE.The inputs are onboard 360° imagery and a preloaded satellite map.
  • B. Learning-based Framework: The height map encodes building geometry relevant to blockage, LoS/NLoS conditions, and reflected paths for downstream BS selection and beam-index prediction.This geometric information is passed to MMST as a communication-relevant environmental prior.
  • B. Learning-based Framework: VBHE comprises image preparation and segmentation, cross-view feature encoding, SVI-to-BEV feature extraction, and cross-view fusion with output.These four blocks form the visual height-map estimation pipeline.
  • B. Learning-based Framework: Building segmentation masks non-building regions before feature encoding, reducing background interference and focusing extraction on building-related visual cues.The building probability map is multiplied with each input image to suppress non-building regions.

B. Cross-View Feature Encoding Block

The cross-view feature encoding block extracts multi-scale representations from masked satellite and street-view images, adapts them to branch-specific domains, and geometrically aligns street-view features with satellite BEV features.

  • B. Cross-View Feature Encoding Block: Two parallel MSCAN encoders extract multi-scale pyramid features from the building-masked satellite and street-view images.The encoders use a two-branch architecture followed by view-specific feature adapters.
  • B. Cross-View Feature Encoding Block: The view-specific adapters project features into a compact branch-level representation and resize them to branch-specific reference resolutions before convolutional fusion.The satellite branch outputs BEV-domain features, whereas the street-view branch remains in the panorama domain.
  • B. Cross-View Feature Encoding Block: Satellite features use all pyramid levels, while street-view features preserve spatial detail and sufficient receptive field for geometric and semantic cues.The different choices reflect the satellite branch’s BEV layout and the street-view branch’s need for later projection.
  • B. Cross-View Feature Encoding Block: Panorama-domain street-view features are projected into the satellite BEV coordinate system before fusion to ensure tensor locations correspond to consistent physical locations.Without this mapping, same-index features would represent different physical positions.
  • B. Cross-View Feature Encoding Block: Ground and rooftop projections map each BEV cell to corresponding panorama coordinates, after which cue encoders resample street-view features for cross-view fusion.The two projection paths provide complementary ground and rooftop cues.

1) Ground and Rooftop Projection:

The projection stage geometrically maps BEV cells on ground and rooftops to panorama coordinates, using camera geometry and building segmentation to establish correspondence and height cues.

  • 1) Ground and Rooftop Projection:: Ground projection maps each BEV cell to panorama coordinates using standard equirectangular projection, with vertical angle and azimuth determined relative to the camera.The panorama maps zenith, horizon, and nadir to vertical coordinates 0, H′/2, and H′.
  • 1) Ground and Rooftop Projection:: The ground-plane azimuth is measured from the VUE position relative to north, while the vertical viewing angle targets the building base on the ground plane.Metric offsets and camera height determine the relevant ground distances and viewing geometry.
  • 1) Ground and Rooftop Projection:: Rooftop projection retains the BEV-cell azimuth and determines a rooftop panorama row from the building segmentation map.The topmost row whose building probability exceeds 0.5 is selected at the fixed panorama column.
  • 1) Ground and Rooftop Projection:: The rooftop elevation angle is mapped through the equirectangular panorama geometry to estimate a geometric rooftop height prior for each BEV cell.This prior is denoted by ĥgeo_m,n and is derived from the camera-to-rooftop elevation angle.

2) Ground Cue Encoder:

The ground cue encoder resamples panorama features at projected coordinates and combines them with satellite BEV features, while reliability matrices and rooftop height cues guide cross-view fusion.

  • 2) Ground Cue Encoder:: The ground cue encoder samples street-view panorama features at the ground-projection coordinates for every BEV cell.The sampled feature is then processed by a learnable ground adapter to refine ground-plane-aligned cues.
  • 2) Ground Cue Encoder:: The resulting BEV-aligned ground feature is concatenated with the satellite BEV feature for subsequent cross-view fusion.Both representations therefore share the same L×L BEV spatial organization.
  • 2) Ground Cue Encoder:: The ground reliability matrix assigns higher reliability to ground cells and lower reliability to building cells because ground projection is valid only for ground regions.Reliability is highest at building–ground boundaries, directing the network toward building footprints.
  • 2) Ground Cue Encoder:: The rooftop cue encoder similarly samples panorama features at rooftop coordinates and applies a learnable rooftop adapter to extract rooftop-aware features.This provides a complementary representation to the ground-plane cue.
  • 2) Ground Cue Encoder:: Rooftop-aware features, the geometric height prior, and the rooftop reliability matrix provide height-related cues for cross-view fusion.The rooftop matrix emphasizes nearby building cells and building–ground boundaries while downweighting ground cells.

D. Cross-view Fusion and Output Block

The framework fuses satellite and street-view BEV features with rooftop-aware information to estimate building footprints and heights. MMST then combines the predicted height map with candidate-link geometry and mobility information to represent propagation-relevant spatial and temporal context.

  • Cross-view Fusion and Output Block: The fusion feature concatenates satellite and street-view BEV features, then incorporates a rooftop-aware feature in shared BEV coordinates.The resulting feature contains both cross-view information and a building-height prior.
  • Cross-view Fusion and Output Block: Two task-specific heads use the fused representation for building-footprint segmentation and height regression.The outputs include a footprint probability map, binary footprint mask, and nonnegative physical height map.
  • Cross-view Fusion and Output Block: VBHE training combines weighted mask-segmentation and height-regression losses supervised by ground-truth footprints and building heights.The mask loss combines cross-entropy, Dice, and GIoU terms.
  • Cross-view Fusion and Output Block: MMST uses the predicted height map, VUE position, and each candidate BS position to estimate the next-slot link state and beam rankings.Its three blocks are a spatial feature encoder, temporal feature encoder, and prediction block with training objective.
  • Cross-view Fusion and Output Block: The spatial encoder extracts height-map context describing surrounding building geometry and LoS features describing propagation conditions along each VUE–BS path.A lightweight convolutional encoder compresses the height map into a geometric context embedding.

2) LoS Feature Extraction:

LoS features are constructed by sampling the predicted height map along each VUE–BS path and summarizing local height margins. These path features are encoded alongside historical relative-position information for temporal link prediction.

  • LoS Feature Extraction: Q points are sampled along each VUE–BS line segment to characterize the propagation condition from the predicted height map.Each sampled point is indexed by its height-map cell coordinates.
  • LoS Feature Extraction: At every sampled point, a local square neighborhood of side length R pixels is used to compute height differences against surrounding cells.The height margin is the sampled-point height minus each neighboring height-map value.
  • LoS Feature Extraction: Each sampled point is represented by average and minimum height margins, blockage ratio, and height-margin variation.These four statistics form the local LoS feature vector.
  • LoS Feature Extraction: The LoS features from all Q sampled points are concatenated and mapped by an MLP into a spatial embedding for each candidate BS.This embedding is combined with other spatial-temporal representations in MMST.
  • LoS Feature Extraction: A relative-position extractor represents each VUE–BS link by distance, azimuth, and elevation before a two-layer LSTM models its historical evolution.The LSTM consumes a window of T relative-position embeddings and returns the final hidden state.

C. Prediction Heads and Training Objective

The prediction block fuses temporal, LoS, and height-map embeddings before producing next-slot LoS, beam, and rate predictions. Training uses task-specific heads and a multi-task objective with uncertainty weighting.

  • Prediction Heads and Training Objective: The prediction head receives the concatenated temporal embedding, LoS feature embedding, and height-map context embedding.
  • Prediction Heads and Training Objective: Four task-specific MLP heads predict next-slot LoS, transmit-beam, receive-beam, and data-rate outputs.Transmit and receive beam rankings are formed by sorting beam probabilities and retaining the first K_beam indices.
  • Prediction Heads and Training Objective: MMST is trained with a multi-task objective combining LoS, beam, and rate losses under homoscedastic uncertainty weighting.The task weights include fixed LoS weighting and learnable uncertainty parameters adjusted during training.
  • Prediction Heads and Training Objective: The LoS label gates beam-ranking and rate objectives because those labels are meaningful only for valid LoS links.

VI. PERFORMANCE EVALUATION

The evaluation uses a two-stage dataset and cross-region protocol to assess environmental reconstruction and communication prediction. VBHE generalizes to unseen regions, while the complete framework achieves strong multi-task performance under the proposed deployment setting.

  • Dataset: The dataset contains 17K visual samples from 18 regions across four New South Wales cities and a separate communication-channel dataset for MMST.The visual data include panoramic street-view images, satellite images, and building footprint and height maps.
  • Performance Evaluation: The framework evaluates VBHE and MMST separately against task-specific baselines, using mIoU and height MAE for reconstruction and classification, rate, and beam metrics for communication.Beam prediction accuracy counts a sample as correct when the ground-truth beam appears among the predicted top-K candidates.
  • Evaluation Protocol: Cross-region testing holds out the test area during training, measuring generalization to geographically unseen regions.Subsequent evaluations use North-West Sydney as the unseen test region.
  • Evaluation of VBHE: VBHE outperforms GP and ST in building segmentation and height prediction, while GP distorts building shapes and ST oversmooths boundaries.VBHE maintains higher mIoU and lower height MAE in both in-region and cross-region evaluations.
  • Evaluation of VBHE: Removing DPT, Ground Projection, or Rooftop Projection reduces mIoU and increases height MAE, supporting the contribution of each VBHE component.

E. Evaluation of MMST

MMST is evaluated for next-slot LoS, rate, beam prediction, throughput across deployment conditions, and inference complexity. It achieves the strongest reported practical performance while retaining deployable computational requirements.

  • Prediction performance: 91.4% LoS classification accuracy and 0.638 bps/Hz rate-prediction MAE are achieved by VBHE-MMST, outperforming the evaluated baselines.VBHE-MMST also achieves the highest reported beam-prediction accuracy in the comparison.
  • Prediction performance: 90.8% BS beam prediction accuracy is achieved by MMST, compared with 89.2%, 85.7%, and 88.8% for the other baselines.
  • Ablation study: Removing any MMST component degrades overall LoS, rate, and beam prediction performance.The ablation identifies complementary contributions from height-map context, LoS, relative-position, and height-map input features.
  • Throughput under BS densities: VBHE-MMST consistently achieves the highest throughput among practical methods across BS densities and remains close to the ground-truth-height-map upper-bound model.At 100% BS density and 32.4 km/h, VBHE-MMST reaches around 8.2 bps/Hz, versus 7.57 bps/Hz for ST-MMST and 7.36 bps/Hz for GP-MMST.
  • Performance under different VUE velocities: VBHE-MMST maintains the highest data rate across evaluated VUE velocities, although average transmission rate decreases as velocity increases.Higher velocity produces faster channel changes and more frequent coverage or blockage-boundary crossings, increasing handover overhead.
  • Performance under different BS antenna numbers: VBHE-MMST achieves the highest data rate across BS antenna configurations, while 5G NR data rate decreases as antenna count grows because exhaustive beam-search overhead increases.The antenna evaluation spans N = 32 to 192 antennas.
Loading 2609.00617v1…