Source-linked AI summary
RemoteVAR: Autoregressive Visual Modeling for Remote Sensing Change Detection
Yilmaz Korkmaz, Vishal M. Patel
TL;DR
Remote sensing change detection requires precise localization despite nuisance differences across multi-temporal images, while VARs remain underexplored for pixel-level discriminative tasks. RemoteVAR addresses this gap with coarse-to-fine autoregressive mask generation conditioned by fused bi-temporal features through cross-attention and refined decoding. On standard benchmarks, it matches or surpasses competitive diffusion-, transformer-, Mamba-, and CNN-based baselines, including top results on WHU-CD and LEVIR-CD.
Problem
Visual autoregressive models have rarely been explored for pixel-level discriminative change detection, despite the task’s need to reconcile bi-temporal context with precise localization.
Method
RemoteVAR generates multi-scale change-mask tokens autoregressively from coarse to fine while conditioning the causal transformer on fused continuous pre/post features through cross-attention.
Results
RemoteVAR achieves F1 = 0.930 and IoU = 0.870 on WHU-CD, and F1 = 0.910 and IoU = 0.834 on LEVIR-CD, with top performance on both datasets.
Takeaways & Limitations
The results establish autoregressive modeling as a competitive alternative for dense remote sensing change detection against diffusion-, transformer-, Mamba-, and CNN-based baselines.
Abstract
from arXiv · showhide
Remote sensing change detection aims to localize and characterize scene changes between two time points and is central to applications such as environmental monitoring and disaster assessment. Meanwhile, visual autoregressive models (VARs) have recently shown impressive image generation capability, but their adoption for pixel-level discriminative tasks remains limited due to weak controllability, suboptimal dense prediction performance and exposure bias. We introduce RemoteVAR, a new VAR-based change detection framework that addresses these limitations by conditioning autoregressive prediction on multi-resolution fused bi-temporal features via cross-attention, and by employing an autoregressive training strategy designed specifically for change map prediction. Extensive experiments on standard change detection benchmarks show that RemoteVAR delivers consistent and significant improvements over strong diffusion-based and transformer-based baselines, establishing a competitive autoregressive alternative for remote sensing change detection. Code will be available \href{https://github.com/yilmazkorkmaz1/RemoteVAR}{\underline{here}}.
I. INTRODUCTION
Remote sensing change detection must distinguish real surface changes from nuisance differences between multi-temporal acquisitions. RemoteVAR frames this task as coarse-to-fine conditional autoregressive prediction, extending visual autoregressive modeling to pixel-level change detection.
- Motivation: Change detection compares satellite observations from different time points to identify alterations in the Earth’s surface.The task supports applications including disaster assessment, climate monitoring, policy and urban planning, land-use mapping, agriculture, and military analysis.
- Motivation: Changing illumination, imperfect registration, and other acquisition differences complicate reliable change localization.
- Related Work: Modern change detection pipelines include convolutional, transformer-based, and diffusion-based neural architectures.
- Approach: RemoteVAR generates change maps stage by stage in discrete token space, progressing from low to high resolution.This coarse-to-fine formulation reflects the need to combine global scene reasoning with precise boundary localization.
- Contribution: RemoteVAR conditions coarse-to-fine decoding on multi-resolution bi-temporal cues and addresses the limited exploration of VARs for discriminative remote sensing tasks.The framework is presented as an alternative to diffusion-based generation, which typically requires many iterative denoising steps.
II. BACKGROUND
Visual autoregressive models generate discrete visual tokens scale by scale, using multi-scale representations that establish global structure before refining details. RemoteVAR applies this autoregressive paradigm to conditional change-map prediction.
- Visual Autoregressive Models: VARs generate discrete visual tokens autoregressively from coarse to fine resolutions, forming global structure before refining details.
- Visual Autoregressive Models: A residual multi-scale VQ-VAE represents images as pyramids of token grids whose finer scales encode remaining details.Latent grids are quantized into codebook indices at varying resolutions.
- Application to Change Detection: RemoteVAR transfers scale-wise discrete token generation to conditional remote sensing change detection.
III. METHODOLOGY
RemoteVAR predicts change masks as discrete tokens across ten progressively finer scales, using bi-temporal conditioning, autoregressive training, and decoder refinement. Its design combines discrete token generation with continuous fused features to improve spatial grounding and boundary recovery.
- Tokenization: A frozen residual VQ-VAE tokenizer maps pre-image, post-image, and training mask inputs into token IDs across 10 resolutions from 1 × 1 to 16 × 16.The tokenizer uses a fixed vocabulary of size V =4096 and scale grids s × s.
- Tokenization: Binary mask tokens are converted into location-aware RGB representations to increase token diversity and mitigate foreground–background imbalance.An efficient inverse mapping recovers the binary mask.
- Autoregressive Training: The model interleaves pre-image, post-image, and mask tokens by scale, adding spatial and scale embeddings while training with teacher forcing only on mask tokens.Pre- and post-image tokens provide self-attention context, while the mask tokens receive the loss.
- Bi-temporal Conditioning: Cross-attention injects fused continuous pre/post features into the causal transformer, providing explicit spatial conditioning without an additional feature-extraction backbone.The VQ-VAE encoder is reused with shared weights for both temporal inputs, and CMX-adapted fusion modules form the cross-attention memory.
- Scale-wise Generation: Predicted mask tokens are upsampled between scales, enabling progressive refinement from coarse grids to finer spatial resolutions.For example, a 10 × 10 prediction is upsampled to 13 × 13 before the next scale is generated.
- Decoder Refinement: The VQ-VAE decoder is refined with UNet-style skip connections from multi-scale fusion modules and binary cross-entropy training while keeping the autoregressive generator fixed.This procedure sharpens boundaries and recovers fine details, including small-object changes.
A. Implementation Details
The experiments use standardized image preprocessing, cosine-scheduled AdamW training, mixed precision, and multiple public change detection datasets. Training runs on eight NVIDIA A5000 GPUs.
- Training Configuration: All images are resized to 256×256 and models are trained for 100 epochs with batch size 48 using AdamW at learning rate 1 × 10−4.Weight decay is 1 × 10−4, with cosine scheduling, linear warmup, gradient clipping, and FP16 training.
- Datasets and Hardware: Training uses the union of WHU-CD, LEVIR-CD, LEVIR-CD+, and S2Looking change detection datasets.The stated goal is to increase training samples and promote token diversity.
- Datasets and Hardware: All experiments run on 8 NVIDIA A5000 GPUs with 24 GB memory each.
B. Experimental setup.
RemoteVAR is evaluated on the LEVIR-CD and WHU-CD building change detection benchmarks using official splits and F1, IoU, and overall pixel accuracy.
- Evaluation uses LEVIR-CD and WHU-CD, two building change detection benchmarks with bi-temporal imagery from Google Earth and aerial sources.LEVIR-CD contains VHR Google Earth image pairs, while WHU-CD provides large-scale aerial imagery collected in different years.
- Performance is reported on the official dataset splits using F1 score, IoU, and overall pixel accuracy.F1 and IoU emphasize the changed class under heavy class imbalance.
C. Baselines
RemoteVAR is compared with representative diffusion, Mamba/SSM, transformer, CNN, and self-supervised change detection methods.
- The comparison includes DDPM-CD, RSMamba, ChangeFormer, BiT, SNUNet, STANet, SeCo, and SaDL-CD.These baselines span diffusion-feature, Siamese Mamba/SSM, transformer, Siamese encoder-decoder CNN, and self-supervised approaches.
- Self-supervised baselines learn representations from unlabeled temporal imagery before fine-tuning for change detection.
D. Results
RemoteVAR achieves the best reported F1 and IoU on both WHU-CD and LEVIR-CD, while qualitative comparisons show cleaner masks and more accurate localization of subtle changes.
- F1 = 0.930 and IoU = 0.870 on WHU-CD, exceeding DDPM-CD at 0.927/0.863 and RSMamba at 0.927/0.865.
- F1 = 0.910 and IoU = 0.834 on LEVIR-CD, marginally exceeding DDPM-CD at 0.909/0.833.RemoteVAR also outperforms the remaining transformer, CNN, and self-supervised baselines by varying margins.
- RemoteVAR produces cleaner masks and detects subtle structural changes with more accurate localization.The authors attribute these results to coarse-to-fine autoregressive prediction and progressive refinement across scales.
- Qualitative comparisons cover WHU-CD in the top row and LEVIR-CD in the bottom row, with true positives white, false positives green, and false negatives red.
E. Ablation Studies
The ablation study on WHU-CD examines how cross-attention conditioning, RGB-based mask token conversion, and early-scale token randomization affect RemoteVAR.
- A “No Cross-Att” variant removes cross-attention conditioning and relies on self-attention over discrete tokens.Unlike original ControlVAR, this variant remains bi-conditioned on both pre- and post-image tokens.
- “No Location Aware RGB Masks” replaces RGB-based mask token conversion with binary mask tokens, increasing token imbalance and reducing mask token diversity.
- “No TokRand” removes the early-scale token-randomization component examined in the WHU-CD ablation study.
V. CONCLUSION
RemoteVAR is a bi-temporal autoregressive framework that generates multi-scale mask tokens coarse-to-fine for remote sensing change detection. It achieves accurate localization and strong benchmark performance, matching or surpassing competitive baselines.
- RemoteVAR generates multi-scale mask tokens in a coarse-to-fine manner for bi-temporal change detection.
- RemoteVAR combines discrete token generation with fused continuous pre/post features injected through cross-attention.
- RemoteVAR matches or surpasses competitive Diffusion-, Transformer-, Mamba-, and CNN-based baselines on public benchmarks.
- The results position autoregressive modeling as a practical and effective alternative for dense remote sensing change detection.