PaperlayerView exact revision on arXiv

Source-linked AI summary

From Pixels to Hierarchical Sequences: Quadtree Mask Encoding for Vision-Language Binary Change Detection

Xiao An, Ruikang Zhang, Chen Zhong, Xuli Shen, Jiaxing Sun, Jiang Wu, Wei He

arXiv:2609.09876v1cs.CV

TL;DR

Dense remote-sensing change detection requires VLMs to produce precise pixel-level masks while comparing bi-temporal imagery, but existing systems often rely on decoders or flat serialization. QUAKE-CD introduces grammar-constrained quadtree mask sequences, grounded reasoning supervision, progressive curriculum learning, and grammar-gated dual-reward RL. It achieves 78.31% accumulated F1 while outperforming decoder-based and flat text-as-mask VLMs and producing more faithful bi-temporal reasoning.

  • Problem

    Existing VLMs for remote-sensing change detection often rely on external mask decoders or flat text-as-mask serialization, limiting native structured pixel-level prediction for sparse, fragmented changes.

  • Method

    QUAKE-CD encodes binary masks as grammar-constrained quadtree sequences and combines grounded QUAKE-CoT traces with progressive curriculum learning and grammar-gated dual-reward reinforcement learning.

  • Results

    78.31% accumulated F1 was achieved, surpassing decoder-based baselines and uniform text-as-mask serializations while producing more faithful bi-temporal explanations.

  • Takeaways & Limitations

    Structured spatial language enables dense mask generation and grounded reasoning to be optimized within a unified autoregressive VLM interface.

  • Takeaways & Limitations

    The benchmarks over-represent built-environment changes, while teacher-trace biases and removal of tiles dominated by very small boundary noise constrain downstream coverage.

Abstract

from arXiv · show

Dense change detection in remote sensing requires vision-language models (VLMs) to compare bi-temporal images and generate accurate pixel-level masks. Existing VLMs are largely confined to change captioning outputs, and the few that produce pixel-level masks still rely on external decoders or flat text-as-mask serialization, which are less effective for small and fragmented changes. We introduce QUAKE-CD, a framework that recasts dense change prediction as syntax-verifiable structured generation. QUAKE-CD represents binary change masks as grammar-constrained quadtree token sequences, making the masks compact, syntactically checkable, and deterministically decodable within an autoregressive generation space. We further construct QUAKE-CoT, which pairs these sequences with chain-of-thought traces grounded in visual evidence, and jointly optimizes textual reasoning and spatial dense prediction through a progressive curriculum followed by grammar-gated dual-reward RL. On QUAKE-CoT, QUAKE-CD achieves 78.31% accumulated F1, outperforming decoder-based and flat text-as-mask VLMs while producing more faithful bi-temporal reasoning.

1 Introduction

QUAKE-CD addresses the difficulty of producing spatially grounded pixel-level change maps by representing masks as hierarchical, grammar-constrained language within the autoregressive VLM space. QUAKE-CoT and grammar-gated training jointly connect mask generation with bi-temporal reasoning, yielding stronger change-detection results.

  • Existing decoder-based VLMs separate dense prediction from autoregressive reasoning, complicating unified end-to-end token-level optimization.Flat patch-grid serialization also allocates tokens uniformly, causing spatial adhesion and poor handling of small fragmented changes.
  • QUAKE recursively refines heterogeneous regions while collapsing uniform areas, preserving fine boundaries and compressing redundant background.Its grammar makes structural errors observable during autoregressive generation.
  • QUAKE-CoT pairs hierarchical mask sequences with chain-of-thought rationales describing change type, location, and extent.A progressive curriculum first teaches valid sequence generation, then integrates bi-temporal rationales.
  • 78.31% accumulated F1 and 73.18% per-image F1 were achieved, surpassing decoder-based baselines and uniform text-as-mask serializations.Reasoning evaluations also found more faithful bi-temporal explanations.
  • Grammar-gated dual-reward reinforcement learning jointly optimizes format validity, pixel-level mask accuracy, and semantic reasoning fidelity.

2 Related Work

Remote-sensing VLMs have expanded beyond captioning, but precise pixel-level interaction remains difficult. Existing approaches commonly use auxiliary mask decoders or sparse coordinate cues that delegate dense prediction to external segmenters.

  • Existing remote-sensing VLMs commonly rely on auxiliary mask decoders or sparse coordinate cues for pixel-level prediction.Sparse cues delegate dense mask generation to external segmentation systems.

3 Methodology

QUAKE-CD formulates dense change detection as structured autoregressive generation, encoding masks with a deterministic quadtree language and pairing them with grounded reasoning. Its dataset, curriculum, reward model, and dual-reward reinforcement learning align syntax, mask accuracy, and semantic faithfulness.

  • Problem Formulation: QUAKE-CD generates reasoning, summary, and quadtree-mask spans, then deterministically decodes valid mask sequences into binary change maps.The sequence uses the tripartite form ythink, yanswer, and yseg, with yseg drawn from the formal language LQ.
  • QUAKE Encoding: QUAKE partitions masks into 32×32 tiles, using uniform tokens or recursively expanded quadtree nodes until the minimum leaf size ℓ.The four node bits identify foreground-containing top-left, top-right, bottom-left, and bottom-right quadrants.
  • QUAKE Encoding: QUAKE grammar supports linear-time syntactic validation, structural grid checking, and deterministic non-parametric decoding.These checks distinguish syntactic validity, structural validity, and semantic correctness.
  • QUAKE-CoT Dataset: QUAKE-CoT pairs deterministic mask encodings with teacher-generated traces covering change type, location, extent, and plausible cause.The dataset contains 27,509 training samples and a 9,301-sample test set formed from validation and test splits.
  • Grammar-Gated Dual-Reward RL: The dual-reward framework combines decoded-mask IoU rewards with a reasoning reward model to optimize spatial and semantic outputs together.Preference pairs use ground-truth traces, empirical rollouts, and hard negatives, while the combined reward is weighted by µ.

4 Experiments

Experiments show that QUAKE-CD improves mask quality, throughput, and grounded reasoning over decoder-based and flat text-as-mask baselines. Ablations attribute these gains to hierarchical encoding, staged curriculum learning, and grammar-gated rewards.

  • Dense Change Mask Generation: 78.31% accumulated F1 surpasses BIT by 1.37% and remains within 3.13% of ChangeFormer, while 73.18% per-image F1 exceeds all specialist models.QUAKE-CD also achieves the highest recall under both evaluation protocols and substantially outperforms PixelLM and LISA-7B.
  • Dense Change Mask Generation: 74.27 pairs/s is at least 1.85× faster than decoder-based VLMs, while accumulated F1 improves over Text4Seg and RSUniVLM by 53.55% and 15.97%.Throughput remains close to flat serialization methods despite QUAKE's additional hierarchical structure.
  • Dense Change Mask Generation: QUAKE-CD exceeds the same-8B baseline by 31.25% in human reasoning score and leads the 235B model by 13.20%, indicating stronger grounding than scaling alone.It also achieves 49.30 BLEU-1 and 40.24 METEOR, while semantics-aware metrics distinguish capacity from grounded supervision.
  • Ablation Studies: F1 peaks at ℓ=8 as reducing ℓ from 16 to 2 expands average sequences from 57 to 523 tokens without monotonic quality gains.Very small leaves increase grammar violations and optimization difficulty, whereas ℓ=16 erodes boundary precision.
  • Ablation Studies: Initializing Stage 2 from Stage 1 improves both F1 measures by roughly 4%, while Quadtree Reward adds about 1% accumulated F1 and raises reranker score from 68.05 to 69.72.The full dual-reward model reaches a reranker score of 71.26.
  • Ablation Studies: QUAKE-CD resolves dense small-building changes more accurately than external-decoder baselines and Text4Seg, whose predictions over-cover broad regions in dense scenes.Its textual descriptions specify change type, location, and extent.

5 Conclusion

The paper concludes that syntax-verifiable quadtree sequences provide a compact, deterministically decodable interface for autoregressive dense prediction. Round-trip fidelity and leaf-size studies support ℓ=8 as a practical balance between precision and sequence complexity.

  • 5 Conclusion: QUAKE-CD formulates dense remote-sensing change detection as structured autoregressive generation under a syntax-verifiable mask grammar.The framework combines quadtree masks, grounded reasoning traces, staged curriculum learning, and grammar-gated dual-reward reinforcement learning.
  • A Quadtree Mask Encoding Details: The grammar is linearly checkable, and deterministic decoding reconstructs masks symbolically without neural overhead during inference or reinforcement-learning reward computation.Validity checks cover wrappers, dimensions, run lengths, recursion depth, and child counts.
  • A Quadtree Mask Encoding Details: A quadtree node recursively expands only active quadrants in top-left, top-right, bottom-left, bottom-right order until reaching the minimum leaf size.Uniform tiles are represented directly, while heterogeneous regions receive finer spatial decomposition.
  • A Quadtree Mask Encoding Details: The encoding is recall-preserving because any leaf containing foreground forces foreground rendering, with precision loss confined to boundary leaves of size ℓ.This construction makes boundary quantization the source of false-positive expansion rather than foreground omission.
  • A Quadtree Mask Encoding Details: 100.0% foreground recall, 91.2% accumulated F1, and 88.0% per-image F1 validate round-trip encoding fidelity on QUAKE-CoT with ℓ=8.The decoded masks serve as supervision targets for supervised fine-tuning and reward signals for reinforcement learning.
  • A Quadtree Mask Encoding Details: ℓ=8 balances compression against precision: ℓ=2 inflates sequence length, whereas ℓ=16 reduces spatial resolution and increases per-leaf false positives.The encoder recursively subdivides heterogeneous 32×32 tiles until the minimum leaf size is reached.

B QUAKE-CoT Dataset Details

QUAKE-CoT augments bi-temporal remote-sensing benchmarks with hierarchical mask sequences and teacher-distilled reasoning traces. The resulting supervision links pixel-level change masks with linguistic explanations.

  • B QUAKE-CoT Dataset Details: QUAKE-CoT pairs pixel-level change masks with chain-of-thought reasoning traces describing grounded visual evidence.The dataset builds on established bi-temporal remote-sensing benchmarks and adds hierarchical mask sequences with teacher-distilled traces.

Data Sources and Curation

QUAKE-CoT is curated from three remote-sensing change-detection benchmarks into aligned image-pair, mask, and reasoning-trace samples. The curation combines deterministic spatial quantization, teacher-generated reasoning, human validation, filtering, and held-out evaluation splits.

  • QUAKE-CoT draws image pairs and binary masks from SYSU-CD, LEVIR-CD, and LEVIR-CD+, covering urban changes and building additions or removals.
  • Each mask is converted into a quadtree sequence after 256×256 normalization, including 16-tile partitioning for 1024×1024 LEVIR images with T=32 and ℓ=8.
  • Teacher traces compare the bi-temporal images using change categories and spatial reasoning rather than binary judgments, then undergo human cross-verification against the images and ground-truth masks.Four graduate students validate the traces, with 98.7% judged factually consistent and 1.3% revised.
  • Filtering removes inconsistent traces and ambiguous boundary-noise tiles while retaining 20% of fully empty-mask tiles to reduce unchanged-sample imbalance.
  • Held-out test splits follow the source benchmarks, and human assessments use common rubrics for factual consistency and spatial grounding.
  • The final corpus contains 36,810 aligned image-pair, mask, and reasoning-trace triples, with traces averaging 60.4 words and 84 Qwen3-VL tokens.

Licenses and Limitations

QUAKE-CoT is redistributed under the licenses of its source datasets and pairs reasoning traces with deterministic quadtree mask sequences. Its main downstream boundaries concern teacher-derived bias, built-environment-heavy data, and reduced representation of very small changes.

  • Licenses: QUAKE-CoT is released under the licenses of SYSU-CD, LEVIR-CD, and LEVIR-CD+, including the associated imagery, quadtree sequences, and reasoning traces.
  • Limitations: Teacher distillation may propagate preferred phrasings, global-appearance cues, or culturally specific descriptors despite human cross-verification.
  • Limitations: The source benchmarks over-represent building additions and removals relative to natural changes such as vegetation succession, waterbody fluctuation, and fine-grained land-cover conversion.
  • Limitations: Removing tiles dominated by sub-tile boundary noise underweights legitimate micro-changes, so QUAKE-CoT should not be the sole benchmark for sub-pixel or thin-linear change detection.
  • Preference Pair Construction: Preference training anchors the reward model with ground-truth and fabricated hard-negative responses, while rollout comparisons retain pairs only when their IoU gap exceeds the margin threshold.The resulting preference set is intended to prioritize spatial fidelity over surface phrasing.
  • Dual-Reward Specification: The reward pipeline combines grammar and structural validation with Tversky-based mask accuracy and a learned score for bi-temporal reasoning traces.Grammar failures receive zero quadtree reward, while the content score weights false negatives more strongly than false positives using α=0.3 and β=0.7.

Oracle Performance Upper Bound

Representation-oracle results show that hierarchical QUAKE encoding preserves all annotated foreground while substantially reducing the precision and F1 losses caused by flat text-as-mask serialization. End-to-end normalization further indicates that QUAKE-CD realizes more of its attainable mask quality, and tiled inference extends the approach to 1024 × 1024 imagery.

  • Oracle Performance Upper Bound: 10.35% and 9.70% higher accumulated and per-image F1, respectively, result from QUAKE’s hierarchical encoding over flat text-as-mask encoding.Both representations preserve 100.00% foreground recall, while accumulated precision rises from 68.04% to 84.03% and per-image precision from 65.23% to 81.43%.
  • Oracle Performance Upper Bound: 85.75% accumulated F1 and 82.52% per-image F1 of the representation oracle exceed the strongest alternative by 8.76% and 5.35%, respectively.QUAKE-CD’s normalized precision remains above 84% under both evaluation regimes.
  • Oracle Performance Upper Bound: 78.10% F1 on LEVIR-CD and 67.62% on LEVIR-CD+ demonstrate tiled inference on original 1024 × 1024 images without high-resolution fine-tuning or architectural modification.Recall is 98.61% on LEVIR-CD and 85.82% on LEVIR-CD+.
  • Oracle Performance Upper Bound: Qualitative tiled predictions recover large changed structures but merge adjacent buildings and expand into surrounding regions, matching the reported high-recall, lower-precision behavior.The examples include curvilinear residential development and rows of newly constructed buildings.

Generalization on Out-of-Domain Dataset

On the held-out WHU Building CD dataset, QUAKE-CD is evaluated without fine-tuning or test-time adaptation and remains competitive with specialist detectors while outperforming VLM baselines. Its higher recall than precision indicates coverage-oriented transfer across resolution, sensor, and scene shifts.

  • Generalization on Out-of-Domain Dataset: The external evaluation shifts spatial resolution, sensor, and scene appearance while preserving native resolution and the same tiled deterministic-decoding protocol.WHU uses 0.2 m/pixel imagery versus 0.3 m/pixel for LEVIR-CD and 0.5 m/pixel for SYSU-CD.
  • Generalization on Out-of-Domain Dataset: 69.10% per-image F1 surpasses the strongest specialist, ChangeFormer at 65.29%, while 71.29% accumulated F1 remains within 4.07% of it.QUAKE-CD’s accumulated recall is 78.40%, the best in the table.
  • Generalization on Out-of-Domain Dataset: 13.04 and 12.45 points separate QUAKE-CD from the strongest VLM baseline, RSUniVLM, on accumulated and per-image F1, respectively.RSUniVLM reaches 58.25% accumulated F1 and 56.65% per-image F1.
  • Generalization on Out-of-Domain Dataset: 78.10% and 82.44% of the accumulated and per-image representation-oracle F1 scores are attained without adaptation.The oracle is computed by encoding and deterministically decoding WHU ground-truth masks with QUAKE.
  • Generalization on Out-of-Domain Dataset: Attention visualizations concentrate token-conditioned responses within designated tiles and aggregated responses on changed structures across varied object scales, shapes, and backgrounds.The overlays are consistent with predicted masks and ground-truth annotations.

F Failure Mode Analysis

Failure analysis distinguishes grammar invalidity from grid-structure mismatch and shows that curriculum training and grammar-gated reinforcement learning reduce both errors. The appendix also documents the prompts and evaluator components used to construct and assess the training pipeline.

  • F Failure Mode Analysis: Grammar invalidity denotes unparsable <seg> spans, whereas structure mismatch denotes parsed grids whose dimensions differ from the expected H/32 × W/32 shape.Examples include malformed quadtree nodes, undersized run-length counts, and unmatched delimiters.
  • F Failure Mode Analysis: Grammar invalidity falls from 1.26% after curriculum supervision to 0.13% after grammar-gated dual-reward RL.The grammar gate assigns zero reward to ungrammatical outputs.
  • F Failure Mode Analysis: Structure mismatch decreases from 12.64% to 2.17% after reinforcement learning, showing that the structure score penalizes grid-dimension drift beyond grammar validation.Pad/crop fallback is triggered exactly when a structure mismatch is detected.
  • F Failure Mode Analysis: The QUAKE system prompt specifies 32×32 tiles, raster ordering, band-wise lines, and delimiter-based tile sequences for structured mask generation.These rules define the spatial layout that structure validation checks.
  • F Failure Mode Analysis: Prompted teacher traces, hard negatives, reranking, and GPT pairwise scoring support QUAKE-CoT construction and preference-based evaluation.The resulting samples pair grounded reasoning and answer traces with QUAKE segmentation sequences.
Loading 2609.09876v1…