PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantum-Grassmann-Plucker Token Mixing for Deep Learning-Based Post-Disaster Damage Assessment

Kooroush Farahkhah, Umut Lagap, Taha Rezaei, Saman Ghaffarian

arXiv:2608.30633v1cs.CVcs.LG

TL;DR

Transitional damage states remain difficult to represent and label consistently across events. This study evaluates Grassmann–Plücker token mixing and quantum-inspired extensions for paired satellite imagery, with QGP leading both seen- and unseen-event tests in accuracy and macro-F1.

  • Problem

    Reliable representation and labeling of transitional damage states under event-to-event variation remain unresolved challenges.

    Source: §5 · p. 42

  • Method

    The study applies Grassmann–Plücker token mixing to represent pairwise image-token relationships with Plücker coordinates, and introduces quantum-inspired and hybrid quantum extensions.

    Source: §2 · p. 8, §6 · p. 44

  • Results

    0.99 higher macro-F1 than standard GP on held-out Tuscaloosa; QGP also led both test sets in accuracy and macro-F1.

    Source: §5 · p. 41, §6 · p. 45

  • Takeaways & Limitations

    GP token mixing is a competitive attention-free alternative to conventional Transformer token mixing for paired satellite image damage classification.

    Source: §6 · pp. 44–45

    Tuscaloosa results are overall point estimates and do not establish general cross-disaster robustness.

    Source: §5 · p. 43

Abstract

from arXiv · show

Timely post-disaster building damage assessment from satellite imagery is a critical engineering decision support task, yet it remains constrained by class imbalance, ambiguous intermediate damage states, and limited cross-event transferability. This study presents, to our knowledge, the first application of Grassmann-Plucker (GP) token mixing to computer vision and introduces two extensions for image classification: the Quantum-inspired Grassmann-Plucker (QGP) head and the Hybrid Quantum Machine Learning Grassmann-Plucker (HQML-GP) head. The GP head represents multiscale relationships among image patch tokens by encoding subspaces formed by token pairs with Plucker coordinates; QGP enriches these coordinates with amplitude-derived probability features, whereas HQML-GP incorporates expectation values generated by a simulated quantum circuit into the geometric token representation. Paired pre- and post-event image patches from the xBD tornado dataset were processed using a frozen six-channel Vision Transformer base encoder with 16 x 16-pixel patches. The three GP-based heads were compared with multilayer perceptron and Transformer baselines under identical training, checkpoint selection, and evaluation protocols. Joplin and Moore tornado samples were used for model development and seen-event testing, while Tuscaloosa was reserved for unseen-event evaluation. QGP led both test sets in accuracy and macro-F1: 83.46% and 64.50% for the seen events, and 66.45% and 52.70% for the unseen event. Although HQML-GP obtained the highest validation macro-F1 of 65.63%, it did not surpass QGP on either test set and required substantially more training time per epoch. These results establish GP token mixing as a competitive attention-free alternative to conventional Transformer-based token mixing for paired satellite image damage classification.

1. Introduction

The introduction motivates efficient, transferable building-damage classification from paired satellite imagery and proposes geometry-aware Grassmann–Plücker token mixing with quantum-inspired and hybrid extensions.

Introduction

  • Building-damage classification faces scarce severe-damage classes, visually ambiguous intermediate states, poor generalization across disaster events, and costly, impractical backbone retraining when event-specific labels are limited, motivating parameter-efficient transfer.Source: §1 · p. 2
  • The study adapts Grassmann–Plücker token mixing from language to paired satellite-image damage classification, explicitly representing multiscale relationships between patch tokens with Plücker coordinates.The prior application was limited to text sequences and did not apply GP mixing to visual patch tokens.Source: §1 · pp. 3–4
  • A frozen six-channel ViT-B/16 encoder supports trainable head-level adaptation, reducing trainable parameters for limited event data and isolating head architecture in controlled comparisons.The comparisons include conventional pooling, attention-based mixing, GP mixing, and its quantum-inspired and hybrid extensions.Source: §1 · p. 3
  • QGP enriches Plücker coordinates with amplitude-derived probability features, while HQML-GP injects simulated quantum-circuit expectation values into the geometric representation.QGP remains entirely classical; HQML-GP processes a global Plücker descriptor through a differentiable quantum circuit.Source: §1 · p. 4
  • The evaluation asks whether GP, QGP, and HQML-GP improve on mean-pooling and Transformer-attention baselines across seen and held-out tornado events.It examines transferability, class-specific behavior, computational efficiency, predictive accuracy, trainable parameters, training cost, and inference speed.Source: §1 · p. 4

2. Background

This section frames xBD’s paired imagery and damage labels as a benchmark amid persistent class-imbalance, uncertain-damage-state, and cross-event-transfer challenges. It motivates efficient head-level geometric token mixing to represent relationships between paired image regions, including possible quantum-inspired and hybrid quantum enrichments, as an open question for paired damage classification.

Background

  • xBD and the xView2 challenge enabled systematic benchmarking with paired pre- and post-disaster imagery, building footprints, and ordinal labels, while exposing class imbalance, ambiguous intermediate states, label uncertainty, and limited cross-event generalisation.The dataset also encouraged models combining building localisation and damage classification.Source: §2 · p. 5
  • In xBD tornado cases, multi-head attention improved high-severity damage recognition, but some channel- and spatial-attention configurations were less reliable, showing attention is not automatically beneficial.The study therefore motivates evaluation using class-specific as well as aggregate metrics, since accuracy alone may conceal poor minority-class performance.Source: §2 · pp. 6–7
  • Frozen-backbone, head-level adaptation avoids retraining the complete feature extractor and enables controlled comparison of how task-specific heads use a shared visual representation.This approach is relevant when event-specific labels are limited and fully fine-tuning large Transformer backbones is computationally demanding.Source: §2 · pp. 7–8
  • Most damage-assessment models aggregate visual information through pooling, self-attention, or dense fusion, but do not explicitly represent geometric subspaces spanned by pairs of local patch embeddings.This gap matters because damage evidence is often relational.Source: §2 · p. 8

2. Background

  • GP token mixing maps token pairs to low-dimensional subspaces encoded by Plücker coordinates, explicitly representing relationships among image regions relevant to damage patterns.This contrasts with conventional token pooling, which does not explicitly represent pairwise token relationships.Source: §2 · p. 8
  • Hybrid quantum-classical methods add small quantum circuits to classical pipelines, while quantum-inspired methods remain classical and use amplitude normalization and probability-based representations.Source: §2 · p. 8
  • The study tests whether quantum-inspired probability features and simulated-circuit expectation values provide measurable nonlinear enrichment beyond geometric token mixing, not whether they demonstrate quantum advantage.These are evaluated as extensions of the standard GP representation.Source: §2 · pp. 8–9
  • Prior work had given less attention to geometric head-level adaptation, and GP mixing had not been systematically evaluated for paired damage classification with a frozen backbone.The added value of quantum-inspired or hybrid circuit-derived features beyond standard GP also remained unclear.Source: §2 · p. 9
  • To address these gaps, the study evaluates GP, QGP, and HQML-GP heads on a common frozen six-channel ViT-B/16 encoder against MLP and Transformer heads under identical protocols.The framework uses paired pre-event and post-event patches and evaluates seen- and unseen-event test sets.Source: §2 · p. 9, Figure 1 · p. 9

3. Materials and Methods · 3.1. Study design

The study isolates head-level token-mixing strategies for building-centred damage classification using paired pre- and post-event imagery and a shared frozen Vision Transformer representation.

3.1. Study design

  • The evaluation tests whether geometry-aware, quantum-inspired, and hybrid quantum-classical token-mixing heads improve classification within a frozen-backbone adaptation framework.The comparison also includes conventional and attention-based token-mixing strategies under an identical visual representation.Source: §3.1 · p. 10
  • Paired pre-event and post-event building patches are concatenated as a six-channel input so the model can analyse both disaster stages jointly.Each sample is centred on an individual building.Source: §3.1 · p. 10
  • A pretrained Vision Transformer accepts the six-channel inputs and remains frozen while only the classification head is optimised.This isolates the head architecture’s contribution under a shared visual representation.Source: §3.1 · p. 10
  • All experiments optimise only the classification head, enabling a controlled comparison of head architectures against the same frozen ViT features.The common experimental protocol holds data splits, preprocessing, backbone, objective, and model selection constant.Source: §3.1 · p. 10
  • Five heads are compared: MLP, shallow Transformer, Grassmann–Plücker (GP), Quantum-inspired Grassmann–Plücker (QGP), and Hybrid Quantum Machine Learning Grassmann–Plücker (HQML-GP).They share data splits, preprocessing, frozen backbone, optimisation objective, and model-selection protocol.Source: §3.1 · p. 10
  • The task is building-centred four-class damage classification, not full-scene building localisation or semantic segmentation.The design focuses on how head-level token mixing uses a shared frozen ViT representation for damage classification.Source: §3.1 · p. 10

3.2. Datasets and event selection

The study uses paired pre- and post-event building patches from xBD tornado events, with Joplin and Moore for model development and seen-event testing and Tuscaloosa reserved for unseen-event testing. Its event-based design distinguishes generalisation to held-out samples from known disasters from transfer to an unseen event.

3.2. Datasets and event selection

  • In xBD, each sample contains paired pre-event and post-event RGB image patches and a building-damage label y in {0,1,2,3}, with 0 = No damage and 1 = Minor damage.Source: §3.2 · p. 11
  • Case IDs 5 and 7 (Joplin and Moore) supply model development and seen-event testing, while case ID 11 (Tuscaloosa) is reserved exclusively for unseen-event testing.The Joplin and Moore samples are divided into training, validation, and seen-event test sets; Tuscaloosa is excluded from model development.Source: §3.2 · p. 11
  • The seen-event test evaluates held-out samples from disasters represented during training, whereas the unseen-event test measures transfer to an event absent from model development.The design reflects potential differences in satellite image characteristics, building typologies, damage patterns, and environmental context across events.Source: §3.2 · p. 11
  • Only the training set is balanced; validation and test sets retain natural class distributions for model selection and final evaluation.Balancing is intended to prevent the majority no-damage class from dominating optimisation while preserving realistic evaluation frequencies.Source: §3.2 · p. 12

3.3. Data preparation and class balancing

Paired pre- and post-event images were prepared as six-channel inputs, and the Joplin–Moore development pool was split and balanced to support controlled training while preserving natural distributions for validation and testing.

3.3. Data preparation and class balancing

  • Pre- and post-event images were resized to 224 × 224, converted to three-channel tensors, and concatenated into a six-channel input.The first three channels represent the pre-event image and the remaining three the post-event image.Source: §3.3 · p. 12
  • ImageNet channel-wise normalization was applied, and no data augmentation was used to keep input processing identical across classification heads.The normalization statistics were repeated for the pre-event and post-event channel groups.Source: §3.3 · pp. 12–13
  • The combined Joplin–Moore development pool was split by class-stratified sampling, assigning 15% of each damage class to validation and the remaining 85% to the initial unbalanced training pool.The validation set retains the natural class distribution and is used exclusively for model selection.Source: §3.3 · p. 13
  • The training set was balanced by randomly selecting 823 samples, while validation and seen-event and unseen-event test sets retained natural class distributions for realistic post-disaster evaluation.Stratified splitting and random under-sampling used a fixed random seed of 42 for reproducibility.Source: §3.3 · p. 13

3.4. Frozen six-channel ViT-B/16 encoder

A pretrained ViT-B/16 is adapted to paired pre- and post-event images through a six-channel patch projection, then frozen as a shared feature extractor while only the classification head is trained.

3.4. Frozen six-channel ViT-B/16 encoder

  • After the projection change, all ViT parameters are frozen, so the encoder acts as a fixed feature extractor and only the downstream classification head is optimized.The encoder is an ImageNet-21k-pretrained ViT-B/16 from checkpoint google/vit-base-patch16-224-in21k.Source: §3.4 · p. 14
  • The original three-channel RGB patch projection is replaced by a six-channel convolution for paired pre- and post-event images, preserving its output dimension and convolution configuration.The projection uses a 16 × 16-pixel kernel and stride.Source: §3.4 · p. 14
  • Six-channel projection weights duplicate the pretrained RGB weights across the two image groups, scaling each copy by one half to avoid approximately doubling activation magnitude.The first three channels process the pre-event image and the remaining three process the post-event image.Source: §3.4 · p. 14
  • At 224 × 224-pixel resolution, the frozen encoder yields 196 patch tokens of hidden dimension 768, and the classification token is excluded from the compared heads.All compared heads operate on the common sequence of patch tokens.Source: §3.4 · pp. 14–15

3.5. Classification head architectures

This section describes five damage-classification heads that use patch tokens from the same frozen six-channel ViT-B/16 encoder and differ in how they transform, mix, and aggregate them. It also details how HQML-GP adds a sample-level feature from a classically simulated quantum circuit to the token-pair geometry.

3.5. Classification head architectures

  • Only the selected head is optimized; the five evaluated architectures are MLP, Transformer, GP, QGP, and HQML-GP.Each receives the same frozen-encoder patch-token sequence.Source: §3.5 · p. 15

3.5.1. MLP and Transformer baseline heads

  • The MLP baseline mean-pools tokens before feed-forward classification, while the Transformer baseline applies self-attention token mixing before pooling.The MLP does not explicitly model interactions between individual patch tokens.Source: §3.5.1 · pp. 16–17

3.5.2. Grassmann–Plücker head

  • The GP head encodes relationships between token pairs as Plücker-coordinate subspace descriptors and fuses these geometric features with original token representations.A learnable element-wise gate balances the geometric and original-token features.Source: §3.5.2 · pp. 18–20

3.5.3. Quantum-inspired Grassmann–Plücker head

  • QGP extends GP with amplitude-derived squared, probability-like features, while keeping the computation fully classical and retaining GP’s geometric token-mixing structure.The original normalized Plücker vector is concatenated with the amplitude-derived features.Source: §3.5.3 · pp. 20–22

3.5.4. Hybrid quantum-classical Grassmann–Plücker head

  • HQML-GP combines GP’s local geometric token representations with a global feature vector produced by a small differentiable quantum circuit.Its input projection, dimensionality reduction, head partitioning, token-pair construction, and Plücker-coordinate computation are shared with GP.Source: §3.5.4 · p. 22
  • A 2,880-dimensional global Plücker descriptor is mapped to four rotation angles for a four-qubit circuit whose expectation values are projected into the model dimension and added to the GP representation.The projected feature is broadcast across token positions and participates in gating, residual feed-forward, pooling, and classification operations through a trainable scalar.Source: §3.5.4 · pp. 23–24
  • The circuit uses four qubits, angle encoding with RY rotations, one trainable RY/RZ layer, and ring-connected CNOT gates, with simulation in PennyLane and differentiable Torch backpropagation.The shared quantum feature combines sample-level circuit information with local token-pair geometry.Source: §3.5.4 · pp. 23–24
  • The classically simulated HQML-GP head investigates whether circuit-derived nonlinear global features improve the GP representation, not whether it provides quantum computational advantage.The quantum component supplements rather than replaces GP aggregation.Source: §3.5.4 · pp. 22, 24

3.6. Training, model selection, and implementation

All classification heads used a shared optimization and checkpoint-selection protocol, with validation macro-F1 guiding early stopping. Implementation differed for HQML-GP because its differentiable quantum circuit was simulated without automatic mixed precision.

3.6. Training, model selection, and implementation

  • All heads used the same training protocol, with AdamW (learning rate 5 × 10^-5; weight decay 1 × 10^-4), batch size 16, and up to 20 epochs.Gradient clipping used a maximum global norm of 1.0.Source: §3.6 · pp. 24–25
  • Head logits were converted to class probabilities with SoftMax, and trainable head parameters were optimized using categorical cross-entropy for four classes.The optimization protocol was shared across heads to ensure a controlled comparison.Source: §3.6 · p. 24
  • Validation macro-F1, rather than accuracy, drove early stopping because the validation set retained its naturally imbalanced class distribution.Training used patience of five epochs, and the selected checkpoint was evaluated on the seen- and unseen-event test sets.Source: §3.6 · p. 25
  • Automatic mixed precision was used for classical heads but disabled for HQML-GP, whose differentiable quantum circuit ran through PennyLane’s default.qubit simulator.The reported hardware was an NVIDIA GeForce RTX 3090.Source: §3.6 · pp. 25–26

3.7. Evaluation metrics

The validation macro-F1-selected checkpoint was evaluated on seen- and unseen-event test sets using classification metrics, alongside measures of computational efficiency.

3.7. Evaluation metrics

  • The validation macro-F1-selected checkpoint was independently evaluated on seen- and unseen-event test sets using accuracy, balanced accuracy, macro precision and recall, class-specific and aggregate F1, and multiclass MCC.The reported metrics also included weighted-F1.Source: §3.7 · p. 27
  • Computational efficiency was compared using trainable parameter count, average training time per epoch, and inference time per image; parameter count excluded the frozen ViT backbone.Inference time was measured in milliseconds per image using the computational configuration described in Section 3.Source: §3.7 · pp. 27–28
  • Balanced accuracy is the unweighted mean of class-specific recall and equals macro recall in single-label multiclass classification; macro-F1 weights classes equally, while weighted-F1 accounts for class support.Precision, recall, and F1 were calculated for each damage class.Source: §3.7 · p. 27

4. Results

The results compare head performance across overall metrics, damage categories, and qualitative cases, showing variation in strengths and errors. They examine confusion among intermediate damage classes and adjacent severity levels, including its worsening under transfer to the unseen event.

4.1. Overall performance, cross-event generalisation

  • HQML-GP led validation macro-F1 at 65.63% versus QGP’s 65.04%, but not either test set; it required 91.51 s per epoch versus QGP’s 35.23 s.Source: §4.1 · pp. 28, 29, 32
  • No head led every criterion: MLP achieved 70.29% seen-event balanced accuracy, while Transformer led unseen-event balanced accuracy at 62.17% and macro precision at 51.82%.Standard GP remained competitive with the MLP and Transformer baselines but did not consistently outperform them.Source: §4.1 · pp. 29, 32
  • On the seen disaster test set, QGP achieved the highest accuracy (83.46%), macro-F1 (64.50%), macro precision (62.22%), weighted F1 (84.82%), and MCC (62.52%).Source: §4.1 · p. 29

4.2. Class-specific performance and error analysis

  • Across both test sets, no-damage and destroyed classes had substantially higher F1 scores than minor- and major-damage classes, exposing difficulty distinguishing intermediate damage states.Shared model errors often involved ambiguous or context-dependent patterns, including debris, shadows, or surrounding-area changes that obscured damage to the target building.Source: §4.2 · pp. 33, 35
  • On the seen-disaster test set, QGP achieved the highest F1 for no damage (92.4%) and minor damage (49.1%); HQML-GP led destroyed (77.9%), while Transformer led major damage.Figure 2 examples are illustrative and should be interpreted with quantitative class-level results, not as estimates of error frequency.Source: §4.2 · p. 35
  • On unseen-event tests, QGP led no-damage F1 (78.4%), HQML-GP led minor-damage F1 (43.2% vs QGP’s 43.0%), and Transformer led major-damage (32.8%) and destroyed F1 (61.3%).Source: §4.2 · p. 36
  • Major damage was the weakest class: unseen-event F1 ranged from 26.6% for HQML-GP to 32.8% for Transformer, and declined substantially from seen-event performance for every architecture.Intermediate damage states remained difficult to distinguish under event-to-event variation.Source: §4.2 · pp. 36, 38
  • Errors clustered between neighboring severity classes, and unseen-event transfer particularly increased confusion between no damage and minor damage.No-damage buildings were often confused with minor damage, while major damage was frequently confused with destruction; direct confusion between no damage and destruction was comparatively uncommon.Source: §4.2 · pp. 36, 38
  • QGP had the highest seen-event no-damage recall (89.2%) and unseen-event no-damage recall (68.3%) and destroyed recall (69.9%); unseen major-damage recall stayed below 60% for every head.Seen-event destroyed-class recall exceeded 75% for every evaluated head.Source: §4.2 · pp. 36, 38

4.3. Model-specific qualitative assessment

  • Each head, including both conventional baselines and GP-based heads, uniquely classified some examples correctly, so the qualitative cases do not show one uniformly superior architecture.The examples illustrate differing decisions from a shared frozen ViT representation, but do not establish which visual features caused those predictions or a specific internal mechanism.Source: §4.3 · p. 40

5. Discussion

QGP achieved the strongest overall test-set point estimates, but class-specific strengths varied across heads. The discussion emphasizes that performance on one held-out tornado does not establish general cross-disaster robustness.

Discussion

  • QGP had the strongest overall test-set point estimates, leading key metrics on both events and improving macro-F1 over standard GP by 0.99 and 2.25 percentage points, respectively.It led accuracy, macro precision, macro-F1, weighted F1, and MCC on the seen-event set, and accuracy, macro-F1, weighted F1, and MCC on Tuscaloosa.Source: §5 · p. 41
  • Class-specific results differed: QGP led no-damage F1 on both sets and minor-damage F1 on the seen set, HQML-GP led unseen minor-damage F1 by 0.2 points, and the Transformer led major-damage F1 on both sets.The Transformer also led destroyed-class F1 on the unseen event.Source: §5 · p. 41
  • No head dominated every metric or damage category: QGP had the strongest overall estimates, while self-attention remained competitive for major-damage recognition and average class-level recall.On Tuscaloosa, the Transformer also achieved the highest balanced accuracy and macro precision.Source: §5 · p. 41
  • HQML-GP's validation macro-F1 advantage over QGP was 0.59 percentage points, but it did not transfer to either test set and required substantially more training time.The quantum-circuit expectation values provided no clear test-set benefit beyond the classical QGP representation under this configuration.Source: §5 · p. 42
  • Errors clustered between neighboring severity levels: no-damage was often confused with minor damage, and major damage with destruction, while intact-to-destroyed confusion was uncommon.The discussion attributes this pattern to ambiguity in intermediate damage states.Source: §5 · p. 42
  • Because the damage categories are ordinal, cross-entropy's equal penalty for all errors fails to distinguish adjacent-class mistakes from errors spanning multiple damage levels.The discussion identifies reliable representation and labeling of transitional states under event-to-event variation as an unresolved problem.Source: §5 · p. 42
  • All models degraded substantially on Tuscaloosa; QGP retained the highest overall accuracy, macro-F1, weighted F1, and MCC, but its macro-F1 lead over the Transformer was only 0.20 points.The Transformer had higher balanced accuracy and macro precision, so the results do not establish general cross-disaster robustness.Source: §5 · pp. 42–43
  • A single held-out tornado cannot establish whether the ranking holds across other tornadoes, regions, hazards, building types, or image-acquisition conditions.The discussion also notes that GP-based heads used fewer than 0.8 million trainable parameters and that QGP offered the most favorable observed balance of predictive performance and reported computational cost.Source: §5 · p. 43

6. Conclusions

The study introduced three Grassmann–Plücker-based heads for paired satellite-image damage classification and compared them under a controlled frozen-encoder protocol. QGP led the test results, while HQML-GP’s validation advantage did not translate to better test performance and came with greater training cost.

6. Conclusions

  • The study introduced GP, QGP, and HQML-GP heads for four-class building-damage assessment from paired pre- and post-disaster satellite imagery.The heads were evaluated using a common frozen six-channel ViT-B/16 encoder and matched data partitions, optimization, checkpoint selection, and evaluation procedures.Source: §6 · p. 44
  • QGP led five of six aggregate metrics on seen events and four of six on held-out Tuscaloosa, with the highest recall for no-damage and destroyed buildings under event transfer.Under the controlled protocol, QGP also outperformed standard GP and the conventional Transformer head while maintaining comparable inference efficiency.Source: §6 · pp. 44–45
  • HQML-GP achieved the highest validation macro-F1 and integrated differentiable quantum-circuit features, but did not surpass QGP on test performance and incurred substantially greater training cost.The conclusions position HQML-GP as exploratory rather than evidence that added hybrid quantum–classical complexity is automatically beneficial.Source: §6 · p. 44
  • The authors identify targeted quantum-inspired enrichment of strong classical representations as a more immediately practical direction than end-to-end replacement of mature deep-learning architectures.QGP outperformed standard GP and the conventional Transformer head under the controlled protocol, while HQML-GP’s additional complexity did not yield stronger test generalization.Source: §6 · p. 45
  • Performance declined on the held-out disaster, models struggled with ambiguous intermediate damage states, and evaluation was limited to one fixed split, one seed, and one unseen tornado event.The conclusions call for multi-seed and multi-event evaluation and component-level ablations of the geometric, probability-feature, and quantum-circuit components.Source: §6 · p. 45

Related papers

Loading 2608.30633v1…