PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantum-Grassmann-Plucker Token Mixing for Deep Learning-Based Post-Disaster Damage Assessment

Kooroush Farahkhah, Umut Lagap, Taha Rezaei, Saman Ghaffarian

arXiv:2608.30633v1cs.CVcs.LG

TL;DR

Post-disaster building damage classification is challenged by class imbalance, ambiguous intermediate states, and limited cross-event transferability. This study evaluates GP, QGP, and HQML-GP token-mixing heads on a frozen six-channel ViT using paired xBD tornado imagery, finding QGP strongest on both seen and unseen tests. The authors note that point estimates on one held-out event do not establish general cross-disaster robustness.

  • Problem

    Post-disaster building damage assessment remains limited by scarce severe classes, ambiguous intermediate damage states, and poor generalisation across events.

  • Method

    The study compares GP, QGP, and HQML-GP classification heads with MLP and Transformer baselines on paired imagery using a common frozen six-channel ViT-B/16 encoder.

  • Results

    QGP outperformed both standard GP and the conventional Transformer head under the controlled evaluation protocol.

  • Takeaways & Limitations

    GP token mixing is presented as a competitive attention-free alternative to conventional Transformer-based token mixing for paired satellite-image damage classification.

  • Takeaways & Limitations

    Overall point estimates on Tuscaloosa do not establish general cross-disaster robustness.

Abstract

from arXiv · show

Timely post-disaster building damage assessment from satellite imagery is a critical engineering decision support task, yet it remains constrained by class imbalance, ambiguous intermediate damage states, and limited cross-event transferability. This study presents, to our knowledge, the first application of Grassmann-Plucker (GP) token mixing to computer vision and introduces two extensions for image classification: the Quantum-inspired Grassmann-Plucker (QGP) head and the Hybrid Quantum Machine Learning Grassmann-Plucker (HQML-GP) head. The GP head represents multiscale relationships among image patch tokens by encoding subspaces formed by token pairs with Plucker coordinates; QGP enriches these coordinates with amplitude-derived probability features, whereas HQML-GP incorporates expectation values generated by a simulated quantum circuit into the geometric token representation. Paired pre- and post-event image patches from the xBD tornado dataset were processed using a frozen six-channel Vision Transformer base encoder with 16 x 16-pixel patches. The three GP-based heads were compared with multilayer perceptron and Transformer baselines under identical training, checkpoint selection, and evaluation protocols. Joplin and Moore tornado samples were used for model development and seen-event testing, while Tuscaloosa was reserved for unseen-event evaluation. QGP led both test sets in accuracy and macro-F1: 83.46% and 64.50% for the seen events, and 66.45% and 52.70% for the unseen event. Although HQML-GP obtained the highest validation macro-F1 of 65.63%, it did not surpass QGP on either test set and required substantially more training time per epoch. These results establish GP token mixing as a competitive attention-free alternative to conventional Transformer-based token mixing for paired satellite image damage classification.

1. Introduction

Post-disaster building damage assessment requires timely, balanced, and transferable classification, but scarce severe classes, ambiguous intermediate states, and event variation remain persistent challenges. The study investigates geometry-aware, quantum-inspired, and hybrid token mixing within a frozen-backbone framework.

  • Accurate and timely damage information supports response prioritisation, recovery planning, and disaster risk reduction.
  • Severe damage classes are scarce, intermediate damage states are visually ambiguous, and models may generalise poorly across disaster events.
  • Fully retraining large visual backbones can be computationally demanding when event-specific labels are limited, motivating parameter-efficient adaptation.
  • Mean pooling and self-attention do not explicitly preserve or represent the geometric subspace structure of relational damage evidence.
  • GP token mixing is adapted from language modelling to visual patch tokens for paired satellite-image damage classification.
  • QGP adds amplitude-derived probability features, while HQML-GP injects expectation values from a differentiable quantum circuit into Plücker-based representations.
  • The study compares GP-based heads with MLP and Transformer baselines using a shared frozen six-channel ViT-B/16 encoder and seen- and unseen-event evaluation.

2. Background

Prior post-disaster damage-assessment research uses remote sensing, specialised architectures, attention, and Transformers to model spatial and temporal context. However, computational complexity, class imbalance, ambiguous damage states, and weak unseen-event transfer motivate efficient, class-aware evaluation and the proposed GP-based heads.

  • xBD enabled systematic benchmarking with paired pre- and post-disaster imagery, building footprints, ordinal labels, and multiple disaster events.
  • Existing CNN, Siamese, U-Net, and multistage systems often rely on specialised feature extraction and fusion architectures.
  • Specialised architectures may increase computational requirements and complicate adaptation to new disaster events.
  • Strong aggregate accuracy does not necessarily imply reliable rare-class recognition or robust transfer to unseen events.
  • Attention can improve recognition of high-severity damage, but some channel- and spatial-attention configurations produce less reliable feature emphasis.
  • Because accuracy alone may conceal minority-class weaknesses, damage systems should also be assessed with macro-F1, per-class F1, balanced accuracy, and MCC.
  • The study adopts GP, QGP, and HQML-GP heads on a common frozen six-channel representation under identical evaluation conditions.

3.1. Study design

The study isolates classification-head token mixing by using paired building-centred image patches, a shared frozen six-channel ViT feature extractor, and identical experimental protocols. Five heads are compared for four-class damage classification.

  • Each sample combines paired pre-event and post-event RGB patches into a six-channel input centred on an individual building.
  • A pretrained Vision Transformer remains frozen while only the classification head is optimised.
  • The controlled design attributes differences to head-level token mixing rather than changes to the visual encoder.
  • Five heads—MLP, shallow Transformer, GP, QGP, and HQML-GP—are evaluated under the same data splits, preprocessing, optimisation, and model selection.
  • The task is building-centred four-class damage classification rather than full-scene localisation or semantic segmentation.

3.2. Datasets and event selection

The experiments use paired building-centred tornado-event patches from xBD, with Joplin and Moore supporting development and seen-event testing and Tuscaloosa held out for unseen-event evaluation. Training is balanced, while validation and test sets retain natural class frequencies.

  • The dataset contains paired pre-event and post-event image patches with a building damage label.
  • Labels comprise four classes: No damage, Minor damage, Major damage, and Destroyed.
  • Joplin and Moore samples form the training, validation, and seen-event test sets for model development and within-event evaluation.
  • Tuscaloosa is excluded from model development and reserved exclusively for unseen-event testing.
  • Seen-event testing evaluates held-out samples from represented disasters, whereas unseen-event testing evaluates transfer to an unobserved tornado event.
  • Event differences can involve satellite-image characteristics, building typologies, damage patterns, and environmental context.
  • Only the training set is balanced; validation and test sets retain natural class distributions for model selection and final evaluation.

3.3. Data preparation and class balancing

Paired pre- and post-event images are converted into six-channel inputs, while stratified splitting and training-only undersampling address class imbalance without altering evaluation distributions.

  • Input preparation: Each pre-event and post-event image is resized to 224 × 224 pixels, bilinearly interpolated, and converted to a three-channel tensor.
  • Input preparation: The paired images are concatenated channel-wise into a six-channel tensor, with pre-event channels first and post-event channels last.
  • Input preparation: No data augmentation is applied, maintaining identical input processing across classification heads and avoiding augmentation-related variability in comparison.
  • Splitting and balancing: 15% of each damage class is assigned to validation, while the remaining 85% forms the initial unbalanced training pool with natural validation frequencies preserved.
  • Splitting and balancing: 823 samples per class are retained through random undersampling because major damage is the smallest class after validation splitting.
  • Splitting and balancing: Only training data are balanced; validation and both test sets retain natural class distributions, using random seed 42 for reproducibility.

3.4. Frozen six-channel ViT-B/16 encoder

A frozen ImageNet-21k-pretrained ViT-B/16 encoder is adapted to paired six-channel imagery, producing a common sequence of 196 patch tokens for all heads.

  • Encoder adaptation: The encoder is a ViT-B/16 model pretrained on ImageNet-21k and used as a fixed visual feature extractor.
  • Encoder adaptation: The original three-channel patch projection is replaced by a six-channel convolution while preserving its output dimension, kernel size, stride, padding, and bias configuration.
  • Encoder adaptation: Pretrained RGB projection weights are duplicated across pre-event and post-event channel groups, with both copies scaled by one half to control activation magnitude.
  • Frozen feature extraction: All remaining ViT parameters and the modified patch projection are frozen, so only the downstream classification head is optimized.
  • Token representation: At 224 × 224 resolution with 16 × 16 patches, the encoder produces 196 patch tokens with hidden dimension 768.
  • Token representation: The classification token is excluded, ensuring all heads operate on the same patch-token sequence and that differences arise from head processing.

3.5. Classification head architectures

Five heads transform the same frozen ViT patch tokens: MLP and Transformer baselines are compared with GP, QGP, and HQML-GP geometry-aware extensions.

  • Shared protocol: All five classification heads receive identical frozen-encoder patch-token representations and differ only in token transformation, mixing, and aggregation.
  • Baseline heads: The Transformer head supplies an attention-based token-mixing baseline using a trainable Transformer encoder layer before pooling and classification.
  • Grassmann–Plücker head: The GP head represents relationships between token pairs through Plücker-coordinate descriptors of geometric subspaces.
  • Quantum extensions: QGP extends GP with fully classical amplitude-derived probability-like features, whereas HQML-GP adds expectation values generated by a differentiable quantum circuit.
  • Baseline heads: The MLP head mean-pools tokens before feed-forward classification and does not explicitly model interactions between individual patch tokens.
  • Grassmann–Plücker head: GP mixing projects tokens to 64 dimensions, partitions them into four heads, and forms pairs using offsets {1,2,4,8,14,28}.
  • Grassmann–Plücker head: A learnable gate combines the original projected token representation with GP-derived geometry before residual feed-forward processing, pooling, and classification.
  • Quantum extensions: QGP retains GP’s geometric token-mixing structure while introducing compact nonlinear enrichment, and its computation does not use a quantum circuit or hardware.

3.6. Training, model selection, and implementation

All classification heads used a shared training and checkpoint-selection protocol, with logits converted to four-class probabilities and optimized using categorical cross-entropy. The study used a frozen ViT backbone and evaluated implementation settings including training stability, hardware, and computational configuration.

  • All heads were trained with the same optimization protocol for controlled comparison.
  • Four-class logits were converted to class probabilities using SoftMax before optimization.
  • Training used batch size 16, a maximum of 20 epochs, and global gradient clipping at norm 1.0.
  • Checkpoints were selected by validation macro-F1 with five-epoch patience, then evaluated on seen- and unseen-event test sets.
  • The six-channel ViT-B/16 backbone was frozen, while HQML-GP used the PennyLane default.qubit simulator without automatic mixed precision.

3.7. Evaluation metrics

The evaluation combined overall, class-balanced, class-specific, agreement, and computational metrics. Test performance was assessed separately on seen-event and unseen-event sets using checkpoints selected by validation macro-F1.

  • The selected validation-macro-F1 checkpoint was independently evaluated on seen-event and unseen-event test sets.
  • Reported predictive metrics included accuracy, balanced accuracy, macro precision, macro recall, class-specific F1, macro-F1, weighted-F1, and MCC.
  • Balanced accuracy was computed as the unweighted mean of class-specific recall values across the four damage classes.
  • Macro precision and macro recall averaged class-specific values, while macro-F1 weighted all classes equally and weighted-F1 reflected class support.
  • Multiclass MCC was calculated from the complete confusion matrix, with values ranging from −1 to 1.
  • Trainable parameters, average training time per epoch, and inference time per image were reported for computational comparison.

4. Results

QGP produced the strongest overall test-set point estimates across seen and unseen events, while class-specific advantages varied by damage category and the unseen event remained difficult. HQML-GP led validation macro-F1 but was slower to train, and intermediate damage states remained the main challenge.

  • Overall performance and cross-event generalisation: 65.63% was HQML-GP’s highest validation macro-F1, followed by QGP at 65.04%, GP at 64.59%, MLP at 64.06%, and Transformer at 63.56%.
  • Overall performance and cross-event generalisation: 83.46% seen-event accuracy and 64.50% macro-F1 were QGP’s leading test results, alongside the highest macro precision, weighted F1, and MCC.
  • Overall performance and cross-event generalisation: 66.45% unseen-event accuracy and 52.70% macro-F1 were QGP’s highest test results, although Transformer led balanced accuracy at 62.17% and macro precision at 51.82%.
  • Computational performance: 91.51 s per epoch was required by HQML-GP, compared with 35.23 s for QGP, 35.53 s for GP, 25.92 s for MLP, and 17.90 s for Transformer.
  • Class-specific performance and error analysis: No-damage and destroyed classes achieved substantially higher F1 than minor- and major-damage classes across both evaluation settings and all heads.
  • Class-specific performance and error analysis: Major-damage F1 ranged from 26.6% to 32.8% on the unseen event and declined for every architecture relative to seen-event performance.
  • Class-specific performance and error analysis: Errors predominantly occurred between neighboring severity categories, with transitional damage discrimination remaining unresolved under event-to-event variation.

5. Discussion

QGP produced the strongest overall test-set point estimates, while standard GP remained competitive with MLP and Transformer baselines. However, performance varied by damage category and degraded substantially on the held-out event, limiting claims of uniform superiority or broad cross-disaster robustness.

  • Overall performance: The standard GP head remained competitive with MLP and Transformer baselines, slightly exceeding them on the seen-disaster test set.These results support GP token mixing as a viable alternative to mean pooling and self-attention under the evaluated protocol.
  • Overall performance: QGP achieved the strongest overall point estimates, leading accuracy, macro-F1, weighted F1, and MCC on both seen and unseen-event test sets.Relative to standard GP, QGP increased macro-F1 by 0.99 percentage points on seen events and 2.25 percentage points on the unseen event.
  • Class-specific behaviour: No single classification head dominated every metric or damage category.QGP led no-damage F1 in both settings and minor-damage F1 on seen events, whereas Transformer led major-damage F1 on both sets and destroyed-class F1 on the unseen event.
  • Quantum extensions: HQML-GP achieved the highest validation macro-F1, exceeding QGP by 0.59 percentage points, but did not surpass QGP on either test set.It also required substantially longer training because its differentiable quantum circuit was evaluated with a classical simulator.
  • Error structure: Errors primarily occurred between neighbouring severity levels, especially no-damage versus minor damage and major damage versus destroyed.This pattern reflects ambiguity in intermediate damage states, while direct intact-versus-destroyed confusion was comparatively uncommon.
  • Cross-event transfer: All architectures experienced substantial performance degradation on the held-out Tuscaloosa tornado, where QGP's macro-F1 exceeded Transformer by only 0.20 percentage points.The Transformer nevertheless achieved higher balanced accuracy and macro precision on Tuscaloosa.

6. Conclusions

The study evaluates geometry-aware Grassmann–Plücker heads, including quantum-inspired and hybrid extensions, for paired satellite-image damage classification under a controlled frozen-backbone protocol. QGP delivered the strongest overall test performance, while hybrid quantum complexity improved validation but not test generalisation and cross-event performance remained limited.

  • The study introduced Grassmann–Plücker, Quantum-inspired Grassmann–Plücker, and Hybrid Quantum Machine Learning Grassmann–Plücker classification heads.These heads were evaluated on paired pre- and post-disaster imagery using a common frozen six-channel ViT-B/16 encoder and matched training conditions.
  • QGP led five of six aggregate metrics on the seen-event test set and four of six on held-out Tuscaloosa.
  • QGP achieved the highest recall for both no-damage and destroyed buildings under event transfer.
  • HQML-GP achieved the highest validation macro-F1, but its validation advantage did not translate into superior test performance and its simulated circuit increased training cost.
  • The results position fully classical QGP as more effective than additional hybrid quantum–classical complexity under the present experimental setting.The authors describe HQML-GP as a promising exploratory extension rather than automatically beneficial additional complexity.
  • Performance declined substantially on the held-out disaster, while models struggled with visually ambiguous intermediate damage states.The evaluation used one fixed split, a single random seed, and one unseen tornado event, limiting the scope of the conclusions.
Loading 2608.30633v1…