PaperlayerView exact revision on arXiv

Source-linked AI summary

TailProp: content-adaptive light- and heavy-tailed propagation for vision

Jiahao Kong, Zihan Li

arXiv:2609.11081v1cs.CVcs.LG

TL;DR

Existing science-inspired vision operators generally adapt within one dynamical family, although visual representations may need different spatial interaction regimes. TailProp addresses this gap with a hierarchical backbone that mixes Gaussian and Cauchy propagation through content-conditioned TPO routing. Across classification, dense prediction, robustness, restoration, and ablations, it consistently outperforms matched propagation baselines, with TailProp-B reaching 84.4% Top-1 accuracy on ImageNet-1K.

  • Problem

    Existing science-inspired vision formulations typically construct and adapt propagation within one dynamical family, despite differing spatial interactions across samples, channels, and stages.

  • Method

    TailProp uses TPO to combine Gaussian and Cauchy stable-process propagators with content-conditioned channel-wise coefficients and fused DCT-domain computation.

  • Results

    TailProp consistently outperforms matched propagation baselines across classification, dense prediction, robustness, restoration, and controlled ablations, reaching 84.4% Top-1 accuracy on ImageNet-1K.

  • Takeaways & Limitations

    Controlled ablations support complementary two-basis propagation as an effective design principle beyond single-basis, same-family, or single-order adaptation.

  • Takeaways & Limitations

    TailProp studies only Gaussian and Cauchy bases, uses spatially shared gating, and relies on a matrix-DCT implementation that is not hardware-optimal.

Abstract

from arXiv · show

Science-inspired vision models show that explicit propagation dynamics can provide structured and interpretable alternatives to conventional token mixing. Existing formulations, however, typically construct and adapt visual propagation within a particular dynamical family, while visual representations can require substantially different spatial interactions across samples, channels, and network stages. We explore cross-regime adaptive propagation and introduce TailProp, a hierarchical vision backbone built upon the Tail Propagation Operator (TPO). TPO uses Gaussian and Cauchy stable-process propagators as complementary bases with rapidly decaying and heavy-tailed spatial influence, and predicts a content-conditioned channel-wise coefficient to adaptively combine them. Because this coefficient is spatially shared, the two responses are fused directly in the DCT domain with a single DCT/IDCT pair, yielding $O(N^{1.5})$ spatial mixing for square feature maps with $N=HW$ and fixed channel width. Across image classification, object detection, semantic segmentation, robustness, and cross-backbone restoration, TailProp consistently outperforms matched propagation baselines; TailProp-B reaches 84.4% Top-1 accuracy on ImageNet-1K, 50.3/44.8 box/mask AP under the 3x Mask R-CNN schedule, and 50.8% mIoU on ADE20K. Controlled ablations further show that these gains are not explained by single-basis propagation, an additional same-family branch, or within-family adaptive order alone, supporting complementary two-basis propagation as an effective design principle for visual representation learning.

1 INTRODUCTION

TailProp addresses the narrowness of single-family propagation by combining complementary Gaussian and Cauchy regimes with content-conditioned routing. Its fused TPO design supports efficient hierarchical vision modeling and improves performance across tasks and controlled comparisons.

  • 1 INTRODUCTION: Visual representations may require rapidly decaying or longer-range spatial influence across samples, channels, and network stages.This motivates adapting across complementary propagation regimes rather than only within one dynamical family.
  • 1 INTRODUCTION: TailProp combines Gaussian and Cauchy propagators through content-conditioned channel-wise mixing and a single fused spectral path.The hierarchical backbone stacks TPO Blocks and TPO Layers across four stages.
  • 1 INTRODUCTION: TPO retains O(N^1.5) spatial mixing for square feature maps with fixed channel width by fusing responses in the DCT domain.Spatially shared channel-wise coefficients enable a single DCT/IDCT pair.
  • 1 INTRODUCTION: TailProp achieves the best accuracy among compared spectral propagation backbones across Tiny, Small, and Base scales, reaching 82.9/84.1/84.4% Top-1 accuracy.The evaluation spans classification, dense prediction, robustness, restoration, and matched ablations.
  • 1 INTRODUCTION: The reported gains are not explained by single-basis propagation, an additional same-family branch, or within-family adaptive order alone.Matched ablations support complementary two-basis propagation and input-conditioned routing.

2 RELATED WORK

Related work spans convolutional locality, transformer attention, efficient state space modeling, and science-inspired propagation dynamics. Recent fractional and nonlocal approaches broaden adaptation within propagation families, motivating TailProp’s cross-regime design.

  • 2 RELATED WORK: CNNs model visual representations through locality, translation-equivariant inductive biases, kernels, hierarchy, and specialized aggregation.The related work describes CNNs as a dominant paradigm for visual representation learning.
  • 2 RELATED WORK: Vision Transformers use self-attention to model global dependencies among image tokens, with hierarchical and efficient variants targeting dense and high-resolution vision.These variants include windowed, cross-shaped, dual, cross-covariance, and simplified hierarchical attention designs.
  • 2 RELATED WORK: State space models provide another route to efficient long-range modeling through continuous multidimensional signals, selective state spaces, and visual scanning adaptations.Visual adaptations include Vim and VMamba.
  • 2 RELATED WORK: Science-inspired vision methods use structured inductive biases from spiking networks, diffusion processes, PDEs, heat equations, and wave equations.Examples include vHeat, HcNet, and WaveFormer.
  • 2 RELATED WORK: Fractional and nonlocal formulations vary governing operators or differential orders to support broader spatial interactions.Examples include fractional reaction-diffusion dynamics and Lévy diffusion governed by a fractional Laplacian.

3 METHOD

TailProp models visual propagation with complementary Gaussian and Cauchy regimes, mixed content-adaptively across channels and implemented efficiently in the DCT domain. Its four-stage backbone uses TPO layers to provide broader spatial responses while retaining O(N^1.5) spatial scaling for square feature maps.

  • 3.2 TAILPROP: A content-conditioned coefficient λ(X) mixes the two spectral responses per channel while remaining spatially shared.The coefficient is predicted from the input feature, allowing adaptation across samples, channels, and network stages without location-wise routing.
  • 3.2 TAILPROP: The spatially shared mixture is fused before inversion with one DCT/IDCT pair, giving O(N^1.5) spatial scaling for square maps with fixed channel width.The separable matrix-DCT implementation has complexity O(C(H^2W + HW^2)).
  • 3.2 TAILPROP: TailProp builds a four-stage hierarchical backbone whose TPO Layers contain residual TPO Blocks and MLP branches.Each TPO Block combines propagation and SiLU gating branches through element-wise fusion, while Tiny, Small, and Base scales vary depth and channel dimensions.
  • 3.1 PRELIMINARIES: GAUSSIAN AND CAUCHY PROPAGATION: TailProp combines Gaussian and Cauchy propagation through content-conditioned channel-wise mixing, covering rapidly decaying and heavy-tailed spatial influence.Gaussian propagation corresponds to α = 2 Brownian diffusion, while Cauchy propagation corresponds to α = 1 symmetric stable Lévy propagation.
  • 3.3 DISCUSSION: The two-basis formulation provides a broader response space than a single adaptive stable order because their mixture yields a frequency-dependent response not generally reducible to one order.The matched Adaptive-α ablation is designed to isolate this distinction.
  • 3.3 DISCUSSION: TailProp avoids pairwise affinities while providing global mixing with O(N^1.5) spatial scaling for square feature maps.This contrasts with self-attention's quadratic token-count scaling and with operators restricted to one dynamical family.

4 EXPERIMENT

TailProp is evaluated across classification, dense prediction, robustness, restoration, and controlled ablations, consistently outperforming matched propagation baselines. Results support complementary Gaussian–Cauchy propagation with content-conditioned routing.

  • Image Classification: 84.4% Top-1 accuracy is achieved by TailProp-B, with TailProp leading matched spectral propagation backbones across Tiny, Small, and Base scales.TailProp-T reaches 82.9% and TailProp-S reaches 84.1%; the strongest prior propagation baseline is exceeded by 0.2 points at Small and Base.
  • Object Detection and Instance Segmentation: 50.3/44.8 APb/APm is obtained by TailProp-B under the 3x schedule, with box and mask AP improving consistently across scales.TailProp-T/S/B reach 47.8/43.1, 49.4/44.2, and 50.3/44.8 APb/APm, respectively.
  • Semantic Segmentation: 50.8 mIoU is achieved by TailProp-B on ADE20K, with TailProp obtaining the highest mIoU at Tiny, Small, and Base scales.TailProp-T reaches 47.8 mIoU and TailProp-S/B reach 50.0/50.8 mIoU with computational cost close to compared spectral backbones.
  • Robustness and Cross-Backbone Generalization: 23.1 and 37.2 Top-1 accuracy are reached on ImageNet-Sketch and ImageNet-A, while restoration transfer reaches 33.51/35.69/34.72 PSNR on Set12, McMaster, and LIVE1.TailProp-B outperforms WaveFormer-B and vHeat-B on both robustness benchmarks, and TailPropIR outperforms the compared restoration backbones.
  • Core Ablation: 82.9 is the best core-ablation result, ahead of Fixed G+C at 82.3 and Learnable G+C/Adaptive-α at 82.2, supporting complementary bases and input-conditioned routing.Gaussian-only/Dual Gaussian reach 81.9 and Cauchy-only reaches 81.8; the controls separate basis diversity from extra branch capacity and single-order adaptation.
  • Mechanism Visualization: Higher λ concentrates TPO response near the source, whereas lower λ preserves stronger far-field influence through relatively stronger Cauchy response.Figure 3 shows sample- and stage-dependent mixing preferences; Figure 4 isolates propagation geometry by fixing the input and varying only λ.

5 CONCLUSION

TailProp adapts propagation across complementary Gaussian and Cauchy regimes through content-conditioned channel-wise gates and a fused spectral path. Results across tasks and ablations support two-basis propagation rather than single-basis or single-order adaptation.

  • 5 CONCLUSION: TailProp mixes Gaussian and Cauchy stable-process bases with content-conditioned channel-wise gates while retaining a fused spectral path.The design adapts between light- and heavy-tailed propagation across complementary dynamical regimes.
  • 5 CONCLUSION: Results across classification, dense prediction, robustness, cross-backbone transfer, and controlled ablations support the two-basis design.The conclusion states that gains are not explained by single-basis or single-order adaptation alone.

6 LIMITATIONS AND FUTURE WORK

TailProp identifies open directions in propagation bases, routing, implementation efficiency, and application scope.

  • TailProp studies only Gaussian and Cauchy bases, leaving broader stable families or learnable basis sets as open alternatives.
  • The spatially shared gate preserves one DCT/IDCT path but leaves efficient location-dependent routing unresolved.
  • The explicit matrix-DCT backend is not hardware-optimal, motivating fused transform kernels.
  • Future work includes extending cross-regime propagation to video, generation, multimodal learning, and embodied perception.

REPRODUCIBILITY STATEMENT

The paper documents its mathematical formulation, implementation details, architecture configurations, and training procedures to support reproducibility.

  • The TPO formulation and implementation are described in Section 3, with DCT/IDCT, boundary, and fused matrix-DCT details in Appendices A and B.
  • Appendix C reports detailed TailProp-T/S/B architecture configurations, while the training materials document optimization and data-processing procedures.

AI USE STATEMENT

The paper uses Neumann-boundary cosine-basis propagation and a separable DCT implementation, fusing Gaussian and Cauchy responses before inversion.

  • Neumann boundaries motivate reflection-based signal extension and cosine bases that diagonalize the corresponding second-difference operators.
  • For each feature-map channel, the two-dimensional transform is implemented separably with one-dimensional DCT matrices along height and width.
  • The discrete frequency grid yields Gaussian and Cauchy factors that are applied elementwise to DCT coefficients.
  • The explicit matrix-DCT implementation costs O(C(H^2W + HW^2)), or O(CN^1.5) for square maps, with O(N^1.5) spatial scaling at fixed channel width.
  • Because the coefficient is channel-wise and spatially shared, TPO fuses the two transfer responses before one inverse DCT rather than computing separate inverses.

C TAILPROP ARCHITECTURE CONFIGURATIONS

TailProp uses a four-stage hierarchical architecture with scale-specific depth, width, regularization, and normalization choices, evaluated under documented dataset and training configurations.

  • TailProp follows a four-stage vHeat-style architecture with a patch-size-4 stem, resolutions 56^2, 28^2, 14^2, and 7^2 for 224-pixel inputs, and a classifier head.
  • TailProp-T/S/B differ by stage depths, channel widths, drop-path rates, and post-normalization or layer-scale settings.
  • ImageNet-1K experiments use 1,281,167 training images, 50,000 validation images, and 1,000 classes from the ILSVRC-2012 snapshot.
  • ImageNet training uses 224-pixel inputs, BF16 autocast, AdamW, cosine decay, 20 warmup epochs, and augmentation including mixup and cutmix.
  • COCO Mask R-CNN and ADE20K UPerNet experiments initialize from corresponding ImageNet-pretrained backbones while matching key decoder, schedule, optimizer, and augmentation settings across methods.

D.4 ROBUSTNESS EVALUATION

The evaluation defines matched controls to isolate propagation basis and adaptive mixing, including fixed, learnable, and input-conditioned Gaussian–Cauchy combinations. It also specifies robustness and restoration evaluation settings for testing generalization beyond standard classification.

  • D.4 ROBUSTNESS EVALUATION: Robustness evaluation uses ImageNet-1K-pretrained classifiers without finetuning on ImageNet-Sketch’s 1,000 classes and ImageNet-A’s official 200-class subset.
  • D.4 ROBUSTNESS EVALUATION: Cross-backbone restoration replaces SwinIR token-mixing modules with TPOs while matching the reconstruction head, losses, optimizer, and iteration budget across denoising and JPEG artifact reduction.
  • E.1 CONTROL DEFINITIONS: Matched controls keep the hierarchical backbone, classifier head, resolution, optimizer, augmentation, and schedule fixed to isolate propagation basis and adaptive mixture.
  • E.1 CONTROL DEFINITIONS: Gaussian-only and Cauchy-only controls use a single propagation basis, while Fixed G+C combines both with Y = 0.5 G(X) + 0.5 C(X).
  • E.1 CONTROL DEFINITIONS: Learnable G+C uses an input-independent channel-wise gate whose final average weights are Gaussian = 0.53 and Cauchy = 0.47.
  • E.1 CONTROL DEFINITIONS: TailProp predicts input-conditioned mixing weights, whereas Dual Gaussian uses a second Gaussian branch and Adaptive α uses one content-adaptive fractional stable order.

F IMPLEMENTATION DETAILS

TailProp implements TPO blocks with separate propagation and gating branches, a fused DCT path, and distributed training infrastructure. Additional diagnostics summarize gate behavior, effective Cauchy contribution, frequency energy, and multi-source propagation responses across stages.

  • F IMPLEMENTATION DETAILS: Each TPO Block projects features into propagation and gating branches, applies TPO and LayerNorm to one branch, SiLU to the other, then combines them by element-wise modulation.
  • F IMPLEMENTATION DETAILS: The input-adaptive TPO forms Gaussian and Cauchy responses on the DCT Laplacian grid, mixes them into one response tensor, and uses a single inverse DCT.
  • F IMPLEMENTATION DETAILS: The content-conditioned gate uses GAP–MLP–Sigmoid with reduction ratio 8 to produce per-sample, per-channel, spatially shared λ(X).
  • F IMPLEMENTATION DETAILS: Training uses PyTorch DistributedDataParallel with NCCL, BF16 autocast by default, distributed sampling, rank-reduced metrics, finite-loss checks, and resumable checkpoints.
  • G ADDITIONAL MECHANISM DIAGNOSTICS: Mechanism diagnostics average λ(X) across channels, positions, blocks, stages, and validation samples before computing effective Cauchy contribution and response-only scores.
  • G ADDITIONAL MECHANISM DIAGNOSTICS: HF energy is normalized DCT energy above ρ ≥ 0.5, with Figure G.1 reporting seed means across seeds 2027, 42, and 3407.
  • ADDITIONAL DIAGNOSTICS: Figures I.1 and H.1 extend propagation inspection through deterministic multi-source responses and additional four-case ERF comparisons using the trained TailProp checkpoint.
Loading 2609.11081v1…