PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning with Volterra Neural Networks: A System Theoretic Perspective

Haoyu Yun, Hamid Krim, Yufang Bao

arXiv:2609.01928v1cs.CV

TL;DR

Explicit high-order operators are costly, motivating a more efficient and structured approach to higher-order filtering. The paper introduces kVNN, which uses order-specific learnable polynomial-kernel atoms instead of explicit high-order tensors and implements them as CNN-compatible layers. Across video action recognition, image denoising, and image classification, kVNN provides a favorable accuracy–efficiency trade-off.

  • Problem

    Explicit high-order operators can require rapidly increasing parameters and computations, while higher-order interactions remain important for signal, image, and video modeling.

  • Method

    kVNN combines Volterra’s order-wise decomposition with learnable polynomial-kernel atoms, assigning separate centers and coefficients to different interaction orders.

  • Results

    Experiments on video action recognition, image denoising, and image classification show competitive or improved performance with low model complexity and a favorable accuracy–efficiency trade-off.

  • Takeaways & Limitations

    kVNN provides an effective, compact higher-order filtering operator that preserves Volterra structure while remaining compatible with CNN-style layer construction.

  • Takeaways & Limitations

    Explicit r-th order Volterra components generally require O(d^r) coefficients, making them expensive as reusable operators in deep networks.

Abstract

from arXiv · show

Higher-order interaction components are important for signal, image, and video modeling, but explicit high-order operators often suffer from rapidly increasing parameter and computational costs. This paper presents kVNN, a learnable kernelized Volterra Neural operator for compact higher-order filtering. The motivation is to use kernelization to improve the efficiency of Volterra-type neural operators while providing a structured interpretation of their higher-order components. The proposed formulation combines the order-wise structure of Volterra filtering with learnable polynomial-kernel atoms, allowing different interaction orders to be represented by separate learnable centers and coefficients. This order-decoupled representation avoids explicit high-order tensor parameterization and can be implemented as a CNN-compatible layer. Experiments on representative vision tasks show that kVNN achieves a favorable accuracy--efficiency trade-off.

1. INTRODUCTION

The paper motivates efficient explicit higher-order filtering and proposes kVNN, which combines Volterra order structure with learnable polynomial-kernel atoms in a compact CNN-compatible operator.

  • Motivation: Explicit higher-order interactions can improve representation capacity for complex local dependencies in signal, image, and video tasks.Examples include video motion patterns, image textures, and multiscale feature couplings.
  • Motivation: Directly parameterizing high-order Volterra tensors causes parameters and computations to grow rapidly with interaction order and input dimension.Kernel methods can represent polynomial-type interactions without explicitly constructing high-dimensional monomial features.
  • Motivation: Existing kernelized convolutional methods may use fixed kernels that couple different interaction orders rather than preserving Volterra’s order-wise decomposition.Kervolution is given as an example of this limitation.
  • Proposed approach: kVNN assigns independent learnable kernel centers and coefficients to different interaction orders, explicitly decoupling linear, quadratic, and higher-order interactions.This forms a structured multi-kernel Volterra representation with compact parameterization.
  • Implementation and evaluation: The operator is implemented as a convolution-like layer that can replace conventional convolutional filters while preserving CNN-style channel organization.The CNN-compatible implementation supports evaluation across video action recognition, image denoising, and image classification.
  • Implementation and evaluation: Experiments report competitive or improved performance with low model complexity, indicating a favorable accuracy–efficiency trade-off.The paper evaluates kVNN on video action recognition, image denoising, and image classification.

2. BACKGROUND

The background frames Volterra filtering as an order-specific basis for nonlinear local operators, while motivating learnable polynomial-kernel atoms to reduce explicit tensor costs and preserve structural interpretation.

  • Volterra filtering: Volterra filtering extends linear filtering through order-specific components, with higher orders describing nonlinear couplings among input variables.A truncated finite-dimensional Volterra model represents the response as a sum of these components.
  • Higher-order representations: Higher-order representation learning includes bilinear, polynomial, tensor-based, and Volterra-inspired architectures for visual and signal modeling.These approaches have been explored for tasks including fine-grained recognition and image classification.
  • Kernelization: Polynomial kernels implicitly encode homogeneous polynomial interactions of different degrees without explicitly constructing all high-dimensional monomial features.This creates a connection between kernel methods and Volterra components.
  • Kernelization: Kernelization is therefore motivated as a way to reduce the complexity of high-order Volterra modeling while preserving a clear functional interpretation.Classical kernel methods usually rely on sample-centered expansions.
  • Existing kernelized convolution: Kernelized convolutional methods generally apply predefined kernel tricks rather than deriving operators from Volterra’s order-wise structure, leaving interaction orders potentially coupled.Their kernel forms are usually fixed in advance.
  • Motivation for kVNN: These limitations motivate compact polynomial-kernel atoms with learnable centers and coefficients assigned separately to different interaction orders.The formulation supports a CNN-compatible higher-order neural filtering layer and avoids explicit tensor parameterization.

3. METHODOLOGY

kVNN represents Volterra interactions with separate learnable polynomial-kernel atoms for each order, avoiding explicit high-order tensors. Its cells provide local approximants that aggregate into CNN-compatible layers and multilayer geometric representations.

  • Learnable Multi-Kernel Volterra Representation: kVNN represents each Volterra order with finite learnable polynomial-kernel atoms, separately parameterizing interaction orders without explicit high-order tensors.Each atom uses learnable centers and coefficients for a specified order.
  • Learnable Multi-Kernel Volterra Representation: The operator sums order-specific learnable responses, so linear, quadratic, and higher-order components remain separately represented rather than coupled in one fixed kernel.This order decoupling supports independent centers and coefficients across interaction orders.
  • Learnable Multi-Kernel Volterra Representation: Explicit r-th order Volterra tensors require O(d^r) coefficients, motivating kVNN’s compact kernelized alternative.The tensor coefficients explicitly capture r-way interactions among input coordinates.
  • Cell-Level Local Approximation: A kVNN cell provides a localized multi-kernel approximation, with separate path counts allowing lower and higher orders to receive different representation capacity.The cell’s finite-atom structure can allocate more paths to higher-order responses while remaining compact.
  • Layer-Level Multi-Chart Representation: Layers aggregate local chart approximants through linear mixing, while multilayer compositions construct global representations through inter-layer transformations.The approximation results assume compact domains, finite chart covers, smooth local representatives, and sufficient localization.
  • Implementation: Multiple kVNN cells operate in parallel like CNN filters, allowing the operator to replace convolutional layers while retaining CNN-style channel organization.The implementation remains compatible with depth-wise filtering and channel scaling.

4. EXPERIMENTS

Experiments evaluate kVNN across video recognition, image denoising, and hybrid image classification, reporting improved accuracy–efficiency trade-offs under controlled backbone settings.

  • Video Action Recognition: On UCF101, kVNN-L third-order filters reach 90.02% accuracy, exceeding VNN-L by 3.86 percentage points at comparable computational cost.In the high-capacity setting, kVNN-H third-order filters achieve the best reported accuracy of 92.67%.
  • Video Action Recognition: Under both model scales, kVNN improves the accuracy–efficiency trade-off compared with Volterra- and kernel-based operators.The comparison uses the same backbone, with latency measured per forward pass and runtime per training step.
  • Image Denoising: For DnCNN and FFDNet denoising, kVNN reduces parameters from 0.55M to 0.37M, lowers GFLOPs by about one third, and improves PSNR by 0.14 dB.For DCANet, PSNR increases from 30.47 dB to 30.56 dB while GFLOPs, latency, and training runtime also decrease.
  • Image Denoising: On Set12 at σ = 25, visual comparisons show better fine-texture and object-boundary preservation with kVNN while suppressing residual noise.The denoising setup trains with random AWGN noise sampled from U(0, 50).
  • Image Classification: In the Conformer-Ti hybrid, replacing convolutional layers with kVNN layers raises Top-1 accuracy from 81.3% to 81.6% while reducing parameters and MACs.The kVNN variant slightly reduces the number of layers while retaining the overall hybrid architecture and training setting.

5. CONCLUSION

The paper concludes that kVNN combines order-wise Volterra filtering with polynomial-kernel atoms to provide compact higher-order filtering across several signal-processing tasks.

  • 5. CONCLUSION: kVNN combines order-wise Volterra filtering with polynomial-kernel atoms in a learnable multi-kernel operator.The formulation avoids explicit high-order tensor parameterization and remains compatible with CNN-style layer construction.
  • 5. CONCLUSION: Experiments on video action recognition, image denoising, and image classification demonstrate an effective and compact higher-order filtering operator.The conclusion also connects kVNN cells and layers with local and multi-chart approximation through its geometric interpretation.
Loading 2609.01928v1…