PaperlayerView exact revision on arXiv

Source-linked AI summary

Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion

Wen Yan, Ligen Shi, Jun Qiu, Haimiao Zhang, Lina Wu, Chang Liu

arXiv:2610.11296v1cs.CV

TL;DR

MLP-based INRs favor low-frequency components, making high-frequency variations difficult to capture. This paper combines MLP and KAN branches using wavelet-domain fusion and band-separation regularization. Experiments and controlled comparisons support the framework's applicability across evaluated modalities and the value of additive fusion and regularization under the tested settings.

  • Problem

    MLP-based INRs exhibit spectral bias: they learn low-frequency components preferentially and have difficulty capturing high-frequency variations.

    Source: §I · p. 1

  • Method

    The framework pairs an MLP branch for smooth structures with a KAN branch for localized detail, fusing their wavelet coefficients additively and regularizing branch frequency responses.

    Source: §I · p. 2

  • Results

    Experiments support applicability across diverse signal dimensions and modalities, while ablations report lower reconstruction quality without band-separation loss and higher fidelity for additive fusion than fixed routing under evaluated settings.

    Source: §VI · p. 13, §VII · p. 14

  • Takeaways & Limitations

    The evaluated results support wavelet-domain coordination of heterogeneous neural representations for multidimensional implicit signal fitting.

    Source: §VII · p. 14

Abstract

from arXiv · show

Implicit Neural Representations (INRs) have emerged as a compelling paradigm for modeling multidimensional signals by mapping continuous coordinates to signal values. However, Multi-Layer Perceptrons (MLP)-based INRs inherently suffer from spectral bias, which favors low-frequency components and suppresses the reconstruction of essential high-frequency details. While existing techniques, such as Fourier feature mappings, mitigate this issue, they often rely on sensitive manual tuning and are prone to spectral artifacts. In this paper, we propose a spatial-frequency-aware INR framework that combines an MLP branch with a Kolmogorov-Arnold network (KAN) branch for complementary frequency-oriented modeling. The MLP branch provides a low-frequency-oriented representation of smooth structures, whereas the KAN branch complements localized variations and fine details. To coordinate the two branches, we integrate the discrete wavelet transform (DWT) and inverse discrete wavelet transform (IDWT) into the output fusion stage. The outputs of the two branches are decomposed into wavelet coefficients, and the corresponding coefficients are additively fused before inverse wavelet reconstruction. A wavelet-domain band-separation regularization further penalizes high-frequency responses in the MLP branch and low-frequency responses in the KAN branch, thereby encouraging complementary frequency-oriented behavior. Experiments on 1D signals, 2D images, 3D volumes and signed distance functions, videos, and 4D light-fields demonstrate the applicability of the proposed representation across the evaluated signal modalities. Results demonstrate improved reconstruction fidelity across the evaluated signal modalities.

I. Introduction

MLP-based INRs favor low-frequency components, motivating a framework that retains this tendency while pairing MLP and KAN branches for complementary signal modeling. Wavelet-domain fusion and band-separation regularization coordinate the branches.

I. Introduction

  • MLP-based INRs exhibit spectral bias, learning low-frequency components preferentially while finding high-frequency variations harder to capture.Existing approaches modify coordinate encodings, activations, or network structures to address this limitation.Source: §I · p. 1
  • The proposed dual-branch INR uses MLP for smooth components and KAN for complementary localized and fine-scale variations.The framework jointly exploits these heterogeneous representations for multidimensional signal fitting.Source: §I · p. 2
  • DWT coefficients from both branches are additively fused before IDWT reconstruction, while band-separation regularization suppresses high-frequency MLP and low-frequency KAN responses.The regularization encourages complementary tendencies without excluding either branch from the fused reconstruction.Source: §I · p. 2

A. Implicit Neural Representation

  • Other INR approaches address high-frequency modeling through coordinate encodings, activation functions, network structures, semantic structure, or learned latent frequency spaces.The proposed framework instead coordinates heterogeneous MLP/KAN representations through wavelet-domain fusion and branch-specific spectral regularization.Source: A. Implicit Neural Representation · p. 3

B. Advances in Kolmogorov-Arnold Networks

The paper uses a standard spline-based KAN alongside an MLP in a dual-branch INR, coordinating their outputs with wavelet decomposition, coefficient fusion, and band-separation regularization.

B. Advances in Kolmogorov-Arnold Networks

  • The paper adopts a standard spline-based KAN, whose learnable univariate edge functions provide a representative, relatively simple architecture for studying heterogeneous MLP/KAN representations.KAN replaces fixed MLP activations with learnable univariate functions on network edges.Source: B. Advances in Kolmogorov-Arnold… · p. 3

C. Wavelet-based Multiresolution Analysis

  • Because DWT alone does not ensure branch specialization, the framework regularizes wavelet-domain responses to constrain each branch's frequency tendencies.DWT is linear, so decomposing and recombining branch outputs does not necessarily change their modeling behavior without additional constraints.Source: C. Wavelet-based Multiresolution… · p. 3

A. Architecture Overview

  • The architecture processes coordinates through parallel MLP and KAN branches, additively fuses their DWT coefficients, and reconstructs the signal with IDWT.Band-separation regularization encourages complementary frequency responses without hard frequency assignment.Source: A. Architecture Overview · p. 3, Figure 2 · p. 3

B. Dual-Branch Spectral Decomposition

  • After wavelet fusion and IDWT, the prediction is expressed as approximation-band and detail-band components encouraged to be dominated by MLP and KAN, respectively.The text identifies the two components as the fused reconstruction's low- and high-frequency parts.Source: B. Dual-Branch Spectral Decomposition · p. 4

1) MLP Branch for Global Structure:

  • The MLP branch parameterizes smooth global structure, with its output serving as a low-frequency-oriented representation encouraged to capture slowly varying structures.Its hidden features use a fixed nonlinear activation function.Source: 1) MLP Branch for Global Structure · p. 4

2) KAN Branch for Localized Details:

The KAN branch uses learnable edge functions to represent localized variations and fine details. Wavelet coefficients from both branches are fused before inverse reconstruction, with band separation encouraging complementary frequency responses.

2) KAN Branch for Localized Details:

  • KAN edge functions are learnable univariate functions, and the branch output responds relatively strongly to localized variations, edges, textures, and fine-scale details.In the spline-based implementation, edge functions use a base function and learnable B-spline coefficients; spline degree and grid size set local order and resolution.Source: 2) KAN Branch for Localized Details · p. 4

C. Wavelet-Domain Fusion and Reconstruction

  • For 2D signals, DWT produces LL, LH, HL, and HH subbands whose corresponding MLP and KAN coefficients are additively fused before IDWT reconstruction.Both branches can contribute to each band; the regularization encourages complementary responses without requiring exclusive subband assignment.Source: Figure 3 · p. 4, C. Wavelet-Domain Fusion and… · pp. 4–5
  • The multidimensional implementation applies a J-level 1D Haar transform along sampled coordinate-axis lines, recursively decomposes approximation coefficients, and retains all detail coefficients.The same fusion and band-separation regularization are applied along each sampled axis.Source: C. Wavelet-Domain Fusion and… · pp. 4–5
  • Band separation encourages suppression of high-frequency MLP responses and low-frequency KAN responses, but does not enforce exclusive subband assignment.The described idealized limiting case fully suppresses those responses; the actual fusion still allows both branches to contribute to each band.Source: C. Wavelet-Domain Fusion and… · p. 4

D. Objective Function and Spectral Constraints

The direct-representation objective combines reconstruction fidelity with wavelet-domain band separation. The spectral penalty discourages MLP detail-band responses and KAN approximation-band responses while preserving additive fusion.

D. Objective Function and Spectral Constraints

  • Reconstruction fidelity is measured by the squared Frobenius error between reconstructed and ground-truth signals, normalized by the number of scalar elements.N is the total number of scalar elements in the signal tensor.Source: D. Objective Function and Spectral… · p. 5
  • The band-separation loss penalizes high-frequency detail coefficients from the MLP branch and low-frequency approximation coefficients from the KAN branch.The weighting parameter α controls the relative weighting of the aggregated MLP high-frequency penalty and KAN low-frequency penalty; the penalty does not impose hard subband assignment.Source: D. Objective Function and Spectral… · p. 5
  • The overall direct-representation objective weights the band-separation penalty by λ_band, which controls the strength of the spectral constraint while preserving additive coefficient fusion.The objective encourages a stronger low-frequency tendency in MLP and relatively stronger detail responses in KAN.Source: D. Objective Function and Spectral… · p. 5

E. Inverse Problem Formulation

The representation extends to inverse problems by fitting observations through a known forward operator. Parameter estimation combines data fidelity with spectral and spatial regularization, including anisotropic total variation for regularly sampled multidimensional signals.

E. Inverse Problem Formulation

  • A known forward operator maps the latent signal to the observation domain, where the measurement model includes observed data and measurement noise.Direct signal representation is the special case where the operator is the identity and the observations equal the signal.Source: E. Inverse Problem Formulation · p. 5
  • Inverse reconstruction estimates network parameters using data fidelity together with spectral and spatial regularization, then evaluates the optimized representation to obtain the latent signal.The data-fidelity loss is defined for the reconstructed signal and observed measurements.Source: E. Inverse Problem Formulation · p. 5
  • For regularly sampled multidimensional signals, anisotropic total variation sums discrete differences along each dimension; its gradient sparsity promotes noise suppression while preserving sharp transitions.The discrete difference operator is applied along each signal dimension.Source: E. Inverse Problem Formulation · pp. 5–6

A. Experimental Setup

The experiments compare the proposed representation with other methods on square waves and audio, using shared optimization settings within each task. The proposed method preserves square-wave transitions on a denser evaluation grid and achieves the highest PSNR on both audio clips.

A. Experimental Setup

  • Across tasks, methods use the same data, preprocessing, sampling, iteration budgets, and evaluation protocols; training runs for 5,000 iterations on 1D tasks.The full model uses λband = 10−3 and α = 0.5; the no-band ablation sets λband = 0.Source: A. Experimental Setup · p. 6, 2) Experimental Details · p. 6

2) Experimental Details:

  • The square-wave experiments use 128 uniformly sampled training coordinates and evaluate predictions at 256 coordinates over the same signal domain.The signal tests fitting of piecewise-constant regions and abrupt transitions.Source: 1) Datasets · p. 6, 3) Experimental Results · p. 6

3) Experimental Results:

  • On the 256-point evaluation grid, the proposed method maintains more stable values and sharper square-wave transitions than comparison methods.The evaluation grid includes coordinates not used during optimization; several other methods show transition blurring, oscillatory artifacts, or amplitude deviations.Source: 3) Experimental Results · p. 6, Figure 5 · p. 6

1) Datasets:

  • Audio fitting uses the “bach” and “counting” clips, divided into nonoverlapping 0.5-second segments that are independently fitted and then concatenated.Both clips are WAV files sampled at 44.1 kHz; all methods use the same segmentation strategy.Source: 2) Experimental Details · p. 6, 1) Datasets · p. 6

3) Experimental Results:

D. Parameterization of 2D Images

The image experiments use a 700-image DIV2K subset, resized to 512 × 512, and report the proposed method’s average performance across PSNR, SSIM, and LPIPS. The provided passages also describe evaluation settings for volumes, SDFs, and videos.

1) Datasets:

  • The image-fitting set comprises the first 700 DIV2K training images, uniformly resized to 512 × 512 and spanning varied structural and textural complexity.Images are evaluated using PSNR, SSIM, and LPIPS.Source: 1) Datasets · p. 7, 2) Experimental Details · p. 7

3) Experimental Results:

  • 45.72 dB PSNR over 700 DIV2K images exceeds the second-best FINER by 6.36 dB; the proposed method also has the best average SSIM and LPIPS.The method better preserves fine textures and structural boundaries across images with differing frequency characteristics.Source: 3) Experimental Results · p. 7, Table I · p. 9

1) Datasets:

  • For volumetric fitting, the experiments use the Head and Standard volumes; SDF reconstruction is evaluated using Chamfer distance and IoU, while volumes and videos use PSNR/SSIM/LPIPS.The Head and Standard volumes have resolutions of 256 × 256 × 256 and 181 × 217 × 181, respectively.Source: 1) Datasets · p. 7, 2) Experimental Details · pp. 8–9
  • The described 3D and video evaluations use Head and Standard volumes and BALL, COLORS, and PARALLAX sequences, with volumes and videos assessed using PSNR, SSIM, and LPIPS.The video sequences are represented as mappings from (x, y, t) to RGB; frames are resized to 480 × 270, with the first 100 frames used for fitting and evaluation.Source: 1) Datasets · p. 8, 2) Experimental Details · pp. 8–9

3) Experimental Results:

Across volume, SDF, and video tasks, the results describe preserved structures and strong reconstruction metrics, with a stated exception for video SSIM on PARALLAX. The SDF and video comparisons include quantitative tables, while the volume passage reports qualitative findings.

3) Experimental Results:

  • For Head and Standard volumes, the proposed method preserves principal anatomical structures with clearer boundaries and fewer local artifacts.The reported qualitative findings include both slowly varying regions and localized structural transitions.Source: 3) Experimental Results · pp. 9–10
  • The proposed method achieves the lowest mean Chamfer distance and highest mean IoU on Sphere, Armadillo, and Dragon.The reported results indicate reconstruction of both smooth and geometrically complex surfaces.Source: Figure 10 · p. 9, 3) Experimental Results · p. 10, Table II · p. 10
  • The proposed method leads on eight of nine video metrics, including PSNR and LPIPS across all three sequences; SIREN instead has higher SSIM on PARALLAX, 0.8545 versus 0.7744.The comparison covers BALL, COLORS, and PARALLAX.Source: 3) Experimental Results · p. 10, Table III · p. 10, Figure 11 · p. 10

1) Datasets:

  • The HCI 4D LF Benchmark contributes four scenes, each containing 9 × 9 regularly arranged subaperture images; all 81 views are used for fitting and evaluation.Source: 1) Datasets · p. 10

2) Experimental Details:

The experiments report that band-separation regularization improves reconstruction quality across tested modalities, while fixed frequency routing trades accuracy for a larger KAN high-frequency share. The supplied passages also specify light-field evaluation, computational trade-offs, and hyperparameter sensitivity.

2) Experimental Details:

  • Light-field fitting samples spatial patches and corresponding 4D coordinates across angular views; PSNR, SSIM, and LPIPS are averaged over reconstructed subaperture views.All methods use identical views, sampling, and optimization conditions and run for 5,000 iterations.Source: 2) Experimental Details · p. 10, Table IV · p. 11

G. Computational Efficiency Analysis

  • The framework generally uses more parameters and adds computational cost in higher-dimensional settings, an accuracy–computation trade-off; KAN convergence times are implementation-dependent.The standalone KAN baseline uses PyTorch, whereas the proposed 2D–4D model uses custom CUDA KAN layers.Source: G. Computational Efficiency Analysis · p. 11, Table V · p. 12

A. Ablation on Network Hyperparameters

  • KAN architecture is highly sensitive, while greater MLP depth or width does not necessarily improve reconstruction; the tested wavelet settings vary only slightly in performance.The main experiments use J = 3 and db1.Source: A. Ablation on Network Hyperparameters · pp. 11–12

B. Ablation on Band-Separation Regularization

  • Removing Lband consistently degrades reconstruction quality on 1D audio, 2D images, 3D volumes, and 4D light fields.The ablation retains DWT/IDWT and additive wavelet-domain fusion, changing only the band-separation loss.Source: B. Ablation on Band-Separation… · p. 12, Figure 13 · p. 13
  • Under both fusion strategies, Lband improves reconstruction quality and increases the KAN high-frequency proportion; fixed routing has lower accuracy despite a larger KAN high-frequency proportion.Additive fusion with band-separation regularization provides a better balance between frequency-oriented coordination and reconstruction flexibility.Source: B. Ablation on Band-Separation… · p. 12

C. Parameter-Controlled Comparison

The framework is evaluated across signal modalities from 1D signals to 4D light-fields, with controlled results supporting wavelet-domain coordination of MLP and KAN branches. Discussion findings also identify configuration dependence on signal characteristics and a trade-off among accuracy, complexity, and efficiency.

C. Parameter-Controlled Comparison

VI. Discussion

  • The MLP has a stronger low-frequency tendency and KAN relatively stronger responses to localized variations, with additive wavelet-coefficient fusion retaining both branches in reconstruction.Source: §VI · p. 13
  • Removing band-separation regularization reduces reconstruction quality across evaluated tasks, while fixed routing concentrates KAN energy in detail bands but substantially lowers accuracy.Source: §VI · p. 13
  • Wavelet level, KAN grid size, spline degree, and branch architecture depend on signal dimensionality and data characteristics, alongside a trade-off among accuracy, model complexity, and computational efficiency.Source: §VI · p. 14

VII. Conclusion

  • The proposed framework combines MLP and KAN branches through wavelet-domain decomposition and additive fusion with band-separation regularization, without exclusive subband assignment.Source: §VII · p. 14
  • Experiments across 1D signals, 2D images, 3D volumes and SDFs, videos, and 4D light-fields show applicability; ablations favor additive fusion over fixed routing and parameter controls indicate gains are not solely due to model size.Source: §VII · p. 14

Related papers

Loading 2610.11296v1…