FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions

1. Introduction

• INRs represent signals by preserving a neural-network mapping from coordinates to corresponding attributes, rather than recording elements individually.The passage contrasts this with image matrices, point clouds, and volumes.Source: §1 · p. 1
• Existing INRs favor learning low-frequency signal components, while predefined positional-encoding bases may mismatch the often-unknown frequency distribution of the signal.The paper notes that existing methods using periodic or non-periodic activations still have limited frequency support.Source: §1 · p. 2
• FINER uses variable-periodic activations and different bias-initialization ranges to select sub-functions with different frequencies, flexibly tuning the supported frequency set.The paper reports advantages over other INRs in image fitting, 3D shape representation, and neural rendering.Source: §1 · p. 2
• FINER tunes frequency by modulating the bias vector, or equivalently the phase of its variable-periodic activations, rather than optimizing weights to manipulate frequency candidates.The paper reports improved shift-invariance and NTK eigenvalue distribution with increased bias-initialization standard deviation.Source: §1 · p. 2

2.1. Implicit Neural Representation

• INRs use multilayer perceptrons to learn continuous coordinate-to-visual-signal functions, including representations of images, videos, and 3D scenes.NeRF is described as a 5D INR that reconstructs scene radiance fields, including density and view-dependent color.Source: §2.1 · p. 2

2.2. Spectral-bias Problem

• Spectral-bias methods include spatial encodings, high-pass filtering, and multi-scale architectures, but unknown signal-frequency distributions make suitable representation design difficult.The cited Fourier-feature approach maps spatial coordinates into MLP inputs; multi-scale approaches aim to capture both low- and high-frequency components.Source: §2.2 · pp. 2–3
• Periodic activations such as SIREN can represent intricate high-frequency details, but may require careful initialization and tuning, converge slowly, and retain a fixed frequency-distribution bias across inputs.The related-work discussion motivates adaptive periodic functions for complex frequency distributions.Source: §2.2 · p. 3

3.1. SIREN and the Capacity-Convergence Gap

• SIREN fits coordinate–attribute pairs with sine-activated layers, using ω0 as a parameter to control frequency.The network seeks to characterize the signal as accurately as possible.Source: §3.1 · p. 3
• SIREN’s supported functions are combinations of harmonics of ω0, but scaling its parameters without bound conflicts with convergence requirements.The text notes that the sine inputs should mainly remain within [−π, π] and describes a capacity–convergence gap.Source: §3.1 · p. 3
• Increasing SIREN’s initialization range enlarges its representable function set but violates convergence guarantees, producing a performance gap in practice.Figure 2 compares standard initialization multiplied by ω0 with initialization using a larger range for 2K image fitting.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2. Variable-periodic Activation Functions

• FINER uses sin((|x| + 1)x), whose frequency changes across shifted input regions, so bias initialization selects regions with different frequencies instead of manually tuning SIREN’s ω0.The text contrasts this behavior with sin(x), whose periodic shifts leave the activation behavior unchanged.Source: §3.2 · p. 4

4.1. Initialization scheme

• FINER initializes biases from a wider U(−k, k) range to avoid the support limits of narrow, zero-centered schemes and use further activation cycles.Traditional narrow initialization limits the supported set to the first cycle, leaving other cycles unused.Source: §4.1 · p. 4

4.2. Geometrical Perspective

• FINER’s supported frequency set contains SIREN’s and expands as the bias-initialization range increases.For bias values far from the origin, the activation’s sub-functions have further-increased frequencies.Source: §4.2 · p. 5

4.3. Neural Tangent Kernel Perspective

• Larger bias-initialization ranges significantly increase FINER NTK eigenvalues above 100 and strengthen diagonality, supporting high-frequency learning.The analysis attributes the stronger diagonal property to increased diagonal kernel elements while non-diagonal elements can be small.Source: Figure 4 · p. 5, §4.3 · p. 5

5. EXPERIMENTS

• The image-fitting evaluation learns a 2D-to-RGB function from 16 natural images at 512 × 512 resolution.The experiments also include five public-dataset shapes for SDF representation; for NeRF, the comparison follows WIRE's 25-image training setting.Source: §5.1 · p. 6, §5.2 · p. 7, §5.3 · p. 8

5.1.2 Comparisons with the State-of-the-arts

• FINER outperforms four classical INR baselines on all three image-fitting metrics, with clearer text than SIREN and PEMLP.WIRE and Gauss also recover the text but introduce artifacts in smooth backgrounds.Source: Table 1 · p. 6, §5.1.2 · p. 7

5.1.1 FINER behaviors under different initializations

• Increasing FINER's bias-initialization range expands its supported frequency set and reveals more high-frequency image detail; at matched initialization, it shows clearer details than SIREN.Figure 5 compares the resulting images and shows each image's Fourier spectrum in its upper-right box.Source: §5.1.1 · p. 6, Figure 5 · p. 6
• FINER's first-layer neurons focus on different frequencies, whereas SIREN neurons encode similar frequencies, which the authors identify as wasted neurons.Figure 6 compares four neurons with the smallest frequencies and four with the largest for each method.Source: §5.1.1 · p. 6, Figure 6 · p. 6

5.2. 3D Shape Representation

• FINER achieves the best results among the compared methods for SDF representation and consistently reconstructs both the smooth pectoral and rough shank.The visual comparison highlights SIREN's loss of shank detail, WIRE's rough pectoral, PEMLP's frequency mismatch, and Gauss's artifacts.Source: Table 2 · p. 7, §5.2 · pp. 7–8, Figure 8 · p. 7

5.3. Neural Radiance Fields Optimization

• FINER achieves the best performance in almost all eight NeRF scenes, and alone retains clear truck-frame holes and highlights that baselines over-smooth.The comparison uses 25 training images rather than the commonly used 100.Source: Table 3 · p. 8, §5.3 · p. 8, Figure 9 · p. 8

• FINER introduces variable-periodic activation functions to address limited use of activation functions' definition domains in current INRs.Source: §6 · p. 8
• Initializing the bias vector to different ranges selects different-frequency sub-functions, allowing FINER's supported frequency set to be flexibly tuned.Source: §6 · p. 8
• The paper demonstrates FINER's advantages over other INRs in image fitting, 3D shape representation, and neural rendering.Source: §6 · p. 8