On the approximation by single hidden layer feedforward neural networks with fixed weights

• The universal approximation property means the network class is dense in the space of continuous functions on a compact set.Source: §1 · pp. 1–2
• Fixed-weight SLFNs were shown to approximate arbitrary univariate functions, but the result did not restrict the number of hidden neurons.Fixed weights are of interest because they reduce computational expense and training time.Source: §1 · p. 2
• For Lipschitz functions, Cao and Xie gave an approximation-error bound dependent on the neuron count and the function’s Lipschitz parameters.The bound is M(1 + ∥σ∥)(b − a)n^-α.Source: §1 · p. 3
• The paper constructs a sigmoidal activation for which weight 1 and two hidden neurons suffice to approximate any continuous univariate function, while coefficients and thresholds depend on ε.It also presents a parameter-computation algorithm and examples, and shows that fixed-weight SLFNs cannot approximate all continuous multivariate functions.Source: §1 · p. 3

• The paper’s λ-increasing condition means the function stays within λ of an increasing or decreasing function on its domain.The section uses this weak monotonicity notion to describe the constructed σ.Source: §2 · p. 4
• The construction takes λ and d = b − a as inputs and defines a function that tends to 1 while remaining λ-increasing on [d, +∞).The envelope h is strictly increasing, lies between 0 and 1, and satisfies 1 − h(d) ≤ λ.Source: §2 · p. 4
• A bijection based on the Calkin–Wilf sequence and finite continued fractions enumerates monic polynomials with rational coefficients.Each polynomial is assigned a positive rational number, with the sequence determining its coefficients.Source: §2 · pp. 4–5
• On successive intervals, the construction places rescaled polynomial pieces within prescribed ranges approaching 1.The bounds use Mn, an increasing sequence converging to 1, and affine maps send [B1, B2] to a range determined by Mn.Source: §2 · p. 5
• Smooth transition functions join the interval-wise definitions, and the final extension on (−∞, d) is strictly increasing and smooth.At d, the left limit equals σ(d), and all positive-order derivatives there are zero.Source: §2 · pp. 6–7

• The algorithm computes σ at any real input, and the paper provides code, a graph, sample values, and plots showing changes as λ decreases.Source: §3 · p. 7
• The activation is sigmoidal, C∞, strictly increasing on (−∞, d), λ-strictly increasing on [d, +∞), and easily computable.Source: §3 · p. 7
• Only two translations and scalings of σ suffice for the paper’s arbitrary continuous-function approximation result.Source: §3 · p. 7

4. Main results

• On every finite segment [a, b], the constructed sigmoidal activation supports approximation of any continuous f to arbitrarily small error using constants c1, c2, θ1, and θ2.The theorem states the approximation holds for all x ∈ [a, b].Source: §4 · p. 7
• The proof approximates a target on [0, 1] by a rational-coefficient polynomial, normalizes it by its leading coefficient, and uses σ’s interval representations.The construction divides [d, +∞) into segments and expresses σ on each through a polynomial-related function.Source: §4 · pp. 8–9
• An affine change of variables extends the approximation argument from [0, 1] to any finite segment [a, b].The proof maps f to the unit interval and then transforms the approximation back to [a, b].Source: §4 · p. 10
• For compact Q ⊂ R, an algorithmically constructed computable, infinitely differentiable σ with specified monotonicity properties supports approximation of every f ∈ C(Q) to any ε > 0.The theorem specifies strict increase on (−∞, d) and λ-strict increase on [d, +∞), for any positive λ.Source: §4 · p. 10
• Monic polynomials are presented as a new numerical-analysis device for networks with limited hidden neurons; compared with [12], they reduce parameter-computation running time enough to support numerical examples.The paper contrasts this practical construction with, which established existence but did not propose constructing or using the activation.Source: §4 · p. 10

5. Numerical results

• For analytic functions with a uniformly convergent Taylor series on [a, b], the procedure uses a Taylor approximation and computes parameters for a two-neuron network.The steps rescale the interval to [0, 1], choose a Taylor polynomial within ε/2, approximate it with a rational-coefficient polynomial, and construct the network.Source: §5 · p. 11
• For the polynomial example, the paper’s sigmoidal function explicitly achieves arbitrarily given precision with two hidden-layer neurons; Heaviside SLFNs use 8, 32, 128, and 532 neurons, increasing as the error bound decreases.Source: §5 · p. 11, Table 5.1 · p. 12
• The cited passage reports an exact representation for the polynomial discussed immediately above, valid on [−1, 1], but does not specify the polynomial within the passage.The reported parameters are c1 ≈ 2059.373597, c2 ≈ −2120.974727, θ1 = −467, and θ2 = −3.Source: §5 · p. 12
• The degree-six polynomial example has no exact representation of the stated form, but the paper reports two-neuron ε-approximators for six error bounds.Table 5.2 presents computed network parameters for those error bounds.Source: §5 · p. 12, Table 5.2 · p. 12
• For two nonpolynomial targets, the reported tables show that changing the error bounds does not change the number of hidden neurons.The targets are 4x/(4 + x^2) and sin x − x cos(x + 1).Source: §5 · p. 12

6. Analysis of the multivariate case

• With one weight vector shared across all units and networks, distinct inputs having the same inner product cannot be distinguished, so approximation fails on compact sets containing both.The functional F(f) = f(x1) − f(x2) annihilates all such networks when w · x1 = w · x2.Source: §6 · pp. 13–14
• Each network in the bounded-weight class is a ridge function, and the cited non-density result for R_k implies that this class is not dense in C(R^d).The section uses H ⊂ R and the non-density of R_k to conclude that H is not dense.Source: §6 · pp. 14–15
• For any continuous activation σ, some multivariate continuous function cannot be approximated arbitrarily well when each network has a uniform bound on its number of pairwise independent weight vectors.The number of neurons may vary; the bound on independent weight vectors is shared across the networks.Source: §6 · p. 15
• The negative result is a limitation of single-hidden-layer networks with restricted weights, and the authors use it to motivate studying universal approximation in the univariate setting.The conclusion concerns a uniform bound on independent weight vectors, not unrestricted multivariate SLFNs.Source: §6 · p. 15