Deep Neural Network Approximation Theory

I. INTRODUCTION

• The Kolmogorov–Donoho framework relates the bits needed to describe a function class to network connectivity and memory for storing topology and quantized weights.Source: §I · p. 2
• The paper extends prior theory to ReLU activations and depth scaling, and establishes a benefit of deep over shallow networks for sufficiently smooth functions.Source: §I · pp. 2–3
• Deep networks are Kolmogorov–Donoho optimal approximants for unit balls in Besov and modulation spaces, with invariance to time shifts, scalings, and frequency shifts.Source: §I · p. 2
• Approximation error decays exponentially in the number of nonzero weights for multiplication, polynomials, sinusoidal and smooth functions, oscillatory textures, and the Weierstrass function.For oscillatory textures and the Weierstrass function, the introduction says no previous methods were known to achieve exponential approximation accuracy.Source: §I · p. 2
• Deep networks transfer optimal approximation from affine dictionaries, such as wavelets, and Weyl–Heisenberg dictionaries to neural networks.Source: §I · p. 3

II. SETUP AND BASIC RELU CALCULUS

• The setup defines ReLU networks as compositions of affine transformations with component-wise ReLU activation, and introduces network composition and linear combinations.Source: §II · p. 4
• Networks are weighted acyclic directed graphs with edges only between adjacent layers; non-output nodes must have outgoing edges and output nodes incoming edges, since otherwise they can be removed.The paper mostly considers mappings from R^d to R, while noting that results generalize to other output dimensions.Source: §II · p. 5
• Compositions and linear combinations are basic network constructions; composing compatible networks yields depth equal to the sum of their depths.The composition lemma also bounds connectivity by 2M(Φ1) + 2M(Φ2) and width by max{2d2, W(Φ1), W(Φ2)}.Source: §II · p. 5
• A network of depth below K can be extended to depth K while preserving its input-output function, with bounded connectivity and width.Lemma II.4 gives M(Ψ) ≤ M(Φ) + d2W(Φ) + 2d2(K − L(Φ)) and W(Ψ) = max{2d2, W(Φ)}.Source: §II · p. 6
• Parallelization combines networks computing f and g into one network mapping x to (f(x), g(x)); the setup also formalizes linear combinations of equal-depth networks.The extension to networks of differing depths follows by applying the depth-extension lemma.Source: §II · pp. 6–7

Squaring

• For every ε ∈ (0, 1/2), squaring is approximated by a network with L ≤ C log(ε^-1), W = 3, and B = 1.The construction uses sawtooth-based linear interpolation, whose number of linear regions grows exponentially with depth and is stated to be optimal.Source: §III · pp. 8–10

Limitations

• The paper states that trained networks generally do not exhibit exponential growth in linear regions with depth, leaving the effect of potentially subexponential growth on approximation limits open.The theory developed here hinges critically on the optimal growth property.Source: §III · p. 10

Multiplication

• For all D > 0 and ε ∈ (0, 1/2), multiplication on [−D, D] uses depth L ≤ C(log(⌈D⌉) + log(ε^-1)), width W ≤ 5, and B = 1.The construction combines two squaring networks through the polarization identity; weights remain bounded by 1 regardless of domain size, trading depth for weight magnitude.Source: §III · pp. 10–12

Polynomials

• Polynomial approximation targets connectivity polylogarithmic in ε^-1, with monomials built by iterative composition of multiplication networks and partial sums approximated in parallel.The theorem gives a depth bound depending on polynomial degree, domain size, coefficient magnitude, and inverse error.Source: §III · p. 12

Continuous functions

• Width 9 suffices for deep ReLU networks to approximate every continuous function on a closed interval to arbitrary accuracy, yielding a finite-width universal approximation result [11],.Source: §III · p. 14

Smooth functions

• L(Ψf,ε) ≤ C(log(ε^-1))^2, with width ≤9 and weights bounded by 1, for f in the specified class S[-1,1].The construction uses Chebyshev-point Lagrangian interpolation, with polynomial approximation error at most 2^-m.Source: §III · p. 15

Smooth-function scope

• Polylogarithmic connectivity in ε^-1 is established for the specified C∞ class, but not for general Sobolev-class k-times differentiable functions [24, Thm. 4].Source: §III · p. 15

Sinusoidal functions

• L(Ψ) ≤ C((log(ε^-1))^2 + log⌈aD+|b|⌉) for approximating sin(ax+b) on [-D,D], with width ≤9 and weights bounded by 1.Cosine targets also admit width-9, weight-bounded approximants; on [-1,1] the stated error is at most ε.Source: §III · pp. 16, 18

Exponential approximation accuracy

• Approximation error ε decays at least exponentially with network connectivity for the approximants considered in this section.Their connectivity grows no faster than polylogarithmically in ε^-1.Source: §III · p. 18

IV. APPROXIMATION OF FUNCTION CLASSES AND METRIC ENTROPY

• Earlier explicit constructions approximate polynomials, smooth functions, and sinusoidal functions with error decaying exponentially in network connectivity.Source: §IV · p. 18
• Hornik and Cybenko showed that single-hidden-layer sigmoid networks approximate continuous functions on bounded domains arbitrarily well.Related shallow-network work addresses approximation bounds, localized-approximation nonexistence, lower bounds, and smooth or analytic functions,,,,,,.Source: §IV · pp. 18–19
• Multilayer-network approximation has been studied for general functions, continuous functions, and functions with their derivatives; some tasks favor depth over a single hidden layer.A function expressible by a small three-layer network may require a very large two-layer network, measured by total neuron count.Source: §IV · p. 19
• Deep ReLU networks can effectively approximate certain solution families of parametric PDEs with many, possibly infinitely many, parameters.Source: §IV · p. 19
• For functions sparse in specific wavelet frames, three-layer networks’ best M-weight approximation rate is at least the best M-term rate in piecewise linear wavelet frames.Source: §IV · p. 19
• The framework begins by reviewing deterministic lossy data compression as a theoretical foundation.The presentation follows, and builds on,.Source: §IV · p. 19

A. Kolmogorov-Donoho Rate Distortion Theory

• The analysis uses L2 error to parallel Kolmogorov-Donoho nonlinear approximation through dictionaries,, although L∞ error is stronger.For finite-measure domains, the L2 norm is bounded by |Ω|^1/2 times the L∞ norm.Source: A. Kolmogorov-Donoho Rate Distortion… · p. 20
• The minimax code length is the least length of an encoder-decoder pair achieving uniform error ε over a compact function class.The framework uses binary encoders and decoders and measures reconstruction error in the L2(Ω)-norm.Source: A. Kolmogorov-Donoho Rate Distortion… · p. 20
• The optimal exponent γ∗(C) describes minimax code-length growth as ε tends to zero, quantifying the class’s description complexity and memory requirements.The passage states that a larger γ∗(C) yields smaller growth and hence lower memory requirements for storing functions with uniformly bounded reconstruction error.Source: A. Kolmogorov-Donoho Rate Distortion… · p. 20

B. Metric entropy

• Metric entropy is log₂ of the covering number, measured in bits; ⌈log₂ N(ε; C, ρ)⌉ bits suffice to encode any x ∈ C with error at most ε.The encoder selects the closest covering-ball center, and the decoder returns that center.Source: B. Metric entropy · p. 21
• For the d-dimensional unit cube, log N(ε; C, ||·||) ≍ d log(ε^-1), with explicit packing sets establishing the scaling.Covering and packing numbers provide essentially the same measure of set size.Source: B. Metric entropy · p. 22
• For a d-dimensional unit ball in its own norm, N(ε; B, ||·||) ≍ ε^-d, so log N(ε; B, ||·||) ≍ d log(ε^-1).The volume-ratio result supplies the scaling for unit balls.Source: B. Metric entropy · p. 23
• The one-parameter function class P has N(ε; P, ||·||L∞([0,1])) ≍ ε^-1 and metric entropy scaling log N(ε; P, ||·||L∞([0,1])) ≍ log(ε^-1).The upper-bound construction gives N(ε; P, ||·||L∞([0,1])) ≤ 1/(2ε) + 2.Source: B. Metric entropy · p. 24
• Lipschitz functions on [0, 1]^d have metric entropy with exponential dependence on d, making the class significantly more massive than unit balls in R^d.The comparison is with the linear dependence on d for unit balls.Source: B. Metric entropy · pp. 24–25
• The examples follow log N(ε; C, ||·||) ≍ ε^-1/γ or ε^-1/γ log(ε^-1)^β; the optimal exponent γ* measures growth crudely, insensitive to logarithmic factors.These scaling laws are used to relate metric entropy to the optimal exponent γ*.Source: B. Metric entropy · p. 25
• The section treats functions on Euclidean domains, though its framework can be extended to functions on manifolds.The extension of this paper’s deep neural network approximation theory to manifolds is identified as future research.Source: B. Metric entropy · p. 25
• Initial results on neural-network approximation of functions on manifolds have been reported,, [66].For broader function-approximation work on manifolds, the section recommends and its references.Source: B. Metric entropy · p. 25

V. APPROXIMATION WITH DICTIONARIES

• Nonlinear best M-term approximation can choose any M basis elements, unlike linear approximation’s restriction to the first M, and can therefore achieve smaller error.For Fourier approximation, this permits selecting the frequencies that best approximate the function rather than only the lowest frequencies.Source: §V · p. 26
• An unrestricted dense dictionary can yield an infinite best M-term rate, but finding its elements may require infinite search and infinitely many bits to encode their indices.Effective approximation restricts the search to the first π(M) dictionary elements for a polynomial π, with coefficients polynomially bounded in M.Source: §V · pp. 26–27
• For compact function classes, every dictionary’s effective best M-term approximation rate is bounded above by the class’s description-complexity rate γ*(C).This gives γ*(C) an operational meaning as a limit on effective approximation.Source: §V · p. 27
• The proof encodes effective M-term approximation indices and coefficients in a bitstring of length proportional to M log(M), preserving error of the same order.The proof is presented as groundwork for the paper’s subsequent theory of best M-weight approximation with neural networks.Source: §V · p. 28

Encoding and decoding

• The encoder represents f with an O(M log(M))-bit string, and the decoder reads its indices and quantized coefficients, orthonormalizes the selected dictionary elements, and establishes claim (21).The coefficient quantization error is bounded using the chosen resolution and the number of terms.Source: §V · pp. 29–30

Known optimal exponents

• Known optimal exponents include γ∗(C) = m/d for specified Besov-space unit balls [76] and γ∗(C) = β(d −1)/2 for β-cartoon-like functions,,.Weighted modulation spaces are also listed among the examples, though their exponent is not fully visible in the supplied passage.Source: §V · p. 31

Compactness conditions

• A well-defined optimal exponent requires a relatively compact function class; effective best M-term approximations induce coverings, so their effective exponent is likewise defined only for compact classes.Translation-invariant norms on unbounded domains can obstruct compactness, while bounded domains or weighted norms can avoid that issue.Source: §V · p. 31

Tail compactness

• Tail compactness makes the effective and unconstrained best M-term exponents coincide, so polynomial search-depth and coefficient-size constraints incur no approximation-rate cost.The condition is illustrated by Besov spaces with wavelet bases and modulation spaces with Wilson bases.Source: §V · p. 32

Exponent-bound techniques

• Optimal-exponent bounds can be derived by constructing encoder-decoder pairs or by embedding orthogonal hypercubes, with an alternative modulation-space method using sequence-space entropy numbers,,.The modulation-space approach uses an isomorphism between weighted modulation spaces and weighted mixed-norm sequence spaces.Source: §V · p. 33

VI. APPROXIMATION WITH DEEP NEURAL NETWORKS

• Best M-weight approximation replaces the number of participating dictionary elements with the number of nonzero network weights.The network weights play the role of coefficients in best M-term approximation.Source: §VI · p. 33
• L2 approximation parallels dictionary theory and permits approximation of discontinuous functions, despite ReLU networks themselves being continuous.Earlier neural-network approximation results were formulated in the L∞ norm.Source: §VI · p. 33
• The unrestricted best M-weight rate optimizes over networks with at most M nonzero weights, arbitrary depth, topology, and weight choices, and benchmarks approximation algorithms.The section states that the optimal exponent γ∗(C) is a fundamental bound on the effective best M-weight approximation rate.Source: §VI · pp. 33–35
• The network’s tree-like structure removes the need for a polynomial-depth search constraint, while its architecture can be encoded using O(M log(M)) bits.The section bounds the possible locations of M nonzero weights by O(M^3).Source: §VI · p. 34
• Effective best M-weight approximation restricts depth to polylogarithmic growth in connectivity and weight magnitudes to polynomial growth.These conditions are presented as natural for the targeted approximation results and as permitting rate-distortion-optimal weight quantization; polynomially growing weights can also be replaced by bounded weights with polylogarithmic depth overhead.Source: §VI · p. 34
• The proof compares network encodings with the class’s minimax code length and establishes a lower bound on connectivity for quantized-weight networks achieving uniform error ε.For M > 0, the displayed encoding of connectivity, layer count, and layer dimensions has length at most M⌈log(M)⌉ + M + 2⌈log(M)⌉ + 1 bits.Source: §VI · pp. 35–36, and satisfies · p. 37

• The network architecture, topology, and quantized weights can be encoded in a bitstring from which the network is uniquely recovered.The encoding yields an encoder-decoder pair with bit length satisfying the established bound.Source: and satisfies · pp. 37–38
• For networks with M(Φ) ≤ ε^-k, Lemma VI.8 constructs an (m, ε)-quantized network by nearest-element weight replacement and bounds the resulting function perturbation.Source: and satisfies · p. 38
• O(ε^-1/γ*(C)) is a lower bound on connectivity growth for networks with polynomially bounded weights and polylogarithmic depth achieving uniform error ε; faster error decay than O(M^-γ*(C)) is impossible.Source: and satisfies · p. 41

VII. THE TRANSFERENCE PRINCIPLE

• For finite-radius balls in BV(R) and W m, Fourier bases are strictly suboptimal in approximation rate.The passage contrasts this with dictionary choices matched to function classes, including wavelets for Besov balls and Wilson bases for weighted modulation-space balls.Source: §VII · p. 41
• Approximating each term in a best M-term dictionary approximation adds only polylogarithmic connectivity per term, yielding overall connectivity Mπ(log(M)).The effective-representability conditions also keep the approximating weights polynomial in M.Source: §VII · p. 42
• For each target error ε, the theorem gives an approximation with L2 error at most ε, logarithmic depth, polylogarithmic connectivity, and polynomially bounded weights.The networks have (⌈π(log(ε−1))⌉, ε)-quantized weights; the theorem's implicit constant is independent of the target function.Source: §VII · pp. 43, 45
• When a dictionary is both optimal for a function class and effectively representable by neural networks, the class is optimally representable by neural networks.The section develops this transfer from dictionary approximation to neural-network approximation.Source: §VII · pp. 42–43
• The transfer theorem also holds on R^n when dictionary elements are compactly supported and support size grows at most polynomially with their index.The section points to Gabor dictionary approximation for the technical argument.Source: §VII · p. 45
• The listed example function classes are optimally represented by affine or Weyl-Heisenberg dictionaries, whose effective neural-network representability supports universal Kolmogorov-Donoho optimality for those classes.The dictionaries include wavelets, the Haar basis, curvelets, Fourier bases, and Wilson bases.Source: §VII · p. 45

• Wavelets, ridgelets, curvelets, shearlets, α-shearlets, and α-molecules are among the affine dictionaries shown to be effectively representable by neural networks.Source: §VIII · p. 45
• Any function class optimally representable by an affine dictionary with a suitable generator is optimally representable by neural networks, provided the generator can be well approximated by ReLU networks.Source: §VIII · p. 45
• For the Haar wavelet example, a neural network realizes a continuous piecewise linear approximation to the mother wavelet, with shrinking transition regions yielding error no greater than ε.Source: §VIII · p. 46

A. Affine Dictionaries with Canonical Ordering

• An affine dictionary uses compactly supported generators on a bounded domain and full-rank affine maps whose eigenvalues are bounded below in absolute value by 1.Source: A. Affine Dictionaries with Canonical… · p. 47
• Canonical ordering allows arbitrary order within each D_j and imposes a very weak restriction on dilation growth, without requiring dilations to be ordered by size.The authors note that they know of no affine dictionaries in the literature that violate this growth condition.Source: A. Affine Dictionaries with Canonical… · p. 47
• The definition separately specifies nondegeneracy through a condition on each sub-dictionary D_s,j.Source: A. Affine Dictionaries with Canonical… · p. 47
• For Haar wavelets, approximating-network weights grow linearly with the dictionary-element index, a behavior the authors say morally extends to general nondegenerate affine dictionaries.The Haar example uses d = 1, Ω = [0, 1], and dilation A_1,j = 2^(j−1).Source: A. Affine Dictionaries with Canonical… · p. 47

B. Invariance to Affine Transformations

• Translations and dilations can be absorbed into a network’s first layer, so the transformed function inherits the generator’s approximation properties.The proposition quantifies effects on connectivity, weight storage, and the approximation domain.Source: B. Invariance to Affine Transformations · p. 48
• For full-rank A and translation e, the affine-invariance proposition bounds the transformed network’s complexity using F = dE∥A∥∞ + ∥e∥∞.The transformed approximation domain is contained in [−F, F]^d, and the connectivity bound is polynomial in log(η^-1) and log⌈F⌉.Source: B. Invariance to Affine Transformations · p. 48

C. Canonically Ordered Affine Dictionaries are Effectively Representable

• A nondegenerate, canonically ordered affine dictionary on bounded Ω with compactly supported generators is effectively representable when each generator has networks with polynomial connectivity in log(ε^-1) and polynomial weight storage in ε^-1.The theorem assumes finitely many generators, each approximable to error ε by a depth-1 network.Source: C. Canonically Ordered Affine… · pp. 48–49
• The dictionary construction uses the affine-invariance result to obtain a connectivity bound independent of dictionary index, while polynomial bounds on affine parameters control weight storage.The proof uses canonical ordering to bound the affine parameters as the dictionary index grows.Source: C. Canonically Ordered Affine… · pp. 49–50
• The effective-representability result extends to Ω = R^d and sufficiently fast-decaying, unbounded-support generators, with an alternative definition of canonical ordering.The extension requires additional technical steps; the stated theorem itself is restricted to bounded Ω and compactly supported generators.Source: C. Canonically Ordered Affine… · p. 50
• If an affine dictionary is optimal for a compact function class, neural networks are also optimal for that class under the stated generator-approximation conditions.Theorem VIII.5 applies to compact classes on bounded Ω with nondegenerate, canonically ordered dictionaries generated by compactly supported functions.Source: C. Canonically Ordered Affine… · p. 50

D. Spline wavelets

• M(Φε) ≤ C log(ε^-1), B(Φε) ≤ 1, and ||Φε − Nm||L∞(R) ≤ ε for each fixed-order cardinal B-spline Nm.The lemma states the approximation bound for ε ∈ (0, 1/2).Source: D. Spline wavelets · pp. 51–52
• The B-spline approximation result also holds in L2(R), with a different constant C.The argument uses the common support [−1, m + 1] of Nm and its approximating networks to bound L2 error by the L∞ error.Source: D. Spline wavelets · p. 52
• Every compact class C ⊆ L2(Ω) optimally representable by a spline wavelet dictionary on a bounded Ω with nonempty interior is optimally representable by neural networks.The dictionary is ordered as specified in the theorem.Source: D. Spline wavelets · p. 53

IX. Weyl-Heisenberg Dictionaries

• Gabor dictionaries comprise time-frequency translates of a generator, ordered so translation and modulation indices grow at most polynomially with the element index.The ordering within each index group is arbitrary; the polynomial-growth condition supports the representability argument.Source: §IX · p. 54
• A generator’s approximation-theoretic properties carry over to its modulated versions, extending neural-network invariance to frequency shifts alongside affine scaling and translation.The section relates modulation invariance to the previously established invariance under affine transformations.Source: §IX · p. 54
• Under bounded-domain or compact-support conditions, Gabor dictionaries are effectively representable by neural networks if their generator’s translations admit polynomial resource bounds.The theorem assumes a bounded domain, or the full domain with compactly supported generator, and polynomial bounds for approximating translations.Source: §IX · p. 57
• Neural networks optimally approximate function classes that are optimally approximated by Gabor dictionaries, when their generator functions are themselves well approximated by neural networks.The section presents this as a central result for Gabor dictionaries, whose mathematical structure differs from affine dictionaries.Source: §IX · p. 60

IX. WEYL-HEISENBERG DICTIONARIES

• Any class optimally representable by G(g, α, β, Ω) is also optimally representable by neural networks.Source: §IX · p. 61
• Gaussian generators admit ε-approximation over all of R^d with shift-independent networks using M ≤ Cd(log(ε−1))^2((log(ε−1))^2 + log(d)) and B ≤ 1.The network and the stated growth rates do not depend on the shift parameter.Source: §IX · pp. 61–62
• The construction extends to Ω = R^d and generators of unbounded support if they decay sufficiently rapidly.Source: §IX · p. 62
• Neural networks provide Kolmogorov-Donoho optimal approximation for classes optimally approximated by Wilson dictionaries with network-approximable generators.The proofs construct network approximations of time-shifted and cosine- and sine-modulated generators; Wilson bases optimally approximate unit balls in modulation spaces.Source: §IX · p. 62
• The same techniques can establish optimal representability for Fourier bases.Source: §IX · p. 62

Oscillatory textures

• The cited wave-atom results achieve only low-order polynomial rates for oscillatory textures, while finite-width deep networks yield exponential rates.The passage notes that concerns the two-dimensional case, whereas this section treats one-dimensional textures.Source: §X · p. 63
• For oscillatory textures, finite-width deep networks improve prior low-order polynomial rates to exponential accuracy; Proposition X.2 gives approximants with W ≤ 32 and B ≤ 1.The result applies to every D, a > 0, f ∈ F_D,a, and ε ∈ (0, 1/2).Source: §X · p. 63

Weierstrass function

• The Weierstrass function is continuous everywhere, and its Hölder smoothness α may be made arbitrarily small by suitable choice of a, as stated for Wp,a.The cited relaxation allows p ∈ (0, r), with r < 1, while changing only the constant C.Source: §X · pp. 65–66
• W ≤ 13 and B ≤ 1 suffice for the Proposition X.3 deep-network approximants to the Weierstrass function, whose approximation error decays exponentially.The stated depth bound is C((log(ε^-1))^3 + (log(ε^-1))^2 log(⌈a⌉) + log(ε^-1) log(⌈D⌉)).Source: §X · p. 65

XI. IMPOSSIBILITY RESULTS FOR FINITE-DEPTH NETWORKS

• For any nonconstant periodic function, some frequency defeats finite-depth networks whose width grows polynomially in the logarithm of frequency, preventing arbitrarily small approximation error.The bound applies at each fixed depth and also rules out polylogarithmic connectivity growth in the highest frequency.Source: §XI · pp. 67–68
• For x ↦ cos(ax), finite-depth networks require faster-than-polylogarithmic connectivity growth in a for arbitrarily small error, while finite-width networks achieve polylogarithmic growth.Source: §XI · p. 67
• Any function that is at least three times continuously differentiable and has nonvanishing second derivative cannot be approximated with finite-depth connectivity scaling polylogarithmically in inverse error.By contrast, finite-width deep networks permit this scaling for various smooth functions, including polynomials and sinusoidal functions.Source: §XI · p. 68

APPENDIX A AUXILIARY NEURAL NETWORK CONSTRUCTIONS

• Scalar multiplication x ↦ ax can be realized with weights bounded by 1, depth at most floor(log(|a|)) + 4, and width at most 3d.Source: Appendix A · p. 69
• Affine maps Ax + b with entries bounded by a can be realized using weights bounded by 1, depth at most floor(log(|a|)) + 5, and width at most max{d, 3d′}.Source: Appendix A · p. 69
• Any network Φ can be represented exactly by a network with weights bounded by 1, width at most max{3d′, W(Φ)}, and depth at most (ceil(log(B(Φ))) + 5)L(Φ).Source: Appendix A · p. 69
• Sums of networks sharing an input can be realized with width at most 2d + 2d′ + max{2d, max_i W(Φ_i)}, independent of the number of networks.The resulting network’s maximum weight magnitude is max{1, max_i B(Φ_i)}.Source: Appendix A · pp. 69–70
• Local approximations with error at most ε/3 on overlapping intervals can be patched into one network of width at most 7 + max{2, max_i W(Φ_i)}.Its maximum weight magnitude is max{1, max_i B(Φ_i)}.Source: Appendix A · p. 71
• For f in S[a,b], a network approximates f within ε with width at most 16 and weights bounded by 1, with depth controlled by interval length, ε, and endpoint magnitudes.The construction handles short intervals by shifting and rescaling, then patches local approximations for longer intervals.Source: Appendix A · pp. 72–73

APPENDIX B TAIL COMPACTNESS FOR BESOV SPACES

• Appendix B uses an orthonormal wavelet basis for L2([0,1]), grouped into scale blocks D_n.Each block D_n contains the wavelets ψ_n,k for k=0,…,2^n−1.Source: Appendix B · p. 74
• The Besov space is independent of the mother wavelet choice when ψ has at least r vanishing moments and belongs to C^r([0,1]) for some r>m.The appendix refers readers to Section 9.2.3 in for details.Source: Appendix B · p. 74
• For q∈[1,2], the Besov unit ball has tail compactness with C=1 and β=m.The dictionary is ordered by wavelet scale.Source: Appendix B · pp. 74–75

APPENDIX C TAIL COMPACTNESS FOR MODULATION SPACES

• A normalized even g yields a tight Gabor frame, from which the Wilson dictionary is an orthonormal basis [17, Thm. 8.5.1].The appendix uses this Wilson dictionary for modulation-space coefficients.Source: Appendix C · p. 75
• For modulation-space functions, the Wilson coefficient sequence is equivalent to the weighted ℓ^s_p,q(Z×N0) norm up to a constant D≥1 [17, Thm. 12.3.1].The stated bounds are D^-1∥f∥_M^s_p,q(R) ≤ ∥c(f)∥_ℓ^s_p,q(Z×N0) ≤ D∥f∥_M^s_p,q(R).Source: Appendix C · p. 76
• For p,q≤2, the modulation-space unit ball has tail compactness with C=(3/2)^−sD and β=s.The result uses the ordered Wilson basis and is derived using its orthonormality.Source: Appendix C · p. 76