PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal estimation for Functional Linear Regression with Noisy Discretized Data

Sixtine Sphabmixay

arXiv:2609.08671v1stat.MLcs.LG

TL;DR

Functional linear regression guarantees have mainly been developed for continuously observed, noiseless curves, leaving regular-grid observations with noise less studied. This paper reconstructs curves from noisy grid data, estimates the slope with penalized least squares, and derives prediction guarantees, including a minimax rate when the grid is sufficiently large.

  • Problem

    Statistical properties have been studied mainly for continuously observed, noiseless curves, while finite-grid observations are more realistic and comparatively less studied.

    Source: §1 · p. 2

  • Method

    The method reconstructs latent curves from noisy regular-grid observations using Fourier projections, then estimates the slope by penalized least squares with data-driven dimension selection.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    For sufficiently large grids, the prediction error reaches the continuous-setting rate, with a matching lower bound showing no minimax loss from grid observation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Prediction performance is assessed against both reconstructed and true curves, with the true-curve risk reflecting the distribution of the covariate.

    Source: §1 · p. 2, §2.3 · p. 6

    The sufficient conditions can be restrictive: independence of Fourier coefficients is a strong requirement, though it can be dispensed with under H3 for some a > 1.

    Source: §3 · p. 7

Abstract

from arXiv · show

In this paper, we consider the scalar-on-function linear regression model under a realistic sampling scheme in which the functional covariates are observed on a regular grid and contaminated by additive noise. We propose a two-step estimation procedure: first, the underlying curves are reconstructed from the discrete noisy observations using a Fourier-based projection method; second, the slope function is estimated by a penalized least-squares criterion over finite-dimensional trigonometric spaces, with data-driven selection of the model dimension. We establish oracle-type inequalities for the prediction error, both with respect to the reconstructed curves and to the true latent curves. Under regularity assumptions on the slope function and polynomial decay of the eigenvalues of the covariate, we derive convergence rates for the prediction error and show that our estimator attains the minimax rate when the number of grid points is sufficiently large. Finally, the proposed method is illustrated on simulated data and on a real meteorological dataset.

1 Introduction

The paper extends functional linear regression to noisy observations on a regular grid, a setting less studied than continuous, noiseless observation. It combines curve reconstruction with penalized slope estimation and derives prediction guarantees under stated regularity and covariance conditions.

1 Introduction

  • Existing statistical guarantees largely concern continuously observed, noiseless curves, while the more realistic finite-grid setting has received comparatively less attention.Cardot et al. studied finite-grid observations, but their results require a sufficiently large grid.Source: §1 · p. 2
  • The method reconstructs curves from noisy discrete observations with a truncated Fourier projection, then estimates the slope by penalized least squares with data-driven basis-dimension selection.The paper derives oracle inequalities for prediction error relative to both reconstructed and true curves.Source: §1 · p. 2
  • When the grid is sufficiently fine, the prediction error attains the continuous-setting rate and a matching lower bound indicates no minimax loss from grid observation.For moderately large grids, the rate also depends on the number of grid points; for small grids, the error decays with grid size.Source: §1 · p. 2
  • The paper illustrates its method on simulated data and real-world meteorological observations.Functional data analysis treats observations naturally expressed as functions rather than finite-dimensional vectors and has applications across several disciplines.Source: §1 · pp. 1–2

2 Estimation procedure

The procedure reconstructs latent curves from noisy grid observations, estimates the slope using reconstructed curves, and evaluates performance through prediction error tied to the true covariate distribution.

2.1 Curve reconstruction

  • A truncated Fourier expansion reconstructs each latent curve from noisy grid observations using discrete inner-product approximations that act as Riemann sums.The chosen grid and Fourier basis yield discrete orthogonality when DNn,p < p.Source: §2.1 · p. 3

2.2 Estimating the slope function

  • Slope estimation minimizes least squares on reconstructed curves and penalizes model complexity, while dimensions exceeding n or near-zero-variance directions can make the empirical matrix singular or ill-conditioned.The penalty is designed to account for model complexity and avoid overfitting.Source: §2.2 · pp. 4–5

2.3 Risk measure

  • Prediction risk measures the discrepancy between fitted and true predictions on the same new curve; unlike L2 error, it weights directions according to the covariate distribution.Directions with small covariance eigenvalues have little influence on prediction but may contribute substantially to L2 error.Source: §2.3 · p. 6

3 Main results

The results give oracle-type prediction bounds for reconstructed curves and then for the true latent curves, under moment, tail, and eigenvalue conditions. They also isolate reconstruction error and identify when Fourier-coefficient independence can be relaxed.

3 Main results

  • The analysis assumes sub-Gaussian observational noise and polynomial eigenvalue decay; Fourier-coefficient independence is restrictive but dispensable when H3 holds for a>1.Larger eigenvalue-decay parameter a corresponds to smoother sample paths; H2 is considered reasonable for light-tailed noise.Source: §3 · p. 7
  • The projections of β onto each model space coincide under the reconstructed and true covariance inner products.This follows because both covariance operators are diagonal with strictly positive eigenvalues on the stated Fourier span.Source: §3 · p. 7
  • Theorems 3.1 and 3.2 give oracle inequalities for prediction risk based on reconstructed curves and on the true underlying curves, respectively.The bounds rely on the assumptions of Proposition 3.1.Source: §3 · p. 8
  • The bounds progress from an empirical reconstructed-data covariance operator to the reconstructed-process covariance and finally to the true-process covariance.The last step controls the discrepancy between reconstructed and true covariance structures.Source: §3 · p. 8
  • The estimator performs nearly as well as the best candidate in the model collection, up to a multiplicative constant and additional remainder terms.The leading oracle criterion differs between the reconstructed-curve and true-curve bounds.Source: §3 · p. 8
  • Reconstruction-error terms quantify the preliminary curve-recovery step, while other remainders are of orders 1/n, 1/p, and 1/(np).These latter terms arise in the proof of the oracle inequalities.Source: §3 · p. 9

4 Convergence rates over Sobolev spaces

The convergence-rate analysis identifies how the number of grid points affects prediction error under Sobolev regularity and polynomial eigenvalue decay. With sufficiently many grid points, the estimator reaches the fully observed minimax rate.

4 Convergence rates over Sobolev spaces

  • The rate has three regimes: with small p, discretization error dominates, so increasing n alone does not improve the convergence rate.For large p, the problem behaves essentially as if the curves were fully observed.Source: §4 · p. 9
  • Under Theorem 4.1’s stated assumptions, its convergence rate coincides with the optimal rate in the literature for functional linear models with fully observed curves, recovering Brunel and Roche.Theorem 4.1 retains Theorem 3.2’s assumptions except that it uses a ≥ 1 rather than a > 1/2 in (H3).Source: §4 · p. 9
  • The rate's dependence on p is driven primarily by observation noise rather than by curve reconstruction itself.Source: §4 · p. 9

5 Lower Bound for the case p large

The lower-bound analysis establishes a minimax benchmark for the large-grid regime under stronger eigenvalue conditions. Together with the upper bound, it shows that the estimator attains the minimax rate over the Sobolev class.

5 Lower Bound for the case p large

  • The lower-bound setting assumes Gaussian observation noise with variance τ^2.Source: §5 · p. 9
  • The lower-bound construction requires two-sided polynomial eigenvalue bounds, unlike the upper-bound proof, which needs only an upper bound.The two-sided condition ensures candidate slopes remain in the Sobolev class and separated in prediction norm.Source: §5 · p. 10
  • For sufficiently large p, the lower bound matches the upper bound, establishing that the estimator attains the minimax rate over W per(k, L).Source: §5 · p. 10

6 Simulations

The simulations assess prediction error across slope functions, grid densities, and sample sizes. Empirical decay rates are close to theoretical rates in several settings, and the estimator’s performance changes little when the regression-noise variance is estimated.

6.1 Simulation setup and data generation

  • Simulations generate Gaussian truncated Karhunen–Loève curves on fine and observation grids, add measurement noise, and assess prediction error by Monte Carlo averaging.The prediction-error assessment uses 50 independent replications.Source: §6.1 · pp. 10–11, §6.3 · p. 12, §6.4 · p. 12

6.2 Estimation of the slope function

  • The procedure reconstructs curves from noisy discrete observations, then estimates β by penalized model selection over Fourier terms, choosing the largest admissible reconstruction dimension.The reconstruction dimension must ensure empirical covariance invertibility with high probability; the slope estimator’s oracle inequality assumes θ>4 and δ>0.Source: §6.2 · p. 11

6.4 Calibration of θ and δ

  • Aggregate prediction error across β1, β2, and β3 is minimized at κ=3, but the simulations use κ=4.01 to satisfy the required κ>4 constraint.Calibration averages prediction errors over 50 replications for each slope function.Source: Figure 3 · p. 13, §6.4 · p. 13

6.5.1 Study of β1

  • With p=7500 and noise variances τ²=σ²=0.1, the log-log error slope versus n is −0.92, close to expected n^-6/7; with n=1000 and τ²=4, the slope versus p is −0.79, matching p^-3/4.The p-grid study also sets σ²=0.1.Source: §6.5.1 · p. 13
  • With p fixed, varying n gives a log-log slope of −0.76, close to expected n−0.8; with n fixed, varying p gives approximately −0.55, consistent with expected p−1/2 behavior.Source: §6.5.1 · p. 15

6.5.2 Study of β2

  • For β2, which does not exactly satisfy periodic boundary conditions, log-log slopes −0.86 and −0.77 remain close to the earlier β1 results and theoretical rates.The simulations use a=2 and the same parameters as the first β1 study.Source: §6.5.2 · p. 15

6.5.3 Study of β3

  • For β3, estimates at n=800 smooth over its discontinuities, while at n=6400 and DNn,p=81 they have lower variance and fit continuous segments better.At the larger sample size, estimates also attempt to fit the discontinuity at t2.Source: Figure 7 · p. 17, §6.5.3 · p. 17

6.6 Effect of the unknown variance σ2

  • For β1 and β2, estimating σ^2 leaves prediction errors in the same order of magnitude as when σ^2 is known, with similar decreases as n or p increases.The comparisons use two regimes: varying n with p=7500 fixed and varying p with n=1000 fixed.Source: Table 1 · p. 18, Table 2 · p. 18, §6.6 · p. 18

7 Application to meteorological data

The meteorological application models annual precipitation from daily temperature profiles across 305 locations. On the held-out test set, its RMSE is 0.58, compared with 0.91 for a training-mean baseline.

7 Application to meteorological data

  • The dataset contains 305 location-level temperature curves with 365 daily measurements, paired with annual precipitation responses and averaged over 2019–2024.The analysis excludes observations with missing values and February 29; it uses an 80/20 training/test split and centers both sets using training-set means.Source: §7 · pp. 18–19
  • The procedure reconstructs the curves, estimates variance and selects the model dimension using training data, then evaluates RMSE on reconstructed test curves.The precipitation response is centered, and the split is 80% training and 20% test.Source: §7 · p. 19
  • The estimated coefficient function is used to describe how the annual temperature profile relates to average annual precipitation.Figure 9 presents the estimated functional coefficient.Source: Figure 9 · p. 21
  • Observed and predicted centered values track the overall precipitation trend, while residuals center near zero but show greater negative-side dispersion and stronger overprediction at some sites.The comparison is shown in Figure 10a; Figure 10b displays the test-set residual distribution.Source: §7 · pp. 21–22, Figure 10 · p. 21
  • RMSE 0.58 for the model versus RMSE 0.91 for the baseline that predicts test rainfall using the training-set average.The paper notes that the baseline is already effective because annual average rainfall is stable and predictable.Source: §7 · p. 22

A Gradient Calculation for the Least Squares Minimization

The appendix establishes differentiability and a gradient expression for the least-squares objective, then uses the gradient-zero condition to characterize its minimizer.

A Gradient Calculation for the Least Squares Minimization

  • The lemma establishes that the objective function F is differentiable and gives its gradient.The gradient is expressed using Φ_m and b, as referenced in Equations (11) and (12).Source: A Gradient Calculation for the Least… · p. 22
  • Because F is convex and continuously differentiable, minimizing it is equivalent to finding a coefficient vector whose gradient is zero.The proof applies this criterion to the finite-dimensional vector eα.Source: A Gradient Calculation for the Least… · p. 22
  • The gradient calculation substitutes the defined reconstructed curves and uses orthonormality of the basis functions to simplify the expression.These steps connect the objective's gradient to the quantities used in the estimator.Source: A Gradient Calculation for the Least… · p. 22

B.1 Proof of Proposition 3.1

These proofs establish intermediate eigenvalue and concentration bounds and use them to control estimator errors through projection, stochastic, and penalty terms. They also examine how discretization noise affects the Fourier coefficients in the analysis.

B.1 Proof of Proposition 3.1

  • The proposition proof separates the estimator error into a projection component and stochastic terms, then bounds the latter using linearity and Cauchy–Schwarz inequalities.It invokes Lemma 1 of Brunel et al. and controls a remaining term using Lemma B.3; the dimension condition DNn,p < n/ln^2 n makes that quantity negligible for sufficiently large n.Source: §B.1 · pp. 23–25
  • Under the stated i.i.d. centered noise assumptions and DNn,p < p, discretization-noise coefficients with disjoint index sets have zero cross-moment.This includes different-parity indices and pairs where one index is 1 and the other is larger; for equal-parity indices, the proof shows disjointness when their frequencies differ.Source: §B.1 · pp. 27–30
  • For equal odd frequencies, the only index coincidences are the identical ones, which the proof uses to derive the corresponding expression for the noise-coefficient product.The even-frequency case likewise permits only matching indices when the frequencies are equal.Source: §B.1 · pp. 29–30
  • Under λ_DNn,p ≥ 2 n^α and α > 2a, monotonicity of the eigenvalues yields the bound required by Lemma B.3.The proof compares the minimum estimated eigenvalue with λ_DNn,p and combines that comparison with the definition of s_n.Source: §B.1 · p. 31
  • A positive smallest eigenvalue for Ψ_m implies that Ψ_m and Φ_m are invertible, supporting the eigenvalue bounds used in the proof.The argument uses positive definiteness and invertibility of the diagonal matrix Λ_m to establish invertibility of Φ_m.Source: §B.1 · p. 32
  • Under (H1) and (H2), independent centered curve and noise observations yield sub-Gaussian vectors, enabling concentration control of Ψ_m around the identity.The proof checks independence, centering, isotropy, and sub-Gaussianity before applying Theorem 4.6.1 of Vershynin.Source: §B.1 · pp. 33–34

B.2 Proof of Theorem 3.1

  • The proof controls the second term of the sum in Equation (58) with Lemma B.10 and focuses on the first through β’s Γ-seminorm projection onto S_m; under (H3), eigenvalues decrease polynomially.Source: §B.2 · pp. 39, 42
  • The B.2 proof invokes Theorem 4.6.1 of Vershynin after computations, using orthonormality and Lemma B.2 in its preceding derivation.Lemma B.11 is stated under (H1), (H2), and DNn,p < p; its displayed conclusion is not included in the supplied passage.Source: §B.2 · pp. 42–43

B.3 Proof of Theorem 3.2

  • Lemma B.12 is stated assuming (H3), DNn,p < p, and β ∈ W per(k, L), with its conclusion omitted from the supplied passage.Source: §B.3 · p. 44
  • For j = 1, ..., DNn, the proof uses λj = O(p^-2a) and Lemma A.3 of Tsybakov to derive a bound.Source: §B.3 · p. 45

C Proof of Theorem 4.1

The appendix derives the rate bound in Theorem 4.1 under polynomial eigenvalue decay and periodic Sobolev regularity, then builds a Tsybakov-style lower-bound argument for Theorem 5.1.

C Proof of Theorem 4.1

  • Under polynomial eigenvalue decay and β in W_per(k, L), the dominant rate involves both sample-size and grid-size terms.The proof uses the polynomial-decay and Sobolev assumptions when invoking Brunel and Roche’s result.Source: C Proof of Theorem 4.1 · pp. 45–46

D Proof of Theorem 5.1

  • The lower-bound proof constructs test functions indexed by binary vectors and checks the conditions needed to apply Tsybakov’s theorem.A large, well-separated family of alternatives is the key ingredient in the stated packing argument.Source: D Proof of Theorem 5.1 · pp. 52–53
  • The proof organizes the lower-bound argument around a function class containing the true slope and a sample model with Gaussian observation noise.The passage states that the test slopes belong to W_per(k, L), with Gaussian centered response and observation noises.Source: D Proof of Theorem 5.1 · p. 52

D.1 Condition 1. of the proof

The proof verifies that its constructed test slopes satisfy the first condition by establishing their smoothness and periodicity and using Fourier-basis properties.

D.1 Condition 1. of the proof

  • The first condition is established by showing the constructed test slopes belong to W_per(k, L).The proof treats the zero slope directly and then checks the derivatives of the alternatives.Source: §D.1 · pp. 53–54
  • The constructed alternatives are 1-periodic, as are the real Fourier basis functions used to form them.The proof also addresses the zero function separately.Source: §D.1 · p. 53
  • The alternatives are built from real Fourier basis functions, whose orthonormality lets the proof express squared L2 norms as sums of squared coefficients.The proof combines the even- and odd-index contributions when forming those sums.Source: §D.1 · pp. 55–56

D.2 Condition 2. of the proof

This proof section establishes the required divergence control by comparing observed-data laws under alternative slope models and relating them to the full-data laws.

D.2 Condition 2. of the proof

  • Because the conditional distributions are Gaussian with the same variance, the proof expresses their divergence through the difference in conditional means.The passage then develops the square and takes expectation under the baseline model.Source: §D.2 · p. 57
  • The proof considers joint laws of the observed sample under the zero-slope model and each alternative, then uses the independent-observation structure for the full sample.The observed laws are marginals of the corresponding full laws for (X, Z, Y).Source: §D.2 · pp. 57–58
  • The proof uses the shared marginal law of Z across models to compare conditional divergences for the observed data with those for the full model.The conditional law of X given Z is also common across the compared models.Source: §D.2 · pp. 58, 61

D.3 Condition 3. of the proof

The proof checks the final condition by characterizing projections of the test slope onto the finite-dimensional space under both stated semi-inner products.

D.3 Condition 3. of the proof

  • The proposition applies the stated semi-norm to the constructed test functions, and the proof invokes the Varshamov–Gilbert bound in deriving the desired result.The constant C4 is stated to depend on k, L, c′, and σ.Source: §D.3 · p. 61
  • For the projection under Γ onto S_m, positive eigenvalues force its Fourier coefficients to match those of β through the retained dimensions.The proof uses the Fourier basis of S_m and the definition of the Γ inner product.Source: §D.3 · p. 62
  • The proof gives the corresponding projection characterization under the empirical semi-inner product eΓ.The displayed coefficient relation again identifies the projection coefficients with those of β.Source: §D.3 · p. 62
Loading 2609.08671v1…