Source-linked AI summary
Functional Additive Mixed Models
Fabian Scheipl, Ana-Maria Staicu, Sonja Greven
TL;DR
The paper addresses flexible regression for correlated functional responses with complex dependence structures and scalar or functional covariate effects. It develops an additive mixed-model framework using spline- and FPC-based representations, and reports reliable recovery of relevant effects across challenging settings, with interpretable applications. The framework remains subject to limitations in FPC truncation and residual-covariance modeling.
Problem
Existing approaches have limited ability to jointly model longitudinal correlation and covariate effects in complex functional data.
Method
The paper develops flexible additive mixed models with functional random effects, tensor-product representations, and standard mixed-model estimation for correlated functional responses.
Results
The method reliably recovers relevant effects, handles small group sizes or few replications, scales to considerable data sets, and produces interpretable application results.
Takeaways & Limitations
The framework supports flexible modeling of complex spatial, temporal, spatio-temporal, and longitudinal functional data while retaining interpretable results.
Takeaways & Limitations
FPC-based modeling depends on estimated FPC quality and truncation choices, while residual modeling can become computationally costly and iid errors may be unrealistic.
Abstract
from arXiv · showhide
We propose an extensive framework for additive regression models for correlated functional responses, allowing for multiple partially nested or crossed functional random effects with flexible correlation structures for, e.g., spatial, temporal, or longitudinal functional data. Additionally, our framework includes linear and nonlinear effects of functional and scalar covariates that may vary smoothly over the index of the functional response. It accommodates densely or sparsely observed functional responses and predictors which may be observed with additional error and includes both spline-based and functional principal component-based terms. Estimation and inference in this framework is based on standard additive mixed models, allowing us to take advantage of established methods and robust, flexible algorithms. We provide easy-to-use open source software in the pffr() function for the R-package refund. Simulations show that the proposed method recovers relevant effects reliably, handles small sample sizes well and also scales to larger data sets. Applications with spatially and longitudinally observed functional data demonstrate the flexibility in modeling and interpretability of results of our approach.
1 Penalized regression for correlated functional data
The framework represents correlated functional-response models with tensor-product bases and penalties, supporting flexible scalar and functional effects, random effects, and dependence structures.
- Tensor-product bases represent additive effects over covariates and the functional-response index, with marginal penalties controlling smoothness in each direction.
- The framework accommodates functional intercepts, scalar-covariate effects, nonlinear scalar effects, and linear or nonlinear functional-covariate effects.
- Functional effects can use observation-specific integration limits, including concurrent models in which covariates and responses are evaluated at the same index.
- Functional random effects support dependence across grouping-factor levels through precision matrices representing structures such as spatial or temporal random fields.
- FPC-based random-effect representations can scale better than flexible spline bases, but performance depends on estimating the covariance kernel and selecting the truncation lag Kt.
- The mixed-model representation separates penalized and unpenalized components, enabling standard additive mixed-model estimation, smoothing-parameter selection, and inference procedures.
2 Empirical evaluation
The empirical evaluation shows that the framework estimates important effects accurately across challenging simulation settings and models longitudinal functional data while accounting for within-subject dependence. Results also demonstrate computational scalability and interpretable disease-related spatial associations in the DTI application.
- Simulation design: The simulations vary model complexity, noise, sample size, replication count, grid density, and random-effect importance across four scenarios.The study fits 1920 models from repeated replications across combinations of these settings.
- Simulation results: Increasing observation quality substantially improves covariate-effect accuracy, whereas more groups mainly reduce covariate-effect errors and more within-group observations also improve response-trajectory estimates.Increasing SNRε from 1 to 5 reduces relative errors about 16-fold, increasing M from 10 to 100 reduces covariate-effect errors about 8-fold, and increasing ni from 3 to 20 reduces response-trajectory errors four- to sixfold.
- Inference: Approximate pointwise confidence-interval coverage was generally near the nominal 95% level, with undercoverage concentrated in small, noisy settings and some overcoverage for selected effects.Coverage for functional random slopes approached 0.95 when ni increased from 3 to 20, while some effects showed overcoverage or low-coverage outliers.
- Computation: Computation times remain manageable for large data sets, although models with multiple random effects become much slower as the number of subjects increases.The largest data sets contained nT = 1.2 · 10^5 observations, and bam() can support large data sets that do not fit into memory.
- Simulation results: Only one replicate produced an rIMSE for y(t) above 0.1, while important predictor effects were estimated with good to excellent accuracy even in difficult settings.Covariate-effect accuracy is most sensitive to noise and random-effect importance, whereas random-effect accuracy depends strongly on random-effect importance and group size.
- DTI application: In the DTI application, accounting for within-subject correlation reveals substantial subject-specific variability and weaker, disease-dependent spatial associations than a naive independence model.Healthy-control associations between OPR and CCA profiles largely vanish or weaken for MS patients, while longitudinal modeling avoids underestimated uncertainty and implausible effects.
3 Discussion and Outlook
The framework combines flexible additive modeling of correlated functional responses with standard additive mixed-model estimation and open-source implementation. Simulations and applications support reliable recovery, scalability, and interpretable analyses, while residual covariance modeling remains an important limitation and research direction.
- The framework accommodates multiple functional random effects, diverse correlation structures, and linear or smooth effects for scalar and functional covariates.Effects and random effects are available through both spline- and FPC-based variants.
- Standard additive mixed-model software provides the basis for estimation and inference, implemented in the documented open-source pffr() function.
- Simulation experiments recover relevant effects reliably, handle small group sizes or few replications, and fit considerable data sets in acceptable time.
- Two applications show that flexible models can represent complex data situations while producing interpretable results about underlying processes.
- Residual modeling is limited because current inference algorithms do not exploit sparse design matrices, making observation-specific residual terms costly for large data sets.Assuming i.i.d. errors may also be unrealistic when functional responses exhibit autocorrelation or heteroscedasticity.
- Further work targets efficient covariance modeling, diagnostics for low-rank problems, comparisons of spline- and FPC-based approaches, and REML inference for function-on-function regression.
A.1 Imposing suitable identifiability constraints
The model requires pointwise identifiability constraints because different decompositions of the mean and random effects can produce the same fitted values. Sum-to-zero constraints and centered functional covariates make response-index-varying effects interpretable as deviations from the overall mean trajectory.
- Additive scalar-response models impose constraints such as sum-to-zero conditions to ensure identifiable decompositions of the predictor.Without such constraints, constants can be shifted between component functions without changing the fit.
- The functional model has the same identifiability problem because arbitrary mean functions can be added to one effect and subtracted from another.
- Pointwise sum-to-zero constraints are imposed for each response index t to prevent equivalent parameterizations.
- Centering covariate trajectories and imposing pointwise constraints makes varying effects directly interpretable as deviations from the overall mean trajectory.Standard mgcv constraints over both subjects and response indices do not provide identifiable or interpretable effects here.
A.2 Limits on the identifiability of complex regression surfaces for low-rank functional covariates
Identifiability of function-on-function regression surfaces depends on conditions that are difficult to verify empirically. Low effective rank and overlap between the covariate covariance kernel and the spline-penalty nullspace can make spline estimates unstable.
- Identifiability of the coefficient surface β(s, t) is theoretically guaranteed under conditions that are difficult to verify empirically.
- The effective rank of a functional covariate is the number of eigenvalues accounting for at least 99.5% of its variability.
- Low effective rank creates a large covariance-operator kernel, which can overlap the tensor-product spline penalty's nullspace and destabilize spline-based estimates.
- Practitioners should check the observed covariance matrix's effective rank and kernel overlap when fitting spline-based function-on-function terms.
B Supplementary Details for the DTI Data Analysis
Figure 7 presents estimated mean profiles, age effects, and coefficient surfaces from a naive independent-response model, while Figure 8 examines residual covariance and correlation under alternative models.
- Figure 7: Figure 7 shows estimated model components with ±2 pointwise standard errors and sign/significance encoded by color at the 95% level.Blue denotes significant negative effects, light blue nonsignificant negative effects, light red nonsignificant positive effects, and red significant positive effects.
- Figure 8: Figure 8 compares observed residuals, empirical covariances, and empirical correlations for model (8) and model (9) with FPC- or spline-based random intercepts.The top row displays residual and covariance diagnostics; the bottom row displays empirical correlations.
C Supplementary Simulation Study Results
The simulations compare the proposed approach with FPC-based terms and WFMM, highlighting differences in relative estimation error, computation time, and model availability across scenarios.
- Comparison with FPC-based approaches: 1.5(1.2−2.1) was the mean factor by which rIMSEs for the FPC-based function-on-function term exceeded spline-based estimates.The comparison used datasets from the second simulation scenario.
- Comparison with FPC-based approaches: 1(0.8−1.4) was the computation-time factor for M = 10, while 1.3(1.1−1.6) was the factor for M = 100, comparing FPC- and spline-based terms.The FPC-based approach took somewhat longer at M = 100 but had about the same computation time at M = 10.
- Comparison with WFMM: WFMM comparisons were possible only for scenario 1 because WFMM cannot fit the additional terms present in the other scenarios.WFMM fits random-effect curves and functional linear effects of scalar covariates, but not the broader term combinations used elsewhere.