Source-linked AI summary
Analysis of family-wise error rates in statistical parametric mapping using random field theory
Guillaume Flandin, Karl J. Friston
TL;DR
The report revisits whether family-wise error analyses undermine parametric random field theory in functional neuroimaging. It reviews the Eklund et al. simulations and concludes that random field theory behaves as assumed when its threshold and smoothing conditions are respected, although the null construction has a limitation.
Problem
The report addresses whether parametric random field theory provides reliable family-wise error control despite concerns about violated distributional assumptions.
Method
It reviews simulations comparing parametric and nonparametric tests across inference types, thresholds, smoothing levels, regressors, and one- versus two-sample t-tests.
Results
Parametric peak-height inference controlled family-wise error acceptably, while spatial-extent inference failed at low cluster-forming thresholds but improved with increased smoothing and two-sample comparisons.
Takeaways & Limitations
The reviewed analyses support using random field theory for spatial-extent inference when its distributional assumptions are respected.
Takeaways & Limitations
Using the same regressor for every subject complicated null-distribution construction and made one- versus two-sample effects difficult to interpret.
Abstract
from arXiv · showhide
This technical report revisits the analysis of family-wise error rates in statistical parametric mapping - using random field theory - reported in (Eklund et al., 2015). Contrary to the understandable spin that these sorts of analyses attract, a review of their results suggests that they endorse the use of parametric assumptions - and random field theory - in the analysis of functional neuroimaging data. We briefly rehearse the advantages parametric analyses offer over nonparametric alternatives and then unpack the implications of (Eklund et al., 2015) for parametric procedures.
1 Introduction
The introduction argues that random field theory offers an efficient and reproducible parametric alternative to nonparametric testing, while emphasizing that its distributional assumptions require validation.
- Assumptions and motivation: Random field theory is important because it provides an efficient and reproducible alternative to nonparametric testing.
- Advantages of parametric approaches: Parametric approaches are more efficient than nonparametric tests under the Neyman-Pearson lemma.Their efficiency follows from using the odds ratio inherent in parametric tests.
- Advantages of parametric approaches: Parametric analyses are reproducible because repeating the analysis yields the same result.Nonparametric p-values depend on samples from the null distribution.
- Advantages of parametric approaches: Parametric approaches avoid exchangeability requirements that complicate nonparametric testing with serial correlations or hierarchical models.
- Advantages of parametric approaches: Parametric approaches are computationally more efficient because they avoid intensive sampling from a null distribution.
- Assumptions and motivation: These advantages depend on distributional assumptions whose violation makes parametric tests inexact.The report therefore examines whether parametric tests and random field theory remain robust to such violations.
2 A review of the Eklund et al simulation results
The review examines family-wise error under varied inference procedures, thresholds, smoothing levels, regressors, and t-tests. It finds that peak-height inference is well behaved, whereas spatial-extent inference depends critically on threshold and smoothing assumptions.
- Simulation design: The simulations varied peak-height versus spatial-extent inference, cluster-forming thresholds, smoothing, regressors, and one- versus two-sample t-tests.They used resting-state fMRI data from two sites with parametric and nonparametric tests.
- Inference results: Parametric peak-height inference provides acceptable family-wise error control.
- Inference results: Spatial-extent inference becomes invalid specifically at low cluster-forming thresholds, consistent with random field theory’s high-threshold approximations.Both the cluster-extent distribution and expected number of maxima are approximations that become inexact at low thresholds.
- Smoothing: Increasing spatial smoothing leads to more exact inference under random field theory’s lattice approximation.The assumption requires the data to be smoother than the voxel size, and the numerical results verify this pattern.
- One- versus two-sample tests: Using the same regressor for all subjects makes one- versus two-sample comparisons difficult to interpret.Systematic resting-state fluctuations correlated with the regressor can produce significant one-sample tests against zero.
- One- versus two-sample tests: Nonparametric false-positive rates exceeded the 95% confidence intervals for the fast, inefficiently estimated E1 regressor.This indicates that the effect was expressed across subjects and did not model null behaviour.
- One- versus two-sample tests: Two-sample comparisons of parameter estimates produced acceptable family-wise error rates for spatial-extent inference.The reproduction used the same data and regressors with close-to-original 3 mm voxels rather than upsampled 2 mm data.
3 Conclusion
The report concludes that the reviewed results endorse random field theory and align with its distributional assumptions. Valid spatial-extent inference requires avoiding low cluster-forming thresholds and insufficient smoothing, while the null construction problem is mitigated by two-sample tests.
- Conclusion: The reviewed results endorse random field theory and show behaviour consistent with its underlying distributional assumptions.
- Conclusion: Random field theory provides valid spatial-extent inference when its distributional assumptions are not violated.The relevant violations involve low cluster-forming thresholds or insufficient smoothing.
- Conclusion: Using the same regressors for every subject created a problem in constructing the null distributions, but two-sample t-tests finessed it.