PaperlayerView exact revision on arXiv

Source-linked AI summary

Network inference - with confidence - from multivariate time series

Mark A. Kramer, Uri T. Eden, Sydney S. Cash, Eric D. Kolaczyk

arXiv:0903.2210v1q-bio.QMq-bio.NC

TL;DR

Threshold-based inference can leave the uncertainty in declared network edges unquantified, despite its sensitivity to threshold choice. The paper uses pairwise coupling tests and multiple-testing control to infer networks with confidence; in simulated neural data, its extremum and bootstrap methods outperform the naive method and reproduce many, but not all, structural edges.

  • Problem

    Choosing a coupling threshold determines which edges are declared, but the expected rate of falsely declared edges is often not quantified.

    Source: §I · p. 1

  • Method

    The procedure computes pairwise coupling, assigns significance-test p-values, and applies multiple-testing control to select edges and quantify false discoveries.

    Source: §I · p. 2, C. Step 3: Control of the false… · p. 5

  • Results

    In simulated neural data, the extremum and bootstrap methods outperform the naive method and qualitatively reproduce many, but not all, structural edges.

    Source: B. Simulated neural data · p. 7

  • Takeaways & Limitations

    Choosing q through formal statistical tests yields functional networks with specified uncertainty levels that may be calibrated across multivariate datasets.

    Source: §V · p. 9

    The FDR guarantee assumes accurately computed p-values and independent tests, with p-value accuracy identified as the more critical assumption.

    Source: C. Step 3: Control of the false… · p. 5

Abstract

from arXiv · show

Networks - collections of interacting elements or nodes - abound in the natural and manmade worlds. For many networks, complex spatiotemporal dynamics stem from patterns of physical interactions unknown to us. To infer these interactions, it is common to include edges between those nodes whose time series exhibit sufficient functional connectivity, typically defined as a measure of coupling exceeding a pre-determined threshold. However, when uncertainty exists in the original network measurements, uncertainty in the inferred network is likely, and hence a statistical propagation-of-error is needed. In this manuscript, we describe a principled and systematic procedure for the inference of functional connectivity networks from multivariate time series data. Our procedure yields as output both the inferred network and a quantification of uncertainty of the most fundamental interest: uncertainty in the number of edges. To illustrate this approach, we apply our procedure to simulated data and electrocorticogram data recorded from a human subject during an epileptic seizure. We demonstrate that the procedure is accurate and robust in both the determination of edges and the reporting of uncertainty associated with that determination.

I. INTRODUCTION

Threshold-based network inference can make edge counts sensitive to the chosen cutoff, while uncertainty in those declarations is often overlooked. The paper proposes a testing-based procedure to infer functional networks from multivariate time series and quantify uncertainty in falsely declared edges.

I. INTRODUCTION

  • When physical connections are unknown, node-pair coupling in multivariate time series can be used to infer associations without directly observing structural connections.This approach has been used in climate and human brain studies.Source: §I · p. 1
  • The number of inferred edges depends critically on the coupling threshold, yet the resulting network’s error rate is often overlooked.The proposed target is a principled threshold tied to a prespecified level of uncertainty in edge declarations.Source: §I · p. 1
  • The paper focuses on uncertainty in edge presence or absence, using the expected rate of falsely declared edges as a measure of network confidence.Other aspects of network structure, such as degree distribution or clustering, are outside this stated focus.Source: §I · p. 1
  • The proposed protocol computes pairwise coupling, tests each measure statistically, and uses multiple-testing procedures to control falsely declared edges.The resulting network is composed of edges selected to achieve a specified level of overall accuracy.Source: §I · p. 2, §II · p. 2

II. GENERAL PARADIGM

  • The three inference steps must be coordinated because the coupling measure shapes the hypothesis test, and inaccurate probability statements undermine uncertainty estimates.The authors describe accurate probabilistic statements for the chosen measure as necessary to propagate uncertainty from time series to the inferred network.Source: §II · pp. 2–3
  • Testing every node pair creates a multiple-testing problem: under independent tests, the expected number of false edges grows with the number of tests, while tightening individual thresholds can reduce power.The classical per-test significance level does not directly control the network-wide false-edge count.Source: §II · p. 3

III. IMPLEMENTATION OF THE GENERAL PARADIGM

The implementation specifies cross-correlation, significance testing, and multiple-testing control as the three parts of the inference procedure.

III. IMPLEMENTATION OF THE GENERAL PARADIGM

  • The implementation defines cross-correlation as its coupling measure, develops significance tests, and integrates them with a procedure for multiple tests.The specific protocols are applied to simulated and observed data.Source: §III · p. 3
  • Across the applications, the authors report that the choice of significance test is critical.Source: §III · p. 3

A. Step 1: choice of coupling measure

The procedure uses maximal absolute cross-correlation as its coupling statistic and assigns p-values through significance tests before network construction.

A. Step 1: choice of coupling measure

  • For each pair of time series, the cross-correlation is computed over lags from −100 to 100 after standardizing the series to zero mean and unit variance.The lag range corresponds to different intervals in milliseconds depending on sampling rate.Source: A. Step 1: choice of coupling measure · p. 3
  • The maximal absolute cross-correlation across lags is the statistic used to test whether to assign an edge between a node pair.The general testing paradigm can in principle use other coupling measures, though more sophisticated measures may make tractable significance tests difficult to derive.Source: A. Step 1: choice of coupling measure · p. 3

B. Step 2: Significance test

  • The maximal cross-correlation tests the null hypothesis of no coupling against the alternative of coupling, and accurate p-values are important for subsequent false-discovery-rate control.Source: B. Step 2: Significance test · p. 3
  • Two p-value approaches are considered: an analytic method designed for the chosen coupling measure and a more general but computationally expensive method.Source: B. Step 2: Significance test · p. 3

1. Analytic method

The analytic method tests the maximum absolute cross-correlation using a statistic designed for that extremum. It accounts for autocorrelation in the time series and for the non-normal distribution of the selected maximum.

1. Analytic method

  • The variance estimate incorporates the time series’ autocorrelation structure, which affects cross-correlation variability even when series are uncoupled.The method uses autocorrelations at different lags to account for expected spurious cross-correlations.Source: §1 · p. 4
  • A normal approximation for the selected maximum can produce inaccurate p-values because finite-sample cross-correlations may be non-normal and extrema are not normally distributed.Using a normal distribution for the maximum could make edge assignment too liberal.Source: §1 · p. 4
  • The extremum method Fisher-transforms cross-correlations and uses the maximum absolute transformed value to test coupling.The transformation is monotone and symmetric about zero, so the lag maximizing the absolute cross-correlation also maximizes its transformed value.Source: §1 · p. 4
  • The method scales transformed cross-correlations by empirically estimated standard deviations before applying an extreme-value approximation to the maximum.The standard deviation for independent, identically distributed sequences is not assumed because the time series are dependent; the approximation is asymptotic for large n.Source: §1 · p. 4

2. Frequency domain bootstrap method

The frequency domain bootstrap estimates the null distribution of maximum cross-correlations using surrogate time series. It requires fewer distributional assumptions than the analytic method, but is computationally expensive and pools results across node pairs under a shared-null assumption.

2. Frequency domain bootstrap method

  • The frequency domain bootstrap is presented as a significance-testing alternative that requires fewer assumptions than the analytic method but is computationally expensive.The analytic method relies on approximate distributional assumptions about the maximum cross-correlation.Source: §2 · p. 4
  • The procedure resamples whitened residuals in the frequency domain to generate surrogate data that preserve the original data’s spectral characteristics.It estimates and smooths an average power spectrum, then uses it to construct the standardized residuals.Source: §2 · pp. 4–5
  • The method compares observed maximum absolute cross-correlations with a bootstrap distribution built from surrogate data to assign p-values.To reduce computation, it pools bootstrap distributions from a subset of node pairs, assuming the null distribution is the same for all pairs.Source: §2 · p. 5

C. Step 3: Control of the false detection rate

This section addresses false edge declarations across the many pairwise tests by controlling the false discovery rate. The procedure depends on accurate p-values, while the tests’ dependence is a less critical assumption in this setting.

C. Step 3: Control of the false detection rate

  • For 4,950 node pairs, a p < 0.05 cutoff would yield roughly 250 expected false edges, while Bonferroni uses p < 10^-5.The Bonferroni threshold is given for a 100-node network and is described as potentially too strict for sparse networks.Source: C. Step 3: Control of the false… · p. 5
  • FDR control requires accurate p-values and independent tests, but test independence is less critical here because tests reuse the same time series.The authors emphasize that inaccurate p-values can undermine FDR principles; dependency extensions are noted but not pursued.Source: C. Step 3: Control of the false… · p. 5
  • The Benjamini–Hochberg linear step-up procedure orders p-values and rejects through the largest rank k with p_k ≤ q·k/m.Here m = N(N − 1)/2; q bounds the expected false-positive proportion among declared edges.Source: C. Step 3: Control of the false… · p. 5

IV. RESULTS

The procedure is applied to three data sets: two with known structural topologies and one electrocorticogram data set from a human subject during an epileptic seizure.

IV. RESULTS

  • The evaluation uses two data sets with known structural topologies and a third recorded from a human epileptic subject during seizure.The known topologies provide comparisons for functional topologies inferred from dynamic data.Source: §IV · p. 5

A. Pink noise data

In colored-noise simulations, the naive, extremum, and bootstrap tests all recover the underlying topology, but the naive method's success depends on errors that happen to offset. The bootstrap also reaches its p-value detection limit for several pairs.

A. Pink noise data

  • Nine naive-method edges exactly match the known topology, with about one false positive expected under q = 0.10.The simulation uses nine nodes with nine directed connections, but the selected coupling measure does not determine edge direction.Source: A. Pink noise data · p. 6
  • Eight extremum-method edges are identified, one fewer than expected, with about one false positive anticipated.The test uses the linear step-up FDR procedure with q = 0.10.Source: A. Pink noise data · p. 6
  • Ten bootstrap edges are identified, one more than expected; six p-values reach the method's 10^-5 detection limit.The analysis pools surrogate distributions from ten of the 36 electrode pairs.Source: A. Pink noise data · p. 6
  • All three tests recover the colored-noise network, but the naive method succeeds through a delicate balance between errors that cannot be relied on generally.Omitting the Fisher transformation raises observed p-values, while using a normal rather than extremum distribution lowers them.Source: A. Pink noise data · p. 6, Figure 2 · p. 6

B. Simulated neural data

In simulated neural data, extremum and bootstrap tests outperform the naive test, which finds no edges; both better-performing tests approximate, but do not exactly recover, the structural network. Their false-positive counts are approximately consistent with the FDR estimates.

B. Simulated neural data

  • The simulation models 1,000 neurons in 20 groups of 50, with weak connections between groups and strong within-group connections.The inferred functional connectivity is compared with the known structural connectivity between groups.Source: B. Simulated neural data · pp. 6–7
  • The extremum method identifies 17 edges, including 14 structural edges, while the bootstrap detects 15 of the 22 true edges.Each method produces three erroneous edges; the FDR procedure expects about two false positives.Source: B. Simulated neural data · p. 7
  • The extremum and bootstrap tests outperform the naive test, while neither exactly recovers the structural topology.The naive test finds no edges; the other methods detect 17 and 18, respectively, and each produces three erroneous edges, approximately matching the expected false-positive count.Source: B. Simulated neural data · p. 7, Figure 3 · p. 7

C. Human ECoG data

In seizure ECoG data, extremum and bootstrap methods produce similar functional networks, whereas the naive method finds no edges; the structural network is unknown, so anatomical validation is unavailable. In shuffled data, the tests find no edges or only two, as expected when coupling is disrupted.

C. Human ECoG data

  • In 1 s of seizure ECoG from 97 electrodes, the extremum and bootstrap methods yield similar networks with 162 and 187 edges, respectively.The extremum network has nine expected false positives and the bootstrap network ten; the naive method finds no edges.Source: C. Human ECoG data · p. 7, Figure 4 · p. 8
  • The extremum network is similar to the bootstrap network, while the simple threshold network provides no estimate of how many edges are false positives.The bootstrap surrogate computation takes about 90 minutes per network; the extremum method is computationally efficient for this coupling measure.Source: C. Human ECoG data · p. 8
  • Because the ECoG structural connectivity is unknown, the inferred seizure networks cannot be validated against anatomical connections.The study instead shuffles the data to disrupt associations between electrode time series.Source: D. Human ECoG data: shuffled · p. 8

D. Human ECoG data: shuffled

  • For shuffled ECoG, naive and extremum tests find no significant coupling, while bootstrap detects two edges, with one false positive expected.The two bootstrap edges match those in the simple-threshold network.Source: D. Human ECoG data: shuffled · p. 8, Figure 5 · p. 8

V. DISCUSSION

The discussion argues that functional-network inference needs statistically valid edge tests and a principled way to control uncertainty. The authors emphasize that test choice matters and propose choosing the FDR threshold q to calibrate network uncertainty.

V. DISCUSSION

  • The authors use cross correlation for its computational efficiency and available analytic significance tests, while noting that more sophisticated measures may not yield more accurate networks without precise edge p-values.They identify measures distinguishing direct from indirect interactions and incorporating information flow as potentially useful.Source: §V · p. 9
  • Simulations with known physical connectivity showed that significance-test choice is critical: the naive test produced inaccurate p-values and networks because it ignored maximizing the absolute cross correlation.The authors used two additional complementary measures to identify appropriate tests for their association measure.Source: §V · p. 9
  • Choosing q in the linear step-up FDR procedure balances detected edges against confidence; at q = 0.5, half of the declared edges are expected to be false positives.The discussion contrasts this choice with raising q to detect edges in an otherwise trivial network.Source: §V · p. 9
  • Conventional association thresholds provide no confidence measure, may need to vary across data instances, and do not propagate measurement error into network measures such as degree or betweenness.The example threshold of maximal cross correlation above 0.75 appears in.Source: §V · p. 9
  • The authors propose choosing q rather than an association threshold to create networks with specified uncertainty levels that may be calibrated across multivariate data populations.Studying uncertainty in network characteristics and extending the approach to weighted networks are future work.Source: §V · p. 9

VII. APPENDIX 2: NEURAL MODEL

The neural model creates groups with strong local and weaker distant connections, then derives noisy population-activity traces for functional-network construction. The human-data appendix describes ECoG recordings and illustrates a one-second seizure interval used for analysis.

VII. APPENDIX 2: NEURAL MODEL

  • The simulated network divides neurons into groups of 50, with 1,200 directed excitatory synapses within each group and weaker distant connections between groups.Local synaptic strengths are drawn uniformly from 0 to 15; distant strengths from 0 to 5, with delays from 0 to 10 ms for both.Source: §VII · p. 10
  • To approximate population postsynaptic activity, the model averages synaptic-state values across each 50-neuron group; these synaptic dynamics do not affect neuron voltage dynamics.The synaptic state approaches 1 with large excitatory input and decays toward 0 with a 20 ms time constant when input is small.Source: §VII · p. 10
  • The authors simulate 5 seconds of group activity, standardize each trace, add zero-mean Gaussian noise with variance 0.55, and downsample from 1000 Hz to 200 Hz.The resulting traces are shown in Fig. 6.Source: §VII · p. 10, Figure 6 · p. 10

VIII. APPENDIX 3: HUMAN SUBJECT DATA

  • The ECoG illustration marks a one-second interval during the second seizure and shows activity from ten of the 97 analyzed electrodes.The upper trace spans 160 seconds from one electrode; the full recording used 100 electrodes, and four seizures were observed during ten days of monitoring.Source: §VIII · pp. 10–11, Figure 7 · p. 11
Loading 0903.2210v1…