PaperlayerView exact revision on arXiv

Source-linked AI summary

Non-uniform state space reconstruction and coupling detection

Ioannis Vlachos, Dimitris Kugiumtzis

arXiv:1007.0394v1nlin.CDcs.ITphysics.data-anq-bio.NCstat.ME

TL;DR

Multivariate reconstruction must identify useful variables and delays despite non-unique embeddings and redundant information. The paper progressively builds purpose-specific embeddings with information criteria, then uses their source components to detect and quantify coupling. The method detects coupling across tested systems and EEG, including changes in information exchange around epileptic seizures.

  • Problem

    Multivariate embeddings can contain redundant components, and the optimal embedding depends on the analysis purpose.

  • Method

    The paper progressively selects variables and delays using information criteria, then derives coupling measures from mixed and projected embeddings.

  • Results

    The scheme detected coupling and measured its strength across nonlinear, stochastic, and epileptic EEG systems, with EEG embeddings showing no antidiametrical components immediately after seizure onset.

  • Takeaways & Limitations

    The form of a mixed embedding can provide a coarse indication of information flow, while S and R provide quantitative coupling measures.

Abstract

from arXiv · show

We investigate the state space reconstruction from multiple time series derived from continuous and discrete systems and propose a method for building embedding vectors progressively using information measure criteria regarding past, current and future states. The embedding scheme can be adapted for different purposes, such as mixed modelling, cross-prediction and Granger causality. In particular we apply this method in order to detect and evaluate information transfer in coupled systems. As a practical application, we investigate in records of scalp epileptic EEG the information flow across brain areas.

I. INTRODUCTION

The paper extends state-space reconstruction from univariate to multivariate time series, where selecting useful variables and delays depends on the reconstruction purpose. It proposes progressively building mixed embedding vectors with information criteria for modelling, prediction, and coupling analysis.

  • Background: Takens’ theorem reconstructs equivalent dynamics from time delays of a single observed time series.The paper positions this theorem as the foundation for system characterization, prediction, and noise filtering.
  • Background: Multivariate embeddings use simultaneous measurements from multiple variables or locations, extending reconstruction beyond scalar observations.Prior applications include noise reduction, nonlinearity testing, and prediction.
  • Motivation: Existing multivariate extensions often overlook dependence among components, although the optimal embedding depends strongly on the reconstruction purpose.This motivates a purpose-specific selection procedure rather than a single universally optimal vector.
  • Contribution: The proposed scheme progressively selects different variables and delays using information criteria, and it also supports two coupling measures for bivariate series.The method is intended to remain adaptable across univariate and multivariate analyses.
  • Multivariate reconstruction: Multivariate reconstruction represents each time series with a variable-specific number of lagged components and can also assign different delays across series.This generalizes fixed-lag embeddings through dimension and delay vectors.

C. Problems and restrictions of multivariate state space reconstruction

Multivariate reconstruction faces identification, scaling, irrelevance, redundancy, and computational-search problems. The proposed non-uniform scheme addresses these issues by progressively selecting components that reduce dependence and explain the system’s future for a chosen modelling purpose.

  • Problems and restrictions: Multivariate identification is non-unique: different component combinations can unfold the attractor, and finite data or noise can change which reconstruction appears optimal.This ambiguity persists even when the total reconstructed dimension is fixed.
  • Problems and restrictions: Different variable ranges can distort delay-vector distances, so the data are scaled to a common range in the study.The implementation uses the range [0, 1].
  • Problems and restrictions: Irrelevance arises when included series are not generated by the dynamical system under investigation, whereas redundancy arises when synchronous or lagged correlations repeat information.Non-uniform lags may reduce redundant information compared with fixed-lag embeddings.
  • Problems and restrictions: Exhaustive component search becomes computationally expensive, reaching 4050 cases for p = 3, M = 3, and maximum lag 10, and 27405 cases when M = 4.These counts motivate a progressive construction procedure.
  • Non-uniform reconstruction: The progressive method selects a subset of candidate lagged components to minimize mutual dependence while maximizing information about a chosen future vector.The future can span variable-specific horizons, and a threshold on successive information gains determines when to stop.
  • Purpose-specific reconstruction: The same scheme supports univariate, cross, mixed, and full modelling by changing which variables populate the candidate and future vectors.Cross and mixed modelling are preferred for checking relations between time series, while full modelling suits invariant-quantity estimation.
  • Estimation constraint: Estimating vector mutual information is difficult because binning requires rapidly increasing data as dimension grows, making the information estimator central to the method.The paper therefore treats estimation accuracy as a practical constraint on embedding construction.

B. Mutual information estimation

The paper compares mutual-information and conditional-mutual-information criteria for progressive embedding selection. K-nearest-neighbor estimation makes CMI less biased in the tested Gaussian simulations and changes later selected components in a Lorenz reconstruction.

  • Information measures: Mutual information is estimated for vector variables, while conditional mutual information is represented through entropy combinations involving X, Y, and Z.The CMI identity is I(X; Y | Z) = I(X; (Y, Z)) − I(X; Z).
  • Estimation: The estimates use K-nearest-neighbor distances, including projected subspaces for the entropy terms required by CMI.This avoids binning estimates whose data requirements grow sharply with dimension.
  • Bias analysis: CMI’s bias is expected to be smaller because it subtracts two biased MI terms, allowing systematic bias to partially cancel.The paper explicitly frames this as an expectation before testing it empirically.
  • Simulation results: Across Gaussian simulations, CMI bias stayed close to zero, whereas MI bias was large and could be positive or negative depending on dimension and correlation.Increasing sample size barely reduced MI bias, and MI bias became larger when dX = 8.
  • Criterion choice: The CMI criterion is therefore judged better suited to the embedding scheme than MI because its lower estimated bias is expected to improve progressive selection.Theoretical equivalence of the criteria does not hold exactly in practice because the estimated conditioning term depends on the candidate component.
  • Embedding example: For the Lorenz reconstruction, MI selected (x_n, z_n−1, z_n−10), whereas CMI selected (x_n, z_n−1, z_n−11) under the stated threshold.The criteria first agree, then diverge at the third cycle and produce different later components.
  • Embedding example: Across ten embedding cycles, MI rose to a maximum and then decreased slightly, while CMI decreased monotonically toward zero as newly added components contributed less.The contrasting trajectories illustrate how bias and conditional information affect stopping and selection.

C. Estimation of information transfer using non-uniform multivariate embedding

The method detects information transfer by examining whether target-predicting embeddings include source components, then quantifies coupling with prediction- and mutual-information-based measures. Its design addresses cases where ordinary delay embeddings can predict the target without revealing the coupling source.

  • Coupling detection: Granger-style transfer is detected when including past values of x improves prediction of y beyond a model using y’s past alone.The approach implements this idea through mixed embeddings and local prediction.
  • Coupling detection: Embedding vectors for predicting y that contain components from x indicate that information from x is transferred to y.This provides a structural coupling signal rather than relying only on prediction-error differences.
  • Practical considerations: The method can require higher stopping thresholds for weak coupling and different future horizons for flows or time series with different complexity.These adjustments are presented as parameter considerations rather than universal settings.
  • Coupling quantification: Coupling strength is evaluated by comparing predictions from mixed embeddings with reduced embeddings containing components from only one source.The mixed embedding combines lagged components from both x and y before local one-step prediction and NRMSE calculation.
  • Prediction measure: The prediction-based measure S compares local model fit using full and reduced projected embeddings, with positive values associated with useful source components.When x is uncoupled, omitting its components leaves S not significant.
  • Information measure: The information-based measure R quantifies the fraction of target information explained by source-derived embedding components and is normalized to range from 0 to 1.The reverse-direction measure is defined analogously.

A. Driven Henon

The driven Henon experiments compare information criteria I0 and I1 for selecting non-uniform embeddings and estimating coupling across strengths, including 20% observational noise. I1 detects driving at weaker coupling, while noise changes embeddings and lowers S values without changing the information-transfer pattern.

  • Across 100 realizations, both criteria selected a single embedding-vector form for each coupling strength, although the forms sometimes differed between criteria.
  • For predicting y, I1 detected the driving at C = 0.1, whereas I0 selected a driver component only from C = 0.2 onward.
  • The monotonic rise of SX→Y with C indicates increasing contribution of x to predicting y, and I1 shifts coupling detection toward smaller C.
  • RX→Y shows results similar to SX→Y for both criteria in the noise-free driven system.
  • With 20% observational noise, embedding vectors change and SX→Y decreases, but information-transfer results remain similar; RX→Y performs better with I1 than I0.

B. Bidirectionally coupled Henon maps

Bidirectionally coupled Henon maps are evaluated with criterion I1 using embedding vectors alongside S and R measures. Embedding forms alone generally cannot identify individual coupling strengths, whereas the quantitative measures distinguish weak coupling and coupling direction.

  • The bidirectional system uses C1 for x driving y and C2 for y driving x, with criterion I1 applied to select embeddings.
  • Except in the asymmetric case C1 = 0.2 and C2 = 0.05, embedding vectors generally have the same form and cannot reveal individual coupling strengths.
  • When C1 = C2 = 0.1, SX→Y and SY→X take equally large values, consistent with equal coupling strength in both directions.
  • For very weak coupling at 0.05, S fails to detect the corresponding direction, while R gives slightly positive values and captures the weak relationship.
  • As coupling increases in one direction, the corresponding R measure increases, and the difference between directional R values indicates the strongest coupling direction.

C. Unidirectionally coupled R¨ossler–Lorenz

The coupled Rössler–Lorenz simulations assess how embedding criteria, threshold A, data length, coupling strength, and future horizon affect coupling detection. Criterion I1 generally detects information transfer more effectively, while finite samples, noise, and generalized synchronization constrain interpretation.

  • Simulation setup: The simulations use y1 from Rössler as the driving series x and y2 from Lorenz as the response series y, with C ranging from 0 to 4.The delay ranges are Lx = Ly = 15, and the future vector initially contains three steps.
  • Embedding criteria: Criterion I1 detects the contribution of x to predicting y for C ≥0.5, whereas I0 detects transfer reliably only for C ≥3.The comparison uses the embedding procedure with A = 0.95 and three-step future vectors.
  • Coupling measures: SX→Y increases almost monotonically with C, while RX→Y follows a similar pattern but begins decreasing from C = 3.The decrease in RX→Y may reflect generalized synchronization occurring for C ≥3, where the variables become functionally related.
  • Threshold and data length: Larger A admits more driving-series components and can produce positive directed measures even for very small C, increasing apparent coupling sensitivity.The study evaluates RX→Y versus A at C = 0.5 and C = 1 using different time-series lengths.
  • Threshold and data length: For C = 0.5, RX→Y detects coupling at A ≥0.95 with N = 8192 and A ≥0.97 with N = 2048; for C = 1, detection spans the examined A range for both N.With N = 512, weak coupling is detected only at larger A and with high variability.
  • Future horizon: Increasing T makes the joint vector higher-dimensional and entropy estimation less accurate, but for weak coupling larger T can improve detection of the driving effect.For C = 0.5 and T ≥4, driving-system components enter the vector explaining y, while uncoupled systems remain unmixed across T.
  • Noise and limitations: With 20% observational noise, embedding forms become more varied and SX→Y and RX→Y retain the noise-free pattern but take smaller values.Criterion I1 can select a response component under strong coupling, consistent with functional dependence between the variables; generalized synchronization can also yield erroneous reverse coupling.

D. Unidirectionally coupled Mackey-Glass

The Mackey–Glass experiments show that progressive mixed embedding tracks coupling direction and strength, while embedding dimension can be underestimated for complex systems. Strong generalized synchronization can make directional attribution ambiguous.

  • The second and third xF choices produced uncoupled embedding dimensions of 3, 4, 9 and 3, 5, 10 for Δ=17, 30, and 100, respectively.These values were just above the corresponding fractal dimensions, unlike the first choice, which substantially underestimated dimension for Δ=100.
  • As coupling increases, components from the driven system decrease while components from the driving system increase in embeddings explaining the driven series.This pattern held across all tested combinations of system complexities and coupling directions.
  • For inverse cross-modelling, embeddings were almost exclusively composed of x components, except at Δ1=17, Δ2=30, C=0.5, where direction was detected incorrectly.The exception occurred under generalized synchronization, which permits components from the two systems to be interchanged.
  • The S and R measures produced qualitatively the same results as the mixed-embedding analysis.

E. Coupled linear stochastic systems

The stochastic-system experiments test whether the embedding scheme identifies regressors and coupling direction in linear dynamic-regression settings. Correct identification improves with record length, but the method is more data-demanding than standard model-order criteria.

  • The proposed scheme was tested on linear dynamic-regression systems in which only y depends on x and C controls dependence strength.The experiments included order-one and mixed-order models with varying sample lengths.
  • For N=4096, the order-one model identified the sole regressor in about half of realizations, while other realizations overestimated model order.For N=512, embedding-vector forms varied even when no driving effect was present or C=0.
  • The correct vector (xn, xn−1) was found in 12 of 100 realizations at N=512 and 78 at N=4096.Intermediate counts were 25 at N=1204 and 50 at N=2048.
  • The correct vector (xn, yn−1) for predicting y was found in 14 realizations at N=512 and increased linearly with sample size.The scheme and resulting S and R measures detected coupling direction but required considerable data.

F. Significance of S and R measures

The study evaluates S and R coupling measures with time-shifted surrogates to test whether apparent directed coupling is statistically significant. On the coupled Rössler–Lorenz system, the test shows good power under the reported conditions.

  • Reliable directed-coupling detection requires testing whether the coupling measure exceeds a significance threshold under a no-coupling null hypothesis.The authors use time-shifted surrogate pairs because the analytic null distribution is unavailable.
  • For N=4096, surrogate testing of SX→Y and RX→Y on the coupled Rössler–Lorenz system showed correct significance and good power.The evaluation used A=0.95, T=3, and 100 realizations.
  • With 40 surrogates, rank ordering yielded α≈0.02 when the original statistic exceeded all surrogate statistics and α≈0.05 when it was second largest.The corresponding reported p-values were 0.0163 and 0.0405.
  • Driving was detected with the same high confidence at C=1 as at larger coupling strengths, although the measures did not reach maximum values at C=1.

V. APPLICATION TO EEG

The embedding procedure is applied to scalp EEG from two epileptic patients to examine information transfer between antidiametrical channels. The results indicate channel-specific exchange patterns and differences between seizure types.

  • EEG records from two patients were segmented into 30-second, N=3000 samples and analyzed across four left-right antidiametrical channel pairs.Surface Laplacian preprocessing was used to improve spatial resolution and reduce common neighboring activity.
  • P3 and P4 showed little or no information exchange, with embeddings almost always lacking components from the antidiametrical channel.
  • For C4, embeddings tended to contain more components from both C3 and C4 than embeddings explaining C3, especially for patient A.
  • Patient A showed channel-dependent embedding dimensions and no apparent change in embedding forms as seizure onset approached.T7 and T8 had substantially more components than the other channels.
  • Differences between corresponding channels in the two patients may reflect their different seizure types.Patient A had generalized tonic-clonic seizure, whereas patient B had partial complex seizure.

VI. DISCUSSION

The proposed progressive embedding scheme performed well in tested systems and supports information-transfer detection across nonlinear, stochastic, and epileptic EEG data. Its threshold choice balances embedding completeness against dimensionality and, under noisy conditions, possible spurious coupling detection.

  • Method: The scheme selects embedding components progressively with information measures, and K-nearest-neighbor estimation was used for greater stability in high dimensions.Mutual-information estimation in high-dimensional vectors can have large bias and variance.
  • Threshold choice: The embedding procedure can under-embed continuous systems when terminated with a strict threshold such as A = 0.95.However, a strict threshold is needed for information-transfer detection.
  • Threshold choice: Larger threshold values can produce large embedding dimensions with many weakly contributing components, while noisy data may yield spurious coupling detection.The extra components do not necessarily impair coupling detection, but specific data conditions can create false positives.
  • Applications: The embedding scheme detects coupling and measures its strength across nonlinear deterministic systems, stochastic linear systems, and epileptic multi-channel EEG.The paper also applies the scheme to invariant estimation, cross-prediction, and mixed prediction.
  • Coupling measures: The prediction measure S and information measure R detected different coupling degrees with good significance and power under appropriate surrogate-data testing.Both measures use mixed and projected embedding vectors.
  • Multivariate extension: The scheme extends beyond bivariate series to investigate direct and indirect coupling among more than two time series.This extension is described as straightforward.
Loading 1007.0394v1…