Multivariate linear regression without prior assumptions

1 Introduction

• The noisy-data covariance is compatible with multiple noise assignments, noise magnitudes, and numbers of independent relations.The ambiguity arises because every measured variable may carry noise, so the covariance alone does not reveal the underlying relationships.Source: §1 · p. 2
• OLS, TLS, and GTLS require prior structural choices about variable partitions or noise variances, and an incorrect choice can bias the model in ways the data cannot reveal.OLS assumes noise-free inputs, TLS treats all variables as equally noisy, and GTLS requires the noisy-variable partition and variances in advance.Source: §1 · p. 2
• QZ-IPCA combines PCA with an iterative QZ-based generalized-eigenvalue solver to identify the full linear model from data geometry alone.The procedure treats OLS, TLS, and GTLS as special cases rather than requiring a prior choice among them.Source: §1 · pp. 2–3
• Across all 32 noise configurations of a five-variable benchmark network, QZ-IPCA identifies relation structure and recovers regression coefficients with less than 6.4% error.It outperforms OLS even with the best input-output partition and identifies rank where standard TLS fails with unequal noise variances.Source: §1 · p. 3

2 Results

• From raw unlabeled data, QZ-IPCA simultaneously recovers variable noise status, per-variable variances, model rank, input-output roles, and regression coefficients.Conventional OLS, TLS, and GTLS require at least three structural quantities to be assumed or specified in advance.Source: §2 · p. 3

2.1 QZ-IPCA correctly identifies model structure across all 32 noise topologies

• Rank was correctly identified in all 32 cases, variance deviations stayed below 0.05, and coefficient error remained within 6.4% across all noise topologies.Source: §2.1 · p. 4, Figure 3 · p. 6

2.2 QZ-IPCA outperforms OLS and standard TLS across representative noise regimes

• OLS’s relative Frobenius norm was approximately 6.62% versus QZ-IPCA’s 1.51% in Case 19, and 4.63% versus 2.35% in Case 32.OLS received the input-output partition identified by QZ-IPCA in each bootstrap replicate; the eigenvalue analysis attributes QZ-IPCA’s advantage to rank identification, which standard TLS cannot resolve under heteroskedasticity.Source: §2.2 · p. 6, Figure 4 · p. 7

2.3 Statistical consistency and practical operating range

• In all-noisy Case 32, variance-estimate bias and confidence-interval width converged toward zero as sample size increased from 50 to 100,000.The authors treat variance consistency as empirical evidence of coefficient-estimate consistency; they also evaluated identification success over eight sample sizes, six SNR values, all 32 configurations, and 1,000 trials per grid point.Source: §2.3 · p. 8

2.4 Robustness to non-Gaussian process excitation

• Under sub- and super-Gaussian excitation, success-rate deviations from the Gaussian baseline stayed within ±5.3 percentage points across 32 cases and 48 operating points.In fully stochastic cases, deviations were confined to the N=100–1000 transition zone without consistent directional bias and converged within the ±3.1% Monte Carlo sampling floor by N=2000.Source: §2.4 · pp. 8–9

3 Conclusion

• QZ-IPCA combines PCA and a QZ-based generalized eigenvalue solver that remains well-posed under singular noise covariance, subsuming OLS, TLS, and generalized TLS as special cases.The framework recovers structural information from data geometry rather than requiring users to select a classical framework a priori.Source: §3 · p. 9
• Across all 32 noise topologies of a benchmark flow network, the method is validated against classical baselines, with its advantage linked to rank identification rather than auxiliary structural knowledge.Source: §3 · p. 9
• Dynamic MIMO systems, where constraint relations hold over time windows, are proposed as a natural extension of the framework.The conclusion also suggests broader application to estimation problems whose standard solutions rely on prior commitments.Source: §3 · p. 9

4.1 System description and problem formulation

• OLS, TLS, and mixed partial-EIV methods require users to specify different aspects of the noise structure, and misspecification biases the resulting model.OLS declares an input-output partition, TLS assumes all variables are equally noisy, and partial-EIV requires the noisy/non-noisy partition.Source: §4.1 · p. 12
• The constraint-matrix formulation avoids prespecifying dependent variables and is unaffected by multicollinearity, while assuming diagonal heteroskedastic noise uncorrelated with the true values.Source: §4.1 · p. 12
• The problem is to estimate the noisy-variable partition, noise covariance, relation count, and constraint matrix from measurements alone, subsuming OLS, TLS, and partial-EIV or GTLS as noise-configuration cases.Source: §4.1 · p. 12

4.2 Related work

• Classical OLS, TLS, GTLS, and partial-EIV procedures estimate models once noise structure is known, with the partition or variances supplied as inputs.Known-noise PCA scaling assumes noise covariance is constant across samples; MLPCA can handle sample-varying, cross-correlated noise through an iterative procedure.Source: §4.2 · p. 13
• IPCA estimates unknown noise covariance only in strict EIV settings and is undefined when any variable is exact, leaving no existing method to recover the full structure simultaneously.For diagonal noise covariance, IPCA requires d(d + 1)/2 ≥ n; sparse non-diagonal covariance would require specifying its nonzero pattern in advance.Source: §4.2 · pp. 13–14

4.3 Unifying identification framework

• Zero eigenvalues of the raw covariance identify relations involving only non-noisy variables and reveal an initial subset of non-noisy variables.Additional non-noisy variables may remain undetected when they also participate in stochastic relations.Source: §4.3 · p. 15
• After identified redundant non-noisy variables are removed, the reduced data matrix can still contain exact variables, making its noise covariance potentially singular.Source: §4.3 · p. 15
• The remaining constraints are obtained from generalized eigenvectors of the reduced data covariance relative to its noise covariance, using QZ to handle singular covariance.Zero noise variances map to infinite generalized eigenvalues rather than causing the QZ algorithm to fail.Source: §4.3 · p. 15
• Under correct variance scaling, the smallest equal finite generalized eigenvalues cluster at unity, identifying the number of noisy relations.Source: §4.3 · pp. 15–16
• QZ-based IPCA iteratively updates noise variances and generalized eigenvalues until convergence and can reveal further non-noisy variables when estimated variances approach zero.Source: §4.3 · p. 16
• Combining the framework's concepts yields the full constraint matrix without prior declarations, while the all-noise-free case terminates with zero noise covariance and the deterministic relations.The appended offset column is fixed as noise-free, so its offset is estimated without measurement noise by construction.Source: §4.3 · p. 16

4.4 Benchmark system and noise topology enumeration

• The benchmark is a five-variable flow network of three process units and five flow streams, governed by three linear mass-balance relations at steady state.Source: §4.4 · p. 16
• The 32 noise configurations enumerate every binary assignment of noisy versus noise-free status across the five variables.Source: §4.4 · p. 16
• For every configuration, QZ-IPCA receives only the raw data matrix, without noise labels, variances, variable roles, or relation count specified.Source: §4.4 · p. 16

4.5 Data generation

• F1 and F2 start at 10 L/min, with normally distributed zero-mean fluctuations; F3–F5 are computed from the network’s linear flow-balance constraints.The excitation variances for F1 and F2 are 4 and 2, respectively.Source: §4.5 · p. 17
• The true channel signal variances are computed empirically from generated data and held fixed for each sample size; channel noise sequences are independent across channels, time steps, and replicates.Noise is statistically independent of the corresponding true signal values; the configuration includes three distinct noise-variance levels.Source: §4.5 · p. 17
• Noise is added independently to variables designated noisy, while designated noise-free variables receive none.For noisy variables, the variances are based on each channel’s true signal power and held fixed across replicate runs at a given sample size and SNR.Source: §4.5 · p. 17

4.6 Bootstrap simulation protocol

• Each of the 32 noise configurations uses 1,000 independently generated data matrices at N = 2000 and SNR = 10, with replicate-specific model, partition, and variance estimates.Reported point estimates are replicate means, and 95% intervals use the non-parametric percentile method across replicates.Source: §4.6 · pp. 17–18

4.7 Performance metrics

• Performance is assessed using exact rank-identification outcomes, per-channel variance deviations, normalized Frobenius coefficient error, and maximum element-wise coefficient error.The maximum element-wise error is reported alongside offset-term error.Source: §4.7 · p. 18
• Across 32 cases, every 95% confidence interval for an individual offset term spans zero, consistent with unbiased recovery relative to the zero constant term.The offset-term confidence intervals are not reported in the table.Source: §4.7 · p. 18

4.8 Comparative benchmarking against OLS and TLS

• OLS is given QZ-IPCA’s replicate-specific input-output partition, so both methods estimate regression matrices under the same partition.This design aims to make the remaining performance gap reflect estimation quality rather than structural ignorance; OLS’s bootstrap distribution also includes partition variation.Source: §4.8 · p. 19
• Standard TLS treats variables symmetrically and, without known variances, operates as PCA on raw covariance; GTLS is excluded because it requires variances and the noisy-variable partition as inputs.The comparison spans all-noise-free, representative mixed-noise, and all-noisy cases.Source: §4.8 · pp. 18–19

4.9 Asymptotic consistency analysis

• Variance-estimator behavior is examined in the all-noisy Case 32 across sample sizes from 50 to 100,000 at SNR = 10, using 1,000 bootstrap runs per size.The stated analysis computes empirical mean residuals and percentile confidence intervals; the supplied passages do not report its outcomes.Source: §4.9 · p. 20

4.10 Operating envelope evaluation

• The operating-envelope evaluation spans eight sample sizes and six SNR values across all 32 configurations, with 1,000 Monte Carlo trials at each operating point.A trial counts as successful only if both relation dimensions and the full noisy/noise-free variable partition are correctly identified; the 95% confidence half-width is approximately ±3.1%.Source: §4.10 · pp. 20–21

4.11 Robustness to non-Gaussian excitation

• The operating-envelope evaluation is repeated with sub-Gaussian uniform and super-Gaussian Laplace excitation, changing only the free-variable excitation distribution.Both regimes preserve the baseline excitation variance; the uniform distribution has kurtosis 1.8 and the Laplace distribution has excess kurtosis 3.Source: §4.11 · p. 21
• Success-rate deviations are compared pointwise with the Gaussian baseline, and deviations within ±0.031 are treated as consistent with it within Monte Carlo sampling noise.The supplied passages describe the comparison procedure but do not report the observed deviations.Source: §4.11 · p. 21

• The study generated its data according to Subsection 4.5 and used no external datasets.Source: Data availability · p. 22
• M.S.K.G. designed and implemented the algorithm; M.S.K.G. and D.D. conducted experiments, and D.D. created figures and drafted the paper.A.K.T. and S.N. jointly supervised the work, suggested experiments, contributed ideas, and provided feedback.Source: Data availability · p. 22
• The authors declare no competing interests.Source: Data availability · p. 22

• A separate supplementary PDF containing extended methods is available as an ancillary file on the preprint’s arXiv abstract page.Source: Additional information · p. 22
• The autonomous pipeline takes raw unlabeled data, uses PCA to isolate non-noisy variables, and applies QZ-IPCA to estimate the constraint matrix, relation count, and noise covariance.The framework is presented as subsuming ordinary, total, and generalized least squares.Source: Figure 5 · p. 24
• The iterative solver updates the noise covariance while testing candidate degrees of freedom, stopping when a hypothesis test supports equality of the smallest d eigenvalues.It uses the QZ algorithm to solve a generalized eigenvalue problem and constructs the constraint matrix from recovered generalized eigenvectors.Source: Figure 6 · p. 25
• In the all-noisy topology, variance-estimate bias and confidence-interval width converge toward zero with increasing N, supporting consistency of coefficient estimates through PCA.The figure examines all five flow variables at SNR = 10.Source: Additional information · p. 26, Figure 7 · p. 26
• For cases 17–32, success exceeds 0.5 at N > 200 regardless of SNR and approaches 1.0 at N ≥ 2000 and SNR ≥ 5.Cases 1–16 achieve perfect or near-perfect success across the grid, with marginal degradation confined to N ≤ 200 in the most structurally complex cases.Source: Additional information · p. 27, Figure 8 · p. 27
• Under sub-Gaussian excitation, deviations from Gaussian-baseline success are bounded by 5.3 percentage points and fall within the Monte Carlo noise floor by N = 2000.In cases with deterministic constraints, deviations are negligible; in fully stochastic cases, they are transient and show no consistent directional bias.Source: Additional information · p. 28, Figure 9 · p. 28
• Under Laplace excitation, deviations are also bounded by 5.3 percentage points and fall within the ±3.1% Monte Carlo noise floor by N = 2000.The text reports no residual bias at large N and describes the finite-sample deviations as concentrated in the transition zone.Source: Additional information · p. 29, Figure 10 · p. 29
• Extended Data Tables 1–2 cover noisy-variable identification and variance estimation, while Table 3 covers model-structure identification and coefficient recovery.Source: Table 1 · p. 30, Table 2 · p. 31, Table 3 · p. 32