Data-Driven LQR with Uncertainty Refinement and Domain Randomization---Part~I: Certified Uncertainty Refinement

I. INTRODUCTION

• Unknown measurement-noise covariances limit prior noisy-data LQR guarantees, while measurement-noise-correlated regressors bias least squares even with abundant data.Indirect finite-sample and certainty-equivalence analyses also require known noise statistics or prescribed bounds.Source: §I · p. 1
• Delayed IVs jointly quantify model uncertainty and unknown noise covariance, while rejecting models whose implied noise violates prior or statistical moment conditions.Multiple instruments provide overidentifying conditions for a certified data-consistency test; the certificates have finite-sample containment, recursive nesting, and delayed-IV-rate convergence.Source: §I · p. 1
• The refinement gain is quantified directionally by a support radius, and separate process- and measurement-noise covariance bounds are available when the refined set certifies A⋆ is nonsingular.The nonsingularity condition is required by Part II.Source: §I · pp. 1–2

A. System Model and Assumptions

• The system has unknown dynamics and i.i.d. Gaussian process and measurement noise with unknown covariances.The state is observed through noisy measurements; the unknown system matrices are constrained by a known matrix zonotope.Source: A. System Model and Assumptions · p. 2
• Delayed instruments replace the current measured state to obtain a statistically centered relation, avoiding the regressor–noise correlation in direct regression.Instruments stack past measurements, past inputs, or exogenous excitation and must be uncorrelated with the effective noise.Source: A. System Model and Assumptions · p. 2

A. IV-Based Closed-Loop Parametrization

• The closed-loop parametrization expresses the dynamics using collected data and realized noise, separating that past noise from future noise whose covariance must be learned.The model coordinates tie the A- and B-matrix errors to the same realized-noise term.Source: A. IV-Based Closed-Loop Parametrization · pp. 2–3

B. Initial Covariance and Self-Normalized Noise Certificates

• The effective noise is one-dependent, so parity splitting yields independent Gaussian subsequences for covariance bounds.The associated quantile factors αT and βT converge to one as T grows.Source: B. Initial Covariance and… · p. 3
• The initial covariance certificate upper-bounds the unknown effective-noise covariance with probability at least 1 − δc.The certificate is conservative and is refined in later sections.Source: B. Initial Covariance and… · pp. 3–4
• For each delayed instrument, projected-noise bounds transfer to a confidence set for the model, and additional instruments remove models without changing the primary instrument’s rate role.With at least two instruments, the moment conditions overidentify the model parameters.Source: B. Initial Covariance and… · pp. 4–5
• Least-squares bias does not vanish with more data, and its residual covariance need not upper-bound the true covariance; delayed instruments instead satisfy the required zero-moment condition.The least-squares bias limit depends on the regressor–noise cross-moment.Source: B. Initial Covariance and… · p. 5
• Refinement on the same data requires no additional confidence allocation because the covariance and self-normalized bounds hold for every valid covariance certificate.The joint confidence event has probability at least 1 − (δc + δs).Source: B. Initial Covariance and… · p. 5

C. Data- and Prior-Consistent Refinement

• Trajectory consistency assigns each candidate model a unique noise sequence, but alone removes no models; refinement comes from the prior and statistical moment constraints.The trajectory-consistent noise set lies in an affine subspace whose dimension is independent of the sample length.Source: C. Data- and Prior-Consistent Refinement · p. 5
• The refined model and noise sets correspond one-to-one, are convex and compact, and contain the true realization when the covariance prior bound holds on the confidence event.The statistical constraints on each candidate noise sequence are equivalent to the corresponding delayed-IV inequalities.Source: C. Data- and Prior-Consistent Refinement · pp. 5–6
• A constrained matrix-zonotope cover approximates the refined model set using primary-instrument interval bounds, supporting halfspaces, and an exact intersection with the prior.The cover supports computation of the covariance envelope over the surviving models.Source: C. Data- and Prior-Consistent Refinement · pp. 6–7
• The covariance-envelope program yields a next certificate that contains the true covariance and is no larger than the current certificate.The envelope is constructed over a cover of the refined model set; its feasibility certifies the covariance bound.Source: C. Data- and Prior-Consistent Refinement · pp. 7–8
• When λmax(M^⊤Mq) < τ < αT, the covariance recursion contracts geometrically at rate τ/αT; multiple instruments provide valid candidates that can be combined by the S-procedure.The limiting bound described in the passage does not depend on the current covariance certificate or the prior.Source: C. Data- and Prior-Consistent Refinement · p. 8
• The refinement uses one LMI of size 2n + m; for Nψ = 1 and a1 = τ, a Schur complement recovers (52), while Nψ ≥ 2 exploits the smaller intersection of the instrument sets.Source: C. Data- and Prior-Consistent Refinement · p. 8

D. Iterative Refinement and Finite- and Large-Data Properties

• Algorithm 1 alternates between constructing statistical model sets, tightening their covers, computing covariance-envelope candidates, and updating the covariance certificate.If no candidate is accepted, the covariance certificate is unchanged; the procedure also stops when its consistency checks fail or trace decrease falls below a tolerance.Source: D. Iterative Refinement and Finite-… · p. 9
• On the joint confidence event, every model set contains the true model, and each covariance certificate remains between the true covariance and the preceding certificate.The model sets and covariance certificates are recursively nested, so excluded models cannot re-enter the refinement.Source: D. Iterative Refinement and Finite-… · p. 9
• Iteration alone cannot remove finite-sample uncertainty; under the large-data assumptions, model-set widths and diameters shrink at O_P(r_T) for fixed iteration count.The stated convergence argument also requires the excitation and confidence-allocation conditions in Assumption 5.Source: D. Iterative Refinement and Finite-… · p. 9
• With a fixed cut budget and an initialization margin, the first accepted covariance update is within O_P(r_T) of the true covariance, and later certificates retain that rate.A positive lower bound on the specified initialization slack can ensure the margin; coefficient slack alone is insufficient.Source: D. Iterative Refinement and Finite-… · pp. 9–10

E. Quantified Refinement Gain

• For nested model sets, the directional envelope objective is monotone: shrinking the set cannot increase its support-based bound.For dimensions above one, a single envelope realizes the directional bound only up to a gap.Source: E. Quantified Refinement Gain · p. 10
• In direction x, the gain is the support-radius drop from replacing the self-normalized set with the refined set.The prior guarantees a gain wherever it is narrower than the self-normalized set; additional instruments may increase the exact gain.Source: E. Quantified Refinement Gain · pp. 10–11
• A strict exact directional gain need not yield a strict update because the envelope bound has a relaxation gap.Cuts reduce the covariance gap, while tighter relaxations reduce the SDP gap.Source: E. Quantified Refinement Gain · p. 11
• Refinement preserves the O(sqrt(log T/T)) rate while improving its constant by removing initialization looseness.The refined set remains within a factor sqrt(ρJ) of the known-covariance set, with ρJ tending to 1.Source: E. Quantified Refinement Gain · p. 11
• The prior’s guaranteed gain is largest in the low-data regime, while refinement otherwise removes initialization looseness and nothing more asymptotically.The cited low-data condition is λmax(Mq⊤Mq) ≥ αT.Source: E. Quantified Refinement Gain · p. 11

F. Certified Separate Covariance Bounds

• Part II requires separate bounds on W and V, but the full covariance certificate bounds only Σtrue = W + V + A⋆V A⋆⊤.The lag-one covariance of the moving-average residual separates the two noise covariances.Source: F. Certified Separate Covariance Bounds · pp. 11–12
• The lag-one moment satisfies Σ1 = −V A⋆⊤, so V is identifiable from it only when A⋆ is nonsingular.The certificate for Σtrue does not require nonsingularity; refinement first supplies consistent estimators of V and W.Source: F. Certified Separate Covariance Bounds · pp. 11–12
• A finite-sample bound for the lag-one covariance jointly controls it with the zero-lag moment, using a 2n-dimensional pivotal argument.The proof partitions dependent products into three independent residue classes and applies Bernstein’s inequality with a net argument.Source: F. Certified Separate Covariance Bounds · pp. 11–12
• With probability at least 1 − (δc + δs + δ1), the certification conditions ensure A⋆ is nonsingular and support bounds on V and W.The lag-one moment bound is extended uniformly over the refined model set; the separate covariance bounds follow using the full covariance certificate.Source: F. Certified Separate Covariance Bounds · p. 12

G. Consistency Test and Self-Consistent Set

• Two or more instruments can reveal model-class misspecification when their IV estimates converge to different limits and their sets separate.Under a correct model, all instruments contain the true parameter; the test does not detect misspecifications whose limiting IV estimates coincide.Source: G. Consistency Test and… · p. 13
• Theorem 5 bounds the probability of false rejection by δc + δs simultaneously over iterations and outer sets.With at least two instruments, the test has power against misspecifications that produce distinct limiting IV estimates.Source: G. Consistency Test and… · p. 13
• The self-consistent set Θsc is a recursion-free confidence set that needs no initial certificate and may be intersected with Θ(j) without affecting containment.Neither set generally contains the other.Source: G. Consistency Test and… · p. 13
• In the low-data regime, KS may be indefinite and Θsc nonconvex, so Algorithm 1 with the prior remains the tool.The self-consistent construction is available when λmax(Mq⊤Mq) < αT.Source: G. Consistency Test and… · pp. 13–14

IV. SIMULATION RESULTS

• The evaluation asks whether guarantees hold, how conservatism changes with T, what a second instrument adds, and whether the test detects a wrong model class.The study uses the fraction of informative records, certificate inflation, and model-set width as evaluation measures.Source: §IV · p. 14
• The single-instrument configuration uses ψ(1), while the two-instrument configuration adds the past-input instrument ψ(2).The prior is a box of half-width h = .10 unless otherwise stated; the algorithm selects the smallest-trace admissible candidate and stops when trace reduction is below 1%.Source: §IV · p. 14
• The tables report containment and single-instrument results, with Table II giving medians over 100 trajectories per T.Its caption defines pinf as the fraction with λq < αT and J as the iteration count.Source: Table I · p. 14, Table II · p. 14

A. Verification of the Guarantees (Q1)

• Containment held in all runs: empirical frequency was 1.00 versus the guaranteed .98 across 100 trajectories at each of six data lengths.The tested properties were θ⋆∈Θ(J) and Σtrue⪯Σ(J).Source: A. Verification of the Guarantees (Q1) · p. 14

B. Tightness Versus Data Length (Q2)

• For T≤200, no record is informative, so the model set remains the prior box and the recursion without the prior cannot improve Σ(0).The prior and SDP lower covariance inflation ρ by a factor of 4, but do not shrink Θ(J) beyond the prior box.Source: B. Tightness Versus Data Length (Q2) · p. 14
• At T=5000, ρ falls from 2.56 to 1.26 and the model-set width ratio ¯w/¯w⋆ falls from 1.29 to 1.11; the refined set closes 53–19% of the oracle gap for T=500–5000.From T=1000 onward, Algorithm 1, candidate (53), and Σsc agree within 1.2%; ¯w/¯w⋆√log T/T stays in [1.5,1.9].Source: B. Tightness Versus Data Length (Q2) · pp. 14–15
• At T=100, covariance inflation ρ is 22, 246, and 1940 for prior half-widths h=.03, .10, and .30, respectively.With little data, the prior sets the result; the prior-only model-set width is ¯w=2h.Source: B. Tightness Versus Data Length (Q2) · p. 15
• At T=1000, model-set widths are .058, .153, and .201 for prior half-widths .03, .10, and .30, while ρ is the same across those priors.The output remains the prior box until the record is informative; Fig. 2 plots certificate inflation and model-set widths.Source: B. Tightness Versus Data Length (Q2) · p. 15, Figure 2 · p. 15, Table III · p. 15

C. Second Instrument and Consistency Test (Q3, Q4)

• An added instrument preserves containment and can only shrink the model set, with the largest gains when it is informative in directions where the first is weak.In the illustrated system, ψ(1) is not informative for ϵ≤.2, while the past-input instrument alone is never informative at T=5000.Source: C. Second Instrument and Consistency… · p. 15
• For ϵ=.1 at T=5000, adding the second instrument reduces ρ from 13.5 to 1.20 and model-set width from .155 to .047, near the oracle value .044.Across the nominal system, ρ falls from 133 to 4.86 at T=500 and from 2.61 to 1.97 at T=1000; the S-procedure candidate bounds residual covariance over the intersected sets.Source: C. Second Instrument and Consistency… · p. 15, Table IV · p. 15
• With AR(1) process noise, two instruments rejected the wrong white-noise class on every trajectory at a=.95 for T≥5000 and at a=.8 for T=2·10^4, but none at T=1000.With one instrument, at most 5% were rejected; the reported test power comes from overidentification.Source: C. Second Instrument and Consistency… · p. 15

D. Closed Loop, Separate Noise Bounds, and Cost

• At T=5000, the least-squares error levels off at .32, while the IV error reaches .047; the least-squares certificate fails on all 30 trajectories.Containment held on all 30 trajectories at each tested T∈{100,1000,5000}; separating W from V required T=2·10^4.Source: D. Closed Loop, Separate Noise Bounds,… · p. 15

• The model, realized-noise, normalized-noise, and covariance certificates contain their true counterparts, are recursively nested, and converge at the delayed-IV rate.Source: §V · p. 16
• Delayed IVs remove errors-in-variables bias while trajectory, prior, and statistical consistency preserve each admissible model’s dependence on its reconstructed noise.Source: §V · p. 16
• Overidentifying instruments tighten the model set and turn the certificates into a test of the model class.Source: §V · p. 16
• In the data-rich regime, a self-consistent set tightens the uncertainty description without recursion.Source: §V · p. 16
• Simulations confirmed the guarantees; the prior mattered only with little data, while long records were needed for certificates to approach known-covariance counterparts.Source: §V · p. 16
• Part II uses these certificates for robust domain-randomized LQR synthesis.Source: §V · p. 16