Performance Enhancement of Parameter Estimators via Dynamic Regressor Extension and Mixing

Aranovskiy Stanislav; Bobtsov Alexey; Ortega Romeo; Pyrkin Anton

Source-linked AI summary

Performance Enhancement of Parameter Estimators via Dynamic Regressor Extension and Mixing

Aranovskiy Stanislav, Bobtsov Alexey, Ortega Romeo, Pyrkin Anton

arXiv:1509.02763v1Sep 9, 2015eess.SY

TL;DR

The paper addresses parameter estimation when standard linear-regression convergence relies on a persistency-of-excitation condition and when nonlinear parameter dependence is only partly monotonic. It proposes DREM, which dynamically extends and mixes regressions to obtain estimators with alternative convergence conditions. The resulting approach establishes convergence without PE in the linear case and supports consistent estimation for factorisable nonlinear regressions with partially monotonic nonlinearities.

Problem
Standard linear-regression estimators require regressor PE for exponential stability, while nonlinear parameterizations are difficult to estimate when monotonicity holds only for some nonlinearities.
Method
DREM applies dynamic operators to generate new regression forms and then mixes them into regressions to which standard parameter-estimation techniques are applied.
Results
DREM establishes parameter convergence for linear regressions without PE and enables consistent estimation for factorisable nonlinear regressions with only partially monotonic nonlinearities.
Takeaways & Limitations
The procedure replaces the standard PE requirement with conditions involving the new regressor, including non-square-integrability of its determinant.

Abstract

from arXiv · show

A new way to design parameter estimators with enhanced performance is proposed in the paper. The procedure consists of two stages, first, the generation of new regression forms via the application of a dynamic operator to the original regression. Second, a suitable mix of these new regressors to obtain the final desired regression form. For classical linear regression forms the procedure yields a new parameter estimator whose convergence is established without the usual requirement of regressor persistency of excitation. The technique is also applied to nonlinear regressions with "partially" monotonic parameter dependence---giving rise again to estimators with enhanced performance. Simulation results illustrate the advantages of the proposed procedure in both scenarios.

I. INTRODUCTION

The paper proposes Dynamic Regressor Extension and Mixing (DREM), which generates and mixes new regressions to design estimators for linear and nonlinear parameterizations. For linear regressions, it replaces the usual PE requirement with different convergence conditions, while for nonlinear regressions it exploits partially monotonic structure.

DREM procedure: DREM applies dynamic operators to an original regression, then mixes the resulting data into a regression suitable for standard parameter-estimation techniques.The procedure has a regression-extension stage and a mixing stage.
Nonlinear regressions: For nonlinear regressions, DREM isolates nonlinearities with usable monotonicity so consistent estimation can address factorisable parameterizations where only some nonlinearities are monotonic.The paper also proposes linearizing a nonlinear mapping around a distinguished point to obtain a separable nonlinear regression.
Linear regressions: For classical linear regressions, DREM designs parameter estimators whose convergence does not require persistency of excitation of the original regressor.The standard linear time-varying estimator is exponentially stable if and only if the regressor is PE, a condition often unmet in applications.
Linear regressions: Each scalar regression is used to estimate an individual parameter with a scalar adaptation law, and the resulting error convergence is characterized by the mixed regressor.The derivation proceeds from the scalar regressions to the estimator and its convergence implication.
Linear regressions: The linear-regression construction filters the original regression, stacks the original and filtered regressors, and premultiplies the result by an adjugate matrix to obtain scalar regressions.The filtering operators are linear and L∞-stable; examples include exponentially stable LTI filters and delay operators.

C. Discussion

The discussion compares the mixed-regressor condition with PE and examines how operator choice affects convergence. It emphasizes that DREM produces convergence conditions involving the determinant and eigenvalues of the extended regressor matrix rather than the standard PE requirement.

Open questions: The discussion asks whether φ(t) ∉ L2 is weaker than PE and how operators can be selected to enforce this condition when the original regressor is not PE.These questions frame the relationship between DREM’s condition and the standard requirement.
Relation to PE: PE implies a non-square-integrable regressor, but the converse need not hold.The discussion gives an example described as neither square integrable nor PE.
Relation to PE: The condition φ(t) ∉ L2 constrains all eigenvalues of the extended regressor matrix, whereas PE is imposed through its minimum eigenvalue.This follows from the determinant being the product of the matrix eigenvalues.
Convergence condition: A necessary condition for convergence of the DREM estimators is that every eigenvalue of the extended regressor matrix is not square integrable.The determinant-based condition therefore differs structurally from the standard PE requirement.

D. An example

The example considers a two-dimensional regressor and a simple filter to construct a mixed scalar regressor. It identifies a class of non-PE regressors for which the mixed regressor is not square integrable.

Example result: The example identifies a class of regressors with m(t) ∉ PE but φ(t) ∉ L2 when a simple LTI filter is used.This class is stated as the target of the proposition in the example.
Example result: The proposition defines a set G of differentiable bounded functions whose derivatives are bounded and not square integrable, then defines the filter operator from this construction.The function φ generated by the resulting regressor satisfies φ(t) ∉ L2.
Example derivation: The proof establishes non-PE behavior from the limiting regressor component and derives the mixed-regressor property by comparing filtered and unfiltered expressions.Exponentially decaying filter-initialization terms are retained in the derivation.
Example derivation: The filter choice α = β = 1 is not essential, because analogous results hold for any exponentially stable LTI filter.Filter selection remains available for verifying the non-square-integrability condition.

E. Simulation results

The classical estimator fails to show convergence for the tested non-PE regressor, whereas DREM substantially improves oscillatory behavior and convergence speed with straightforward gain tuning.

The classical estimator is stable but not exponentially stable because the regressor m(t) is not persistently exciting.
After 500 time units, the classical parameter errors have not converged, and increasing γ from 3 to 10 worsens the transient behavior.
DREM produces significant improvements in oscillatory behavior and convergence speed for the same regressor and parameter setting.
The classical-estimator simulations use transient errors and integral curves with γ=3 and γ=10, respectively, under the stated initial-condition settings.
DREM gain tuning is straightforward, with simulations linking the selected gains to the observed behavior.

III. PARAMETER ESTIMATION OF “PARTIALLY” MONOTONIC REGRESSIONS

DREM is applied to factorisable nonlinear regressions with only partial monotonicity, isolating the monotonic nonlinearities and extending the approach through local factorisation procedures.

The proposed nonlinear-regression application targets factorisable nonlinearities where some, but not all, functions satisfy a monotonicity condition.
DREM generates a new regressor containing only the “good” nonlinearities that satisfy the desired monotonicity property.
The considered regression uses measurable signals y and m, a known mapping ψ, and an unknown parameter vector θ.
Figures 3 and 4 evaluate DREM transient errors and integral curves using the stated regressor, gains, and initial-condition settings.
The nonlinear regression can be transformed into a linear regression in η:=ψ(θ), allowing application of the standard gradient estimator.
The paper discusses overparameterisation and gives a local procedure for factorising nonlinear dynamical-system regressions with non-factorisable parameter dependence.

A. First example

The first example applies DREM to a scalar nonlinear regression with one strongly monotonic parameter function and one nonmonotonic function. The resulting estimator converges under a condition on the mixed regressor that differs from PE, whereas overparameterization cannot recover the parameter globally.

Setup: The example uses a regression whose parameter dependence includes a strongly monotonically increasing ψ1.The stated assumption provides ρ1 > 0 for the monotonicity condition.
DREM construction: DREM filters and mixes the regression to produce a new scalar regression involving only the strongly monotonic function ψ1.The procedure applies an operator H, stacks the original and filtered regressions, and multiplies them by [m2f −m2].
Convergence: If Φ(t) /∈L2, the resulting estimation error ˜θ(t) converges to zero.The convergence follows from the Lyapunov analysis of the scalar error equation.
Regressor condition: A delay-operator construction gives m(t) /∈PE but Φ(t)̸ ∈L2, satisfying the convergence condition for DREM.The paper identifies this class of regressors and states that the constructed Φ is not square-integrable.
Simulation: The overparameterized estimator shows poor convergence, while the DREM estimator converges to zero and can be accelerated by increasing γ.The overparameterized approach also cannot reconstruct θ globally because cos(θ) is not injective on R.
Monotonicity qualification: With only strict or plain monotonicity, the analysis guarantees nonincrease of the Lyapunov function and requires additional PE-like conditions for convergence.DREM also applies when the monotone function is decreasing or when ψ2 is the monotone component.

B. Second example

The second example extends DREM to a two-parameter, three-function factorisable regression with two monotonic nonlinearities. Filtering and annihilation remove the nonmonotonic component and yield convergence governed by determinant conditions on the resulting regressor matrix.

Setup: The example assumes that ψ2 and ψ3 satisfy the monotonicity condition and groups them into the “good” vector ψg.The corresponding mapping is required to satisfy a strong P-monotonicity property for some positive definite P.
DREM construction: DREM filters one row, stacks the resulting regression, and applies a left annihilator to eliminate the first-column contribution.This constructs a 2 × 2 regression matrix for the two estimated parameters.
Convergence: If det2{Φ(t)} ≥κ > 0, the estimation error is exponentially stable.The Lyapunov derivative is bounded by a negative term proportional to det2{Φ(t)} and the largest eigenvalue of Γ.
Convergence: Without the uniform determinant bound, ˜θ(t) still converges to zero when det{Φ(t)} /∈L2.The result follows by integrating the Lyapunov inequality.
Design freedom: The determinant behavior can be shaped through choices of the operator H and the rows selected for filtering and mixing.The stated design objective is to make det{Φ(t)} ideally uniformly bounded away from zero.
Verification: The nonlinear monotonicity condition can be checked locally through an LMI when prior parameter-set information is available.The test uses vertex-computable vectors vi and feasibility of P∇ψg(vi) + [∇ψg(vi)]⊤P > 0.

C. A general result

The general result treats factorisable nonlinear regressions with some monotonic parameter functions and constructs a square regressor by filtering and annihilating the remaining components. It establishes asymptotic or exponential convergence from determinant conditions.

General formulation: The general formulation separates the regression into monotonic components ψg and remaining components ψb.The reordered model uses mg, mb, ψg, and ψb with dimensions specified for the general factorisable regression.
Design objectives: DREM must eliminate ψb while producing a square or tall regressor so PE can be relaxed.The formulation imposes assumptions designed to make these two objectives possible.
Construction: Filtering selected rows and stacking the resulting regressions creates the extended regression used for elimination.When n = p, no operators are needed because mb is tall and admits a full-rank left annihilator; analogous simplifications occur when n > p.
Dimensioning: The construction uses nf = p −n operators so a q-row left annihilator can produce a square new regressor.The required operator count follows from the dimension of the component to be eliminated.
Main result: Under the stated assumptions, det{Φ(t)} /∈L2 implies lim t→∞|˜θ(t)| = 0.This is the asymptotic convergence conclusion of Proposition 4.
Main result: If det2{Φ(t)} ≥κ > 0, the parameter error tends to zero exponentially fast.The uniform determinant condition strengthens the asymptotic result to exponential convergence.

D. Discussion on overparameterisation

Linearizing a nonlinear regression introduces overparameterization, which can degrade performance and restrict estimation validity. It can also prevent recovery of the original parameters and require more stringent excitation conditions.

Overparameterization can slow convergence because estimation searches a larger parameter space.
It imposes more stringent reference-signal conditions to provide the persistency of excitation needed for parameter convergence.
The transformed estimator generally cannot recover the true parameter θ unless an appropriate mapping is injected.
Overparameterization introduces conservativeness when prior knowledge is incorporated into restricted parameter estimation.
Because the parameter mappings are generally only locally invertible, the estimates have a reduced domain of validity.

E. Generating separable regressions for nonlinear dynamical systems

The paper constructs state-dependent nonlinear regressions for dynamical systems and locally linearizes them into separable parameterizations. DREM can then be applied when enough transformed parameter components have monotonic dependence, although PE still fails in regulation as the state deviation vanishes.

The system model contains known functions F0 and F1, a measurable state x, and constant unknown parameters θ around an operating point x∗.
Classical filtering generates a state-dependent regression, while exponentially decaying terms are neglected to obtain a nonlinear parameterization.
Estimating the general nonlinear regression is difficult, so Ξ is linearized around x∗ to obtain a separable nonlinear regression.
The first-order approximation represents the regression using m and ψ, with the lifted parameter dimension p defined as n + n2.
DREM applies when at least q components of ψ(θ) have monotonicity properties.
In regulation tasks, PE is not satisfied when the state deviation ˜x(t) tends to zero.

IV. CONCLUDING REMARKS AND FUTURE RESEARCH

The paper concludes that DREM changes the convergence requirements for linear and partially monotonic nonlinear parameter estimation. Future work focuses on making its design degrees of freedom more systematic in structured applications.

For linear regressions, DREM establishes parameter convergence without the usual PE condition, replacing it with non-square-integrability of the new regressor vector.
For nonlinear regressions with partially monotonic parameterizations, the procedure also yields parameter estimators with enhanced performance.
The method has many degrees of freedom for verifying convergence, but selecting them systematically remains difficult at the paper’s level of generality.