PaperlayerView exact revision on arXiv

Source-linked AI summary

Improved estimators for a general class of beta regression models

Alexandre B. Simas, Wagner Barreto-Souza, Andréa V. Rocha

arXiv:0809.1878v1stat.MEstat.CO

TL;DR

The paper addresses bias in maximum likelihood estimation for flexible beta regression models, especially when regression structures are nonlinear or include precision covariates. It derives analytical and bootstrap corrections and finds that corrective and parametric-bootstrap methods generally perform best, with different advantages in bias, MSE, and computation.

  • Problem

    Bias-corrected beta-distribution estimators do not generally extend to regression structures, while nonlinear regression MLE bias is problematic in small samples.

  • Method

    The paper extends beta regression with nonlinear mean and precision structures and derives O(n^-1) bias corrections using Cox–Snell, Firth, and bootstrap approaches.

  • Results

    The corrective method generally has the smallest bias, whereas the parametric bootstrap generally has the smallest MSE in simulations.

  • Takeaways & Limitations

    The corrective method offers a non-computer-intensive alternative to the parametric bootstrap, while both improve on the MLE in the reported simulations.

  • Takeaways & Limitations

    The model assumes rank conditions, functionally independent regression parameters, and strictly monotonic twice-differentiable link functions.

Abstract

from arXiv · show

In this paper we consider an extension of the beta regression model proposed by Ferrari and Cribari-Neto (2004). We extend their model in two different ways, first, we let the regression structure be nonlinear, second, we allow a regression structure for the precision parameter, moreover, this regression structure may also be nonlinear. Generally, the beta regression is useful to situations where the response is restricted to the standard unit interval and the regression structure involves regressors and unknown parameters. We derive general formulae for second-order biases of the maximum likelihood estimators and use them to define bias-corrected estimators. Our formulae generalizes the results obtained by Ospina et al. (2006), and are easily implemented by means of supplementary weighted linear regressions. We also compare these bias-corrected estimators with three different estimators which are also bias-free to the second-order, one analytical and the other two based on bootstrap methods. These estimators are compared by simulation. We present an empirical application.

1 Introduction

The paper extends beta regression to nonlinear mean and precision structures, then develops second-order bias corrections for maximum likelihood estimators. It compares analytical and bootstrap corrections through simulation and applications.

  • Beta regression models responses in the standard unit interval, where observations can represent rates or proportions.
  • The proposed model allows nonlinear regression for the mean and a potentially nonlinear regression structure for the precision parameter.The precision structure naturally accommodates heteroscedasticity, and its covariates need not be distinct from mean covariates.
  • Maximum likelihood estimators in nonlinear regression are typically biased at order O(n^-1), with small samples making this bias more problematic.
  • Existing bias corrections for beta-distribution parameters do not extend to regression structures, while prior beta-regression work derived O(n^-1) bias expressions for a narrower model.
  • The paper derives closed-form second-order biases and defines corrective, preventive, and bootstrap bias-adjusted estimators.The biases concern model parameters, response means, and precision parameters; auxiliary weighted linear regressions facilitate computation.

2 The model

The model parameterizes beta responses by their mean and precision, then links both quantities to covariates through flexible regression structures. Regularity and asymptotic results support maximum likelihood inference.

  • Under the mean–precision parameterization, Y follows B(µ, φ) with E(Y)=µ and Var(Y)=µ(1−µ)/(1+φ).Here φ acts as a precision parameter.
  • The log-likelihood is constructed from the beta density after expressing its shape parameters as µiφi and (1−µi)φi.
  • Each observation yi is modeled as beta distributed with mean µi and precision φi, both determined by functional regression relations.
  • The mean and precision predictors may be nonlinear, may use overlapping covariates, and are connected through strictly monotonic twice-differentiable link functions.Examples include logit, probit, and complementary log-log links for the mean.
  • The MLEs solve the nonlinear score equations and can be computed by numerical maximization, such as with BFGS.
  • Mean and precision parameters are not orthogonal because the cross-information block Wβθ is nonzero, although the MLEs and estimated information matrix are consistent.

3 Bias correction of the MLEs of β and θ

This section derives O(n^-1) bias formulas for the regression estimators and turns them into analytical and bootstrap corrections. The analytical bias can be computed through a formal weighted linear regression.

  • The Cox–Snell formula is used to derive second-order biases for β and θ and thereby construct bias-corrected estimators.
  • The joint bias B(ζ̂) is expressed in matrix form using the inverse Fisher information, derivative matrices, projection-related quantities, and weighted vectors.
  • The bias equals the coefficient vector from a formal weighted linear regression of ξ=ξ1+ξ2 on the columns of P with W as weights.
  • The O(n^-1) bias separates into a component generalizing the linear beta-regression result and a component caused by nonlinear mean and precision functions.The nonlinearity component vanishes when both regression functions are linear.
  • The corrective estimator replaces the MLE by subtracting an estimated O(n^-1) bias, while Firth’s preventive method modifies the score function before estimation.
  • Parametric and nonparametric bootstrap corrections estimate bias from resampled estimator means and subtract that estimate from the MLE.Parametric samples use the fitted distribution, whereas nonparametric samples use the empirical distribution without parametric assumptions.

4 Bias correction of the MLEs of µ and φ

This section derives second-order bias expressions for estimators of the response means and precision parameters, then uses them to construct bias-corrected alternatives. It also presents analytical, modified-score, and bootstrap-based adjustment routes, with link-function derivatives supplied for implementation.

  • Bias derivation: The section derives O(n^-1) biases for the maximum likelihood estimators of µ and φ because practitioners need accurate response-mean and precision estimates.The biases of µ and φ must be computed separately; bias-free regression-parameter estimators do not generally make transformed estimates bias-free to the same order.
  • Bias derivation: Taylor expansion and expectations are used to obtain the O(n^-1) biases of the linear predictors η1 and η2, with matrix expressions built from link derivatives and asymptotic covariance matrices.The notation includes derivative vectors for η1 and η2, matrices F and G, and diagonal matrices containing second derivatives of the inverse links.
  • Bias correction: The resulting explicit bias formulae for µ and φ support second-order bias-corrected estimators based on estimated analytical biases.Unknown quantities in the bias terms are replaced by their maximum likelihood estimates.
  • Alternative methods: Alternative corrections use the modified score function associated with Firth’s preventive method and parametric bootstrap estimates of the O(n^-1) biases.Bootstrap bias vectors are formed for both µ and φ and used to define additional corrected estimators.
  • Implementation: Tables 1 and 2 list first and second derivatives of common link functions for the mean and precision submodels, respectively.The tables are intended to facilitate practical application of the bias formulae.

5 Some special cases

The paper specializes its general bias-correction results to four commonly used beta-regression settings. For each setting, it provides model-specific scores, information matrices, and second-order bias expressions.

  • Scope: Four special cases are treated: linear beta regression, linear beta regression with dispersion covariates, nonlinear beta regression, and nonlinear beta regression with linear dispersion covariates.The section gives closed-form expressions for scores, Fisher information, and O(n^-1) biases of β, θ, µ, and φ.
  • 5.1 The linear beta regression model: In linear beta regression, the mean uses a link function while φ is constant, yielding ˜X = X and ˜Z = 1.The section specializes the general score and information expressions and shows agreement with Ferrari and Cribari-Neto (2004).
  • Bias correction: When both predictor structures are linear, the nonlinear terms ξ2 and ω2 vanish, simplifying the analytical bias and modified-score expressions.The simplified results agree with the findings of Ospina et al. (2006) for the corresponding linear models.
  • 5.2 The linear beta regression model with dispersion covariates: For linear beta regression with dispersion covariates, both mean and precision have linear regression structures, so ˜X = X and ˜Z = Z.The score vector retains the general form after replacing the expanded design matrices, and the section derives model-specific information and bias expressions.
  • 5.3 The nonlinear beta regression model: In the nonlinear beta regression case, the mean structure is nonlinear while φ remains constant and ˜Z = 1.The section derives the specialized score, Fisher information, parameter bias, modified score, and mean-bias expressions.
  • 5.4 The nonlinear beta regression model with linear dispersion covariates: The final case combines a nonlinear mean structure with a linear precision regression, retaining ˜X and setting ˜Z = Z.Its score, information matrix, parameter biases, modified score, and biases of µ and φ are obtained from the general framework.

6 Numerical results

Monte Carlo experiments compare maximum likelihood, analytical bias-correction, preventive, and bootstrap estimators across sample sizes and parameters. Corrective and parametric-bootstrap methods generally improve bias or MSE, while preventive correction performs poorly and nonlinear models often outperform linearized versions.

  • n = 20: For n = 20, corrective estimators reduced bias and MSE relative to MLEs, while preventive estimators had larger bias and MSE.The parametric bootstrap had the smallest MSE across parameters, although its bias was not always smallest.
  • n = 20: For n = 20, parametric bootstrap schemes had the best MSE for µ, while corrective MSE was always no larger than MLE MSE.Nonparametric bootstrap schemes had the largest MSE, although estimates of the means were generally close across methods.
  • n = 20: For n = 20, corrective estimation had relatively small bias when φ was large, whereas Taylor-based bootstrap correction was omitted because it performed worse than regular bootstrap correction.The comparison concerns bias across the corrective and bootstrap schemes for the precision parameter.
  • n = 40: For n = 40, corrective and parametric-bootstrap estimators generally performed best for β and θ, while preventive estimation remained poor.Nonparametric bootstrap performance exceeded MLE for θ but was worse for β; for µ, corrective estimators had the best MSE.
  • Precision parameter φ: For φ, MLE had the largest bias and MSE, parametric bootstrap had the smallest MSE, and corrective estimation had smaller bias when the true φ was large.Both corrective and parametric-bootstrap estimators improved on MLE with respect to bias and MSE.

7 Application to real data

The application fits a linear beta regression model with covariates for both the mean and precision parameters to crude-oil distillation data. Tests reject the reduced model without a precision covariate, while corrected and uncorrected mean estimates remain similar but precision estimates differ considerably.

  • Data and model: The data model crude-oil conversion after fractional distillation using crude-oil level and vaporization temperature as explanatory variables.The sample contains n = 32 observations; the mean structure uses an intercept, nine level indicators, and temperature, while the precision structure uses an intercept and one covariate.
  • Data and model: The mean uses a logit link, whereas the precision parameter uses a log link, with coefficients estimated by maximum likelihood using BFGS.
  • Model assessment: The LRT and score test reject θ2 = 0, supporting inclusion of a regression structure for the precision parameter.The reported standard error for θ2 is 0.00361, and the asymptotic-normality test gives p-value 5.65 × 10^-5.
  • Bias correction: Corrected and uncorrected estimates for the mean parameters are very similar, but they differ considerably for the precision parameter φ.

8 Conclusions

The paper develops a general beta regression model with potentially nonlinear mean and precision structures, derives second-order bias corrections, and compares analytical and bootstrap estimators. Simulations favor the corrective and parametric bootstrap methods for reducing bias and mean squared error, with the parametric bootstrap performing best overall in the reported comparisons.

  • Conclusions: The model allows nonlinear regression structures for both the mean and precision parameters.
  • Conclusions: The derived O(n^-1) bias formulae for maximum likelihood estimators generalize the results of Ospina et al. (2006).
  • Conclusions: The paper defines analytical corrective and preventive estimators alongside parametric and nonparametric bootstrap bias-correction schemes.
  • Conclusions: Simulations favor the corrective and parametric bootstrap methods over the other methods for bias reduction and mean squared error.
  • Conclusions: The parametric bootstrap had the best overall performance in the reported simulation comparisons.

Appendix

The appendix supplies derivative, cumulant, matrix, and bias-expression details underlying the paper’s second-order bias calculations. It also records a minor correction concerning expressions in the appendix of Ospina et al. (2006).

  • Appendix: The appendix gives explicit expressions for cumulants and their derivatives used in the bias calculations.
  • Appendix: Expressions of the type in equation (8) of Ospina et al. (2006) are incorrect when a is set to φ, although the paper’s main-text formulae are correct.
  • Appendix: The appendix defines diagonal matrices and related matrix quantities needed to compute each term in the paper’s bias expression.
  • Appendix: The displayed matrix constructions distinguish blocks associated with β and θ and use previously defined matrices such as K, P, F, and G.
Loading 0809.1878v1…