PaperlayerView exact revision on arXiv

Source-linked AI summary

Model Averaging and its Use in Economics

Mark F. J. Steel

arXiv:1709.08221v3stat.APstat.COstat.ME

TL;DR

Model averaging addresses model uncertainty, which can otherwise produce overconfident or biased inference and predictions. The paper surveys Bayesian and frequentist methods, implementation strategies, and economic applications, emphasizing principled use and sensitivity to assumptions.

  • Problem

    Model uncertainty makes single-model inference and prediction potentially overconfident and biased, while economic analyses often involve very large model spaces.

  • Method

    The paper surveys Bayesian and frequentist model averaging, explains numerical implementation, and emphasizes sensitivity analysis across priors and sampling models.

  • Results

    The survey identifies predictive advantages of Bayesian model averaging and reviews applications across economics, including finance and forecasting.

  • Takeaways & Limitations

    Model averaging with appropriately chosen model spaces can provide compelling evidence when analyses encompass competing economic theories.

  • Takeaways & Limitations

    Model averaging remains sensitive to prior assumptions, and the review is not intended to be exhaustive.

Abstract

from arXiv · show

The method of model averaging has become an important tool to deal with model uncertainty, for example in situations where a large amount of different theories exist, as are common in economics. Model averaging is a natural and formal response to model uncertainty in a Bayesian framework, and most of the paper deals with Bayesian model averaging. The important role of the prior assumptions in these Bayesian procedures is highlighted. In addition, frequentist model averaging methods are also discussed. Numerical methods to implement these methods are explained, and I point the reader to some freely available computational resources. The main focus is on uncertainty regarding the choice of covariates in normal linear regression models, but the paper also covers other, more challenging, settings, with particular emphasis on sampling models commonly used in economics. Applications of model averaging in economics are reviewed and discussed in a wide range of areas, among which growth economics, production modelling, finance and forecasting macroeconomic quantities.

1 Introduction

The paper surveys model averaging as a formal response to model uncertainty, emphasizing Bayesian methods while also covering frequentist approaches and applications in economics. It connects methodological developments with practical implementation, prior sensitivity, computational tools, and empirical uses.

  • Its main statistical setting is uncertainty about covariate inclusion in normal linear regression, while extensions address dependence over time or space and endogeneity.
  • The survey reviews major methodological contributions in model averaging, with particular attention to developments relevant to economists.It presents a formal but accessible account of recent developments without claiming to be exhaustive.
  • Model averaging connects forecasting, econometrics, statistics, shrinkage, variable selection, and theoretical results on model uncertainty.
  • The paper discusses practical implementation, including numerical methods, prior sensitivity, freely available computational resources, and applications across economic fields.Applications include growth theory, production modelling, finance, and forecasting, while recommendations emphasize easily elicited and robust prior structures.
  • Covariate selection is treated through prior point masses at zero, allowing formal exclusion and averaging across many different models.This contrasts with shrinkage priors that retain all covariates while shrinking coefficients toward zero.
  • The paper focuses mainly on Bayesian model averaging because of its statistical properties, computational tractability, and interpretable posterior probabilities.BMA is described as easier to implement in large model spaces than frequentist model averaging because efficient MCMC algorithms are available.

2 Model uncertainty

Model uncertainty is pervasive in economic modelling because theories, empirical implementations, and assumptions can vary. Model averaging addresses this uncertainty by combining inference across candidate models, while requiring careful definition of the model space.

  • Economic modelling faces uncertainty about theories, empirical measures, functional forms, heterogeneity, and independence assumptions.
  • Model selection conditions inference on one chosen model, whereas model averaging combines all models in the considered space using Bayesian or frequentist weights.
  • Model averaging adjusts inferential precision for variation across models, whereas relying on a single model can produce biased or overconfident conclusions.
  • The choice between selection and averaging depends on the decision problem, with averaging suited to prediction and estimation of non-model-specific covariate effects.
  • Model uncertainty can materially alter policy estimates: estimated net lives saved per execution ranged from -63.6 to 20.9 across homicide-process assumptions.
  • The model space must be explicitly defined, potentially including uncertainty about both functional form and covariate inclusion; excluded models cannot contribute to averaged inference.
  • The survey excludes most nonparametric machine-learning approaches because they prioritize predictive flexibility and lack probability-based uncertainty intervals or structural interpretability.

3 Bayesian model averaging

Bayesian model averaging provides a formal Bayesian response to model uncertainty by averaging with posterior model probabilities and integrating out model-specific parameters for prediction.

  • BMA uses exact or approximate posterior model probabilities as averaging weights and integrates out model-specific parameters for prediction.

3.1 Prior Structures

The section examines how prior assumptions shape Bayesian model averaging, emphasizing prior elicitation, robustness, and the need for proper model-specific priors. It discusses common parameter and model priors, their computational implications, and differences between fully Bayesian and empirical-Bayes procedures.

  • Prior assessment: Prior quality can be assessed by whether it reflects user beliefs, satisfies objective-prior desiderata, and leaves inference robust across sensible choices.The paper notes that model-averaging results are often surprisingly sensitive to prior assumptions.
  • Prior assessment: Proper priors on model-specific parameters are required because arbitrary constants from improper priors can make posterior model probabilities undetermined.Such constants affect model marginal likelihoods and therefore posterior model comparisons.
  • Priors on model parameters: The main parameter prior specification combines Jeffreys-style priors for common parameters with g-priors for model-specific regression coefficients.The g-prior covariance depends on the design matrix because the analysis conditions on the regressors.
  • Priors over models: Random and fixed prior structures over model size differ in multiplicity adjustment: hierarchical priors down-weight medium-sized models, whereas fixed-w priors depend strongly on the chosen mean model size.The paper recommends sensitivity analysis and robust hierarchical prior structures because particular prior choices can materially affect results.
  • Empirical Bayes versus Hierarchical Priors: Comparisons between empirical-Bayes and fully Bayesian procedures find context-dependent differences, with Scott and Berger recommending fully Bayesian procedures overall.The cited applications include GDP-growth data, and empirical Bayes can assign probability one to the full or null model when either has the largest marginal likelihood.

3.2 Properties of BMA

BMA has consistency, predictive, calibration, and shrinkage properties under stated conditions, but these results often depend on assumptions such as model-space coverage and prior specification.

  • Consistency and paradoxes: Under suitable conditions, several priors deliver model-selection consistency, with posterior probability converging to one for the data-generating model.Consistency results depend on the prior for g, the model-space prior, and how the number of regressors grows with sample size.
  • Consistency and paradoxes: In high-dimensional regression, BMA contracts toward sparse data-generating models at an optimal rate when the prior sufficiently penalizes model complexity.The result is stated for linear regression and sparse true models.
  • Consistency and paradoxes: BMA can resolve the information paradox when suitable hyperpriors on g satisfy the required integrability condition.The Zellner–Siow, hyper-g, and benchmark beta priors are identified as solutions under stated conditions.
  • Predictive performance: BMA predicts at least as well as any single model under log predictive score, while empirical studies report improved predictive performance and coverage.The squared-error optimality result for point forecasts requires that the model space include the data-generating model.
  • Frequentist performance of point estimates: BMA point estimates minimize prior-weighted MSE, and intervals are calibrated in the sense that average coverage is at least their nominal posterior probability.These guarantees concern the Bayesian framework and do not ensure frequentist properties under a different data-generating distribution.

3.3 BMA in practice: Numerical methods

BMA computation is difficult because model spaces can be enormous, so practical implementations use model-space search, MCMC, or direct calculation when feasible.

  • Search methods: Occam’s window reduces computation by removing models dominated in posterior probability or containing more likely nested submodels.It was proposed for graphical models and extended to linear regression.
  • MCMC methods: MCMC methods address large model spaces by sampling promising models, commonly through the random-walk Metropolis sampler MC3.MC3 proposes neighboring models, while extensions add swap, add, and delete moves.
  • Computational challenge: With k = 100 potential covariates, exhaustive enumeration requires K = 2^100 = 1.26 × 10^30 models.This scale makes complete enumeration impractical even with fast processors.
  • Alternative strategies: Search-based methods can be biased by the search procedure, whereas posterior-sampling MCMC methods are potentially more precise.Both strategies remain useful, and some software implementations rely on search-based estimation.
  • Alternative strategies: For complicated models, direct marginal-likelihood computation is another option when the number of candidate models is manageable.MCMC proposals based on approximate marginal likelihoods may not always work well.

3.4 Role of the prior

Prior assumptions strongly affect posterior model probabilities, especially the complexity penalty. Hyperpriors on g and w flatten this penalty and increase the relative role of model fit.

  • Prior sensitivity: Prior choices affect posterior model probabilities more strongly than posterior inference conditional on a given model.This makes prior specification a central issue in BMA.
  • Penalty decomposition: The posterior-odds decomposition separates model-space complexity, marginal-likelihood complexity, and lack-of-fit penalties.For fixed g and w, the complexity penalty increases with g and decreases with w.
  • Hyperprior effects: Random w and g produce less extreme posterior-odds patterns than fixed hyperparameter values and penalize models near size k/2.The behavior reflects a multiplicity penalty that fixed w cannot induce.
  • Hyperprior effects: Integrating over g and w makes the model-size penalty flatter, reduces dependence on arbitrary prior assumptions, and increases the relative importance of model fit.These effects are illustrated using the posterior odds in Figure 3.

3.5 Data Robustness

BMA conclusions can be sensitive to data revisions, but shared samples, adaptive priors, and informative model-space restrictions can improve robustness.

  • Data revisions: Different versions of Penn World Table data produced substantial variation in posterior inclusion probabilities for some growth covariates.The comparison used versions 6.0–6.2 and found similar sensitivity across methods.
  • Scope and limitations: Inference from BMA remains challenging when complex model spaces are assessed using relatively few observations and data quality is uncertain.The paper notes that stronger prior information can mitigate changes caused by data modifications.
  • Sources of instability: Using the same countries roughly halves the changes across datasets, while fixing g at a large value amplifies the effect of small fit differences.A hyper-g prior dramatically reduces instability by allowing g to adjust to the data.
  • Robustness interventions: Restricting the model space to include initial GDP per capita and regional dummies enhances robustness across newer Penn World Table vintages.This restriction effectively imposes a more informative prior on the model space.

3.6 Collinearity and Jointness

The paper argues that marginal inclusion probabilities can be inadequate when covariates are collinear or jointly selected. It reviews jointness measures and related methods for summarizing dependence among regressors.

  • Jointness criteria: The paper proposes evaluating jointness measures by interpretability, calibration, extreme-jointness behavior, and universal definition.These criteria require meaningful scales, maximum values when variables always appear together, and definition whenever at least one variable has positive inclusion probability.
  • Jointness measures: Ley and Steel’s jointness measure is a posterior odds ratio comparing joint inclusion with inclusion of either regressor alone.For two variables, it compares the probability of including both with the probabilities of including exactly one.
  • Jointness measures: Doppelhofer and Weeks’ measure is the log posterior-odds ratio for including i when j is included versus when j is excluded.Ley and Steel note that their own measure is undefined when a variable is always included or excluded.
  • Comparisons: Empirical comparisons find substantial disagreement between major jointness measures, whereas results appear more robust across prior choices.Composite indicators have been suggested to combine information from different jointness concepts.
  • Collinearity: Strong collinearity can produce multimodal posterior distributions over models, making joint summaries more appropriate than marginal summaries.Ghosh and Ghattas recommend routinely calculating joint inclusion probabilities for correlated covariates.

3.7 Approximations and hybrids

The paper reviews approximations and hybrid approaches for model averaging when analytic marginal likelihoods are unavailable or when researchers combine Bayesian and frequentist reasoning. These methods include BIC- and AIC-based weights and averaging classical estimates.

  • Approximations: Complex Bayesian models often lack analytic marginal likelihoods, motivating approximations based on BIC or the Schwarz criterion.The normal linear regression model can have a closed-form marginal likelihood under the stated prior structure, unlike many more complex models.
  • Approximations: Smoothed AIC weights can approximate posterior model probabilities and provide an alternative to BIC-based weighting.The paper cites an application of smoothed AIC to pricing determinants of credit default swaps.
  • Approximations: BIC assumes a true model exists and can over-weight the best model, whereas AIC targets the model that best fits the unknown model.Hansen reports particularly poor BIC-weight performance relative to smoothed AIC weights when the sample size is large.
  • Hybrids: BACE and BAMLE illustrate hybrids that use asymptotic or maximum-likelihood ideas within model-averaging procedures.BACE uses Bayesian averaging of classical estimates, while BAMLE generalizes the approach to panel data.

3.8 Prior robustness: illusion or not?

Similar conclusions under selected prior settings can create an illusion of prior robustness. The paper emphasizes that prior choices remain consequential and recommends transparent, less subjective prior structures.

  • Figure 4: The left panel compares equal-posterior-probability contours for fixed w with the prior settings used by FLS and SDM.The plotted examples use n = 88, k = 67, k_i = 8, k_j = 7, and R2_j = 0.75.
  • Limitations: Similar outcomes for selected prior combinations do not establish general robustness, because the literature finds that prior choices can matter crucially.The paper therefore favors making prior assumptions explicit rather than treating implicit assumptions as irrelevant.
  • Robustification: Hyperpriors can reduce sensitivity to particular hyperparameter choices and further robustify the analysis.The paper notes that a hyperprior on g can be adopted and discusses a Beta(1, (k − m)/m) hyperprior on w.

3.9 Other sampling models

The paper extends Bayesian model averaging beyond normal linear regression to non-normal, flexible, dependent, outlier-prone, endogenous, panel, and spatial sampling models. These extensions often require new priors, approximations, or more demanding computation.

  • Generalized linear models: Generalized linear models extend the normal regression framework to non-normal responses, including binomial, Poisson, multinomial, and gamma outcomes.BMA for GLMs uses approximations such as Laplace-based Bayes factors and specialized prior structures.
  • Generalized linear models: GLM prior developments include locally orthogonal g-priors, wider hyperprior classes, and data-dependent structures whose results rely on approximations.Recommended hyperpriors include the hyper-g/n prior and the benchmark beta prior.
  • Generalized linear models: For probit models, posterior inference can use data augmentation, while model uncertainty typically requires MCMC over models and parameters.Because parameter dimension varies across models, reversible-jump Metropolis-Hastings methods are typically used.
  • Flexible models: Flexible additive models combine covariate uncertainty with alternative powers of regressors and require MCMC exploration of a large model space.Fractional polynomials are paired with a hyper-g prior in the cited additive-model approach.
  • Outliers and non-normality: Outlier-aware approaches jointly address variable selection and atypical observations, while Student-t models provide more robust inference against outliers and unmodelled heterogeneity.Some growth-data models additionally account for heteroscedastic measurement error, data vintages, and vintage fixed effects.
  • Dependent and endogenous data: BMA has also been applied to transformed-response models, time-series models, endogenous regressors, panel data, and spatial autoregressive panels.The spatial panel formulation incorporates spatial proximity through a normalized weight matrix and includes regional and time effects.

4 Frequentist model averaging

Frequentist model averaging constructs estimators by combining estimates from candidate models with weights chosen through criteria such as information criteria, Mallows’ criterion, or cross-validation. The section also reviews extensions, optimality results, computational trade-offs, and forecast-combination evidence.

  • Frequentist model averaging: Frequentist model averaging combines estimators from candidate models using weights in the unit simplex, making weight selection the central design choice.The weights may be chosen using information criteria, predictive-error criteria, cross-validation, or other frequentist objectives.
  • Information criteria: Information-criterion weights use AIC or BIC values, while modified AIC adds a small-sample correction recommended when n/k < 40.BIC imposes a stronger complexity penalty than AIC; the modified AIC recommendation is attributed to Burnham and Anderson.
  • Frequentist estimators: Mallows model averaging selects weights by minimizing Mallows’ criterion, which is asymptotically equivalent to squared predictive error in linear regression.The resulting estimator is described as easily implementable and asymptotically optimal under the stated criterion.
  • Frequentist estimators: Jackknife model averaging selects weights by leave-one-out cross-validation and is asymptotically optimal over linear estimators, including under heteroscedasticity and non-nested candidate models.The result is stated in terms of reaching the lowest possible squared errors over the class of linear estimators.
  • Extensions: Frequentist averaging has been extended to heteroscedastic linear models, generalized linear models, mixed-effects models, partial linear models, and other nonstandard settings.The reviewed methods use GLS, plug-in criteria, or other model-specific constructions and often establish asymptotic optimality.
  • Forecast combinations: Forecast combinations can outperform single models, but empirically data-adaptive weighting often performs worse than equal weighting in mean squared forecast error.This tension is known as the forecast combination puzzle.

5 Applications in Economics

Applications of model averaging in economics address empirical uncertainty across competing theories, covariates, instruments, and specifications. Reviewed studies cover growth, energy, public spending, natural resources, competition, and regional development, often producing more nuanced conclusions than limited model sets.

  • Overview: Model averaging has become a rapidly growing approach for economic applications where model uncertainty was previously handled informally or ignored.The review highlights applications that generate new empirical evidence across multiple economic problems.
  • Growth regressions: Growth applications use model averaging to examine competing theories, determinants, instruments, and variable representations.The growth literature is characterized by many proposed determinants and difficulty resolving the resulting open-ended empirical theory.
  • Institutions: All three main growth theories—geography, integration, and institutions—were found to play an important part in development after expanding the model and instrument space with 2SBMA.This contrasts with earlier conclusions based on more limited sets of models and instruments.
  • Institutions: Model averaging over alternative instruments and growth determinants produces more complete and nuanced inference and can downweight inappropriate instruments.The review notes that settler mortality receives little weight in one instrumental-variables equation, while instrument relevance can depend on the outcome specification.
  • Energy consumption: In U.S. data from 1949 to 2010 with up to 32 covariates, BMA identified energy consumption as an important determinant of aggregate GDP growth, without assessing possible endogeneity feedback.The study also examined energy intensity, energy efficiency, and the share of nuclear power.
  • Government spending and savings: For high-public-debt countries, BMA found public investment likely has a larger GDP impact than public consumption, while another study identified positive roles for government consumption and government GDP shares.The reviewed fiscal results concern developed economies during fiscal consolidation and depend on the specified prior structure.
  • Natural resources: BMA studies report a robust positive effect of oil rents on growth, with the interpretation varying according to whether interactions and heredity-prior restrictions are included.One specification attributes the effect mostly to institutional quality interacting with oil revenues; another finds the oil-rent main effect dominates.
  • Other applications: Other applications find no robust effect of fiscal federalism, positive effects of financial-market competition, and robust roles for several regional growth factors.The regional factors include initial income, human capital, infrastructure accessibility, and population age structure.

5.3 VAR and DSGE modelling

Model averaging is applied across VAR, DSGE, crisis, finance, and forecasting models to account for uncertainty about model structure and predictors. These applications identify heterogeneous determinants, changing predictor importance, and limits to crisis and exchange-rate forecasting.

  • VAR modelling: BMA has been applied to VAR models for forecasting inflation and output growth, modelling international spillovers, and analysing structural shocks.
  • DSGE modelling: BMA in DSGE models finds evidence of a structural break around 1984, after which technology shocks account for all stochastic trends.
  • Crises and finance: Early-warning studies find that crisis indicators are dimension-specific, with no single signal explaining all four dimensions of the 2008 crisis.
  • Crises and finance: Model-averaging studies link contagion to institutional similarity, distance, and political risk, while pre-crisis credit and real-activity growth are associated with lower crisis resilience.
  • Forecasting: BMA produces slightly better exchange-rate forecasts than a random-walk benchmark, while financial stress remains difficult to predict out of sample.
  • Finance: BMA applications identify robust financial determinants including trust for stock-market depth and liquidity, rule of law for financial development, and tail dependence for credit-default-swap pricing.

5.5 Production modelling

Production modelling faces substantial uncertainty about inefficiency distributions and covariate inclusion, which can materially affect firm-efficiency inference. Bayesian and frequentist model averaging extend across stochastic-frontier, productivity, trade, education, and other economic applications.

  • Production modelling: Uncertainty about one-sided inefficiency distributions can strongly affect firm-efficiency inference because cross-sectional data often provide only one observation per firm.
  • Production modelling: BMA averages inference on elasticities, returns to scale, and firm-specific efficiencies across competing stochastic-frontier distributions and specifications.
  • Production modelling: Frequentist model averaging addresses uncertainty over covariates in the frontier, symmetric-error variance, and inefficiency-distribution parameters.
  • Production modelling: The exponential inefficiency distribution tends to cluster firms near high efficiency because it places substantial mass near zero, unlike more flexible alternatives.
  • Other applications: Model averaging is used to study export diversification, corruption, spatial mobility, democracy, urban inequality, and resource-use efficiency across varied economic settings.
  • Forecasting: Dynamic model averaging for Chinese house prices finds significantly higher forecasting accuracy than other models, with predictor importance varying substantially over time.

6 Software and resources

The paper highlights freely available software for Bayesian and frequentist model averaging, spanning R, MATLAB, STATA, Gauss, Fortran, gretl, WinBUGS, and related implementations. Package comparisons indicate broadly similar inference for moderate model spaces, with speed and memory trade-offs.

  • Resources: Publicly available resources include early BMA code, dedicated software pages, and implementations for Bayesian and frequentist model averaging.
  • R packages: R packages cover BMA, endogenous regressors, dynamic model averaging, model confidence sets, and related model-selection tasks.
  • Package comparison: Across several R packages, real-data results are very similar and inference is generally reliable within 10 minutes for problems with up to about k = 100 covariates on a simple PC.
  • Package comparison: BAS is reported as fastest overall but has high memory requirements, whereas BayesVarSel is preferable when memory is restricted.
  • Package comparison: BAS also offers Bayesian Adaptive Sampling, which uses sampling without replacement but produces less accurate estimates than the main approach.
  • Other environments: Implementations also exist in MATLAB, STATA, Gauss, Fortran, gretl, WinBUGS, and MATLAB spatial-model code.

7 Conclusions

The conclusion presents model averaging as a formal response to pervasive economic model uncertainty and argues that ignoring alternative models produces excessive confidence. It emphasizes explicit prior sensitivity, practical implementation, and model averaging’s role in explaining divergent empirical findings.

  • Model uncertainty: Ignoring model uncertainty while retaining a single model can produce conclusions with excess confidence because alternative models are not incorporated.
  • Model uncertainty: Model selection is valid only under restrictive conditions, such as identifying a true model within the model space or overwhelming dominance by one model.
  • Approaches: Model averaging spans Bayesian and frequentist approaches, with BMA attractive for its finite-sample interpretation and posterior inclusion probabilities.
  • Prior sensitivity: BMA results depend on prior assumptions, so researchers should make priors explicit and investigate which prior features drive sensitivity.
  • Applied practice: Sensitivity analysis across priors and sampling models is indispensable for mapping assumptions to empirical results and supporting applied conclusions.
  • Interpretation: Model averaging can unify empirical findings by revealing how differences in model spaces, BMA priors, or FMA weights generate divergent results.
Loading 1709.08221v3…