Source-linked AI summary
Econometric applications of high-breakdown robust regression techniques
Asad Zaman, Peter J. Rousseeuw, Mehmet Orhan
TL;DR
Robust regression is rarely used in applied econometrics despite the prevalence of outliers and misconceptions about robustness. The paper presents high-breakdown procedures and applies them to published regression models, finding that a few observations can substantially alter results.
Problem
Robust regression is rarely used in applied econometrics despite common outliers and misconceptions about large samples, outlier detection, and robust methods.
Source: §1 · p. 1Method
Results
Deleting a few outlying points substantially affects regression results; fit often improves and regressor significance changes, with implications for theory.
Source: §6 · p. 7Takeaways & Limitations
The authors recommend applying high-breakdown techniques, even in simple regression, to avoid misleading effects of bad leverage points.
Source: §6 · p. 7Outlier treatment requires understanding why an observation arose and may require subject-matter knowledge beyond the paper’s mechanical analysis.
Source: §2.1 · p. 4
Abstract
from arXiv · showhide
A literature search shows that robust regression techniques are rarely used in applied econometrics. We list several misconceptions about robustness which lead to this situation. We show that most data sets are not normal, least squares performs very poorly even in large data sets with small numbers of outliers, and that commonly used techniques for achieving robustness fail to do so. We then provide newly developed techniques from the statistics literature which are easy to understand, and achieve robustness. We show the practical use of these techniques by re-analyzing three regression models from recent literature, and arriving at different conclusions from those reached by the authors.
1. Introduction
Robust regression is rarely used in applied econometrics despite the prevalence of outliers, and the paper argues that misconceptions about large samples, outlier detection, and what counts as robust help explain this gap. It challenges those assumptions and introduces robust procedures intended to support more routine use.
1. Introduction
- Practitioners face misconceptions about large samples, visual or residual-based outlier detection, choosing among robust methods, interpreting results, and the gains robust analysis can offer.The authors aim to reduce these barriers and facilitate more routine use of selected robust regression procedures.Source: §1 · p. 1
1.1. OLS and large samples
- OLS has a 0% breakdown value, so even a small share of deviant observations can systematically distort estimates in large samples.The paper cites earnings-function and stock-return examples in which a few or 1% of extreme observations alter findings.Source: §1.1 · p. 2
1.2. Removing outliers
- OLS residuals and one-at-a-time sensitivity checks can miss outliers, especially when outliers mask one another; visual detection is also difficult in high dimensions.The paper concludes that OLS output supplemented by sensitivity analysis may still provide no signal of distortionary outliers.Source: §1.2 · p. 2
1.3. What is robust?
- The paper defines robustness as resisting changes from modifying or removing a small percentage of observations, a property many commonly labelled robust methods lack.It notes that regression M-estimators and their generalizations have low or zero breakdown, and that the Krasker–Welsch estimator can change substantially with a small share of deviant observations.Source: §1.3 · p. 2
2. Robust regression via LTS
The paper presents least trimmed squares (LTS) as a high-breakdown regression approach and explains how it can help identify influential observations. It also cautions that outlier treatment requires judgment and knowledge of how the data were generated.
2. Robust regression via LTS
- LTS orders squared residuals and estimates coefficients by minimizing the sum after trimming a specified percentage of the observations.The paper presents LTS among methods designed to remain reasonable in the presence of many bad observations.Source: §2 · p. 3
- LTS has high breakdown, but its objective is highly nonlinear and non-differentiable with multiple local maxima, making global optimization challenging.The paper points to efficient optimization techniques developed for this problem.Source: §2 · p. 3
- Robust regression can flag outliers and help identify bad leverage points, but removing too many observations risks changing the relationship being estimated.The paper describes bad leverage points as high-leverage observations with large LTS residuals and recommends robust estimates of predictor center and covariance for leverage assessment.Source: §2 · p. 3
2.1. Interpretation and value of robust analysis
- Outlier analysis is not mechanical: measurement errors may warrant removal, while valid extremes may signal behavior outside the model and require subject-matter investigation.The paper describes its mechanical analysis as input to deeper analysis that requires knowledge of the data-gathering and generating processes.Source: §2.1 · p. 4
3. Growth study of De Long and Summers
In the 61-country growth regression, LTS identifies Zambia as an outlier, and excluding it changes the OLS inference on non-equipment investment and improves model fit.
3. Growth study of De Long and Summers
- The analysis focuses on the longest period, 1960–1985, and includes all 61 countries in the original study’s main regression.Source: §3 · pp. 4–5
- Removing Zambia, identified by LTS as an outlier, makes non-equipment investment and the constant significant and substantially improves fit relative to the original OLS analysis.The comparison is reported in Table 1.Source: Table 1 · p. 5, §3 · p. 5
- Zambia’s standardized LTS residual is −5.41, which the authors identify as evidence that it is a very significant outlier.Source: §3 · p. 5
4. Solow model applied to the OECD countries
In the 22-country Solow-model example, robust screening identifies three bad leverage points; excluding them improves fit and changes the inference on population growth.
4. Solow model applied to the OECD countries
- The example uses 22 countries and one arbitrarily selected version of the original paper’s more general equation.Source: §4 · p. 5
- Excluding Canada, Turkey, and New Zealand improves fit and changes population growth, ln(N), from borderline significant in the original equation to insignificant.The regression results are reported in Table 2.Source: Table 2 · p. 6, §4 · p. 6
- LTS residual screening flags five countries, but Canada, Turkey, and New Zealand are classified as bad leverage points and removed; Japan and Norway are retained.Source: §4 · pp. 5–6
5. Benderly and Zwick’s return data
For Benderly and Zwick’s 1954–1981 stock-return regression, LTS identifies two moderate outliers, and excluding them changes inflation’s significance in the robust analysis.
5. Benderly and Zwick’s return data
- The regression explains annual US common-stock returns from 1954 to 1981 using output growth and inflation.Source: §5 · p. 6
- LTS identifies 1979 and 1980 as mild leverage points with standardized residuals of 2.60 and 2.82, while OLS reveals no outliers among the 28 years.Their robust distances are 3.65 and 3.55, respectively.Source: §5 · p. 6
- After deleting two moderate outliers, inflation becomes significant in the robust analysis, whereas the original OLS results suggest it is not significant.The paper also reports substantial improvement in fit; regression results are displayed in Table 3.Source: §5 · p. 6, Table 3 · p. 7
6. Concluding remarks
Across applications to recent economic regressions, high-breakdown methods reveal observations that can materially alter fits and regressor significance; the authors judge these benefits worth the added computation.
6. Concluding remarks
- Deleting a few outlying points often improves the fit and changes regressor significance, with implications for theory.The authors report this pattern across several linear regression models from recent economic literature.Source: §6 · p. 7
- Classifying observations as regular, good or bad leverage points, and vertical outliers, provides diagnostic insight into how observations affect OLS fits.Good leverage points support precise fits, while bad leverage points pull OLS in the wrong direction; removing one vertical outlier from Zambia’s data drastically changed De Long and Summers’ results.Source: §6 · p. 7
- High-breakdown techniques take somewhat more computation time than OLS, but the authors judge their benefits to outweigh that cost.Source: §6 · p. 7