PaperlayerView exact revision on arXiv

Source-linked AI summary

Group recovery after trimming, and level-free flagging, in robust clusterwise regression

Samir Orujov

arXiv:2609.31019v2stat.COstat.MEstat.ML

TL;DR

Robust clusterwise regression requires a trimming level that is often unknown, and an overly generous level can discard genuine groups. The paper proposes group recovery and level-free ESF, then compares them with fixed-level trimming. Up to 20% outliers, TCLUST-REG at level 0.30 with reweighting and recovery matched ESF’s mean accuracy; higher levels were more accurate beyond 25%.

  • Problem

    Trimming methods require an often-unknown fraction to discard: too low can leave outliers that break the fit, while too high can discard genuine units or a small group.

    Source: §1 · p. 2

  • Method

    The paper adds a post-fit search that can recover a group among discarded units and ESF, which flags units using exact clusterwise least-squares fits on successive small subsamples without setting a trimming level.

    Source: §1 · p. 3

  • Results

    Up to 20% outliers, TCLUST-REG at level 0.30 with reweighting and recovery had mean accuracy 0.914 against ESF’s 0.915 over 89 cells; beyond 25%, levels 0.35 and 0.40 beat ESF.

    Source: §1 · p. 3, §6 · p. 22

  • Takeaways & Limitations

    When contamination is unknown, ESF with recovery needs no trimming level and is accurate up to about a fifth of outliers; if contamination may exceed a quarter, the paper recommends levels 0.35 or 0.40 with recovery unless outliers may be uniformly spread.

    Source: §6 · p. 23

    The flagged fraction estimates contamination only under Gaussian errors, and the stated large-sample result excludes small groups under moderate contamination and repeated or refitted applications.

    Source: §3 · p. 10

Abstract

from arXiv · show

Trimming methods for robust clusterwise regression discard a fixed fraction of the data. Too low a level breaks the fit; too generous a level can trim away a small group. We first propose a group-recovery step that can follow any trimming or flagging method: it searches the discarded units for a line, tests whether those near it form a peak rather than a band, and restores the line as a group when the likelihood of Gaussian groups plus uniform noise improves by a margin like that of the Bayesian information criterion. In simulations it repaired the failures of a generous TCLUST-REG level with unequal groups, but not with three or four groups. The second proposal, ESF (exact-subsample flagging), is a flagging procedure without a trimming level: it solves the clusterwise least-squares problem exactly on small subsamples, flags units far from the best fit, and draws later subsamples from the rest. Two constants stand in for the level: a subsample size, set from a lower bound on the smallest group proportion, and a cap on the flagged set. It is meant for data about whose contamination nothing is known: TCLUST-REG at the fixed level 0.30, followed by reweighting and the recovery step, was as accurate as ESF on average up to a fifth of outliers, and higher levels were more accurate beyond. The flagged fraction estimates the contamination only when the errors are close to Gaussian. On taxi fares with known tariffs, ESF found both in every sample.

1 Introduction

The paper addresses the unknown trimming level in robust clusterwise regression, where too little trimming leaves outliers and too much can discard genuine groups. It proposes post-fit group recovery and level-free exact-subsample flagging, and compares their performance with fixed-level trimming methods.

1 Introduction

  • The trimming level is consequential: too little leaves enough outliers to break the fit, while too much can discard genuine observations or an entire small group.This motivates methods that avoid choosing a fixed trimming fraction or check what a fit has discarded.Source: §1 · p. 2
  • Trimming-level alternatives include reweighting, scanning levels or group counts, group-specific trimming, bounded losses, and mixture models, while multiple-model fitting uses a user-set noise threshold.The cited approaches do not ask whether discarded units form a missed group.Source: §1 · p. 2
  • The group-recovery step searches discarded or flagged units for a concentrated line and retains it when Gaussian groups plus uniform noise yield a preferred likelihood.It can follow different trimming or flagging methods and is intended to recover small genuine groups lost with outliers.Source: §1 · pp. 2–4
  • ESF fits clusterwise least-squares models exactly on small random subsamples, flags units far from the best fit, and draws later subsamples only from units not yet flagged.Its subsample size depends on a lower bound for the smallest group proportion, while recovery is not attempted when the flagged set exceeds n/3.Source: §1 · p. 3
  • In the simulation study, TCLUST-REG at fixed level 0.30 with reweighting and recovery matched ESF’s mean accuracy with up to 20% outliers; levels 0.35 and 0.40 beat ESF beyond 25%.No cell fell below 0.85 for the fixed level, compared with two for ESF.Source: §1 · p. 3
  • The paper also compares exact solvers and an exact-fit ablation, derives partial results for ESF and recovery, and evaluates methods in 7,750 simulated data sets.Its applications include fishery, tone-perception, and taxi-fare data; taxi trips have recorded tariffs and known groups.Source: §1 · p. 4
  • No method is best everywhere, and the recovery and ESF constants were held fixed across the experiments.Source: §1 · p. 4

2 ESF and the recovery step

ESF builds robust clusterwise-regression fits from exactly solved small subsamples, flags units poorly explained by the fit, and can be paired with a recovery step that searches flagged units for a missed group. Its subsample size and flagged-set cap replace a trimming level, but exact fitting limits it to small subsamples and the recovery step has stated scope limits.

2.1 Exact fits on small subsamples

  • Exact subsample fits use depth-first branch and bound over label vectors, pruning partial labelings whose residual sum of squares exceeds the best value found.A replicate is admissible only if each of its K groups has at least d + 1 units, requiring mπmin ≳ d + 1.Source: §2.1 · pp. 4–5

2.2 Cost of the exact fit

  • For m ≤20, the depth-first search was 30 to 180 times faster than Gurobi, but the programme overtook it from about m = 40 with two groups.Replacing exact fits with one hard-alternation start lost up to 0.11 with four groups, while elemental fits lost up to 0.25.Source: §2.2 · p. 5

2.3 Sequential flagging

  • ESF draws admissible replicates in stages, scores them with bounded loss on unflagged units, and flags units far from the best-scoring replicate, recomputing flags at each stage.The final labels use a weighted plurality of replicate labels, followed by concentration and reweighting steps.Source: §2.3 · pp. 5–6
  • A unit is flagged for a large nearest-line residual relative to its group scale or for robustly distant covariates; the residual scale is 1.4826 times the median absolute residual.Covariate distance is tested against the 0.975 quantile of χ2_p for sufficiently large groups.Source: §2.3 · p. 5

2.4 Is the flagged set a group?

  • The recovery step searches flagged units for a line forming a peak rather than a band, then compares fits by the likelihood of Gaussian lines plus uniform noise.It is not attempted when at least a third of units are flagged; its peak test can still accept a sufficiently narrow uniform-noise band.Source: §2.4 · pp. 7–8

2.5 Tuning constants and defaults

  • ESF’s subsample size is set near (d + 1)/πmin using a lower bound on the smallest group proportion, while a cap of n/3 guards recovery when the fit may have broken down.The cut-off c = 2.5 gives a clean-unit flagging probability of about 0.012 at the true parameter; the flagged fraction is not an estimate of contamination when errors are not close to Gaussian.Source: §2.5 · p. 9

3 Theoretical results

The theoretical results give partial guarantees for one stage of ESF, one application of reweighting, and the population operations of group recovery. They do not establish how ESF progresses from one stage to the next.

3 Theoretical results

  • A stage containing B/L admissible subsamples has a clean replicate with probability 1 − (1 − πℓ)^(B/L), where each replicate is clean with probability πℓ = pℓaℓ.The exact result is conditional on the data and all draws before that stage; it does not cover whether earlier stages improve later clean-subsample chances.Source: §3 · pp. 9–10
  • At a fixed input θ = θ0, the reweighting map’s flagged fraction converges to ε + (1 − ε)κ, with κ within η of Σk πk 2Φ(−csk/σk).The refitted coefficients converge to a contamination-dependent limit within O(η) of θ0k when covariates are bounded; without overlap or contamination, θ0 is a fixed point and the flagged fraction tends to 2Φ(−c).Source: §3 · p. 10
  • For a fixed-parameter recovery candidate, adding a separated Gaussian group can improve population log likelihood by its peak-density gain less a noise-mass loss.For the stated example R/σ∗ = 40, π∗ = 0.1, π0 = 0.2, and ξ ≈ 0, the gain is at least 0.09n against a margin of 8.6 when d = 2 and n = 300; the result does not cover the candidate’s reweighting refit.Source: §3 · p. 11
  • When two lines through one group are separated by δ, replacing them with the group’s line has population log-likelihood change bounded by (φ(1)a² + 1/4a⁴)πS, where a = δ/(2σS).The peak test fails when a band of outliers fills its window and passes a line of half-width below about 3.4 scales.Source: §3 · p. 11

4 Simulation study

Across simulations, performance depended on contamination level and group structure: fixed-level TCLUST-REG with reweighting and recovery performed strongly at lower contamination, while higher levels were more robust beyond 25%. ESF and the recovery step helped in particular settings, but small groups, clustered or high-leverage outliers, and departures from Gaussian errors exposed limits.

4.1 Design

  • The simulations used eight designs with n = 300, varied group structure, noise, and outlier types, and evaluated clean-unit classification accuracy alongside other measures.Methods received the true number of groups; clean units were assigned to their nearest fitted line, and failed fits counted as zero accuracy.Source: §4.1 · pp. 11–12

4.2 Up to 20% of outliers

  • 0.897 was the highest mean accuracy across all 123 cells, for TCLUST-REG at level 0.40 with reweighting and recovery, compared with 0.888 for ESF.The level 0.35 was closest to serving both contamination ranges, with mean accuracy 0.912 against ESF's 0.914 up to 20% of outliers.Source: Table 2 · p. 14, §4.2 · p. 15

4.3 Beyond 25% of outliers

  • At ε = 0.30 and 0.35, TCLUST-REG at level 0.25 fell to accuracies of 0.53–0.62 in four designs, while ESF remained above 0.87.At higher contamination, level 0.40 with reweighting and recovery was ahead of ESF by more than 0.03 in 13 of 34 cells.Source: §4.3 · pp. 15–16, Table 3 · p. 16

4.4 Covariate screen, other measures, further checks and cost

  • The covariate screen protected ESF against high-leverage outliers, while its flagged fraction tracked contamination closely only when ESF worked.Where ESF worked, the mean flagged fraction was within about three percentage points of ε; additional checks found no alternative pipeline that outperformed the leading methods, and ESF took a median 0.08 seconds per data set.Source: §4.4 · p. 17

4.5 Sample size, groups, covariates and outliers

  • With groups of 80% and 20% or 85% and 15%, recovery raised ESF accuracy up to 20% contamination, but at 30% it rarely applied and accuracy remained 0.75.Heavy-tailed, skewed, and heteroscedastic errors left accuracy near the best competitor but caused the flagged fraction to count error tails or spread as well as outliers.Source: §4.5 · p. 18

4.6 The recovery step: confirmation, sensitivity and repeated runs

  • On new small-group data, recovery raised accuracy by 0.09–0.28 in all 12 cells, although ESF remained 0.08–0.09 below TCLUST-REG in two cells.Recovery was sensitive to the response range set by extreme units, and ESF's results varied across random seeds; recovery after TCLUST-REG could also hurt with dense, narrow uniform-noise bands.Source: §4.6 · pp. 18–19

5 Three data sets

Three real-data examples test robust clusterwise regression on fishery prices, tone perception, and taxi tariffs, illustrating how recovery and flagging outcomes depend on the data and method settings.

5.1 The fishery data

  • In 19 of 20 seeds, recovery replaced the fishery fit's line at intercept 2.57 with a cheap-flow line at 2.16 and flagged 3% of flows.TCLUST-REG's answer depended on trimming level; with reweighting and recovery, level 0.25 returned the cheap group and dearer line from all ten starts.Source: §5.1 · p. 20

5.2 The tone perception data

  • With ten outliers added, ESF returned the same two tone-perception lines within 0.01 for every seed, while several alternatives lost a line.The recovery step found no group among the flagged trials.Source: §5.2 · pp. 20–21

5.3 Taxi fares from an airport

  • ESF found both taxi tariffs in all 20 samples, reaching the shared best-method accuracy of 0.978; the recovery step changed none of its fits.The recovery pipeline also reached 0.975–0.979 accuracy, with coefficient errors of 0.10 and 0.14 versus ESF's 0.23.Source: §5.3 · p. 21

6 Discussion

The discussion compares ESF with trimming and recovery, gives method-selection guidance, and identifies limits involving contamination, group structure, and error distributions.

6 Discussion

  • Recovery reduced cells with mean accuracy below 0.85 from 31 of 89 to four after generous TCLUST-REG trimming, but the four remaining cells had three or four groups.The recovery step also worked after the trimmed cluster-weighted model.Source: §6 · p. 22
  • At fixed K, recovery tests one candidate at a time and can accept narrow bands of uniform noise as groups after generous trimming at 30% or 35% outliers.When K is in doubt, the discussion points instead to scanning across the number of groups.Source: §6 · p. 22
  • The flagged fraction estimates contamination only under near-Gaussian errors; on tone and taxi data it exceeded contamination despite correct fitted lines.For Gaussian-error simulations where ESF worked, it lay within about three percentage points of the contamination fraction.Source: §6 · p. 22
  • Up to 20% outliers, mean accuracy over 89 cells was 0.914 for ESF versus 0.915 for TCLUST-REG at level 0.30 with reweighting and recovery; with four groups, accuracy was 0.95 versus 0.88.The level-0.30 pipeline had no cell below 0.85 where ESF had two; the four-group comparison favors ESF because trimming can discard parts of several groups at once.Source: §6 · p. 22
  • When contamination is known below a fifth, level 0.25 with equal weights, reweighting, and recovery was most accurate; for unknown contamination, ESF needs no level and is accurate up to about a fifth.For possible contamination above a quarter, the discussion recommends levels 0.35 or 0.40 with reweighting and recovery, unless outliers may be uniformly spread.Source: §6 · pp. 22–23
  • ESF breaks down when clean subsamples become rare; in these designs this occurred from 25% contamination with uniform noise or high-leverage points and from 30% with three groups.Small groups were recovered only up to about 20% outliers, and not on every data set at that level; tight-cloud outliers were fitted as part of a group from 20% onward.Source: §6 · p. 23
  • K was treated as known; BIC selected the true K for at least 90% of two- or four-group data sets up to 20% outliers, but only 50%–80% for three overlapping groups.BIC also counted a tight cloud of outliers as a group.Source: §6 · p. 23

Declarations

The declarations report no specific funding or competing interests, identify data and code sources, and disclose the use of a language model during preparation.

Funding.

Competing interests.

Data availability.

  • Fishery and tone data are distributed with FSDA and mixtools; the taxi extract and raw simulation output are in the named repository and Zenodo archive.Source: Data availability · pp. 23–24

Code availability.

  • The code, table- and figure-producing scripts, and analysis plans are available on GitHub and archived at Zenodo.Source: Code availability · p. 24

Use of large language models.

  • Claude assisted with code, simulations, and drafting; the author reviewed and edited its contributions and takes responsibility for the article.Source: Use of large language models · p. 24
Loading 2609.31019v2…