PaperlayerView exact revision on arXiv

Source-linked AI summary

INCONVENIENT: A Benchmark to Assess LLM Generalization Capabilities from Convenience Samples

Sara Rezaeimanesh, Chad Klochko, Kundan Thind, Mohammad Ghassemi

arXiv:2610.21038v1cs.LGcs.CL

TL;DR

Convenience sampling can distort the feature–target relationships learned from training data, while many tabular shift benchmarks partition both training and evaluation across domains. INCONVENIENT directly perturbs training relationships while keeping population evaluation fixed, and reports substantial performance losses across tasks.

  • Problem

    Convenience-based samples may distort feature–target relationships, while existing tabular shift benchmarks often change both training and evaluation distributions rather than isolating biased training against a fixed population.

    Source: §1 · pp. 1–2

  • Method

    INCONVENIENT creates training samples with controlled coefficient amplification, attenuation, or sign reversal and evaluates models on the original population distribution.

    Source: §1 · p. 2, §3.1 · p. 3, §3.2.3 · p. 5

  • Results

    Across 200 datasets and around 15,500 retained shifts, seven standard baselines lose 20%–45% of performance on average with deployment distribution held fixed.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Because biased samples contain no signal identifying the bias, closing the gap requires information from outside the training data, a capability INCONVENIENT is designed to measure.

    Source: §6 · p. 10

    The setting assumes no information about the specific shift and no access to unlabeled data from the population during training.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Machine learning models for tabular prediction typically assume that training data are representative of the population on which they will be deployed. In practice, however, data are often collected based on convenience, availability, or cost, resulting in systematic sampling biases that can distort feature--target relationships and undermine real-world generalization. Existing tabular distribution-shift benchmarks primarily construct out-of-distribution (OOD) settings by partitioning observations across naturally occurring domains. While such partitions may induce changes in P(Y | X), the resulting shifts are neither explicitly controlled nor isolated, and the evaluation distributions themselves are often not representative of the broader target population. We introduce INCONVENIENT, a benchmark designed to study robustness to convenience-sample bias. Rather than relying on naturally occurring domain partitions, INCONVENIENT directly perturbs feature--target relationships in the training distribution while preserving the original population distribution as the deployment environment. This setting more closely reflects real-world scenarios in which models are developed from unrepresentative samples and subsequently deployed on a broader population. The distinction is particularly relevant for large language models (LLMs), whose pretraining may encode population-level knowledge that extends beyond the relationships observed in a biased training dataset. Unlike existing distribution-shift benchmarks, INCONVENIENT provides explicit control over both the direction and severity of relationship shifts, enabling a systematic assessment of robustness to convenience sampling. The benchmark comprises 200 tabular classification and regression datasets spanning diverse application domains and generates around 15,500 shifted training distributions which lead to substantial downstream degradation.

1 INTRODUCTION

Convenience-based data collection can make training samples unrepresentative, so their feature–target relationships may differ from those in the deployment population. INCONVENIENT creates controlled relationship shifts in training while evaluating on the original population.

1 INTRODUCTION

  • Convenience sampling can distort training relationships because data are collected for availability, accessibility, or cost rather than population representativeness.Examples include single-hospital medical data, platform-specific consumer data, and scientific samples drawn from easy-to-recruit populations.Source: §1 · p. 1
  • Existing tabular shift benchmarks partition naturally occurring domains, changing both training and evaluation distributions and leaving fixed-population deployment with biased training samples less directly addressed.They also do not explicitly isolate shifts in the feature–target relationship.Source: §1 · p. 2
  • INCONVENIENT perturbs feature–target relationships in unrepresentative training samples while retaining the original population as the test distribution.It controls shift direction and severity through coefficient amplification, attenuation, and sign reversal.Source: §1 · p. 2
  • The benchmark contains 200 tabular prediction datasets and around 15,500 shifted training distributions, with convenience-sample training causing substantial downstream degradation.The supplied passage does not quantify that degradation.Source: §1 · p. 2

2 RELATED WORK

Prior tabular shift benchmarks largely evaluate naturally occurring domain differences, while INCONVENIENT targets biased training samples with population-level evaluation. This setting also tests whether pretrained knowledge can help when observed relationships contradict population relationships.

2 RELATED WORK

  • Selection depending on X alone can be corrected by importance weighting, whereas selection depending on Y or the X–Y relationship is not identifiable from the biased sample.The passage attributes these learning-theoretic results to Zadrozny (2004) and Cortes et al. (2008).Source: §2 · p. 2
  • WILDS, TableShift, and WhyShift evaluate naturally occurring domain partitions; WhyShift decomposes observed shifts and finds concept shift dominates many real tabular tasks.These approaches change training and evaluation distributions, unlike INCONVENIENT’s fixed-population deployment setting.Source: §2 · p. 2
  • TabFSBench synthetically adds or removes feature columns, but does not target feature–target relationships directly.INCONVENIENT instead constructs controlled coefficient changes with known direction and severity.Source: §2 · pp. 2–3
  • Prior work suggests LLMs encode knowledge useful in low-data tabular prediction, including approaches that elicit explicit priors over model coefficients.Examples include TabLLM (Hegselmann et al., 2023), AutoElicit (Capstick et al., 2025), and LoID (Rezaeimanesh et al., 2026).Source: §2 · p. 3
  • INCONVENIENT weakens, amplifies, or reverses training relationships while preserving the original population for evaluation, isolating settings where recovery requires prior knowledge beyond the observed sample.The benchmark targets the gap between limited-data settings and data that actively contradict population relationships.Source: §2 · p. 3

3 BENCHMARK CREATION

INCONVENIENT constructs convenience samples by selecting examples that produce specified coefficient changes, then evaluates against the original population. It varies the shift operation and severity across semantically meaningful tabular datasets.

3.1 PROBLEM STATEMENT

  • The test distribution is the population, while convenience and population samples have different joint distributions; the setup assumes no shift-specific information or unlabeled population data during training.This models practitioners training on available convenience samples without knowing the precise sampling bias.Source: §3.1 · p. 3

3.2 DATASET SELECTION

  • The benchmark restricts datasets to semantically meaningful feature representations, excluding image and audio data whose dimensions generally lack interpretable meaning for methods using feature names or task descriptions.The collection comprises 200 tabular datasets from OpenML, Kaggle, and TALENT, spanning classification and regression tasks.Source: §3.2 · p. 4

3.2.1 REFERENCE MODEL

  • A reference linear model is fit on the training pool to obtain baseline coefficients: unregularized logistic regression for classification and ordinary least squares for regression.Each dataset is split into a 50% training pool and 50% held-out test set.Source: §3.2.1 · p. 4

3.2.2 COEFFICIENT MANIPULATION OPERATIONS

  • Three coefficient operations create distinct relationship shifts: amplification strengthens target coefficients, attenuation weakens them, and sign flipping reverses their direction.Amplification uses α > 1 and tests factors from 1.15 to 5.0; attenuation uses 0 < γ < 1 and tests factors from 0.2 to 0.9.Source: §3.2.2 · p. 4

3.2.3 GREEDY SUBSET SELECTION ALGORITHM

  • A greedy algorithm scores examples by their influence on targeted coefficients, incrementally adds high-scoring examples, refits a model, and stops when the target criterion is met or the pool is exhausted.If the shift criterion remains unmet after verification, the generated training set is excluded; selection can also alter covariate, label, and concept distributions.Source: §3.2.3 · p. 5

3.2.4 SHIFT SEVERITY CONTROL

  • Shift severity can be varied through minimum subset size, the importance of targeted features, and coefficient magnitude for amplification or attenuation.Feature importance is estimated using absolute feature–target correlation; the paper expects more informative features and larger coefficient changes to have greater downstream impact.Source: §3.2.4 · p. 5

4 EXPERIMENTAL SETUP

The evaluation compares standard tabular methods trained on representative and convenience samples, using held-out population tests. It also measures collateral coefficient changes and tests whether an LLM can identify perturbed features from domain knowledge.

4.1 EVALUATION

  • Convenience training variants are compared with in-distribution training on a test set, using AUC-ROC for classification and R² for regression; variants with insufficient degradation or out-of-range performance are removed.The procedure also removes coefficient-manipulation sets whose degradation is subsumed by previously constructed shifts.Source: §4.1 · p. 6

4.2 MEASURE OF COLLATERAL EFFECT

  • Collateral effect is measured as relative L2 change in non-target coefficients, comparing logistic or ridge coefficients fitted on population and convenience training sets.The measure expresses non-target coefficient change as a fraction of its original magnitude, making it scale-invariant.Source: §4.2 · p. 6

4.3 BASELINES

  • The benchmark evaluates seven methods across model families, tuning them with TPE and a budget of 20 evaluations per method–dataset combination.Hyperparameter selection uses the convenience training set’s 80/20 train-validation split; models are retrained on combined training and validation data before held-out population testing.Source: §4.1 · pp. 5–6, §4.3 · pp. 6–7

4.4 LLM-BASED COEFFICIENT ANOMALY DETECTION

  • Claude-Haiku-4.5 is given OOD coefficients to rank suspicious features, and its rankings are compared with actual coefficient-change rankings using Recall@K.The experiment measures overlap between the model’s top-K suspicions and the true top-K most-changed features.Source: §4.4 · p. 7

5 RESULTS

The results assess whether controlled convenience-sample shifts degrade tabular learners, propagate beyond targeted features, and yield predictable difficulty controls; they also test LLM coefficient-anomaly detection.

5.1 CONVENIENCE SAMPLES DEGRADE EVERY BASELINE

  • Ridge and MLP suffer among the largest degradations despite weak in-distribution performance, and internally consistent biased training and validation data do not reveal reversed or erased relationships.Ridge and MLP fall to under 10% R2; correction must come from outside the data when no unlabeled population data are available.Source: §5.1 · pp. 7, 9
  • 0.18–0.24 AUC on classification (20–29% relative drop) and 0.35–0.45 R2 on regression (53–82% relative drop) accompany convenience-sample training across 6,489 and 8,970 shifts, respectively.Figure 2 places baselines below the in-distribution versus out-of-distribution diagonal; Figure 3 reports averages across shifts.Source: §5.1 · p. 7, Figure 3 · p. 8, Figure 2 · p. 8

5.2 COLLATERAL EFFECTS CONFIRM THE SHIFTS ARE STRUCTURAL

  • 26%, 61%, and 55% median relative changes for regression (amplify, attenuate, flip-sign), versus 109%, 107%, and 104% for classification, accompany correlation-structure propagation.The perturbation propagates through the feature-space correlation structure rather than remaining local to the targeted feature.Source: §5.2 · p. 9

5.3 THE SEVERITY KNOBS CONTROL DIFFICULTY

  • Only minimum fraction and feature-importance bin behave as reliable difficulty controls; amplification and attenuation magnitude show inconsistent trends.At αmin = 0.1, degradation reaches 0.09–0.23 R2 for regression and 0.08–0.12 AUC for classification. Top-bin perturbations cost 0.18–0.29 R2 and 0.07–0.11 AUC, while bottom-bin perturbations cost less than 0.07 in regression and 0.04 in classification; amplification/attenuation trends vary across tasks.Source: §5.3 · p. 9

5.4 LLM-BASED COEFFICIENT ANOMALY DETECTION

  • Across 300 randomly selected shifts, the LLM achieves 37.4% overall Recall@5, with performance varying by coefficient operation.Recall@5 is 42.7% for amplified coefficients, 36.7% for sign flips, and 32.7% for attenuations, versus 16.8% for random selection.Source: §5.4 · p. 9

6 CONCLUSION

INCONVENIENT evaluates models trained on non-representative samples against the general population; across 200 datasets and about 15,500 shifts, seven baselines lose 20%–45% of performance on average. The paper also reports its AI-use practices and provides tools and materials for reproducing and extending the evaluations.

6 CONCLUSION

  • Across 200 datasets and approximately 15,500 retained shifts, seven standard baselines lose 20%–45% of performance on average when the deployment distribution is held fixed.The benchmark assesses models trained on non-representative samples and tested on the general population.Source: §6 · pp. 9–10

AI USE STATEMENT

  • Claude-Haiku-4.5 supported an LLM-based coefficient anomaly detection experiment, while Claude-Sonnet-4.5 and ChatGPT-5 assisted with code, tables, and manuscript editing.The authors state that they reviewed AI-assisted work and take responsibility for the final content and analyses.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • A command-line interface generates and evaluates shifts, downloads the collection of approximately 15,500 shifts, and supports reproducing Table 1 and testing other models.A usage guide and documentation are included in a README; the repository provides datasets and full results, while the supplementary material includes scripts.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 INAPPLICABLE METHODS

The appendix explains why common robustness methods may not fit convenience sampling: the setting withholds information those methods use, while some methods cannot recover the population relationship from the biased sample. It motivates evaluating methods that can draw on pretrained knowledge of feature meanings.

A.1 INAPPLICABLE METHODS

  • Domain-generalization methods require domain labels and at least two training domains, whereas INCONVENIENT supplies a single convenience sample with no domain structure.The setting models a practitioner working from one hospital, platform, or recruitment pipeline.Source: §A.1 · p. 13
  • Partitioning the sample into synthetic environments does not reveal the population relationship: global selection makes the distorted relationship stable across partitions, so invariance-seeking may preserve it.The authors state that invariance is informative only when some environment exhibits the population relationship, which none does here.Source: §A.1 · p. 13
  • Label-shift methods target changes in P(y) and require target-label proportions or unlabeled target data, while correcting P(y) alone leaves the manipulated P(y | x) untouched.Label shift occurs as a by-product of influence-based selection, not as the mechanism of the shift.Source: §A.1 · p. 13
  • Selection depending on Y or the X–Y relationship cannot be corrected by reweighting the biased sample, according to the cited sample-selection literature (Cortes et al., 2008).The population conditional is not identifiable from the sample, so no weighting of convenience examples restores the removed relationship.Source: §A.1 · p. 13
  • Convenience-sample validation favors the smallest DRO uncertainty radius, collapsing the method toward ERM; the authors leave DRO for a future benchmark version.The validation split shares the training sample’s bias, and the authors say an empirical demonstration of this failure remains future work.Source: §A.1 · p. 13
  • Several tabular neural architectures are excluded on cost and expected informativeness, not principle; the authors expect architectural variation to shift the gap marginally rather than close it.They argue that these architectures lack information about feature meanings and identify the MLP and TabPFN as bracketing the included data-driven family.Source: §A.1 · p. 13
  • Nearly the entire robustness toolkit is inapplicable or insufficient when it requires information removed by convenience sampling, motivating evaluation of methods with pretrained knowledge of feature meanings.The baseline suite is narrower than TableShift.Source: §A.1 · pp. 12–14

A.2 DATASETS

This section introduces the dataset inventory used in INCONVENIENT.

A.2 DATASETS

  • The section states that it names all datasets used in INCONVENIENT.Source: §A.2 · p. 14

A.2.1 REGRESSION (80 DATASETS)

The regression dataset inventory lists datasets from OpenML, Kaggle, and TALENT.

A.2.1 REGRESSION (80 DATASETS)

  • OpenML contributes 72 datasets to the listed regression inventory.Source: §A.2.1 · p. 14
  • Kaggle contributes 1 dataset to the listed regression inventory.Source: §A.2.1 · p. 14
  • TALENT contributes 7 datasets to the listed regression inventory.Source: §A.2.1 · p. 14

A.2.3 MULTICLASS CLASSIFICATION (81 DATASETS)

The appendix lists sources for the multiclass-classification datasets and describes Table 1 as reporting mean performance across shift configurations.

A.2.3 MULTICLASS CLASSIFICATION (81 DATASETS)

  • OpenML supplies 78 datasets listed for the multiclass-classification collection.The dataset list continues across two passages.Source: §A.2.3 · pp. 14–15
  • Kaggle supplies two datasets: Insurance Csv and Hotel Guests Dataset.Source: §A.2.3 · p. 15

A.3 DETAILED PERFORMANCE NUMBERS

  • Table 1 reports mean performance across datasets for configurations varying by task type, baseline method, operation, feature-importance bin, and minimum fraction.Source: Table 1 · p. 15
Loading 2610.21038v1…