A budget-dependent crossover between coverage- and response-based training-set selection for machine-learned interatomic potentials

1 Introduction

• At a fixed training-set size, the central question is whether to retain diverse structures or configurations where a model struggles most.The paper frames this as compression of an existing labelled pool, where redundant examples are discarded.Source: §1 · p. 1
• Coverage methods select for structural or information-theoretic diversity, while active-learning approaches use uncertainty or extrapolation to target model weakness.The value of disagreement can be tested by training on selected configurations and evaluating their predictions.Source: §1 · pp. 1–2
• A single-budget comparison can miss shifts in selector performance, motivating learning curves and direct DFT errors across retained-data fractions.Coverage and model disagreement address different needs, and their balance can change as the subset grows.Source: §1 · p. 2
• The study compares Random, validation-selected FPS/DPP coverage, and response-guided selection at 1%, 5%, and 20% on GAP-20 Carbon and pooled rMD17.The response witness prioritises disagreement between a coverage-trained model and a full-data reference; each selected subset trains a separate MACE model.Source: §1 · p. 2
• Coverage better reproduces the full-data outcome at 1% and 5%, but the response witness is closer at 20% and lowers direct DFT force error in all eight dataset–endpoint comparisons.The paper also examines endpoint-specific trade-offs and complementary selection-score analyses.Source: §1 · p. 2
• The authors frame retained-data size and selection rule as a joint decision, judged by the predictive accuracy of the resulting model.Their analyses connect selection scores to the information represented and to outcomes after retraining.Source: §1 · p. 2

2 Methods

• The protocol links selection criteria to force errors after training, varying selector and retained-data budget on common Carbon and rMD17 panels with paired MACE seeds.Descriptor analyses test learned similarity and HEA energy prediction; foundation-error and response selections are evaluated through subset-model retraining.Source: §2 · p. 2

2.1 Atomistic datasets, splits and retained-data budgets

• The fixed panels contain 1,400 structures each, with 1,000 training, 200 validation, and 200 test structures; budgets of 10, 50, and 200 are labeled 1%, 5%, and 20%.Carbon is stratified across five physical groups, while pooled rMD17 includes 140 frames from each of ten molecules with temporally blocked splits.Source: §2.1 · p. 2

2.2 Coverage selectors and diagnostic scores

• Validation selects the primary FPS/DPP coverage comparator at each budget; learned-similarity, HEA-residual, frozen-model-error, and response-witness analyses test other selection signals.Random is the unstructured reference, and QUESTS-MSC provides an additional coverage comparison.Source: §2.2 · pp. 3–4

2.3 MACE models and force-error endpoints

• Each selected subset and the full-data reference train separate MACE models under the same architecture, loss, optimiser, and epoch-based schedule.Performance is measured against both the matched full-data model and DFT labels across four force endpoints; the shared schedule does not fix gradient-update counts across budgets.Source: §2.3 · p. 4

2.4 Foundation-error and response-based selections

• The response witness ranks candidates using disagreement between coverage and full-data models, while foundation-model selections use frozen force error or its residual after novelty and nuisance adjustment.Response coordinates distinguish energy shifts, distributed force mismatch, and largest local force mismatch; a primary repair setting replaces 5% of the coverage subset with candidates from occurred-but-not-learned cells.Source: §2.4 · p. 5

2.5 Uncertainty estimation and paired comparisons

• Primary selector contrasts use paired training-seed intervals, while grouped bootstrap intervals quantify variation across Carbon groups, rMD17 molecules, or HEA groups.The paired intervals measure training-seed variability conditional on the selected subsets.Source: §2.5 · p. 5

2.6 Manuscript and analysis-code preparation

• Generative-AI tools assisted language revision, methods exposition, numerical consistency checks, and analysis and plotting-code preparation.The manuscript points to supplementary documentation and acknowledgments for details.Source: §2.6 · p. 5

3.1 Selector ordering changes across retained-data budgets

• Witness absolute gaps exceed FPS/DPP at 1% and 5% across all four endpoints in both datasets, but are lower at 20%; the sampled reversal lies between 5% and 20%.At 20%, witness-minus-FPS/DPP absolute-gap differences ranged from −0.46 to −4.78 percentage points, with all paired-seed 95% intervals below zero.Source: Figure 2 · p. 6, §3.1 · p. 6
• At 5%, FPS/DPP’s Mean absolute gap was 1.80% in Carbon and 1.68% in rMD17, versus Random’s 2.10% and 1.92%, with the advantage across all endpoints.All eight paired training-seed intervals for Random minus FPS/DPP were above zero.Source: §3.1 · p. 6, Table 1 · p. 7
• At 20%, response-witness lowers held-out force errors by 0.46–5.89% versus FPS/DPP across all eight comparisons; six endpoints also fall below full-data error.All four rMD17 endpoints and the Carbon High-force tail and High-novelty mask were below the matched full-data reference.Source: §3.1 · p. 6, Table 2 · p. 7

3.2 Response emphasis creates endpoint-specific trade-offs at 5%

• At 5%, rMD17 High-response cell error fell 7.87%, while its other three reported errors rose 13.20–58.52%; Carbon errors rose 6.33–59.45%.All four Carbon grouped-bootstrap intervals excluded zero.Source: §3.2 · p. 7
• At 5%, repair replaced 3/50 Carbon structures and left rMD17 unchanged, with no detectable error reduction; at 1%, its absolute-gap effects varied across endpoints.At 1%, repair reduced all four rMD17 mean gaps by 0.77–2.66 percentage points, while Carbon’s tail and novelty gaps fell and its Mean and Validation-selected gaps rose.Source: §3.2 · p. 7

3.3 Frozen foundation-model error largely tracks embedding novelty

• Frozen MACE-MP-0 force error closely tracked embedding novelty; residualizing novelty did not improve direct error selection, and embedding coverage had the best Carbon-defect force-vector MAE.Novelty correlations were 0.86 for rMD17, 0.89 for amorphous Carbon and 0.91 for Carbon defects.Source: §3.3 · pp. 7–8

3.4 Learned similarity preserves the coverage ranking

• Learned similarity rankings remained close to Gaussian QUESTS, with 0.980–0.990 Spearman correlations and 0.667–0.754 selected-set Jaccard overlap at 5%.Geometry and coverage accounted for 0.821–0.989 of fitted weight, while the residual channel received no more than 0.030; learned-minus-Gaussian ΔAUROC intervals included zero across Carbon, rMD17 and HEA.Source: §3.4 · p. 8

4.1 Data budget is part of selector performance

• At 20%, the response witness has smaller full-data gaps in all eight dataset–endpoint comparisons and lowers direct held-out DFT force errors across all eight comparisons.All eight paired-seed comparisons favor the witness over FPS/DPP, and six errors fall below the full-data reference.Source: §4.1 · p. 8, §4.2 · p. 9
• At 1% and 5%, coverage has smaller absolute gaps to the full-data outcome than the response witness; at 5%, FPS/DPP is closer than Random across all eight force endpoints.The crossover is located between the sampled 5% and 20% budgets; intermediate learning-curve points would show whether it is gradual or sharp.Source: §4.1 · p. 8

4.3 Selection scores and training value are different quantities

• Frozen-model force error tracks embedding novelty, and error-based selection yields higher Carbon-defect force-vector MAE than embedding coverage.Figure 4 summarizes novelty correlations and explained error variance alongside held-out Carbon-defect MAE for five training seeds.Source: Figure 4 · p. 9
• Learned similarity preserves a ranking close to coverage, while learned-minus-Gaussian leave-one-group-out ∆AUROC intervals include zero for all three reported comparisons.Figure 5 summarizes overlap and ranking, learned channel contributions, and ∆AUROC uncertainty.Source: Figure 5 · p. 9

4.2 Response emphasis exposes endpoint trade-offs

• At 5%, response selection lowers rMD17 High-response cell error but raises its Metadata-score mask, High-force tail, and High-novelty mask errors; every reported Carbon response endpoint worsens.A gain in one force regime can therefore accompany losses elsewhere.Source: §4.2 · pp. 8–9
• Response repair removes Carbon’s 1.00% uncovered high-response atom mass without measurable improvement, while the rMD17 repair subset is unchanged because coverage already includes every eligible high-response cell.The reported Carbon outcome intervals include zero.Source: §4.2 · p. 9

4.3 Selection scores and training value are different quantities

• In the auxiliary HEA25 regression, G4 descriptors reduce normalized out-of-fold squared error to 0.240 of the G0 value, with a grouped-bootstrap interval of [0.222,0.278].This regression tests descriptor representation of the energy target; ranking, overlap, and matched-budget retraining address different questions about selection value.Source: §4.3 · p. 10

4.4 Implications for atomistic data selection

• Selector comparisons should use matched budgets, direct force errors and full-data deviations, with multiple endpoints and uncertainty measures suited to the comparison.Paired training-seed intervals measure optimization variability; grouped resampling measures variation across specified scientific groups. The relevant choice is the rule and endpoint at the intended retained-data budget.Source: §4.4 · p. 10

• At 20%, witness-selected models reduce direct held-out force errors by 0.46–5.89% relative to FPS/DPP, with all eight paired-seed intervals favoring the witness.Six errors are below the full-data reference.Source: §5 · p. 10
• The response witness has larger absolute gaps than coverage at 1% and 5%, but smaller gaps at 20%.At 5%, FPS/DPP is closer to the full-data outcome than Random across all eight force endpoints.Source: §5 · p. 10
• Selection scores should be connected to retrained-model predictions across budgets and judged against the force accuracy required of the final model.Learned similarity preserves the coverage ranking, while frozen-model-error selection increases Carbon-defect force MAE relative to embedding coverage.Source: §5 · p. 10

• The authors report no conflicts of interest.Source: Conflicts of interest · p. 10

• GAP-20 Carbon, revised MD17, and HEA25 are publicly available, and the pretrained embedding and error-estimation model is MACE-MP-0.Source: Data availability · p. 10
• The Zenodo archive includes aggregate budget-curve values, seed-resolved primary comparisons, manuscript source files, and a data dictionary.Source: Data availability · p. 10
• A portable analysis entry point regenerates paired statistics and quantitative figures from CSV tables without training models.The archive also includes a MACE configuration for shared compression-training settings.Source: Data availability · p. 10

Notes and references

• The bibliography spans foundational interatomic potentials and atomistic representations, later graph architectures, and datasets used in machine-learned interatomic-potential research.The supplementary information also identifies the Carbon and revised MD17 dataset records.Source: Notes and references · pp. 11–12
• The listed literature also covers structural selection, active learning, uncertainty estimation, and data-pruning studies.The bibliography includes works on coverage and batch selection as well as active-learning workflows and data pruning.Source: Notes and references · p. 12

SUPPLEMENTARY INFORMATION

• The supplementary information provides extended methods, diagnostic controls, five-seed outcome summaries, budget-dependent learning curves, and a constructed methodological positive control.Its source-data tables are archived at the cited Zenodo record.Source: SUPPLEMENTARY INFORMATION · p. 13

1.1 Datasets, splits and budgets

• Both analysis panels contain 1,000 training, 200 validation, and 200 held-out test structures; selectors draw from an 800-structure candidate pool disjoint from response calibration.Carbon samples across five physical groups, while pooled rMD17 includes ten molecules with temporally blocked sampling.Source: §1.1 · p. 13

1.2 MACE outcome comparisons

• Each selected subset and the 1,000-structure reference is trained with the shared MACE protocol and compared on shared validation and test partitions.Outcome seeds are paired with their full-data references; the common epoch ceiling does not equalize gradient-update budgets across subset sizes.Source: §1.2 · pp. 13–14, Table S0 · p. 14

1.3 Coverage baselines and candidate signals

• Random, validation-selected FPS/DPP coverage, QUESTS-MSC, and candidate signals including learned similarity, frozen-model error, and response selection provide the comparisons.FPS/DPP is selected by validation force MAE; candidate analyses also include HEA energy residual and foundation-model fine-tuning.Source: §1.3 · p. 15

1.4 Definitions and procedures

• Held-out force-vector errors are evaluated across fixed atom groups, including the High-force tail, validation-selected stratum, and novelty and metadata masks.The validation-selected stratum is frozen before test evaluation, and novelty thresholds are fitted on calibration data.Source: §1.4 · p. 15
• The learned metric combines standardised geometry/coverage, species/composition, and residual channels; grouped evaluation excludes held-out groups from fitting.The HEA descriptor ladder tests prediction of a non-spin DFT energy target, from basic composition and structure features through frozen-model representations.Source: §1.4 · p. 16
• The response witness ranks candidates using disagreement between a coverage-trained model and a full-data reference, with response cells fitted on calibration data.Selected subsets are retrained with paired seeds for held-out testing; because construction uses a full-data model, the witness is a retrospective compression diagnostic, not a pre-labelling acquisition rule.Source: §1.4 · pp. 16–17

2.1 Static physical proxy diagnostics

• Learned chemical/physical similarity closely preserves Gaussian QUESTS ranking: rank correlations are 0.990 for Carbon, 0.990 for rMD17 full, and 0.980 for HEA.Top-5% Jaccard overlaps are 0.754, 0.724, and 0.667, respectively; learned-minus-Gaussian AUROC intervals include zero for every dataset.Source: §2.1 · p. 17, Figure S1 · p. 18, Table S1 · p. 18
• The G4 descriptor reaches 0.240 of the G0 normalised out-of-fold squared error for the non-spin HEA energy target, with a grouped-bootstrap interval of [0.222, 0.278].The descriptor ladder's error decreases at each step; this tests descriptor adequacy rather than selected-subset value.Source: §2.1 · p. 17, Table S2 · p. 18

2.2 Foundation-model error analysis

• Embedding novelty explains 0.75, 0.80, and 0.84 of frozen-model error variance in rMD17 full, amorphous Carbon, and Carbon defects; error-based Carbon-defect selections exceed embedding coverage error.Residual-error and raw-error selections are higher than embedding coverage by 36.39 and 92.84 meV Å−1, respectively.Source: §2.2 · p. 19, Figure S2 · p. 19

2.3 Response witness and repair

• At 5%, Carbon response witness raises all four risks by 6.33% to 59.45%, while repair intervals include zero; rMD17 repair makes no replacement.Coverage already spans every high-response cell in rMD17 at this budget.Source: §2.3 · p. 19, Figure S3 · p. 20

2.4 Outcome-gap evaluation

• At 5%, FPS/DPP has smaller gaps than Random on all four endpoints in both datasets, while response witness has larger gaps than coverage.The paired-seed bootstrap intervals for Random minus FPS/DPP are above zero in all eight comparisons.Source: §2.4 · p. 19, Figure S4 · p. 20
• Response-witness gaps are largest at 1% but fall below FPS/DPP at 20% across all four endpoints in both datasets.The 1% rMD17 baseline comparison is mixed across endpoints, while the 20% reversal is supported by paired-seed intervals.Source: §2.4 · p. 21
• At 20%, response witness has smaller absolute gaps and lower direct DFT risk than FPS/DPP across both datasets and all four endpoints.The reported paired-seed intervals support this reversal.Source: §2.4 · p. 21

2.5 Confidence-interval precision and ranking stability

• At 5%, Carbon witness intervals indicate higher risk at all four endpoints; Carbon repair intervals include zero, while rMD17 repair matches FPS/DPP.Positive values in the response summary indicate higher risk.Source: §2.5 · p. 21, Table S4 · p. 22
• All eight Random-minus-FPS/DPP absolute-gap intervals are above zero, and the intervals quantify variation across five matched training seeds.The 5% contrast uses paired training-seed intervals.Source: §2.5 · p. 22
• The top-budget margin measures score separation between the structures at the coverage boundary; its stability interpretation assumes ideal score calibration and excludes ties.The paper treats this margin as a descriptive diagnostic alongside rank correlation, overlap and group discrimination.Source: §2.5 · p. 22
• Learned-metric rankings remain close to coverage overall, despite membership changes near the budget boundary, particularly for HEA at small budgets.Leave-one-group-out AUROC differences remain centred near zero.Source: §2.5 · p. 22

2.6 Learned-metric positive-control capacity test

• With an injected independent signal, the learned metric shifts its weight to that channel (0.878), lowers coverage rank correlation to 0.226, and raises evaluation AUROC from 0.482 to 0.797.The evaluation pool is separate from the metric-fit pool.Source: §2.6 · p. 22
• The injected-signal control enriches the selected subset for the high-value group by 4.67×, showing that the metric can reorder selections in this constructed setting.The physical datasets separately test behavior with measured energy, force and structural attributes.Source: §2.6 · p. 24

2.8 Data and code map

• The archive links aggregate and seed-resolved source tables to analysis scripts and figure renderers; reproduce.sh recalculates paired contrasts and regenerates figures without model training.Derived files are kept separate from recorded source tables.Source: §2.8 · p. 24

2.9 Generative-AI assistance during preparation

• AI-assisted tools supported language editing, proofreading and quality control, but did not generate or modify primary data or perform the reported numerical analyses.The authors state they verified the output and take responsibility for the submitted work.Source: §2.9 · p. 24

2.7 Uncertainty available for each outcome contrast

• The 5% coverage contrast and 20% response crossover have paired-seed intervals, while the 1% comparison reports five-seed mean ordering.Grouped intervals resample scientific groups with paired seeds; paired-seed intervals resample five matched training seeds.Source: §2.7 · p. 24, Table S10 · p. 28
• Table S11 reports matched-seed contrasts for the 5% coverage comparison and 20% response crossover, including signed witness gaps from full-data DFT risk.Its bootstrap intervals enumerate all 55 paired-seed resamples; paired-t intervals use four degrees of freedom.Source: Table S11 · p. 29

Notes and references

• References [2] and [4] identify the Carbon GAP 20 and revised MD17 datasets, respectively.Source: Notes and references · p. 31
• References [7] and [8] cite MACE and SOAP, while [3] cites work on machine learning and interatomic potentials.Source: Notes and references · p. 31
• The list also includes Drautz’s 2019 work [9] and a 2025 Journal of Chemical Physics paper [10].Source: Notes and references · p. 31