PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Materials Properties from Scarce Labels and Unlabeled Crystals

Wentao Li, Yizhe Chen, Jiangjie Qiu, Yijun Li, Leyi Zhao, Xiaonan Wang

arXiv:2608.30682v1cs.LGcs.AI

TL;DR

Learning materials properties from scarce labels and unlabeled crystals requires using uncertain unlabeled evidence without allowing misleading pseudo-labels to dominate. SemiMat provides a controlled benchmark and MatRank a reliability-weighted, ranking-aware objective; across the retained blocks, the fixed objective achieves the strongest aggregate benchmark results, with evidence boundaries remaining.

  • Problem

    Materials property learning must use abundant unlabeled crystal structures despite scarce, uneven labels and the risk that uncertain continuous pseudo-labels become misleading supervision.

  • Method

    SemiMat controls inputs, backbones, splits, checkpoint selection, held-out reporting, and metrics, while MatRank combines anchor pseudo-targets, reliability weighting, weak–strong consistency, and ranking signals.

  • Results

    Across the retained 24 backbone-task blocks, one fixed MatRank objective gives the lowest aggregate held-out test NMAE (0.896) and best average method rank (2.208).

  • Takeaways & Limitations

    The evidence supports MatRank as a strong semi-supervised regression baseline for this benchmark, with reliability-aware values and candidate ordering relevant to materials screening.

  • Takeaways & Limitations

    The generated-pool study tests one MP20pretrained MatterGen source, and MatRank cannot certify stability, synthesizability, or intended chemical-domain membership.

Abstract

from arXiv · show

Learning materials properties from scarce labels and unlabeled crystals is a central challenge for data-driven materials discovery. We present SemiMat, a controlled benchmark for semi-supervised materials property regression, and MatRank, a reliability-weighted objective for continuous pseudo-label uncertainty. SemiMat fixes labeled and unlabeled crystal inputs, graph-backbone interfaces, validation-only checkpoint selection, held-out test reporting, normalized MAE (NMAE), and method-rank summaries across six scarce-label tasks, four graph backbones, and five predefined split runs. MatRank builds pseudo-targets from labeled anchors, weights them by local reliability and weak-prediction agreement, trains weak and strong graph views consistently, and adds ranking signals so that unlabeled crystals shape both values and candidate order. Across the retained 24 backbone-task blocks, one fixed MatRank objective gives the lowest aggregate held-out test NMAE (0.896) and best average method rank (2.208). The component, OOD, and generated-pool diagnostics identify where the gain is reliable and where further screening evaluation remains necessary. Code is available at https://github.com/littlepeachs/SemiMat.

Introduction

SemiMat and MatRank address the challenge of learning continuous materials properties from scarce labels and unlabeled crystals. The benchmark controls evaluation, while MatRank uses reliability-aware pseudo-labeling and ranking signals to guide learning from unlabeled structures.

  • Scarce experimental and high-fidelity labels make it necessary to use abundant unlabeled crystal structures without turning uncertain predictions into misleading supervision.
  • Continuous materials regression is difficult because task scales differ, numerically sharp pseudo-labels can still be wrong, and unlabeled pools may help one task while harming another.
  • Controlled comparisons are needed because apparent semi-supervised gains can depend on properties, backbones, splits, unlabeled pools, seeds, and aggregation rules.
  • SemiMat fixes inputs, graph-backbone interfaces, checkpoint selection, held-out reporting, metrics, and split structure across six tasks, four backbones, and five predefined runs.
  • MatRank combines labeled-anchor pseudo-targets, local reliability, weak-prediction agreement, weak–strong consistency, and ranking signals so unlabeled structures shape values and candidate order.

Related Work

SemiMat situates semi-supervised crystal property prediction within broader materials machine learning and semi-supervised learning. It treats unlabeled structures as part of the protocol while using ranking to complement continuous regression.

  • Uneven label coverage leaves experimental, stability-related, tensorial, and high-fidelity materials targets smaller than the pool of candidate structures.
  • SemiMat treats unlabeled structures as part of the learning protocol rather than unused background data.
  • Crystal graph models provide the backbone layer, encoding atomistic geometry through differing message-passing, convolutional, and geometric inductive biases.
  • Semi-supervised learning commonly uses pseudo-labeling, consistency regularization, teacher–student targets, and augmentation-based label guessing.
  • Unlabeled data can hurt when the protocol or pool is mismatched, and continuous pseudo-labels make confidence calibration harder than in classification.
  • MatRank therefore adds pairwise learning-to-rank supervision as a complement to pointwise regression.

Problem Setup

SemiMat formalizes scarce-label regression with labeled graphs, scalar targets, and an unlabeled candidate pool. It evaluates methods using original-scale MAE, block-normalized NMAE, and within-block method rank without using test labels for training or selection.

  • The setup contains labeled materials graphs with scalar properties and an unlabeled candidate pool available during training.
  • SemiMat fixes the task, split, backbone, seed, and unlabeled pool before comparing algorithms.
  • Test labels are excluded from training, checkpoint selection, and hyperparameter tuning when assessing whether unlabeled structures improve regression.
  • Reported MAE is computed after predictions are returned to the original property scale.
  • NMAE normalizes each method’s five-split mean MAE by the supervised mean MAE within the same backbone-task block because raw MAE scales differ across datasets.
  • Lower NMAE is better, supervised training equals 1.0 in every block, and method rank summarizes local MAE ordering within each backbone-task block.

SemiMat Framework

SemiMat fixes the data, model, training, and evaluation protocol for controlled semi-supervised materials regression. MatRank uses reliability-weighted pseudo-targets, weak–strong consistency, and ranking signals so unlabeled crystals influence predictions and candidate order.

  • SemiMat Framework: SemiMat fixes matched labeled and unlabeled structures, graph interfaces, backbones, algorithms, seeds, splits, and reporting metrics for each benchmark block.The same validation split selects checkpoints, while held-out test data are reserved for final reporting.
  • SemiMat Framework: MatRank applies one unchanged objective across datasets, backbones, and indexed splits rather than selecting methods per dataset or backbone.This design exposes aggregate behavior alongside local exceptions when tasks differ in label noise, target scale, and geometric complexity.
  • SemiMat Framework: MatRank combines labeled-anchor pseudo-targets, reliability-weighted weak–strong consistency, cross-set and labeled-batch ranking, and an auxiliary ranking-classifier head.The regression branch uses both absolute consistency and relative order, while the ARC weight increases when batch-level anchor reliability is low.
  • SemiMat Framework: Anchor reliability reflects local neighborhood sharpness and agreement with the weak-view regressor, reducing anchor influence when either condition is unreliable.High-reliability anchors move strong predictions toward nearby labeled values; low-reliability anchors rely more on weak-prediction consistency.
  • SemiMat Framework: The fixed loss combines supervised learning with ARC, consistency, cross-set, weak-view, and other ranking-related terms.Lcons is reliability-weighted unlabeled regression consistency; Lcross compares labeled examples with unlabeled stable targets; Lweak and Lsmooth regularize weak-view predictions; Llabel preserves labeled-batch order.
  • SemiMat Framework: Ranking remains a training signal while held-out test tables evaluate regression MAE, because candidate order can remain informative when continuous pseudo-labels are uncertain.The component ablation removes anchor, cross-rank, feature-smoothness, and labeled-rank terms from the same DimeNet++ implementation.

Experiments

Experiments cover six scalar materials-property tasks spanning electronic, energetic, mechanical, and response-property regimes.

  • Experiments: The benchmark evaluates 2D band gap, piezoelectric tensor, exfoliation energy, elastic anisotropy, experimental formation enthalpy, and experimental band gap.The task tables were converted from processed Matminer task tables, with primary-source and processed-table provenance reported in the supplement.

Results

Across the controlled SemiMat evaluation, MatRank improves aggregate held-out regression while diagnostics distinguish broad gains from task-, split-, and unlabeled-pool-dependent cases. The results support a fixed objective whose reliability weighting and ranking signals remain useful across backbones, OOD splits, and a generated unlabeled pool.

  • Main benchmark: MatRank achieves the lowest average NMAE, 0.896, and best average method rank, 2.208, across 24 backbone-task blocks.It also has the lowest backbone-level NMAE for SchNet, DimeNet++, ET, and M3GNet.
  • Main benchmark: MatRank leads many 2D gap, piezoelectric, and elastic-anisotropy blocks, while Mean Teacher and RDA remain strongest on selected formation-enthalpy, experimental-gap, and exfoliation-energy cases.These local exceptions motivate reporting the complete task-backbone matrix rather than only aggregate scores.
  • Stability: Figure 2 separates stable gains from boundary cases by plotting relative gain over supervised training against five-split variability.Lower-right blocks improve over supervised training while remaining below median variability; near-zero-gain blocks are treated as boundary cases.
  • Ablation: The full MatRank objective has the best average NMAE on DimeNet++, although several ablations remain competitive on individual tasks.Removing anchor pseudo-labels, cross-set ranking, feature smoothness, or labeled-rank preservation can improve a local task but weakens aggregate behavior.
  • OOD evaluation: MatRank improves all six tasks under both element-level and label-level OOD splits, with larger reductions for element-level shifts and smaller but consistent reductions for label-tail shifts.The label-level setting is a harder stress because unlabeled targets near the held-out high-label tail are more difficult to estimate.
  • Generated-pool evaluation: A MatterGen-generated unlabeled pool produces aggregate performance similar to MP-5k and is slightly better on 2D gap, formation enthalpy, and experimental gap.Because effect sizes are small, the result supports pool tolerance rather than universal superiority of generated pools.

Discussion

SemiMat and MatRank provide a controlled, auditable framework for reliability-aware semi-supervised materials regression, with evidence that remains bounded by task, backbone, split, and unlabeled-source variation.

  • Benchmark scope: The benchmark keeps task, backbone, split, unlabeled source, checkpoint selection, and aggregation choices explicit so local exceptions remain visible.Its reporting unit separates raw held-out test MAE from aggregate NMAE and method-rank summaries.
  • Main finding: The evidence supports one fixed MatRank objective as best on aggregate held-out test NMAE and method rank across 24 retained backbone-task blocks.The claim is based on main comparison, ablation, OOD, and generated-pool diagnostics rather than a single table.
  • Mechanism: MatRank combines labeled-anchor reliability, weak–strong consistency, and order-aware losses without selecting a different objective for each dataset or backbone.The design treats unlabeled crystals as structured but uncertain evidence and uses relative order when absolute values are less reliable.
  • Evidence quality: Held-out, validation-selected reporting, matched baseline views, and shared DimeNet++ diagnostics reduce the risk that aggregate gains are reporting artifacts.The diagnostics test component effects, distribution shift, and an alternative generated unlabeled source.
  • Robustness boundaries: OOD results strengthen the benchmark claim beyond one in-distribution table, while generated-pool results show source substitution without algorithm changes.Neither diagnostic establishes invariance to every shift, universal superiority of generated crystals, or a deployment guarantee.
  • Limitations: The current scope covers scalar regression tasks and one MP20pretrained MatterGen source, not vector, tensor, distributional, or all crystal-generation settings.MatRank also cannot certify stability, synthesizability, or chemical-domain membership; those remain data-curation and validation checks.

Conclusion

The paper addresses semi-supervised materials property learning when reliable labels are scarce by pairing a controlled benchmark with a reliability-weighted, ranking-aware objective. Across the benchmark, MatRank achieves the strongest aggregate performance, but the evidence supports it as a benchmark baseline rather than a universal materials-discovery solution.

  • SemiMat and MatRank target learning materials properties from scarce labels and unlabeled crystals without allowing uncertain pseudo-labels to dominate training.
  • SemiMat supplies the controlled benchmark contract, while MatRank supplies one reliability-weighted, ranking-aware objective under that contract.
  • MatRank achieves the lowest aggregate held-out test NMAE and best average method rank across six tasks, four backbones, and five split runs.
  • The evidence supports MatRank as a strong semi-supervised regression baseline for this benchmark, not as a universal solution to materials discovery.
Loading 2608.30682v1…