Leakage-Safe Machine Learning for Hydrogen Embrittlement Detection in 316L Stainless Steel: A Region-Held-Out Evaluation of Texture and Deep Features in SEM Micrographs

1 Introduction

• SEM captures visual signs of hydrogen damage, while manual microstructural analysis is slow and open to human bias.Automated computer vision and machine learning are presented as ways to scale analysis.Source: §1 · p. 2
• Region-held-out evaluation tests whether models generalize to unseen microstructural domains instead of benefiting from spatial leakage or region-specific features.Image-level splits can mix images from the same region across training and test sets, while region separation tests on a zone the model has not seen.Source: §1 · p. 2
• Neural-network adoption in metallography is limited by scarce annotated datasets, information-dense micrographs, and hard-to-interpret decisions.The introduction also notes that models trained on natural images or specialized medical tasks can yield inaccurate boundary delineation and ambiguous labels in 316L microstructures.Source: §1 · pp. 2–3
• The study builds a pipeline for region-held-out SEM-based hydrogen-embrittlement detection in 316L and examines model performance and architecture on distinct domains.The broader motivation is validation on previously unseen material regions.Source: §1 · p. 2
• Spatial correlations and hidden stratification can let models use artifacts or local imaging features instead of underlying damage, producing overly optimistic metrics on unseen regions.The introduction identifies background artifacts, specimen-preparation textures, and local imaging features as possible shortcuts.Source: §1 · p. 2

2 Methodology

• The primary task uses 31 316L images from 14 regions, with region identity as the leakage-prevention unit for data splits.The subset comprises 8 AR and 6 H2 regions.Source: §2 · p. 3
• The compared inputs include LBP and GLCM texture descriptors, self-supervised global embeddings, and a CNN trained on raw images.LBP and GLCM were used individually and in combined-feature classifiers; the autoencoder was pretrained on unlabeled images.Source: §2 · p. 4
• Fourteen-fold Leave-One-Region-Out cross-validation withholds each region in turn, pools predictions, and evaluates balanced accuracy, F1, precision, and recall.The evaluation emphasizes H2, the minority class, because missed H2 cases matter most for safety.Source: §2 · p. 4
• A group-level permutation test reruns the LORO pipeline under permuted region-to-label assignments to form a null distribution of balanced accuracy.The test uses 500 draws from the 3,003 possible assignments.Source: §2 · p. 4
• Grad-CAM is applied to a CNN trained on the full 316L image set to locate image regions that distinguish AR from H2 surface morphologies.This interpretability analysis is not run with a held-out split.Source: §2 · p. 4

3.1 Overview of LORO classification performance

• For 316L AR vs. H2 in 14-fold LORO cross-validation, LBP+SVM performed best: balanced accuracy 0.79 ± 0.36, F1 0.75, and H2 precision 0.82, exceeding CNN balanced accuracy 0.67 ± 0.45.Source: §3.1 · p. 5, Table 1 · p. 5

3.2 Pooled confusion matrix and H2 recall

• LBP+SVM correctly classified 16 of 18 AR images (specificity = 0.89) and 9 of 13 H2 images (recall = 0.69), missing 4 H2 images and flagging 2 AR images.The authors note that the four false negatives mean the model currently misses roughly one in three H2-exposed regions, limiting its use as a stand-alone screening tool.Source: §3.2 · pp. 5–6, Table 2 · p. 6, Figure 1 · p. 6

3.3 Fold-level variance and the cost of region-held-out evaluation

• Across folds, LBP+SVM’s aggregate-results standard deviation was ±0.36; in a two-image fold, one misclassified image could shift the fold’s score by 0.5 or more.The variance reflects the difficulty of generalizing to an entirely new specimen region and provides a more realistic estimate of practical performance.Source: §3.3 · pp. 6–7

3.4 Statistical significance via permutation testing

• LBP+SVM’s observed balanced accuracy of 0.7906 exceeded the null in a group-level permutation test (3 of 500 permutations reached or exceeded it; p = 0.008).The test permuted region-to-label assignments and reran the full LORO pipeline; the authors concluded the result was not due to a chance arrangement of region labels.Source: §3.4 · p. 9
• LBP+GLCM+SVM was not significant under the same permutation procedure (observed BA = 0.62, p = 0.174), supporting the authors’ concern that added GLCM features contributed noise.The paper describes the comparison as differing only in the added GLCM features and notes that the test could distinguish the two representations.Source: §3.4 · p. 9
• Observed balanced accuracy was 0.79, with permutation-test p = 0.008, supporting a statistically significant hydrogen-charging signature under the Leave-One-Region-Out protocol.The null distribution was generated from 500 permutations of region-to-label assignments.Source: Figure 4 · p. 9, §3.4 · p. 9

3.5 Grad-CAM interpretability and physical plausibility

• Grad-CAM activations varied across the six sampled images but tended to localize on grain-boundary networks and surface relief rather than global image properties.The maps were generated for three AR and three H2 images using a SmallCNN trained on all 316L images.Source: §3.5 · p. 10
• H2 activation hotspots often coincided with visually different surface texture, qualitatively matching descriptions of hydrogen-induced changes in grain-boundary cohesion and surface texture.This spatial agreement suggests physically plausible features rather than imaging artifacts or background, but is qualitative evidence.Source: §3.5 · p. 10
• The LBP histogram comparison links the best classical model, LBP+SVM, directly to an inspectable image representation, addressing a common black-box criticism.The comparison provides a direct view of texture-descriptor values used by the classifier.Source: §3.5 · p. 10
• The interpretability analysis is qualitative and based on only six images; it supports, but does not replace, the quantitative LORO and permutation-test results.It should be read as converging evidence rather than evidence in its own right.Source: §3.5 · p. 10

4 Conclusion

• 0.79 balanced accuracy, 0.69 H2 recall, and 0.82 H2 precision were achieved by LBP+SVM, which outperformed the tested CNN, self-supervised embeddings, and GLCM-based models.The six feature-classifier combinations were evaluated with 14-fold Leave-One-Region-Out cross-validation.Source: §4 · p. 10
• p = 0.008 in the group-level permutation test indicated that LBP+SVM performance was not due to a chance arrangement of the 14 specimen regions.The test sampled 500 permutations from 3,003 possible region-to-label arrangements.Source: §4 · p. 10
• Grad-CAM activations concentrated on localized surface and grain-boundary features, but no single activation pattern was consistent across sampled images.The CNN was trained on the full dataset without a held-out split, so this interpretability result is qualitative rather than held-out validation.Source: Figure 5 · p. 11, §4 · p. 13
• The authors conclude that simple, low-parameter texture descriptors can outperform deep and self-supervised representations in small-sample SEM classification tasks.They also present region-held-out evaluation with group-level permutation testing as a leakage-safe validation framework.Source: §4 · p. 13
• The 31-image study is a starting point; adding regions and specimens is proposed to reduce fold-level variance and increase confidence in the metrics.The protocol could also be extended to other alloy systems, for which pilot data are not yet large enough for region-held-out evaluation.Source: §4 · p. 13

• MA developed the methodological framework and handled implementation, experimental design, model training and evaluation, performance analysis, and the original draft.Source: Author Contribution Statement · p. 13
• AS supervised the research, while AS and NAN contributed to manuscript review and editing; NAN also provided computational resources.Source: Author Contribution Statement · p. 13
• MY and HZ prepared and edited figures and tables, MZS contributed proofreading and language correction, and all authors approved the final manuscript.Source: Author Contribution Statement · p. 13

• The SEM micrographs analyzed in the study are available from the corresponding author on reasonable request.Source: Data Availability Statement · p. 13
• A metadata convention defines specimen regions using a material-condition-region-image naming pattern.Source: Data Availability Statement · p. 13
• The documented specimen-region convention determines the grouping used for Leave-One-Region-Out cross-validation.Source: Data Availability Statement · p. 13

• Code for feature extraction, self-supervised pretraining, Leave-One-Region-Out validation, permutation testing, and Grad-CAM analysis is openly available under the MIT license.Source: Code Availability Statement · p. 14
• The reported results were produced using tagged release v1.0, and all experiments used fixed random seed 42.Source: Code Availability Statement · p. 14
• Experiments ran on CPU with Python 3.13 and PyTorch 2.7.0.Source: Code Availability Statement · p. 14

• The authors declare no known competing financial interests or personal relationships that could have influenced the reported work.Source: Declaration of Competing Interest · p. 14