Source-linked AI summary
STAR-XPC: Set Transformer for Estimating X-ray Transmission in Polymer Composites
Can Erer, H. Birkan Yilmaz, Nebahat Aral
TL;DR
Estimating X-ray transmission through polymer composites is challenging because mixtures and spectra are complex, while Monte Carlo simulation is computationally costly. STAR-XPC predicts transmission from unordered constituent sets and physical descriptors; cross-validation reports NMSE% = 0.0294% and R^2 = 0.9988.
Problem
Composite transmission is difficult to estimate because mixtures are heterogeneous, beams are polychromatic, and unordered constituents create a permutation-invariance requirement.
Source: §1 · pp. 1–2Method
STAR-XPC embeds constituent molecules with ChemBERTa and uses a permutation-invariant Set Transformer with bulk physical descriptors to predict transmission.
Source: §1 · p. 2, §3.2.3 · p. 5, §3.3.1 · p. 5Results
NMSE% = 0.0294% and R^2 = 0.9988 across cross-validation on 4,697 simulated configurations.
Source: §6 · p. 11Takeaways & Limitations
Inference takes sub-milliseconds per query versus hours per Monte Carlo run, supporting interactive composite design within the studied scope.
Source: §6 · p. 11ChemBERTa was pretrained on drug-like organics, while several shielding constituents are outside that distribution, and the dataset contains only 13 distinct constituent molecules.
Source: §5.2 · pp. 10–11
Abstract
from arXiv · showhide
Evaluating the energy transmission ratio of an X-ray beam through a polymer composite is a challenging task in radiation shielding studies. The reference solution with Monte Carlo transport simulation is highly accurate but computationally expensive: each of our reference simulations is capped at $4\times10^9$ photon histories and a 20,000 s (over 5 hours) time limit, while closed-form analytical methods fail to represent such complex scenarios. We present an artificial neural network model that maps the unordered multi-set of molecular constituents, their mass fractions, and four bulk physical descriptors directly to the transmission ratio $I(t)/I_0$ in 1-3 milliseconds. The model is a Set Transformer, which makes it permutation invariant by construction and accepts mixtures of different sizes up to 11 constituents within a fixed-size padded set. Each constituent is embedded with a chemistry-pretrained transformer language model, ChemBERTa. In our proposed method, training data are produced by an automated PENELOPE/penEasy pipeline using the radiation source at 30, 80, 100, and 120 kV tube voltages over polyethylene-based shielding composite slabs of thicknesses spanning 0.15-2.5 mm. After preprocessing, the dataset contains $n = 4697$ simulated configurations on which a leakage-free, jointly stratified 10-fold cross-validation is performed, attaining mean squared error MSE = $(1.12 \pm 0.56)\times10^{-4}$, normalized-MSE percentage NMSE% = 0.0294% $\pm$ 0.0147%, and $R^2 = 0.9988$. The corresponding per-sample percentage errors give an alpha-trimmed ($\alpha = 0.10$) absolute percentage error (APE) of 1.66% and a median APE of 1.15%. Our model's inference is several orders of magnitude faster than the underlying Monte Carlo simulation (sub-millisecond per query vs. several hours for the reference run) with effective accuracy in predicting $I(t)/I_0$.
1. Introduction
Estimating X-ray transmission in polymer composites is difficult because mixtures are heterogeneous, beams are polychromatic, and design choices are combinatorial. STAR-XPC addresses the computational cost and ordering challenges with a permutation-invariant set transformer for transmission prediction.
1. Introduction
- 30–120 kV beams span varying spectra, while composite constituents can have interaction cross sections differing by orders of magnitude.The design space also includes arbitrary material mixtures and thicknesses, requiring engineers to test many candidates.Source: §1 · p. 1
- Closed-form mixture methods assume narrow-beam geometry and a monoenergetic source, limiting their treatment of broad-beam, polychromatic composites.Spectrum hardening and scattered photons further complicate direct transmission estimates.Source: §1 · p. 1
- Monte Carlo reference runs are accurate but costly: a 20,000 s limit and up to 4 × 10^9 photon histories constrain evaluation across the design space.The time limit is binding for most configurations, so completed histories and precision vary.Source: §1 · p. 2
- Flattened constituent inputs impose arbitrary ordering, whereas STAR-XPC's set architecture encodes the physical invariance of attenuation to constituent order.The proposed pipeline combines automated PENELOPE/penEasy data generation, ChemBERTa constituent embeddings, and a Set Transformer.Source: §1 · p. 2
- STAR-XPC treats composites as unordered sets and builds permutation invariance into its set-transformer architecture.The model maps constituent tokens enriched with bulk physical descriptors to the energy transmission ratio.Source: §1 · p. 2
2. Background Information
The background explains the idealized physics behind transmission, the computational trade-off in Monte Carlo transport, and why set-based attention suits unordered mixtures. Together, these topics motivate a learned surrogate for polychromatic composite transmission.
2.1. Radiation Attenuation Physics
- The model target is the Monte Carlo transmission ratio, which includes scattered photons and characteristic fluorescence reaching the scoring region.This differs from the narrow-beam idealization described for a slab of thickness t and energy E.Source: §2.1 · p. 2
- At fixed energy, mass attenuation is additive under the standard assumption that mixture chemistry does not alter atomic wavefunctions.The analytic prediction combines mass additivity with Beer-Lambert attenuation for narrow-beam, monoenergetic conditions.Source: §2.1 · p. 2
2.2. Monte Carlo Simulation Method
- PENELOPE samples particle histories from the linear Boltzmann transport equation through a Markov-process construction.PenEasy drives the transport library with input files and provides source, geometry, tally, and variance-reduction components.Source: §2.2 · p. 3
- Halving Monte Carlo uncertainty quadruples cost; reference runs take about 5 CPU-hours for 2% relative uncertainty, motivating a learned surrogate.The reference runs use up to 4 × 10^9 histories.Source: §2.2 · p. 3
2.3. Machine Learning for Physics Applications
- Flattened inputs and sequence models impose an arbitrary constituent order, while padding is compatible with invariance only when tokens are interchangeable and the model is permutation invariant.The target is an aggregate property of an unordered multiset.Source: §2.3 · p. 3
- ISAB processing is permutation equivariant, and pooling by multihead attention produces a fixed-length permutation-invariant set summary for the prediction head.The Set Transformer replaces Deep Sets' sum aggregator with attention.Source: §2.3 · pp. 3–4
3. Methodology
STAR-XPC development combines data generation, feature engineering, set-transformer design, and a cross-validation protocol.
3. Methodology
- The methodology begins with dataset generation and feature engineering for STAR-XPC.These are two of the process components identified for model development.Source: §3 · p. 4
- The model-development process includes designing a set-transformer architecture.The architecture design is listed as a distinct component of developing STAR-XPC.Source: §3 · p. 4
- Cross-validation is included in the protocol for developing the proposed model.The paper identifies cross-validation alongside data generation, feature engineering, and architecture design.Source: §3 · p. 4
3.1. Data Generation Pipeline
An automated PENELOPE/penEasy pipeline generated simulations for polyethylene-based composites under four standardized beam qualities. Filtering yielded 4,697 valid configurations.
3.1. Data Generation Pipeline
- The simulations used fixed beam qualities at 30, 80, 100, and 120 kV, generated once and shared across composites.The qualities were N-30, RQR 6, RQR 8, and RQR 9, respectively.Source: §3.1 · p. 4
- Samples used polyethylene with twelve possible inorganic or elemental shielding materials, with up to three shielders mixed per configuration.The polymer was represented as a constant-indicator token, and its mass fraction was drawn from a fixed discrete list.Source: §3.1 · p. 4
- The mixture's bulk density was computed using the additivity-of-volumes rule, and mean excitation energy came from PENELOPE's material utility using Bragg additivity.The constituent elemental excitation energies came from PENELOPE's database.Source: §3.1 · p. 4
- 4,697 valid samples remained after excluding invalid transmission ratios, missing bulk descriptors, or missing additive-identity entries.The target is transmitted energy normalized by the source-spectrum integral; sample transmission ratios had a median of 0.539.Source: §3.1 · p. 4
3.2. Input Feature Engineering and Representation
STAR-XPC represents each composite through constituent molecular embeddings and mass fractions, paired with bulk physical descriptors. The representation uses fixed-size padded sets and ChemBERTa-derived constituent features.
3.2.1. Molecular Embeddings
- Each constituent’s SMILES is embedded with ChemBERTa by averaging its four upper hidden layers, mask-weighting token states, and applying ℓ2 normalization.The molecular strings are generated through a deterministic lookup, including for inorganic compounds and pure elements.Source: §3.2.1 · p. 4
3.2.2. Composition Features
- Composites use 11 slots—up to 10 material slots plus the polymer—with normalized mass fractions; empty slots have zero embeddings and fractions but retain global descriptors.Padding tokens are not masked; their identity preserves permutation invariance, while the fixed slot count makes the trained network specific to N = 11.Source: §3.2.2 · pp. 4–5
3.2.3. Global Physical Parameters
- Removing mean excitation energy increased MAE in 9 of 10 folds, although the descriptor is deterministic from constituent composition and mass fractions.The paper describes it as a smooth proxy for mixture high-Z content.Source: §3.2.3 · p. 5
- Four global descriptors encode tube voltage, bulk density, slab thickness, and Bragg-additivity mean excitation energy, and are concatenated to every component token.An explicit voltage descriptor lets one network handle four beam settings without a per-voltage head; mean excitation energy is retained as an engineered compositional summary, not a direct photon-attenuation parameter in the stated range.Source: §3.2.3 · p. 5
3.3. Set Transformer Architecture
STAR-XPC processes padded constituent tokens with a Set Transformer and predicts the energy transmission ratio. On CPU, its forward pass takes milliseconds per sample, while training uses a squared-error objective and a cosine learning-rate schedule.
3.3.1. Core Architecture
- The model maps an 11 × 773 padded token set to a scalar transmission-ratio prediction using permutation-equivariant ISAB blocks, PMA pooling, an SAB block, and a linear head.Each token combines a 768-dimensional molecular embedding, mass fraction, and four global descriptors; the network has 3,004,417 trainable parameters.Source: §3.3.1 · p. 5, Table 1 · p. 5, Figure 1 · p. 6
- 2.6 ms per sample at batch size 1 and 0.99 ms at batch size 256; both CPU inference times are several orders of magnitude below multi-hour reference runs.The comparison is reported regardless of hardware for the PENELOPE/penEasy reference runs.Source: §3.3.1 · p. 5
3.3.2. Training Configuration
- Training minimizes raw-target mean squared error with AdamW, batch size 256, and a cosine-annealed learning rate over 200 epochs; all reported runs completed without early stopping.Early stopping was configured after 50 consecutive epochs without validation-loss improvement.Source: §3.3.2 · p. 5
3.4. Cross-Validation Protocol
Generalization is estimated with jointly stratified 10-fold cross-validation, with normalization statistics fitted only on training data. The stratification combines shielding-material identity and target-value quantile bins.
3.4. Cross-Validation Protocol
- 10-fold cross-validation is jointly stratified by main shielding material and empirical-quantile bins of the transmission target.The target spans several decades and the chemistry distribution is nonuniform, motivating stratification on both dimensions.Source: §3.4 · p. 6
- The number of target bins is adaptively set to keep at least 10 samples per stratum, and each fold uses a 90/10 train-plus-validation/test split.When early stopping is used, the train-plus-validation portion is further divided 89/11; the separate single-run training uses an 80/20 train/validation split.Source: §3.4 · p. 6
- Normalization statistics are fitted on each fold’s training rows only, and the resulting Z-scoring is applied unchanged to validation and test partitions.The paper notes that leaking validation or test statistics could boost perceived accuracy without changing underlying generalization.Source: §3.4 · p. 6
- Reported evaluation metrics are RMSE, MSE, MAE, and normalized-MSE percentage (NMSE%).Source: §3.4 · p. 6
4. Results and Analysis
STAR-XPC is compared with flattened regression baselines and evaluated across cross-validation, material conditions, and held-out compositions. It performs well overall, while permutation tests and generalization experiments probe whether performance extends beyond the training arrangements and mixtures.
4.1. Baseline Comparison
- On held-out unseen combinations, STAR-XPC reaches mean NMSE% 0.0496% versus 0.0804% for SVR.Table 2 compares in-distribution cross-validation with held-out composition tests.Source: Table 2 · p. 7, §4.1 · p. 7
4.2. Detailed Cross-Validation Performance of STAR-XPC
- Across 4,697 configurations, 10-fold cross-validation yields MSE = (1.12 ± 0.56) × 10^-4 and R^2 = 0.9988; MAPE0.10 is 1.66%.Untrimmed MAPE is 203%, driven almost entirely by 129 near-opaque samples with y < 0.01.Source: §4.2 · pp. 7–8
4.1. Baseline Comparison
- Across 40 slot reorderings, STAR-XPC’s prediction standard deviation is ≤1.3 × 10^-7, while SVR’s is 0.067–0.098 for the same sample.Random Forest is less sensitive than SVR but still changes by up to 0.034 for one sample.Source: §4.1 · p. 7, Figure 3 · p. 8
4.2. Detailed Cross-Validation Performance of STAR-XPC
- Training and validation losses plateau near O(10^-4) and closely track each other throughout training.The paper interprets the similar curves as indicating no overfitting; the single-run model’s loss plot marks its best-validation epoch.Source: §4.2 · p. 7, Figure 4 · p. 8
4.3.1. Energy-level performance
- α-trimmed MAPE decreases with tube voltage, from 1.65% at 80 kV to 1.25% at 100 kV and 1.13% at 120 kV.Absolute and squared-error metrics remain small across these regimes but do not follow a monotonic trend.Source: §4.3.1 · p. 8, Table 3 · p. 9
4.3.2. Thickness-dependent accuracy
- As thickness increases, MAPE0.10 and NMSE% increase, while MAE has no trend across the reported thickness bins.The paper relates this pattern to transmission’s exponential decay with thickness: relative error scales with thickness while absolute error remains bounded.Source: Table 4 · p. 9, Table 5 · p. 9, §4.3.2 · p. 9
4.3.3. Chemistry-stratified performance
- Compound fillers are estimated more accurately than elemental shielders in the reported material comparison.For four element–compound pairs, the compound has lower MAE and MAPE0.10; the paper leaves a proposed attenuation-based explanation for future testing.Source: Table 6 · p. 9, §4.3.3 · p. 9
4.4. Generalization Capability of STAR-XPC
- All six held-out compositions achieve NMSE% ≤0.129% and MAPE0.10 ≤6.86%.For unseen Pb and Gd, tested without retraining, NMSE% is 0.192% and 0.198%, respectively.Source: §4.4 · pp. 9–10, Figure 5 · p. 10
5. Discussion
The Set Transformer builds constituent-order invariance into the model, but its demonstrated scope is limited by the training mixtures, fixed set size, polyethylene-only samples, and uncertain embeddings for out-of-domain shielding species.
5.1. Set Transformer Architecture Benefits
- The Set Transformer enforces constituent-order invariance by construction, matching the physical symmetry of mixture attenuation.The same network handles 1-, 2-, and 3-component mixtures; training data contain at most three shielding materials per sample.Source: §5.1 · p. 10
- Accuracy on mixtures with more than three shielding materials remains unvalidated, and changing the fixed set size N = 11 changes predictions.Higher-order mixtures contain fewer padding tokens than training samples, while unmasked padding tokens carry global descriptors and receive attention.Source: §5.1 · p. 10
5.2. Limitations and Challenges
- The polymer is fixed as polyethylene, so its token embedding has no discriminative power; varying polymers would require a richer, polymer-aware embedding pipeline.A polymer-aware encoder fine-tuned on engineering polymers is offered as an example, and studying varying polymers is beyond this paper's scope.Source: §5.2 · p. 10
- ChemBERTa's drug-like molecular pretraining domain differs from several shielding constituents, and their embeddings are not validated to encode chemically meaningful structure.The dataset contains only 13 distinct constituent molecules; the paper notes evidence against a pure lookup-table interpretation because the model generalizes to unseen Pb and Gd.Source: §5.2 · pp. 10–11
6. Conclusions and Future Directions
STAR-XPC estimates X-ray transmission in polyethylene/shielder composites using a permutation-invariant Set Transformer trained on PENELOPE/penEasy simulations. It achieves low cross-validated error and reports generalization to held-out compositions and unseen materials, with inference orders of magnitude faster than Monte Carlo runs.
6. Conclusions and Future Directions
- The permutation-invariant Set Transformer predicts transmission for polyethylene/shielder composites from a PENELOPE/penEasy-generated dataset.The model predicts only the transmission ratio; other tally metrics and physics-informed regularizers are proposed for future investigation.Source: §6 · p. 11
- NMSE% = 0.0294% (R2 = 0.9988) across four voltages, with NMSE% ≤0.129% on six held-out compositions and ≤0.2% on unseen Pb and Gd.The evaluation uses n = 4697 simulated configurations spanning 12 main shielding material chemistries.Source: §6 · p. 11
- Sub-millisecond inference per query on a single GPU is orders of magnitude faster than hours per Monte Carlo run, supporting interactive composite design.The stated application scope includes medical imaging, industrial radiography, and shielding optimization.Source: §6 · p. 11