Source-linked AI summary
How Architecture and Training Affect TPC Representations Across Experiments
Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp, Daniel Bazin, Connor L. Cross, Hoi Yan Ian Heung, Andrew J. Jones, Ruchi Mahajan, Saiprasad Ravishankar, Pranjal Singh, Benjamin Votaw, Chris Wrede
TL;DR
The paper examines whether learned representations transfer when the learning task changes. Using probes and random-weight controls, it finds that architecture supplies substantial task-relevant structure that remains useful across tasks, experiments, and detectors.
Problem
The paper investigates whether representations learned in one setting transfer effectively when the learning task changes within an experiment.
Method
The authors use probes to measure task-relevant structure before adaptation and architecture-matched random-weight controls to distinguish architecture from encoder training.
Results
Architecture supplies a large fraction of task-relevant structure; random-encoder probes outperform supervised-training gains in all 16 linear-probe settings and 14 of 16 RBF comparisons.
Takeaways & Limitations
Both architectures embed informative event-level information, and this structure remains useful across changes in task, experiment, and detector.
Abstract
from arXiv · showhide
Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned representations to be reused beyond the experiments in which they were developed. This work evaluates the reusability of representations across experiments and detector systems using probes on frozen encoders. These probes reveal task-relevant structure before downstream adaptation, complementing fine-tuning. Together with random-weight controls, they distinguish contributions from architecture and encoder training that downstream performance alone cannot resolve. Time projection chamber (TPC) data provide a useful testbed because events from TPC systems can be represented as variable-length sparse tensors, while detector geometries, event topologies, and scientific tasks can differ substantially. We investigate whether fixed-dimensional TPC event representations can be reused across classification tasks, experiments, and detector systems. Sparse ResNet and PointNet-style encoders produce 512-dimensional embeddings for four datasets from the GADGET II TPC and AT-TPC. Randomly initialized encoders isolate the contribution from architecture before supervised training. We then train each encoder on a classification task, freeze its parameters, and train a linear or nonlinear probe for each downstream task. We find that this architecture-induced structure remains useful across experiments and detector systems. The randomly initialized PointNet-style representation is highly informative on several tasks. The two architectures organize their embedding spaces differently, but neither exhibits a large, systematic loss of utility cross-detector. These results show that architecture is a major source of task-relevant structure in TPC embeddings and should be treated explicitly when assessing representation learning and developing reusable detector models.
I. INTRODUCTION
TPC analyses face diverse event signatures across tasks, experimental conditions, and detector geometries, while raw data are sparse, variable-length point clouds. This work tests whether fixed-length representations can be reused across tasks, experiments, and detectors instead of training bespoke models repeatedly.
- TPC outputs are sparse, variable-length point clouds containing spatial coordinates and ionization charge.
- TPC machine-learning applications span particle classification, event identification, track segmentation, rare-event searches, background rejection, and kinematic reconstruction.
- Event signatures vary with the physics task, experimental conditions, and detector geometry, so existing machine-learning solutions are often bespoke.
- The study evaluates representation reuse when tasks, experiments, or detector systems change, using fixed-length embeddings of variable-length, permutation-invariant point clouds.
- The approach targets TPC analyses where labeled samples may be limited and event selection can precede computationally intensive reconstruction or fitting.
II. BACKGROUND
Frozen-encoder probes measure task-relevant information already present in representations, while random-weight controls separate architecture-induced structure from supervised training. The study compares sparse convolutional and PointNet-style encoders across substantially different TPC systems.
- Freezing an encoder and training a task-specific model on its outputs tests representation reuse without changing the encoder.
- The encoders use sparse convolutional neural networks and PointNet-style models suited to variable-length point-cloud inputs.
- Linear probes test linear separability, whereas nonlinear probes exploit more complex structure in the same fixed representation.
- Randomly initialized networks provide nonlinear feature maps whose probe performance can be compared with pretrained outputs to separate architectural and training contributions.
- The GADGET II and AT-TPC systems have substantially different geometries and event topologies, making cross-detector transfer nontrivial.
1. GADGET II TPC
The paper uses two contrasting TPC systems: GADGET II for low-energy β-delayed decay spectroscopy and AT-TPC for inverse-kinematics nuclear-reaction studies. Both reconstruct three-dimensional event topology from drift-time, pad-position, and charge information.
- 1. GADGET II TPC: GADGET II is a radioactive-ion-beam implant-decay detector optimized for low-energy β-delayed charged-particle spectroscopy.
- 1. GADGET II TPC: GADGET II reconstructs three-dimensional event topology from ionization electrons drifting to a position-sensitive MICROMEGAS readout plane.
- 1. GADGET II TPC: The detector has a cylindrical active volume with a 40 cm drift region and an approximately 8 cm diameter pad plane.
- 2. AT-TPC: The AT-TPC is a large active-target chamber for inverse-kinematics nuclear-reaction studies, using the gas as both reaction target and tracking medium.
- 2. AT-TPC: The AT-TPC is approximately 100 cm long and 58 cm in diameter, with 10,240 triangular pads and finer inner-pad spatial resolution.
III. METHODS
The study spans four experiments across GADGET II and AT-TPC, representing events as sparse (x, y, z, q) points. Encoders trained on simple classification tasks are evaluated with downstream probes under within-experiment, cross-experiment, and cross-detector transfer.
- Data and learning tasks: Four experiments provide two implant-decay datasets from GADGET II and two active-target datasets from AT-TPC.
- Data and learning tasks: Events are represented as sparse (x, y, z, q) points, with pad positions supplying transverse coordinates and drift time supplying the longitudinal coordinate.
- Data and learning tasks: Each detector uses a comparatively simple classification task to train an encoder before evaluating its representations on more complex downstream tasks.
- Data and learning tasks: Evaluations cover within-experiment transfer, cross-experiment transfer, and cross-detector transfer.
- GADGET II 20Mg dataset: The GADGET II 20Mg encoder is trained on binary proton-versus-alpha identification and probed on a three-class 20Mg task, cross-experiment 21Mg, and cross-detector 16O and 10B tasks.
2. GADGET II 21Mg dataset
The study uses TPC datasets from GADGET II and AT-TPC to test classification transfer across tasks, experiments, and detector systems. Events retain variable-length spatial and charge information, while encoders produce fixed 512-dimensional representations.
- Datasets and tasks: The 21Mg dataset defines three classes: 1285 keV protons, 1930 keV protons, and 2153 keV alpha particles.
- Datasets and tasks: The 16O dataset includes three-class 0–2, 3, and 4–5 track categories and a binary 0–2-track versus 3–5-track task used for encoder training.
- Datasets and tasks: The 10B dataset is used only as a downstream probe for cross-experiment and cross-detector representation tests.
- Event representation: Each event is represented as a variable-length set of spatial coordinates and ionization charge, preserving topology and charge without padding.
- Encoders: Both encoders produce 512-dimensional event representations but differ substantially in capacity.
1. ResNet backbone
The paper compares a sparse ResNet14 backbone with a Minkowski PointNet-style backbone for embedding variable-length TPC events. Both use global pooling to produce event-level representations, but they organize point information differently.
- ResNet backbone: Sparse ResNet14 applies convolutions across three spatial dimensions, treats charge as a feature channel, and ends with global max pooling and a fully connected head.
- ResNet backbone: Deeper ResNet variants increased training time without clear performance gains.
- PointNet-style backbone: PointNet-style processing combines information across inputs only through global pooling, preserving permutation invariance.
- PointNet-style backbone: The PointNet-style backbone uses shared point-wise transformations implemented with 1 × 1 convolutions and symmetric global pooling.
- PointNet-style backbone: Its sparse-tensor implementation supports variable-length inputs and preserves the variable-length structure of detector events.
3. Training and optimization
The evaluation freezes trained or random encoders and measures downstream information with fixed linear and nonlinear probes. Results show that architecture supplies useful task-relevant structure before supervised training, while training adds architecture-dependent gains.
- Training procedure: Five models were trained for each encoder task using five-fold cross-validation, with held-out folds used for encoder evaluation.
- Training procedure: Coordinate and charge scaling parameters were determined from training and validation data, while weighted sampling reduced class-imbalance effects.
- Frozen representations: Each trained or randomly initialized encoder was frozen before downstream probes were trained on its 512-dimensional event embeddings.
- Probe evaluation: Linear and RBF SVM probes measure complementary access to downstream class information through linear and nonlinear decision boundaries.
- Within-experiment results: Random-encoder probes substantially outperform the naïve classifier, showing that architectures provide useful task-relevant representations before supervised training.
- Within-experiment results: Encoder training yields additional gains, more pronounced for ResNet, while PointNet’s random representation is already especially effective for 20Mg.
2. Cross-Experiment Transfer
Representations remain useful across experiments and detector systems, with random PointNet embeddings often especially informative and supervised training adding architecture-dependent benefits. PCA further shows that similar probe performance can arise from different latent geometries.
- Cross-experiment transfer: Changing the experiment while retaining the detector does not eliminate task-relevant structure in the frozen representations.
- Cross-experiment transfer: Across GADGET II and AT-TPC transfer pairs, random PointNet representations remain particularly informative, while ResNet benefits more from encoder training.
- Cross-experiment transfer: Transferred class structure can be accessed through nonlinear boundaries, as trained representations show stronger performance with an RBF probe.
- Cross-detector transfer: Cross-detector reuse shows no pronounced penalty from changing detector systems; random embeddings remain informative and supervised training generally adds benefit.
- Cross-detector transfer: Cross-detector gains are clearest for ResNet and RBF probes, while PointNet changes comparatively little because its random representation is near the performance ceiling.
- PCA visualizations: Training on the 16O binary task reorganizes both embedding spaces and makes track-multiplicity distinctions more apparent.
- PCA visualizations: The architectures produce substantially different latent geometries, so similar downstream probe performance does not imply similar embedding-space organization.
- PCA visualizations: PCA visualizations are interpreted qualitatively because PCA identifies directions of greatest variance rather than directions maximizing class separation.
V. DISCUSSION
Architecture supplies a large fraction of task-relevant structure in TPC representations, with random embeddings often useful across tasks, experiments, and detectors. Frozen probes and random-weight controls expose these contributions beyond end-to-end performance.
- Architecture supplies a large fraction of task-relevant structure in TPC representations.
- The random-encoder probe-to-modal-classifier improvement exceeds supervised-training improvement in all 16 evaluated architecture–task settings.
- The same pattern holds in 14 of 16 RBF-probe comparisons across changes in task, experiment, and detector.Encoder training generally adds information, but gains vary across architectures and tasks.
- Random PointNet representations remain particularly informative across GADGET II and AT-TPC transfer pairs.PointNet often begins from a strong random baseline and consequently has less room to improve.
- PCA projections show that ResNet and PointNet organize the same events differently, even when probes attain similar performance.The random ResNet representation and random PointNet representation form visibly different embedding organizations and distributions.
- Frozen probes complement fine-tuning by measuring task-relevant structure accessible before adaptation, while random-weight controls distinguish architecture from encoder training.Reporting these diagnostics alongside fine-tuning provides a more complete evaluation of reusable scientific representations.
FUNDING
The work acknowledges support from U.S. Department of Energy and National Science Foundation awards, including resources from the Facility for Rare Isotope Beams. Processed data are intended for public release, while additional FRIB experimental data require applicable sharing policies and co-spokesperson approval.
- FUNDING: The work was supported by the U.S. Department of Energy under awards DE-SC0024587 and DE-SC0023633.
- FUNDING: The work also received support from the U.S. National Science Foundation under award OAC-2311263.
- FUNDING: The study used resources of the Facility for Rare Isotope Beams, a DOE Office of Science User Facility.
- DATA AVAILABILITY: Processed data used to generate the reported results will be made publicly available at publication in accordance with the FRIB Data Management and Sharing Plan.
- DATA AVAILABILITY: Access to additional FRIB experimental data is subject to applicable data-sharing policies and approval of the experiment co-spokespersons.
Appendix A: Training Details
Training used five-fold experiments with held-out test folds, pooled non-test data, and an 80:20 training-validation split. ResNet14 and PointNet-style encoders shared settings unless otherwise indicated, while randomly initialized encoder baselines were not trained.
- Cross-validation: In each five-fold experiment, fold i was held out for testing, while run i was used only to construct training batches.
- Data splitting: The four non-test folds were pooled and divided into training and validation subsets using an 80:20 split.
- Preprocessing: Scaling parameters were calculated from the training and validation subsets and held fixed when processing the corresponding test subset.
- Encoders: ResNet14 and PointNet-style encoder settings were used across encoder-training datasets unless otherwise indicated.
- Optimization: Learning-rate reduction and early stopping were both based on validation loss, with final-model hyperparameters listed in Table VI.
- Baselines: The randomly initialized encoder baselines used the same per-run seed convention but were not trained.