PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Aware Self-Supervised Pretraining on Multi-Domain Raman Spectra

Charalampos Davalas, Georgios Levantinos, Anastasios Dimitriou, Nikolaos Papanikolaou, Georgios Th. Papadopoulos, Dimitrios Michail

arXiv:2610.23955v1cs.LGcs.CVeess.IV

TL;DR

Raman spectroscopy produces abundant unlabeled measurements, while labels remain costly and conventional supervised models use only labeled data. This paper pretrains a multi-domain masked autoencoder with physics-aware reconstruction and denoising, finding benefits especially when labels are scarce and frozen representations are evaluated with attention pooling.

  • Problem

    Raman measurements are abundant but labeled datasets remain small, and how pretraining should be designed and used across Raman tasks remains insufficiently characterized.

    Source: §1 · pp. 1–2

  • Method

    A masked autoencoder is pretrained without labels on ten heterogeneous Raman datasets using masked intensity reconstruction, sparse Voigt-profile reconstruction, and denoising.

    Source: §1 · pp. 2–3

  • Results

    Pretraining helps most when labeled data are scarce relative to task difficulty, attention pooling improves on linear probing, and pretraining's value grows with encoder scale.

    Source: §1 · pp. 2–3, §7 · p. 17

  • Takeaways & Limitations

    Frozen representations contain useful information that linear probing may understate, and pretraining's advantage is less evident on large datasets or near-saturated tasks.

    Source: §7 · p. 17

Abstract

from arXiv · show

Raman spectroscopy generates large volumes of unlabeled data that are difficult to exploit with conventional supervised learning. This work introduces a multi-domain self-supervised learning framework based on a masked autoencoder (MAE) for learning transferable Raman spectral representations from heterogeneous biological, chemical, forensic, and materials datasets. The proposed pretraining strategy incorporates physics-aware objectives by combining masked intensity reconstruction with a sparse Voigt-profile decoder that models spectral peak structure through differentiable peak centers, widths, and amplitudes. A denoising objective is further included to improve robustness to spectral corruption. The learned encoder is evaluated on substance classification, component identification, concentration estimation, and spectral denoising under linear probing, attention-based probing, end-to-end fine-tuning, and training from scratch. Experimental results show that self-supervised pretraining is most beneficial for challenging tasks with limited labeled data. Attention-based probing also reveals substantially more useful information in the frozen representation than linear probing, indicating that conventional linear evaluation may underestimate representation quality. In addition, the benefits of pretraining become more pronounced with increasing encoder capacity. Overall, the proposed framework demonstrates that physics-aware, multi-domain self-supervised pretraining can improve label efficiency, robustness, and transferability across diverse Raman spectroscopy tasks.

1 Introduction

Raman measurements are abundant, but labels remain costly, leaving most spectra unused by conventional supervised learning. The paper introduces physics-aware pretraining across heterogeneous datasets and evaluates when it helps downstream tasks.

1 Introduction

  • Raman spectra are plentiful but labeled datasets remain small, so supervised models trained on one dataset and task leave most measurements unused.The forensic GBL concentration dataset, for example, contains about 1,500 spectra.Source: §1 · p. 1
  • Raman SSL had been explored with narrower objectives or data, while the paper asks how multi-domain pretraining and peak-structured reconstruction affect downstream use.Different setups distort peaks, and Voigt profiles provide a parametric approximation of band structure rather than a claim about underlying components.Source: §1 · p. 2
  • A single encoder is pretrained without labels on ten heterogeneous datasets using masked intensity reconstruction, sparse Voigt-profile reconstruction, and denoising.The Voigt decoder predicts peak centers, widths, and amplitudes under a sparsity penalty; augmentations simulate calibration shifts, intensity scaling, and detector noise.Source: §1 · pp. 2–3
  • The evaluation spans four downstream tasks, twelve dataset–task pairs, and three encoder scales, comparing frozen-feature probes, fine-tuning, and training from scratch.Because downstream datasets contribute unlabeled spectra to pretraining, this evaluation measures encoder reuse across datasets and tasks; unseen acquisition-domain transfer is left for future work.Source: §1 · pp. 2–3
  • On 30-class bacterial identification, fine-tuning reaches macro-F1 0.65 vs. 0.33 for training from scratch; GBL concentration estimation reaches 0.84 vs. 0.74.The bacterial identification gain holds at all three encoder scales; R6G detection in beverages is 1.00 vs. 0.96.Source: §1 · pp. 2–3

2 Related Work

Prior work established masked reconstruction, deep learning, and physics-informed methods across spectroscopy, but multi-domain Raman pretraining with peak decomposition remained unexamined. Raman benchmarking has also largely assessed supervised models trained separately for each target.

2 Related Work

  • Masked autoencoders and other SSL methods span language, images, scientific signals, and Raman, but prior Raman MAE work used one bacterial dataset and per-pixel reconstruction.Broader spectral foundation models span modalities, while contrastive approaches rely on carefully designed augmentations and positive/negative pairs.Source: §2 · p. 3
  • Deep learning and transfer methods address Raman classification, quantification, and instrument variability; synthetic spectra also provide an additional training source.The cited work includes CNNs, transformers, domain adaptation, and a dataset of more than 220,000 computed Raman and infrared spectra.Source: §2 · pp. 3–4
  • Physics-informed spectroscopy methods encode constraints in recovery or supervised unmixing, while sparse Voigt decomposition had not been used as an SSL reconstruction objective.Voigt fitting is established, but symmetric profiles can be unreliable for asymmetric or overlapping bands.Source: §2 · p. 4
  • Standard SSL evaluation uses frozen-feature linear probing and end-to-end fine-tuning, with random initialization comparisons measuring pretraining's sample-efficiency gains.The methods assess discriminative power, adaptability, and gains attributable to pretraining under limited labels.Source: §2 · p. 4
  • RamanBench unifies 74 public datasets and 163 targets, but evaluates supervised models trained from scratch rather than cross-domain SSL or physics-aware reconstruction.Its findings include no single model family generalizing across all datasets; this work instead studies a single encoder pretrained across heterogeneous unlabeled corpora.Source: §2 · p. 4

3 Datasets

The study assembles experimental Raman spectra from biological, chemical, forensic, and materials domains for unlabeled pretraining. Its datasets support substance classification, component identification, concentration estimation, and denoising.

3 Datasets

  • The pretraining corpus spans biological, chemical, forensic, and materials domains; Table 1 reports windowing, split counts, spectral ranges, and resolutions.Paired table rows show how windowed sampling changes split size and resolution; all spectra are used unlabeled during pretraining.Source: §3 · p. 5, Table 1 · p. 6
  • The evaluation includes bacterial identification across 30 isolates, liver tissue classification, and mineral classification in 192-class and 1,332-class subsets.The bacterial spectra have low signal-to-noise conditions; the mineral subsets assess different levels of class diversity and difficulty.Source: §3 · pp. 5–6
  • Quantification and component tasks include adenine concentration thresholding, four-bin GBL concentration classification, R6G presence detection, and sugar-mixture concentration estimation.The GBL task includes concentrations of 0, 0.1, 1, and 10 mg/mL; R6G mixtures include positive and negative beverage samples.Source: §3 · pp. 5–7
  • The MDA-Spectral dataset pairs short-acquisition low-SNR spectra with long-acquisition high-SNR targets for denoising.The broader hyperspectral dataset contains over 1.5 million Raman spectra from in vitro breast cancer cells.Source: §3 · p. 7
  • DeepRaman's augmented training corpus is used for pretraining, while experimental multi-component and liquid/powder mixtures are reserved for downstream component identification.The source training spectra span four excitation wavelengths: 1064, 785, 532, and 488 nm.Source: §3 · pp. 7–8

4 Methodology

The Raman MAE combines masked reconstruction, physics-aware peak modeling, and denoising, with shared preprocessing and several masking strategies. Its design uses a one-dimensional ViT and evaluates three model scales.

4 Methodology

  • The MAE combines masked intensity reconstruction, sparse Voigt-profile reconstruction, and denoising in a multi-objective pretraining framework.The one-dimensional ViT encoder processes visible tokens during masked reconstruction; three model scales, MAE-S, MAE-B, and MAE-L, are evaluated.Source: §4 · p. 8, §4.3 · p. 10

4.1 Preprocessing

  • Global min-max scaling maps wavenumber axes and intensities to [0, 1] before tokenization, while spectral augmentations simulate acquisition distortions.Augmentations include spectral shifts, intensity scaling, Gaussian noise, and continuous peak shift and stretch.Source: §4.1 · p. 9

4.2 Masking Strategies

  • Four masking strategies remove patches during pretraining: random contiguous, peak-centered, Voigt peak, and front masking.Peak-centered masking targets the dominant peak; Voigt peak masking assigns that region to the physics-aware objective, while front masking removes the final contiguous region.Source: §4.2 · pp. 9–10

4.3 Reconstruction and Denoising Objectives

  • The objective pairs masked-patch pixel reconstruction with sparse Voigt-profile decomposition and denoising from corrupted spectra.The Voigt branch combines peak-weighted and global reconstruction with sparsity regularization; denoising uses Poisson shot noise and additive Gaussian noise, with a two-layer MLP predicting the clean spectrum.Source: §4.3 · p. 10

5 Experiments

The experiments assess learned Raman representations on four analytical tasks using frozen-feature probes, end-to-end fine-tuning, and training from scratch. The task and protocol choices span different forms of prediction and representation adaptation.

5.1 Downstream Tasks

  • Four downstream tasks cover substance classification, component identification, concentration estimation, and spectral denoising.The dataset-to-task mapping distinguishes classes or concentration levels from component presence-or-absence decisions; denoising targets a high-SNR spectrum rather than a discrete label space.Source: §5.1 · p. 11
  • Component identification compares query and reference spectra, concentration estimation predicts discrete concentration levels, and denoising reconstructs clean spectra from noisy observations.Denoising is evaluated by fine-tuning and training from scratch, using MSE, PSNR, and SSIM.Source: §5.1 · p. 11

5.2 Evaluation Protocols

  • Four protocols assess frozen representations, pretrained initialization, and the contribution of pretraining: linear probing, attention probing, fine-tuning, and training from scratch.Linear probing trains a classifier on the frozen [CLS] embedding; attention probing learns task-specific pooling over frozen token embeddings, while fine-tuning adapts the encoder end to end and scratch training uses random initialization.Source: §5.2 · pp. 12–13

6 Results

Transfer learning’s benefit varies by task: it is strongest in data-constrained settings, while larger or simpler datasets often narrow the gap with training from scratch. Adaptation depth also matters, and denoising performance is similar across initialization strategies.

6 Results

  • For RRUFF-S, scratch training leads on the 192-class subset, while fine-tuning and scratch training have nearly identical F1 on the 1332-class subset.On the 1332-class subset, fine-tuning has considerably better AUC despite similar threshold-dependent metrics.Source: §6 · pp. 13–14, Table 4 · p. 14
  • Transfer learning provides the greatest benefit in data-constrained settings, while its advantage diminishes as downstream dataset size and diversity increase; pretraining may also improve probability discrimination and confidence estimation in complex multiclass settings.On 30-class Bacteria, fine-tuning achieved approximately 66% accuracy versus 39% for a randomly initialized model; both were trained for 30 epochs. Liver Cancer approaches produced comparable F1 scores of 88% to 90%, while scratch training achieved the highest performance on RRUFF-S 192-class data and fine-tuning and scratch produced nearly identical F1 scores on the 1332-class subset.Source: §6 · pp. 13–14
  • Fewer candidate components generally correspond to higher component-identification performance, while the 13-component DeepRaman benchmark remains modest across protocols.DeepRaman multi-component accuracy ranges from 0.5903 to 0.5975 and F1-scores from 0.5660 to 0.5729; protocol differences are small.Source: §6 · p. 14, Table 5 · p. 15
  • For R6G component identification, fine-tuning reaches accuracy, AUC, AUPRC, and F1-score of 1.0000 ± 0.0000, while attention probing also approaches perfect performance.On Raman Sugars, fine-tuning performs best overall, but scratch training is very similar; attention probing substantially exceeds linear probing.Source: §6 · p. 15, Table 5 · p. 15
  • On GBL concentration estimation, fine-tuning outperforms scratch training, whereas scratch training matches or exceeds fine-tuning on Raman Sugars.GBL fine-tuning reaches accuracy 0.8733 and F1-score 0.8351; Raman Sugars scratch training reaches accuracy 0.9460 and F1-score 0.9461.Source: §6 · p. 16, Table 6 · p. 16
  • On MDA-Spectral denoising, fine-tuning yields slightly better MSE and PSNR than scratch training, while scratch training yields slightly better SSIM.Overall denoising performance is comparable between initialization strategies and both significantly outperform the baseline.Source: §6 · pp. 16–17, Table 7 · p. 17

7 Conclusion

The paper pretrains a multi-domain Raman masked autoencoder using reconstruction, sparse Voigt-profile, and denoising objectives. Across downstream tasks, pretraining’s value depends on task difficulty and data availability, and the authors identify held-out instruments and sample types as a next transfer test.

7 Conclusion

  • Pretraining combines per-pixel reconstruction, sparse Voigt-profile reconstruction, and denoising on unlabeled spectra from ten biological, chemical, forensic, and materials datasets.The Voigt decoder uses a few profiles with centers, widths, and amplitudes as approximate peak parameters.Source: §7 · p. 17
  • Fine-tuning helps most when labels are scarce relative to task difficulty, while frozen features benefit more from attention pooling than a linear head.The value of pretraining grows with encoder scale, but it brings little or no benefit on large datasets or near-saturated tasks.Source: §7 · p. 17
  • Future work will pretrain larger encoders on larger, more diverse corpora and test transfer to instruments and sample types held out from pretraining.Source: §7 · p. 17

A Voigt Profile Mathematics

The Voigt branch models masked spectral regions with differentiable peak profiles, combining spectral fidelity with a preference for sparse peak representations.

A Voigt Profile Mathematics

  • The Voigt profile is a Lorentzian–Gaussian convolution, approximated for computational efficiency by a differentiable pseudo-Voigt combination.Its parameters include amplitude, center, and Lorentzian and Gaussian widths.Source: A Voigt Profile Mathematics · p. 20
  • Each masked patch is reconstructed from at most Kmax candidate profiles using a differentiable Voigt parameterization.The profiles represent peak-like structure rather than assuming every Raman band has an ideal Voigt line shape.Source: A Voigt Profile Mathematics · p. 20
  • Softplus enforces positive amplitudes and widths, while sigmoid rescaling bounds peak centers to the patch wavenumber range.The presence score is produced by a sigmoid and profiles below threshold τ = 0.5 are discarded at inference.Source: A Voigt Profile Mathematics · p. 20
  • The Voigt loss combines reconstruction fidelity with a sparsity penalty that encourages explaining masked regions with few active profiles.The peak-weighted reconstruction and masked-region reconstruction terms complement the sparsity objective.Source: A Voigt Profile Mathematics · p. 20

B Model Size Ablation

Across encoder sizes, frozen representations improve on several tasks, while the fine-tuning advantage over training from scratch widens on selected tasks. The scale effect varies by task.

B Model Size Ablation

  • From MAE-S to MAE-L, linear-probe F1 rises by 0.11 to 0.57 on seven of eleven prediction tasks, while changing by at most 0.03 on the other four.The largest gains are on the 192-class RRUFF-S subset (0.05 to 0.62) and Bacteria (0.16 to 0.45).Source: B Model Size Ablation · p. 21
  • The fine-tuning margin over training from scratch widens by 0.05 to 0.22 F1 on five tasks as encoder size increases.These are both RRUFF-S subsets, Raman Sugars identification and concentration, and GBL.Source: B Model Size Ablation · p. 21
  • On four of those five tasks, scratch-trained performance degrades from MAE-S to MAE-L, consistent with larger models overfitting limited labeled data.Fine-tuned performance does not show this degradation.Source: B Model Size Ablation · p. 21
  • On the 1,332-class RRUFF-S subset, fine-tuning shifts from 0.12 F1 below scratch with MAE-S to 0.09 above it with MAE-L.With MAE-L, fine-tuning exceeds the published baseline on all four metrics.Source: B Model Size Ablation · p. 21
  • Denoising also favors pretraining at larger scale: fine-tuning has higher MSE than scratch with MAE-S but lower MSE with MAE-L.The fine-tuning margin narrows by more than 0.02 F1 only on R6G, by 0.07, where scratch approaches perfect performance with MAE-L.Source: B Model Size Ablation · p. 21

C Qualitative Analysis of Maximum Peak Reconstruction

The Voigt decoder compresses localized peak structure into a small set of parameters, while the per-pixel branch predicts masked intensities directly. Their reconstructions emphasize complementary aspects of the spectrum.

C Qualitative Analysis of Maximum Peak Reconstruction

  • The Voigt decoder represents a masked region with a compact set of peak parameters rather than predicting every intensity independently.Each profile uses amplitude, center, Gaussian width, Lorentzian width, and presence score; the representation captures dominant morphology with fewer quantities than per-pixel reconstruction.Source: C Qualitative Analysis of Maximum Peak… · p. 21
  • The Voigt reconstruction captures the dominant peak’s position and overall width, smoothing small fluctuations instead of reproducing every local variation.It functions as structured compression of peak morphology, not a replacement for dense spectral reconstruction.Source: C Qualitative Analysis of Maximum Peak… · p. 21, Figure 3 · p. 25
  • The per-pixel decoder predicts masked-region intensities directly and can model broader spectral variation without a fixed peak-family constraint.Its representative reconstruction follows the overall signal shape and recovers dominant peak locations, but reproduces smaller fluctuations less precisely.Source: C Qualitative Analysis of Maximum Peak… · pp. 21, 25, Figure 4 · p. 25
  • The two branches provide different training signals: dense reconstruction encourages local fidelity, while Voigt reconstruction encourages a lower-dimensional description of peak position, amplitude, and width.The Voigt objective treats the profile as a general representation of peak-like structure rather than requiring ideal Voigt bands.Source: C Qualitative Analysis of Maximum Peak… · pp. 21, 25
Loading 2610.23955v1…