PaperlayerView exact revision on arXiv

Source-linked AI summary

Data Augmentation of Spectral Data for Convolutional Neural Network (CNN) Based Deep Chemometrics

Esben Jannik Bjerrum, Mads Glahder, Thomas Skov

arXiv:1710.01927v1cs.LG

TL;DR

The paper studies predicting tablet drug content from NIR spectra and how preprocessing affects CNN modeling. It benchmarks data augmentation and EMSC, finding that their combination works best and that CNNs perform strongly in extrapolation tests.

  • Problem

    The study examines CNN prediction of tablet drug content from NIR spectra and compares preprocessing approaches for that task.

    Source: Introduction · p. 2

  • Method

    CNNs were trained with spectral data augmentation and EMSC preprocessing and compared with PLS models.

    Source: Introduction · pp. 1–2

  • Results

    CNNs nearly always surpassed standard PLS models, and data augmentation followed by EMSC worked best; CNNs also predicted higher assay values and spectra from another instrument.

    Source: Discussion · p. 5, Conclusion · p. 8

  • Takeaways & Limitations

    CNN kernel activations resembled smoothing, derivative or slope recognition, thresholding, and spectral-region selection.

    Source: Conclusion · p. 8, Discussion · p. 8

    PLS tuning used information from the external test set, which the authors say can overestimate real performance.

    Source: Discussion · p. 5

Abstract

from arXiv · show

Deep learning methods are used on spectroscopic data to predict drug content in tablets from near infrared (NIR) spectra. Using convolutional neural networks (CNNs), features are ex- tracted from the spectroscopic data. Extended multiplicative scatter correction (EMSC) and a novel spectral data augmentation method are benchmarked as preprocessing steps. The learned models perform better or on par with hypothetical optimal partial least squares (PLS) models for all combinations of preprocessing. Data augmentation with subsequent EMSC in combination gave the best results. The deep learning model CNNs also outperform the PLS models in an extrapolation chal- lenge created using data from a second instrument and from an analyte concentration not covered by the training data. Qualitative investigations of the CNNs kernel activations show their resemblance to wellknown data processing methods such as smoothing, slope/derivative, thresholds and spectral region selection.

Introduction

The paper applies CNNs to spectroscopic analysis and examines spectral data augmentation alongside EMSC for tablet-content prediction. It compares these preprocessing approaches with hypothetical optimal PLS models and tests extrapolation across assay values and instruments.

Introduction

  • CNNs scan neighboring spectral points with shared-weight kernels, whose activations feed later convolutional or dense layers.The paper describes spectra as one-dimensional pictures suitable for this architecture.Source: Introduction · p. 1
  • Prior work applied CNNs to spectroscopic classification and regression, including Raman and VIS-NIR spectra and spectral-region selection.The cited examples include tablet classification and NIR/Raman regression.Source: Introduction · p. 1
  • Spectral augmentation expands training data by adding random offsets, slope changes, and multiplications to existing spectra.These perturbations simulate baseline offsets, slope differences, and intensity differences in recordings.Source: Introduction · p. 1, Figure 1 · p. 1
  • Dropout randomly removes activations during training to disrupt neuron interdependence and limit overfitting.The layer is inactive during sampling after training.Source: Introduction · pp. 1–2
  • The study compares CNNs with hypothetical optimal PLS models across preprocessing choices and tests extrapolation across assay values and instruments.CNN hyperparameters and dropout regularization are also tuned.Source: Introduction · p. 2

Methods

The methods define tablet-spectrum dataset splits, preprocessing variants, CNN architecture, and model-tuning procedures. They also specify how PLS component counts were selected.

Datasets

  • Training and validation used instrument-one spectra, while test sets used instrument two; extrapolation splits separated concentrations below 212 mg, 212–228 mg, and above 228 mg.Modeling used the 600–1798 nm spectral region; the standard split successively reserved 20% for test and validation.Source: Datasets · p. 2
  • Outliers were identified using PLS with 10-fold cross-validation over 1–30 components, then flagged above 2.5 times the standard deviation of absolute prediction error.The selected component count minimized average Huber loss.Source: Datasets · p. 2
  • Augmentation added nine variants per training sample, with offsets and multiplications varied by ±0.10 training-set standard deviations and slopes between 0.95 and 1.05.EMSC used order one and the training-set average spectrum as its reference; datasets were also normalized using the training-set global mean and twice its standard deviation.Source: Datasets · p. 2

Neural Networks

  • The CNN used two one-dimensional convolutional layers followed by flattening, dropout, a dense layer, and a single-neuron linear output.Input first passed through a Gaussian-noise layer; training used Huber loss and Adadelta.Source: Neural Networks · p. 2

Hyperparameter tuning and training

  • Bayesian optimization searched the listed layer and convolutional hyperparameters, using 20 random starts and up to 40 expected-improvement iterations.Networks were trained for 40 or 200 epochs during search, then final models for 100 or 250 epochs, depending on augmentation.Source: Hyperparameter tuning and training · pp. 2–3, Table 1 · p. 3
  • PLS component counts were selected by the lowest Huber loss on the test set after training on pooled training and validation data.Training, cross-validation, and corrected cross-validation losses were also recorded.Source: Hyperparameter tuning and training · p. 3

Results

CNNs generally outperform PLS on RMSE and Huber loss, with correlations roughly on par; data augmentation followed by EMSC gives the best performance. The extrapolation comparison favors CNN predictions, but its test set differs from the other datasets.

Results

  • CNN hyperparameter tuning used 20 random starts before Bayesian optimization; the EMSC example found its best solution at iteration 41.The optimization selected validation loss for the CNNs.Source: Results · p. 3, Figure 2 · p. 3
  • Augmentation enlarged training and validation sets tenfold while leaving test sets unaugmented.The dataset variants included global scaling, data augmentation, and EMSC treatments.Source: Results · p. 3, Table 2 · p. 4
  • For the PLS example, test Huber loss is lowest at five components, versus 10 for CV and 9 for corrected CV; using validation minima generally selects more components and lowers test performance.The red test-loss curve reaches its minimum at five components.Source: Results · p. 3
  • Training and validation losses ended close together for both EMSC histories, while the non-augmented dataset had a rougher validation-loss curve.The reported optimized CNN settings generally used low dropout; layer two usually had at least as many kernels as layer one, except for DA EMSC.Source: Results · pp. 3, 5
  • CNNs generally perform better than PLS on RMSE and Huber loss, with correlations roughly on par; augmentation followed by EMSC gives the best performance.Data augmentation does not necessarily improve CNN RMSE, and it appears to deteriorate PLS performance unless combined with EMSC.Source: Results · p. 5
  • On the extrapolation dataset with augmentation and EMSC, CNN predictions are closer to the ideal line than PLS predictions, but this test set is not directly comparable to the others.The extrapolation dataset uses a different test set from the other datasets.Source: Results · p. 5

Discussion

CNNs outperformed the benchmark despite a tuning advantage for PLS, and augmentation followed by EMSC gave the best preprocessing results. The discussion also links extrapolation performance and learned kernel behavior to validation design and convolutional features.

Discussion

  • Data augmentation followed by EMSC gave the best preprocessing results, although EMSC nearly—but not completely—removed the added spectral variation.The authors suggest residual variation may help the network focus on invariant features while retaining EMSC baseline correction.Source: Discussion · p. 5, Figure 6 · p. 7
  • Extrapolation performance was good only when hyperparameters were optimized with an extrapolation validation set rather than a randomly selected one.The authors suggest linear and semi-linear activation functions may contribute to extrapolation, while stressing validation should mimic future performance.Source: Discussion · p. 5, Figure 5 · p. 6
  • CNN models performed better in the benchmark despite PLS tuning access to external-test information; Table 5 compares test performance across models.PLS models represent hypothetical optimal models for the selected preprocessing, while CNN tuning did not use the external test set from instrument two.Source: Discussion · p. 5, Table 5 · p. 6
  • First-layer kernels combine thresholding with smoothing and slope sensitivity, with one example responding mainly to low spectral values.The qualitative examples show activation patterns alongside the spectrum and kernels.Source: Discussion · pp. 5, 7, Figure 7 · p. 7
  • Second-layer kernels form narrower, more specific features across spectral and kernel dimensions, with some redundancy and apparent spectral-region selection.The authors note that dropout may encourage redundancy, allowing kernels to substitute when another activation is dropped.Source: Discussion · pp. 7–8, Figure 8 · p. 7
  • Despite several million weights, the models were hard to overfit; the authors suggest early convolutional layers regularize and reduce noise for later layers.The initial convolutional layers have relatively few weights and operate across spectrum segments.Source: Discussion · p. 8

Conclusion

CNNs modeled tablet compound concentration and showed little tendency to overfit. The authors report that augmentation with EMSC worked best, that CNNs surpassed PLS in nearly all tests, and that learned activations resembled familiar spectral operations.

Conclusion

  • Data augmentation and EMSC were both beneficial, and their combination worked best despite EMSC removing most of the simulated offsets.Residual augmentation remained after EMSC correction.Source: Conclusion · p. 8
  • CNNs showed little tendency to overfit and nearly all tests surpassed standard PLS models, which were treated as hypothetical optimal for the chosen component count.The CNNs also performed well in a hard extrapolation test involving higher assay values than the training set and spectra recorded on different instruments.Source: Conclusion · p. 8
  • Qualitative kernel assessments found behavior resembling smoothing, derivative or slope recognition, thresholding, and spectral-region selection.The models used dropout as a regularization technique.Source: Conclusion · p. 8

Conflict of interests

E. J. Bjerrum owns Wildcard Pharmaceutical Consulting, which is usually contracted by biotechnology and pharmaceutical companies to provide third-party services.

Conflict of interests

Loading 1710.01927v1…