Source-linked AI summary
Decision Tree and K-Means Analysis of Raman Spectra for Edible Oils: A Physics-Informed AI Approach
Amrita Shaw, Chandrasekar S. N., Sai Muthukumar V., Jhinuk Gupta, Deepak L. N. Kallepalli
TL;DR
Edible-oil authentication in processed foods requires methods that address food-matrix interference while remaining accurate and interpretable. The study combines Raman spectroscopy with unsupervised structure analysis, Decision Trees, and NNLS-based Physics-Informed AI. Pure oils were perfectly classified with four variables, while NNLS correction improved matrix-sample performance and reduced model complexity.
Problem
Edible-oil authentication matters for food quality, safety, and compliance, but matrix interference can reduce spectral separability and classification performance.
Method
The framework uses t-SNE and K-means to examine spectral organization, Decision Trees for interpretable classification, and NNLS decomposition to separate additive matrix contributions.
Results
86.4% and 85.4% accuracy were achieved for paper-subtracted and paper-plus-potato-subtracted datasets, respectively, compared with approximately 64.4% for original chips spectra.
Takeaways & Limitations
The results support physically meaningful, interpretable, and compact Raman representations for Frugal AI, Edge AI, and embedded food-quality monitoring.
Takeaways & Limitations
Original food-matrix spectra remained limited by matrix-induced overlap, and t-SNE was used only for qualitative visualization rather than clustering.
Abstract
from arXiv · showhide
Authentication of edible oils in processed foods is important for food quality, fraud prevention, and regulatory compliance. This study establishes an integrated Raman spectroscopy and machine-learning framework that links intrinsic spectral organization, interpretable classification, and Physics-Informed Artificial Intelligence (PI-AI). Five edible oils were investigated in pure form and within a fried-potato-chip matrix using t-SNE, K-means clustering, Decision Trees, and Non-Negative Least Squares (NNLS)-based spectral decomposition. Unsupervised analyses revealed substantially stronger class organization and separability in pure oils, whereas food-matrix effects introduced pronounced spectral overlap. Decision Trees achieved 100% classification accuracy for pure oils using only four Raman variables from the original 1866-feature spectral space. These four variables, consistently identified by both pre-pruned and post-pruned models, represented only approximately 0.21% of the available spectral information while retaining perfect test-set performance. For matrix-containing samples, NNLS-based PI-AI spectral decomposition substantially improved classification by separating oil-related signatures from paper and potato contributions. Optimized post-pruned models achieved accuracies of 86.4% and 85.4% for paper-subtracted and paper-plus-potato-subtracted datasets, respectively, while reducing the number of important Raman variables to only five and four. The compact four-feature representation further reduced the data footprint by 99.44% without loss of classification accuracy. Collectively, these findings demonstrate that accurate Raman-based oil identification can be achieved through physically meaningful, highly compact, and interpretable spectral representations, providing a promising foundation for Frugal AI, Edge AI, portable sensing, and embedded food-quality monitoring.
1. Introduction
The study addresses whether Raman classification performance reflects intrinsic spectral organization or merely the selected learning algorithm. It combines unsupervised structure analysis, interpretable Decision Trees, and physics-informed spectral modeling to improve understanding across pure oils and food matrices.
- Motivation: Raman spectroscopy offers a rapid, non-destructive alternative to laboratory methods that generally require extensive preparation and solvent extraction.Conventional approaches include gas chromatography, high-performance liquid chromatography, and nuclear magnetic resonance.
- Research gap: The unresolved question is whether predictive success arises mainly from learning-algorithm choice or natural class structure in Raman spectral space.This distinction matters for developing accurate, interpretable, and transferable models across food matrices.
- Study approach: t-SNE and K-means evaluate natural grouping and separability before Decision Trees generate transparent predictive models.The workflow combines unsupervised and supervised learning for Raman-based edible-oil identification.
- Physics-informed modeling: Non-Negative Least Squares provides a physics-informed framework by preserving additive spectral mixing and enforcing non-negative component contributions.This models measured Raman spectra as combinations of multiple chemical constituents.
- Deployment rationale: Compact and efficient spectral representations align the framework with Frugal AI for resource-constrained deployment.Frugal AI emphasizes reliable performance with reduced data requirements, model complexity, computational cost, and energy consumption.
2.1 Sample Collection and Preparation
Five commercially available edible oils were analyzed in pure form and after extraction from a representative fried-food matrix. Four datasets were constructed to compare ideal pure-oil measurements, fried-food spectra, and NNLS-corrected matrix conditions.
- Sample collection: Five oils were investigated: sunflower, soybean, groundnut, palm, and vanaspati oil.Samples included both pure oils and oils extracted from a representative fried-food matrix.
- Protocol basis: The sampling strategy and Raman measurement protocol followed methodologies established in previous studies.Those studies demonstrated solvent-free Raman analysis for edible-oil authentication in pure and food-matrix conditions.
- Dataset construction: Four datasets represented pure oil, fried food, paper-subtracted fried food, and paper-plus-potato-subtracted fried food conditions.The corrected datasets were constructed using NNLS-based removal of matrix contributions.
2.2 Raman Spectral Acquisition
Raman spectra were acquired using a rapid, solvent-free approach with minimal sample preparation. Measurements covered approximately 500–3000 cm⁻¹ to capture molecular-vibration bands characteristic of edible oils.
- Acquisition procedure: Measurements used rapid, solvent-free sampling requiring minimal sample preparation.The instrumentation and acquisition parameters followed previous studies.
- Spectral coverage: The spectral range covered approximately 500–3000 cm⁻¹ across fingerprint and high-wavenumber regions.This range captured bands associated with C=C stretching, CH₂ deformation, and other edible-oil vibrations.
2.3 Spectral Preprocessing
Spectral preprocessing corrected negative intensities introduced during spectral correction and prepared comparable non-negative profiles for clustering and classification. A global baseline shift preserved relative spectral relationships before visual inspection and downstream analysis.
- Baseline correction: Negative intensities reached −7.58 in oils and −20.35 in chips before preprocessing.These values motivated a global baseline shift before machine-learning analysis.
- Non-negative transformation: Adding the absolute minimum intensity produced non-negative spectral distributions for subsequent K-means analysis.The transformation preserved relative spectral relationships.
- Quality inspection: The processed spectra were visually inspected using mean Raman plots for each oil class before clustering and classification.These processed profiles served as inputs to all reported analyses.
2.4 Physics-Informed Spectral Decomposition Using NNLS
NNLS provides a physics-informed decomposition that represents Raman spectra as additive, non-negative combinations of reference components. The resulting coefficients and background-subtracted spectra support chemically meaningful downstream classification of oil-containing fried-food samples.
- Physics-Informed Spectral Decomposition: NNLS models each observed Raman spectrum as a linear combination of reference spectral components with non-negative coefficients.This constraint reflects the physical reality that Raman contributions and chemical concentrations cannot be negative.
- Feature Extraction: The NNLS coefficients were used as physics-informed features for downstream machine-learning analysis.This preserves chemically meaningful relationships while incorporating prior knowledge about additive spectral mixing.
- Physics-Informed Spectral Decomposition: Each fried-food spectrum was decomposed using reference spectra for oil, paper, and potato contributions.The decomposition represents the measured signal as an additive combination of three physically meaningful components.
- Background Subtraction: Two corrected datasets isolated oil-related signals by subtracting either paper alone or both paper and potato contributions.The paper-subtracted dataset retained oil and potato contributions, whereas the paper-potato-subtracted dataset preferentially retained oil signal and residuals.
2.5 Computational Environment and Reproducibility
The study used a controlled computational environment and shared analysis materials to support reproducibility. The same workflow covered preprocessing, spectral analysis, NNLS decomposition, clustering, and Decision Tree modeling.
- Computational Environment: All preprocessing, spectral analysis, NNLS decomposition, unsupervised learning, and Decision Tree modeling were performed in Python-based notebook and VS Code environments.The workflow was implemented in Jupyter Notebook and Visual Studio Code.
- Computational Environment: Identical software versions and package dependencies were maintained across analyses to ensure consistency and reproducibility.The stated environment included Python 3.13.9 with Anaconda, Pandas 2.3.3, and NumPy.
- Reproducibility: Jupyter notebooks and corresponding VS Code project files were provided for preprocessing, clustering, NNLS decomposition, feature extraction, and machine-learning analysis.The materials were intended to support reproducibility, future deployment, and practical implementation.
- Analysis Workflow: t-SNE was used to visualize local organization in standardized high-dimensional Raman data before clustering or classification.The embeddings were exploratory and were not used as direct inputs to K-means.
- Observed Spectral Organization: Pure-oil spectra formed more compact and distinguishable class regions than fried-food spectra, which were broader and more intermingled.The contrast was attributed in the passages to matrix-derived contributions and frying-induced compositional changes.
3.2 Influence of t-SNE Perplexity on Cluster Stability
The t-SNE perplexity analysis tested whether apparent spectral organization depended on neighborhood-scale parameter choice. Pure-oil separation remained stable, whereas fried-food overlap persisted across perplexities and dimensions.
- Perplexity: Perplexity controls the effective number of local neighbors considered when t-SNE constructs pairwise relationships.Lower values emphasize local neighborhoods, while higher values incorporate a broader view of the data manifold.
- Pure-Oil Stability: Pure-oil class organization remained largely preserved across perplexities of 5, 10, 20, 40, 50, and 75.Sunflower, groundnut, and vanaspati oils consistently occupied relatively distinct regions.
- Fried-Food Stability: Fried-food embeddings consistently showed broader distributions and greater inter-class overlap across the tested perplexity values.Changing perplexity altered visual arrangement but did not substantially improve separation.
- Three-Dimensional Validation: Three-dimensional t-SNE confirmed the two-dimensional trend: pure oils remained more compact and distinguishable, while fried-food samples stayed comparatively diffuse.The three-dimensional embeddings used a perplexity of 50.
- Overall Stability: Across neighborhood assumptions, stable pure-oil embeddings and persistent fried-food overlap supported genuine differences in spectral organization rather than visualization artifacts.The analysis therefore treated the separability contrast as robust to the tested perplexity choices.
3.3 Elbow Analysis and Silhouette Scores for K-Means Clustering
K-means quantified the spectral organization suggested by t-SNE using compactness and separation metrics. Pure-oil spectra showed stronger intrinsic clustering than fried-food spectra, supporting a link between spectral organization and downstream learning effectiveness.
- K-Means Method: K-means partitions standardized Raman samples into similarity-based clusters without using true labels.The analysis used the elbow method and silhouette scores to assess cluster compactness and separation.
- Elbow Analysis: WCSS measures the total squared distance between samples and their assigned cluster centroids.Lower WCSS indicates greater compactness, and elbow analysis identifies where additional clusters yield diminishing improvement.
- Elbow Analysis: WCSS decreased with increasing cluster number, with the pure-oil reduction becoming more gradual after approximately five clusters.Five clusters were selected for subsequent comparison with the five investigated oil categories.
- Silhouette Analysis: Silhouette analysis evaluates how well each sample fits its assigned cluster relative to neighboring clusters.Values range from -1 to +1, with higher values indicating stronger cohesion and separation.
- Results: Pure-oil silhouette scores remained substantially higher than fried-food scores across all tested cluster numbers.This quantitative pattern matched the stronger compactness and separation observed for pure oils in t-SNE.
- Overall Interpretation: Elbow and silhouette analyses together indicated stronger intrinsic clustering in pure-oil spectra than in fried-food spectra.Their agreement supported a close relationship between spectral organization and subsequent machine-learning effectiveness.
3.4 K-Means Cluster Composition in Pure Oils and Fried-Food Samples
K-means recovered the intrinsic class organization of pure-oil Raman spectra with high fidelity, whereas fried-food spectra showed broader overlap and weaker class resolution. The cluster assignments therefore supported the t-SNE-based distinction between compact pure-oil groups and matrix-affected samples.
- Pure oils: K-means recovered dominant pure-oil class structure with high fidelity, concentrating GNO, SO, and VO mostly within single clusters while PO and SOYO showed limited overlap.The clustering was not perfectly one-to-one, with overlap likely associated with similarities in lipid composition and unsaturation.
- Pure oils: Strong agreement between true-label and K-means projections showed that the dominant spectral organization identified by t-SNE was also present in the original high-dimensional feature space.Figure 2 contains true oil labels, whereas Figure 4 contains K-means assignments projected onto the same t-SNE coordinates.
- Fried-food samples: Fried-food spectra formed less compact clusters with broader class boundaries and more pronounced overlap than pure-oil spectra.This pattern agreed with weaker t-SNE separation and lower silhouette scores for the fried-food dataset.
- Fried-food samples: 79 SOYO spectra occurred in Cluster 0 and 82 in Cluster 2, preventing a distinct SOYO-dominant cluster from appearing in the fried-food visualization.SOYO was therefore distributed across multiple clusters and substantially mixed with other oil categories.
- Overall comparison: Overall, pure-oil spectra exhibited stronger intrinsic class organization, while food-matrix contributions spread classes across multiple clusters and weakened natural boundaries.The progression from visualization to clustering metrics and label composition provided a basis for subsequent supervised classification.
3.5 Decision Tree Classification: Comparison Between Pure Oils and Fried-Food Samples
Decision Trees classified pure oils perfectly, but original fried-food spectra generalized poorly because matrix interference weakened spectral separability. NNLS-based matrix correction and pruning improved matrix-sample classification while reducing feature requirements, and pure-oil accuracy remained perfect with only four Raman variables.
- Baseline performance: 100% accuracy, precision, recall, and F1-score were achieved on the independent pure-oil test set, compared with 62.59% test accuracy for original chips spectra.The pure-oil result matched compact class organization, whereas the chips result showed a substantial train-test gap consistent with matrix-induced complexity.
- Matrix correction: 77.04% test accuracy followed paper subtraction, increasing from 62.59% by approximately 14%.The improvement was obtained through NNLS-based spectral representation without changing the classifier architecture, training strategy, or hyperparameters.
- Pre-pruning: 100% test accuracy was retained after pre-pruning for pure oils, showing that all five classes remained completely separated with a simpler tree.The result indicates that pure-oil discrimination did not require a highly complex decision-tree structure.
- Pre-pruning: 64.8% test accuracy for original chips spectra represented only a modest increase from 62.6% after pre-pruning, while matrix-corrected datasets reached 73.7% and 76.7%.Residual matrix-induced overlap continued to produce ambiguous boundaries in the original chips dataset.
- Compact pure-oil model: Four Raman variables near ~1127, ~1273, ~1322, and ~1649 cm⁻¹ achieved perfect pure-oil classification from 1866 original features.The four bands contributed nearly equally, combining complementary information from characteristic lipid vibrations rather than relying on one dominant marker.
- Compact pure-oil model: 99.44% data-footprint reduction was achieved by the four-feature representation while maintaining 100% test-set accuracy.The reduced dataset occupied approximately ~81 KB versus ~14.3 MB for the full 1866-feature dataset.
- Matrix effects: Matrix-containing classifiers required more features because potato and paper contributions obscured oil-specific Raman signatures.The original chips pre-pruned model used approximately 29 non-zero important features, compared with 15 after paper subtraction and 11 after both subtractions.
4. Conclusions and Future Outlook
The study integrates spectral-structure analysis, interpretable classification, and NNLS-based PI-AI to identify edible oils using compact Raman representations. Results support physically meaningful and computationally efficient analysis, while motivating broader validation across food matrices and processing conditions.
- The framework combines t-SNE, elbow and silhouette analyses, K-means clustering, interpretable Decision Trees, and PI-AI for edible-oil analysis.
- Only approximately 0.21% of the original Raman feature space was required to discriminate five edible oils, with the same four variables selected by pre-pruned and post-pruned models.
- 86.4% and 85.4% accuracies were achieved for paper-subtracted and paper-plus-potato-subtracted datasets, while important variables decreased to five and four, respectively.NNLS-based spectral decomposition separated paper and potato contributions using non-negative, additive Raman components.
- The results indicate that classification performance depends strongly on spectral information quality and organization, with NNLS preprocessing often improving performance more than classifier optimization alone.
- Future work should test additional matrices, adulterated and recycled oils, blended formulations, varied frying and storage conditions, and lightweight real-time analytics.