PaperlayerView exact revision on arXiv

Source-linked AI summary

TasteBench: Multimodal Benchmark for Sensory Prediction, from Molecules to Sustainable Foods

Anna T. Thomas, Sohum Patnaik, Caroline Cotto, Benjamin Sanchez-Lengeling

arXiv:2610.02599v1cs.AI

TL;DR

Sensory evaluation by human panels is costly and limits fast screening of sustainable-protein formulations. TasteBench provides multimodal food-ranking and molecular taste-classification benchmarks, calibrates performance against panel reliability, and finds its best food-ranking model competitive with the median panelist.

  • Problem

    Sustainable-protein developers lack fast computational proxies for sensory similarity, which currently requires costly human-panel evaluation.

    Source: §1 · p. 1

  • Method

    TasteBench benchmarks food-level similarity ranking and molecular taste classification using multimodal features and privacy-preserving competitions.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    .661 pairwise accuracy on panelist-rated pairs is competitive with the median panelist (.650), and the best model reaches .683 across all within-category pairs.

    Source: §1 · p. 2, §5.2 · p. 7

  • Takeaways & Limitations

    The .825 reliability ceiling and human-panel comparisons provide a calibrated framework for measuring progress in computational sensory prediction.

    Source: §1 · p. 2, §6 · p. 8

    The panel is a screened two-city US consumer sample, so generalization to other markets or trained panelists is untested.

    Source: §6 · p. 9

Abstract

from arXiv · show

Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory, that accelerate drug and materials discovery. Evaluating whether a novel food tastes like its animal-based target requires expensive human sensory panels, bottlenecking the design-build-test loop. We introduce TasteBench, a multimodal benchmark and privacy-preserving competition for sensory prediction, spanning two tasks: a food-level ranking task built on 21K+ human evaluations across 215 plant-based foods in 24 product categories, yielding 935 within-category ranking pairs, and a supporting molecular-level taste classification task over 15K flavor molecules. To enable rigorous interpretation of model performance, we characterize the ground truth: inter-rater agreement among panelists is low (Krippendorff's $\alpha = .077$), and the split-half reliability ceiling of panel-aggregated rankings is .825, establishing the range within which ML systems on this benchmark should be assessed. We evaluate baselines across four input modalities; on the same pairs panelists rated, the best model achieves .661 pairwise accuracy, competitive with the median individual panelist (.650), and .683 across all within-category pairs. TasteBench provides the evaluation infrastructure and baselines for measuring progress on computational screening for sustainable protein discovery.

1 Introduction

TasteBench addresses the costly, low-throughput sensory testing that slows sustainable-protein development with a benchmark for food-level ranking and molecular taste classification. It calibrates model scores against human-panel reliability and reports results competitive with individual panelists.

1 Introduction

  • Sustainable-protein sensory evaluation relies on slow, expensive human panels, limiting how many formulations can be tested in a large combinatorial design space.A sensory predictor could rank candidates for panel testing, while closed-loop formulation optimization is described as a more ambitious future use.Source: §1 · pp. 1–2

A multimodal, multiresolution benchmark

  • TasteBench combines food-level ranking across 215 products and 935 pairs with molecular taste classification, using four feature-cost tiers in privacy-preserving public competitions.The benchmark draws on 21K+ evaluations and 15K+ molecules; decoy data protects brand privacy.Source: §1 · p. 2

Ground truth characterization for ML evaluation

  • The median panelist reaches .650 pairwise accuracy against the panel mean, inter-rater agreement is α = .077, and split-half reliability sets a .825 ceiling.The authors use these measures to define the interpretive range for model performance.Source: §1 · p. 2

Computational baselines and empirical findings

  • .661 pairwise accuracy on panelist-rated pairs is competitive with the median panelist (.650), while the best model reaches .683 across all within-category pairs.Molecular encoders with .84–.90 molecular-classification accuracy have statistically indistinguishable food-level ranking accuracy (≈.68).Source: §1 · p. 2

1 Introduction

  • The introduction identifies open questions about multimodal integration, the chemistry/semantic gap, and augmenting small psychophysical datasets with web data.Earlier work examined text-only food prediction or molecule-level taste prediction, but not transfer of molecular representations to food-level sensory prediction.Source: §1 · p. 2

2 Datasets

TasteBench combines de-identified sensory data for plant-based meat and dairy analogs with product features and a separate molecule taste dataset. Its public competitions preserve brand privacy while supporting benchmark use.

2 Datasets

  • The food benchmark evaluates 215 plant-based and vegetarian meat and dairy analogs across 24 categories, forming 935 within-category ranking pairs from NECTAR ratings.NECTAR’s 2025 and 2026 reports cover 121 meats and 94 dairy-free products; 32 animal-based products serve as references.Source: §2 · p. 3
  • NECTAR product identities are de-identified, while public sensory targets and product features support benchmarking without revealing brands.The benchmark also includes Taste Like products without sensory ratings to prevent inference about NECTAR products’ performance from public TASTY Awards.Source: §2 · p. 3
  • FoodAtlas links ingredients to associated compounds and reported concentrations, which are mapped to canonical SMILES for molecular featurization.FoodAtlas contains 285,077 triplets, including 4,318 evaluated by human experts.Source: §2 · p. 3
  • FartDB provides 15,025 molecules labeled as sweet, bitter, sour, umami, or undefined, with a held-out test split of 2,254 molecules.The dataset is heavily skewed toward sweet molecules, with only 53 umami training molecules and 6 umami test molecules.Source: §2 · p. 3

3 Benchmark Design

TasteBench evaluates sensory prediction at food and molecule scales, with a food-level ranking task and a supporting molecular taste-classification task. Its public competitions separate benchmark access from brand-identifying information.

3 Benchmark Design

  • TasteBench’s two scales comprise food ranking over 21K+ evaluations and molecule classification over 15K flavor molecules.The food data span 215 plant-based foods in 24 categories.Source: Figure 1 · p. 4, §3 · p. 4
  • Food-level prediction ranks products within categories by mean panel-rated similarity to the animal-based counterpart, using pairwise accuracy, rank correlations, and Recall@k.Inputs span ingredient text, nutrition, and molecular composition, which do not require a physical product, and appearance, which does.Source: §3 · p. 4
  • The molecular task predicts one of five taste classes from a molecule’s canonical SMILES and tests whether molecular embeddings transfer to food-level similarity prediction.The task extends prior FART work by evaluating a D-MPNN trained on the same data splits.Source: §3 · p. 4
  • Public competitions let participants submit predictions and receive scores without product identities; Taste Like products help anonymize NECTAR products.Released code, weights, and files support reproducibility, while the identity linkage is restricted and is not needed to use or reproduce the benchmark.Source: §3 · pp. 4–5

4 Baseline Models

The baseline pipeline combines nutritional, visual, ingredient-text, and molecular features to predict food similarity. It compares distance, foundation-model, supervised, and ensemble approaches for ranking products within categories.

4 Baseline Models

  • Food-level prediction draws on four modalities—nutrition, appearance, ingredient text, and molecular composition—while molecular taste classification tests whether better chemical classifiers improve downstream representations.The models predict mean similarity ratings for plant-based products relative to their animal-based counterparts.Source: §4 · p. 5, §4.2 · p. 6

4.1 Features & the Molecular Taste Sub-Task

  • Nutrition uses six macronutrients, ingredient lists become 1024-dimensional Qwen3 embeddings, and images become 1024-dimensional DINOv3 embeddings.Missing image embeddings are imputed from five nearest neighbors within the same category; category subsets encode meat, nonsweet dairy, cheese, and sweet dairy.Source: §4.1 · p. 5
  • Molecular features use FoodAtlas compounds and canonical SMILES; FART embeddings are aggregated by log concentration and ingredient label weight, then compared with D-MPNN and fine-tuned ChemBERTa-2 embeddings.The alternative encoders also test whether molecular taste-classification representations transfer to food-level ranking.Source: §4.1 · pp. 5–6

4.2 Food Similarity Models

  • Food similarity baselines include multimodal rank fusion, zero-shot Gemini and Qwen models, five supervised models, and ensembles combining Gemini with Bradley-Terry.Supervised models use leave-one-product-out cross-validation; ensemble evaluation uses nested leave-one-out cross-validation to prevent label leakage.Source: §4.2 · p. 6

5 Experiments and Results

The best ensemble exceeds the median panelist on food-level ranking, while panelist agreement is low and split-half reliability bounds model performance. Molecular taste-classification accuracy does not distinguish encoders that are statistically indistinguishable on food-level ranking.

5.1 Ground truth reliability and human performance

  • .661 accuracy on within-block pairs exceeds the median panelist’s .650 by 1.1 pp; panel agreement is low (Krippendorff’s α = .077), and split-half reliability is .825.On all within-category pairs, the best model achieves .683, matching approximately k∗≈2 panelists; .825 is an approximate ceiling for model performance.Source: §5.1 · p. 7, Table 1 · p. 7, Figure 2 · p. 7

5.2 Ensembles exceed median human similarity matching performance

  • .683 pairwise accuracy for the NNLS ensemble over Bradley-Terry and Gemini 3.1 Pro exceeds Gemini 3.1 Pro at .654 and the median untrained panelist at .650.NNLS exceeds either component alone, indicating complementary signal; the best model ranks first under micro-, macro-, and partial-pooling aggregation.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 Molecular-classification quality does not transfer to food-level ranking

  • Despite .84–.90 molecular-task accuracy, the three encoders yield statistically indistinguishable downstream food-level pairwise accuracy.On the held-out FART test split, the GNN scores .904 [.890, .915] versus FART’s .894 [.880, .906] (McNemar p=0.07); ChemBERTa-2 scores .842 [.827,.857].Source: §5.3 · pp. 7–8, Table 3 · p. 8

6 Discussion

TasteBench baselines are presented as screening tools, but molecular classification gains do not map to better food rankings; the authors also flag modality, contamination, and benchmark-scope limits.

6 Discussion

  • Current models exceed the median untrained panelist on pairwise ranking, while the .825 reliability ceiling bounds future progress.The authors describe current accuracy as practically useful for screening tools.Source: §6 · p. 8
  • Zero-shot multimodal LLMs may benefit from pretraining contamination, while text and images cannot fully capture physical texture, taste, or olfaction.Brand-blind intervention changed accuracy by only −.005 (p = .71), bounding the latent contribution.Source: §6 · p. 8
  • TasteBench invites community augmentation with large-scale web data to further close the gap to human perception.Its human reliability ceiling is presented as a calibrated framework for measuring algorithmic progress.Source: §6 · pp. 8–9
  • Across 935 within-category pairs, molecular-encoder quality does not transfer to product-level ranking when the compound block is swapped.The three encoders span six points of molecular-classification accuracy but yield statistically indistinguishable food-level ranking.Source: §6 · p. 8, Table 4 · p. 9
  • Biased models could narrow formulation choices, and overreliance on screening before sufficient reliability could prematurely eliminate promising candidates.The human baseline and reliability ceiling serve as partial safeguards against overconfidence.Source: §6 · p. 9
  • The benchmark covers meat and dairy, with a screened two-city US consumer panel; generalization to other markets or trained panelists is untested.Per-category samples are small, and evaluation focuses on similarity rather than other sensory dimensions.Source: §6 · p. 9

A Related Work

Prior taste-prediction work addressed molecular classification and text-based sensory tasks, while olfactory benchmarks offer methodological precedents. TasteBench connects molecular-level predictions to product-level sensory outcomes.

Taste Prediction

  • FART predicts sweet, bitter, sour, and umami classes from molecular tastants, achieving over 91% accuracy.Earlier molecular methods used taste-specific binary classifiers with Morgan fingerprints or molecular descriptors and tree-based models.Source: A Related Work · p. 13
  • Thomas et al. (2025) found LLMs could exceed food-scientist performance on experimental design, but used text-only inputs and did not improve foundation-model performance.TasteBench adds a food-level ranking task linking molecular predictions to product-level outcomes.Source: A Related Work · p. 13

Olfactory Prediction

  • Olfactory work provides precedents in predicting odor qualities from molecular structure, aggregating molecule embeddings for mixtures, and benchmarking real-world odor classification.The paper describes the molecule-to-mixture aggregation problem as structurally analogous to its compound-to-food pipeline.Source: A Related Work · p. 13

B Theoretical Analysis

A truncated bivariate-normal model relates a predictor’s correlation with true sensory quality and the candidate pool’s variability to expected improvement from screening. The appendix also documents data protections, feature selection, category grouping, and food-level model procedures.

B Theoretical Analysis

  • The analysis uses a truncated bivariate normal to relate predicted values above a threshold to expected improvement in the target property.S denotes the target property, P the predicted value, and ρ their correlation; the displayed proposition uses the standard normal density and cumulative distribution functions.Source: B Theoretical Analysis · p. 13
  • 32% illustrative improvement follows from a predictor correlation of 0.81 and selecting the top 5% of candidates, but realized gains depend on the unobserved internal-pool variance.The example uses a market-product variance ratio of 0.19; expected improvement increases with pool variance and predictor correlation.Source: B Theoretical Analysis · p. 13

C.2 Privacy-Preserving Kaggle Competition

  • Leaderboard scores are rounded to two decimals, submissions are limited to five daily, and an 11% private holdout is unavailable for probing and frozen afterward.These measures address fine-grained pairwise inference that could reveal panel rankings.Source: §C.2 · p. 14

C.3 Dataset Licenses and Terms of Use

  • The appendix lists NECTAR data under CC BY-NC 4.0 with NDA-gated access, Taste Like under the same gated route, and FoodAtlas under Apache 2.0.FartDB is listed under MIT, pinned to a specified commit.Source: §C.3 · p. 14

C.4 LLM prompt

  • The LLM prompt uses XML blocks for features, and feature ablation removes the corresponding blocks.The documented prompt inputs include ingredients, nutrition, and images.Source: Figure 4 · p. 15, §C.7 · p. 17

C.5 Nutrition feature selection

  • Nutrition features are selected only where nutritional distance is treated as a valid proxy for sensory distance, with subsets chosen by product type.The selected feature groups are summarized in Table 5.Source: §C.5 · p. 15, Table 5 · p. 15

C.6 Category Subsets

  • Because per-category samples are small, the 24 categories are grouped into four subsets, and each product receives a four-dimensional one-hot subset indicator.The 215 products are grouped as meat, nonsweet dairy, cheese, and sweet dairy.Source: §C.6 · p. 16

C.7 Food-Level Model Training Details

  • Food-level models use leave-one-product-out predictions, with pairwise training comparisons restricted to training-fold products; the held-out product never enters a training pair.Features are standardized and high-dimensional modalities are reduced within each fold; ensemble models use nested LOOCV to prevent partner-level label leakage.Source: §C.7 · pp. 16–17

C.8 GNN Training Details

The molecular task trains a directed-bond message-passing classifier and selects its configuration by validation macro-F1 before test-set evaluation. The reported experiments ran on CPU without GPU acceleration.

C.8 GNN Training Details

  • The classifier uses directed bond-level message passing, mean atom aggregation, and a single-hidden-layer classification network, each with hidden dimension 300.Molecular graphs are featurized from canonicalized SMILES with atom and bond features.Source: §C.8 · p. 17
  • Training uses Adam, cross-entropy loss, and validation-loss early stopping, with a maximum of 50 epochs and patience 10.The learning rate follows a Noam-like schedule; the random seed is 42.Source: §C.8 · p. 17
  • The selected GNN configuration had the highest validation macro-F1 among 12 settings, with selection committed before FART test-set or NECTAR evaluation.The selected configuration led the runner-up by 0.8 percentage points.Source: Table 6 · p. 17, §C.8 · p. 17

C.9 Compute Resources

  • All supervised, distance-based, and D-MPNN runs used CPU only on a single MacBook Pro; no GPU acceleration was used.Reproduction takes under a few hours, plus API calls for LLM evaluation.Source: §C.9 · p. 18

D Reproducibility

The appendix inventories reproducibility artifacts and reports further analyses of feature subsets, category variation, aggregation choices, molecular classes, and category-balanced evaluation. Across these analyses, small category samples and overlapping uncertainty intervals constrain interpretation of subgroup differences.

D Reproducibility

  • Deterministic out-of-fold predictions and scripts support reviewer verification without external data or model training, while regenerating food-level predictions requires gated NECTAR access.The appendix also inventories LLM outputs, molecular checkpoints, embeddings, and confidence-interval caches.Source: D Reproducibility · pp. 18–19

E.1 Model Ablation

  • The feature-subset ablations find no smaller input set matching the headline SNCTI and ingredients-plus-image configurations within their confidence intervals.The sweeps cover 15 non-empty feature subsets for supervised and distance models and seven non-empty subsets for LLMs.Source: §E.1 · p. 19, Table 7 · p. 20, Table 8

E.2 Per-category model comparison

  • Performance varies substantially across the 24 categories, but small category samples yield wide confidence intervals, so category-level leaders warrant caution.The reported category breakdown covers models from the main results table.Source: §E.2 · p. 20, Table 9 · p. 20

E.3 Gemini ensemble comparison

  • Table 10 compares supervised models alone with their combinations with Gemini 3.1 Pro using a nested-LOOCV NNLS meta-learner.The supplied passage describes the comparison scope but not its outcome.Source: §E.3 · p. 21, Table 10 · p. 22

E.4 Per-category NNLS metrics

  • Across categories, NNLS achieves R@1 = .292, R@2 = .500, and R@3 = .542; expected screening uplift scales with the product of sensory variability and predictor correlation.The table also reports category-level inputs and evaluation metrics for the NNLS ensemble.Source: §E.4 · p. 22, Table 11

E.5 Ingredient Aggregation Ablation

  • Accuracy differences among five compound-to-ingredient aggregation rules fall within 95% CI overlap, so log-concentration weighting is adopted for theoretical rather than empirical advantage.The comparison covers Bradley–Terry and the BT+Gemini NNLS ensemble on within-block and all-pairs scopes.Source: §E.5 · p. 23, Table 12 · p. 24

E.6 Per-class breakdown for molecular prediction

  • FART and the D-MPNN have overlapping per-class F1 confidence intervals; on umami, their apparent precision–recall differences are uncertain with only six test molecules.One additional correct umami prediction shifts F1 by approximately 0.17.Source: §E.6 · p. 24, Table 13 · p. 24

E.7 Category-balanced evaluation and human comparison

  • The best model ranks first under micro, macro, and partial-pooling aggregation, while macro-averaging lowers model scores by .000 to .055 and largely preserves ordering.The macro–micro ordering correlation is .881; the partial-pooling estimate is .682.Source: §E.7 · p. 24, Table 14 · p. 25, Table 15 · p. 25

E.8 Recognition and contamination audit

The audit finds little evidence that product recognition or contamination explains the LLM’s performance, but reasoning-based tests cannot rule out latent use of product reputation.

Recognition audit

  • 12.5% of pairs prompted product identification and 2.4% reputation references, neither of which was associated with higher accuracy.The cleaned ingredient text contains no brand tokens, and images show plated food, so identifications were inference rather than input leakage.Source: §E.8 · p. 25

Margin decomposition

  • Recognized pairs did not show a larger Gemini-over-Bradley–Terry margin; Bradley–Terry scored .672 on recognized pairs versus .605 on the rest.The interaction was not significant, and the stronger Bradley–Terry score on recognized pairs indicates that those pairs were intrinsically easier.Source: §E.8 · p. 25

Interventional control

  • The brand-blind prompt changed accuracy by −.005 (.639 to .634, McNemar p = .71), while brand mentions fell from 10.8% to 1.7%.Both prompts were rerun in the same session because the unconstrained prompt flips ∼29% of predictions across runs at temperature 0.Source: §E.8 · pp. 25–26

Training-data contamination

  • The dairy TASTY winners were released after both model releases, and Gemini’s documented January 2025 training cutoff predates the March 2025 meat winners.The dairy-over-meat accuracy gap is also present for the memorization-free Bradley–Terry baseline.Source: §E.8 · p. 26

Caveats

  • The recognition tests rely on surfaced reasoning and cannot exclude latent use of product reputation when it correlates with true taste performance.Post-release NECTAR collections are proposed as a prospective hold-out.Source: §E.8 · p. 26

F Architecture with Worked Example

The worked example traces how TasteBench combines a supervised multimodal scorer with a zero-shot LLM scorer to rank products within a category. The LLM reads raw non-molecular modalities, while the supervised model uses encoded and fused features.

F Architecture with Worked Example

  • Four modalities are encoded into vectors, standardized, and concatenated as SNCTI input to Bradley–Terry, which is fit on within-category feature-pair differences.Text, image, and compound blocks are reduced by per-modality PCA to 95% explained variance within each LOOCV fold; nutrition and category-subset blocks are not PCA-reduced.Source: F Architecture with Worked Example · p. 26
  • The ensemble combines supervised Bradley–Terry and zero-shot LLM scores by non-negative least squares to produce a predicted within-category ranking.Figure 5 illustrates the pipeline for one plant-based burger in a ten-product category.Source: F Architecture with Worked Example · pp. 26–27, Figure 5 · p. 28
  • The LLM reads raw non-molecular inputs rather than encoded or fused features, and its score is a product’s win rate over within-category comparisons.The deployed ensemble uses ingredients plus image; ingredients plus nutrition plus image is evaluated as an ablation.Source: F Architecture with Worked Example · p. 27, Figure 5 · p. 28
Loading 2610.02599v1…