PaperlayerView exact revision on arXiv

Source-linked AI summary

What an odour descriptor corpus can and cannot measure: valence, attenuation, and the ceiling of the public record

Stylianos Kampakis, Fabio Rovai

arXiv:2609.12875v1cs.LG

TL;DR

Public odour descriptor corpora share molecules and words, but whether those words measure comparable variables and what the record can measure remain open questions. The paper audits the corpora using conditional and construct analyses and benchmarks against a reliability-adjusted human panel. The descriptor record captures only a limited share of perceptual similarity, while pleasantness ratings supply a dimension it lacks.

  • Problem

    Shared molecules and descriptor words in public odour corpora do not establish that the recorded variables are commensurate, leaving both comparability and the record’s measurement ceiling unresolved.

    Source: §1 · p. 1

  • Method

    The paper audits pinned public corpora, uses matched molecule–descriptor observations to separate criterion from construct, and measures prediction against a reliability-adjusted human-panel criterion.

    Source: §1 · p. 2, §2.2 · p. 3, §2.3 · p. 3

  • Results

    Five people rating pleasantness exceed molecular structure plus the full descriptor record, while the record’s missing perceptual dimension cannot be recovered by inference.

    Source: §5 · p. 7

  • Takeaways & Limitations

    For benchmarking resources, pooling needs correction for systematic offsets and descriptor-level heterogeneity, while valence should be collected at source rather than derived.

    Source: §5 · p. 7

    Only four descriptors are shared often enough across both corpora and an independent human panel to score, preventing a downstream demonstration.

    Source: §5.1 · p. 8

Abstract

from arXiv · show

Machine olfaction trains on pooled public descriptor corpora, but whether a shared descriptor word measures the same thing across corpora has not been tested, nor has the ceiling of what any of them can measure. We audit four corpora from Pyrfume. Conditioning on the molecule makes McNemar's test the exact conditional test of the corpus effect. Corpora disagree heterogeneously across descriptors ($I^2 = 80\%$) and non-uniformly with labelling breadth ($z = 17.2$), so no single offset repairs pooling. Median tetrachoric agreement is 0.795 against median $\kappa$ of 0.413: sources largely concur on which molecules deserve a word and differ on how readily they apply it. Of 109 descriptors with an estimable effect, 36 show large differential functioning on the ETS scale. Against a human panel's reliability, Morgan fingerprints with the full RDKit descriptor block reach 32.9\% of achievable; adding every label from two merged corpora reaches 33.9\%. The gap does not close with model capacity, encoding choice, more molecules, or more words. The missing variance is valence. One pleasantness rating per molecule reaches 54.6\% of achievable (57.1\% on an independent older instrument). Valence recovered from descriptors ($\rho = 0.457$) yields only 16.6\%, so it must be measured. Five raters exceed structure plus the full descriptor record; fifteen to twenty saturate. We release a descriptor crosswalk and twenty machine-checked theorems.

1 Introduction

The paper asks whether shared descriptor words are comparable across public odour corpora and what those records can measure. It separates criterion from construct, quantifies the perceptual ceiling, and identifies valence as the missing dimension.

1 Introduction

  • 111 of 666 GoodScents terms match Leffingwell’s 113 after normalisation, leaving 555 terms that a merge either discards or invents.Matching vocabulary alone does not establish that shared terms mean the same thing.Source: §1 · p. 1
  • The study tests whether shared descriptor words across public corpora measure comparable things and what the record can measure at all.The paper computes its reported results from pinned sources.Source: §1 · pp. 1–2
  • Structure and every descriptor label in two merged corpora reach about a third of achievable performance, while one pleasantness rating reaches more than half.Figure 1 expresses performance as a share of the panel criterion’s reliability; it also shows replicates from an instrument three decades older.Source: Figure 1 · p. 2
  • The paper contributes an exact conditional analysis, a criterion–construct distinction with equating constants, and a measure of how much perceptual similarity the record explains.It also identifies valence as the missing majority and assesses the measurement needed to supply it.Source: §1 · p. 2

2 Materials and methods

The study audits four Pyrfume corpora and uses matched molecule–descriptor observations to separate corpus criterion effects from construct agreement. It evaluates prediction against a reliability-adjusted human-panel criterion.

2.1 Corpora

  • The audit uses Dravnieks, Keller and Vosshall, Leffingwell, and GoodScents as distributed in Pyrfume.Leffingwell and GoodScents are the only well-powered pair, sharing 2,225 molecules and 111 normalised descriptors; both are trade compilations without human observers.Source: §2.1 · p. 2

2.2 The exact conditional analysis

  • Each molecule–descriptor cell has two corpus observations, so conditioning on their pair total removes molecule and molecule-by-descriptor nuisance effects and yields McNemar’s exact test.The conditional maximum-likelihood estimate is the discordant log odds ratio; descriptors with zero discordant cells are reported separately as unbounded.Source: §2.2 · p. 3

2.3 Criterion and construct

  • A saturated bivariate probit estimates two thresholds for criterion differences and a tetrachoric correlation for construct agreement.It is exactly identified with zero degrees of freedom and assumes no latent molecule trait, local independence, or unaffected anchor items.Source: §2.3 · p. 3

2.4 The perceptual criterion

  • The perceptual criterion is the correlation between mean molecule profiles from 55 individual raters, with performance reported as a share of the panel reliability permits.Reliability comes from the panel’s split half, corrected to full-panel size; structure uses Morgan fingerprints and the RDKit physicochemical block, with molecule-grouped splits to prevent pair leakage.Source: §2.4 · p. 3

3 Results

The corpora show both substantial descriptor-level variation in conditional effects and high construct agreement. The results distinguish differences in how readily sources apply words from disagreement over which molecules merit them.

3.1 Corpora disagree heterogeneously and non-uniformly

  • Across 2,225 shared molecules, mean descriptor-set Jaccard agreement is 0.359, and only 5.3% have identical positive descriptor sets.Identical identifiers and vocabulary do not imply identical labels.Source: §3.1 · p. 3
  • I2 = 80% and τ = 0.52, with a prediction interval of 0.39 to 3.21; the pooled difference is negligible, but descriptor effects vary substantially.Cochran’s Q is 528 on 108 degrees of freedom, and the figure displays descriptor-level effects with the random-effects pooled estimate and prediction interval.Source: §3.1 · pp. 3–4, Figure 2 · p. 4
  • The conditional log odds ratio shifts from −0.511 for molecules with one to four descriptors to +0.485 for those with seven to fifteen, with a trend of +0.390 per stratum (z = 17.2).The direction of the corpus effect turns with labelling breadth, so a per-descriptor offset is insufficient.Source: §3.1 · p. 4

3.2 The disagreement is criterion, not construct

  • Median tetrachoric agreement is 0.795 (bootstrap interval 0.768 to 0.821), versus median Cohen’s κ of 0.413; ρ supplies agreement level, not descriptor ordering.The correlation between ρ and κ is 0.865 by both Pearson and Spearman measures.Source: §3.2 · p. 4

3.3 Differential item functioning and equating

The public record reaches only about a third of achievable perceptual similarity, and its unexplained share is not closed by added descriptor labels. The analysis also reports descriptor-level differential functioning and equating constants.

3.3 Differential item functioning and equating

  • Of 109 estimable descriptors, 56 show negligible, 17 intermediate, and 36 large differential functioning on the ETS delta scale.The largest effects are for rum, gasoline, chamomile, aldehydic, and catty.Source: §3.3 · p. 4
  • The paper reports 110 probit-threshold equating constants with median absolute value 0.102 and range −0.512 to +1.515.These are shifts between criteria; item-response equating methods are not performed or claimed because a unidimensional model is misspecified for these descriptors.Source: §3.3 · p. 4

3.4 The ceiling of the public record

  • 32.9% of achievable performance comes from the full RDKit structural block, rising to 33.9% with every label in two merged corpora; the gap does not close with model capacity, encoding, or volume.Tanimoto on Morgan fingerprints alone reaches 14.6%; the descriptor contribution is flat in molecules labelled and vocabulary size.Source: §3.4 · p. 5

3.5 The missing variance is valence

Pleasantness identifies valence as the missing perceptual axis, and descriptor-derived valence recovers only a fraction of what direct ratings capture. The paper tests this conclusion against alternative explanations and an independent instrument.

3.5 The missing variance is valence

  • Splitting the 55 subjects between affective ratings and criterion removes the shared-rater confound.Source: §3.5 · p. 5
  • Dropping pleasantness collapses the affective set from 0.504 to 0.148, while dropping intensity or familiarity moves it by under 0.01, identifying valence as the axis.Keller records pleasantness, intensity, and familiarity as non-character responses; trade corpora record none of them.Source: §3.5 · p. 5

3.6 Five objections

  • 57.1% of achievable against 28.3% for structure plus every descriptor label is reached by Dravnieks’s evaluative-descriptor proxy on 64 shared molecules.The proxy comes from a different panel, decade, and response format.Source: §3.6 · p. 5
  • 16.6% of achievable is reached by valence recovered from corpus labels, versus 53.9% for measured valence on the same molecules and folds.Labels predict pleasantness out of fold at ρ = 0.457, but the recovered coordinate remains insufficient.Source: §3.6 · p. 5
  • The observed 0.486 exceeds all 200 molecule-identity shuffles, whose mean is +0.002 and maximum +0.048, under a null preserving the valence marginal.Source: §3.6 · p. 5
  • Figure 3 reports that descriptor labels add nothing to chemistry against a mouse olfactory bulb but add reliably against a human panel.Bars are 95% intervals over molecule-grouped folds.Source: Figure 3 · p. 6

3.7 How much measurement is needed

Five pleasantness raters exceed structure plus the full descriptor record, with gains saturating around fifteen to twenty. A tetrachoric reanalysis identifies valence as an interpretable leading factor, while the evidence still rejects a unidimensional item-response model.

3.7 How much measurement is needed

  • Five raters exceed structure plus every descriptor label; moving from five to twenty-seven buys only 1.7 points, making fifteen to twenty the practical point.The paper reports this as the only axis on which collection buys anything, with gains that then stop.Source: §3.7 · p. 6

3.8 A correction to our own dimensionality evidence

  • 21.2% of trace is the tetrachoric leading factor, whose molecule scores correlate with measured pleasantness at −0.473.The factor loads negatively on apple, floral, honey, and rose, and positively on coffee, savory, onion, and sulfurous; the earlier Pearson-based estimate was 3.2%.Source: §3.8 · p. 6
  • The refusal of a unidimensional item-response model stands: the first-to-second eigenvalue ratio is 1.22, and 53 of 111 eigenvalues exceed one.Source: §3.8 · p. 6

3.9 Confirmation on an untouched pair

A preregistered analysis on an untouched corpus pair confirmed heterogeneous corpus effects and attenuation, but its prediction of a small average effect failed. The authors conclude that pooling needs correction for both a systematic offset and descriptor-level spread.

3.9 Confirmation on an untouched pair

  • The specification named an unused pair and four numbered predictions before execution; the analysis ran once on 834 shared molecules and 158 shared descriptors.Source: §3.9 · p. 6
  • The small-average-effect prediction failed: the mean corpus effect was +1.426 in log odds against a threshold of 0.5.Source: §3.9 · pp. 6–7
  • 64.1% heterogeneity was confirmed, with Q = 317 on 114 degrees of freedom; the attenuation gap was +0.463 and median tetrachoric construct agreement was 0.835.Source: §3.9 · p. 6
  • The finding that these examined corpora agree on average is pair-specific; pooling must correct for a large systematic offset and descriptor-level spread that no offset removes.Source: §3.9 · p. 7

4 Formal results

The formal results characterize exact conditional testing, recoverability, representational capacity, and aggregation. Their theorems are machine-checked in Lean 4.

4 Formal results

  • Twenty theorems are machine-checked in Lean 4 against Mathlib, with no sorry and no axioms beyond the three standard ones.Source: §4 · p. 7
  • McNemar’s test is the exact conditional test of the corpus effect because conditioning cancels the nuisance term; the discordant log odds ratio is its conditional maximum-likelihood estimate.Source: §4 · p. 7
  • A crosswalk can recover a target from a recording if and only if the recording never merges states the target must distinguish.One merged pair refutes every possible crosswalk, distinguishing reversible calibration failure from irreversible information failure.Source: §4 · p. 7
  • A recording of k binary descriptors takes at most 2^k values, so a target requiring more distinctions is unreachable by any crosswalk.The bound does not depend on the number of raters.Source: §4 · p. 7
  • A three-rater majority applies a label with probability 3p^2 − 2p^3; under a no-signal model, this is prevalence shrinkage that a different threshold can undo.A fixed vote requirement drives agreement to one, whereas unanimity drives it to zero.Source: §4 · p. 7

5 Discussion

The public descriptor record omits valence, and adding compounds or quality words does not close the gap; the discussion recommends collecting valence directly. The main effect rests on one criterion dataset, and the record cannot presently validate its own labels.

5 Discussion

  • Five people rating pleasantness can exceed structure plus the full descriptor record, while fifteen to twenty raters saturate the measurement.The discussion concludes that the dominant missing dimension must be measured rather than inferred from the public descriptor record.Source: §5 · p. 7
  • Pooling requires correction for a systematic offset and heterogeneous descriptor-level effects, while a versioned crosswalk makes unmatched vocabulary visible.The released crosswalk includes all 668 terms: 111 matched and 557 retained as unmatched.Source: §5 · p. 7

5.1 Limitations

  • The main effect rests on one criterion dataset, with external replication on 64 molecules, and the criterion measures word application rather than perception unmediated by language.PubChem CID links do not encode stereochemistry, concentration, solvent, or purity; the Dravnieks concentration field is reproduced by a stimulus-name substring test.Source: §5.1 · p. 7
  • The record cannot presently validate its own labels because only four descriptors support downstream scoring and no public human perceptual criterion is large enough.The only two human descriptor panels share two usable descriptors, and training-regime differences are around 0.005 in AUC.Source: §5.1 · p. 8

Data and code availability

The source snapshot is pinned and its files were checked against a manifest, but raw rows and analysis code are not publicly released. The appendix gives procedures sufficient for reimplementation.

Data and code availability

  • The source snapshot is pinned to a commit, and every file was checked against a manifest of byte counts and digests.The source snapshot is identified so inputs can be retrieved independently.Source: Data and code availability · p. 8
  • Raw corpus rows are not redistributed because reuse positions do not permit it, and the analysis code and derived reports are not publicly released.Source: Data and code availability · p. 8
  • The appendix gives procedures in sufficient detail to reimplement the analysis, including seven pitfalls that produced incorrect development results.Source: Data and code availability · p. 8
Loading 2609.12875v1…