PaperlayerView exact revision on arXiv

Source-linked AI summary

English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck

Vassili Philippov, Amro Salman, Dmitrii Andreev, Penny Hands, Emil Kaiumov, Pavel Katunin, Anton Nikolaev

arXiv:2609.17554v1cs.CL

TL;DR

English all-words WSD now faces a label-quality bottleneck: frontier models approach saturation while annotation errors and fine-grained senses affect benchmark rankings. The paper introduces auditable benchmark repair and evaluation, shows gains from repairing training labels, and finds that coarse granularity improves agreement and accuracy while cost becomes binding.

  • Problem

    As frontier models approach benchmark saturation, residual label errors and fine-grained sense distinctions make WSD rankings and progress difficult to interpret.

  • Method

    The paper builds lexEN as a human-adjudicated correction layer, SenseBench as an auditable evaluation harness, and repaired training corpora for unchanged supervised systems.

  • Results

    Frontier LLMs cluster near 95% on lexEN-v1, while relabeling training data lifts classic systems by several F1 points on untouched test sets.

  • Takeaways & Limitations

    Benchmark labels and sense granularity now constrain WSD measurement, while repaired labels and coarser inventories improve evaluation or model performance within the studied settings.

  • Takeaways & Limitations

    The reviewed lexEN subset is model-selected rather than random, so its 211 changed labels cannot estimate a corpus-wide Maru2022 error rate.

Abstract

from arXiv · show

In English all-words word sense disambiguation (WSD), the labels, not the models, have become the bottleneck: frontier LLMs are accurate enough that the errors surviving in the gold standard decide benchmark rankings -- in the test sets we score on and, as we show causally, in the corpus we train on. We release lexEN, a WSD evaluation benchmark built as a conservative, human-adjudicated correction layer over Maru2022's ALL_NEW benchmark (211 labels changed, 56 removed), and SenseBench, an auditable LLM WSD evaluation harness and living leaderboard (57 models, 192 runs). The task is inventory-constrained multiple choice (the model picks from the supplied WordNet senses), so the reported accuracies are a ceiling on what models achieve without that help. On lexEN-v1 the frontier LLMs converge near 95% (best, 95.6%), the top three families are statistically indistinguishable, and accuracy trades off against reasoning effort and cost across a ~2,500x price span. Relabeling SemCor with frontier models and retraining BEM, ESCHER, and ConSeC unchanged lifts them by several F1 points on test sets the relabeling never touched; we release the relabeled corpora and Glite LENS, a 298M bi-encoder trained on the repaired labels -- to our knowledge the strongest reported (83.6 Raganato ALL, 87.4 Maru ALL_NEW) -- serving at ~$0.13 per million items. On hard items, fine-grained WordNet senses are partly ill-posed even for experts (three-reviewer Fleiss kappa=0.537); coarsening raises annotator agreement and model accuracy together across four inventories, placing a top model inside the expert agreement band at coarse granularity (statistically equivalent under three of four) but significantly below it at fine. The binding constraint is now cost.

1 Introduction

English all-words WSD has entered a regime where label quality and sense granularity constrain benchmark conclusions more than model architecture. The paper responds with lexEN, SenseBench, training-label repair, and analyses of cost and human agreement.

  • 1 Introduction: Label errors can decide rankings when model differences are smaller than annotation noise, making benchmark quality central to measurement.The paper argues that a better-characterized benchmark must identify decidable items, expert agreement, and meaningful system differences.
  • 1 Introduction: Frontier LLMs cluster near 95% on lexEN-v1, with a best score of 95.6% and statistically indistinguishable top three families.The task is inventory-constrained multiple choice rather than open generation.
  • 1 Introduction: lexEN changes 211 labels and removes 56 unanswerable items through conservative review, while SenseBench provides immutable prompts, artifact verification, cost tracking, confidence intervals, and a living leaderboard.The correction layer and harness are designed to make close model comparisons auditable.
  • 1 Introduction: Relabeling SemCor and retraining BEM, ESCHER, and ConSeC unchanged lifts them by several F1 points on untouched test sets, providing a controlled training-side complement to test-set repair.The paper releases the relabeled SemCor-GPT5.5 and SemCor-Gemma corpora.
  • 1 Introduction: Glite LENS reaches 83.6 on Raganato ALL and 87.4 on Maru ALL_NEW, while serving at roughly $0.13 per million items.It is a 298M bi-encoder trained solely on the relabeled corpus and is reported as the strongest bi-encoder in the paper.
  • 1 Introduction: At fine WordNet granularity, models and experts disagree on hard items, whereas coarse-graining raises both agreement and model accuracy across tested inventories.The paper identifies cost as the next binding constraint and treats fine-grained residual disagreement as partly shared by humans and models.

2 Related Work

Prior WSD research established strong supervised baselines, emerging LLM evaluations, and longstanding concerns about sense granularity and annotation quality. This paper combines those strands through an auditable correction layer and reproducible evaluation protocol.

  • 2 Related Work: Prior supervised systems raised English all-words F1 from the 65–70 range toward the low 80s before progress appeared to plateau near 79–83%.The paper uses MFS, BEM, ESCHER, and ConSeC as reference baselines for measuring LLM performance and label-noise sensitivity.
  • 2 Related Work: Existing LLM WSD studies are difficult to compare because they vary in inventories, datasets, task formats, metrics, and model versions.SenseBench addresses these differences with immutable prompts, raw-artifact reverification, and rescoring against multiple gold layers.
  • 2 Related Work: The evaluation pipeline begins with Raganato ALL, incorporates Maru2022’s ALL_NEW re-annotation, and adds professional lexicographer review to construct lexEN.Figure 1 frames the correction process as model-assisted selection followed by independent human review and frozen adjudication.
  • 2 Related Work: The literature motivates coarser, more reproducible sense inventories because fine-grained distinctions are not reliably reproduced by either annotators or systems.The paper follows this coarsening line while retaining a single audited gold layer and releasing reviewer choices and rationales.
  • 2 Related Work: Model-assisted benchmark repair focuses expert effort on suspicious examples, and lexEN adopts this targeted-audit pattern with expert adjudication of retained labels.This approach connects benchmark construction to annotation-error detection and governance work.
  • 2 Related Work: The paper’s training intervention differs from noisy-label robustness methods by correcting annotations at the source and retraining otherwise unchanged systems on cleaned corpora.It also connects to work using language models as data annotators.

3 lexEN: Constructing an Auditable Correction Layer

lexEN is a conservative, auditable correction and coarsening layer over Maru2022, using model-assisted triage, independent lexicographer review, and frozen majority adjudication. It corrects suspicious labels while making fine-versus-coarse disagreement measurable.

  • 3.6 Release and Governance: The released correction process preserves item-level lineage and offers a reusable recipe: model-panel triage, independent professional review, frozen majority adjudication, and full release.The immutable artifact records reviewer choices, evidence, cannot-answer types, and the rule producing each label.
  • 3.1–3.4 Construction: 363 suspicious items were triaged, 211 labels were corrected, and 56 unanswerable items were removed through conservative human adjudication.Three professional lexicographers reviewed selected items independently; retention required a two-of-three sense majority, while majority cannot-answer or three-way disagreement caused removal.
  • 3.2 Model-Assisted Triage: The model panel flags items where frontier predictions dispute Maru2022, concentrating review on a hard tail rather than estimating corpus-wide annotation error.The selection rule is fixed: at least six of eight GPT-5.5 variants must dispute the source label, with SANDWiCH and CatBoost grading convergence.
  • 3.3 Lexicographer Review: Review is provenance-hidden but not cue-free, because highlighted candidate senses can still anchor lexicographers despite hiding source labels and model identities.Reviewers work independently and see full candidate inventories, while earlier annotations and triage-plausible senses may be highlighted.
  • 3.4 Adjudication: The frozen two-of-three rule ratifies a shared expert sense, removes unresolved items, and never lets one reviewer override the others.A corrected label reflects at least two independent professional judgments rather than a single annotator’s opinion.
  • 3.5 The Glite Coarsening Layer: Coarsening deterministically re-grades identical labels and predictions, with the qualitative accuracy lift remaining across inventories; Glite also most closely matches five professional dictionaries among six candidate mappings.The coarsening layer supports reproducible fine- and coarse-granularity analyses without re-annotating at the coarse level.

4 SenseBench: An Auditable Evaluation Protocol for LLM WSD

SenseBench makes LLM WSD comparisons reproducible by fixing prompts, retaining raw calls, and mechanically re-verifying scores. It evaluates constrained-index multiple choice, yielding clean cross-system comparisons but accuracies that are upper bounds relative to unaided sense generation.

  • 4.1 Task formulation: SenseBench evaluates every model under identical constrained-index multiple-choice conditions, supplying WordNet candidate senses and scoring the selected index.The target span and candidate inventory are provided, making the task mechanically verifiable across systems.
  • 4.1 Task formulation: Constrained selection removes output-matching confounds but makes reported accuracies upper bounds that are not directly comparable to open-generation WSD results.Models receive the candidate list, and chance accuracy depends on candidate count rather than the whole inventory.
  • 4.2 Prompts: Every prompt is an immutable registered object, so prompt identifiers make leaderboard accuracies comparable only when the models share exactly the same task framing.Controlled prompt pairs vary context and gloss enrichment, while intermediate prompts isolate individual factors.
  • 4.3 Run artifacts: Each run records model, prompt, dataset hash, decoding, usage, cost, code version, date, predictions, and raw request-response artifacts.These artifacts pin each leaderboard row to a model version and evaluation configuration.
  • 4.3 Run artifacts: Continuous integration re-parses raw calls and re-scores frozen labels, failing builds when regenerated predictions do not reproduce published accuracy bit for bit.The public artifacts provide the evidence and procedure needed to reconstruct each reported number.
  • 4.4 Statistical methodology: Close system comparisons use paired bootstrap tests that preserve per-item correlation and call differences significant only when the 95% paired interval excludes zero.This distinguishes appropriate head-to-head ordering from comparisons based on separate marginal confidence intervals.

5 Results I: Label Noise and the Current Frontier

Re-scoring fixed predictions shows that correcting gold labels benefits frontier LLMs far more than supervised systems, while coarse-grained evaluation largely closes the remaining gap to expert agreement. The corrected benchmark places frontier families near 95%, with top-family differences not statistically decisive and residual fine-grained disagreement shared by humans and models.

  • 5.1 Label correction lifts the LLMs most: 10.2 points is GPT-5.5’s gain from Raganato to lexEN, while supervised systems gain only 2.7–3.7 points across the same label span.The frontier models rise monotonically as labels improve, whereas supervised systems gain only a fraction of a point on lexEN’s Maru2022-to-lexEN correction.
  • 5.1 Label correction lifts the LLMs most: 98.6–98.7% is the frontier range under coarse lexEN labels, leaving under two points of headroom at the practically meaningful granularity.Across public coarse inventories, accuracies also rise substantially, supporting the conclusion that much fine-level error reflects over-specified distinctions.
  • 5.2 The lexEN-v1 leaderboard and cross-family agreement: 95.6% is the best lexEN-v1 score, with frontier families clustering near 95% and the top three not statistically separable.GPT-5.5 reaches 95.6%; paired comparisons do not establish a reliable separation among the leading families, including on the hard subset.
  • 5.2 The lexEN-v1 leaderboard and cross-family agreement: 95.3% is Claude-Fable-5’s self-answered accuracy versus 95.2% for its mixed run, and excluding fallback-served corrections leaves the correction effect essentially unchanged.The fallback was a mechanical repair pass for filtered responses, not an ensemble, and the robustness check preserves the correction-step result.
  • 5.2 The lexEN-v1 leaderboard and cross-family agreement: The corrected-tail comparison does not eliminate shared training-data, WordNet, frequency-prior, or semantic biases across frontier families.The authors note that non-panel consensus is only a computational stand-in for a random unflagged control.

6 Results II: Repairing the Training Labels

The section tests whether training-label quality constrained classic supervised WSD systems and finds that repairing labels improves unchanged architectures on unseen benchmarks. The gains concentrate on non-first-sense items, while repaired labels and modern architectures contribute comparably and combine to yield a strong low-cost bi-encoder.

  • 6.1 Relabeling SemCor: The relabeling disagrees with SemCor gold on 21.30% of GPT-5.5-labeled instances and 21.85% of Gemma-labeled instances, with the highest disagreement on verbs.Verb disagreement is 29.90% for GPT-5.5 and 30.86% for Gemma; the two relabelers disagree with the corpus in almost the same places.
  • 6.2 Retraining lifts the classic systems: Every retrained system improves on unseen benchmarks when only the training gold changes, with +2.4 to +3.4 points on Raganato ALL and roughly doubled gains on Maru ALL_NEW.GPT-5.5 relabeling raises ESCHER from 79.57 to 82.39, ConSeC from 81.62 to 84.06, and BEM from 77.66 to 81.06 on Raganato ALL.
  • 6.2 Retraining lifts the classic systems: +11.7 to +17.6 points of retraining gain occur on non-first-sense items, versus a 0.5–3.3-point loss on first-sense items across both unseen surfaces.The pattern replicates with Gemma relabeling, with +11.2 to +17.9 points on non-first-sense items.
  • 6.2 Retraining lifts the classic systems: The training-label lift widens as evaluation labels become cleaner, mirroring the test-side correction pattern from Raganato to Maru to lexEN.This monotonic increase is reported for every retrained system.
  • 6.4 Glite LENS: Glite LENS reaches 83.6 on Raganato ALL and 87.4 on Maru ALL_NEW, exceeding published SemCor-only systems and matching ConSeC+WNGE despite using roughly 3× less labelled data.It scores 90.5 on lexEN-v1 and serves at roughly $0.13 per million items; the lexEN result is confirmatory because training labels share a model with relabeling.
  • 6.4 Glite LENS: On Raganato ALL, repairing labels moves BEM by +3.4 points and modernizing the architecture at fixed repaired labels by about +2.6, for roughly +6 points together.The decomposition treats label quality and architecture as separable contributors rather than a hard ceiling from either axis alone.

7 Results III: Human Agreement and the Granularity Ceiling

Fine-grained WordNet senses are partly ill-posed even for expert lexicographers, while coarsening substantially improves both human agreement and model performance. At coarse granularity, frontier models enter the expert agreement band; at fine granularity, they remain significantly below it.

  • Inter-Annotator Agreement: Fine-grained agreement is limited: no two reviewers exceed 59% exact agreement, so the benchmark measures distinctions experts cannot reliably reproduce.The pooled κ calculation uses lemma-specific candidate sets and a global sense-selection space.
  • Inter-Annotator Agreement: 63.1% three-way agreement and Fleiss κ=0.740 under Glite coarse replace 35.5% and κ=0.537 at fine granularity.42.7% of non-unanimous fine items become unanimous after coarsening.
  • Easy versus Hard: On hard items, coarse accuracy recovers to 87.0, 86.6, and 87.3 for GPT-5.5, Gemini-3.1-Pro, and Claude-Fable-5, versus fine scores near two-thirds.The coarse lift persists across three public coarsenings, reaching 78–81%.
  • The Human Envelope: At fine granularity, reviewer–model κ=0.551 is significantly below reviewer–reviewer κ=0.600, with paired difference −0.049 and a confidence interval excluding zero.The model enters the human envelope only after coarsening.
  • The Human Envelope: At coarse granularity, reviewer–model κ=0.812 and reviewer–reviewer κ=0.805 are statistically indistinguishable, with paired difference +0.007.The confidence interval includes modestly worse model agreement, so the result is interpreted as the same agreement band rather than model superiority.
  • The Human Envelope: The coarse human-envelope result is robust across inventories: every coarse reviewer–model difference confidence interval includes zero, unlike fine WordNet.Per-model coarse reviewer-agreement κ values are 0.807, 0.814, and 0.816 for GPT-5.5, Gemini-3.1-Pro, and Claude-Fable-5.

8 Discussion

The discussion separates repairable label noise from the deeper granularity ceiling and argues that benchmark quality, human agreement, and inference cost now define progress. Frontier models can match expert agreement on coarse inventory-constrained WSD, but broad deployment makes price decisive.

  • Discussion: Label noise affects both evaluation and training: correcting test labels improves measurement, while relabeling SemCor raises supervised systems on untouched test sets.The paper treats these as distinct bottlenecks: contingent label noise is repairable, whereas fine-grained sense disagreement reflects the inventory.
  • Granularity and the Human Ceiling: At coarse granularity, top-model agreement with experts is statistically equivalent to expert–expert agreement across four inventories; fine WordNet WSD remains below that ceiling.The paper’s novelty is demonstrating this granularity-relative conclusion for frontier models on corrected all-words labels.
  • Cost: At scale, inference price becomes a budget constraint because practical WSD applications operate on billions of words and research infrastructure repeatedly re-scores benchmarks.The paper therefore treats cost as a first-class leaderboard axis alongside accuracy.
  • Cost: The cost frontier also has an equity dimension because compute-gated disambiguation disproportionately constrains under-resourced languages, institutions, and public deployments.A one-time relabeling budget may be more reachable than standing frontier-inference costs, but does not erase the disparity.
  • Cost: Offline frontier relabeling offers an amortizable alternative: pay once to repair training data, then deploy a supervised model at much lower recurring cost.The relabeling comparison is indicative rather than like-for-like because local GPU and cloud API costs are measured differently.
  • Benchmark Trustworthiness: Benchmark governance must address both label error and contamination through immutable artifacts, provenance, resolved model versions, and evaluation dates.The release makes benchmark versions fixed and runs re-locatable, although the contamination canary detects only verbatim echoing.

9 Conclusion

The paper argues that labels, not models, have become the bottleneck in English all-words WSD and addresses this with corrected evaluation, auditable measurement, and repaired training data. Its results make progress granularity-relative and leave cost as the practical frontier.

  • Conclusion: lexEN and SenseBench replace annotation-sensitive evaluation with a conservative corrected benchmark and an auditable, cost-aware leaderboard.The paper targets settings where leading systems already differ by roughly a point.
  • Conclusion: The remaining challenge is to disambiguate at the practically meaningful granularity everywhere useful while keeping the cost low enough for deployment at scale.The conclusion frames affordability, rather than accuracy alone, as necessary for making broad coverage possible.

Limitations

The paper’s claims are bounded by model-selected review, inventory-dependent coarse scoring, reviewer heterogeneity, proprietary-system drift, uneven reasoning controls, and the narrow inventory-constrained task. Several training and evaluation results also require caution because relabeling and recipe choices can overlap with evaluation conditions.

  • Scope of Review: The reviewed subset is model-selected, so 211 corrected labels cannot estimate Maru2022’s corpus-wide error rate among 4,554 unreviewed items.Unreviewed items retain their Maru2022 labels unchanged.
  • Inventory Dependence: The agreement and accuracy claims depend on WordNet and the released Glite many-to-one map, so coarse results are inventory-specific rather than universal.The map covers 10,412 sense keys over 6,505 concepts, with explicit markers for uncovered keys; OntoNotes could not be evaluated.
  • Reviewer Heterogeneity: The three-reviewer panel cannot average away reviewer idiosyncrasy, including differing rates of no_sense_applies judgments.The frozen two-of-three rule absorbs some heterogeneity but does not make reviewer rates interchangeable.
  • Task Scope: SenseBench evaluates inventory-constrained disambiguation with supplied target spans and candidate senses, not end-to-end lexical semantics.All reported human-agreement and model results are conditioned on this narrower task definition.
  • Reproducibility: Proprietary endpoints may drift despite pinned versions, dates, and hashes, so frozen artifacts—not live endpoints—are the reproducible objects.Re-querying the same endpoint is not guaranteed to reproduce the original outputs.
  • Reasoning Effort: The leaderboard mixes vendor-specific reasoning tiers because only 8 of 52 models expose more than one tier, so best-tier comparisons can conflate capability with available effort.The paper supplies matched-tier comparisons and paired bootstrap analyses to address this limitation.
  • Experimental Generality: The prompt ablation covers only two open Gemma models, while Glite LENS recipe choices used lexEN-fine feedback and include limited seed controls.These design choices constrain how broadly component and recipe conclusions should be generalized.
  • Training-Side Evidence: SemCor disagreement rates are not verified error rates because relabeled training items lacked lexicographer adjudication, and lexEN shares labeling-function overlap with some training results.Training claims therefore rely on independent Raganato and Maru test sets, treating lexEN as confirmatory.

Ethics Statement

The review was conducted by professional lexicographers under informed-consent, compensation, and public-release procedures, with reviewer independence safeguards. The coarse-granularity results require particular care because the grouping was developed by the authors for Glite’s product, although alternative public coarsenings were also evaluated.

  • Three professional lexicographers reviewed items under informed consent, professional compensation, and consent to release their names, judgments, and rationales.
  • Reviewers were blind to the source label, flagging panel member, and one another’s choices, although PH is a paper co-author.
  • Coarse-granularity results use an author-developed inventory created for Glite’s language-learning product rather than a third-party standard.
  • The released coarsening map enables reproducibility, and predictions were also re-graded under CSI, WordNet supersenses, and WordNet Domains.

Reproducibility Statement

SenseBench is designed for rerunning and verifying results through immutable prompts, resolved model versions, dates, and artifact hashes. The training relabeling and released bi-encoder are likewise accompanied by auditable records, code, checkpoints, and prediction files.

  • Every SenseBench run is pinned to an immutable prompt, resolved model version, evaluation date, and artifact hashes for re-checking raw API responses.
  • Relabeled SemCor layers preserve the original corpus while releasing relabeled gold keys and per-instance records of keys, candidates, chosen senses, models, and prompt hashes.
  • The relabeling layers are released under CC BY-NC with the complete research record available in the cited repository.
  • Glite LENS includes training and inference code, checkpoints, and prediction key files so reported results can be reproduced or re-derived.
  • LLMs served both as study objects and methodological instruments, while authors checked manuscript claims, numbers, and interpretations against released artifacts.

A Full Agreement Report

Across source corpora, polysemy levels, and parts of speech, expert agreement is substantially higher for coarse than fine-grained senses. The detailed breakdown therefore supports a granularity-dependent ceiling on fine-grained WSD evaluation.

  • By source corpus: Fine agreement ranged from 29.5% to 46.2% across source corpora, while coarse agreement ranged from 59.1% to 73.1%.Every source roughly doubled unanimity under coarsening, with broadly preserved source ordering.
  • By polysemy degree: Fine agreement fell from 52.5% on two-candidate targets to 27% on targets with 8+ senses, while coarsening recovered agreement at every polysemy level.The fine-grained disagreement was concentrated on highly polysemous targets.
  • By part of speech: Coarsening raised all-three-agree rates for NOUN 36.9 →62.6, VERB 29.7 →67.0, ADJ 32.7 →55.1, and ADV 58.8 →70.6.Verbs gained the most, while adverbs started highest and therefore had the least room to improve.
  • Agreement remained low at fine granularity and recovered under coarsening across all measured corpora, polysemy bands, and parts of speech.The reported ceiling is attributed to the sense inventory rather than to one data slice.

B Extended Examples

The reviewed examples show why corrections, retained labels, model disagreement, and unanswerable items require separate treatment. They also expose concrete WordNet coverage gaps and cases where models converge on the wrong sense.

  • Further corrections and retained items: Three reviewers unanimously corrected Maru2022’s “make certain” label for see to the plain “perceive” sense.
  • Model-agreement patterns: Models matched the expert majority on country, but jointly selected the wrong “basic” satellite for underlying despite unanimous human selection of “implicit.”
  • Model-agreement patterns: Fine-grained reviewer splits could disappear under coarsening, as with local, where a 2–1 human split became unanimous at the coarse level.
  • Unanswerable items: Several items were unanswerable because WordNet lacked the needed revenue, cognate-object, company, mainland-Europe, or fixed-phrase senses.
  • Unanswerable items: Some items were unanswerable because the source text was non-native or otherwise defective, making the intended sense unrecoverable.
  • Further corrections and retained items: Only 211 of 363 flagged items were corrected because retained cases such as argument showed that the original label still fit.

C Bi-encoder Configuration and Component Attribution

Glite LENS is a 298M-parameter dual-tower gloss bi-encoder trained with contrastive in-batch negatives on relabeled SemCor. Its gains arise from interacting backbone, recipe, and context choices, while target-span pooling is foundational and serving costs are measured directly.

  • Configuration: Glite LENS uses two independent 149M-parameter ModernBERT-base towers, scores context and structured candidate-gloss encodings by dot product, and trains for three epochs on 226,036 relabeled SemCor instances.The candidate gloss includes the headword, part of speech, definition, synonyms, and example sentences.
  • Component Attribution: The full recipe’s +4.5-point gap over the BEM anchor decomposes across backbone, recipe, and context axes, while glosses and negatives add little marginally.The attribution compares add-one and remove-one variants on lexEN fine.
  • Component Attribution: Components interact: structured glosses hurt when added alone to the anchor, so the recipe functions as a co-adapted stack rather than a sum of parts.Add-one and remove-one readings therefore disagree about individual component contributions.
  • Serving Cost: Serving costs $0.126 per million items at roughly 0.9 ms per item under steady-state bf16 inference on a single RTX 5090 with the gallery precomputed.Model-load time is excluded and the GPU is priced at market hourly rental rates.

D Reviewer Brief

The reviewer brief standardizes fine-grained WordNet adjudication by requiring one best sense by default, explicit ambiguity and inventory-defect flags, and confidence ratings. It also records practical guidance for difficult constructions and specifies how reviewer verdicts feed the benchmark.

  • Task and Granularity: Reviewers choose one WordNet sense that best fits each target sentence, reserving ambiguity comments for cases where two distinct meanings are genuinely supported.The task is not to merge near-identical senses for a learner’s dictionary.
  • Component Attribution: The component-attribution table compares add-one changes to an 85.5 lexEN-fine BEM anchor with remove-one changes from a 90.0 full recipe, using a canonical 90.6-to-81.7 pooling remeasurement.The table is lexEN fine F1 only, with single-seed cells except for the pooling remove-one result.
  • Noun Adjuncts: Noun adjuncts normally receive the standard noun sense, except when lexicalization creates a genuinely distinct adjective meaning.The brief uses “plastic” meaning “fake” or “superficial” as the exception.
  • Phrasal Verbs and Multiword Expressions: Phrasal verbs and fixed expressions absent from WordNet are marked inventory_inadequate because the headword’s plain senses cannot cover the usage.Such inventory gaps are treated as a recurring reason items leave lexEN.
  • Cannot-Answer Flags: The protocol distinguishes no_sense_applies, inventory_inadequate, and input_defective, each requiring a free-text explanation; a guarded sense pick may coexist with a flag.These flags determine which items are removed from lexEN.
  • Confidence and Records: Reviewers assign high, medium, or low confidence and use WordNet examples as well as definitions, but the exported records do not preserve the confidence field.The released records retain selected sense keys, typed flags, notes, and free-text comments.
Loading 2609.17554v1…