Source-linked AI summary

FormBench: A Benchmark for Fine-Grained Retrieval in Formulation Patents

Vineeth Venugopal, Elsa Olivetti, Jonathan Welch

arXiv:2610.30175v1cs.IR

TL;DR

Fine-grained retrieval in formulation patents requires distinguishing passages that share chemical entities but differ in functional role and context. FormBench evaluates this challenge using ontology-based triplets and random and structured corpora. Commercial API encoders show little retrieval signal, while task-adaptive Qwen3-Embed exceeds BM25; the paper also notes limits in relevance judgments and patent coverage.

  • Problem

    Generic-web-trained language models are not designed to distinguish passages that share domain-specific entities but differ in functional role and application context, a failure existing benchmarks do not expose.

    Source: §1 · p. 1
  • Method

    FormBench pairs synthetic queries with anchor, hard-negative, and soft-negative passages and evaluates models on formulation-patent corpora with random or structured distractors.

    Source: §1 · p. 1, §5.3.1 · p. 6
  • Results

    Qwen3-Embed + MNRL reaches 0.4085 on C1 and 0.4835 on C0, exceeding BM25, while commercial API encoders score below the BM25 lexical floor.

    Source: §1 · pp. 1–2
  • Takeaways & Limitations

    The benchmark shows that general-purpose embeddings can fail at fine-grained retrieval despite performing well on coarse patent classification, while task-adaptive training improves retrieval.

    Source: §6 · p. 8, §7 · p. 11

    Relevance tiers are a computational proxy rather than a precise fine-grained proxy for human judgments, and coverage is limited to English-language USPTO utility patents from 1995–2022.

    Source: §7 · p. 11

Abstract

from arXiv · show

We introduce FormBench, a platform for testing the ability of models to retrieve precise chemical information from technical documents. FormBench contains 55,352 queries and two retrieval corpus variants each with ~1M passages: FormBench-Random (C0), with randomly sampled distractors, and FormBench-Structured (C1), with structurally proximate distractors. Each query is paired with an anchor passage, a hard negative, and a soft negative - derived from a materials ontology through entity overlap scoring, providing a deeper graded signal on a model's discriminatory power. We evaluate 10 model configurations spanning commercial API encoders (OpenAI, Voyage), open-source encoders (Nomic, GTE-large, Qwen3-Embed), and their task-adaptive fine-tuned (TAPT) variants. We find: (1) commercial APIs score very poorly across all configurations (OpenAI-3-large: 0.009 on C1, 0.213 on C0); (2) TAPT on FormBench triplets lifts Qwen3-Embed to 0.4085 on C1, 0.4835 on C0; (3) BM25 remains competitive (0.375 on C1, 0.467 on C0), exceeded only by TAPT-Qwen3; (4) corpus size has a significant impact on retrieval behavior, as demonstrated through a corpus-size ablation; (5) the API failure traces to within-domain geometric collapse; (6) APIs outperform even the TAPT variants on classification and clustering tasks. FormBench is released in BEIR-compatible format at huggingface.co/datasets/vinven7/FormBench under CC-BY-4.0.

1 Introduction

FormBench targets fine-grained retrieval in formulation patents, where passages can share chemical entities while differing in functional roles and application context. Its results show that general-purpose embeddings often fail to discriminate such passages, while task-adaptive training can improve retrieval.

1 Introduction

  • Retrieval is difficult because passages may share domain-specific entities yet differ in their functional roles and application contexts.The discriminative signal is ontological rather than topical, and generic-web-trained language models are not designed for this distinction.Source: §1 · p. 1
  • FormBench contains 55,352 formulation-patent queries and contrasts random distractors with ontologically adjacent near-misses requiring functional-level discrimination.The two corpus variants share queries and relevance judgments, and are released in BEIR format.Source: §1 · p. 1
  • Qwen3-Embed + MNRL reaches 0.4085 on C1 and 0.4835 on C0, exceeding the BM25 lexical baseline and outperforming commercial API encoders.Commercial APIs score below the BM25 lexical floor; the benchmark also reports that TAPT’s retrieval gains do not transfer to classification or clustering.Source: §1 · pp. 1–2
  • The benchmark combines graded relevance, two corpus variants, ten model configurations, BM25, corpus-size ablation, and geometric analysis of API failure.The stated contributions include binary and graded nDCG@10, MRR, recall at 100, and latency analysis.Source: §1 · p. 2

2 Related Work

Prior benchmarks and patent or chemistry retrieval work provide relevant evaluation settings, but do not target fine-grained composition matching over full patent descriptions. FormBench addresses this gap with ontologically derived relevance and functional near-miss passages.

2 Related Work

  • Specialized benchmarks show that high aggregate MTEB scores do not predict performance on professional text; FormBench extends this line to formulation chemistry.The paper focuses on dense bi-encoder evaluation, where embedding geometry is interpretable and domain adaptation is tractable.Source: §2 · p. 2
  • PatenTEB and DAPFAM use binary, citation-derived relevance over titles and abstracts, while FormBench targets composition matching over full descriptions.ChemTEB’s retrieval evaluation uses about 33,000 passages and 233 queries across two splits adapted from general-domain QA.Source: §2 · p. 2
  • FormBench uses entity-overlap tiers for graded relevance and functional-role near-misses, targeting within-domain composition matching rather than citation-based or topic-level retrieval.The paper states that this design exposes commercial API models’ lack of discriminatory signal, a gap invisible without structured within-domain negatives.Source: §2 · p. 2

3 The FORMBENCH Dataset

FormBench is built from USPTO formulation-patent passages, entity-based relevance tiers, and synthetic queries checked for quality and anchor matching. Its random and structured corpus variants contrast broad topical retrieval with harder within-domain discrimination.

3 The FORMBENCH Dataset

  • The three BEIR-format releases share query and qrel files; FormBench-Sample provides 63,058 scored passages for rapid prototyping and reviewer verification.The full Random and Structured configurations contain about 997,000 and 994,000 passages, respectively, and are released under CC-BY-4.0.Source: §3 · p. 2

3.1 NER and Ontological Screening

  • 590,247 USPTO utility patents yielded about 3.1 million candidate passages after CPC filtering and overlapping description chunking.The patents cover 1995–2022 and formulation areas including pharmaceutics, cosmetics, adhesives, coatings, agrochemicals, and food.Source: §3.1 · p. 3
  • The NER pipeline processed 3.1 million passages and extracted 234.7 million entity mentions across 3,090,473 unique surface forms.The Llama-3-8B adapter achieved 85.9% exact-match Jaccard on a 250-passage held-out test set.Source: §3.1 · p. 3
  • Entity-set Jaccard tiers assign hard negatives to similarity 0.30–0.50 and soft negatives to 0.05–0.15, supporting graded relevance and triplet construction.The passage defines similarity from canonical entity sets after normalization.Source: §3.1 · p. 3

3.2 Synthetic Query Generation

  • Claude Sonnet 4 generated formulation-relevant questions from cross-patent anchor, hard-negative, and soft-negative triplets.Queries address composition, process conditions, performance properties, or application context; nomenclature-only, hardware, and boilerplate questions were excluded.Source: §3.2 · p. 3
  • Claude Haiku 4.5 screened queries for quality and checked, in shuffled passage order, whether the selected answer matched the original anchor.Queries failing either stage were eliminated; 55,352 queries were retained and split by patent into train, development, and test sets.Source: §3.2 · p. 3
  • Expert re-annotation ranked the anchor first in 93.1% of judgements and the hard negative above the soft negative in 59.7%.The expert study covered 72 judgements over 49 triples; its full study of 100 triples was ongoing.Source: §3.2 · p. 4

3.3 Corpus Variants

  • FormBench-Random fills distractor slots with random patent chunks, while FormBench-Structured uses passages from the anchor patent and other patents in the same chemical macro-domain.Random tests coarse-grained domain retrieval; Structured is the primary evaluation configuration and isolates the added difficulty of within-domain distractors.Source: §3.3 · p. 4

4 Evaluation Framework

FormBench evaluates retrieval across corpus difficulty and relevance granularity, using binary nDCG@10 as its primary metric. The evaluation uses exact top-100 search and tests open-source encoders in zero-shot and TAPT conditions alongside commercial API baselines.

4 Evaluation Framework

  • C0 tests retrieval among topic-diverse distractors, whereas C1 tests discrimination among near-misses from the same chemical macro-domain.The C0/C1 performance gap is presented as diagnostic of whether models have learned domain-specific discrimination.Source: §4 · p. 4
  • Binary nDCG@10 treats only the anchor as relevant, while graded nDCG@10 scores anchors, hard negatives, and soft negatives as 3, 2, and 1.Binary nDCG@10 is the primary metric; graded nDCG@10 gives partial credit for retrieving compositionally similar passages near the top.Source: §4 · p. 5

4.1 Evaluation Infrastructure and Latency

  • Corpus passages are indexed with FAISS flat inner product, and exact nearest-neighbor search retrieves the top 100 results for each query.The evaluation records per-query latency and embedding throughput; API round-trip latency is not directly comparable to on-device GPU throughput.Source: §4.1 · p. 5

4.2 Models Evaluated

  • The model evaluation covers three open-source encoder families under zero-shot and TAPT conditions, plus four commercial API baselines evaluated zero-shot.The open-source families are Nomic Embed, GTE-large, and Qwen3-Embedding; commercial baselines are OpenAI text-embedding-3-large/small and Voyage-4-large/3.5.Source: §4.2 · p. 5

5 Experiments

The experiments compare lexical and embedding-based retrieval on random and structurally difficult corpora, testing base models, task-adaptive fine-tuning, and corpus-size sensitivity. Qwen3-Embed with TAPT is the only reported neural configuration to exceed BM25 on both corpora.

5 Experiments

  • All 10 configurations are evaluated, with BM25 included as a lexical baseline; the benchmark experiments test whether general-purpose retrieval models provide signal and whether TAPT learns from the training split.The evaluation is framed as benchmark validation for discriminativeness, solvability, and adaptation.Source: §5 · p. 5

5.1 BM25 Lexical Baseline

  • 0.3751 on C1 and 0.4665 on C0: BM25 ranks second on both corpora and provides a reference for learned embeddings.Queries are generated from passages, so lexical overlap with anchors is expected.Source: §5.1 · pp. 5–6

5.2 Base Model Results

  • On C1, commercial APIs score 0.0025–0.0090 and open-source base models score 0.1475–0.2317; none reaches BM25’s 0.3751.OpenAI-3-large exceeds OpenAI-3-small by only 0.0011 despite twice the dimensionality.Source: §5.2 · p. 6

5.3.1 Protocol

  • TAPT trains on query–anchor–hard-negative triplets with MNRL, pulling matched passages closer and pushing hard negatives apart in embedding space.The anchor is the positive; the hard negative comes from Jaccard-tier pairs, and other in-batch positives also serve as negatives.Source: §5.3.1 · p. 6

5.4 C0 vs. C1 Corpus Structure

  • OpenAI-3-large falls from 0.213 on C0 to 0.009 on C1, while TAPT models show a smaller C0/C1 gap of approximately 0.070–0.075.The passage attributes the API change to replacing random distractors with structured near-misses; BM25’s gap is 0.091.Source: §5.4 · p. 6, Figure 2 · p. 8
  • Patent-domain encoders retain 81–88% of their C0 score on C1, compared with 4% for OpenAI-3-large, but patent pretraining does not yield fine-grained formulation discrimination.Chunk-and-pool encoding lowers PaECTER from 0.1334 to 0.1239 and PatentSBERTa from 0.1064 to 0.0963.Source: §5.4 · pp. 6, 8

5.3.2 Results

  • 0.4085 on C1 and 0.4835 on C0: Qwen3-Embed + MNRL is the only TAPT configuration to exceed BM25 on both corpora.Nomic + MNRL reaches 0.3668 and GTE + MNRL reaches 0.3561 on C1.Source: §5.3.2 · p. 6, Table 1 · p. 7

5.5 Corpus-Size Sensitivity

  • As the C0 corpus grows from distractor fraction 0.01 to 0.75, Voyage-3.5 drops 82%, OpenAI-3-large 53%, and Qwen3+MNRL 31%.All models degrade as corpus size increases; TAPT-fine-tuned models degrade most slowly.Source: Figure 3 · p. 8, §5.5 · p. 8

6 Analysis

The analysis investigates why commercial API embeddings fail on fine-grained formulation retrieval and what TAPT changes. It finds that API models lack within-domain ranking separation, while TAPT improves retrieval but not classification or clustering; lexical overlap and external-benchmark results further qualify these findings.

6.1 Embedding Geometry: Why Commercial APIs Fail

  • ∆=0.096 for OpenAI text-embedding-3-large, versus 0.177–0.206 for TAPT models; anchor rank-1 rates are 0.3% and 24.5%, respectively.The analysis attributes the narrow API similarity band to within-domain anisotropy and says contrastive training on hard negatives directly optimizes pairwise separation.Source: §6.1 · p. 9, Figure 4 · p. 9

6.2 Downstream Task Evaluation: What TAPT Does and Does Not Learn

  • 80.4% classification accuracy for OpenAI-3-large versus 78.0% for the best TAPT model, GTE-large; TAPT adds no improvement on classification or clustering.The paper reports this as a retrieval–classification inversion, while noting that its exact mechanism is unclear.Source: §6.2 · p. 10, Table 3 · p. 10

6.3 Analysis of Lexical Similarity

  • 77%: BM25’s C1 score falls from 0.3765 to 0.0853 under progressive query rewriting, while neural models fall within a 43–52% band.OpenAI-3-large remains near zero (0.0089 → 0.0060); query-meaning preservation was not independently checked, so some drop may reflect lost information.Source: §6.3 · p. 10, Table 4 · p. 10

6.4 External Validation

  • Within 0.001 of published ChemRxivRetrieval scores for three of four reference models, the pipeline’s scorer is independently validated; on FormBench, ChEmbed-prog remains below BM25 and 1.66× below Qwen3-TAPT.TAPT slightly improves both tested models on ChemRxivRetrieval, but the gains were not significance-tested; ChemTEB has only 45 test queries and is too small for conclusions.Source: §6.4 · p. 11, Table 5 · p. 11

7 Conclusion

The conclusion presents FormBench as a benchmark exposing general-purpose embeddings’ difficulty distinguishing chemically similar passages within a domain. It reports that structured-hard-negative TAPT improves retrieval, while noting limits in relevance judgments and corpus coverage.

7 Conclusion

  • 0.4085 on C1 and 0.4835 on C0: Qwen3-Embed + MNRL is the only configuration to exceed BM25, while OpenAI text-embedding-3-large scores 0.009 on C1.The same API models that fail at retrieval lead on CPC classification probes.Source: §7 · p. 11
  • FormBench’s graded relevance uses entity-set Jaccard as a computational proxy, not a precise fine-grained proxy for human judgments; binary nDCG@10 is more robust to tier-boundary uncertainty.The qrel tiers show 64.7% hard-over-soft agreement in LLM-as-judge validation.Source: §7 · p. 11
  • Coverage is limited to English-language USPTO utility patents from 1995–2022; the authors identify human-authored queries and multilingual, non-US sources as future extensions.The authors say human-authored formulation-chemist queries would directly address the principal validity limitation.Source: §7 · p. 11

A Dataset Construction Details

The dataset construction pipeline gathers formulation-patent passages, identifies and normalizes entities, and uses entity overlap to form graded relevance pairs. It generates and filters synthetic queries, then splits them by patent to prevent cross-split patent overlap.

A.1 Patent Corpus and NER Pipeline

  • 590,247 USPTO utility patents (1995–2022) were segmented into overlapping chunks of at most 6,000 characters, yielding about 3.1 million candidate passages.CPC filtering retained formulation-related subclasses; passages shorter than 200 characters or without alphabetic content were discarded.Source: §A.1 · p. 14
  • A Llama-3-8B LoRA NER adapter extracted 234.7 million entity mentions; normalization reduced 3,090,473 raw surface forms to 2,854,881 canonical forms.Normalization combined deterministic cleanup, embedding-based clustering, and LLM adjudication of ambiguous clusters.Source: §A.1 · pp. 14–15, Table 6 · p. 15
  • Jaccard overlap over nine discriminative NER categories assigns hard negatives score 2 at 0.30–0.50 and soft negatives score 1 at 0.05–0.15; anchors score 3.Pairs from the same patent receive no non-zero score.Source: §A.1 · p. 15

A.2 Query Generation Pipeline

  • Claude Sonnet 4 generated formulation queries in four modes, and Claude Haiku 4.5 filtered them for grammaticality, specificity, and passage match, retaining 55,352 queries.The filtering pass rate was approximately 72%; patent-level stratified 80/10/10 splitting yielded 44,413/5,480/5,459 train/dev/test queries with no patent overlap.Source: §A.2 · pp. 15–16

B.1 Model Architectures

The section specifies the evaluated encoders and retrieval setup, then reports training dynamics across DAPT and TAPT. DAPT did not improve retrieval over zero-shot baselines, while TAPT learning curves converged at different rates across architectures.

B.1 Model Architectures

  • The evaluated open-source encoders are Nomic, GTE-large, and Qwen3-Embed, with maximum sequence length 8,192 and encoding length 2,048.The 2,048-token encoding length covers 99.87% of corpus passages without truncation.Source: Table 7 · p. 16, §B.1 · p. 16

B.2 Evaluation Protocol

  • Retrieval uses exact FAISS inner-product search over the top 100 results, with metrics computed against test-split qrels for 5,459 queries.The corpus is encoded in shards on OLCF Frontier; API latency includes round-trip network and inference time and is not directly comparable to GPU throughput.Source: §B.2 · p. 16

B.3.1 DAPT Checkpoint Trajectories

  • Nomic D2 fell from 0.1465 zero-shot to 0.0082 after DAPT, while subsequent TAPT reached 0.3665 versus 0.3668 for TAPT-only.GTE D1 also remained below its zero-shot score; overall, DAPT offered no benefit on FormBench.Source: §B.3 · p. 16, §B.3.1 · p. 17

B.3.2 TAPT Learning Curves

  • DAPT-initialised Nomic and GTE converge below their TAPT-only counterparts on C1, indicating that DAPT initialisation hinders rather than helps contrastive fine-tuning.The figure also shows zero-shot baselines for each architecture.Source: Figure 5 · p. 17
  • All three TAPT-MNRL architectures converged within the first 50–60% of training steps, but their best-checkpoint trajectories differed.Qwen3 peaked at ckpt-54 (75% of steps) before modest decline, while Nomic and GTE improved through the full training budget.Source: Figure 6 · p. 18

B.3.3 Hyperparameters

  • TAPT runs use MNRL with in-batch negatives, while Matryoshka wraps MNRL with multi-resolution dimension training.The reported runs use five epochs and bf16 precision on OLCF Frontier.Source: Table 10 · p. 18

C Analysis Details

  • The C1 efficiency frontier compares FAISS search latency with binary nDCG@10, and TAPT models dominate the Pareto frontier.Open markers denote base models, filled markers TAPT, and diamonds APIs.Source: Figure 7 · p. 19

C.1 CPC Clustering Results

CPC clustering favors API models, while TAPT retrieval gains do not translate into better clustering. The case studies illustrate both successful fine-grained retrieval and failures to distinguish highly specific concepts or adjacent passages.

C.1 CPC Clustering Results

  • API models lead CPC clustering at both seven-class and 80-class granularity, while TAPT models cluster no better than zero-shot counterparts.The 80-class API-leading pattern is stronger at finer granularity; Figure 10 shows TAPT developing finer structure within CPC clusters despite the clustering results.Source: Figure 8 · p. 19, §C.1 · p. 19, Figure 9 · p. 20

D.1 Case t7795_B — architectural coatings

  • Nomic-base and Qwen3-Embed TAPT-MNRL both retrieve the architectural-coatings anchor at rank 1, while OpenAI ranks it below 100.OpenAI instead retrieves a calcium-carbonate coating passage, matching surface chemistry vocabulary rather than the wax-seeded-latex concept.Source: §D.1 · p. 21

D.2 Case t14820_B — infrared-sensitive printing

  • All three TAPT-MNRL models retrieve the infrared-sensitive printing anchor at rank 1, improving on a best rank of 3 among base models.The case analysis attributes this to triplet fine-tuning making the electron-migration mechanism a retrieval signal.Source: §D.2 · p. 22

D.3 Case t14087_B — lubricant viscosity (hard case: all models fail)

The examples expose three distinct retrieval challenges: a rare cross-domain concept pair that defeats every model, a fine-grained coolant distinction where TAPT regresses, and adjacent passages within the correct patent.

D.3 Case t14087_B — lubricant viscosity (hard case: all models fail)

  • Every model ranks the lubricant anchor at least 90th, as C1 near-misses obscure the rare pairing of molecular-sieve crystallite size with low-viscosity lubricant formulation.Nomic retrieves a catalyst-deposition size passage, matching the sieve concept but missing the lubricant context; the query targets a crystallite size no greater than about 0.5 microns.Source: §D.3 · p. 23

D.4 Case q_01891 — automotive coolant corrosion

  • Qwen3-Embed TAPT-MNRL ranks the 2025 magnesium-specific coolant anchor second, while the base models retrieve it at rank 1.The TAPT model instead prefers a related 2023 heat-transfer-fluid patent; the authors suggest its richer representation in training triplets as a possible explanation.Source: §D.4 · p. 24

D.5 Case t15855_A — phosgene production catalyst (same-patent confusion)

  • All non-OpenAI models identify the correct phosgene patent but miss the targeted coating-properties passage in favor of an adjacent arrangement passage.The authors describe this as within-patent chunk discrimination, distinct from OpenAI’s cross-patent confusions.Source: §D.5 · p. 25

E Validation of Graded Qrel Tiers

An LLM-as-judge study and a paired domain-expert subsample test whether the benchmark’s anchor, hard-negative, and soft-negative relevance tiers reflect meaningful ordering. The results support the tiers on average, while indicating limits at fine-grained boundaries.

E Validation of Graded Qrel Tiers

  • 92.6% of 1,000 GPT-4o-judged tuples place the anchor first, and 64.7% rank the hard negative above the soft negative.The hard-over-soft rate is above chance with a lower confidence bound 11.8 percentage points above 50%; 35% of tuples disagree, consistent with close cases near tier boundaries.Source: E Validation of Graded Qrel Tiers · p. 26, Table 11 · p. 26
  • In the reported evaluation, perfect anchor > hard negative > soft negative ordering was achieved in 60.3% of tuples overall, with the reported interval [57.6, 63.8].The LLM-as-judge study found that graded tiers are meaningful on average but not a precise proxy for fine-grained human relevance judgments, supporting binary nDCG@10 as the primary metric.Source: E Validation of Graded Qrel Tiers · p. 26
  • On the paired 200-tuple subsample, human and GPT-4o hard-versus-soft ordering agrees in 78.6% of tuples.Human HN > SN is 61.1% and GPT-4o HN > SN is 64.3%; their intervals overlap.Source: E Validation of Graded Qrel Tiers · pp. 26–27, Table 12 · p. 27
  • The human validation uses one annotator and reports no multi-annotator inter-rater agreement statistics.The authors identify a second domain expert annotating an overlapping subsample as a natural extension.Source: E Validation of Graded Qrel Tiers · p. 27

F Responsible AI

The paper documents representational and deployment risks for FormBench, alongside its release, reproducibility, and maintenance provisions. Its scope is limited by patent and time coverage, while proprietary use may warrant legal review.

F Responsible AI

  • Four documented biases limit coverage: USPTO geography, English-only text, 1995–2022 coverage, and disproportionate representation of large patent-producing institutions.Niche or academic formulation research that does not produce USPTO patents is absent.Source: F Responsible AI · p. 27
  • Retrieval models fine-tuned on FormBench could accelerate large-scale identification of chemically similar formulations, raising potential competitive-intelligence or intellectual-property concerns.The authors advise legal review for proprietary deployments; CC-BY-4.0 requires attribution but does not restrict commercial use.Source: F Responsible AI · pp. 27–28
  • The work used OLCF Frontier and about 1.2 billion commercial-API tokens, but its total carbon footprint was not formally estimated.The OLCF allocation was subject to the institution’s carbon accounting.Source: F Responsible AI · p. 28

G Maintenance Plan

  • The dataset is hosted indefinitely on HuggingFace under CC-BY-4.0, with three BEIR-format corpus configurations downloadable through the datasets library.A single load_dataset call retrieves the files.Source: G Maintenance Plan · p. 28
  • Versioned releases preserve existing IDs and qrel scores within major versions; users should pin a version tag for reproducible evaluation.Breaking qrel-scoring changes require a major version increment; the current release is v1.0.Source: G Maintenance Plan · p. 28
  • A reference script reproduces both-corpus evaluations, while three TAPT checkpoints are released under Apache 2.0 and support SentenceTransformer loading.New models can be evaluated by implementing encode(texts); dataset issues and release enquiries can be sent to the authors or repository Discussions.Source: G Maintenance Plan · p. 28
Loading 2610.30175v1…