PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparse Coverage: Semantic Center Representations for Patent Prior-Art Retrieval

You Zuo, Kim Gerdes, Éric de la Clergerie, Benoît Sagot

arXiv:2608.16918v1cs.IRcs.AI

TL;DR

Patent prior-art retrieval must find relevant documents in long, structured technical texts, but single dense vectors can bottleneck multiple components and constraints. Sparse Coverage maps local span embeddings to sparse semantic centers for inverted-index retrieval, matching or exceeding strong dense encoders on document-level recall in several CLEF-IP 2013 configurations while remaining competitive for passage retrieval.

  • Problem

    Single-vector patent representations may compress multiple technical components, functions, constraints, and alternatives, limiting semantic coverage in recall-oriented prior-art retrieval.

  • Method

    Sparse Coverage maps local contextual span embeddings to a coverage-oriented vocabulary of semantic centers, activating nearby centers for sparse inverted-index retrieval.

  • Results

    99.31 Recall@100 is achieved at the document level with PaECTER, NP+tok. units, and V = 50k, exceeding the strongest dense baseline in the reported setup.

  • Takeaways & Limitations

    Sparse Coverage is an effective high-recall first-stage retriever that combines local semantic evidence with sparse index-based candidate selection.

  • Takeaways & Limitations

    Sparse Coverage is not a universal replacement for dense ranking because dense encoders remain stronger on some document-level and passage-level ranking metrics.

Abstract

from arXiv · show

Patent prior-art retrieval is a recall-oriented search task over long and highly structured technical documents. Dense retrieval improves semantic matching, but single-vector representations may compress multiple technical components, functions, and constraints into a single embedding. We propose Sparse Coverage, an unsupervised semantic retrieval framework that maps local span embeddings to a sparse vocabulary of embedding-space centers. The centers are selected with a coverage-oriented k-center objective, and spans activate nearby centers to produce sparse representations compatible with inverted-index retrieval. Experiments on CLEF-IP 2013 show that Sparse Coverage matches or exceeds the document-level recall of strong dense patent encoders in several configurations, while remaining competitive for passage-level retrieval. By combining local semantic evidence with sparse inverted-index search, Sparse Coverage provides an effective first-stage retrieval approach for patent search.

1 Introduction

Patent prior-art retrieval is a recall-oriented search problem over long, structured, specialized documents, where single-vector dense representations may compress diverse technical content into one embedding. Sparse Coverage addresses this bottleneck by mapping local span embeddings to sparse semantic centers compatible with inverted-index retrieval and evaluating the approach on CLEF-IP 2013.

  • Motivation: Patent prior-art search retrieves earlier patents with related technical ideas for examination, invalidity analysis, and technology landscaping.The task is especially challenging because patent documents are long, highly structured, and written in specialized technical language.
  • Problem: Single fixed-dimensional vectors create a capacity bottleneck by compressing multiple components, functions, constraints, and alternatives into one embedding-space point.Dense neural retrievers improve semantic matching beyond lexical overlap, but long technical queries and documents remain difficult to represent compactly.
  • Contribution: Sparse Coverage represents text through local semantic regions activated by contextual span embeddings, relaxing the single-vector bottleneck while preserving inverted-index search.Patent text is decomposed into encoder tokens, noun phrases, or hybrid span units and embedded with a pretrained encoder.
  • Evaluation: The framework is evaluated for document- and passage-level retrieval on the CLEF-IP 2013 prior-art benchmark against lexical, dense, neural sparse, and late-interaction baselines.The experiments also analyze how retrieval behavior changes with vocabulary size V.

2 Related Work

Prior-art retrieval has progressed from structure-aware sparse search to patent-oriented neural encoders and neural alternatives to single-vector dense retrieval. Sparse Coverage is closest to embedding-based inverted indexing but selects geometrically covering span-embedding centers rather than clustering meanings within lexical tokens.

  • Patent retrieval: CLEF-IP established large-scale EPO benchmarks for document-level and claim-to-passage patent retrieval.These benchmarks derive from patent examination reports.
  • Patent retrieval: Early patent retrieval used sparse search with query engineering, structure-aware heuristics, noun-phrase units, and metadata such as IPC codes.Approaches included field-based lexical matching, keyphrase extraction, query expansion, and patent-metadata filtering.
  • Patent retrieval: Recent patent-oriented neural models include PatentSBERTa, PaECTER, SearchFormer, QaECTER, PatEmbed, and self-supervised intra-document-view encoders.These models provide strong semantic representations for patents.
  • Neural retrieval beyond single-vector embeddings: Neural sparse models retain inverted-index retrieval through contextual term weighting or vocabulary expansion, while SparseEmbed combines sparse lexical activations with contextual embeddings.The passage characterizes these methods as improving lexical sparse retrieval.
  • Neural retrieval beyond single-vector embeddings: Late-interaction systems such as ColBERT preserve local matching evidence through token-level dense representations and MaxSim scoring, but require specialized indexing and scoring.This contrasts with single-vector encoders, which may lose local matching evidence.
  • Closest prior approach: Sparse Coverage builds an embedding-derived inverted index using centers that geometrically cover span-embedding space rather than clustering meanings within lexical tokens.The centers are retrieval-oriented semantic units optimized for coverage rather than reconstruction.

3 Methodology

Sparse Coverage builds an offline semantic vocabulary from local span embeddings, then uses adaptive center activation and sparse aggregation to represent queries and corpus units. Inverted-index scoring combines shared-center semantic matching with normalization and frequency-based suppression of generic centers.

  • Span construction: Local spans are formed from encoder tokens, noun phrases, or a hybrid of noun phrases and uncovered encoder tokens, then mean-pooled into embeddings.The pretrained encoder maps text to contextual token embeddings before span construction.
  • Center selection: Semantic centers are selected with a coverage-oriented k-center objective, using farthest-first traversal so rare or peripheral embedding regions are not ignored.The vocabulary size V controls the granularity of semantic covering.
  • Center activation: Each span activates centers whose center-specific radii cover it, while a top-K cap retains the highest-similarity centers to limit dense-region overlap.High-percentile radii reduce outlier sensitivity and allow different activation thresholds across semantic regions.
  • Center-level aggregation: Center weights use the maximum similarity across activating spans, with corpus-side span-count normalization reducing the advantage of longer corpus units.Queries use the same center vocabulary, but the described length normalization is applied on the corpus side.
  • Sparse interaction scoring: Retrieval traverses an inverted index over centers, suppressing the most frequent generic centers and applying IDF weighting to calibrate their document-frequency effects.Document frequency is computed after center activation because geometric span coverage does not guarantee balanced document-frequency statistics.

4 Experimental Settings

Experiments use the English CLEF-IP 2013 claims-to-passages benchmark with document- and passage-level evaluation. The protocol constructs a controlled candidate corpus, applies two-stage retrieval metrics, and compares Sparse Coverage with lexical, dense, neural sparse, and late-interaction baselines under defined preprocessing and model settings.

  • Dataset: Evaluation uses 48 usable English patent-claim queries from CLEF-IP 2013, with passage-level relevance annotations and ancestor independent claims prepended for dependent queries.The benchmark originally defines 50 English topics, but two are excluded because their source XML and distributed family documents lack claims.
  • Dataset: The evaluation corpus contains 17,323 usable documents and 1.4M passages, after constructing a 25,000-document pool with all 90 relevant documents and hard-negative sampling.Negative sampling uses 75% IPC-based hard negatives and 25% uniformly sampled negatives; queries average about 3.4 relevant documents and 67.5 relevant passages.
  • Protocol and metrics: Passages are first ranked and deduplicated by document, with the top-100 unique documents defining document metrics and the candidate set for passage reranking.All passages from those top-100 documents are then pooled and reranked by their original retrieval scores.
  • Protocol and metrics: Document retrieval reports Recall@100, mAP, and PRES@100, while passage retrieval reports official CLEF-IP mAP(D).Recall@100 and PRES@100 use the top-100 document ranking, whereas mAP uses the full deduplicated ranking.
  • Baselines: Sparse Coverage is compared with BM25, four fixed-vector dense encoders, SPLADE-v2, and ColBERTv2, using sentence-aligned chunking for claims exceeding encoder limits.Dense baselines are SPECTER2, BERT-for-Patents, PaECTER, and PatentMap-V0-SecPair-Claim; SPLADE and ColBERTv2 are off-the-shelf baselines.
  • Sparse Coverage settings: The default Sparse Coverage setting uses soft assignment to up to K = 5 centers per span, removes the top 1% frequent centers, applies max aggregation and α = 2.0 IDF weighting, and evaluates V ∈ {10k, 20k, . . . , 50k}.Vocabularies cover encoder-token, noun-phrase, and hybrid noun-phrase-plus-token spans across BERT-for-Patents, PaECTER, and PatentMap-V0 embedding spaces.

5 Results

Sparse Coverage delivers strong high-recall retrieval, including the best reported document Recall@100 and passage MAP(D) configurations, while its performance depends on span granularity and vocabulary size. It also offers a favorable cost–recall trade-off, although dense encoders remain stronger on some ranking metrics.

  • 99.31 Recall@100 is achieved with PaECTER, NP+tok. units, and V = 50k, exceeding the strongest dense, neural sparse, and late-interaction baselines.PatentMap-V0 with noun-phrase units also reaches 97.71 Recall@100, slightly above its dense counterpart.
  • 30.03 MAP(D) is achieved by BERT-for-Patents with encoder-token units at V = 40k, while V = 30k reaches 29.54 MAP(D).Converting the dense BERT-for-Patents baseline into Sparse Coverage yields more than 27 absolute points in document Recall@100 and more than 10 absolute points in MAP(D), corresponding to relative improvements above 40% and 50%, respectively.
  • Dense PaECTER and PatentMap-V0 remain stronger on document-level mAP and PRES@100, while PatentMap-V0 achieves the highest passage Recall@1000.Sparse Coverage is therefore positioned as a high-recall candidate generator rather than the strongest method for every ranking metric.
  • Vocabulary size and span granularity affect results: document Recall@100 peaks at intermediate V for encoder-token and noun-phrase units, whereas hybrid NP+tok. units benefit more consistently from larger vocabularies.Increasing V generally improves document mAP, but its effects on document Recall@100 and passage MAP(D) are less monotonic; fine centers can also weaken matches when spans activate neighboring centers.
  • Encoder-token units achieve the highest average document mAP and passage MAP(D), while noun-phrase units remain useful for document recall, including 97.71 Recall@100 for PatentMap-V0 at V = 20k.The result indicates that fine-grained local evidence is especially important for early ranking and passage-level retrieval.
  • Sparse Coverage occupies a favorable cost–recall region: many configurations exceed BM25’s Recall@100 while scanning fewer postings, and larger V trades shorter posting lists against more center lists accessed per query.This opposing balance makes retrieval cost non-monotonic in V and can allow larger vocabularies to improve recall in some settings.

6 Conclusion

Sparse Coverage is an unsupervised framework that maps local patent span embeddings to sparse semantic center activations for inverted-index retrieval. On CLEF-IP 2013, it is an effective first-stage retriever, matching or exceeding strong dense patent encoders on document-level recall in several configurations while remaining competitive for passage-level retrieval.

  • 6 Conclusion: Sparse Coverage maps local patent span embeddings to sparse semantic center activations for inverted-index retrieval.The framework is unsupervised.
  • 6 Conclusion: On CLEF-IP 2013, Sparse Coverage matches or exceeds strong dense patent encoders for document-level recall in several configurations.This result supports its effectiveness as a first-stage retriever.
  • 6 Conclusion: Sparse Coverage remains competitive for passage-level retrieval while using shared center activations to represent patents.The shared activations combine local patent-span evidence for retrieval.

Références

The references span prior work on patent passage and prior-art retrieval, patent text representations, sparse inverted-index methods, and theoretical foundations for embedding-based search. They also include benchmark, evaluation, clustering, and domain-specific NLP studies.

  • Patent retrieval and representations: The bibliography includes studies of patent passage retrieval, prior-art search, patent embeddings, classification, and citation-informed representation learning.These works include Andersson et al. (2013), Verma and Varma (2011), Vowinckel and Hähnke (2023), Ayaou and Cavallucci (2025), and Ghosh et al. (2024).
  • Sparse retrieval: The references connect sparse retrieval to learned representations, contextualized inverted lists, and embedding-based inverted-index construction.They cite SPLADE v2, COIL, and efficient sparse retrieval through embedding-based inverted-index construction.
  • Methods and evaluation: The bibliography includes k-center clustering and recall-oriented retrieval evaluation as methodological foundations.Gonzalez (1985) addresses minimizing maximum intercluster distance, while Magdy and Jones (2010) propose a score metric for recall-oriented retrieval applications.
  • Related resources and limitations: Additional references cover scientific document representation benchmarks, biomedical NLP, BERT for patents, and theoretical limitations of embedding-based retrieval.These works include SciRepEval, SciSpaCy, Leveraging the BERT algorithm for Patents, and On the theoretical limitations of embedding-based retrieval.

A Statistics of Evaluation Data

The default CLEF-IP 2013 English evaluation corpus contains 17,323 usable documents and about 1.4M passages, with only 90 relevant documents and long, structurally complex queries. This highly imbalanced, recall-oriented setting motivates span-level representations and document-level recall evaluation.

  • Corpus construction: 17,323 usable documents and about 1.4M passages remain after XML parsing and passage extraction from the initially targeted 25,000-document pool.The pool targeted all relevant documents, 75% IPC-based hard negatives, and 25% random negatives.
  • Evaluation imbalance: 90 of 17,323 documents are relevant, containing 9,117 passages mixed with more than 1.4M passages from non-relevant documents.Each query averages 3.4 relevant documents, making document-level recovery a small-set search problem.
  • Evaluation rationale: The benchmark’s imbalance motivates recall-oriented metrics such as Recall@100 and PRES@100, alongside span-level representations and document-level recall evaluation.The long queries’ technical constraints, components, and functional relations motivate span-level representations.
  • Query complexity: 585 words is the mean query length after ancestor expansion of dependent claims, with a 407-word median.Queries may contain several technical constraints, components, and functional relations.

B Examples of CLEF-IP 2013 Data

CLEF-IP 2013 queries are citing patent claims, while retrieval targets are cited prior-art documents and passages. The examples illustrate extreme variation in query length, specialized terminology, and the fine-grained imbalance between citing and cited units.

  • Query and ground-truth structure: Queries represent citing-document claims, and retrieved targets are cited prior-art documents and passages.The examples explicitly pair each source claim set with ground-truth relevant documents and passages.
  • Representative examples: PSG-7 contains 1 claim spanning 79 words, with 2 relevant documents containing 12 relevant passages.Its query concerns activating and deactivating automatic noise cancellation in a mobile station.
  • Representative examples: PSG-34 contains 7 claims spanning 332 words, with 3 relevant documents containing 53 relevant passages.The query concerns a rubber composition for preparing cable connector seals.
  • Representative examples: PSG-26 contains 8 claims spanning 457 words, with 8 relevant documents containing 429 relevant passages.The query concerns a vehicle curve-approach control device using curve and turning-motion information.

C Analysis of Center Construction

Center-vocabulary behavior depends jointly on encoder geometry, span granularity, and vocabulary size. Larger vocabularies improve embedding compactness and relative sparsity, but compact geometry can coincide with greater posting-list imbalance.

  • Embedding compactness: Larger vocabularies produce finer span-embedding partitions, with median Voronoi radius decreasing as V increases.PaECTER has substantially smaller radii than PatentMap and BERT-for-Patents, while encoder-token units are consistently more dispersed than phrase-aware units.
  • Representation sparsity: Activation rates decrease consistently as V grows, making span representations sparser relative to the full vocabulary.Encoder-token units activate a larger fraction of centers than noun-phrase and hybrid units; PaECTER has lower encoder-token activation rates than the other encoders.
  • Posting list balance: Compact embedding geometry does not imply balanced posting lists: PaECTER has the strongest overall skew, especially for noun-phrase units.BERT-for-Patents shows a similar but weaker pattern, while skew tends to increase with V even as average posting-list length decreases.
  • Overall implications: The analysis motivates combining geometric center selection with Top-K sparsification, stop-center suppression, and IDF weighting.The k-center objective encourages embedding-space coverage but does not control how often centers are activated across documents.

D Additional Hyperparameter Ablations

The ablations show that Sparse Coverage benefits from controlled center overlap, IDF weighting, and corpus-side span-count normalization. Soft assignment with a small top-K cap, α = 3.0 in the sweep, and the default γ = 0.5 produce the strongest trends, although the main experiments retain α = 2.0.

  • Ablation setup: The sweep covers assignment strategy, IDF exponent α, and corpus-side normalization exponent γ across 20 configurations per setting.It averages PaECTER and PatentMap-V0 with encoder-token and noun-phrase units over V ∈{10k, 20k, 30k, 40k, 50k}.
  • Assignment strategy: Soft assignment outperforms hard nearest-center assignment, while K = 3 and K = 5 perform similarly.Radius-based overlap helps recover matches across neighboring centers, whereas excessive overlap can add noisy activations and longer posting lists.
  • IDF weighting: IDF weighting improves averaged retrieval results: α = 0 is weakest, while α = 3.0 performs best among tested values.The main experiments use α = 2.0 as a moderate setting rather than tuning on the evaluation set.
  • Corpus-side normalization: Square-root span-count normalization gives the best average results, whereas γ = 0 reduces retrieval quality.The effect is especially strong for encoder-token units because longer corpus units have more opportunities to activate query centers and can otherwise receive inflated scores.

E Full Sparse Coverage Results

Full Sparse Coverage results across encoder, span-unit, and vocabulary-size combinations show that performance varies by configuration. Encoder-token units generally favor ranking-oriented performance, while noun-phrase and hybrid units can achieve very high document-level recall in specific configurations.

  • Configuration effects: Performance depends jointly on the encoder and span unit across the full Sparse Coverage configurations.Table 4 reports results across all encoder, span-unit, and vocabulary-size combinations.
  • Span-unit trends: Encoder-token units tend to provide stronger ranking-oriented performance.This trend is reported across the full non-averaged results.
  • Span-unit trends: Noun-phrase and hybrid units can achieve very high document-level recall in specific configurations.The strongest configuration depends on the encoder and span unit.
  • Results presentation: Table 4 presents the full results, with best overall results highlighted in bold and second-best overall results underlined.The table covers CLEF-IP 2013 and varies encoder, span unit, and vocabulary size.
Loading 2608.16918v1…