PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Embedding Geometries and Clustering Algorithms for Unsupervised Patent Claim Classification

Sujith Subramanian, P N Bhargav Teja, Susen Sarkar, Rajasekhara Reddy Duvvuru Muni

arXiv:2610.12997v1cs.AIcs.CL

TL;DR

Code-restricted patent search can miss relevant prior art filed under another classification code. This paper tests hyperbolic and matched Euclidean VAEs trained on patent text, finding more cross-category retrieval with the hyperbolic model. It also reports a measurable domain-level predictor and boundaries on the evidence.

  • Problem

    Code-restricted patent search cannot find relevant prior art filed under a different code, motivating tests of whether unsupervised patent-text models can recover those connections.

    Source: §I · p. 1

  • Method

    The paper compares a hyperbolic VAE with a matched Euclidean VAE on patent text, evaluating cross-category retrieval and agreement with official classification.

    Source: §I · p. 1

  • Results

    The hyperbolic model retrieves cross-category prior art 52% more often than the Euclidean model on the primary retrieval comparison.

    Source: 1) Primary Result: Cross-Category… · p. 3

  • Takeaways & Limitations

    Classification breadth is a concrete, significant predictor of where the approach performs well across domains.

    Source: §IX · p. 6

    Retrieval is measured by CPC-boundary crossing alone, and the breadth-performance correlation is based on only ten domains and warrants replication.

    Source: §VIII · p. 6

Abstract

from arXiv · show

Prior-art retrieval systems are structurally organized around classification codes, making cross-code discovery difficult. This paper asks if an unsupervised representation trained on patent text can recover relevant connections that code-restricted search cannot, and if hyperbolic space helps. We trained a hyperbolic and a matched Euclidean Variational Autoencoder (VAE) on 45,000 US patents (96 runs, 6 seeds). The hyperbolic model retrieves genuinely cross-category prior art 52% more often than the Euclidean model (18.76% vs. 12.34% of top-10 results, 3,000 queries). This represents a practical gain for the motivating search problem. The hyperbolic model agrees less with official classification (NMI 0.2392 vs. 0.3206), which we explain geometrically: the two measures capture different goals, and only one aligns with prior-art search needs. In a study of 10 technology domains (50,000 more patents), we find that a domain's classification breadth--the number of distinct CPC subgroups spanned--predicts approach performance (Pearson r=-0.642, p=0.045, N=10), and that patents per subgroup is an even stronger predictor (r=0.715, p=0.020). This provides a checkable signal for practitioners to determine where the method is most valuable. We report all findings with statistical transparency and release a complete, 91-test automated implementation.

I. INTRODUCTION

The paper tests whether unsupervised patent-text models can recover cross-category prior art missed by code-restricted search and whether hyperbolic geometry improves retrieval. It presents the retrieval gain, lower classification agreement, cross-domain study, and reproducible implementation as contributions.

I. INTRODUCTION

  • The hyperbolic model surfaces genuine cross-category prior art 52% more often than the Euclidean model on 3,000 benchmark queries.The paper frames this as a practical response to the structural gap in code-restricted search.Source: §I · pp. 1–2
  • The hyperbolic model agrees less well with official classification, a secondary result the paper explains geometrically rather than treating as contradictory.The paper presents the two findings as distinct measures of retrieval and classification agreement.Source: §I · p. 2
  • The extension tests generalization across 10 real technology domains and 50,000 further patents, using a corrected comparison after resolving a training defect.The defect had prevented two patent-pretrained models from training.Source: §I · p. 2
  • Classification breadth predicts performance across domains (Pearson r=-0.642, p=0.045, N=10), giving practitioners a checkable signal before deployment.The passage describes this predictor as statistically significant.Source: §I · p. 2
  • The paper releases a fully reproducible, tested implementation covering both studies, with 91 automated tests.Source: §I · p. 2

II. RELATED WORK

The paper situates its work among patent-text retrieval, hyperbolic representation learning, and unsupervised patent grouping, identifying a gap in geometry-focused retrieval evaluation and cross-domain analysis. Its primary study uses official USPTO patent data spanning 2004–2018.

II. RELATED WORK

  • Prior work applies patent sentence embeddings, hyperbolic representation learning, and unsupervised grouping, but the paper identifies no prior evaluation of hyperbolic geometry specifically for prior-art retrieval and cross-domain generalization.It compares geometry’s effect on retrieval separately from the embedding model itself.Source: §II · p. 2

III. DATA

  • The primary study uses the Harvard USPTO Patent Dataset, official USPTO bulk data from 2004–2018, scoped to CPC prefixes G06N, G06F, and G06Q for 45,000 patents.These prefixes cover AI, general digital data processing, and business data processing.Source: §III · p. 2
  • The primary-study patents average 2.26 CPC codes each, motivating a retrieval task that can cross classification codes.Source: §III · p. 2

IV. SYSTEM OVERVIEW

The system compares Euclidean and hyperbolic conditions while changing only the latent-model stage. Hyperbolic geometry is motivated by branching structure, and the VAE uses geometry-specific latent operations against a matched Euclidean comparison.

IV. SYSTEM OVERVIEW

  • Only the latent-model stage differs between the Euclidean and hyperbolic conditions compared throughout the paper.Source: Figure 1 · p. 2

A. Why Hyperbolic Geometry

  • Hyperbolic space’s exponentially growing volume gives branching technical hierarchies more room to spread than Euclidean space’s polynomially growing volume.The paper gives this geometric contrast as its reason for testing hyperbolic against Euclidean geometry.Source: A. Why Hyperbolic Geometry · p. 2, Figure 2 · p. 2

B. Text Representation and Hyperbolic VAE

  • Patent text is encoded as 384-dimensional MiniLM embeddings, and the hyperbolic VAE maps encoder outputs to the Poincaré ball and samples with a wrapped-normal distribution.Sampling draws Euclidean noise, parallel-transports it to the tangent space, then maps it onto the manifold.Source: B. Text Representation and Hyperbolic… · p. 2
  • The matched Euclidean VAE keeps the architecture identical while using standard Gaussian reparameterization and analytic KL; the decoder is Euclidean in both conditions.The hyperbolic KL term uses a single-sample Monte Carlo estimate because no closed-form solution is available above latent dimension two.Source: B. Text Representation and Hyperbolic… · p. 3, Figure 3 · p. 3

C. Why Evaluate Against CPC While Motivating the Work by Its Limits

The paper treats CPC agreement as a measure of reproduction of a legal-administrative taxonomy, not as a direct measure of technical similarity. It also describes stability and scalability fixes used in the implementation.

C. Why Evaluate Against CPC While Motivating the Work by Its Limits

  • CPC agreement measures reproduction of the existing legal-administrative taxonomy, whereas cross-category retrieval tests whether a method finds results code-restricted search cannot reach.The paper does not treat CPC as a gold standard for technical similarity.Source: C. Why Evaluate Against CPC While… · p. 3

D. Numerical Stability, Scalable Clustering, and a Corrected Defect

  • A curvature-relative safety clamp resolved negative mean KL estimates near the manifold boundary, and a sparse k-nearest-neighbor graph replaced an infeasible dense distance computation.The sparse graph used k=15 and showed 0.997 correlation with true Poincaré distance for near-origin points; connectivity-constrained Ward linkage clustered 45,000 points in under one minute.Source: D. Numerical Stability, Scalable… · p. 3
  • For cluster-count selection, the implementation replaces an unstable inconsistency-coefficient method with a largest-relative-jump detector that recovered a known count exactly.The reported decisive jump ratio was 53.8×.Source: D. Numerical Stability, Scalable… · p. 3
  • A matrix-shape mismatch that blocked two patent-pretrained embedding methods across domains, geometries, and seeds was fixed by deriving input dimension from the data.The paper says all Study 2 results reflect this correction, verified across the embedding dimensions used.Source: D. Numerical Stability, Scalable… · p. 3

E. Experimental Design

The study compares embedding and geometry configurations across six seeds and evaluates cross-category retrieval. Hyperbolic geometry delivers the central retrieval gain over the matched Euclidean model.

E. Experimental Design

  • The comparison varies text-field variant, geometry, latent dimension, and clustering method across six independent seeds in 96 Study 1 training runs.Retrieval is measured as the share of top-10 neighbors outside a query’s CPC subclass.Source: E. Experimental Design · p. 3

A. Single-Domain Results (Study 1)

2) Lower Classification Agreement as Expected Structural Evidence:

The hyperbolic model’s lower CPC agreement is presented as evidence of organizing patents differently from the administrative taxonomy, not as a contradiction of its retrieval result. Baseline comparisons show that simpler methods can score better on classification agreement.

2) Lower Classification Agreement as Expected Structural Evidence:

  • Hyperbolic geometry’s lower CPC agreement is described as the direct signature of organizing patents along technical rather than administrative lines.The authors connect that organization to more frequent cross-category prior-art retrieval.Source: 2) Lower Classification Agreement as… · p. 4
  • The hyperbolic VAE has the lowest fine-grained CPC agreement (NMI=0.2392), while Euclidean agrees more closely than every tested hyperbolic variant across six seeds.The authors interpret this divergence as expected because CPC is a legal-administrative taxonomy, not technical-similarity ground truth.Source: 3) Baseline Comparison · p. 4, 2) Lower Classification Agreement as… · p. 4

3) Baseline Comparison:

  • BERTopic has the highest NMI in the baseline comparison but operates with approximately 50% outliers.Table I compares NMI against fine-grained CPC codes.Source: 3) Baseline Comparison · p. 4, Table I · p. 4
  • Several simpler methods, including two requiring no VAE training, outperform the trained hyperbolic VAE on classification agreement.The authors identify cross-category retrieval, rather than classification agreement, as their primary measure of practical value.Source: 3) Baseline Comparison · p. 4

4) Qualitative Character:

The primary configuration yields clusters with mixed qualitative coherence, while Study 2 extends the framework to ten real technology domains. Its reduced-scale geometry comparison is described as directionally consistent with Study 1 but noisier.

4) Qualitative Character:

  • The primary configuration produces 25 clusters, ranging from a sharply coherent 2,101-patent touchscreen-related cluster to broad, overlapping vocabularies.The authors attribute the broad clusters to genuine technical heterogeneity rather than a labeling artifact.Source: 4) Qualitative Character · p. 4

B. Multi-Domain Extension (Study 2): Where Does This Approach Work Best

  • Study 2 extends the framework to ten real domains, with 5,000 patents per domain, and Table II lists their technology domains and CPC codes.The extension addresses Study 1’s single-domain scope limitation.Source: B. Multi-Domain Extension (Study 2):… · p. 4, Table II · p. 4
  • At the smaller per-domain scale, the Phase 1 geometry comparison is directionally consistent with Study 1 but noisier; its bars represent 30 observations each with one-standard-deviation error bars.Phase 1 compares embedding and geometry configurations on AI and software patents; Phase 2 applies the winning configuration to the other nine domains.Source: B. Multi-Domain Extension (Study 2):… · p. 4, Figure 5 · p. 5

1) Geometry Comparison at Reduced Scale:

At reduced per-domain scale, Euclidean has a small classification-agreement advantage, while the embedding-method comparison finds the tested pretrained methods essentially tied. Across ten domains, subgroup breadth and patents per subgroup are associated with mean representation quality.

1) Geometry Comparison at Reduced Scale:

  • 0.0476 is Euclidean’s classification-agreement score at 5,000 patents per domain, versus 0.0372–0.0408 for hyperbolic, with a 0.0217 Euclidean standard deviation.The result is directionally consistent with Study 1 but noisier, attributed to the roughly ninefold smaller corpus per domain.Source: 1) Geometry Comparison at Reduced Scale · p. 5

2) Embedding Method Comparison:

  • After correcting the dimension-mismatch defect, all five embedding methods trained; patent-pretrained methods and MiniLM were essentially tied at 0.0499–0.0508.All three roughly doubled TF-IDF and the hybrid method, which scored 0.0246–0.0263.Source: 2) Embedding Method Comparison · p. 5
  • Across the ten domains, pharmaceutical scores lowest and electric vehicles highest, with the latter 3.7 times higher; AI and software are near, but not at, the bottom.The comparison applies the winning Phase 1 configuration across all ten domains, with six seeds each.Source: Figure 6 · p. 5
  • Classification breadth correlates negatively with mean score across ten domains (Pearson r=-0.642, p=0.045), while patents per subgroup correlates more strongly (r=0.715, p=0.020, N=10).Electric vehicles and wireless/5G have 3.18 and 3.31 patents per subgroup; semiconductors and materials analysis have 1.28 and 1.32.Source: 3) A Practical Predictor:… · p. 5, Figure 7 · p. 5
  • The authors attribute the subgroup-density pattern to the number of examples available for discovering each category’s structure.They present a fast, data-only calculation as a preliminary signal for likely performance, noting that the correlation is based on only ten domains and warrants replication.Source: 3) A Practical Predictor:… · pp. 5–6

4) Multi-Label Evaluation and Label-Quality Validation:

Study 2 accounts for patents’ multiple classification codes and validates label quality against terms already confirmed as discriminative.

  • Because patents average 2.26 CPC codes, Study 2 credits a match against any of each patent’s real CPC, IPC, or USPC codes.Source: 4) Multi-Label Evaluation and… · p. 6
  • Labeling outputs are checked against a fixed list of terms confirmed as genuinely discriminative in Study 1.Source: 4) Multi-Label Evaluation and… · p. 6
  • The validation caught a grammatical filter excluding valid adjectival terms such as “virtual” and a failure to credit terms within multi-word phrases.The authors describe this check as reusable for label-quality validation generally.Source: 4) Multi-Label Evaluation and… · p. 6

VI. DISCUSSION

The discussion relates lower classification agreement to the retrieval objective and presents domain density as a practical signal for expected performance and geometry choice.

VI. DISCUSSION

  • The lower CPC agreement is the structural signature of the retrieval gain: a model seeking relevant art beyond a query’s code should disagree with that code more.Source: §VI · p. 6

VII. PRACTICAL IMPLICATIONS AND LESSONS

  • Before model training, practitioners can compute patents per CPC subgroup as a fast, cheap signal for expected performance in a new domain.Source: §VII · p. 6
  • Domains below roughly two patents per CPC subgroup scored 0.060–0.090, while domains above roughly 2.5 scored 0.107–0.223, with no overlap across the ten studied domains.The paper presents this as an observed heuristic, not a validated threshold, pending replication with more domains.Source: §VII · p. 6
  • Within a domain passing the density check, the paper recommends hyperbolic geometry for cross-category discovery and Euclidean geometry for closer legal-classification alignment.Source: §VII · p. 6

VIII. LIMITATIONS AND FUTURE WORK

The authors qualify the domain findings by study scale and metric scope, and conclude that hyperbolic retrieval and classification agreement answer different questions.

VIII. LIMITATIONS AND FUTURE WORK

  • Study 2 used 5,000 rather than 45,000 patents per domain, yielding a noisier geometry signal; its significant breadth-performance correlation covers only ten domains and warrants replication.Source: §VIII · p. 6
  • The trained VAE underperforms simpler untrained baselines on classification agreement, uses a Euclidean decoder, and is evaluated with metrics across studies that are not directly comparable in absolute terms.Source: §VIII · p. 6
  • Retrieval is assessed only through CPC-boundary crossing; independent expert review of result relevance remains a proposed next step.Source: §VIII · p. 6

IX. CONCLUSION

  • The hyperbolic VAE retrieved cross-category prior art 52% more often than the matched Euclidean model, while agreeing less well with official classification.Source: §IX · p. 6

Related papers

Loading 2610.12997v1…