PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Patent Drafting from Inventor-Style Disclosures

Lekang Jiang, Wenjun Sun, Stephan Goetz

arXiv:2608.21249v1cs.CL

TL;DR

The paper addresses the gap between real-world patent drafting from informal inventor disclosures and benchmarks built from later-stage legalistic inputs. It introduces Dis2Pat and Patent-MAF, a locally deployable multi-agent framework, and reports that the framework is a strong baseline against open-source systems and competitive with closed-source models. The dataset remains limited by reliance on pseudo-disclosures and primarily English-language evaluation.

  • Problem

    Complete, legally coherent patent-application generation from early-stage inventor-style disclosures remains insufficiently studied because prior work typically uses later-stage, structured, or legalistic inputs.

    Source: §1 · p. 1

  • Method

    The paper introduces Dis2Pat, built from controlled LLM-generated pseudo-disclosures, and Patent-MAF, which coordinates manager, drafter, and polisher agents for local patent drafting.

    Source: §1 · pp. 1–2, §4 · p. 4

  • Results

    Patent-MAF is a strong locally deployable baseline that outperforms evaluated open-source models and remains competitive with state-of-the-art closed-source models.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Dis2Pat establishes a challenging benchmark for disclosure-to-patent drafting, while Patent-MAF provides a privacy-preserving baseline for this setting.

    Source: §7 · p. 9

    Dis2Pat relies on pseudo-disclosures that may not fully capture the diversity, incompleteness, or ambiguity of real-world inventor disclosures, and Patent-MAF is evaluated primarily on English-language patents.

    Source: §3.1 · p. 3, Limitations · p. 9

Abstract

from arXiv · show

While recent large language models (LLMs) have achieved promising results on individual patent drafting tasks, they fundamentally fail to investigate the core challenge of real-world patent drafting: generating a complete and legally coherent patent application directly from early-stage invention materials. Prior work predominantly assumes later-stage, highly structured, or already legalistic inputs. However, real patenting workflows begin with informal, de-legalized disclosures authored by inventors. To bridge the gap, we introduce Dis2Pat, a disclosure-to-patent dataset that reflects realistic patenting workflows by requiring the generation of complete patent applications directly from inventor-style, de-legalized disclosures. Given the inherent difficulty of long-form, legally constrained patent drafting and the strong privacy requirements, we further propose a strong baseline named Patent-MAF. It is a multi-agent framework for locally deployable patent drafting. Benchmark results reveal that current LLMs exhibit limitations in patent drafting, while Patent-MAF provides a strong baseline that consistently outperforms evaluated open-source models and remains competitive with large closed-source models.

1 Introduction

Real-world patent drafting requires turning informal inventor disclosures into complete, legally structured applications, a challenge largely bypassed by prior benchmarks. Dis2Pat addresses this gap with a disclosure-to-patent dataset, while Patent-MAF provides a privacy-aware multi-agent baseline that performs strongly against open- and closed-source models.

  • Complete patent-application generation from early-stage invention materials remains an open problem despite progress on individual claim and specification tasks.Source: §1 · p. 1
  • Most existing benchmarks use later-stage, highly structured inputs and therefore bypass translation from inventor-style disclosures into legally structured applications.Source: §1 · p. 1
  • Dis2Pat requires models to generate complete applications, including claims and a full specification, from de-legalized inventor-style disclosures accompanied by figures.Source: §1 · pp. 1–2
  • Patent-MAF uses a manager, specialized drafter agents, and a polisher to support locally deployable patent drafting under privacy and confidentiality constraints.Source: §1 · p. 2
  • Dis2Pat introduces a benchmark incorporating inventor-style disclosures, associated figures, and patent texts for disclosure-to-patent generation.Source: §1 · p. 2
  • Patent-MAF consistently outperforms evaluated open-source models and achieves performance competitive with state-of-the-art closed-source models.Source: §1 · p. 2

2 Related Works

Prior patent-generation research covers claim drafting, specification generation, and end-to-end application drafting, but commonly relies on formal or technically structured inputs. Dis2Pat instead targets inventor-style, nonlegal disclosures with drawings.

  • Existing patent-generation work spans claim-centric generation, specification generation, and end-to-end patent-application drafting.Source: §2 · p. 2

Claim Drafting

  • Claim-drafting studies progressed from short inputs such as abstracts to larger datasets using full patent specifications.Source: §2 · p. 2

End-to-End Patent Drafting

  • Draft2Patent transforms patent drafts into complete applications, while Pap2Pat converts scientific articles into patent applications.Source: §2 · p. 3

Disclosure-to-Patent

  • Dis2Pat focuses on natural-language, nonlegal inventor-style disclosures accompanied by drawings, aligning more closely with real-world drafting workflows.Source: §2 · p. 3

3 The Dis2Pat Dataset

Dis2Pat constructs a disclosure-to-patent dataset from granted patents because authentic invention disclosures are confidential and legally constrained. The resulting pseudo-disclosures remove legalistic structure while preserving technical content, and the dataset is filtered and evaluated for quality.

3.1 Construction

  • Because real-world invention disclosures are not publicly accessible, Dis2Pat creates pseudo-disclosures from publicly available granted patents using LLM-assisted rewriting.Source: §3.1 · p. 3
  • The construction process extracts technical content, removes patent-specific legal style, and blurs feature-focused patent language.Source: §3.1 · p. 3
  • Each pseudo-disclosure uses seven invention-centric components covering the title, motivation, core idea, operation, differences, benefits, and optional variants.Source: §3.1 · p. 3

3.2 Statistics and Quality Analysis

  • Pseudo-disclosures average 1,196 tokens, substantially shorter than original patents, while the technical term density decreases from 0.05 to 0.001.Source: §3.2 · pp. 3–4
  • Patent professionals evaluated 100 sampled disclosures for hallucination, missing details, contradiction, and de-legalization quality.Source: §3.2 · p. 4
  • An approximately 0.1 Cohen’s d between evaluation subsets indicates negligible score variation and supports evaluation reliability.Source: §3.2 · p. 4

4 Patent-MAF

Patent-MAF generates claims and specifications from invention disclosures through coordinated manager, drafter, and polisher agents. Its design emphasizes local privacy-preserving deployment, domain-grounded role decomposition, and multimodal figure use.

  • Patent-MAF first has a manager structure the disclosure and coordinate specialized drafting agents, then uses a polisher to refine the patent text.Source: §4 · p. 4, Figure 1 · p. 5
  • The claim drafter generates a complete claim set, while the specification drafter produces the background, summary, and detailed embodiments.Source: §4 · p. 5
  • Patent-MAF runs all agents on open-source models without proprietary API calls, supporting privacy-aware local deployment for sensitive intellectual property.Source: §4 · p. 5
  • Its manager, drafter, and polisher roles mirror professional patent-prosecution division of labor rather than generic task decomposition.Source: §4 · p. 5
  • The vision-enabled specification drafter uses figures to resolve spatial and structural ambiguities.Source: §4 · p. 5

5 Experiments

The experiments evaluate patent claims and specifications using automatic, model-based, and expert metrics across text-only and multimodal systems, including Patent-MAF variants and strong baselines.

Experimental Comparisons

  • Experiments compare Patent-MAF and ablations with diverse open-source and closed-source LLM baselines under the same decoding configuration.Ablations remove the manager, visual input or claim fine-tuning, and the polisher.Source: §5.1 · pp. 5–6

Patent-MAF Configuration

  • Patent-MAF combines a disclosure-structuring manager, specialized drafting agents, and a global consistency and style polisher.The specification drafter processes text and figures, while the claim drafter uses fine-tuning for patent-claim conventions.Source: §5.1 · p. 6

Evaluation Setup

  • Models are evaluated with lexical-overlap, semantic-similarity, LLM-judge, and expert human metrics for both specifications and claims.The evaluation includes BLEU, ROUGE-1, ROUGE-L, BERTScore, and BERT-for-Patent measures alongside quality assessments.Source: §5.2 · p. 6
  • Specification evaluation measures completeness, logical clarity, and legal quality.Completeness covers technical background, problem formulation, solutions, implementation details, and embodiments.Source: §5.2 · p. 6
  • Claim evaluation measures technical scope, claim legality, and logical clarity.These criteria assess whether claims capture the inventive concept and comply with professional drafting standards.Source: §5.2 · p. 6

6 Results and Analysis

Results show that visual inputs help specification drafting but not claim generation, while Patent-MAF achieves strong specification performance and benefits substantially from coordinated components.

Specification Generation

  • Visual inputs slightly improve specification quality, including Qwen3-VL-72B’s increase from 81.1 to 81.8 and GPT-5’s BERT-for-Patent increase from 95.7 to 96.4.The analysis attributes the benefit to visual grounding of spatial and structural descriptions.Source: §6.1 · p. 6
  • 85.4 overall content quality makes Patent-MAF the strongest evaluated open-source system and higher than Qwen3-VL-72B at 81.8 and GPT-5 at 85.1.Patent-MAF also records BERTScore 82.3 and BERT-for-Patent 96.1, compared with GPT-5’s 81.8 and 96.4.Source: §6.1 · p. 7

Claim Generation

  • Visual inputs do not improve claim generation: Qwen3-VL-72B falls from 80.7 to 79.8 and GPT-5 from 91.1 to 90.7.The same downward trend extends to text-overlap and semantic-similarity metrics.Source: §6.2 · p. 7

Ablation Study

  • Removing the polisher causes the largest claim-generation degradation, reducing the overall score by 6.6 points from 86.7 to 80.1.Removing fine-tuning lowers the score to 81.6, while removing the manager lowers it to 84.2.Source: §6.2 · p. 8

Overall Patterns

  • Specification drafting benefits more from multimodal inputs and global consistency refinement, whereas claim generation is more sensitive to textual modeling and task-specific fine-tuning.The two tasks therefore exhibit distinct component sensitivities.Source: §6.3 · p. 8

Human Evaluation

  • Patent-MAF is preferred over all open-source baselines across completeness, clarity, legality, and consistency in majority-of-cases pairwise evaluations.Against GPT-5, GPT-5 is preferred more often on completeness, clarity, and legality, while consistency is evenly matched and ties remain substantial.Source: §6.3 · p. 8

Overall Patterns

  • Explicit task decomposition and refinement appear particularly beneficial for comprehensive coverage and coherence between claims and specifications.This conclusion is reported even in comparison with a state-of-the-art closed-source model.Source: §6.3 · p. 9

7 Conclusion

The paper contributes Dis2Pat, a realistic disclosure-to-patent benchmark, and Patent-MAF, a locally deployable multi-agent baseline for privacy-sensitive drafting.

Contributions

  • Dis2Pat models disclosure-to-patent drafting from inventor-style disclosures with associated figures and patent texts.Its inputs are designed to represent realistic, de-legalized drafting materials.Source: §7 · p. 9
  • Patent-MAF coordinates manager, specialized drafter, and polisher agents to generate coherent patent applications with claims and specifications.The framework is optimized for locally deployable open-source models and avoids reliance on external APIs.Source: §7 · p. 9

Conclusion

  • Overall results establish Dis2Pat as a challenging benchmark and Patent-MAF as a strong locally deployable baseline.The conclusion positions the framework within disclosure-to-patent drafting rather than isolated drafting tasks.Source: §7 · p. 9

Limitations

The study’s main limitations concern the realism and scope of its data and evaluation setting.

Dataset Scope

  • Dis2Pat uses pseudo-disclosures extracted and rewritten from granted patents, which may not capture the diversity, incompleteness, or ambiguity of real inventor disclosures.The authors identify partnerships for anonymized or partially synthetic disclosures as a future direction.Source: Limitations · p. 9

Evaluation Scope

  • Patent-MAF is evaluated primarily on English-language patents, leaving multilingual and jurisdiction-specific drafting as future work.The paper specifically mentions EPO and CNIPA standards.Source: Limitations · p. 9

Experimental Scope

  • Inference hyperparameters remain fixed rather than tuned, preserving fair comparisons but limiting inference-time optimization.This choice is stated as an experimental constraint.Source: Limitations · p. 9

Ethics Statement

The paper documents artifact licensing, data-content safeguards, and the absence of ethics-board review. It also situates the work within patent-document structure, NLP challenges, dataset construction, and implementation details.

Ethics and licensing

  • The dataset is released under CC-BY-SA-4.0, while the listed models and artifacts use their respective stated licenses.Llama-3 uses the Meta Llama 3 Community License, Qwen-3 uses Apache 2.0, and GPT models are accessed through OpenAI’s API under a commercial license.Source: Ethics Statement · p. 9
  • The data contains no potential personal information or offensive content, and no ethics review board was involved.Source: Ethics Statement · p. 9

Patent context

  • Patent documents combine legally authoritative claims with supporting specifications, figures, and other standardized components.Source: A Patent Background · p. 11
  • Patent drafting requires long-context reasoning, specialized-language interpretation, and strict legal precision and consistency.Source: A Patent Background · p. 11

Dataset and implementation

D Evaluation Details

The evaluation combines LLM-based judging, robustness checks, and expert assessment across generated patent specifications and claims. Patent-MAF’s trends generalize to another open-source vision backbone, but the framework incurs additional token and latency costs.

Automated evaluation

  • DeepSeek-V3 with Chain-of-Thought prompting evaluates generated patent specifications and claims using dedicated assessment prompts.Source: D Evaluation Details · p. 12
  • GLM-5.2 agrees with DeepSeek-V3 on approximately 85% of specification comparisons and 82% of claim comparisons.Source: D Evaluation Details · p. 13

Human evaluation

Additional experiments

  • Llama-3.2-90B-Vision produces similar performance trends when used with Patent-MAF, supporting generalization across open-source backbones.Source: §F.1 · p. 13, Table 6 · p. 13

Statistical analysis and overhead

  • Patent-MAF’s reported automated-metric improvements are statistically significant, while its multi-agent process uses approximately 2.5x the tokens of a single-shot baseline.The added token usage increases latency because of intermediate reasoning by the Manager and Polisher agents.Source: §F.2 · p. 13, §F.3 · p. 13

F.4 Qualitative Analysis

The qualitative analysis identifies failures in information routing and terminology alignment across independently generated patent sections. The Polisher can resolve some cross-section inconsistencies but cannot restore omitted technical information.

Error patterns

  • Manager routing omissions cause downstream drafters to produce incomplete claims or specifications when key technical features are not extracted.Source: §F.4 · p. 13
  • The Polisher cannot recover information missing from the disclosure structure because it performs consistency rather than completeness checking.Source: §F.4 · p. 13
  • Independent Claim and Specification Drafters occasionally use divergent terms for the same objects.Source: §F.4 · p. 13
  • The Polisher successfully resolves some terminology inconsistencies between claims and specifications.Source: §F.4 · p. 13

Evaluation materials

Related papers

This paper cites

Loading 2608.21249v1…