PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Patent Drafting from Inventor-Style Disclosures

Lekang Jiang, Wenjun Sun, Stephan Goetz

arXiv:2608.21249v1cs.CL

TL;DR

The paper addresses the gap between real-world patent drafting from informal inventor disclosures and benchmarks built from later-stage legalistic inputs. It introduces Dis2Pat and Patent-MAF, a locally deployable multi-agent framework, and reports that the framework is a strong baseline against open-source systems and competitive with closed-source models. The dataset remains limited by reliance on pseudo-disclosures and primarily English-language evaluation.

  • Problem

    Complete, legally coherent patent-application generation from early-stage inventor-style disclosures remains insufficiently studied because prior work typically uses later-stage, structured, or legalistic inputs.

  • Method

    The paper introduces Dis2Pat, built from controlled LLM-generated pseudo-disclosures, and Patent-MAF, which coordinates manager, drafter, and polisher agents for local patent drafting.

  • Results

    Patent-MAF is a strong locally deployable baseline that outperforms evaluated open-source models and remains competitive with state-of-the-art closed-source models.

  • Takeaways & Limitations

    Dis2Pat establishes a challenging benchmark for disclosure-to-patent drafting, while Patent-MAF provides a privacy-preserving baseline for this setting.

  • Takeaways & Limitations

    Dis2Pat relies on pseudo-disclosures that may not fully capture the diversity, incompleteness, or ambiguity of real-world inventor disclosures, and Patent-MAF is evaluated primarily on English-language patents.

Abstract

from arXiv · show

While recent large language models (LLMs) have achieved promising results on individual patent drafting tasks, they fundamentally fail to investigate the core challenge of real-world patent drafting: generating a complete and legally coherent patent application directly from early-stage invention materials. Prior work predominantly assumes later-stage, highly structured, or already legalistic inputs. However, real patenting workflows begin with informal, de-legalized disclosures authored by inventors. To bridge the gap, we introduce Dis2Pat, a disclosure-to-patent dataset that reflects realistic patenting workflows by requiring the generation of complete patent applications directly from inventor-style, de-legalized disclosures. Given the inherent difficulty of long-form, legally constrained patent drafting and the strong privacy requirements, we further propose a strong baseline named Patent-MAF. It is a multi-agent framework for locally deployable patent drafting. Benchmark results reveal that current LLMs exhibit limitations in patent drafting, while Patent-MAF provides a strong baseline that consistently outperforms evaluated open-source models and remains competitive with large closed-source models.

1 Introduction

Real-world patent drafting requires turning informal inventor disclosures into complete, legally structured applications, a challenge largely bypassed by prior benchmarks. Dis2Pat addresses this gap with a disclosure-to-patent dataset, while Patent-MAF provides a privacy-aware multi-agent baseline that performs strongly against open- and closed-source models.

  • Complete patent-application generation from early-stage invention materials remains an open problem despite progress on individual claim and specification tasks.
  • Most existing benchmarks use later-stage, highly structured inputs and therefore bypass translation from inventor-style disclosures into legally structured applications.
  • Dis2Pat requires models to generate complete applications, including claims and a full specification, from de-legalized inventor-style disclosures accompanied by figures.
  • Patent-MAF uses a manager, specialized drafter agents, and a polisher to support locally deployable patent drafting under privacy and confidentiality constraints.
  • Dis2Pat introduces a benchmark incorporating inventor-style disclosures, associated figures, and patent texts for disclosure-to-patent generation.
  • Patent-MAF consistently outperforms evaluated open-source models and achieves performance competitive with state-of-the-art closed-source models.

2 Related Works

Prior patent-generation research covers claim drafting, specification generation, and end-to-end application drafting, but commonly relies on formal or technically structured inputs. Dis2Pat instead targets inventor-style, nonlegal disclosures with drawings.

  • Existing patent-generation work spans claim-centric generation, specification generation, and end-to-end patent-application drafting.
  • Claim Drafting: Claim-drafting studies progressed from short inputs such as abstracts to larger datasets using full patent specifications.
  • End-to-End Patent Drafting: Draft2Patent transforms patent drafts into complete applications, while Pap2Pat converts scientific articles into patent applications.
  • Disclosure-to-Patent: Dis2Pat focuses on natural-language, nonlegal inventor-style disclosures accompanied by drawings, aligning more closely with real-world drafting workflows.

3 The Dis2Pat Dataset

Dis2Pat constructs a disclosure-to-patent dataset from granted patents because authentic invention disclosures are confidential and legally constrained. The resulting pseudo-disclosures remove legalistic structure while preserving technical content, and the dataset is filtered and evaluated for quality.

  • 3.1 Construction: Because real-world invention disclosures are not publicly accessible, Dis2Pat creates pseudo-disclosures from publicly available granted patents using LLM-assisted rewriting.
  • 3.1 Construction: The construction process extracts technical content, removes patent-specific legal style, and blurs feature-focused patent language.
  • 3.1 Construction: Each pseudo-disclosure uses seven invention-centric components covering the title, motivation, core idea, operation, differences, benefits, and optional variants.
  • 3.2 Statistics and Quality Analysis: Pseudo-disclosures average 1,196 tokens, substantially shorter than original patents, while the technical term density decreases from 0.05 to 0.001.
  • 3.2 Statistics and Quality Analysis: Patent professionals evaluated 100 sampled disclosures for hallucination, missing details, contradiction, and de-legalization quality.
  • 3.2 Statistics and Quality Analysis: An approximately 0.1 Cohen’s d between evaluation subsets indicates negligible score variation and supports evaluation reliability.

4 Patent-MAF

Patent-MAF generates claims and specifications from invention disclosures through coordinated manager, drafter, and polisher agents. Its design emphasizes local privacy-preserving deployment, domain-grounded role decomposition, and multimodal figure use.

  • Patent-MAF first has a manager structure the disclosure and coordinate specialized drafting agents, then uses a polisher to refine the patent text.
  • The claim drafter generates a complete claim set, while the specification drafter produces the background, summary, and detailed embodiments.
  • Patent-MAF runs all agents on open-source models without proprietary API calls, supporting privacy-aware local deployment for sensitive intellectual property.
  • Its manager, drafter, and polisher roles mirror professional patent-prosecution division of labor rather than generic task decomposition.
  • The vision-enabled specification drafter uses figures to resolve spatial and structural ambiguities.

5 Experiments

The experiments evaluate patent claims and specifications using automatic, model-based, and expert metrics across text-only and multimodal systems, including Patent-MAF variants and strong baselines.

  • Experimental Comparisons: Experiments compare Patent-MAF and ablations with diverse open-source and closed-source LLM baselines under the same decoding configuration.Ablations remove the manager, visual input or claim fine-tuning, and the polisher.
  • Patent-MAF Configuration: Patent-MAF combines a disclosure-structuring manager, specialized drafting agents, and a global consistency and style polisher.The specification drafter processes text and figures, while the claim drafter uses fine-tuning for patent-claim conventions.
  • Evaluation Setup: Models are evaluated with lexical-overlap, semantic-similarity, LLM-judge, and expert human metrics for both specifications and claims.The evaluation includes BLEU, ROUGE-1, ROUGE-L, BERTScore, and BERT-for-Patent measures alongside quality assessments.
  • Evaluation Setup: Specification evaluation measures completeness, logical clarity, and legal quality.Completeness covers technical background, problem formulation, solutions, implementation details, and embodiments.
  • Evaluation Setup: Claim evaluation measures technical scope, claim legality, and logical clarity.These criteria assess whether claims capture the inventive concept and comply with professional drafting standards.

6 Results and Analysis

Results show that visual inputs help specification drafting but not claim generation, while Patent-MAF achieves strong specification performance and benefits substantially from coordinated components.

  • Specification Generation: Visual inputs slightly improve specification quality, including Qwen3-VL-72B’s increase from 81.1 to 81.8 and GPT-5’s BERT-for-Patent increase from 95.7 to 96.4.The analysis attributes the benefit to visual grounding of spatial and structural descriptions.
  • Specification Generation: 85.4 overall content quality makes Patent-MAF the strongest evaluated open-source system and higher than Qwen3-VL-72B at 81.8 and GPT-5 at 85.1.Patent-MAF also records BERTScore 82.3 and BERT-for-Patent 96.1, compared with GPT-5’s 81.8 and 96.4.
  • Claim Generation: Visual inputs do not improve claim generation: Qwen3-VL-72B falls from 80.7 to 79.8 and GPT-5 from 91.1 to 90.7.The same downward trend extends to text-overlap and semantic-similarity metrics.
  • Ablation Study: Removing the polisher causes the largest claim-generation degradation, reducing the overall score by 6.6 points from 86.7 to 80.1.Removing fine-tuning lowers the score to 81.6, while removing the manager lowers it to 84.2.
  • Overall Patterns: Specification drafting benefits more from multimodal inputs and global consistency refinement, whereas claim generation is more sensitive to textual modeling and task-specific fine-tuning.The two tasks therefore exhibit distinct component sensitivities.
  • Human Evaluation: Patent-MAF is preferred over all open-source baselines across completeness, clarity, legality, and consistency in majority-of-cases pairwise evaluations.Against GPT-5, GPT-5 is preferred more often on completeness, clarity, and legality, while consistency is evenly matched and ties remain substantial.
  • Overall Patterns: Explicit task decomposition and refinement appear particularly beneficial for comprehensive coverage and coherence between claims and specifications.This conclusion is reported even in comparison with a state-of-the-art closed-source model.

7 Conclusion

The paper contributes Dis2Pat, a realistic disclosure-to-patent benchmark, and Patent-MAF, a locally deployable multi-agent baseline for privacy-sensitive drafting.

  • Contributions: Dis2Pat models disclosure-to-patent drafting from inventor-style disclosures with associated figures and patent texts.Its inputs are designed to represent realistic, de-legalized drafting materials.
  • Contributions: Patent-MAF coordinates manager, specialized drafter, and polisher agents to generate coherent patent applications with claims and specifications.The framework is optimized for locally deployable open-source models and avoids reliance on external APIs.
  • Conclusion: Overall results establish Dis2Pat as a challenging benchmark and Patent-MAF as a strong locally deployable baseline.The conclusion positions the framework within disclosure-to-patent drafting rather than isolated drafting tasks.

Limitations

The study’s main limitations concern the realism and scope of its data and evaluation setting.

  • Dataset Scope: Dis2Pat uses pseudo-disclosures extracted and rewritten from granted patents, which may not capture the diversity, incompleteness, or ambiguity of real inventor disclosures.The authors identify partnerships for anonymized or partially synthetic disclosures as a future direction.
  • Evaluation Scope: Patent-MAF is evaluated primarily on English-language patents, leaving multilingual and jurisdiction-specific drafting as future work.The paper specifically mentions EPO and CNIPA standards.
  • Experimental Scope: Inference hyperparameters remain fixed rather than tuned, preserving fair comparisons but limiting inference-time optimization.This choice is stated as an experimental constraint.

Ethics Statement

The paper documents artifact licensing, data-content safeguards, and the absence of ethics-board review. It also situates the work within patent-document structure, NLP challenges, dataset construction, and implementation details.

  • Ethics and licensing: The dataset is released under CC-BY-SA-4.0, while the listed models and artifacts use their respective stated licenses.Llama-3 uses the Meta Llama 3 Community License, Qwen-3 uses Apache 2.0, and GPT models are accessed through OpenAI’s API under a commercial license.
  • Ethics and licensing: The data contains no potential personal information or offensive content, and no ethics review board was involved.
  • Patent context: Patent documents combine legally authoritative claims with supporting specifications, figures, and other standardized components.
  • Patent context: Patent drafting requires long-context reasoning, specialized-language interpretation, and strict legal precision and consistency.
  • Dataset and implementation: Dis2Pat uses controlled LLM extraction from granted patents to create pseudo-disclosures while removing legal language and explicit feature structure.
  • Dataset and implementation: Experiments use LoRA fine-tuning and standardized inference settings, including temperature 0.3 and a 16,384-token generation limit.

D Evaluation Details

The evaluation combines LLM-based judging, robustness checks, and expert assessment across generated patent specifications and claims. Patent-MAF’s trends generalize to another open-source vision backbone, but the framework incurs additional token and latency costs.

  • Automated evaluation: DeepSeek-V3 with Chain-of-Thought prompting evaluates generated patent specifications and claims using dedicated assessment prompts.
  • Automated evaluation: GLM-5.2 agrees with DeepSeek-V3 on approximately 85% of specification comparisons and 82% of claim comparisons.
  • Human evaluation: Human evaluation uses reference and candidate claims assessed by one licensed patent attorney and one practitioner-in-training.
  • Human evaluation: Inter-annotator agreement reaches κ = 0.79 for Completeness and κ = 0.73 for Consistency.
  • Additional experiments: Llama-3.2-90B-Vision produces similar performance trends when used with Patent-MAF, supporting generalization across open-source backbones.
  • Statistical analysis and overhead: Patent-MAF’s reported automated-metric improvements are statistically significant, while its multi-agent process uses approximately 2.5x the tokens of a single-shot baseline.The added token usage increases latency because of intermediate reasoning by the Manager and Polisher agents.

F.4 Qualitative Analysis

The qualitative analysis identifies failures in information routing and terminology alignment across independently generated patent sections. The Polisher can resolve some cross-section inconsistencies but cannot restore omitted technical information.

  • Error patterns: Manager routing omissions cause downstream drafters to produce incomplete claims or specifications when key technical features are not extracted.
  • Error patterns: The Polisher cannot recover information missing from the disclosure structure because it performs consistency rather than completeness checking.
  • Error patterns: Independent Claim and Specification Drafters occasionally use divergent terms for the same objects.
  • Error patterns: The Polisher successfully resolves some terminology inconsistencies between claims and specifications.
  • Evaluation materials: The paper provides separate prompts for specification and claim quality assessment and instructions for expert overall-quality ranking.
Loading 2608.21249v1…