Generative Gap Filling

Method

• The masking design creates knowable ground truth by redacting terms that parties negotiated, priced, drafted, and signed, allowing interpretive predictions to be graded.The design uses real executed contracts, realistic triggering scenarios, and readers who see the masked agreement without knowing the original term.Source: INTRODUCTION · pp. 5–6

Results

• 88% of the time, language models recovered masked negotiated contract terms, compared with about half of lay subjects and 59% of seasoned lawyers.Law students barely exceeded lay readers, while differently trained and prompted models repeatedly returned the same answer.Source: INTRODUCTION · p. 6

Robustness

• Across more than 100 additional contracts, AI predictions remained accurate, moved with altered deal language, and exceeded every human group even for off-market deals.Changing individual terms shifted predictions toward the contractual leverage implied by the modified language.Source: INTRODUCTION · p. 7

Mechanisms

• About two-thirds of model success reflected general deal-structure expectations, while one-third reflected inferences from specific contract language.Models were nearly perfect for commercially normal masked terms and right about 60% of the time for idiosyncratic outcomes.Source: INTRODUCTION · p. 7

Implications

• The findings challenge the assumption that courts lack textual guidance in gaps and suggest AI can support empirical interpretation while choice-of-model clauses discipline model selection.Parties can name the system whose reading receives weight before a dispute arises, reducing post-dispute model-selection concerns.Source: INTRODUCTION · pp. 8, 10

The literature’s shared premise

• Courts ordinarily claim to reconstruct hypothetical party intent, but as textual meaning diminishes, gap filling increasingly relies on context, extrinsic evidence, and policy.This has fueled competing approaches based on defaults, commercial norms, formalism, or refusal to fill gaps.Source: §I · pp. 11–12
• The literature’s central premise is that documents eventually stop reliably evidencing the parties’ actual deal, leaving judges to import materials from outside the contract.The paper identifies this shared premise beneath disagreements over what counts as a gap and how gaps should be filled.Source: §I · pp. 11–13

The empirical question

• The paper argues that “true gaps” are likely rare because false gaps occupy more territory than assumed and incomplete documents may still carry substantial evidence of the deal.Measuring that evidence could narrow the domain for defaults, penalties, and policy while distinguishing silences where intent survives from those where it is genuinely missing.Source: §I · pp. 13, 17, 18

Three kinds of silence

• Silence can reflect strategic disagreement, strategic non-drafting, or inadvertence, and these forms differ in how much evidence of the parties’ preferences remains.Strategic disagreement may leave no convergent preference to recover, whereas deliberate or inadvertent non-drafting preserves evidence about priorities, party types, and deal structure.Source: §I · pp. 15–17

FILLING

• The authors introduce masking contract terms to test whether contractual texts are richer sources of the parties’ intent than the literature has assumed.The technique is presented as necessary for experimentally evaluating the gap-filling hypothesis.Source: FILLING · p. 18

A. Ground Truth and Interpretation

• Contract interpretation lacks an agreed accuracy criterion, despite contract law’s general commitment to recovering parties’ intended meaning at formation.This complicates comparisons among textualism, contextualism, judges, juries, arbitrators, and generalists.Source: A. Ground Truth and Interpretation · p. 18
• Survey, judicial-opinion, and corpus-based methods offer alternative benchmarks but do not necessarily establish what the contracting parties themselves meant.The paper therefore treats document-supplied answers as a more direct ground-truth source than population beliefs or linguistic proxies.Source: A. Ground Truth and Interpretation · pp. 19–20
• The approach adapts machine learning’s masked-prediction logic: hide part of a statement, then predict it from the remaining context.This creates labeled examples with known answers and lets researchers evaluate prediction accuracy directly.Source: A. Ground Truth and Interpretation · p. 20
• Masked provisions in real, negotiated contracts provide a ground-truth benchmark for evaluating whether interpreters can recover the parties’ original language.The method redacts a consequential provision and scores predictions against what the parties actually wrote.Source: A. Ground Truth and Interpretation · p. 20
• The task most directly represents agreements where parties reached and priced a term but failed to commit it to writing.It does not directly address disputes involving unwanted judicial gap filling, disagreement over the term, or expressly normative inquiries.Source: A. Ground Truth and Interpretation · pp. 20–21
• The paper presents the masked-provision task as novel in legal interpretation because its answer key comes from the document itself rather than external surveys or legal materials.The nearest cited legal analogue masks case holdings to test case-law knowledge, not interpretation of agreements.Source: A. Ground Truth and Interpretation · p. 20
• Masking a written provision may provide more external information than predicting a genuinely omitted term, especially through cross-references or textual allusions.The authors note that their described cases involved masking written terms rather than terms later lost from the agreement.Source: A. Ground Truth and Interpretation · p. 21

B. Three Scenarios

• The authors implemented the method in three real-contract scenarios involving genuine interpretive disputes with real stakes, using recent PACER contracts unlikely to appear in model training data.They drafted the contractual disputes themselves rather than relying on the parties’ briefs.Source: B. Three Scenarios · p. 21

• The scenario used a talent-agency engagement contract in which the artist canceled a performance after a slip-and-fall injury, with the commission clause masked.The contract covered a $40,000 artist fee, travel expenses, performance obligations, promotion, and agency brokerage services.Source: §1 · pp. 21–22
• The correct answer was full commission because the cancellation did not result from the agency’s gross negligence.The masked clause made commission payable even when the performance did not occur, subject to the agency’s gross-negligence exception.Source: §1 · pp. 22–23
• Contract language outside the mask established that cancellation did not affect the agency’s commission, but did not specify the commission amount or gross-negligence limitation.A follow-up question tested the limitation directly, and the correct answer was that the agency lost commission only for serious misconduct or extreme carelessness.Source: §1 · pp. 23–24

Contingency Fee

• The correct answer was that attorneys remained entitled to their contingency fee even when the client settled without their participation.The masked clause directly addressed settlements made without the attorneys’ knowledge or consent.Source: §2 · p. 25
• The agreement’s fee percentages, broad definition of gross recovery, lien provision, and discharge protections supplied clues to the masked settlement term.These provisions helped readers infer that unauthorized settlement did not eliminate the attorneys’ fee entitlement.Source: §2 · p. 25
• The applicable amount was the pre-complaint percentage rate specified in Paragraph 4(a), because the clause promised the “full contingent fee.”The alternatives included a post-complaint penalty rate, a reduced percentage, and an hourly rate, but the contract language supported option A.Source: §2 · pp. 25–26

Scenario

• Respondents evaluated whether Milo’s had to purchase and pay for 110,000 excess bottles produced after its forecast and subsequent refusal.The dispute involved CKS producing, storing, and returning bottles after Milo’s said it expected additional campaign-related demand.Source: §3 · pp. 26–27

Question 1

• The correct liability answer was C: Milo’s forecast caused CKS to produce the inventory, making Milo’s responsible for its price.The clause covers inventory held because of a purchase order or forecast; the extended holding period alone was not the operative basis.Source: §3 · p. 27

Interpretation

• The masked clause distinguishes forecast-related inventory from arbitrary inventory CKS independently produced for other buyers.Answer B was sensible but atextual, while answer D improperly treated the three-month holding period as dispositive.Source: §3 · p. 27

Question 2

• The correct pricing answer is A: CKS may invoice at then-current market prices or indexed raw material prices plus storage costs.Source: §3 · pp. 27–28

Hypotheses and preregistration

• The study preregistered hypotheses about interpretation accuracy, political concordance, legal training, scenario complexity, and racial concordance, with planned robustness checks.The authors leave racial and political concordance discussions to a different paper.Source: C. Measures and Methods · p. 28

Human-participant sample

• The study collected 501 completions, removed two duplicates and 34 attention failures, and analyzed 465 participants.Lay respondents were recruited through Prolific; participants received approximately $12 per hour, with two $100 gift-card bonuses raffled among correct respondents.Source: C. Measures and Methods · pp. 28–29

Sample composition

• Lay respondents were fairly representative of the U.S. population, but were more educated and more liberal than the population at large.The comparison concerns respondents who passed the attention check.Source: C. Measures and Methods · p. 29, Table 1 · p. 29

Human-participant sample

• The human sample included 77 law students near the end of their 1L year plus 2Ls and 3Ls, and 48 lawyers and judges with 18.5 median years in practice.The lawyer sample was a convenience sample weighted toward law-firm lawyers and in-house counsel.Source: C. Measures and Methods · p. 29

Human respondents

• 55%: lay respondents correctly predicted the hidden term overall, with accuracy ranging from 32.2% in Artist to 71.1% in Contingency Fee.The pooled rate was more than twice chance, but performance varied considerably by scenario.Source: Figure 1 · p. 30, D. Results · pp. 30–31
• Nearly 60%: lawyers correctly filled contractual gaps, outperforming other human groups on Artist and Contingency Fee but not Bottles.Law students were marginally better than lay respondents overall.Source: D. Results · pp. 31–32, Figure 2 · p. 31, Figure 3 · p. 32

Headline unmasking results

• 88.3%: LLMs correctly unmasked the agreements, far exceeding human performance across the scenarios.The models were strong interpreters across each scenario, although Bottles was their most challenging packet.Source: Figure 4 · p. 33, D. Results · p. 33

Model variation

• 100%: Opus 4.6 achieved perfect scenario-balanced unmasking, while panel scores ranged from 70.0% for GLM 5.1 to 97% for leading alternatives.Gemini 3.1 Pro, Grok 4.2, and GPT 5.4 ranged between 88% and 97%, Qwen 3.6 reached 82%, and GLM 5.1 remained above the human rate.Source: Figure 5 · p. 34, D. Results · p. 34

Strict reconstruction

• 26%: human respondents answered both headline and follow-up questions correctly, versus 6.25% expected by chance; pooled LLM performance was about twice human performance.Legal training again monotonically improved performance on strict reconstruction.Source: Figure 6 · p. 35, D. Results · p. 35
• None of 84 LLM runs that correctly answered Bottles’ headline question answered its follow-up correctly, whereas humans performed substantially better.The models likely over-weighted section 1’s product prices, showing that some contractual issues remained especially difficult for LLMs.Source: Figure 7 · p. 36, D. Results · p. 36

Perturbed Contracts

• 88% accuracy fell to 60% with commercially flipped contracts and 68% without contracts, confirming that perturbing the text reduced model performance.The flipped variants reversed each agreement’s commercial direction while holding questions and answer options fixed; the no-contract prompt supplied only the title.Source: E. Robustness Checks: Perturbed… · pp. 37–38, Figure 8 · p. 38
• Removing contracts preserved or insignificantly increased accuracy for Bottles and Contingency Fee scenarios but reduced Artist Contract performance to barely above chance.Baseline expectations can decode deals matching industry norms, whereas the Artist Contract required its particularized text.Source: Figure 9 · p. 39, E. Robustness Checks: Perturbed… · p. 39

Test Hacking

• Answers were consistent across opposing party framings roughly 85% of the time, supporting the reliability of the fresh-agreement yes-or-no evaluation.The question was asked separately about whether party A or party B should prevail.Source: E. Robustness Checks: Perturbed… · p. 39
• 53% reconstruction accuracy from dispute descriptions alone increased by 7.3 percentage points to 61% when models saw the masked contract.An advanced model graded open-ended reconstructions using a rubric, reflecting the difficulty of evaluating provisions directly.Source: E. Robustness Checks: Perturbed… · p. 40
• 26% accuracy rose to 49% when hard disputes in 145 fresh SEC agreements included the contract, indicating performance was not merely multiple-choice test hacking.Models were asked who should win and to reconstruct a masked provision; the hard-dispute analysis was exploratory and defined difficulty using the other two models’ answers.Source: E. Robustness Checks: Perturbed… · pp. 39–40, Figure 10 · p. 41

F. Diversifying Commercial Contexts: 119 Recent Contracts

• 91.1% and 88.4% accuracy on 112 post-cutoff contracts matched GPT-5.4 and Claude Opus 4.6’s overall rates.Both models’ training ended in August 2025, before those contracts were filed.Source: F. Diversifying Commercial Contexts:… · p. 41
• 87% of masked clauses were recovered correctly across 119 contracts, with individual-model accuracy ranging from 84% to 92%.This result closely matched the main study’s accuracy.Source: Figure 11 · p. 42, F. Diversifying Commercial Contexts:… · p. 42
• Accuracy was highest for promissory notes (98%), securities and subscription agreements (96%), and credit facilities (95%), but lower for indemnification (61%), registration rights (67%), and employment agreements (76%).Models performed better on templated, market-standard documents than on negotiated or bespoke terms.Source: Figure 12 · p. 43, F. Diversifying Commercial Contexts:… · p. 43
• Models answered correctly 93.6% of default-conforming clauses but only 59.8% of anti-default clauses, suggesting they combine boilerplate expectations with contract-specific information.Accuracy was unrelated to contract length, clause length, or mask position.Source: F. Diversifying Commercial Contexts:… · p. 44
• Pairwise error correlations averaged .54 across model pairs, ranging from .36 to .75, while excluding the item-generating model changed accuracy from 87.4% to 86.6%.The distribution of errors across contracts remained unchanged after exclusion.Source: F. Diversifying Commercial Contexts:… · pp. 42, 44
• The fifteen trickiest contracts—12.6% of the sample—accounted for about 75% of errors, and all six models missed the same five contracts.An exploratory audit classified those five masked clauses as reversing conventional market or legal defaults.Source: F. Diversifying Commercial Contexts:… · pp. 43–44, Figure 13

G. General Discussion of Results

• Human readers performed above chance, while legal expertise reached nearly 60% on interpretive tasks and lawyers generally outperformed students and lay readers without statistically significant differences.Lawyers’ domain knowledge helped predict a counsel-fee provision, but expertise also produced errors when a bespoke deal departed from a standard pattern.Source: G. General Discussion of Results · pp. 45, 47
• Contractual provisions are interdependent, so surrounding text carries mutual information that can reconstruct missing terms rather than merely signaling general commercial conventions.Price, risk allocation, excuse, and termination provisions inform one another, allowing information elsewhere in the agreement to support gap filling.Source: G. General Discussion of Results · p. 45
• 88.3% accuracy for masked terms and 54% on term limits exceeded chance and every human group, with differently trained models converging on similar answers.Chance was 25% for term prediction and 6.25% for the stricter measure; model results were less variable than human responses.Source: G. General Discussion of Results · p. 46
• Perturbed and missing-contract scenarios weakened performance for less market-anchored deals, while testing over 100 unseen agreements preserved the main results and reduced concern about cherry-picking.Models were pulled toward convention but still extracted information from the parties’ actual contract text.Source: G. General Discussion of Results · p. 46
• Models remained vulnerable to conspicuous defaults: on Bottles, they overwhelmingly selected a wrong answer based on a number elsewhere instead of the operative local language.This mirrors lawyers’ tendency to apply sensible industry-standard terms when the parties had contracted around them.Source: G. General Discussion of Results · pp. 47–48
• Removing contract language reduced model accuracy in some settings, while familiar agreements sometimes produced unchanged performance; internally reversed agreements also made models excessively obstinate.These results show that models use textual information but can remain anchored to familiar contingencies and prior predictions.Source: G. General Discussion of Results · p. 48
• Manufactured gaps limit external validity, and litigated gaps may be unusually difficult because predictable disputes settle while unresolved cases involve unforeseen contingencies or unaddressed disagreements.The method infers the real deal from written provisions, which contract-production theory suggests are linked to unwritten terms through deal types, risk allocation, and pricing.Source: G. General Discussion of Results · pp. 48–49

III. THE PRACTICE AND PERILS OF GENERATIVE GAP FILLING

• The practice section asks how courts and practitioners should deploy models that uncover concealed contract terms more accurately than humans, including lawyers.When agreements were internally flipped or masked clauses reversed legal or market defaults, accuracy settled near 60%, still exceeding every human group.Source: §III · p. 49, G. General Discussion of Results · p. 49

Proposed role

• Generative gap filling would treat a model’s prediction as contested evidence for adjudication, not as a judgment that replaces judges.Parties would disclose their queries and outputs, while the court would weigh them in an explained opinion open to challenge and appeal.Source: A. Generative Gap Filling Within… · pp. 51, 53, 54

Doctrinal problem

• Existing gap-filling doctrine gives judges multiple discretionary inputs, yet experimental evidence suggests they may not reliably recover what contracting parties would have agreed.Different judges can reach different answers from the same record, making hypothetical-bargain analysis insufficiently disciplining.Source: A. Generative Gap Filling Within… · p. 52

Judicial oversight

• Courts could evaluate prompts, model weaknesses, corpus appropriateness, and sensitivity analyses, while appointing a neutral expert when an independent query is needed.The proposed process makes inferential work public, contestable, and reason-giving rather than dependent on a decisionmaker’s inarticulable intuitions.Source: A. Generative Gap Filling Within… · p. 54

Procedural safeguards

• The method’s legitimacy depends on disclosure of the model, version, prompt, system instructions, run settings, and deployment harness so opposing parties can replicate or contest the output.The harness matters because web access, private transaction data, and other tools can materially change a model’s result.Source: A. Generative Gap Filling Within… · p. 55

Limits and enforcement

• Severe sanctions should deter fabricated or manipulated outputs, and courts must decide whether a model’s prediction is probative for the particular gap.The authors also expect model battles to remain uncommon in ordinary cases because their data suggest the method is stable enough for run-of-the-mill disputes.Source: A. Generative Gap Filling Within… · pp. 55–56

B. Choice of Model Clauses & Equilibrium Drafting Effects

• Choice of Model clauses specify the model, harness, and prompt protocol governing disputes about contractual silence.They extend parties’ control over interpretive methods to generative AI, analogous to incorporating technical standards by reference.Source: B. Choice of Model Clauses &… · pp. 56–58
• Model panels provide contestability and replication rather than independent verification because current models’ errors are strongly correlated.A strict-majority panel returned no answer on eight of 119 contracts, so clauses should specify aggregation, tie, and abstention rules.Source: B. Choice of Model Clauses &… · p. 57
• Undated references to a model should capture subsequent versions, while parties seeking to freeze the model at signing must say so explicitly.This proposed default addresses replacement of an unavailable model version.Source: B. Choice of Model Clauses &… · pp. 58–59
• Pre-signing model testing makes inadvertent gaps less likely, leaving more remaining silences deliberate for sophisticated parties.The parties may either endorse the model’s prediction, effectively incorporating it, or strategically accept an unfavorable prediction without resolving disagreement.Source: B. Choice of Model Clauses &… · pp. 59–60
• Endorsed silence invites enforcement, while strategic silence may require good faith, unconscionability, penalty-default, or equity-based doctrines.Courts must distinguish the two forms of silence rather than treating them identically.Source: B. Choice of Model Clauses &… · pp. 60–61
• Drafting-stage evidence can show model use, bilateral access, and whether negotiations failed, helping courts diagnose the kind of silence involved.Discovery may examine the models, predictions, and negotiation record, while normative judgment remains about the output’s significance, access, and public-policy implications.Source: B. Choice of Model Clauses &… · p. 61

C. Generative Gap Filling in the Chambers

• A judge using AI for general linguistic meaning conducts a resource inquiry resembling corpus research, but should disclose the tool, query, and weight given.Transparency allows the method to be contested on appeal.Source: C. Generative Gap Filling in the… · pp. 62–63
• Case-specific judicial gap filling effectively commissions an expert report without party cross-examination, requiring advance disclosure of model type, version, prompt, and settings or a neutral expert.Using models to draft decisions is not recommended and is not supported by the evidence.Source: C. Generative Gap Filling in the… · p. 63

Consumer contracts

• Consumer contracts raise distinctive intent and fairness concerns because they are often created without bargaining or reading, weakening the case for intent-based interpretation.Background concerns about fairness, resource distribution, and thin consent further complicate the inquiry.Source: D. How Far Does This Go? · p. 64
• For consumer contracts, model-based gap filling may favor firms, but it is cheaper than surveys and more contestable because courts can ground decisions in contract text.The approach also relies on majoritarian inferences generated by the forms’ provisions.Source: D. How Far Does This Go? · pp. 64–65

One-off deals

• One-off deals among parties outside majoritarian linguistic communities risk model error because textual methods may miss important social context.Calibrating models to linguistic subcommunities would add complexity and expense, challenging routine use by trial judges.Source: D. How Far Does This Go? · p. 65
• When a contract appears singular or bespoke, courts should require a Choice of Model Clause before using AI to improve textual interpretation.Such deals should reduce reliance on ordinary-meaning textualism and technologically enhanced textualism absent the parties’ stated agreement.Source: D. How Far Does This Go? · p. 65

Reliability and bias

• Model use risks hallucination, prompt sensitivity, sycophancy, randomness, and deceptive plausibility that may encourage judges and lawyers to become overconfident.The authors regard these risks as especially serious for busy and hubristic judges, but suggest training, operational protocols, and adversarial use can reduce them.Source: E. Gaps in Generative Gap Filling and… · p. 66

Human comparison

• Human judgment also varies with hidden conditions such as energy, mood, temperature, time of day, and case sequence, so AI’s incremental error remains an empirical question.The paper argues that models should be compared with existing human inconsistency rather than judged against an idealized decisionmaker.Source: E. Gaps in Generative Gap Filling and… · p. 67

Comparative reliability

• 19.7–22.9 percentage points for models versus 21–23 points for typical humans shows comparable dispersion, while model outputs also converge across systems and settings.GPT-4.1 deviated by 19.7 points and Claude Opus 4.1 by 22.9, compared with 21–23 points for individual human respondents.Source: E. Gaps in Generative Gap Filling and… · pp. 67–68

Epistemic limits

• Although legal interpretation lacks pre-agreed answers and model reasoning may not reflect operative causes, the study supplies ground truth by testing recovery of terms parties would write.This design evaluates whether legal actors and interpretive approaches recover actual parties’ intent from surrounding contractual context.Source: E. Gaps in Generative Gap Filling and… · pp. 68–69

Lawyer evaluation

• The same method can test whether lawyers genuinely interpret contracts or instead project priors, including whether they can put ideological biases aside.The authors frame this as a way to evaluate lawyers’ competence using performance rather than unverified explanations of reasoning.Source: E. Gaps in Generative Gap Filling and… · p. 69

• AI can serve lawyers as an agent by supporting narrower, more contestable opinions, contractual disclosure, adversarial presentation, and Choice of Model clauses.The authors argue that this approach can reduce interpretive disputes and leave courts focused on the normative question of what parties would have wanted.Source: §IV · pp. 70–71
• The method depends on human bargaining: visible contract terms carry information about hidden terms because they record social tradeoffs made off the page.That human origin makes masked-term prediction a proxy for recovering parties’ intent and enables comparison against the actual term the parties wrote.Source: §IV · p. 71
• AI-generated contracts may preserve contractual form while lacking human intent, leaving existing doctrines unable to assign clause-level meaning to principals who never read, drafted, or contemplated the clause.The authors suggest that intent might instead migrate to principals’ instructions and model prompts, but they expressly leave the necessary doctrine unresolved.Source: §IV · p. 72
• The recommendations remain relevant because human-drafted agreements will dominate litigation for some time, whereas machine-written contracts may transmit a clear signal without anyone having sent it.The radio-signal metaphor therefore supports generative gap filling for human agreements but leaves courts’ reasons for interpreting machine-written agreements uncertain.Source: §IV · p. 73