Hallucination is the last thing you need

• Existing responses include fact-enhanced models, retrieval mechanisms, and vendor search or comparison systems for checking generated output.Source: §1 · p. 1
• Hallucinated case law has already produced legal consequences, including a US sanctions hearing and a court order requiring disclosure of generative-AI use.Source: §1 · p. 1
• Subtle, non-obvious errors generated during legal research could contaminate common law and undermine confidence in the legal profession.Source: §1 · p. 2
• The paper proposes a theoretical approach to reducing legal hallucinations through model design, pending further exploration as data labelling reaches critical mass.Source: §1 · p. 2
• The authors state that they have formal legal training but are not qualified lawyers and provide computational analysis rather than legal advice or judgment.Source: §1 · p. 2

2 Common law contamination

• Monolithic LLMs may produce unclear overlapping outputs across case law, legislation, and case summaries, motivating multi-length tokenisation and vertically aligned ensemble models.Source: §2 · p. 2

2.1 Multi-length tokenisation

• Judgments and especially legally significant quotations should be treated as structured factual data rather than left to probabilistic next-token generation.Source: §2.1 · p. 3
• Word-level tokenisation can distort quotations, while whole-quote tokenisation may become off-topic; complex legal text therefore requires integral units at multiple lengths.Source: §2.1 · p. 3

2.2 Ensemble models

• The proposed ensemble assigns separate models to problem understanding, commentary based on experience, and factual verbatim quotations supporting legal arguments.Source: §2.2 · p. 4, Commentary Model · p. 5, Fact Model · p. 5
• The problem model uses early judgment summaries and claims or defences as inputs, with commentary functioning like an answer to the represented legal problem.Source: Problem Model · p. 5
• The design aims to protect quotations from ambiguous hallucinations, but handovers may still fabricate case law or propagate upstream hallucinations as misleading text chunks.Source: Linking them together · pp. 6–7
• A classification or key/value handover could link semantic commentary and case citations to a fact model that outputs an entire verbatim quote.Source: Linking them together · pp. 6–7
• Training on whole judgments may dilute frequently cited quotations, omit uncited material, and struggle with new judgments until courts improve data labelling.Source: Judgement as Fact Issue · p. 7

3 Hallucination Evaluation

• The evaluation tests whether GPT can continue legal judgment sequences verbatim, distinguish semantic matches from unrelated intent, and identify subtle differences and their likely source.The intended categories include verbatim, close-to-verbatim, semantically similar, unrelated, and error-attribution outcomes.Source: §3 · pp. 8, 9, 15
• Older GPT models often followed contextual sequence patterns rather than reproducing judgment wording exactly, while temperature may have encouraged more imaginative outputs.Outputs preserved semantic elements such as notice and onerous-term concepts but diverged from the precise quotation.Source: §3 · p. 8
• The model produced a close-to-verbatim quotation, correcting the judgment’s “read” typo to “red” but adding “the more unreasonable the clause,” which was not materially misleading here.This suggests probabilistic next-token selection can correct human spelling errors while also inserting extra text.Source: §3 · p. 10
• The model generated semantically similar legal text about contractual conditions covering breaches and excluding negligence, despite not matching the source wording.The output preserved the judgment’s central meaning that the condition must cover the breach and not arise from negligence.Source: §3 · p. 11
• In legislative text, the model preserved wording omitted by the judgment but also produced discrepancies, with legalese ellipsis and likely next-token preferences contributing to hallucinated wording.The analysis specifically attributes “application” replacing the expected phrase to common legalese ellipsis and token probability.Source: §3 · pp. 13–14
• The model produced semantically similar output for libel liability and contract-term notice, but the latter appeared to conflate related summaries rather than faithfully continue the quotation.For libel, it retained the point that anyone involved in publication may be liable; for notice, only part of the semantic mapping was convincing.Source: §3 · pp. 14, 17
• The model sometimes generated an unrelated continuation and denied that the cited case existed, showing that absent case knowledge can undermine legal uses requiring detection of misrepresented authorities.This limitation may matter less when citing obscure cases affirmatively than when checking an opposing argument against databases such as Lexis or Westlaw.Source: §3 · p. 15
• The model produced non-verbatim matches with similar semantic intent, but sometimes ignored the instruction to continue a sequence and instead treated the input as a question.In one example, searching the generated text retrieved Barclays v O'Brien, yet the model had missed the prompt’s intended continuation.Source: §3 · p. 18
• The model generated a verbose contract-law passage with similar meaning despite no online source matching the quotation, suggesting synthesis across summaries and verbatim judgment text.The paper attributes this possibility partly to model temperature and the prescriptive nature of the judgment quotation.Source: §3 · p. 19
• The evaluation also identified unrelated-intent hallucinations, including conflation of passages about non-pecuniary contract damages and definitions of unlawful means.The damages example combined sections concerning the central object of a contract, while the unlawful-means example shared only limited words such as “induce” and “act.”Source: §3 · pp. 20, 22
• A Salomon example shows that the model can produce a semantically sensible but instructionally wrong quotation by retrieving a verbatim passage connected to the surrounding legal meaning.The generated passage was linked to the judgment’s discussion of a company being wholly owned and controlled by one person.Source: §3 · p. 21
• The blank-insertion test exposed subtle hallucinations in which “reasonable expectation of confidentiality” replaced “relevant legal context” while arguably making better contextual sense.This test used textdavinci-003 because GPT-4 was unavailable in OpenAI Insert within Playground.Source: §3 · p. 24
• The evaluation identified valid but misattributed wording, including a “cause of action” definition from Letang v Cooper with “simply” omitted.The omission reflected wording also used more commonly online, but the generated quote did not correctly match the cited judgment.Source: §3 · p. 28

3.1 Discussion

• More than 50% of twenty trials produced non-verbatim matches with similar semantic intent, usually concise summaries rather than verbatim quotations.The results show that semantic preservation does not ensure faithful reproduction of legal language.Source: §3.1 · p. 32
• The model produced one flawless Donoghue v Stevenson match, likely aided by the quote’s extensive circulation in case summaries, judicial opinions, and academic publications.The authors infer that the model places greater emphasis on well-known quotations.Source: §3.1 · p. 32
• Gillick v West Norfolk showed that contaminated square-bracket commentary can produce a partly accurate legislative preamble followed by factual inaccuracies against the statute.The example underscores the importance of interpreting editorial brackets and distinguishing court commentary from quoted legal text.Source: §3.1 · pp. 32–33
• Non-verbatim outputs ranged from comparable meaning to unrelated intent, with subtle substitutions such as “relevant legal context” replacing “reasonable expectation of confidentiality” requiring careful evaluation.The Dundee v Prasan example illustrates how seemingly minor changes can alter intended legal meaning without immediately appearing erroneous.Source: §3.1 · pp. 32–33
• The authors conclude that consumer and enterprise ChatGPT lack precise safeguards against legal hallucination, although LLMs perform effectively in verification, contract review, and dispute resolution.They support continued development while warning that uncontrolled common-law contamination could make legal-research use unlawful and advocating access to justice data.Source: §3.1 · pp. 33–34

• YCNBot detects case law in generated output and blocks the “Copy and Paste” functionality to mitigate risk.Source: §5 · p. 34
• The check relies on regular expressions and common case-name formats rather than direct access to a case-law database.Source: §5 · p. 34
• The authors plan to add a similar legislation-detection algorithm with the same blocking effect.Source: §5 · p. 34
• The latest YCNBot version containing this feature is available for cloning.Source: §5 · p. 34