LLMediator: GPT-4 Assisted Online Dispute Resolution

1. Introduction

• Laypeople may struggle to understand applicable legal rules or resolve disputes, while court proceedings can impose substantial monetary, temporal, and psychological costs.The introduction notes that unresolved legal issues were estimated to affect 1.5 billion people globally at any point in time.Source: §1 · p. 1
• Online negotiation and mediation offer cooperative, non-adversarial alternatives to court; PARLe-OPC reports 70% of cases settled, including 45% at negotiation and 25% at mediation.The platform is described as a Quebec consumer-protection implementation.Source: §1 · p. 1
• LLMediator is an experimental platform exploring how LLMs such as GPT-4 can support parties and mediators in reaching settlements through ODR.The paper positions the platform as a way to further improve ODR usability and efficacy.Source: §1 · p. 1
• It can reformulate user messages, draft mediator messages using prior party messages and mediator instructions, and experimentally respond autonomously to discussions.The paper demonstrates selected features and conducts initial qualitative evaluations.Source: §1 · pp. 1–2

2. Related Work

• The paper places its approach among AI legal tools and uses augmented intelligence to support negotiation and mediation rather than provide direct instructions or advice.It notes prior GPT applications in legal exams, reasoning, information, and annotation.Source: §2 · p. 2
• Prior negotiation tools include BATNA information, game-theory methods, and dialogue structuring; this work detects inflammatory language and suggests alternatives.The suggested wording may be more amenable to amicable resolution.Source: §2 · p. 2
• Earlier mediation systems monitored negotiations for situations requiring human intervention and suggested standard messages, while other work explored ChatGPT-generated mediator questions and party priorities.LLMediator evaluates LLM-generated suggestions for mediators and possible autonomous negotiation intervention.Source: §2 · p. 2
• Huang and Ouyang used pretrained LLMs with low-code fine-tuning to generate judge perspectives for ODR scenarios.Source: §2 · p. 2

3. Example use cases

• The web-based ODR prototype lets parties or a mediator use an asynchronous chat, and adds message reformulation, mediator drafts, and possible autonomous intervention.Figure 1 depicts a dispute about a broken camera.Source: §3 · p. 2, Figure 1 · p. 3

3.1. F1 - Reformulating inflammatory messages

• For inflammatory messages, LLMediator detects language that may derail settlement and offers a GPT-4 reformulation the user can send, edit, or decline.Figure 2 illustrates the detected message and proposed alternative.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2. F2 - Drafting messages for the mediator

• A mediator can request a GPT-4 draft based on earlier party messages, then send or edit it and add instructions to emphasize particular issues.The example instructions guide discussion toward considering the camera's packaging.Source: §3.2 · p. 3

3.3. F3 - Autonomously intervening in the negotiation?

• Autonomous intervention could be triggered by a party request, inactivity, or inflammatory messages, including in low-value disputes or where mediators are scarce.The authors present this as experimental and say its substantial risks require careful study before deployment.Source: §3.3 · pp. 3–4

• The technical considerations include when features activate, how the LLM is prompted, and what happens to its resulting text.Source: §4 · p. 4
• Figure 4 shows mediator-provided instructions being incorporated when LLMediator generates a message.Source: Figure 4 · p. 4

• LLMediator uses GPT-4, described as a multimodal pre-trained large language model created by OpenAI [13].Source: §4.1 · p. 4
• The system interacted with the model through the OpenAI API using the 8k-token version.Source: §4.1 · p. 4
• The passage notes GPT-4’s performance on varied tasks, including passing the Uniform Bar Examination.Source: §4.1 · p. 4

4.2. F1 - Reformulating inflammatory messages

• LLMediator aims to reduce tension by reformulating inflammatory messages in a more neutral tone while preserving their semantic content.Source: §4.2 · pp. 4–5

4.2.1. Detect a message requiring intervention

• The system detects messages using keyword search and also lets users manually request a reformulation; more sophisticated detection is deferred to future work.Source: §4.2.1 · p. 5

4.2.2. Reformulating the message

• After detection, GPT-4 is prompted to retain the message’s content while making it less confrontational and more encouraging of amicable settlement.Source: §4.2.2 · p. 5

4.2.3. Showing the reformulation to the user

• Users can accept, edit, or reject the reformulation and send their original message, preserving control over their communication.Source: §4.2.3 · pp. 5–6

4.2.4. Additional examples

• Table 1 presents examples of original and reformulated messages.Source: Table 1 · p. 6, §4.2.4 · p. 6

4.3. F2 - Drafting messages for the mediator

• The platform generates ad hoc intervention suggestions for mediators, rather than relying only on predefined messages.Source: §4.3 · p. 6

4.3.1. Generating the message suggestion

• The draft uses the 10 latest conversation messages and any additional mediator instructions as context for GPT-4.Source: §4.3.1 · p. 6

4.3.2. Further examples

• Table 2 gives intervention examples for a dispute scenario involving a follow-up question about postal-service fault.Source: §4.3.2 · p. 6

4.4. F3 - Autonomously intervening in the negotiation?

• Direct intervention lets LLMediator act as mediator in the negotiation, but remains experimental and requires substantial study before deployment.The authors identify substantial risks in this use case.Source: §4.4 · p. 6

4.4.1. Triggers

• The system could intervene after inactivity, heated discussion, every few messages, or a party’s request; only party-requested intervention is implemented.The implemented trigger reflects the paper’s focus on evaluating whether LLMs can interact usefully with users.Source: §4.4.1 · pp. 6–7
• Table 2 gives a broken-camera dispute example of suggested interventions, with optional instructions from the mediator.Source: Table 2 · p. 7

4.4.2. Generating the message

• After a trigger, prior messages go to the LLM, whose generated response is sent directly to the parties rather than to a human mediator.The message-generation procedure follows the one described for mediator drafts in section 4.3.1.Source: §4.4.2 · p. 7

4.4.3. Additional examples

• Table 3 presents generated messages for several example disputes to illustrate the types of responses LLMediator can produce.Source: §4.4.3 · p. 7

5. Discussion

• LLMediator offers three levels of support: reformulating messages, drafting mediator interventions, and potentially intervening directly in negotiations.The discussion frames these features as selected capabilities of an experimental platform.Source: §5 · p. 7

5.1. The large language models

• The authors report that GPT-4 handled varied contexts and instructions, preserving key message content while changing tone and generating relevant mediator responses without task-specific training.The model also produced mediation techniques such as clarifying facts, encouraging respectful exchanges, and suggesting solutions; the authors say more evaluation is needed to understand performance and failure modes.Source: §5.1 · pp. 7–9

5.2. F1 - The reformulation of inflammatory messages

• F1 offers a neutral reformulation for user review rather than sending it automatically, leaving users able to edit the suggestion while retaining a more neutral tone.The authors describe this user-controlled approach as less risky, while noting that real-world ODR deployment still requires empirical evaluation of risks.Source: §5.2 · p. 9

5.3. F2 - Drafting messages for the mediator

• F2 drafts context-sensitive interventions that mediators can edit or supplement with instructions, but risks include anchoring, overreliance, and an unverified assumption that review is faster than drafting.The examined drafts adapted to conversation context by calming discussion, clarifying facts, or suggesting compromises.Source: §5.3 · p. 10

5.4. F3 - Autonomously intervening in the negotiation?

• F3 sends mediator-like messages directly to parties without human review, creating risks from biased or inaccurate content; deployment would require considerable study.If made sufficiently safe, the approach could provide personalized, always-available mediation, including where disputes are too small for mediator involvement or mediators are scarce.Source: §5.4 · p. 10

6. Future Work

• Evaluate whether LLMediator's reformulations and interventions are relevant and useful, including through mediator assessments and negotiation role-play, while examining potential bias.The proposed participants include experienced mediators and students taking the parties' roles.Source: §6 · pp. 10–11
• Prompt engineering could improve results, while the reformulation and AI-mediator intervention triggers remain incompletely developed and need further exploration.Source: §6 · p. 11
• Conversation summaries for mediators or adjudicators are another proposed way to integrate LLMs into ODR.Source: §6 · p. 11

7. Conclusion

• LLMediator is an experimental platform built to support negotiation and mediation using LLMs such as GPT-4.Source: §7 · p. 11
• The system can reformulate inflammatory messages and suggest interventions for the mediator or send them directly to the parties.Source: §7 · p. 11
• Examples using fictitious disputes and discussion of efficacy lead the authors to call the approach promising and a starting point for further research.Source: §7 · p. 11