Source-linked AI summary
Measuring Moral Inconsistencies in Large Language Models
Vamshi Krishna Bonagiri, Sreeram Vennam, Manas Gaur, Ponnurangam Kumaraguru
TL;DR
Moral scenarios lack a correct answer, limiting ground-truth-based assessment of LLM consistency. The paper proposes Semantic Graph Entropy, using semantic representations and Rules of Thumb to measure consistency, and reports stronger alignment with human judgments alongside notable inconsistency in state-of-the-art LLMs.
Problem
Moral scenarios can lack a correct answer, making consistency difficult to assess with approaches that rely on ground-truth tasks.
Source: §1 · p. 1Method
Results
Preliminary findings suggest SGE correlates better with human judgments and that most state-of-the-art LLMs are notably inconsistent.
Source: §3 · p. 3Takeaways & Limitations
The findings highlight uncertainty within LLMs, and the authors plan to investigate its origins and develop long-term solutions.
Source: §3 · p. 3
Abstract
from arXiv · showhide
A Large Language Model (LLM) is considered consistent if semantically equivalent prompts produce semantically equivalent responses. Despite recent advancements showcasing the impressive capabilities of LLMs in conversational systems, we show that even state-of-the-art LLMs are highly inconsistent in their generations, questioning their reliability. Prior research has tried to measure this with task-specific accuracy. However, this approach is unsuitable for moral scenarios, such as the trolley problem, with no "correct" answer. To address this issue, we propose a novel information-theoretic measure called Semantic Graph Entropy (SGE) to measure the consistency of an LLM in moral scenarios. We leverage "Rules of Thumb" (RoTs) to explain a model's decision-making strategies and further enhance our metric. Compared to existing consistency metrics, SGE correlates better with human judgments across five LLMs. In the future, we aim to investigate the root causes of LLM inconsistencies and propose improvements.
1 Introduction
Consistency matters for reliable LLM decisions, especially on moral queries that can affect users. The paper introduces SGE to assess consistency where moral scenarios lack a correct answer, and reports that mainstream LLMs are inconsistent.
1 Introduction
- Moral queries can affect users, but scenarios without a correct answer are difficult to assess using ground-truth-based measures.The introduction notes that related work measures uncertainty at the word or token level and uses tasks with ground truth.Source: §1 · p. 1
- SGE is an information-theoretic consistency metric, evaluated through paraphrases, LLM responses and RoTs, semantic graphs, and human evaluation.The pipeline generates quality paraphrases, obtains responses and Rules of Thumb, then creates a semantic graph and calculates SGE.Source: §1 · p. 1, Figure 1 · p. 1
- Table 1 reports stronger alignment between SGE and human assessments and higher SGE scores for paraphrased questions in MIC.Its caption specifies 10K MIC samples for † results and a 10% MIC subset for ‡ results because of API limitations.Source: Table 1 · p. 2
- Mainstream LLMs are inconsistent in moral responses, and the proposed approach is unsupervised, works for BlackBox LLMs, and quantifies consistency.The paper also describes this as highlighting moral uncertainty.Source: §1 · p. 2
2 Methodology and Results
The method samples paraphrases of MIC questions, represents model responses as semantic graphs, and uses graph entropy to measure consistency. Preliminary results favor SGE's human alignment and indicate that response consistency varies across models.
2 Methodology and Results
- The study samples 10,000 MIC questions and generates ten paraphrases per question, filtering for Parascore above 0.8 before obtaining model responses.Vicuna-13b generated the paraphrases; an SBERT DeBERTa model produced semantic representations of the answers.Source: §2 · p. 2
- SGE represents answer meanings as graph vertices, assigns vertex probabilities using an information functional, and scales entropy by average semantic distance before taking its complement.The passage defines graph distance using cosine distance and describes the complement as enabling comparison with existing metrics.Source: §2 · p. 2
- Replacing answers with Rules of Thumb shows better correlation with human evaluations and increases the reported reliability of the metric.The human NLI annotation task used 1,000 randomly selected questions, with Fleiss Kappa of 0.68 among three annotators.Source: §2 · p. 2
- Generated paraphrases are relatively consistent, whereas model answers are less consistent, indicating inconsistency even among leading LLMs.This comparison uses paraphrased questions as a baseline.Source: §2 · p. 2
- GPT-4 shows the highest consistency among the evaluated LLMs, while Falcon-7B is more consistent than Llama2-13B despite fewer parameters and a lower benchmark rank.The authors suggest this may hint that different training strategies lead to better consistency.Source: §2 · pp. 2–3
3 Conclusion and Future Work
The paper presents SGE as a consistency metric and reports preliminary evidence that it aligns better with human judgments while exposing inconsistency in state-of-the-art LLMs. Future work will strengthen the metric, investigate the origins of uncertainty, and seek long-term solutions.
3 Conclusion and Future Work
- The paper introduces SGE to measure LLM consistency.The conclusion identifies reinforcing SGE through graph entropy properties as a future direction.Source: §3 · p. 3
- Preliminary findings suggest SGE correlates better with human judgments and that most state-of-the-art LLMs are notably inconsistent.The authors describe these findings as highlighting uncertainty within LLMs.Source: §3 · p. 3
- Future work aims to pinpoint the origins of uncertain behavior and devise long-term solutions.The authors also aim to reinforce SGE via graph entropy properties.Source: §3 · p. 3