NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly

• Existing models trained on grounded rules-of-thumb often struggle to uncover new norms and model richer context, while prior norms are primarily Reddit-based and US-centric.Source: §1 · p. 1
• The paper introduces norm discovery from conversation context and a collection of English and Chinese conversations spanning genres, topics, and cultures.Source: §1 · p. 1
• NORMSAGE uses GPT-3 prompting to propose norms, then explanation-aware correctness and grounding checks to assess candidates against their source dialogues.Source: §1 · pp. 1–2
• Human judgments select NORMSAGE norms as best in at least 70% of cases, while Chinese-versus-English relevance and correctness differ by no more than 5%.Human annotators identify culture-specific norms with 80% accuracy.Source: §1 · p. 2
• NormsKB contains over 20,000 unique norms extracted from multilingual, multicultural conversations, with correctness and grounding explanations.Source: §1 · p. 2

2.1 Task Formulation

• The task derives a set of relevant norms in English from a conversation scenario in a predefined target language, such as English or Chinese.Source: §2.1 · p. 2
• Background context or speaker-profile information can optionally enrich discovery, while verification produces correctness and grounding verdicts, confidence scores, and explanations.Grounding distinguishes entailment, irrelevance, and contradiction.Source: §2.1 · p. 2
• The task grounds discovered norms in dialogue examples with confidence and natural-language explanations, enabling self-supervised verification and adherence or violation checks.Source: §2.1 · p. 2

2.2 Source of Data

• Source data is sought for realistic dialogue exchanges and diversity across topics and societal or cultural groups.Source: §2.2 · pp. 2–3
• Table 1 describes each raw-data source by language, topic domain, spoken-token count, and dialogue-turn count.Source: Table 1 · p. 3
• The collection expands a predominantly single-culture TVQA source with cross-cultural television and movie dialogue, plus Chinese and English real-world chats, negotiations, and documentaries.Wikipedia summaries are collected when available, and transcripts are chunked every five lines.Source: §2.2 · p. 3

3.1 Initial Norm Discovery

• The dvr(·) operator prompts GPT-3 with a context description, dialogue, and directed norm-discovery question to generate candidate norms.Source: §3.1 · p. 3
• Prompt structure enhancement adds frames for expected norm structure or cultural indicators for topic-specific taxonomies.Cultural indicators are extracted by prompting over available background summaries.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2 Self-Verification with Correctness Checking & Explainable Grounding

• The cor(·) operator generates a correctness verdict and explanation, filters candidates below a tunable 0.7 correctness-probability threshold.Source: §3.2 · p. 4
• The grd(·) operator judges whether dialogue entails, contradicts, or is irrelevant to a candidate norm, explaining its verdict and filtering scores below 0.6.Source: §3.2 · p. 4
• Before adding discoveries to NormsKB, the framework discards norms whose BERT embeddings exceed a 0.95 cosine-similarity threshold against an existing norm.Source: §3.2 · p. 4

4.1 Intrinsic Norm Discovery Evaluation

• NORMSAGE outperforms the baselines across all five norm-quality dimensions when enhanced with frame or structured indicators.The reported comparisons also identify poor portability from curated Reddit rules-of-thumb to in-situ conversations as a baseline limitation.Source: Table 2 · p. 5, §4.1 · p. 5

4.1.1 Multi-Lingual Norm Discovery

• Figure 3 presents Chinese conversation examples that the authors describe as high-quality in detail and correctness.The authors state that the prompting method can extend to other languages on which GPT-3 was pretrained.Source: Figure 3 · p. 5, §4.1.1 · pp. 5–6
• NormSAGE shows relative consistency across Chinese and English conversations, with relevance and correctness differences of at most 9.5%.English norms score slightly higher for insightfulness (≤9.5%) but lower for relatableness (≤2.1%); the authors suggest dialogue nature may contribute.Source: §4.1.1 · pp. 5–6, Table 4 · p. 6

4.1.2 Culture-Specific Norm Discovery

• Human annotators identify culture-specific norms with 80% accuracy, with Fleiss Kappa agreement of 60.8%.The authors note that some errors may reflect subjectivity in judging which culture is more likely to discuss a topic.Source: Table 5 · p. 6, §4.1.2 · p. 6

4.2 Extrinsic Evaluation on Norm Correctness Self-Verification

• Zero-shot prompting approaches achieve the best norm-correctness verification performance, while cor(·)-NORMSAGE provides high-quality explanations unlike T0pp’s circular explanations.T0pp has higher accuracy and AUC than cor(·)-NORMSAGE; the authors suggest the shared GPT-3 backbone may bias verification toward norms it generated.Source: Table 6 · p. 7, §4.2 · p. 7

4.4 Resource Contribution

• NORMSAGE discovers over 20,500 unique norms, including 1,250 culture-specific norms, and runs discovery at 8.6 seconds per dialogue.Grounding takes 3.8 seconds per dialogue; discovery is over 10× faster than human annotation, requires no researcher GPU, and NORMSAGEmini performs competitively on discovery and verification.Source: §4.4 · p. 8

4.3 Extrinsic Evaluation on Norm Grounding

• For three-class norm grounding, grd(·)-NORMSAGE achieves 68% accuracy without speaker localization and 80% with it.Speaker localization is important for prompting-based three-class grounding, and NORMSAGE explanations are preferred over human-written ones in around 40% of cases.Source: §4.3 · pp. 7–8, Table 8 · p. 8, Table 7 · p. 8

5 Related Work

• Earlier studies examined cooperative principles, politeness implicatures, and relationships between norms and law.Source: §5 · p. 8
• Moral-judgment modeling has used an evocative-word lexicon grounded in moral-foundation theory and Reddit anecdotes (Emelin et al., 2021; Sap et al., 2019a; Lourie et al., 2021).Source: §5 · p. 9
• Prior norm-discovery approaches catalog manually curated Reddit rules-of-thumb and train models to generate them, whereas this work discovers norms from conversations without manual curation.Source: §5 · p. 9
• Modeling social and moral dynamics supports applications including cyberbullying and hate-speech detection, detoxification, debiasing, model alignment, and social-media understanding.Source: §5 · p. 9
• Norm discovery and grounding relate technically to prompt engineering and explainable natural-language inference, and NORMSAGE operates without intensive prompt-tuning or fine-tuning.Source: §5 · p. 9

6 Conclusions

• NORMSAGE combines large-language-model knowledge elicitation with self-verification to discover dialogue norms across multilingual and diverse social and cultural groups.Source: §6 · p. 9
• The conclusion reports improved discovery of dialogue norms across diverse groups, including multilingual conversations and culture-specific norms, with natural-language explanations for interpretable discovery and violation detection.Source: §6 · p. 9
• The authors propose that discovering norms that may differ across cultures can help people navigate communication barriers and find common ground to avoid norm violations.Source: §6 · p. 9

Limitations

• Discovered norms apply to the time period aligned with the conversation scenario from which each norm is derived.Source: Limitations · pp. 9–10
• GPT-3 was used instead of GPT-4 because GPT-4 had a rate limit of 25 request calls every 3 hours at the time of the experiments.Source: Limitations · p. 10
• The authors caution that computational models may be biased, model behavior may change over time, and generated explanations may not entail predictions or be grounded in the input.Source: Limitations · p. 10

Ethical Considerations

• Discovered norms are not intended as a global, universally binding ethical system, but as discrete intuitions that help explain latent assumptions.Source: Ethical Considerations · p. 10
• The system does not distinguish cultural-hierarchy levels such as Cantonese versus Chinese culture, relies on source-data cultural references, and may produce norms viewed as stereotypical.Source: Ethical Considerations · p. 10
• The authors position the transparent, flexible system as a human-AI collaboration tool for domain experts to use in moderation.Source: Ethical Considerations · p. 10
• The paper reports research-use-only conversation data and requires a Data Usage Agreement, while noting that source conversations may include explicit content and burden annotators.Source: Ethical Considerations · p. 10

A.1 Speaker Profiling and Cultural Indicator Extraction for Norm Discovery with Frames

• Cultural indicators are used to guide norm discovery toward better structure and culture-specific insights, with dialogue content and a directed task question in the prompt.The prompting logic is similar to that used for norm discovery, correctness verification, and relevance grounding.Source: §A.1 · pp. 13–14
• For each person, the extraction asks for an affiliated country- or state-level culture in adjective form when available, and skips people without that information.Source: §A.1 · p. 14
• An Outsourced dialogue’s Wikipedia summary provides context that yields “Todd Anderson: American” and “Puro: Indian” profiles, which are plugged into the norm-discovery input.The profiles follow a consistent textual pattern that is easy to parse.Source: §A.1 · p. 14

A.2.1 Assessment Metric Guidelines

• Norms are assessed for relevance, well-formedness, correctness, insightfulness, and relatableness.The guidelines define relevance by connection to the situation, well-formedness by structure, correctness by agreement that the norm holds, insightfulness by enlightening social understanding, and relatableness by balancing generality and specificity.Source: §A.2.1 · p. 14
• Examples contrast a conversation-relevant norm with an unrelated one, and a self-contained norm with a fragment that omits the assessed behavior.Source: §A.2.1 · p. 14
• The correctness examples contrast assigning tasks by expertise with assigning them alphabetically.Source: §A.2.1 · p. 14
• Insightfulness and relatableness examples contrast general claims about using expertise with narrower claims tied to specific software proficiency.Source: §A.2.1 · p. 14
• Annotators rate discovered norms on a 1–5 Likert scale.Source: §A.2.1 · p. 14