PaperlayerView exact revision on arXiv

Source-linked AI summary

CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, Ji Zhang, Chao Peng, Fei Huang, Jingren Zhou

arXiv:2307.09705v1cs.CL

TL;DR

Chinese benchmarks have largely emphasized knowledge and reasoning rather than human-values alignment. CVALUES evaluates safety and responsibility with expert-collected prompts and human and automatic methods. Most Chinese LLMs perform well on safety, while responsibility remains an area for improvement.

  • Problem

    Chinese benchmarks largely test knowledge and reasoning rather than human-values alignment, leaving this alignment insufficiently assessed in the Chinese context.

    Source: §1 · pp. 1–2

  • Method

    CVALUES assesses safety and responsibility using adversarial safety prompts, expert-induced responsibility prompts, and both human and multiple-choice automatic evaluation.

    Source: §1 · p. 2

  • Results

    Most Chinese LLMs perform well on safety, but have considerable room to improve on responsibility, and human and automatic evaluation assess different aspects of alignment.

    Source: §1 · p. 3, §7 · p. 9

  • Takeaways & Limitations

    The authors conclude that both human and automatic evaluation matter for assessing Chinese LLMs' human-values alignment and that CVALUES can support research on this alignment.

    Source: §7 · p. 9

Abstract

from arXiv · show

With the rapid evolution of large language models (LLMs), there is a growing concern that they may pose risks or have negative social impacts. Therefore, evaluation of human values alignment is becoming increasingly important. Previous work mainly focuses on assessing the performance of LLMs on certain knowledge and reasoning abilities, while neglecting the alignment to human values, especially in a Chinese context. In this paper, we present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs in terms of both safety and responsibility criteria. As a result, we have manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains by professional experts. To provide a comprehensive values evaluation of Chinese LLMs, we not only conduct human evaluation for reliable comparison, but also construct multi-choice prompts for automatic evaluation. Our findings suggest that while most Chinese LLMs perform well in terms of safety, there is considerable room for improvement in terms of responsibility. Moreover, both the automatic and human evaluation are important for assessing the human values alignment in different aspects. The benchmark and code is available on ModelScope and Github.

1 Introduction

Existing Chinese benchmarks largely test knowledge and reasoning rather than human-values alignment. CVALUES evaluates Chinese LLMs on safety and responsibility, using human and automatic evaluations; results show stronger safety than responsibility.

1 Introduction

  • Existing Chinese benchmarks focus on abilities such as knowledge and reasoning, while leaving human-values alignment unexamined.A cited Chinese safety benchmark uses InstructGPT as evaluator, which the authors say may be unreliable for Chinese culture and policies.Source: §1 · p. 2
  • CVALUES introduces two ascending Chinese-context criteria: safety, which excludes harmful or toxic content, and responsibility, which adds positive guidance and humanistic care.Responsibility also accounts for effects on society and the world.Source: §1 · p. 2, Figure 2 · p. 2
  • The benchmark collects safety prompts across 10 scenarios and expert-induced responsibility prompts from 8 domains, then evaluates Chinese LLMs through human annotation and automatic multiple-choice prompts with opposing options.Source: §1 · p. 2
  • Most Chinese LLMs perform well on safety, but have considerable room to improve on responsibility; human and automatic evaluations assess different aspects of values alignment.Automatic multiple-choice evaluation tests comprehension of unsafe or irresponsible behavior, while human evaluation assesses generated responses.Source: §1 · p. 3
  • The authors publicly release the benchmark and code, alongside CVALUES-COMPARISON, a dataset of 145k prompts with paired positive and negative responses.They hope CVALUES will support research toward more responsible Chinese AI.Source: §1 · p. 3

2 The CVALUES Benchmark

CVALUES defines safety and responsibility as ascending values criteria, gathers adversarial safety prompts and expert-induced responsibility prompts, and evaluates models through human and multiple-choice methods.

2.1 Definition and Taxonomy

  • CVALUES frames safety as a baseline against harmful content and responsibility as a higher criterion requiring positive guidance, humanistic care, and attention to social impact.The authors motivate the higher criterion by noting that prior work mainly focused on safety and that responsible guidance is more demanding than vague or neutral replies.Source: §2.1 · p. 3

2.2 Data Collections

  • Initially, crowdworkers manually wrote safety test prompts based on the taxonomy, but these were easy to solve, so they were asked to attack an early ChatPLUG chatbot.Questions that successfully triggered safety issues were labeled by scenario and added to the test prompts. Separately, the project collected 800 questions from 8 experts, each providing 100.Source: §2.2 · pp. 3–4

2.3 Evaluation

  • CVALUES evaluates values performance through both human evaluation and automatic evaluation.Human evaluation is intended to provide reliable and comprehensive comparisons; the automatic method is presented as lightweight and reproducible.Source: §2.3 · p. 4, §2.3.1 · p. 4, §2.3.2 · p. 4

2.3.1 Human Evaluation

  • For responsibility, domain experts assess ChatPLUG-13B responses to their own prompts, selecting the best response or neither and rating the selected response from 1 to 10.ChatPLUG-13B generates three candidates per prompt, and domain scores are averages; experts evaluate this representative model because annotating all model responses would be costly and time-consuming.Source: §2.3.1 · p. 4

2.3.2 Automatic Evaluation

  • For multiple-choice evaluation, prompts pair safe and unsafe responses, swap their positions to reduce position bias, and use the same construction approach for responsibility.When a question initially has responses from only one category, ChatGPT is instructed to rewrite one response into the opposite category.Source: Figure 3 · p. 4, §2.3.2 · p. 5
  • The automatic evaluation set contains 4,312 prompts: 2,600 for safety and 1,712 for responsibility; accuracy is reported both including and excluding refusals.The authors note that models may refuse to decide on security or ethics grounds.Source: §2.3.2 · p. 5

3 Results of Human Evaluation

The human evaluation assesses Chinese LLMs using raw safety and responsibility prompts. Most models perform well on safety, but ChatPLUG-13B shows uneven responsibility scores across domains and substantial room for improvement.

3.1 Experimental Settings

  • The human evaluation tested 10 Chinese-input-capable LLMs with raw test prompts, including a pretraining-only model and instruction-tuned models.Chinese-LLaMA-13B was pre-trained only; the other listed models were instruction-tuned with SFT/RLHF.Source: §3.1 · p. 5, §3 · p. 5

3.2 Results on Values of Safety

  • Most evaluated Chinese LLMs performed well on safety; ChatGPT ranked first, while Chinese-Alpaca-Plus-7B and ChatGLM-6B had similar safety scores.The authors suggest safety data during instruction tuning improves scores and note that pretraining-only Chinese-LLaMA-13B performed very poorly.Source: §3.2 · p. 5

3.3 Results on Values of Responsibility

  • ChatPLUG-13B scored above 7 in five responsibility domains, led by Environment Science at 8.7, but performed poorly in Law and Social Science.The paper attributes the Law results to weak legal reasoning and susceptibility to inducement traps; Social Science responses lacked comprehensiveness and empathy, and the expert scored any identified issue as 1.Source: §3.3 · p. 6
  • ChatPLUG-13B still has substantial room to improve responsibility; tests of bad cases also found similar problems in ChatGPT and ChatGLM-6B.The authors argue that exploring values alignment across domains is worthwhile.Source: §3.3 · p. 6

4 Results of Automatic Evaluation

Automatic evaluation used multiple-choice prompts to assess safety and responsibility. ChatGPT ranked first, and the results show a broad responsibility gap relative to safety, alongside differences between models in refusals and human-evaluation performance.

4.1 Experimental Settings

  • The automatic evaluation used multiple-choice prompts; Table 4 reports results under two settings, with Acc∗ excluding failed cases when models refuse.Chinese-LLaMA-13B, ChatPLUG, and BELLE-7B-2M were excluded for invalid answers, unsuitable multiple-choice ability, or failure to follow instructions, respectively.Source: §4 · p. 6, §4.1 · p. 7, Table 4 · p. 7

4.2 Results and Analysis

  • ChatGPT ranked first in the multiple-choice evaluation, with Ziya-LLaMA-13B-v1.1 second, only 2.3 points behind.The remaining listed order was Ziya-LLaMA-13B-v1, ChatGLM-6B, Chinese-Alpaca-Plus-13B, Chinese-Alpaca-Plus-7B, and MOSS.Source: §4.2 · p. 7
  • Responsibility accuracy was generally much lower than safety accuracy; for ChatGLM-6B, responsibility was 10.2 points lower.The authors conclude that current models need to improve their responsibility alignment.Source: §4.2 · p. 7, Table 4 · p. 7
  • Chinese-Alpaca-Plus-7B and -13B had large gaps between Avg. and Avg.∗, which the authors infer reflects helpfulness sacrificed for harmlessness and more false rejections.Other models were described as relatively balanced on this measure.Source: §4.2 · p. 7
  • Ziya-LLaMA achieved high multiple-choice accuracy but low human-evaluation scores, suggesting comprehension of safe choices did not ensure aligned generation.The authors note that it could still offer suggestions for harmful behavior.Source: §4.2 · p. 7

5 Discussion

The discussion contrasts Chinese LLMs’ generally good safety with weaker responsibility alignment and explains why human and automatic evaluations capture different aspects. It recommends combining the two evaluation methods in practical development.

Overall values performance of Chinese LLMs

  • Instruction-tuned models generally perform well on safety, but rejection alone is insufficient for responsibility, which requires responses aligned with human values.The authors say adding some security data during instruction tuning can help models learn to reject risky prompts more effectively.Source: §5 · p. 7

The differences between human and automatic evaluation

  • Multiple-choice prompts test comprehension of unsafe or irresponsible behavior, whereas raw adversarial prompts test values understanding and generation; Ziya-LLaMA scored highly on the former but poorly in manual evaluation.The authors suggest it can distinguish unsafe responses without being well-aligned enough to avoid harmful generation.Source: §5 · pp. 7–8

The practical suggestions for evaluating values

  • The authors recommend combining multiple-choice evaluation, potentially with varied option difficulty, and manual evaluation to assess both values understanding and generation alignment.They suggest manual evaluation once a model reaches a certain level of understanding.Source: §5 · p. 8

6 Related Work

The related work describes Chinese LLMs developed through pretrained models and alignment methods, then contrasts capability benchmarks with the less-explored assessment of Chinese social values. CValues contributes both human and automatic evaluation of safety and responsibility.

6.1 Large Language Models

  • Chinese LLMs such as ChatGLM, MOSS, ChatPLUG, BELLE, and Ziya-LLaMA build on pretrained models and use supervised fine-tuning or RLHF; this paper evaluates their safety and responsibility values.The authors distinguish their human-labeled values benchmark from earlier work focused mainly on helpfulness.Source: §6.1 · p. 8

6.2 Evaluation Benchmarks

  • Chinese benchmarks have largely assessed helpfulness, knowledge, and reasoning, while responsibility and Chinese social values remain under-explored; an earlier safety benchmark used InstructGPT as evaluator, which may not know Chinese culture and social values.CValues provides evaluation of Chinese social values through both human evaluation and automatic multiple-choice question answering.Source: §6.2 · pp. 8–9

7 Conclusion

CVALUES evaluates Chinese LLMs on safety and responsibility using human and automatic evaluation. Most models perform well on safety, while responsibility remains an area for improvement.

7 Conclusion

  • CVALUES evaluates Chinese LLM alignment with human values across both safety and responsibility criteria.The paper describes it as a comprehensive benchmark.Source: §7 · p. 9
  • The evaluation combines human assessment for reliable comparisons with multi-choice prompts for automatic testing.Source: §7 · p. 9
  • Most Chinese LLMs perform well on safety, but responsibility has considerable room for improvement; both evaluation modes matter across different aspects.Source: §7 · p. 9

A Taxonomy and Examples

The taxonomy describes safety scenarios and responsibility-related risks through categories of potentially harmful model responses.

A Taxonomy and Examples

  • Responsibility categories include dangerous topics, crimes, physical or mental health harms, and exposure of personal privacy.Source: A Taxonomy and Examples · p. 12
  • Other responsibility categories cover sensitive-topic bias or inaccuracy, subjective or biased output, unethical behavior, and maliciously induced unsafe content.Source: A Taxonomy and Examples · p. 12
  • Safety examples include following attack instructions to generate unsafe or fake content and producing unsafe content outside the listed scenarios.Source: A Taxonomy and Examples · p. 12

B Data Statistics

The appendix directs readers to statistics tables for safety, responsibility, and multi-choice prompts.

B Data Statistics

  • Prompt statistics are presented in Table 6 for safety, Table 7 for responsibility, and Table 5 for multi-choice prompts.Source: B Data Statistics · p. 12

C The Labeling Tool for Professional Experts

The section identifies a screenshot of the professional-expert labeling tool.

C The Labeling Tool for Professional Experts

D CVALUES-COMPARISON Construction

CVALUES-COMPARISON is constructed through a four-step pipeline that turns seed prompts into paired safe and unsafe responses. The pipeline yields 145k comparison samples, split into 116k training and 29k test samples.

D CVALUES-COMPARISON Construction

Related papers

Loading 2307.09705v1…