Source-linked AI summary
Analogical Reasoning on Chinese Morphological and Semantic Relations
Shen Li, Zhe Zhao, Renfen Hu, Wensi Li, Tao Liu, Xiaoyong Du
TL;DR
Chinese analogical reasoning lacked a reliable benchmark covering the language’s morphological and semantic regularities. This paper constructs CA8 from Chinese lexical knowledge and evaluates how representations, context features, and corpora affect reasoning, finding that CA8 is a demanding benchmark whose performance depends on these factors.
Problem
Existing Chinese analogy evaluation relies on CA_translated, which has 134 unique words in three semantic relations and excludes morphological knowledge.
Method
The paper models 68 morphological and 28 semantic relations, builds the balanced CA8 dataset with 17,813 questions, and tests vector representations, context features, and corpora.
Results
CA8 is harder than CA_translated; the best configuration achieves 68.0%, while character and ngram features consistently improve CA8 performance and character features especially benefit morphology.
Takeaways & Limitations
CA8 and the released embeddings provide a benchmark and resource base for evaluating Chinese word embeddings and supporting Chinese NLP tasks.
Abstract
from arXiv · showhide
Analogical reasoning is effective in capturing linguistic regularities. This paper proposes an analogical reasoning task on Chinese. After delving into Chinese lexical knowledge, we sketch 68 implicit morphological relations and 28 explicit semantic relations. A big and balanced dataset CA8 is then built for this task, including 17813 questions. Furthermore, we systematically explore the influences of vector representations, context features, and corpora on analogical reasoning. With the experiments, CA8 is proved to be a reliable benchmark for evaluating Chinese word embeddings.
1 Introduction
Chinese linguistic regularities differ from those captured by existing analogy benchmarks because Chinese uses function words, reduplication, and culturally grounded semantic relations. The paper addresses this gap by introducing a Chinese analogical reasoning study and benchmark.
- Chinese lexical knowledge: Chinese uses function words and reduplication to express grammatical and semantic information.Figure 1 illustrates comparative and superlative marking with function words and an “every” meaning from reduplication.
- Chinese lexical knowledge: Chinese semantic relations can reflect social and cultural knowledge, including associations involving Li-bai and Du-fu.
- Benchmark gap: The existing CA_translated dataset contains only 134 unique Chinese words across three semantic relations and omits morphological knowledge.
- Paper contribution: The study investigates Chinese linguistic regularities through analogical reasoning and examines vector offset methods for morphological and semantic relations.
- Paper contribution: The released benchmark and 36 open-source pre-trained embeddings provide resources for evaluating Chinese word embeddings and Chinese NLP tasks.
2 Morphological Relations
The paper characterizes Chinese morphology through reduplication and semi-affixation rather than conventional inflectional structure. It identifies reduplication patterns and inventories semi-affixes that alter meaning or part of speech.
- Morphological framework: Chinese morphology is analyzed through reduplication and semi-affixation, reflecting a system distinct from languages with clear roots and affixes.
- Reduplication: Reduplication repeats a morpheme to create a word that is semantically or syntactically distinct from the original.The paper finds reduplication abilities across nouns, verbs, adjectives, adverbs, and measure words, with six summarized repetition patterns.
- Reduplication: Reduplication can encode kinship, distributive meanings, brief actions, adjective intensification, or adverb formation.
- Semi-affixation: Chinese semi-affixes are components that behave like affixes while also functioning as independent lexemes.
- Semi-affixation: The paper identifies 21 semi-prefixes and 41 semi-suffixes, including forms that create ordinal numbers or nominalize adjectives.For example, dì- attaches to numerals and -zi nominalizes adjectives.
3 Semantic Relations
The paper organizes Chinese semantic knowledge into 28 relations across geography, history, nature, and people, including Chinese-specific and cross-lingual relation types.
- 28 semantic relations cover geography, history, nature, and people.
- Chinese province analogies connect provinces with abbreviations, capitals, and representative dramas.Examples include Anhui–Zhejiang, Wan–Zhe, Hefei–Hangzhou, and Huangmei Opera–Yue Opera.
- Some relations are inherited from English analogy datasets, including country–capital and family-member relations.
- The dataset also introduces relations proposed for broader multilingual use, such as scientists–findings and companies–founders.
4 Task of Chinese Analogical Reasoning
The task retrieves an answer for “a is to b as c is to ?” using curated Chinese word pairs and the balanced CA8 benchmark, evaluated alongside CA_translated.
- The analogy task retrieves an answer to “a is to b as c is to ?”.
- 1852 unique Chinese word pairs are collected after dictionaries and word-segmentation specifications guide word-pair inclusion.
- The experiments use shared SGNS and PPMI hyper-parameter settings and solve analogies with 3COSMUL.
- CA8 contains 17813 questions and limits each relation to at most 50 word pairs to reduce imbalance.
- CA8 and CA_translated are both used for evaluation in the experiments.
5 Experiments
Experiments examine how vector representations, context features, and training corpora affect Chinese analogical reasoning, using CA8 and related comparisons. Results show distinct representation strengths, benefits from Chinese-aware features, and strong effects from corpus size and domain.
- 5 Experiments: The experiments vary sparse and dense representations, character, word, and ngram context features, and six corpora differing in size and domain.The corpora include Chinese Wikipedia, Baidubaike, People’s Daily News, Sogou News, Zhihu QA, and their Combination.
- 5.1 Vector Representations: SGNS representations perform better on CA8 morphological relations, while PPMI representations have advantages on semantic relations.The paper relates this pattern to SGNS favoring frequent word pairs and PPMI being more sensitive to infrequent, specific pairs.
- 5.1 Vector Representations: CA8 provides balanced morphological and semantic questions, making it a reliable benchmark for studying dense and sparse vectors.Unlike CA_translated and several English analogy datasets, CA8 covers both relation types with balanced categories.
- 5.2 Context Features: Ngram and character features significantly and consistently improve performance across almost all CA8 categories.Character features are especially advantageous for morphology; SGNS with character features doubles accuracy on morphological questions.
- 5.2 Context Features: 68.0% is the best configuration accuracy reported for CA8, whereas some CA_translated categories already leave little room for improvement.The paper characterizes CA8 as substantially harder for representation methods than CA_translated.
- 5.3 Corpora: Accuracies increase with corpus size, while corpus domain affects which relations benefit most.News data help geography relations, Zhihu QA helps reduplication, and the large, varied Combination corpus performs much better across morphology and semantics.
6 Conclusion
The paper proposes a Chinese analogical reasoning task grounded in Chinese linguistic regularities and examines representation, context, and corpus effects. It presents the benchmark and released embeddings as resources for Chinese NLP.
- 6 Conclusion: The task models 68 morphological relations and 28 semantic relations in Chinese analogical reasoning.The study applies a vector offset method while examining vector representations, context features, and corpora.
- 6 Conclusion: The released benchmark and embedding sets provide a basis for evaluating Chinese word embeddings and supporting Chinese NLP tasks.The paper describes this as an approach combining linguistic analysis with representation models.