Source-linked AI summary
Chinese NER Using Lattice LSTM
Yue Zhang, Jie Yang
TL;DR
Chinese NER faces segmentation errors and character-based models' limited use of explicit word information. The paper addresses this with a lexicon-informed lattice LSTM that jointly encodes characters and matching words, and it reports the best results across varied Chinese NER datasets. The approach's accuracy may nevertheless depend on lexicon quality.
Problem
Chinese NER is vulnerable to segmentation errors, while character-based methods do not fully exploit explicit word and word-sequence information.
Method
The model builds a character-centered lattice from all lexicon-matched subsequences and uses gated LSTM cells to route information from word paths to characters.
Results
The lattice model significantly outperforms character- and word-sequence LSTM-CRF models across Chinese NER datasets from different domains.
Takeaways & Limitations
Lattice LSTM uses explicit word information without relying on word segmentation and can select lexicon words in context for NER disambiguation.
Takeaways & Limitations
Lexicon quality may affect accuracy because noise words can confuse NER, although the lattice model may learn to select more correct words during training.
Abstract
from arXiv · showhide
We investigate a lattice-structured LSTM model for Chinese NER, which encodes a sequence of input characters as well as all potential words that match a lexicon. Compared with character-based methods, our model explicitly leverages word and word sequence information. Compared with word-based methods, lattice LSTM does not suffer from segmentation errors. Gated recurrent cells allow our model to choose the most relevant characters and words from a sentence for better NER results. Experiments on various datasets show that lattice LSTM outperforms both word-based and character-based LSTM baselines, achieving the best results.
1 Introduction
Chinese NER must balance segmentation-related errors against the limited word information in character-based models. The paper introduces a lattice LSTM that integrates lexicon-matched words with characters and reports superior results across Chinese NER datasets.
- Chinese NER can suffer error propagation when segmentation mistakes create incorrect entity boundaries, especially in open-domain settings.
- Character-based methods avoid segmentation errors but do not fully exploit explicit word and word-sequence information.
- The model constructs a word-character lattice by matching each sentence against a large automatically obtained lexicon.The lattice exposes potential words such as 长江大桥, 长江, and 大桥 alongside the character sequence.
- Lattice LSTM gated cells dynamically route information from different paths to each character, allowing the model to select useful contextual words during NER training.
- The model significantly outperforms both character- and word-sequence LSTM-CRF models across Chinese NER datasets from different domains.
2 Related Work
Prior Chinese NER work established character sequence labeling as a strong approach while exploring segmentation, lexicon, and other external information. This paper positions lattice LSTM as a neural representation that uses word information without requiring a word segmentor.
- Character sequence labeling has been a dominant Chinese NER approach and has empirically outperformed statistical word-based methods.
- With suitable representation settings, the same character-over-word conclusion holds for neural NER, while lattice LSTM outperforms both alternatives.
- Earlier methods incorporated segmentation information through soft features, joint segmentation and NER, or related structured and multitask approaches.
- NER research has also used lexicon features, language modeling, pretrained character models, and cross-domain or cross-lingual multitask learning.
- The paper presents lattice LSTM as a character-centric lattice representation for mixed characters and lexicon words, designed for segmentation-free Chinese NER.
3 Model
The model uses LSTM-CRF architectures for character, word, and lattice representations, with the lattice model integrating lexicon-matched words into character-level processing. Its gated recurrent structure controls information flow from multiple character and word paths while retaining CRF sequence labeling.
- 3.1 Character-Based Model: The character-based baseline applies a bidirectional LSTM to character embeddings and uses a CRF for sequence labeling.Character hidden states are formed from left-to-right and right-to-left LSTM directions before CRF decoding.
- 3.1 Character-Based Model: Character representations can be augmented with character bigram embeddings or segmentation-label embeddings.The bigram and softword variants concatenate these additional embeddings with character embeddings.
- 3.2 Word-Based Model: The word-based model represents each word with a word embedding, optionally combined with within-word character CNN or LSTM representations, then applies sequence labeling.The word representation is formed by combining word-level and character-level information before bidirectional LSTM processing.
- 3.3 Lattice Model: The lattice model receives the character sequence together with every character subsequence matching a word in lexicon D.The lexicon is built from automatically segmented large raw text, and matched subsequences provide additional word paths.
- 3.3 Lattice Model: Word cells represent recurrent states for lexicon subsequences, while additional gates control their contributions to character-cell updates.Character cells receive information from multiple recurrent paths, including character inputs and matched words; word cells have input and forget gates but no output gate.
- 3.3 Lattice Model: The lattice model dynamically focuses on relevant words during NER labeling and uses a standard CRF layer for character-level sequence labeling.Training back-propagates through the model parameters, and the reported configuration excludes single-character words because including them slightly reduces final performance.
4 Experiments
Experiments compare character-, word-, and lattice-based Chinese NER across multiple datasets and settings. The lattice LSTM consistently improves performance while avoiding explicit word segmentation, though lexicon quality and sentence length remain practical boundaries.
- Experimental Settings: Experiments evaluate word-character lattice LSTMs across Chinese NER datasets and compare word-based and character-based neural models using precision, recall, and F1-score.The datasets include OntoNotes, MSRA, Weibo NER, and an annotated Chinese resume corpus.
- Development Experiments: 69.64% F1 is the best development result among character representations, achieved by combining character and character-bigram information.Character-bigram and softword representations individually improve the character-based LSTM-CRF baseline.
- Development Experiments: 71.62% development F1 is achieved by the lattice LSTM-CRF, significantly exceeding both word-based and character-based methods without character bigrams or word segmentation.Lattice word information improves the best character-based development result from 62.5% to 71.6%.
- Final Results: On OntoNotes, lattice word information raises F1 from 88.81% to 93.18%, compared with 91.87% after adding bichar and softword information.The lattice model achieves the best result with automatic segmentation.
- Final Results: The lattice model significantly outperforms word-based and character-based models on Weibo and resume NER, achieving state-of-the-art results with p < 0.01.The same conclusion is reported across these test domains.
- Discussion: Lattice performance decreases as sentence length increases, but it is more robust than word+char+bichar and char+bichar+softword baselines.The authors attribute the decline partly to the exponentially increasing number of word combinations in the lattice.
- Discussion: Lexicon quality may affect NER accuracy because noisy words can confuse the model, although lattice training may learn to select more correct words.The influence of lexicon quality is left for future work.
5 Conclusion
The lattice LSTM-CRF delivers consistently superior Chinese NER performance across domains while remaining independent of word segmentation. It uses contextual freedom to select lexicon words for NER disambiguation.
- The lattice LSTM-CRF consistently outperforms word-based and character-based LSTM-CRF models across different domains.
- The lattice method is fully independent of word segmentation.
- The model uses contextual freedom to choose lexicon words for NER disambiguation.