PaperlayerView exact revision on arXiv

Source-linked AI summary

Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation

Yushi Yao, Zheng Huang

arXiv:1602.04874v1cs.LGcs.CL

TL;DR

Classical Chinese word-segmentation methods focus on hand-crafted contextual features. This paper applies BLSTM to segmentation, learning character-level features automatically, and reports good performance and generalization on simplified and traditional Chinese datasets.

  • Problem

    Classical Chinese word-segmentation systems focus on feature engineering, while this paper investigates a model that does not depend on predesigned features.

    Source: §1 · p. 1, §4 · p. 4

  • Method

    The paper applies bidirectional LSTM to Chinese word segmentation, automatically learning character-level features without hand-crafted features or prior knowledge.

    Source: §1 · p. 1, §5 · p. 4

  • Results

    Experiments on SIGHAN Backoff 2005 show good performance and generalization on both simplified and traditional Chinese datasets.

    Source: §5 · p. 4

  • Takeaways & Limitations

    The authors conclude that BLSTM networks with word embeddings are an effective tagging solution worth further exploration.

    Source: §5 · p. 4

Abstract

from arXiv · show

Recurrent neural network(RNN) has been broadly applied to natural language processing(NLP) problems. This kind of neural network is designed for modeling sequential data and has been testified to be quite efficient in sequential tagging tasks. In this paper, we propose to use bi-directional RNN with long short-term memory(LSTM) units for Chinese word segmentation, which is a crucial preprocess task for modeling Chinese sentences and articles. Classical methods focus on designing and combining hand-craft features from context, whereas bi-directional LSTM network(BLSTM) does not need any prior knowledge or pre-designing, and it is expert in keeping the contextual information in both directions. Experiment result shows that our approach gets state-of-the-art performance in word segmentation on both traditional Chinese datasets and simplified Chinese datasets.

1 Introduction

The paper applies bidirectional LSTM networks to Chinese word segmentation and constructs higher-level character features with BLSTM. Its training framework integrates character embeddings and character tagging without external datasets.

1 Introduction

  • Chinese sentence structure motivates considering both past and future context during segmentation.The related-work discussion contrasts this with prior Chinese segmentation using LSTM to memorize only past context.Source: §1 · p. 1
  • The paper applies BLSTM to Chinese word segmentation and constructs higher-level character features with the network.The authors describe this as the first application and improvement of bidirectional LSTM on Chinese word-segmentation benchmark datasets.Source: §1 · p. 1
  • The training framework integrates embedding generation and character tagging without requiring external datasets.Source: §1 · p. 1

2 BLSTM network architecture

BLSTM uses recurrent layers to retain sequential context from both directions. Its LSTM units update a cell state through three gates, and the bidirectional layer concatenates forward and backward outputs.

2.1 LSTM unit

  • An LSTM unit combines a cell state with forget, input, and output gates that control retained, stored, and emitted information.The recurrent part is updated using the current cell state and passed to the next iteration.Source: §2.1 · p. 2

2 BLSTM network architecture

  • BLSTM propagates in forward and backward directions to retain contextual features from both past and future.Source: §2.2 · p. 2

2.2 BLSTM Network

  • The BLSTM layer concatenates its forward and backward outputs into a single output vector.The concatenated output has dimension 2d when each directional output has dimension d.Source: §2.2 · p. 2

3 Training Method

The method converts Chinese word segmentation into character tagging and uses character embeddings with BLSTM context to predict each character’s tag. It also stacks BLSTM layers while transforming their outputs to control dimensionality.

3 Training Method

  • Characters receive B, M, E, or S tags to mark word beginnings, middles, endings, or single-character words.Source: §3 · p. 2

3.1 Training framework

  • Characters are mapped to dense embeddings, processed by a BLSTM, and passed through hidden and softmax layers to select each character’s most probable tag.The embeddings are retrieved from a matrix using character IDs defined by a dictionary built from the training set.Source: §3.1 · p. 3

3.2 Model variant

  • Stacked BLSTM layers are intended to extract higher-level contextual features, while a transformation matrix keeps each layer’s output dimension equal to its input dimension.The output is initially twice the input dimension because it combines forward and backward LSTM layers.Source: §3.2 · p. 3

4 Experiments

The experiments evaluate the model on simplified and traditional Chinese benchmark datasets and examine embedding size and BLSTM depth. The reported findings include competitive performance against a character-embedding LSTM and diminishing returns from deeper networks.

  • Evaluation used SIGHAN Backoff 2005 simplified Chinese datasets PKU and MSRA and traditional Chinese datasets AS and HK.Source: §4 · p. 3
  • Adding BLSTM layers produced slight improvement, but gains were less effective beyond three layers and training took longer.Source: §4 · p. 3, Table 1 · p. 4
  • 200-dimensional embeddings performed best on HKCityU; larger dimensions increased and destabilized the error rate.Source: §4 · p. 3, Table 2 · p. 4
  • Unlike the compared prior systems, which focused on feature engineering, this approach relies on BLSTM automatic feature learning rather than predesigned features.Source: §4 · p. 4
  • The model achieved competitive performance against Chen’s character-embedding LSTM, and results suggest BLSTM may perform better for segmentation.Source: §4 · p. 4

5 Conclusions

The conclusion presents BLSTM as an approach for Chinese word segmentation that learns character-level features automatically without handcrafted features or prior knowledge. Experiments on SIGHAN Backoff 2005 show good performance and generalization across simplified and traditional Chinese datasets.

  • The paper applies bidirectional LSTM networks to Chinese word segmentation as a sequential tagging task.Source: §5 · p. 4
  • The model automatically extracts discriminative character-level features without requiring handcrafted segmentation features or prior knowledge.Source: §5 · p. 4
  • Experiments on SIGHAN Backoff 2005 show good performance and generalization on both simplified and traditional Chinese datasets.Source: §5 · p. 4

Related papers

Loading 1602.04874v1…