PaperlayerView exact revision on arXiv

Source-linked AI summary

Representing and Parsing Korean Constituency Structure at Different Levels of Granularity

Jungyeul Park, KyungTae Lim, Zihao Huang, Eunkyul Leah Jo, Yige Chen, Chulwoo Park

arXiv:2608.27035v1cs.CL

TL;DR

Korean constituency parsing must decide how phrase structure relates to morphologically complex eojeols and non-overt elements. The paper derives three aligned Penn Korean Treebank representations and evaluates canonical transition-based parsers under shared gold-annotation conditions. Morpheme+XPOS provides the strongest results after projection to the eojeol domain, while the paper separately motivates eojeol as an interpretable surface terminal domain.

  • Problem

    Korean constituency parsing requires a principled choice of terminal domain because eojeols contain internal morphology and resources differ in representing morphology and null elements.

  • Method

    The paper removes null elements, aligns Penn Korean phrase structure with overt eojeols, and compares Morpheme+XPOS, Eojeol+XPOS, and Eojeol+UPOS using canonical transition-based parsers.

  • Results

    Morpheme+XPOS gives the strongest results after projection to Eojeol+UPOS, while terminal granularity substantially affects Korean constituency parsing.

  • Takeaways & Limitations

    Fine-grained morphological and XPOS representations provide valuable evidence for the evaluated parsers, while eojeol remains a motivated surface domain for annotation.

  • Takeaways & Limitations

    The experiments define a surface constituency parsing task rather than a full reconstruction of the original empty-category analysis.

Abstract

from arXiv · show

Korean constituency parsing raises a representational challenge because the terminal units of a phrase-structure tree do not straightforwardly correspond to simple surface words. Korean eojeols are morphologically complex spacing units, and existing constituency resources differ in how they represent eojeol-internal morphology and non-overt elements. This paper compares three constituency parsing representations derived from the Penn Korean Treebank: Morpheme+XPOS, Eojeol+XPOS, and Eojeol+UPOS. We construct these representations by removing null elements, aligning Penn Korean phrase structure with overt eojeol tokens, preserving Penn Korean phrase labels where possible, and varying the terminal and preterminal layers. We then evaluate canonical non-binary transition-based constituency parsers in top-down, in-order, and bottom-up orders under a shared modeling and evaluation setup. All experiments use gold terminal segmentation and gold preterminal labels and therefore evaluate constituency parsing conditioned on gold morphosyntactic annotation. Eojeol terminals yield shorter transition sequences, but Eojeol+UPOS parsing substantially underperforms the morphologically richer conditions. Eojeol+XPOS narrows this gap, while Morpheme+XPOS gives the strongest results even after its predictions are projected to the eojeol terminal domain. Under these gold-annotation conditions, the results show that fine-grained morphological and XPOS representations provide valuable evidence for the evaluated parsers. This empirical finding concerns the information available for parsing and does not by itself determine the linguistically preferable terminal domain. Independently, linguistic and resource-design considerations motivate eojeol as a stable and interpretable surface domain for phrase-structure annotation, with morpheme-level and XPOS information retained as aligned morphosyntactic evidence.

1 Introduction

Korean constituency parsing requires explicit decisions about terminal units and morphosyntactic representation because eojeols combine morphology within spacing-based surface units. The paper derives and compares aligned morpheme- and eojeol-based representations while separating empirical parser evidence from linguistic claims about the preferred terminal domain.

  • Motivation: Korean phrase-structure trees must determine whether bound morphemes or eojeols count as constituency terminals.Functional morphology contributes to Korean syntax, but its projection as independent terminals is a representational choice.
  • Resource and conversion setting: The Penn Korean Treebank combines phrase-structural annotation, terminal-internal morphology, and explicit null elements, complicating comparisons with surface-oriented targets.The conversion removes null elements, recovers overt eojeol terminals, and preserves phrase labels where possible.
  • Representations and evaluation: The study compares Morpheme+XPOS, Eojeol+XPOS, and Eojeol+UPOS targets derived from the same aligned source representation.These targets vary terminal granularity and preterminal POS granularity while retaining aligned morphosyntactic information.
  • Representations and evaluation: Three canonical non-binary transition-based parsers are evaluated in top-down, in-order, and bottom-up transition orders.The comparison uses a shared parsing and evaluation setup.
  • Findings and interpretation: Fine-grained morphological and XPOS representations provide valuable evidence under the evaluated parsing conditions, while this result does not determine the linguistically preferable terminal domain.The paper separately motivates eojeol as an overt, interpretable surface domain with morpheme-level information retained as aligned annotation.

2 Background

Korean constituency parsing makes segmentation granularity part of the formal tree representation because eojeols are morphologically complex spacing units. The paper uses the Penn Korean Treebank as a controlled source for deriving comparable surface- and morphology-oriented targets.

  • Korean wordhood and parsing: Korean eojeols are spacing units that may combine stems, particles, auxiliaries, endings, suffixes, and punctuation.This makes the choice between eojeol-level and morpheme-level terminals consequential for constituency structure.
  • Korean wordhood and parsing: Terminal granularity changes the formal object a constituency parser is trained to produce, not merely its input preprocessing.The choice also affects tree interpretation and parser design.
  • Representational alternatives: Morpheme-based trees expose functional morphemes as terminals, whereas eojeol-based trees keep surface units intact and encode morphology elsewhere.Both representations retain linguistic information but place it in different layers of the treebank.
  • Representational alternatives: The study separates terminal granularity from POS granularity by comparing Morpheme+XPOS, Eojeol+XPOS, and Eojeol+UPOS.The same source phrase-structural analysis supports the comparison.
  • Resource scope: The Penn Korean Treebank is a controlled starting point because its overt terminals can be aligned with eojeols and internal morphological annotation.Other resources introduce additional conversion issues, including Sejong debinarization and theoretically motivated KAIST structural decisions.

3 Morpheme- and eojeol-based treebank representations

The paper defines three treebank representations from a shared Penn Korean phrase-structural source, differing in terminal and POS granularity. This design preserves a common structural basis while testing where morphosyntactic information is placed.

  • Shared design: All three representations share sentence data and source phrase labels where possible, differing primarily in terminal yield and preterminal annotation.The aligned representation holds the source phrase-structural analysis constant.
  • Morpheme+XPOS: Morpheme+XPOS treats internal morphemes as terminals with fine-grained Korean XPOS preterminals.Particles, endings, suffixes, and other functional material can appear directly in the tree yield.
  • Evaluation implications: The aligned morpheme-to-eojeol mapping projects Morpheme+XPOS predictions into the eojeol span domain for normalized evaluation.Eojeol targets define shorter sequences over overt spacing units, while morpheme targets provide a finer-grained yield.
  • Eojeol-based representations: Eojeol+XPOS treats each eojeol as one terminal while preserving its internal XPOS sequence in a combined preterminal.This retains detailed morphosyntactic information without projecting morphemes as independent constituency terminals.
  • Eojeol-based representations: Eojeol+UPOS uses the same eojeol terminal domain but assigns each terminal a single coarse UPOS category.This produces a smaller, more comparable preterminal inventory while removing Korean-specific distinctions expressed by XPOS morphology.

4 Canonical non-binary transition systems

The paper uses three canonical non-binary transition systems for projective constituency trees, differing in when labels and child sequences are introduced. Their reductions directly support unary, binary, and higher-arity constituents without artificial binarization.

  • Common framework: All three systems construct projective constituency trees over the input sequence, but differ in the order of label and child introduction.Top-down, in-order, and bottom-up parsing respectively introduce labels before children, after the left corner, or after all children.
  • Top-down parsing: Top-down parsing introduces a phrase label before constructing its children.A nonterminal marker records the opened constituent, and Reduce combines all items above that marker.
  • In-order parsing: In-order parsing introduces a constituent label after constructing its left-corner child.The marker is placed immediately above the left corner before remaining children are built.
  • Bottom-up parsing: Bottom-up parsing constructs all children before introducing the mother label at reduction time.Its arity-specific Reduce-X#k action combines exactly k stack items into a labeled constituent.
  • Common framework: Non-binary systems are used because the converted trees contain constituents with more than two children.Direct construction avoids artificial binary intermediate nodes, and reduction arity is determined by the stack or action specification.

5 Experiments

The experiments compare three aligned Penn Korean Treebank representations with three non-binary transition orders under shared modeling and evaluation conditions. Results are evaluated with gold terminals and preterminals, including normalized comparison in the Eojeol+UPOS domain.

  • Experimental design: The study compares Morpheme+XPOS, Eojeol+XPOS, and Eojeol+UPOS across top-down, in-order, and bottom-up parsers.The representations vary terminal and preterminal layers while using the same data splits, architecture, training procedure, and evaluation protocol.
  • Evaluation conditions: All systems receive gold terminal sequences and gold preterminal labels, so the evaluation is conditioned on gold morphosyntactic annotation.Preterminals are supplied rather than predicted, accounting for essentially 100% scores at tree height 1.
  • Data and representations: The converted treebank removes null elements, aligns Penn Korean phrase structure with overt eojeol tokens, and preserves phrase labels where possible.The resulting representations use morphemes or eojeols as terminals with XPOS or UPOS preterminals.
  • Evaluation metrics: Normalized evaluation compares all systems in Eojeol+UPOS by converting Morpheme+XPOS and Eojeol+XPOS predictions before scoring.Projection can slightly increase labeled F1, so the normalized score measures recovery of eojeol-level structure rather than improvement from projection itself.
  • Overall results: 84.76 F1 with EVALB and 82.17 F1 with jp-evalb are achieved by bottom-up Morpheme+XPOS after conversion to Eojeol+UPOS.This is the strongest normalized result reported in the supplied passages.
  • Overall results: Eojeol+XPOS improves strongly over Eojeol+UPOS, especially for in-order and bottom-up parsing.The comparison indicates that fine-grained Korean XPOS information is more useful than coarse UPOS labels under Eojeol-level evaluation.

6 Analysis

The analysis compares transition lengths and normalized constituency accuracy across representations, transition orders, constituent arities, and tree heights. Morpheme-based and XPOS-rich representations recover stronger eojeol-level structure than native Eojeol+UPOS, while bottom-up parsing is generally strongest.

  • Transition statistics: Bottom-up parsing yields much shorter derivations than top-down and in-order parsing because reductions directly construct completed constituents.Top-down and in-order require one shift per terminal plus separate constituent-opening and completion actions.
  • Transition statistics: Morpheme+XPOS requires more transitions than Eojeol+XPOS and Eojeol+UPOS, whose counts are identical because they share eojeol terminals.The two eojeol representations differ only in their preterminal label inventories.
  • Arity-specific results: Morpheme-based systems recover higher-quality eojeol-level structure than native Eojeol+UPOS, especially for binary and higher-arity constituents.Eojeol+XPOS substantially improves over Eojeol+UPOS, showing that fine-grained Korean XPOS information remains useful after unified evaluation.
  • Arity-specific results: Bottom-up parsing gives the highest F1 for unary, binary, ternary, four-child, and high-arity constituents after morpheme-to-eojeol conversion.In-order parsing is generally close to bottom-up, whereas top-down parsing is weaker across most arities.
  • Tree-height results: At height 2 under top-down parsing, Morpheme+XPOS reaches 91.09 F1, compared with 86.36 for Eojeol+XPOS and 78.69 for native Eojeol+UPOS.The representation effect is largest immediately above the preterminal layer, and the morpheme-based advantage persists at higher tree heights.
  • Tree-height results: Richer morphosyntactic information improves both local phrase-structure decisions and the recovery of larger constituents.The arity- and height-based analyses identify morpheme-level information as useful evidence for recovering eojeol-level phrase structure.

7 Discussion

The discussion separates the empirical value of fine-grained morphosyntactic information from the independent design choice of eojeol constituency terminals. It argues for eojeol-based output with aligned morpheme and XPOS annotation, while identifying conversion, gold-alignment, and resource-scope boundaries.

  • Empirical conclusions: Under gold segmentation and preterminal annotation, Morpheme+XPOS and Eojeol+XPOS recover more eojeol-level constituency structure than Eojeol+UPOS.This establishes the value of fine-grained morphosyntactic information under the evaluated parsing conditions.
  • Terminal-domain choice: The parsing results do not independently determine whether morphemes or eojeols are the linguistically preferable constituency terminals.The argument for eojeol instead rests on linguistic and representational considerations.
  • Terminal-domain choice: Eojeols are defended as a consistent, linguistically transparent terminal domain while morpheme-level information and XPOS labels remain aligned annotation.This separates phrase-level grouping from eojeol-internal morphology and preserves morphosyntactic evidence for parsing.
  • Resource design: An eojeol-based representation gives constituency trees a direct surface interpretation and aligns them with Korean dependency analyses.Shared terminals allow constituency and dependency analyses to be compared without reconstructing word boundaries from morpheme sequences.
  • Empirical interpretation: Eojeol+XPOS substantially improves over Eojeol+UPOS despite identical eojeol terminal yields, isolating the contribution of fine-grained XPOS information.The comparison shows that the performance difference is not attributable only to terminal-sequence length.
  • Transition systems: Bottom-up parsing generally performs best, in-order parsing is often close, and top-down parsing is weaker across Korean and corresponding English and Chinese experiments.The reported ordering is therefore not specific to Korean.
  • Limitations: The study is limited to one Penn Korean Treebank conversion and its specific choices for null deletion, labeling, punctuation, UPOS mapping, and projection.The converted task is surface constituency parsing rather than recovery of the original empty-category analysis, and future work should test alternative targets.
  • Limitations: Because evaluation uses gold eojeol boundaries and fixed gold morpheme-to-eojeol alignment, it does not test boundary recovery, morphological alignment, or end-to-end predicted annotation.The experiments motivate but do not instantiate an eojeol-output parser using aligned morpheme sequences.

8 Conclusion

The paper supports an eojeol-based surface constituency representation while retaining morpheme sequences and fine-grained POS tags as aligned evidence. Its experiments show that terminal granularity and transition-system choice substantially affect Korean constituency parsing.

  • The resulting trees define constituency over surface eojeol sequences while preserving morpheme sequences and fine-grained Korean POS tags as aligned annotation.
  • The experiments show that terminal granularity substantially affects Korean constituency parsing, and native and converted scores are not interchangeable because representations define different span spaces.
  • Morpheme+XPOS produces the strongest results after projection to the eojeol domain, followed by Eojeol+XPOS, under gold segmentation and preterminal conditions.
  • Richer morphological or XPOS information provides valuable evidence for the evaluated transition-based parsers, but this finding does not determine the linguistically preferable terminal domain.
  • The paper’s eojeol preference follows linguistic and resource-design considerations: eojeols provide overt surface boundaries and a shared alignment domain with other Korean resources.Morpheme-level and XPOS information remain available as aligned morphosyntactic evidence.
  • Bottom-up and in-order parsing generally outperform top-down parsing, especially with rich morphological information, and the gains extend beyond binary constituents.The arity-based analysis covers ternary, four-child, and higher-arity structures.
  • Direct non-binary transition systems support Korean constituency parsing without artificial binarization, including naturally represented nominal, modifier, adjunct, and clausal structures.
  • These results provide a conservative foundation for surface-oriented Korean constituency parsing, treebank conversion, and constituency–dependency interoperability.
Loading 2608.27035v1…