Source-linked AI summary
Head-Driven Phrase Structure Grammar Parsing on Penn Treebank
Junru Zhou, Hai Zhao
TL;DR
The paper addresses how to combine constituent and dependency representations within HPSG. It formulates a simplified HPSG with two decoding methods, and reports state-of-the-art results across English and Chinese benchmark treebanks for both parsing tasks.
Problem
Constituent and dependency structures provide complementary syntactic information, motivating a unified HPSG representation that combines them.
Method
The paper formulates a simplified HPSG combining constituent and dependency structures and explores two parsing methods for its converted tree representations.
Results
The model achieves state-of-the-art results on PTB and CTB for both constituent and dependency parsing.
Takeaways & Limitations
Joint learning of constituent and dependency structures is reported as superior to separate learning, with joint score training providing further improvement.
Takeaways & Limitations
HPSG’s precise constraints make it difficult to broadly cover unseen real-world texts, and the simplified representation uses head words as HEAD values for convenience.
Abstract
from arXiv · showhide
Head-driven phrase structure grammar (HPSG) enjoys a uniform formalism representing rich contextual syntactic and even semantic meanings. This paper makes the first attempt to formulate a simplified HPSG by integrating constituent and dependency formal representations into head-driven phrase structure. Then two parsing algorithms are respectively proposed for two converted tree representations, division span and joint span. As HPSG encodes both constituent and dependency structure information, the proposed HPSG parsers may be regarded as a sort of joint decoder for both types of structures and thus are evaluated in terms of extracted or converted constituent and dependency parsing trees. Our parser achieves new state-of-the-art performance for both parsing tasks on Penn Treebank (PTB) and Chinese Penn Treebank, verifying the effectiveness of joint learning constituent and dependency structures. In details, we report 96.33 F1 of constituent parsing and 97.20\% UAS of dependency parsing on PTB.
1 Introduction
The paper combines constituent and dependency structures into a simplified HPSG representation and introduces parsing methods evaluated on both structures. Experiments report state-of-the-art results on PTB and CTB for constituent and dependency parsing.
- Constituent structure captures phrasal continuity, while dependency structure indicates relations among words; their close relation supports combining them in HPSG.
- The work explores two parsing methods for simplified HPSG trees containing both constituent and dependency information.
- The parser is evaluated against both annotated constituent trees and converted dependency trees, enabling comparison with existing parsers for each task.
- The paper formulates a simplified HPSG by combining constituent and dependency tree structures for richer syntactic representation.
- The model achieves state-of-the-art results on PTB and CTB for both constituent and dependency parsing.
2 Simplified HPSG on PTB
The paper simplifies HPSG by using dependency-derived head words alongside constituent categories, then converts the resulting trees into division-span or joint-span structures for parsing.
- 2.1 Tree Preprocessing: Simplified HPSG assigns each constituent its category and a head word shared with the dependency parent of its children.This operationalizes the HPSG Head Feature Principle while using head words as HEAD values.
- 2.1 Tree Preprocessing: PTB dependency conversion uses Stanford PTB-SD rules, with a special category # dividing phrases that contain multiple head words.The conversion leaves only 50 heads as errors in Penn Treebank.
- 2.2 Span Representations of HPSG: Division span structures encode head position by splitting a phrase around its head and prefixing categories with H to distinguish the sides.The head is identified through the category marked with H in the converted span structure.
- 2.2 Span Representations of HPSG: Joint span structures recursively combine constituent spans with dependency arcs between the heads of their child phrases.A joint span contains child-span categories and dependency relations, including the root arc for the complete tree.
- 2.2 Span Representations of HPSG: Both converted representations encode constituent and head information in span-like trees that can be processed by constituent-style parsers.The paper therefore uses standard span parsing machinery for the simplified HPSG structures.
3 Our Model
The model predicts simplified HPSG trees using token representations, self-attention encoding, span and dependency scoring, and CKY-style decoding. Joint span parsing combines constituent and dependency structures under a weighted objective.
- 3 Our Model: The model combines token representations, a self-attention encoder, scoring modules, and CKY-style decoding to predict simplified HPSG trees.Token representations concatenate character, word, and POS embeddings, while the encoder factors content and position information.
- 3.4 Decoder for Division Span HPSG: Span vectors concatenate forward and backward encoder differences, which feed one-layer networks producing scores for span categories.The span scorer uses the concatenated representation sij as input to generate a score vector.
- 3.4 Decoder for Division Span HPSG: The constituent decoder finds the highest-scoring tree with a CKY-style algorithm in O(n3) time and trains it with a hinge-loss margin objective.The loss compares the correct tree against alternatives using Hamming loss on category spans.
- 3.5 Decoder for Joint Span HPSG: Dependency scores use biaffine attention, and dependency training minimizes cross-entropy over the correct parent and dependency label.The biaffine score combines bilinear and linear terms from child and parent representations.
- 3.5 Decoder for Joint Span HPSG: The joint span decoder uses a CKY-style dynamic program to maximize a global score over span and dependency structures, then backtracks to construct the HPSG tree.Complete and incomplete spans support binarization, while dependency scores influence split-point selection.
- 3.5 Decoder for Joint Span HPSG: The weighting parameter λ controls the decoder’s structure: λ=1 yields constituent parsing, λ=0 yields dependency parsing, and intermediate values produce joint HPSG parsing.The joint parser has O(n5) time and O(n3) space complexity because it searches over sub-heads and split points.
4 Experiments
Experiments evaluate simplified HPSG parsing on English and Chinese treebanks, testing encoder depth, constituent–dependency balance, decoding speed, and benchmark performance. Joint prediction consistently outperforms separate learning and reaches state-of-the-art results.
- 4.1 Setup: 12 self-attention layers perform best on the English development set; adding more layers provides almost no improvement and can reduce accuracy.The remaining experiments therefore use 12 layers.
- 4.3 Moderating constituent and Dependency: λ=0.5 gives the best performance on both F1 and UAS, and joint decoding outperforms either constituent-only or dependency-only decoding.λ=0 and λ=1 correspond to dependency-only and constituent-only modes, respectively.
- 4.4 Joint Span HPSG Parsing: The Joint span decoder has O(n^5) complexity but is not much slower than the Division span decoder because neural-network computation dominates runtime.The decoder consumes only a small fraction of total training and inference time.
- 4.6 Main Results: 89.40 F1, 91.21% UAS, and 89.15% LAS are achieved by the best model on CTB, while PTB also reaches new state-of-the-art results.Results are reported under both CTB data splittings because constituent and dependency parsing use different splits.
- 4.6 Main Results: Joint HPSG parsing is more effective than separate constituent or dependency learning, with dependency parsing benefiting more from the Joint than Division decoder.The authors attribute this empirically to the dependency score in joint loss.
- 4.6 Main Results: 96.33 F1, 97.20% UAS, and 95.72% LAS are achieved by the Joint XLNet model on PTB, defeating other ensemble models.These results cover constituent and dependency parsing.
5 Related Work
Related work situates HPSG among lexicalized grammar frameworks, approximation methods, neural parsing, and joint constituent–dependency learning. The paper proposes a graph-based simplified HPSG to combine both syntactic representations.
- Background: HPSG is a constraint-based, highly lexicalized, non-derivational generative grammar framework, alongside earlier lexicalized approaches such as LTAG.The paper identifies HPSG as developed by Pollard and Sag.
- Limitations and Approximations: HPSG’s precise constraints make broad coverage of unseen real-world text difficult, forcing implementations to trade linguistic precision against parsing coverage.Prior approximation work includes grammar-based, corpus-driven, and PCFG approaches.
- Neural Parsing: Neural constituent and dependency parsers achieve state-of-the-art results, motivating research on their shared grammar and machine-learning characteristics.Prior work has explored joint learning of the two structures.
- Novelty: The paper proposes the first graph-based parsing model that formulates constituent and dependency structures as a simplified HPSG.This approach is presented as exploiting the strengths of both representation forms.
6 Conclusions
The paper presents simplified HPSG with two decoding methods evaluated on constituent and dependency parsing. Experiments report state-of-the-art results on English and Chinese benchmarks and favor joint learning over separate learning.
- 6 Conclusions: Simplified HPSG with two decoding methods achieves new state-of-the-art results on both parsing tasks across Chinese and English benchmark treebanks.The paper evaluates both constituent and dependency outputs.
- 6 Conclusions: Joint learning of constituent and dependency structures outperforms separate learning, while combining their scores during training further improves parsing.The conclusion frames the contribution as exploring the relation between the two structures as well as building a high-performance parser.