PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification

Zhi-Xiu Ye, Zhen-Hua Ling

arXiv:1906.06678v1cs.CL

TL;DR

Few-shot relation classification must address relations with very few training samples, while prototypical networks encode queries and support sets independently. MLMAN matches them interactively at local and instance levels, aggregates support instances into query-weighted class prototypes, and achieves state-of-the-art performance on FewRel.

  • Problem

    Few-shot relation classification must handle relations with very few training samples, while prototypical networks represent query instances and support sets independently.

    Source: §1 · pp. 1–2

  • Method

    MLMAN interactively encodes queries and support sets through multi-level matching, then uses query-based attention to aggregate support representations into class prototypes.

    Source: §1 · p. 2, §6 · p. 8

  • Results

    MLMAN achieves state-of-the-art performance on the FewRel dataset.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The authors conclude that MLMAN is effective for few-shot relation classification on FewRel.

    Source: §6 · p. 9

    The few-shot task is constrained by the limited number of labeled support samples available for each relation class.

    Source: §3 · p. 3

Abstract

from arXiv · show

This paper presents a multi-level matching and aggregation network (MLMAN) for few-shot relation classification. Previous studies on this topic adopt prototypical networks, which calculate the embedding vector of a query instance and the prototype vector of each support set independently. In contrast, our proposed MLMAN model encodes the query instance and each support set in an interactive way by considering their matching information at both local and instance levels. The final class prototype for each support set is obtained by attentive aggregation over the representations of its support instances, where the weights are calculated using the query instance. Experimental results demonstrate the effectiveness of our proposed methods, which achieve a new state-of-the-art performance on the FewRel dataset.

1 Introduction

MLMAN addresses few-shot relation classification with interactive query–support matching and query-weighted aggregation, and the paper reports new state-of-the-art performance on FewRel.

1 Introduction

  • Few-shot relation classification addresses the long-tail problem by providing only a few support instances, such as one or five, for each relation.Relation classification identifies the semantic relation between two entities in text.Source: §1 · p. 1
  • Prototypical networks independently encode queries and support instances, average support embeddings into class prototypes, and classify by distances; they have also been applied to few-shot relation classification (Snell et al., 2017; Han et al., 2018).Source: §1 · p. 2
  • MLMAN interactively encodes queries and support sets through local- and instance-level matching, then aggregates support representations into class prototypes.Local representations are softly matched and pooled; an MLP computes query-to-support-instance matching degrees used as aggregation weights, and the layers are jointly estimated.Source: §1 · p. 2
  • An auxiliary loss measures inconsistency among support representations within each class.It is designed to encourage support-instance representations in each class to be close.Source: §1 · p. 2
  • MLMAN achieves new state-of-the-art performance on the FewRel few-shot relation classification dataset.The introduction reports this as a contribution but provides no quantitative comparison here.Source: §1 · p. 2

2 Related Work

Related work moves from relation classifiers that need substantial training data to metric-based few-shot learning, motivating MLMAN’s interactive query-to-support-set matching.

2.1 Relation Classification

  • Neural relation classifiers used position features, CNNs, or attention-based networks, but required substantial training data and could not quickly adapt to unseen classes.Examples include Zeng et al. (2014), Wang et al. (2016), and Zhou et al. (2016).Source: §2.1 · p. 2

2.2 Metric Based Few-Shot Learning

  • For few-shot relation classification, Han et al. (2018) used prototypical baselines and Gao et al. (2019) added hybrid attention for noisy samples; MLMAN interactively encodes queries and class prototypes.Metric-based methods classify support and query samples through learned projection functions; prototypical networks use the mean of a class’s support representations (Snell et al., 2017).Source: §2.2 · pp. 2–3

2.3 Sentence Matching

3 Task Definition

Few-shot relation classification evaluates a model on query sentences drawn from N relation classes, given only K labeled support examples per class. Training mimics this support-query setup on meta-training relations, which are disjoint from the meta-test relations.

3 Task Definition

  • A sample pairs a sentence and two entity positions with a relation label; meta-train and meta-test use disjoint relation-label spaces.An N-way-K-shot task has K labeled support samples for each of N classes and evaluates on query samples from those classes; the limited N × K examples make training from scratch difficult.Source: §3 · p. 3
  • Following Vinyals et al. (2016), training mimics few-shot test conditions by sampling N classes and K support instances per class, then drawing query instances from the remaining samples.The meta-training and meta-test relation-label spaces are disjoint, but meta-training episodes are used to help classification on meta-test relations.Source: §3 · p. 3

4 Methodology

MLMAN models query–support relationships interactively through contextual encoding, local and instance-level matching and aggregation, class-level scoring, and inconsistency-aware joint training.

4.2 Local Matching and Aggregation

  • Attention matches query and support contexts locally, after which fused representations pass through a bidirectional LSTM and are pooled into instance embeddings.The local aggregation concatenates max- and average-pooling results.Source: §4 · p. 4, §4.2 · p. 5

4.4 Class Matching

  • An MLP computes the class-level matching score between the query representation and the class prototype.The experiments use the same function for instance-level and class-level matching.Source: §4 · p. 4, §4.4 · p. 5

4.1 Context Encoder

  • CNNs encode each query and support sentence using word embeddings and entity-relative position features to produce word-level context representations.Each word representation concatenates its word embedding with two position-feature vectors.Source: §4.1 · p. 4

4.3 Instance Matching and Aggregation

  • Query-conditioned attention weights support-instance representations by their matching scores to form each class prototype, rather than taking a simple mean.The instance-level matching degree between the query and each support instance supplies the aggregation weight.Source: §4.3 · p. 5

4.4 Class Matching

  • Sharing the same matching function for instance-level and class-level comparisons led to better performance in the experiments.The model shares weights W2 and v across the two matching functions in each training iteration.Source: §4.4 · p. 5

4.5 Joint Training with Inconsistency Measurement

  • The joint objective adds a support-set inconsistency measure based on average Euclidean distance from support instances to their class prototype.Its hyperparameter λ was set to 1 without tuning in the experiments.Source: §4.5 · p. 5

5 Experiments · 5.1 Dataset and Evaluation Metrics

Experiments use FewRel, a crowdsourced-filtered dataset of 100 relations, and evaluate four few-shot configurations with results averaged over 10 training repetitions and tested on 20,000 independent samples.

5.1 Dataset and Evaluation Metrics

  • The evaluation covers 5-way 1-shot, 5-way 5-shot, 10-way 1-shot, and 10-way 5-shot, matching Han et al. (2018).Results are reported as the mean and standard deviation across 10 training repetitions and tested using 20,000 independent samples.Source: §5.1 · p. 6
  • FewRel contains 100 relations with 700 instances each, split into 64 training, 16 validation, and 20 test relations.The dataset was generated by distant supervision and filtered through crowdsourcing to remove noisy annotations.Source: §5.1 · p. 6
  • FewRel sentences average 24.99 tokens, and the dataset contains 124,577 unique tokens.Source: §5.1 · p. 6

5.2 Training Details and Hyperparameters

The section specifies the model’s training setup, embedding treatment, hyperparameter search, and optimization settings.

5.2 Training Details and Hyperparameters

  • Training used N = 20 support instances to construct train-support sets for 5-way and 10-way tasks, based on prior findings about harder training tasks.The authors cite Munkhdalai and Yu (2017) as motivation for this choice.Source: §5.2 · p. 6
  • The context encoder used fixed 50-dimensional GloVe embeddings; unknown words mapped to a fixed zero-vector <UNK> token.The embeddings are attributed to Pennington et al. (2014) 4.Source: §5.2 · p. 6
  • Grid searches covered dc ∈ {100, 150, 200, 250}, dh ∈ {100, 150, 200, 250}, and R ∈ {5, 10, 15}.Optimization used mini-batch SGD with initial learning rate 0.1, decayed to one tenth every 20,000 steps; dropout of 0.2 preceded CNN and LSTM layers.Source: §5.2 · p. 6

5.3 Comparison with Previous Work

On the FewRel test set, MLMAN outperforms the compared models by a large margin, which the paper attributes to interactions between query instances and support sets at multiple levels.

5.3 Comparison with Previous Work

5.4 Ablation Study

Ablations on FewRel’s development set assess individual model components. Models 2–7 matched the complete model on 5-way and 10-way 1-shot tasks.

5.4.1 Instance Matching and Aggregation

  • On 5-shot tasks, query-conditioned attentive aggregation outperformed max and average pooling significantly at the 1% t-test level.For one illustrated class A support set, the five instance weights were 0.03, 0.46, 0.25, 0.08, and 0.18; the most query-similar instance received the highest weight.Source: §5.4.1 · p. 7
  • Untying the instance- and class-level matching-score parameters made model 3 perform much worse than the complete model, supporting parameter sharing.Source: §5.4.1 · p. 7

5.4.2 Inconsistency Measurement

  • Removing Jincon worsened performance and increased mean within-class support-instance distance from 0.0199 to 0.0346.These means were calculated over 20,000 support sets in 5-way 5-shot tasks; Jincon’s accuracy benefit was larger with attentive aggregation (model 5 versus 6) than without it (model 1 versus 2).Source: §5.4.2 · p. 7

5.4.3 Local Matching

  • Removing local matching significantly reduced few-shot classification performance, while concatenation helped suppress support instances with low query similarity.The attention matrix also illustrates matching local contexts, including head and tail entities, the phrases “son” and “daughter,” and the shared keyword “married.”Source: §5.4.3 · p. 8, Figure 2 · p. 8

5.4.4 Class Matching

  • With local matching, the learnable MLP outperformed Euclidean distance by a large margin; without local matching, the MLP performed worse than Euclidean distance.The authors suggest local matching creates complex query–support correlations better handled by an MLP than by Euclidean distance.Source: §5.4.4 · p. 8

6 Conclusions

MLMAN uses interactive multi-level matching and attention-based aggregation to classify few-shot relations, achieving state-of-the-art performance on FewRel; future work will study distant-supervision data and extend the model to zero-shot learning.

6 Conclusions

  • MLMAN interactively encodes query and support instances through local matching and aggregation, then attention-aggregates support instances into class prototypes and scores classes with a learnable MLP.An additional objective is designed to improve consistency among support-instance representations within each class.Source: §6 · pp. 8–9
  • MLMAN achieves state-of-the-art performance on FewRel, with future work planned on distant-supervision data and extending the model to zero-shot learning.The paper identifies both directions as future work.Source: §6 · p. 9
Loading 1906.06678v1…