Source-linked AI summary
DPGN: Distribution Propagation Graph Network for Few-shot Learning
Ling Yang, Liangliang Li, Zilun Zhang, Xinyu Zhou, Erjin Zhou, Yu Liu
TL;DR
Few-shot learning requires predicting unlabeled queries from only a few labeled examples, while prior graph methods primarily model instance-level relations. DPGN combines explicit distribution-level and instance-level relations in dual complete graphs to propagate labels, reporting 5%∼12% supervised and 7%∼13% semi-supervised improvements on benchmarks.
Problem
Few-shot learning must predict unlabeled query data from a few labeled support examples, while prior graph approaches mainly model pairwise relations and overlook distribution relations.
Method
DPGN uses point and distribution graphs to represent instance-level and 1-vs-N distribution-level relations, jointly updating them through label propagation and distribution losses.
Results
5%∼12% improvement is reported for supervised few-shot classification and 7%∼13% for semi-supervised tasks on few-shot learning benchmarks.
Takeaways & Limitations
Explicit distribution propagation complements instance-level modeling and supports label propagation between labeled and unlabeled examples in few-shot tasks.
Abstract
from arXiv · showhide
Most graph-network-based meta-learning approaches model instance-level relation of examples. We extend this idea further to explicitly model the distribution-level relation of one example to all other examples in a 1-vs-N manner. We propose a novel approach named distribution propagation graph network (DPGN) for few-shot learning. It conveys both the distribution-level relations and instance-level relations in each few-shot learning task. To combine the distribution-level relations and instance-level relations for all examples, we construct a dual complete graph network which consists of a point graph and a distribution graph with each node standing for an example. Equipped with dual graph architecture, DPGN propagates label information from labeled examples to unlabeled examples within several update generations. In extensive experiments on few-shot learning benchmarks, DPGN outperforms state-of-the-art results by a large margin in 5% $\sim$ 12% under supervised setting and 7% $\sim$ 13% under semi-supervised setting. Code will be released.
1. Introduction
Few-shot learning addresses prediction from a few labeled support examples, while prior graph-based approaches mainly model pairwise relations and overlook broader distribution relations. DPGN introduces dual graph propagation to combine instance- and distribution-level information, with reported gains across supervised and semi-supervised benchmarks.
- Few-shot learning predicts unlabeled query data from a few labeled support examples.
- Prior few-shot graph methods mainly model pairwise node or edge relations and ignore substantial distribution relations.
- The point graph generates distribution features by gathering each example’s relations to all support samples, while distribution information enhances instance features.
- DPGN explicitly incorporates distribution propagation and combines distribution-level relations with instance-level relations through a dual complete graph.
- 5%∼12% average accuracy improvement is reported in supervised few-shot classification, while semi-supervised tasks show 7%∼13% improvement over existing graph-based methods.
2. Related Work
Related few-shot methods use graph propagation, metric learning, distribution learning, or meta-learning to exploit structure from limited examples. DPGN is presented against these approaches as a dual-graph method for propagating instance and distribution information.
- Graph-based few-shot methods propagate labels or update node and edge features using pairwise sample similarities.
- Metric-learning approaches classify few-shot queries through embedding distances, class prototypes, or learned pointwise relations.
- Distribution-learning research estimates sample-generating distributions, while some classification methods assign distributions rather than one-hot labels.
- Meta-learning methods learn knowledge across task batches, including gradient-based parameter updates and episodic adaptation.
- DPGN uses a dual complete graph consisting of point and distribution graphs for transductive propagation across repeated generations.
3. Method
DPGN combines instance-level and distribution-level relations through dual complete graphs that exchange information across repeated generations. It initializes point features and derives distribution features from instance similarities, then propagates distribution information back to refine point representations and predict labels.
- 3.2. Distribution Propagation Graph Networks: DPGN uses a point graph and a distribution graph to combine instance-level and distribution-level relations for every example.The point graph represents instance relations, while the distribution graph represents 1-vs-N relations derived from them.
- 3.2.1 Point-to-Distribution Aggregation: Feature embeddings are extracted for all support and query samples, and their instance similarities initialize the point graph.The initial point-node features come from the embedding extractor, while point-graph edges encode instance similarity.
- 3.2.1 Point-to-Distribution Aggregation: P2D aggregation constructs distribution features by integrating point-graph edge information and updating concatenated features with a fully connected layer and ReLU.The distribution graph is built after point-graph edges are produced or updated, and its node features aggregate instance relations.
- 3.2.1 Point-to-Distribution Aggregation: Each distribution feature is an NK-dimensional vector whose j-th entry represents the relation between sample x_i and sample x_j.NK is the total number of support samples in the task.
- 3.2.2 Distribution-to-Point Aggregation: D2P aggregation sends encoded distribution information back into the point graph so node features incorporate distribution-level information before the next generation.The cyclic update alternates distribution processing and point-graph refinement across generations.
- 3.3. Objective: DPGN predicts each node’s class from final-generation point-graph edges and trains with weighted point and distribution losses.The point loss uses cross-entropy on sample predictions, while the distribution loss supports learning discriminative distribution features and faster, better convergence.
4. Experiments
Experiments across four few-shot benchmarks evaluate DPGN against existing methods, including semi-supervised, transductive, high-way, and ablation settings. Results consistently support the value of combining point-level and distribution-level relations.
- Main Results: DPGN achieves state-of-the-art performance across miniImageNet, tieredImageNet, CIFAR-FS, and CUB-200-2011, especially against graph-based methods.The evaluation compares graph and non-graph methods using matched backbones.
- Semi-supervised Few-shot Learning: 11%∼16% and 7%∼13% gains over TPN and EGNN, respectively, are reported in semi-supervised miniImageNet classification.These results concern few-shot average classification accuracy as the label ratio decreases.
- Transductive Propagation: DPGN’s transductive accuracy increases by a large margin over its non-transductive setting.Unlike TPN and EGNN, DPGN uses distribution similarities between query samples in a dual graph architecture.
- High-way classification: DPGN surpasses graph-based and non-graph methods in high-way miniImageNet classification as the number of ways increases.More ways provide more distribution-level information for queries, according to the reported interpretation.
- Impact of Distribution Graph: Keeping distribution-edge dimensions from 0 to 5 boosts 5way-1shot miniImageNet performance by nearly 10% absolute.Accuracy increases with retained dimensions, while the incremental gain decreases as more dimensions are kept.
- Generation Numbers: Six update generations are selected as a trade-off between test accuracy and convergence time, while final heatmaps show correct predictions for five queries.The cyclic updates refine instance-level similarities and enlarge distances between samples from different classes.
5. Conclusion
DPGN is presented as a dual complete graph network that explicitly combines instance-level and distribution-level relations for few-shot learning. Experiments report improvements over recent state-of-the-art methods in both supervised and semi-supervised settings.
- DPGN explicitly combines instance-level and distribution-level relations in a dual complete graph network for few-shot learning.The architecture uses point and distribution graphs with label propagation and transduction.
- 5%∼12% improvement is reported over recent state-of-the-art algorithms in the supervised few-shot learning task.
- 7%∼13% improvement is reported over recent state-of-the-art algorithms in the semi-supervised few-shot learning task.