PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Multiple Instance Neural Networks

Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai, Wenyu Liu

arXiv:1610.02501v1stat.MLcs.LG

TL;DR

MIL must predict bag labels from weakly supervised, variable-sized collections of instances, making conventional neural-network inputs and instance-level supervision unsuitable. The paper proposes end-to-end networks centered on learned bag representations, studies deep supervision and residual connections, and evaluates them across MIL benchmarks. The networks achieve state-of-the-art or competitive benchmark performance while remaining fast to train and test.

  • Problem

    MIL predicts bag labels from weakly supervised bags with ambiguous instance labels and variable numbers of instances, which conventional neural networks are not designed to handle.

  • Method

    The paper proposes mi-Net and MI-Net, including a new embedded-space network that directly learns fixed-length bag representations, and studies deep supervision, residual connections, and MIL pooling.

  • Results

    The proposed networks achieve state-of-the-art or competitive performance on several MIL benchmarks, with deep supervision improving robustness and MI-Net performing strongly across benchmark datasets.

  • Takeaways & Limitations

    End-to-end neural networks can provide effective and efficient solutions for traditional MIL while directly learning bag-level representations.

Abstract

from arXiv · show

Recently neural networks and multiple instance learning are both attractive topics in Artificial Intelligence related research fields. Deep neural networks have achieved great success in supervised learning problems, and multiple instance learning as a typical weakly-supervised learning method is effective for many applications in computer vision, biometrics, nature language processing, etc. In this paper, we revisit the problem of solving multiple instance learning problems using neural networks. Neural networks are appealing for solving multiple instance learning problem. The multiple instance neural networks perform multiple instance learning in an end-to-end way, which take a bag with various number of instances as input and directly output bag label. All of the parameters in a multiple instance network are able to be optimized via back-propagation. We propose a new multiple instance neural network to learn bag representations, which is different from the existing multiple instance neural networks that focus on estimating instance label. In addition, recent tricks developed in deep learning have been studied in multiple instance networks, we find deep supervision is effective for boosting bag classification accuracy. In the experiments, the proposed multiple instance networks achieve state-of-the-art or competitive performance on several MIL benchmarks. Moreover, it is extremely fast for both testing and training, e.g., it takes only 0.0003 second to predict a bag and a few seconds to train on a MIL datasets on a moderate CPU.

I. INTRODUCTION

MIL learns bag labels from weakly supervised bags whose instances have ambiguous labels and variable composition. The paper develops end-to-end neural networks that process variable-sized bags, learn bag representations, and incorporate deep-learning techniques.

  • MIL setting: MIL trains bag classifiers from labeled bags, assuming a positive bag contains at least one positive instance while negative bags contain only negative instances.The central difficulty is ambiguity in instance labels, especially within positive bags containing diverse instances.
  • Motivation: Conventional neural networks are difficult to apply because MIL provides only bag labels, while bags contain varying numbers of instances rather than fixed-length inputs.These differences from fully labeled, fixed-input supervised learning motivate specialized multiple instance neural networks.
  • Existing and proposed networks: MINNs accept variable-sized bags and use multiple instance supervision to learn instance representations layer by layer.The paper distinguishes mi-Net, which includes instance classifiers, from MI-Net, which directly constructs a fixed-length bag representation.
  • Existing and proposed networks: MI-Net learns a bag representation before applying a bag classifier, unlike earlier networks that estimate instance probabilities.This embedded-space design is proposed as a new multiple instance neural network in the paper.
  • Network components: MIL pooling layers aggregate instance probability distributions or feature vectors into bag feature vectors using differentiable max, mean, or log-sum-exp operations.These layers connect variable-instance data with conventional neural-network components and are studied experimentally.
  • Training enhancements: Deep supervision is reported as the most effective among the studied deep-learning additions, while residual connections also improve the networks.The paper attributes deep supervision’s effectiveness to better use of hierarchical network features.

II. RELATED WORK

Prior MIL research includes neural networks, feature engineering, ensembles, scalable Fisher Vector coding, and weakly supervised computer-vision models. The paper positions MI-Net as a deep, embedded-space alternative focused on bag representation learning.

  • Neural MIL: Earlier neural MIL methods estimated instance probabilities and aggregated them with log-sum-exp or max operations to obtain bag probabilities.Subsequent work added feature selection with Diverse Density and PCA, or integrated ensemble methods.
  • Scalable embedded-space methods: miFV scales embedded-space MIL by transforming instance features into a high-dimensional Fisher Vector representation using a pretrained Gaussian mixture model.The paper contrasts this feature-coding approach with MI-Net’s learned instance features.
  • Weakly supervised vision: Computer-vision studies used max pooling to locate positive image patches or log-sum-exp pooling for weakly supervised semantic segmentation.The proposed mi-Net extends this neural MIL direction to more general MIL problems while adding deep supervision.
  • Scope: The paper’s related-work discussion frames its contribution as a series of neural-network frameworks for MIL rather than a new pooling method alone.Its stated focus is end-to-end MIL networks and associated experimental studies.

A. Notations

MIL learns bag labels from labeled bags containing variable numbers of instances, while instance labels remain unavailable during training. MINNs address this ambiguity either by inferring instance probabilities or by directly learning bag representations.

  • MIL formulation: MIL uses labeled bags of instances, with negative bags containing only negative instances and positive bags containing at least one positive instance.Only bag labels are provided during training, making instance labels latent in positive bags.
  • MINN strategies: MINNs also include a second strategy that learns bag representations directly and classifies bags without calculating instance probabilities.This embedded-space strategy is newly proposed in the paper.
  • MINN formulation: A MINN processes each instance through nonlinear layers and aggregates the resulting instance outputs into a bag-level representation or probability.The network applies layer transformations to instances and uses a MIL pooling operation across the bag.
  • mi-Net: mi-Net learns instance probabilities before a final MIL pooling layer produces the bag probability.The instance probability is a scalar in [0, 1], and the pooling layer aggregates these scores into the bag output.
  • mi-Net: Because instance labels are unavailable, mi-Net treats them as latent variables and infers them during network training through differentiable aggregation.The pooling function must satisfy MIL constraints while remaining differentiable for back-propagation.

C. MI-Net: A new Embedded-Space MIL Algorithm

The paper proposes MI-Net, an embedded-space MIL architecture that learns bag representations rather than inferring instance probabilities. A MIL pooling layer handles variable-sized bags before a final classifier predicts the bag probability.

  • MI-Net contribution: MI-Net avoids inferring instance probabilities by directly learning bag representations for classification.The proposed networks belong to the embedded-space MIL paradigm.
  • Plain MI-Net: A plain MI-Net uses three fully connected layers followed by one MIL pooling layer to convert any number of instance features into one bag feature vector.The pooled representation is passed to a one-neuron fully connected layer with sigmoid activation.

D. MI-Net with Deep Supervision

Deep supervision extends MI-Net by producing bag predictions from multiple intermediate feature levels, while residual connections adapt residual learning to bag representations. The architecture is designed to improve supervision across hierarchies and robustness through score averaging.

  • MI-Net with Deep Supervision: Deeply supervised MI-Net adds an instance-score layer and MIL pooling layer after each intermediate fully connected feature-learning layer.Training applies supervision at each level, and testing averages the scores from all levels.
  • Effect of Deep Supervision: Deep supervision uses multiple feature hierarchies: lower layers receive stronger training supervision, while averaged bag probabilities provide a more robust test label.The paper sets the weights of the different levels equally.
  • Architecture: Three fully connected layers of sizes 256, 128, and 64 learn instance features, with three MIL pooling layers generating bag features for separate bag classifiers.Each bag feature connects to a one-neuron fully connected layer for bag-label prediction.
  • Residual Connections: Residual MI-Net learns residuals of bag representations using fully connected layers, ReLU, and MIL pooling before final bag classification.This adapts residual learning from convolutional networks to the bag-representation setting.

F. MIL Pooling Methods

MIL pooling aggregates instance scores or feature vectors into a bag-level output, bridging variable-sized multiple-instance data with conventional neural networks. The paper compares max, mean, and log-sum-exp pooling, with log-sum-exp providing a tunable approximation between max and mean.

  • MIL pooling aggregates instance scores or feature vectors into a bag score or representation.It provides the interface between multiple-instance data and conventional neural networks.
  • The evaluated pooling methods are max pooling, mean pooling, and log-sum-exp pooling.
  • The log-sum-exp pooling parameter r controls smoothness, approaching max pooling for large r and mean pooling for small r.

G. Training Loss

Training uses cross-entropy loss on bag scores, applies the loss at each bag-score level for deep supervision, and optimizes the differentiable networks with SGD backpropagation.

  • Cross-entropy loss is used to train the networks to predict bag labels from bag scores.The loss is defined as Loss(S_i, Y_i) = −{(1 − Y_i) log(1 − S_i) + Y_i log S_i}.
  • The loss is added at each bag-score level to provide deep supervision.
  • Because all network components are differentiable, the models are trained with standard backpropagation and stochastic gradient descent.

B. Experimental Setup

The experiments evaluate multiple-instance networks on benchmark datasets using fully connected architectures, repeated cross-validation, and average prediction accuracy for bag classification.

  • The evaluated networks use four fully connected layers, with dropout applied after the first three layers at a 0.5 ratio.
  • The reported evaluation uses average prediction accuracy for bag classification on five MIL benchmarks.
  • Experiments run on a moderate CPU system with an Intel i7-4790K processor and 32GB RAM.

C. Experimental Results

The proposed networks achieve state-of-the-art or highly competitive MIL performance across benchmark tasks. MI-Net variants, especially with deep supervision, generally outperform competing methods, while pooling choice also affects results.

  • C. Experimental Results: The networks achieve state-of-the-art performance on Elephant, Fox, and text categorization, with nearly best accuracies on other datasets.
  • C. Experimental Results: MI-Net and its two variations outperform five competing algorithms on the average accuracy across all 20 text-categorization datasets.
  • C. Experimental Results: MI-Net with deep supervision achieves almost all best results on five benchmark datasets, while residual connections also obtain good results.
  • F. MIL Pooling Methods: Max, mean, and log-sum-exp pooling produce different results across datasets, with max pooling preferred in the reported comparison.
  • B. The Influence of Deep Supervision: Deep supervision boosts performance for all evaluated datasets and networks and is described as essential for learning good instance features.

C. The Influence of Residual Connections

Residual connections improve MI-Net performance on most of five MIL benchmarks, while depth and width provide limited and dataset-dependent benefits.

  • Residual connections: Residual connections improve MI-Net performance on five MIL benchmarks, except for Elephant and Tiger.The residual variant learns bag-representation residuals and generally outperforms MI-Net without residual connections.
  • Network depth and width: MI-Net with DS performs best in most cases at depth 4 with layer widths 256, 128, 64, and 1.The final fully connected layer is fixed at one neuron to output bag scores.
  • Network depth and width: Deeper and wider MI-Net variants sometimes outperform shallower and thinner networks, but the advantage is not obvious.For MI-Net with RC, performance is similar across depths, wider networks are unnecessary, and overly thin networks may perform worse.

VI. CONCLUSION

The paper proposes neural-network frameworks for multiple instance learning that emphasize bag-level representation learning and integrate deep-learning techniques. These methods perform competitively on MIL benchmarks while offering very fast per-bag computation.

  • Conclusion: The proposed frameworks learn bag-level representations rather than estimating instance-level labels.The paper distinguishes its approach from previous MIL networks by focusing on bag representation learning.
  • Conclusion: Deep supervision and residual connections further boost the performance of the proposed MIL networks.
  • Conclusion: Testing takes about 0.0003 second per bag, while training takes about 0.0008 second per bag.These timings are reported for forward and backward passes, respectively.
Loading 1610.02501v1…