PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Scale Dense Networks for Resource Efficient Image Classification

Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, Kilian Q. Weinberger

arXiv:1703.09844v5cs.LG

TL;DR

The paper addresses image classification when computation is limited or variable at test time. It introduces MSDNet, a dense multi-scale CNN with early exits, and reports strong accuracy across anytime and budgeted batch settings. The framework adapts computation to example difficulty using one model.

  • Problem

    The paper asks how image classifiers can adapt computation to easy and difficult inputs under anytime and shared batch-test budgets.

  • Method

    MSDNet combines intermediate early-exit classifiers, dense connectivity, and multi-scale features maintained throughout one convolutional network.

  • Results

    Across three image-classification datasets, MSDNet maintains high accuracy throughout anytime evaluation and performs on par with state-of-the-art CNNs while drastically reducing computation in budgeted batch classification.

  • Takeaways & Limitations

    A single MSDNet can dynamically adapt test-time computation to example difficulty while providing competitive classification results across both resource-constrained settings.

  • Takeaways & Limitations

    The paper identifies extending resource-aware architectures beyond object classification and combining MSDNets with further efficiency methods as future work.

Abstract

from arXiv · show

In this paper we investigate image classification with computational resource limits at test time. Two such settings are: 1. anytime classification, where the network's prediction for a test example is progressively updated, facilitating the output of a prediction at any time; and 2. budgeted batch classification, where a fixed amount of computation is available to classify a set of examples that can be spent unevenly across "easier" and "harder" inputs. In contrast to most prior work, such as the popular Viola and Jones algorithm, our approach is based on convolutional neural networks. We train multiple classifiers with varying resource demands, which we adaptively apply during test time. To maximally re-use computation between the classifiers, we incorporate them as early-exits into a single deep convolutional neural network and inter-connect them with dense connectivity. To facilitate high quality classification early on, we use a two-dimensional multi-scale network architecture that maintains coarse and fine level features all-throughout the network. Experiments on three image-classification tasks demonstrate that our framework substantially improves the existing state-of-the-art in both settings.

1 INTRODUCTION

The paper targets test-time resource constraints by adapting computation to image difficulty, using MSDNet’s dense early exits and multi-scale features to support flexible classification.

  • 1 INTRODUCTION: The paper asks how CNNs can avoid wasting computation on easy images while retaining accuracy on difficult examples.It focuses on automatically using smaller networks for easy inputs and larger networks for hard inputs or greater computational availability.
  • 1 INTRODUCTION: It studies anytime prediction and budgeted batch classification as two settings where computation is constrained at test time.Anytime systems output predictions whenever required, while batch systems distribute a fixed budget unevenly across easy and hard examples.
  • 1 INTRODUCTION: MSDNet combines dense connectivity with multi-scale feature maps and intermediate classifiers to enable resource-efficient image classification.Dense connectivity reduces interference among classifiers, while maintaining coarse and fine features supports accurate early exits.
  • 1 INTRODUCTION: Across three image-classification datasets, MSDNets provide predictions throughout computation and match state-of-the-art accuracy while drastically reducing computational requirements.The authors report dynamic resource adaptation with a single model and competitive results in both evaluation settings.

2 RELATED WORK

Prior work reduces computation mainly through model compression or resource-aware traditional models, while related neural architectures provide ideas for multi-scale feature construction.

  • 2 RELATED WORK: Most computation-efficient CNN research reduces inference cost by pruning or quantizing weights after or during training.These methods exploit redundant weights while aiming to preserve or improve performance.
  • 2 RELATED WORK: Resource-efficient machine-learning studies incorporate feature-computation costs into training, often for decision trees and related traditional models.The paper is inspired by this line of work but explores a different neural-network architecture.
  • 2 RELATED WORK: MSDNet borrows multi-scale construction ideas from neural fabrics and related architectures while using fewer scales and omitting sparse channel connectivity and up-sampling paths.The supplied comparison states that MSDNets are at least one order of magnitude more efficient and typically more accurate than neural fabrics.

3 PROBLEM SETUP

The paper formulates two test-time resource-constrained classification problems: prediction under a per-example variable budget and batch classification under a shared fixed budget.

  • 3 PROBLEM SETUP: Together, the settings impose computational constraints at prediction time while allowing computation to be allocated unevenly across examples.This shared framing distinguishes per-example anytime budgets from a soft cumulative batch constraint.
  • 3 PROBLEM SETUP: Anytime prediction assigns each test example a finite, nondeterministic computational budget and minimizes expected loss under the joint distribution of inputs and budgets.The budget may depend on events requiring an immediate prediction and is modeled through P(x, B).
  • 3 PROBLEM SETUP: Budgeted batch classification classifies a test set within a known cumulative budget while minimizing loss across all examples.The formulation permits spending less computation on easy examples and more on difficult ones.

4 MULTI-SCALE DENSE CONVOLUTIONAL NETWORKS

MSDNets combine multi-scale feature maps, dense connectivity, and intermediate classifiers to support early exits while reusing computation. The architecture and evaluation procedure adapt computation to resource budgets and input difficulty.

  • Architecture: MSDNets use cascaded classifiers with dense connectivity so features computed for earlier exits can be reused by later classifiers.Dense connections also reduce interference between intermediate classifiers and the final classifier.
  • Architecture: Multi-scale feature maps preserve coarse-level features throughout the network, enabling intermediate classifiers to make predictions earlier.Classifiers use the coarsest-scale features, while the network maintains finer-scale representations for later computation.
  • Test-time computation: At test time, anytime classification returns the most recent prediction when computation stops, whereas batch classification exits when confidence exceeds classifier-specific thresholds.The batch procedure uses the maximum softmax probability as its confidence measure.
  • Test-time computation: Exit probabilities and classifier costs determine thresholds that approximately satisfy the expected batch-computation budget.Validation data is used to set thresholds so the desired fraction of samples exits at each classifier.
  • Efficiency improvements: Network reduction removes finer scales across depth, while lazy evaluation propagates examples only along computation paths needed for the next classifier.These strategies reduce both training and testing computation and avoid unnecessary work when evaluation stops early.

5 EXPERIMENTS

Experiments evaluate MSDNets for anytime and budgeted batch classification, showing strong accuracy-efficiency trade-offs across ImageNet and CIFAR-100. Additional analyses examine dynamic evaluation, early exits, and a more efficient DenseNet variant.

  • 5.1 ANYTIME PREDICTION: For anytime prediction on ImageNet, MSDNet substantially outperforms ResNet and DenseNet ensembles at every evaluated budget, with ∼4%−8% higher accuracy from 0.1×10^10 to 0.3×10^10 FLOPs.The comparison is shown as top-1 accuracy versus computational budget in Figure 5.
  • 5.2 BUDGETED BATCH CLASSIFICATION: Budgeted batch evaluation propagates easy examples only through early classifiers while sending hard examples through the full network, adapting computation to example difficulty.The procedure is evaluated on batches of M =256 images for CIFAR-100.
  • 5.2 BUDGETED BATCH CLASSIFICATION: At an average budget of 1.7×10^9 FLOPs on ImageNet, MSDNet reaches ∼75% top-1 accuracy, ∼6% above an equally expensive ResNet and using ∼2−3× fewer FLOPs than DenseNet for the same accuracy.Dynamic evaluation also permits precise computational-budget tuning that individual ResNet and DenseNet models do not provide.
  • 5.2 BUDGETED BATCH CLASSIFICATION: On CIFAR-100 budgeted batch classification, MSDNet consistently outperforms all baselines, matching a 110layer ResNet with 1/10th the computation and reaching up to ∼5 times DenseNet efficiency.It also substantially outperforms ResNetMC and DenseNetMC models with multiple intermediate classifiers.
  • 5.2 BUDGETED BATCH CLASSIFICATION: Figure 6 suggests that early classifiers correctly recognize prototypical ImageNet examples, whereas the last classifier handles non-typical images that fail early prediction.The visualization contrasts examples exited correctly at the first classifier with examples corrected at the last layer.
  • 5.3 MORE COMPUTATIONALLY EFFICIENT DENSENETS: Doubling DenseNet growth rates after transition layers creates DenseNet*, which is more computationally efficient than DenseNet but remains substantially worse than MSDNet in both test-time budget settings.DenseNet* improves over the original DenseNet under anytime and budgeted batch evaluation.

6 CONCLUSION

MSDNet is designed for test-time CPU budgets by maintaining coarse and fine features while connecting layers densely. The authors report efficiency across budgets and identify segmentation and further efficiency techniques as future directions.

  • 6 CONCLUSION: MSDNet maintains features at multiple resolutions throughout a densely connected network, enabling intermediate classifiers across computation depths.The architecture decouples depth from feature coarseness and supports early exits without classifiers interfering with one another.
  • 6 CONCLUSION: A single MSDNet can outperform competitive baselines across computational budgets from highly limited CPU settings to almost unconstrained settings.
  • 6 CONCLUSION: Future work extends resource-aware architectures beyond object classification, including image segmentation, and combines them with compression and more efficient computation.

A DETAILS OF MSDNET ARCHITECTURE AND BASELINE NETWORKS

The appendix specifies reduced multi-scale architectures for CIFAR and ImageNet, with dense convolutional feature layers and intermediate classifiers. Baseline early-exit networks are also described for comparison.

  • A DETAILS OF MSDNET ARCHITECTURE AND BASELINE NETWORKS: CIFAR MSDNets use three scales with 6, 12, and 24 output channels, while each classifier uses down-sampling convolutions, average pooling, and a linear layer.The feature transformations follow a Conv(1 × 1)-BN-ReLU-Conv(3 × 3)-BN-ReLU design.
  • A DETAILS OF MSDNET ARCHITECTURE AND BASELINE NETWORKS: The ImageNet MSDNet uses four scales producing 16, 32, 64, and 64 feature maps per layer, with network reduction applied to lower computation.Input preprocessing begins with a 7×7 convolution and 3×3 max pooling, both using stride 2.
  • A DETAILS OF MSDNET ARCHITECTURE AND BASELINE NETWORKS: Reduced networks use transition layers between blocks to progressively decrease the number of maintained scales.The illustrated anytime architecture has three scales and three blocks.
  • A DETAILS OF MSDNET ARCHITECTURE AND BASELINE NETWORKS: Budgeted-batch CIFAR MSDNets range from 10 to 36 layers, with classifiers attached at cumulative layer positions in the reduced network.
  • A DETAILS OF MSDNET ARCHITECTURE AND BASELINE NETWORKS: ResNetMC and DenseNetMC use six intermediate classifiers attached at fixed depths within residual blocks or dense blocks.

B.1 ABLATION STUDY

The ablation study examines how multi-scale feature maps, dense connectivity, and intermediate classifiers contribute to MSDNet performance on CIFAR-100.

  • B.1 ABLATION STUDY: The CIFAR-100 ablation isolates the effects of multi-scale feature maps, dense connectivity, and intermediate classifiers.

B.2 RESULTS ON CIFAR-10

On CIFAR-10, MSDNets are evaluated in anytime and budgeted-batch settings against the same baselines used for CIFAR-100. They outperform all baselines by a significant margin in both settings.

  • B.2 RESULTS ON CIFAR-10: MSDNets outperform all baselines by a significant margin in both anytime prediction and budgeted batch classification on CIFAR-10.The results are shown in the left and right panels of Figure 11, respectively.
  • B.2 RESULTS ON CIFAR-10: ResNet and DenseNet models with multiple intermediate classifiers perform relatively poorly in the CIFAR-10 experiments.
Loading 1703.09844v5…