Source-linked AI summary
Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization
Ruyi Ji, Longyin Wen, Libo Zhang, Dawei Du, Yanjun Wu, Chen Zhao, Xianglong Liu, Feiyue Huang
TL;DR
FGVC is difficult because subordinate categories have high intra-class and low inter-class visual variation, while single CNNs may miss their subtle differences. ACNet addresses this with a convolutional binary neural tree whose routed branches use attention to learn coarse-to-fine representations. Experiments on CUB-200-2011, Stanford Cars, and Aircraft report favorable performance against state-of-the-art methods.
Problem
FGVC must distinguish subordinate categories despite high intra-class and low inter-class visual variation from deformation, occlusion, and illumination, and single CNNs struggle with these subtle differences.
Method
ACNet combines convolutional operations along complete binary-tree edges, node routing functions, branch-specific attention, and leaf predictions for weakly supervised FGVC.
Results
Experiments on CUB-200-2011, Stanford Cars, and Aircraft show favorable performance against state-of-the-art methods.
Takeaways & Limitations
Different root-to-leaf paths focus on different discriminative regions, while the tree supports coarse-to-fine hierarchical feature learning for FGVC.
Abstract
from arXiv · showhide
Fine-grained visual categorization (FGVC) is an important but challenging task due to high intra-class variances and low inter-class variances caused by deformation, occlusion, illumination, etc. An attention convolutional binary neural tree architecture is presented to address those problems for weakly supervised FGVC. Specifically, we incorporate convolutional operations along edges of the tree structure, and use the routing functions in each node to determine the root-to-leaf computational paths within the tree. The final decision is computed as the summation of the predictions from leaf nodes. The deep convolutional operations learn to capture the representations of objects, and the tree structure characterizes the coarse-to-fine hierarchical feature learning process. In addition, we use the attention transformer module to enforce the network to capture discriminative features. The negative log-likelihood loss is used to train the entire network in an end-to-end fashion by SGD with back-propagation. Several experiments on the CUB-200-2011, Stanford Cars and Aircraft datasets demonstrate that the proposed method performs favorably against the state-of-the-arts.
1. Introduction
FGVC distinguishes visually similar subordinate categories, but deformation, occlusion, illumination, and related variation make the task difficult. ACNet addresses weakly supervised FGVC with a hierarchical tree, branch-specific attention, and convolutional representation learning.
- 1. Introduction: FGVC distinguishes subordinate object categories, such as bird species and flowers, under high intra-class and low inter-class visual variance.The cited challenges include deformation, occlusion, and illumination.
- 1. Introduction: A single CNN struggles to describe differences between subordinate classes, motivating models that attend to multiple object regions.Object-part attention uses object and part attentions to exploit subtle local differences.
- 1. Introduction: ACNet combines convolutional operations on tree edges with node routing functions to select root-to-leaf computational paths.The architecture is designed for weakly supervised FGVC.
- 1. Introduction: The tree supports coarse-to-fine hierarchical feature learning, with different branches focusing on different local object regions and leaf predictions combined into the final decision.The introduction describes the final decision as the summation of predictions from all leaf nodes.
- 1. Introduction: ACNet uses a complete binary tree with prespecified depth and soft decisions, avoiding pruning error and reducing training time relative to adaptive tree growth.The paper also introduces an attention transformer and evaluates the method on CUB-200-2011, Stanford Cars, and Aircraft.
- 1. Introduction: The paper reports effectiveness across three challenging datasets and identifies the attention transformer as a component for coarse-to-fine hierarchical feature learning.An ablation study examines the influences of different components.
2. Related Works
Prior FGVC work uses supervised object or part annotations, weak image-level supervision, attention mechanisms, and neural decision trees. ACNet differs by applying attention on each tree branch while combining routing and representation learning.
- Deep supervised methods: Deep supervised FGVC methods use object, part, or keypoint annotations to guide CNN training and model fine-grained object differences.Examples include object and part detectors, local discriminative-region attention, part-stacked CNNs, and detection-classification subnetworks.
- Deep weakly supervised method: Weakly supervised FGVC methods require only image-level annotations and learn discriminative regions through multi-attention, recurrent attention, or multiple attention-region features.The cited methods use mutually reinforcing part generation, feature learning, and region-based representations.
- Decision tree: Neural decision-tree research combines routing, representation learning, and leaf nodes, while ACNet integrates sub-branch selection with representation learning.The related work frames decision trees as interpretable classification structures.
- Attention mechanism: Attention mechanisms guide models toward discriminative regions or channels, including spatial, feature-map, and channel weighting approaches.The related work cites distillation, cascade attention, channel weighting, and CBAM-style attention.
- Attention mechanism: Unlike the cited attention approaches, ACNet applies attention on each branch of its tree architecture to seek discriminative regions for classification.This branch-level design connects the attention mechanism to the tree structure.
3. Attention Convolutional Binary Neural Tree
ACNet combines a full binary tree, branch routing, convolutional edge operations, and attention transformers for weakly supervised fine-grained recognition. Its hierarchical paths aggregate leaf predictions while modeling localized and multiscale discriminative features.
- Architecture: ACNet represents its topology as a full binary tree whose nodes route samples and whose edges apply attention-transformer operations.For tree height h, the model has n = 2^h − 1 nodes and k = 2^h − 2 edges.
- Backbone network: The backbone uses constrained convolution and pooling settings with 448 × 448 inputs to preserve relatively small receptive fields for localized discriminative regions.The truncated backbone is pretrained on ILSVRC CLS-LOC and can also use other pretrained networks.
- Branch routing: Branch routing modules produce probabilities that determine whether samples follow left or right sub-branches at each tree node.A routing probability greater than 0.5 selects the left path; otherwise, the right branch dominates the decision.
- Attention transformer: Attention transformers operate along tree edges to capture discriminative features, with ASPP-based larger receptive fields addressing limited empirical receptive fields.Different branches focus on different local object regions, while the asymmetric design uses two left-edge transformers and one right-edge transformer to capture different feature scales.
- Label prediction: ACNet computes the final prediction by summing leaf predictions weighted by accumulated root-to-leaf routing probabilities.The resulting confidence vector is normalized so the confidences across subordinate classes sum to 1.
- Training: Training combines negative logarithmic likelihood losses for leaf and final predictions, using SGD and a two-stage optimization schedule.The first stage fixes the truncated VGG-16 parameters, while the second fine-tunes the entire network.
4. Experiments
Experiments evaluate ACNet on three fine-grained datasets and analyze how its tree, leaf-node, routing, attention, and ASPP components affect performance. ACNet achieves strong reported accuracy, while ablations support coarse-to-fine specialization and complementary leaf predictions.
- Dataset evaluations: ACNet achieves 87.8% and 88.1% top-1 accuracy on CUB-200-2011 with different backbones, outperforming the compared weakly supervised methods.
- Dataset evaluations: ACNet outperforms most compared methods on Aircraft, reaching 93.0% top-1 accuracy with ResNet-50 versus DCL’s 92.4%.With VGG-16, ACNet obtains 91.2% versus DCL’s 91.5%.
- Tree architecture: 3.025% top-1 accuracy improvement results when adding the tree architecture to the VGG-16 baseline.The comparison uses a VGG-16-only variant and a variant integrating the tree architecture.
- Tree architecture: 87.8% top-1 accuracy is achieved with tree height 3, whereas height 4 causes a 2.3% drop attributed to overfitting.The authors also report insufficient parameters for representing subordinate-category variation when h ≤2.
- Leaf-node specialization: 87.8% top-1 accuracy results from fusing four leaf predictions, exceeding their individual accuracies of 85.8%, 86.2%, 86.7%, and 87.0%.Different leaf nodes attend to background, head, wings, and tail regions.
- Tree architecture: 1.6% top-1 accuracy improvement is obtained with the asymmetrical tree over the symmetrical architecture, reaching 87.8%.The authors speculate that asymmetry fuses features with different receptive fields.
- Attention and routing components: 0.4% average top-1 accuracy improvement is obtained by adding attention, while Tree-ASPP improves average top-1 accuracy by 0.5%.The reported ASPP effect is attributed to parallel dilated convolutions with different dilation rates.
- Attention and routing components: 0.275% average top-1 accuracy is lost after removing the global context block from branch routing.Grad-CAM visualizations show branch and leaf responses, with deeper nodes focusing on increasingly detailed object regions.
5. Conclusion
ACNet addresses weakly supervised FGVC by combining attention-guided tree paths with coarse-to-fine hierarchical feature learning. Its final decision aggregates predictions from leaf nodes.
- ACNet uses attention transformers within convolutional operations along tree edges to focus different root-to-leaf paths on discriminative regions.
- The tree architecture organizes feature learning hierarchically from coarse to fine across different computational paths.
- ACNet produces its final prediction by max-voting among leaf-node predictions.