Source-linked AI summary
Fine-Grained Visual Classification of Aircraft
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, Andrea Vedaldi
TL;DR
Fine-grained aircraft recognition requires distinguishing many visually similar models, motivating a dedicated benchmark beyond commonly studied animal and pet domains. The paper introduces FGVC-Aircraft, constructs its hierarchy and dataset from enthusiast-supported collections, and evaluates baseline recognition across hierarchy levels. The baseline reaches 58.48% variant accuracy and 71.30% manufacturer accuracy, while the dataset remains constrained by possible photographer-related correlations.
Problem
Fine-grained aircraft recognition must distinguish subtle visual differences across aircraft models while addressing variation in purpose, structure, history, and branding.
Method
The paper builds FGVC-Aircraft from 10,000 images, hierarchical annotations, online resources, and aircraft-enthusiast contributions, then defines three recognition tasks.
Results
58.48% variant accuracy and 71.30% manufacturer accuracy are obtained by evaluating a baseline across the hierarchy.
Takeaways & Limitations
FGVC-Aircraft establishes aircraft recognition as a distinct FGVC domain with variation modes different from frequently studied object classes.
Takeaways & Limitations
The dataset may contain unwanted correlations because it relies on a small set of photographers and their regional or temporal dependencies.
Abstract
from arXiv · showhide
This paper introduces FGVC-Aircraft, a new dataset containing 10,000 images of aircraft spanning 100 aircraft models, organised in a three-level hierarchy. At the finer level, differences between models are often subtle but always visually measurable, making visual recognition challenging but possible. A benchmark is obtained by defining corresponding classification tasks and evaluation protocols, and baseline results are presented. The construction of this dataset was made possible by the work of aircraft enthusiasts, a strategy that can extend to the study of number of other object classes. Compared to the domains usually considered in fine-grained visual classification (FGVC), for example animals, aircraft are rigid and hence less deformable. They, however, present other interesting modes of variation, including purpose, size, designation, structure, historical style, and branding.
1. Introduction
FGVC-Aircraft introduces a hierarchical aircraft dataset for fine-grained recognition, targeting visually distinguishable yet often subtle differences among aircraft models. The paper also motivates aircraft as a complementary FGVC domain and presents a data-collection strategy and baseline benchmark.
- Dataset and problem: FGVC-Aircraft contains 10,000 airplane images spanning 100 visually distinguishable aircraft models organised hierarchically.The model differences are often subtle, making classification challenging while remaining visually measurable.
- Motivation: Aircraft offer FGVC variation through size, designation, purpose, propulsion, structure, historical style, and airline branding.Aircraft are also largely rigid, reducing deformability effects and highlighting these other variation modes.
- Contributions: The paper contributes detailed model annotations, an enthusiast-supported online data-collection method, and baseline aircraft-model identification results.The authors suggest that the collection strategy may apply to other object classes.
2. The dataset: content, tasks, and evaluation
FGVC-Aircraft provides 10,000 aircraft images with hierarchical labels for variant, family, and manufacturer recognition. It defines dataset protocols and evaluates models using class-normalised average accuracy.
- Dataset content: The dataset contains 10,000 airplane images annotated with the dominant aircraft’s model and bounding box.Images are distributed across 100 variants, with 100 example images per variant and balanced training, validation, and test subsets.
- Label hierarchy: The hierarchy contains 100 visually detectable variants, 70 families of related variants, and 30 manufacturers grouping families.Variants merge models judged visually indistinguishable, while families represent a higher-level intermediate-difficulty distinction.
- Evaluation protocol: Bounding boxes may be used for classifier training but must not be used during testing.The dominant aircraft is generally centred, supporting fine-grained discrimination rather than object detection.
- Tasks and evaluation: The benchmark defines variant, family, and manufacturer recognition tasks evaluated by class-normalised average classification accuracy.The metric averages the diagonal elements of the normalised confusion matrix.
3. Dataset construction
The dataset was constructed by leveraging aircraft-spotter collections, filtering and diversifying images, validating bounding boxes, and manually building the visual label hierarchy.
- Initial data collection: Aircraft enthusiasts and spotters supplied annotated visual data that made collecting detailed aircraft labels feasible.The authors obtained explicit research-use permissions from participating photographers and retained only permitted data.
- Initial data collection: About 70,000 downloaded images were reduced to the 100 most frequent variants, retaining at least 120 images per variant.The original collection spanned thousands of models and had a highly skewed class distribution.
- Diversity maximisation: The selection procedure maximised internal diversity using photographer, time, airliner, and airport information before incrementally selecting 100 images per variant.This was intended to reduce unwanted correlations caused by photographer and regional dependencies.
- Annotation: Roughly 110 images per variant underwent three-way Mechanical Turk bounding-box annotation, after which invalid or unboxed images were discarded.The remaining annotations were averaged to obtain final bounding boxes.
- Hierarchy construction: The hierarchy was built largely by manual inspection, with visual evidence from example images, Wikipedia, and manufacturer websites used to merge indistinguishable models.Some distinctions, such as aircraft-length differences, were subtle and difficult to estimate from monocular images.
4. Baselines
The baseline evaluates aircraft recognition across hierarchical labels, with performance measured using class-normalised average accuracy. Variant recognition reaches 58.48%, while manufacturer recognition reaches 71.30%, and merging variant predictions performs better than training directly at each hierarchy level.
- Evaluation: The benchmark defines variant, family, and manufacturer recognition tasks evaluated with class-normalised average accuracy.For variant recognition, the task has 100 classes and uses the average diagonal of the normalised confusion matrix.
- Baseline: The strong baseline combines a χ2-kernel nonlinear SVM, bag-of-visual-words, dense SIFT, a 600-word dictionary, and a spatial pyramid.It is trained on entire images while ignoring the provided bounding boxes.
- Results: The baseline handles distinctive categories well but struggles with subtle Airbus and Boeing family variations.The Eurofighter Typhoon has an error of 5.9%, whereas subtle family differences produce large intra-family confusion.
- Results: 58.48% accuracy is obtained for variant classification, compared with 71.30% for manufacturer classification.The hierarchical measurements use one model trained for variant recognition, with labels merged below the evaluated level.
- Results: Training directly for a hierarchy level performs significantly worse than merging labels from a model trained for variant classification.This comparison is reported for the hierarchical evaluation strategy.
5. Summary
FGVC-Aircraft is a large aircraft dataset for fine-grained visual categorisation, covering multiple hierarchical levels and distinctive modes of variation. Its images were collected from aircraft spotter collections to increase internal diversity, with planned expansion to additional models and object categories.
- Summary: FGVC-Aircraft contains 10,000 images spanning 100 airplane model variants, 70 families, and 30 manufacturers.The dataset is intended for fine-grained visual categorisation and introduces aircraft as a distinct recognition domain.
- Summary: Aircraft provide modes of variation that differ from frequently used fine-grained visual categorisation classes.The paper specifically highlights aircraft as a novel domain for the wider computer vision community.
- Summary: Images were obtained from aircraft spotter collections to maximise internal diversity and reduce unwanted photographer-specific correlation.The authors plan to expand the dataset as more photographers permit use of their photos and to apply the construction approach to other object categories.