Source-linked AI summary
No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, D. Sculley
TL;DR
Large open image data sets are widely reused, but their geographic representativeness for developing-world applications is uncertain. The paper measures geo-diversity in ImageNet and Open Images and examines classifiers trained on them across locations, finding amerocentric and eurocentric bias alongside strong geographic performance differences. It concludes that data-set appropriateness and broad geo-representation should be considered when developing such systems.
Problem
Because large data sets are costly and open benchmarks are widely reused, the paper asks whether their geographic representation is appropriate for developing-world applications.
Method
The paper estimates country-level geo-diversity from textual, contextual, and URL metadata and evaluates classifiers trained on the data sets using images from varied geographical locations.
Results
The data sets show observable amerocentric and eurocentric bias, while classifier confidence and performance differ across locales, including poorer performance on images from Ethiopia and Pakistan than US-based images.
Takeaways & Limitations
The findings emphasize assessing data-set appropriateness and prioritizing broad geo-representation when creating data sets for ML in the developing world.
Takeaways & Limitations
Geographically diverse evaluation images were collected using internet tools that relied at least partly on image classifiers, including classifier-based filtering.
Abstract
from arXiv · showhide
Modern machine learning systems such as image classifiers rely heavily on large scale data sets for training. Such data sets are costly to create, thus in practice a small number of freely available, open source data sets are widely used. We suggest that examining the geo-diversity of open data sets is critical before adopting a data set for use cases in the developing world. We analyze two large, publicly available image data sets to assess geo-diversity and find that these data sets appear to exhibit an observable amerocentric and eurocentric representation bias. Further, we analyze classifiers trained on these data sets to assess the impact of these training distributions and find strong differences in the relative performance on images from different locales. These results emphasize the need to ensure geo-representation when constructing data sets for use in the developing world.
1 Introduction: Data and the Developing World
Open image data sets are widely reused because creating large data sets is costly, especially where resources for new data are limited. The paper argues that geographic non-representation can undermine predictive performance in developing-world applications.
- Creating large data sets from scratch is costly, so practitioners commonly use freely available ImageNet and Open Images.
- This reuse is particularly desirable for developing-world ML applications, where resources for creating new data sets may be limited.
- If these data sets do not represent locations of interest, models trained on them may suffer reduced predictive performance.
- The paper assesses geo-diversity and classification differences across geographical locations, finding observable amerocentric and eurocentric bias in both assessments.
2 Background: ImageNet and Open Images
ImageNet and Open Images are popular public benchmark data sets that practitioners often use when application-specific data are unavailable. Their country distributions prominently feature the US and Great Britain, while publicly released pretrained models support standard image-classification use.
- ImageNet and Open Images are popular public data sets generally considered academic benchmarks, but they were not necessarily constructed for every use case.
- Practitioners commonly fall back to these standard data sets when no robust application-specific data source is available.
- In both data sets, the US and Great Britain are among the top represented locations.
- Publicly available pretrained classifiers use the Inception V3 architecture and provide competitive performance across standard benchmarks.
3 Analyzing Geo-Diversity
The study estimates country-level geo-diversity using proxy metadata and examines geographic representation in Open Images and ImageNet. Both data sets are dominated by a small group of North American and European countries, with China and India sparsely represented.
- Country-level locations were recovered for many images using textual, contextual, and URL metadata, despite the difficulty of identifying every image’s geography.
- The study accepts possible noise in country identification because it is considered unlikely to change the larger trends.
- Open Images: More than 32% of the Open Images sample was US-based, while 60% came from the six most represented countries across North America and Europe.
- Open Images: China and India accounted for only 1% and 2% of Open Images images, respectively.
- ImageNet: Around 45% of the ImageNet sample was US-based, while China and India represented 1% and 2.1%, respectively.
4 Analyzing Classification Behavior Based on Geo-Location
The study tests classification across geographically varied images using crowdsourced and geo-located web data, finding location-linked performance differences in models trained on ImageNet and Open Images.
- Stress-test data collection: Crowdsourced raters in Hyderabad supplied community-specific image URLs, while geo-located web images extended testing across 15 countries and people-related labels.The collection sought to reduce amerocentric and eurocentric bias and included labels such as bridegroom, police officer, and greengrocer.
- Classifier performance: Non-US groom images tended to receive lower Open Images model likelihoods than US groom images.Figure 4 compares likelihood densities for groom, bridegroom, butcher, greengrocer, and police officer categories.
- Geo-dependent mis-classifications: ImageNet-trained classifiers often misclassified Hyderabad-sourced groom and bridegroom images as attire-related categories despite highlighting human faces.Observed alternatives included chain mail, cloth, academic gown, and vestment.
- Classifier performance: Performance differences appeared in both ImageNet and Open Images classifiers, indicating the pattern was not particular to a single data set.Some classes had similar prediction distributions across countries, suggesting better representation for those classes.
- Classifier performance: US groom and bridegroom images clustered at high log likelihood, whereas Ethiopia and Pakistan images were more uniformly distributed, indicating poorer classifier performance.The trend was confirmed across several other countries in different world regions.
- Scope: The analysis focused on humans, although developed–developing-country distributional differences may also occur for sports, transportation, and wildlife.This broadens the listed scope of potential geo-dependent behavior beyond the evaluated human labels.
5 Discussion
The discussion concludes that ImageNet and Open Images may lack sufficient geo-diversity for broad representation across the developing world. It therefore emphasizes assessing dataset appropriateness and prioritizing broad geo-representation when creating new datasets.
- Dataset scope: ImageNet and Open Images may not provide sufficient geo-diversity for broad representation across the developing world.The datasets were designed for specific purposes, so problems may arise when they are later adopted for other uses.
- Implications: Dataset appropriateness should be assessed before training models intended for use in the developing world.The discussion presents this as an importance of the study’s findings.
- Implications: New datasets should prioritize broad geo-representation as a first-class goal to aid machine learning in the developing world.
- Empirical context: Images from Ethiopia and Pakistan were classified less consistently than images from the United States and Australia.Figure 5 uses log likelihood assigned to the bridegroom class to align images from different countries.