Source-linked AI summary
Information Density Imbalance in Visual Object Detection
Ziwei Zhao, Yanxi Lu, Yuwei Hu, Shiyang Su, Mingxuan Wang, Chenyue Zhou, Jiayi Chen, Hehan Li, Xiaoshuai Hao, Andi Zhang, Yanbiao Ma
TL;DR
Object-detection category bias persists even when instance counts are relatively balanced, so instance quantity alone is insufficient to explain performance disparities. The paper introduces information density, studies its relationship with accuracy and training, and integrates it into three loss functions. Across Pascal VOC, COCO-LT, and LVIS, information density reduces model bias while improving overall performance.
Problem
Object-detection category bias is not fully explained by instance-number imbalance, including in datasets with relatively balanced category counts.
Method
The paper defines information density from instance diversity and total instance area, then integrates it into three detection losses and dynamically updates it during training.
Results
Information density shows a significant negative correlation with category accuracy, and incorporating it into loss functions reduces bias while improving overall performance across Pascal VOC, COCO-LT, and LVIS.
Takeaways & Limitations
Information density provides a measure of category learning difficulty and a basis for designing fairer object-detection losses and training strategies.
Takeaways & Limitations
The authors note that factors beyond information density may also affect model bias or performance.
Abstract
from arXiv · showhide
In object detection, the number of instances is typically used to determine whether a dataset exhibits a long-tailed distribution, implicitly assuming that the model will perform poorly on categories with fewer instances. This assumption has led to extensive research on category bias in datasets with imbalanced instance numbers. However, even in datasets where instance numbers are relatively balanced, models still exhibit category bias, indicating that instance count alone cannot explain this phenomenon. In this work, we first introduce the concept and measurement of information density. We then observe a significant negative correlation between a category's information density and its accuracy, and we investigate how the training process impacts this relationship. Empirical studies suggest that information density imbalance may be a potential source of category bias. To preliminarily validate the potential of information density, we made simple improvements to three advanced object detection loss functions using this concept. Experiments on the Pascal VOC, COCO-LT, and LVIS datasets demonstrate that information density can significantly reduce model bias while effectively enhancing the overall performance of existing loss functions. This study provides a new perspective for understanding the generalized bias phenomenon in object detection models and offers new tools for designing fairer loss functions and training strategies.
1 INTRODUCTION
Instance count alone does not explain category bias in object detection, motivating measures of deeper dataset factors. The paper introduces information density and uses it to improve loss functions and reduce bias across datasets.
- Motivation: The Pascal VOC analysis found a Pearson correlation of -0.171 between category instance count and detection accuracy.This weak correlation suggests that factors beyond instance quantity influence category performance.
- Motivation: Instance diversity and total instance area are investigated as deeper factors related to category accuracy.Highly diverse categories may be harder to learn, while larger instances may be easier to detect.
- Information Density: Information density is defined as the ratio of instance diversity to the total area of instances.It represents the amount of information per unit area and captures the combined impact of diversity and instance area.
- Information Density: Information density has a significant negative correlation with category detection accuracy, and this correlation strengthens during training.The findings suggest that current optimization objectives overlook bias associated with information-density imbalance.
- Method and Evaluation: The paper incorporates information density into Seesaw Loss, Equalized Focal Loss, and C2AM Loss, with evaluations on Pascal VOC, COCO-LT, and LVIS.The proposed improvements significantly alleviate model bias and improve overall performance across non-long-tailed and long-tailed datasets.
2 RELATED WORK
Related work has largely addressed object-detection bias through long-tailed datasets, sampling, loss design, architectural changes, and training strategies. The paper emphasizes that bias can persist with balanced sample sizes, while object-detection research has insufficiently investigated underlying class-difficulty factors.
- Long-Tailed Object Detection: Long-tailed object detection amplifies model bias through artificially constructed datasets but can divert attention from bias in balanced scenarios.The cited discussion states that instance-number imbalance does not fully explain widely observed object-detection bias.
- Long-Tailed Object Detection: Long-tailed recognition methods include data re-sampling, specialized loss functions, architectural improvements, decoupled training, and data augmentation.These approaches target imbalance through different data, objective, architecture, and training modifications.
- Long-Tailed Object Detection: Re-sampling balances data by increasing the sampling frequency of tail-class examples, while specialized losses reduce excessive suppression of tail classes.Examples include class-aware sampling, repeat factor sampling, EQL variants, Seesaw Loss, Equalized Focal Loss, ACSL, and LOCE.
- Long-Tailed Object Detection: Module improvements group classes using prior knowledge, while decoupled training separates feature learning from later classification adjustment.The related-work discussion presents these as additional strategies for long-tailed detection.
- Methods for Measuring Class Difficulty: Balanced-sample classification studies link class performance disparities to perceptual-manifold geometry and spectral features, but object detection has not explored these factors sufficiently.Reported geometric factors include manifold volume, curvature, and intrinsic dimensionality.
3 INFORMATION DENSITY IMBALANCE
This section defines information density by combining instance diversity with total instance area, then examines how these factors relate to category detection accuracy during training. Experiments find that information density is negatively correlated with accuracy, with this relationship strengthening over epochs and motivating loss-function improvements.
- Instance Diversity Measurement: Instance diversity is estimated from classification-module embeddings using a perceptual-manifold volume and Ledoit–Péché nonlinear covariance shrinkage.The embeddings are arranged as X ∈ R^p×m, and nonlinear eigenvalue transformation improves covariance-estimation stability.
- Empirical Correlations: Instance diversity is negatively correlated with category accuracy, whereas total instance area is positively correlated with detection accuracy.The experiments use Faster R-CNN models with multiple backbones and loss functions on Pascal VOC and MS COCO.
- Information Density Definition: Information density combines instance diversity and total instance area into a unified measure of information per unit area.It is defined as instance diversity divided by the total area of all ground-truth bounding boxes.
- Empirical Correlations: Information density shows an even more significant negative correlation with category average precision than the component factors considered separately.The measure captures the combined influence of category diversity and instance area.
- Training Dynamics: The correlation between information density and category average precision strengthens as training progresses and remains significant at the end of training.The authors attribute the evolving relationship to the model learning prominent features first and more complex features later.
4 METHOD
The method embeds information density into three object-detection losses and introduces a dynamic update strategy that reduces storage while preserving the calculated information density.
- Loss-function improvements: The method incorporates information density into Seesaw, Equalized Focal, and C2AM losses to address category bias.The resulting variants include IDG-Seesaw, an information-density-guided EFL modification, and an information-density-based C2AM modification.
- Loss-function improvements: IDG-Seesaw increases attention to classes with higher information density by modifying the balancing factor and its associated negative gradients.When class i has greater information density than class j, the negative gradient for class j is increased.
- Loss-function improvements: The information-density-guided EFL modification changes its weighting term to account for information-density imbalance between classes.Other details remain consistent with the original Equalized Focal Loss.
- Loss-function improvements: The information-density-guided C2AM modification replaces original weight-norm ratios with information density to better reflect class-diversity effects on decision-space allocation.C2AM originally addresses imbalanced classifier-weight norms and assigns larger decision spaces to classes with greater diversity.
- Dynamic update strategy: The dynamic update strategy stores embeddings in a queue, computes local covariance statistics, and merges them into global covariance estimates for information-density updates.The strategy uses multiple local sample covariance matrices to calculate the global sample covariance matrix.
- Dynamic update strategy: The new training framework reduces storage utilization by merging local covariance matrices while keeping the calculated information density unchanged.This provides a lower-cost way to update information density during training than repeatedly extracting embeddings for the entire dataset.
5 EXPERIMENTS
Experiments across Pascal VOC, COCO-LT, and LVIS v1.0 show that information-density-guided losses improve accuracy, especially for underperforming categories, while reducing model bias.
- 5.1 Datasets and Evaluation Metrics: Experiments evaluate information-density-guided improvements on non-long-tailed Pascal VOC and long-tailed COCO-LT and LVIS v1.0 datasets.The study uses Faster R-CNN with R-50-FPN and R-101-FPN backbones and compares modified Seesaw, EFL, and C2AM losses with their originals.
- 5.3 Evaluation on Pascal VOC: 3.1%, 1.8%, 4.3%, 2.7%, and 4.6% gains occur for aeroplane, boat, bottle, chair, and potted plant under Seesaw with R-50-FPN.The improvements are especially pronounced for the two worst-performing categories, bottle and potted plant.
- 5.4 Evaluation on COCO-LT and LVIS v1.0: 1.0%, 1.3%, and 1.1% mAP gains are obtained by IDG-Seesaw, IDG-FL, and IDG-C2AM on COCO-LT with R-50-FPN.Rare-category AP b 1 gains are 5.2%, 4.7%, and 5.5%, respectively; R-101-FPN increases them to 5.8%, 5.5%, and 5.6%.
- 5.4 Evaluation on COCO-LT and LVIS v1.0: 3.1%, 3.5%, and 3.4% APr gains are achieved by IDG-Seesaw, IDG-FL, and IDG-C2AM on LVIS v1.0 with R-50-FPN.With R-101-FPN, the respective APr gains are 4.3%, 3.2%, and 2.9%.
- 5.5 Effectiveness in Reducing Model Bias: 37.23% and 45.89% reductions in model bias are reported for IDG-Seesaw on COCO-LT and IDG-FL on LVIS v1.0, respectively.Model bias is measured using variance in Average Precision across classes.
- 5.5 Effectiveness in Reducing Model Bias: Information-density imbalance complements gradient- and decision-space-imbalance approaches because reducing gradient imbalance does not necessarily address this bias source.The authors urge consideration of information-density imbalance when designing object detection models.
6 CONCLUSION AND FUTURE WORK
The paper concludes that information density captures category learning difficulty beyond instance count and can guide future data augmentation, model-capacity, and architecture studies.
- 6 CONCLUSION AND FUTURE WORK: Information density cannot be explained solely by instance number and is negatively correlated with category accuracy.The paper presents it as a measure of category detection difficulty and reports the strongest improvements for poorly performing categories.
- 6 CONCLUSION AND FUTURE WORK: High-information-density categories may benefit from Copy-Paste or generative data expansion.These approaches are proposed as directions for using information density to guide data augmentation.
- 6 CONCLUSION AND FUTURE WORK: Information density may help estimate model or backbone capability limits and support model-size choices that balance deployment cost and performance.Higher information density is described as requiring models to handle more information per unit area.
- 6 CONCLUSION AND FUTURE WORK: The correlation between information density and accuracy may inform adaptive modules or branches for categories with high information density.The paper frames this as an architecture-design direction related to model capacity and information compression.