Source-linked AI summary
TACO: Trash Annotations in Context for Litter Detection
Pedro F Proença, Pedro Simões
TL;DR
TACO addresses the lack of contextual annotated data for litter detection in the wild by providing a crowdsourced dataset and supporting tools. The paper describes its taxonomy and transplantation tools, then evaluates Mask R-CNN on two litter tasks, finding promising but error-prone performance and a need for substantially more annotations.
Problem
Litter is poorly represented in general image datasets, while in-the-wild detection requires annotated photos showing trash in diverse contexts.
Method
The paper constructs TACO with hierarchically segmented litter annotations, develops transplantation tools, and evaluates Mask R-CNN on classless and ten-class tasks.
Results
Mask R-CNN results are promising overall, but performance is poor for tiny objects such as cigarettes and remains prone to errors on new or rare backgrounds.
Takeaways & Limitations
TACO is a useful starting point for litter detection research and invites further annotation contributions to improve its coverage.
Takeaways & Limitations
The dataset needs significantly more annotated images, and the network configuration performs poorly on tiny objects.
Abstract
from arXiv · showhide
TACO is an open image dataset for litter detection and segmentation, which is growing through crowdsourcing. Firstly, this paper describes this dataset and the tools developed to support it. Secondly, we report instance segmentation performance using Mask R-CNN on the current version of TACO. Despite its small size (1500 images and 4784 annotations), our results are promising on this challenging problem. However, to achieve satisfactory trash detection in the wild for deployment, TACO still needs much more manual annotations. These can be contributed using: http://tacodataset.org/
I. INTRODUCTION
Litter detection in natural environments lacks both effective control mechanisms and representative annotated data. TACO addresses this gap with a crowdsourced dataset of segmented litter images from diverse environments.
- Effective technological and regulatory means for controlling litter are currently lacking.
- Deep-learning litter monitoring requires annotated photos showing trash in context rather than isolated images on white backgrounds.
- TACO collects segmented and hierarchically annotated litter photographs from diverse environments such as beaches and cities.
- Trash in the wild is difficult to detect because it can be deformable, transparent, aged, fragmented, occluded, or camouflaged.
II. TACO DATASET
TACO combines high-resolution mobile-phone images with contextual scene tags and hierarchical litter annotations. Its taxonomy supports flexible task-specific class groupings but exhibits substantial size and class imbalance.
- TACO stores mostly high-resolution mobile-phone images, with Flickr managing the images and a server managing annotations and crawling for additional images.
- The dataset annotates litter using 60 categories organized into 28 super categories, including Unlabeled litter for ambiguous or uncovered objects.
- Scene tags describe image backgrounds and are not mutually exclusive.
- Classes can be rearranged for particular tasks because all annotated objects can be treated as litter and some classes are visually indistinguishable.
- Most cigarette annotations occupy less than 64 × 64 pixels, creating a size-related challenge for detection.
A. Litter transplants
TACO supports transplanting segmented litter into under-represented environments to augment data. A distance-transform soft mask smooths object silhouettes, while the approach remains limited for transparent objects and realistic lighting or camera effects.
- Transplantation can augment under-represented backgrounds when segmentation masks are sufficiently precise, especially for closed-set environments.
- Copying objects with hard masks creates edge artifacts, so TACO mixes transplanted objects pixelwise with target images using a truncated distance transform.
- Transplantation does not work well for transparent objects and omits many lighting and camera factors needed for realism.
- The resulting soft mask smooths the transplanted silhouette and makes its boundary more even than a hard-mask transplant.
- The Transplanter GUI embeds the transplantation function and additional features intended to make the process easy and seamless.
III. EXPERIMENTS
The study evaluates Mask R-CNN on classless litter detection and on detection with classification across ten litter classes. Because TACO is small, results use four-fold cross-validation and mask-based Average Precision.
- Mask R-CNN is evaluated on TACO-1, which detects and segments litter without class labels.
- Mask R-CNN is evaluated on TACO-10, which detects, segments, and distinguishes ten litter classes.
- Four-fold cross-validation uses random 80% training, 10% validation, and 10% test splits for each fold.
- Performance is measured with Average Precision averaged over Intersection-over-Union thresholds using instance masks.
A. Implementation
The implementation adapts Mask R-CNN with a ResNet-50 Feature Pyramid Network for TACO litter detection and segmentation.
- A. Implementation: Mask R-CNN uses a default ResNet-50 Feature Pyramid Network backbone with 1024×1024-pixel resized and padded inputs.Models were initialized from Mask R-CNN weights trained on COCO.
B. Prediction Scoring
The paper compares prediction-ranking scores derived from Mask R-CNN class probabilities, emphasizing a ratio score that incorporates litter confidence.
- B. Prediction Scoring: The scoring study compares three prediction-ranking scores derived from Mask R-CNN’s classification-head probabilities.The background probability is included as p_N+1 among the class probabilities.
- B. Prediction Scoring: ratio_score combines confidence in a specific litter class with confidence that the object is litter rather than background.A small ϵ prevents NaN values.
C. Results
Mask R-CNN performance is evaluated across TACO tasks and prediction scores, with ratio_score improving the multi-class setting while small and visually similar objects remain difficult.
- C. Results: ratio_score improves AP on TACO-10 without decreasing AP on TACO-1, and correlates well with prediction Intersection-over-Union.The score expresses confidence that an object belongs to a class rather than being background.
- C. Results: Poor cigarette detection contributes substantially to low TACO-10 performance because many small ground-truth objects are missed.The paper identifies missed objects smaller than 20×20 px.
- C. Results: Detection is better for Cans and Bottles, although many bottles are classified as cans.Plastic bags and Other are also confused because of similar materials.
- C. Results: The examples include transparent-object detections, but also false-positive seashell detections and errors on rare backgrounds such as ocean waves.These examples illustrate both successful detections and remaining errors in unfamiliar settings.
IV. CONCLUSIONS AND FUTURE WORK
TACO and its tools provide a starting point for litter detection, but the dataset requires substantially more annotations and improved methods for tiny objects.
- IV. CONCLUSIONS AND FUTURE WORK: TACO’s dataset and tools support litter detection and have received positive feedback from researchers and entrepreneurs.The authors invite contributions of additional annotations using the tools.
- IV. CONCLUSIONS AND FUTURE WORK: Tiny-object detection is poor with the current network configuration and significantly affects overall AP.Higher input resolution increases memory use, while sliding windows sacrifice surrounding context.