PaperlayerView exact revision on arXiv

Source-linked AI summary

WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models

Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz, Mahedi Hasan, Israt Khandaker, Ibrahim Khalil Shanto, Muhammad Nomani Kabir

arXiv:2608.22950v1cs.CV

TL;DR

Floating-waste monitoring requires models to localize, classify, count, and explain multiple objects in cluttered water-surface scenes. WADE benchmarks this setting with reasoning annotations and compact VLM evaluations, finding that fine-tuning improves grounding and hallucination control while most annotated instances remain undetected.

  • Problem

    Small, overlapping, partially submerged waste objects resemble vegetation and reflections, while existing models often describe objects without accurately spatially grounding them.

  • Method

    WADE provides 2,167 rural Bangladesh images, 13,608 boxes, ten waste categories, and class-level reasoning chains, evaluating six VLMs across prompting and fine-tuning regimes.

  • Results

    Existing VLMs struggle primarily with spatial grounding, while QLoRA fine-tuning of Qwen3-VL-2B raises recall from 0.0248 to 0.2339 and class-aware F1 from 0.0257 to 0.2163.

  • Takeaways & Limitations

    Plausible reasoning does not establish successful grounding, so reliable floating-waste monitoring requires geometric metrics and qualitative inspection alongside generated explanations.

  • Takeaways & Limitations

    The study cannot isolate reasoning annotations’ contribution because fine-tuning jointly supervises boxes, labels, and reasoning chains without a box-and-label-only condition.

Abstract

from arXiv · show

Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquatic-waste datasets provide limited geographic coverage, sparse multi-instance annotations, and little supervision beyond boxes and labels. Compact vision-language models (VLMs) therefore remain insufficiently evaluated for jointly localizing, classifying, counting, and explaining floating waste. We introduce WADE, a reasoning-annotated benchmark containing 2,167 images from rural Bangladesh, 13,608 bounding boxes, and ten waste categories. Each annotation is associated with class-level recognition rules covering visual cues, likely confusions, and discriminative features. We evaluate six VLMs under zero-shot, two-shot, reasoning-guided, and fine-tuned settings using detection, counting, and hallucination metrics. For resource-efficient adaptation, we jointly fine-tune Qwen3-VL-2B on boxes, labels, and reasoning chains using QLoRA. Fine-tuning increases recall from 0.0248 to 0.2339 and F1 from 0.0257 to 0.2163, while reducing image-level hallucination from 0.6836 to 0.0883. However, over three-quarters of instances remain undetected, establishing WADE as a challenging benchmark for dense floating-waste grounding with compact VLMs.

1. Introduction

WADE addresses the underexplored problem of grounding every floating-waste instance in cluttered waterways, where objects are small, occluded, overlapping, unevenly distributed, and easily confused with vegetation. The benchmark combines multi-instance localization with category labels and structured visual reasoning using in-situ imagery from rural Bangladesh.

  • Motivation: Floating waste threatens aquatic ecosystems and transports land-based pollution toward marine environments, motivating scalable computer-vision monitoring.Existing datasets such as TACO and FloW have supported litter and floating-waste detection.
  • Challenges: Small size, vegetation similarity, partial submersion, overlap, and uneven scene distribution make real water-surface waste difficult to detect.These conditions are illustrated in the WADE task overview.
  • Challenges: Grounding every visible instance differs from identifying one salient object because VLMs may miss objects, produce inaccurate boxes, or hallucinate unsupported categories.Visual reasoning annotations may connect predictions with discriminative image evidence, but their role in multi-instance floating-waste grounding remains underexplored.
  • WADE Benchmark: WADE contains 2,167 in-situ images, 13,608 bounding boxes, and ten categories from rural Bangladeshi waterways.The dataset averages 6.28 instances per image and provides broad coverage for multi-instance floating-waste grounding.
  • WADE Benchmark: Each instance has a structured class-level reasoning chain specifying a primary visual cue, a confusable category, and a separating rule.WADE is introduced as a reasoning-annotated benchmark for multi-instance floating-waste grounding with compact VLMs.

2. Related Work

Prior work spans camera-based, multispectral, industrial, and language-guided approaches to waste and environmental observation. Environmental datasets increasingly add natural-language explanations to spatial annotations, motivating WADE’s annotation perspective.

  • Camera-based river monitoring demonstrated continuous automated observation, while reflections, organic material, and environmental variation remained important challenges.
  • MARIDA uses multispectral Sentinel-2 imagery with pixel-level annotations to distinguish marine debris from water, vegetation, foam, ships, and other surface features.
  • Language-guided detectors connect visual localization with natural-language concepts, progressing from GLIP’s phrase grounding to Grounding DINO’s open-set localization and GLaMM’s pixel-linked textual responses.
  • VisText-Mosquito combines object detection, water-surface segmentation, and natural-language explanations, illustrating the value of interpretable text alongside spatial annotations.

3. The WADE Dataset

WADE is a multi-instance floating-waste grounding dataset with 2,167 field images, 13,608 annotated instances across ten categories, and validated class-level reasoning chains. It combines naturally occurring scenes from rural Bangladesh with object-level annotations and site-disjoint evaluation partitions.

  • Dataset Overview: WADE contains 2,167 field images and 13,608 annotated instances across ten floating-waste categories, averaging 6.28 instances per image.Each instance includes a bounding box, category label, and validated class-level reasoning chain.
  • Data Collection: Images span ponds, canals, and rivers across 390 rural-Bangladesh locations, covering winter and monsoon conditions and varied daylight levels.All annotated objects occurred naturally, without placement or rearrangement during collection.
  • Taxonomy: The taxonomy includes ten categories, including water hyacinth, plastic bottle, organic waste, industrial waste, polythene, algal bloom, solid waste, fabric waste, wood debris, and foam waste.Biological surface materials are included because they frequently coexist with waste and create difficult visual confounders.
  • Reasoning Annotations: Each category has a scene-invariant reasoning chain covering appearance, visual cues, likely confusions, contrastive rules, partial-visibility evidence, and failure modes.The same category-level chain is associated with every instance of that category, providing consistent supervision without writing repetitive explanations for every box.
  • Dataset Splits: The dataset is partitioned 75/10/15 into 1,625 training, 217 validation, and 325 test images, with all images from each water-body site kept in one partition.This produces image-disjoint and collection-site-level location-disjoint evaluation.

4. Experimental Setup

WADE is evaluated as a structured generative grounding task requiring models to enumerate waste instances with boxes, labels, and reasoning chains. The study compares six VLMs across four evaluation regimes using detection, counting, hallucination, and human- or LLM-based assessment protocols.

  • Task and evaluation regimes: WADE requires each model to enumerate visible target instances and output a normalized bounding box, taxonomy label, and class-level reasoning chain.Six VLMs are evaluated under zero-shot, two-shot, reasoning-guided prompting, and reasoning-supervised QLoRA fine-tuning.
  • Model suite: Six models comprise four open-weight VLMs, one larger 11B open-weight model, and two commercial API systems.The open-weight models are LLaMA-3.2-11B-Vision, InternVL2-2B, Qwen3-VL-2B-Instruct, and LFM2.5-VL-1.6B; commercial systems are GPT-4o-mini and Gemini-2.5-Flash.
  • Prompting protocols: Two-shot prompting uses the same two training-set demonstrations for every test image, each containing an image and complete ground-truth boxes, labels, and reasoning chains.The fixed demonstrations control variation caused by exemplar selection in multimodal in-context learning.
  • Fine-tuning limitation: Reasoning’s independent contribution cannot be quantified because fine-tuning jointly uses boxes, category labels, and reasoning chains without a box-and-label-only ablation.The vision encoder and base language model remain frozen while only low-rank adapters are optimized.
  • Evaluation metrics: Detection is measured with class-aware precision, recall, and F1 at IoU ≥0.5, plus class-agnostic F1 and recall at IoU ≥0.75.Predicted and ground-truth boxes are matched one-to-one by descending IoU; mean IoU is computed only over matched pairs.
  • Evaluation metrics: Counting uses MAE and RMSE, hallucination uses instance-level and image-level CHAIR, and qualitative assessment combines three LLM judges with two model-blind human evaluators.Average precision is omitted because textual boxes lack confidence scores required to rank predictions.

5. Results and Analysis

WADE exposes severe grounding failures in compact and larger VLMs under dense floating-waste conditions, with prompting offering limited or conservative changes. Domain-specific fine-tuning substantially improves Qwen3-VL-2B’s localization and hallucination control, although counting errors and missed instances remain substantial.

  • Zero-shot grounding: 0.1174 F1 makes Gemini-2.5-Flash the strongest zero-shot model, while Qwen3-VL-2B is the strongest compact open model at 0.0257 F1.All zero-shot models perform poorly at IoU = 0.5, including large and commercial VLMs.
  • Zero-shot grounding: 93% of the ground-truth count yields only 2.48% instance recall for Qwen3-VL-2B, showing incorrect localization dominates label disagreement.Removing the class constraint raises F1 only from 0.0257 to 0.0347.
  • Prompting: 0.0385 precision and 0.0403 class-agnostic F1 follow reasoning-guided prompting for Qwen3-VL-2B, but recall falls from 0.0248 to 0.0137.Image-level hallucination also falls from 0.6836 to 0.3077, while the box-count ratio decreases from 0.93 to 0.36.
  • Fine-tuning: 0.2339 recall and 0.2163 F1 after fine-tuning represent increases from 0.0248 and 0.0257, respectively, with recall at IoU = 0.75 rising to 0.1041.Fine-tuned Qwen3-VL-2B also exceeds Gemini-2.5-Flash in precision, recall, and F1, but the result reflects the complete WADE supervision package rather than reasoning chains alone.
  • Limitations: 0.0883 image-level CHAIR after fine-tuning replaces 0.6836 before adaptation, but counting worsens to 2.934 MAE from 2.540 and remains above Gemini-2.5-Flash’s 1.366.The best model still recalls only 23.39% of instances at IoU 0.5 and 10.41% at IoU 0.75.

6. Discussion

WADE exposes a persistent gap between plausible object descriptions and accurate spatial grounding, with prompting and reasoning guidance failing to ensure exhaustive localization. Fine-tuning improves multiple grounding outcomes, but judged reasoning quality can exceed box quality and the benchmark remains far from saturated.

  • Discussion: Zero-shot models may generate approximately correct box counts while achieving limited overlap with annotated instances, indicating inaccurate localization beyond taxonomy confusion alone.Relaxing class-aware matching yields only limited improvement.
  • Discussion: Two-shot demonstrations generally decrease performance, showing that output-format examples cannot compensate for inadequate visual grounding.Prompting does not resolve the grounding limitation.
  • Discussion: Reasoning-guided prompting improves Qwen3-VL-2B precision and hallucination control but reduces recall and generated boxes, making predictions more selective yet less exhaustive.This precision–recall trade-off is undesirable for dense environmental monitoring, where exhaustive enumeration is essential.
  • Discussion: Fine-tuned Qwen3-VL-2B simultaneously improves precision, recall, strict-IoU recall, and hallucination control, outperforming the strongest evaluated commercial model despite its compact size.The experiment jointly supervises bounding boxes, labels, and class-level reasoning chains.
  • Discussion: LLM judges and human raters score generated reasoning higher than bounding-box quality, so fluent explanations cannot substitute for geometric metrics and qualitative inspection.Models may articulate plausible visual cues or contrastive rules despite inaccurate predicted coordinates.
  • Discussion: 23.39% of instances are recalled at IoU 0.5 and 10.41% at IoU 0.75 by the best system, establishing a baseline with substantial room for improvement.Suggested directions include small-object representation, resolution-aware encoding, dense-instance enumeration, localization-aware multimodal training, dedicated detectors, and geographically disjoint evaluation.

7. Limitations

WADE’s current findings are limited by unisolated supervision effects, restricted geographic coverage, scene-invariant reasoning annotations, and potential evaluator bias. The study also lacks formal inter-annotator agreement reporting and specialized detector comparisons.

  • Supervision and baselines: Reasoning annotations’ contribution cannot be isolated because box-and-label-only fine-tuning was not evaluated, and specialized detectors such as YOLO or DETR were not compared.The fine-tuning experiment jointly supervises bounding boxes, category labels, and class-level reasoning chains.
  • Geographic generalization: All collection sites are in rural Bangladesh, so the location-disjoint split does not establish generalization to other countries, urban drainage systems, coastal environments, or capture platforms.Images from the same pond, canal, or river site cannot occur across different partitions.
  • Annotation scope: One scene-invariant reasoning chain per category omits instance-specific deformation, severe occlusion, and scene-dependent ambiguity; 95% of boxes and labels were accepted without modification, but no formal inter-annotator agreement coefficient was reported.The annotation design provides consistent and scalable supervision but does not capture instance-specific factors.
  • Evaluation: Evaluator bias may affect judged scores because the judge panel overlaps with validation models and both human raters are project-team members.Judged scores are therefore treated as supporting comparative evidence rather than absolute measures, while commercial-system results may change as APIs are updated.

8. Conclusion

WADE is introduced as a benchmark for dense floating-waste grounding with compact vision-language models. It combines richly annotated field imagery with naturally crowded and visually ambiguous waterway scenes, and evaluates six VLMs.

  • Dataset and annotations: WADE contains 2,167 field images and 13,608 annotated instances across ten waste categories.Each instance includes a bounding box, category label, and associated class-level reasoning chain.
  • Dataset and annotations: Each waste instance is annotated with a bounding box, category label, and class-level reasoning chain.The reasoning chain accompanies the spatial and categorical annotations.
  • Benchmark conditions: The dataset captures crowded waterway scenes containing small objects, occlusion, reflections, vegetation, and substantial visual ambiguity.These naturally occurring conditions characterize the benchmark’s dense floating-waste grounding setting.
  • Evaluation: The evaluation covers six compact vision-language models.The passage identifies six VLMs as the evaluated model set.

A. Extended WADE Dataset Details … C.4. Box Matching and Detection Metrics

WADE combines geographically diverse, naturally varied floating-waste imagery with validated object-level annotations, class-level reasoning chains, site-disjoint splits, and standardized VLM evaluation protocols. Its experiments specify multiple prompting and QLoRA regimes, while detection, counting, and hallucination metrics address complementary failure modes.

  • A.1. Collection Conditions: 2,167 images span approximately 500 ponds, canals, and rivers across 390 rural-Bangladesh locations, with smartphone capture reflecting practical monitoring conditions.Most originals were captured at 3096 × 4128 pixels.
  • A.1. Collection Conditions: 1,000 winter and 1,167 monsoon images, plus 680 morning, 950 midday, and 537 low-light images, expose models to reflections, turbidity, occlusion, and weak boundaries.No waste was introduced, repositioned, or staged for photography.
  • A.2. Taxonomy Design: The taxonomy preserves visually confusable biological materials and separates industrial from general solid waste, while retaining natural class imbalance.Water hyacinth and plastic bottles are most frequent; foam waste and wood debris are less frequent, without removal or synthetic resampling.
  • B.1. Bounding-Box Protocol: Annotators tightly boxed every identifiable target, including small, occluded, and overlapping objects, recording normalized coordinates, labels, and reasoning chains; 95% passed review unchanged.Insufficiently visible objects were excluded, and reviewers checked boxes, categories, and image-level attributes.
  • B.2. Reasoning-Chain Schema: Each of ten categories has a validated nine-field reasoning chain encoding shared discriminative knowledge, with fallback cues supporting partially occluded objects.Fallback cues include a visible cap, rigid curved edge, or cylindrical outline.
  • B.3. Reasoning Validation: Three language models scored each reasoning chain from 0 to 10; acceptance required all scores ≥8 and a mean ≥9, with failed chains revised and resubmitted.This scalable quality-control procedure does not replace human evaluation.
  • B.4. Split Construction: Site-grouped partitioning yields 1,625 training, 217 validation, and 325 test images under an approximately 75/10/15 ratio, preventing image- and site-level overlap.All images and annotations from each pond, canal, or river remain in one partition.
  • C.1. Prompt and Serialization; C.2. Evaluation-Regime Details; C.3. QLoRA Configuration; C.4. Box Matching and Detection Metrics: All regimes require fixed JSON entries with identifiers, labels, normalized boxes, and reasoning chains, varying only conditioning through zero-shot, two-shot, reasoning-guided, or QLoRA settings.QLoRA freezes the vision encoder and backbone, updates language-model adapters, and trains on serialized ground-truth JSON responses; evaluation uses IoU matching, precision/recall/F1, count ratio, MAE, RMSE, CHAIRi, and CHAIRs.

C.5. Judge-Based and Human Evaluation

The section combines anonymized LLM-as-judge scoring with external human comparative evaluation of model predictions. Both protocols assess boxes, reasoning, and hallucination control, but provide reference-based or relative evidence rather than absolute visual or geometric correctness.

  • LLM-as-judge evaluation: LLM judges receive predicted and ground-truth JSON without raw images or model identities, enabling anonymized reference-based scoring.Gemini-3.5-Flash, GPT-5.4, and Claude-Opus-4.8 serve as separate judges.
  • LLM-as-judge evaluation: 0 to 10 scores measure box agreement, reasoning consistency, and hallucination control.Scores are averaged across test images and then across the three judges, with no acceptance threshold.
  • Evaluation interpretation: Human assessments use visible source content without ground truth, so they represent external comparative visual evaluations rather than absolute geometric accuracy.The LLM-based scores are likewise treated as comparative evidence alongside geometric grounding metrics because judges lack raw images.
  • Human comparative evaluation: Seven anonymized model–regime conditions are compared by two external evaluators using source images and predictions, excluding few-shot outputs.The conditions include zero-shot outputs from five models plus reasoning-guided and fine-tuned Qwen3-VL-2B.
  • Human comparative evaluation: 0 to 10 human scores assess relative box quality, reasoning quality, and hallucination control across the anonymized predictions.The final score for each model–regime condition is averaged across the two evaluators.

D. Judge and Human Evaluation Results

The fine-tuned Qwen3-VL-2B achieves the strongest overall performance, receiving the highest scores across all evaluated axes under both LLM-judge and human protocols. The protocols agree on system rankings but differ in what they reward, so judge scores are interpreted comparatively rather than absolutely.

  • The fine-tuned model receives the strongest scores on every evaluation axis under both protocols.
  • 6.8, 8.5, and 7.4 are the LLM panel’s scores for box quality, reasoning quality, and hallucination control, respectively.
  • 7.5, 7.5, and 8.5 are human raters’ corresponding scores for box quality, reasoning quality, and hallucination control.
  • Gemini-2.5-Flash ranks as the strongest zero-shot system, while fine-tuned Qwen3-VL-2B ranks as the strongest system overall.
  • Judge panels favor fluent reasoning, whereas human raters are more sensitive to spatial accuracy and unsupported detections.
Loading 2608.22950v1…