PaperlayerView exact revision on arXiv

Source-linked AI summary

SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality

Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, Ranjay Krishna

arXiv:2306.14610v1cs.CVcs.CLcs.LG

TL;DR

Vision-language compositionality benchmarks contain exploitable biases that allow image-blind models to outperform vision-language models. SugarCrepe replaces rule-based negatives with language-model-generated captions and adversarial refinement, revealing that prior improvements were overestimated and current models still lack compositionality.

  • Problem

    Existing compositionality benchmarks contain nonsensical and non-fluent hard negatives that create exploitable distributional gaps.

  • Method

    SugarCrepe uses large language models to generate plausible, fluent hard negatives and adversarial refinement to reduce benchmark biases.

  • Results

    Re-evaluation shows that recent compositionality improvements were hugely overestimated, while current pretrained models still lack compositionality.

  • Takeaways & Limitations

    More innovative techniques are needed to improve vision-language compositionality under less biased evaluation.

  • Takeaways & Limitations

    The study focuses on compositionality benchmarks formulated as image-to-text retrieval and identifies only two human-interpretable dataset biases.

Abstract

from arXiv · show

In the last year alone, a surge of new benchmarks to measure compositional understanding of vision-language models have permeated the machine learning ecosystem. Given an image, these benchmarks probe a model's ability to identify its associated caption amongst a set of compositional distractors. Surprisingly, we find significant biases in all these benchmarks rendering them hackable. This hackability is so dire that blind models with no access to the image outperform state-of-the-art vision-language models. To remedy this rampant vulnerability, we introduce SugarCrepe, a new benchmark for vision-language compositionality evaluation. We employ large language models, instead of rule-based templates used in previous benchmarks, to generate fluent and sensical hard negatives, and utilize an adversarial refinement mechanism to maximally reduce biases. We re-evaluate state-of-the-art models and recently proposed compositionality inducing strategies, and find that their improvements were hugely overestimated, suggesting that more innovation is needed in this important direction. We release SugarCrepe and the code for evaluation at: https://github.com/RAIVNLab/sugar-crepe.

1 Introduction

Vision-language compositionality benchmarks use image-to-text retrieval with compositional distractors, but distributional artifacts make them hackable. SugarCrepe addresses these artifacts and shows that existing improvements were overestimated.

  • Motivation: Recent benchmarks test whether models select an image’s correct caption over a compositional hard negative.The evaluation formulation contrasts descriptions such as “a girl in white facing a man in black” with a compositional distractor.
  • Benchmark vulnerability: A blind model that never sees the image can outperform pretrained vision-language models across these benchmarks.The paper attributes this vulnerability to a distributional gap between positive and hard-negative captions.
  • Benchmark vulnerability: Existing hard-negative generation introduces captions that are implausible or non-fluent, creating exploitable differences from positive captions.Examples include logically invalid descriptions and incomprehensible text produced by rule-based procedures.
  • SugarCrepe: SugarCrepe uses ChatGPT to generate plausible, natural hard negatives and adversarial refinement to reduce remaining biases.The workflow replaces simple rule-based templates and subsamples examples to maximally remove identified shifts.
  • Findings: Re-evaluation shows that data augmentation improvements are hugely overestimated on existing benchmarks and limited on SugarCrepe.The authors also report that current pretrained CLIP models still lack compositionality.

2 Related Work

Prior work introduced compositionality benchmarks and training strategies, while dataset-debiasing research developed methods for more faithful evaluation. SugarCrepe connects these lines by applying adversarial refinement to compositionality benchmarks.

  • Evaluating vision-language compositionality: Recent benchmarks report limited compositional understanding despite vision-language models’ strong downstream-task performance.They test word-order sensitivity and links between objects and attributes.
  • Improving vision-language compositionality: Proposed improvements use additional data, models, losses, or explicit training to distinguish hard negatives from correct captions.The paper notes that apparent benchmark gains may not establish actual compositional acquisition.
  • Debiasing dataset evaluation: Dataset-debiasing work aims to make evaluations more faithful by reducing artifacts that inflate measured capabilities.Adversarial filtering iteratively trains classifiers and removes easy negatives for each instance.
  • Benchmark artifacts: Procedurally generated hard negatives in existing benchmarks often lack logical sense or grammatical fluency.Table 1 summarizes the two artifact types targeted by SugarCrepe.
  • Debiasing dataset evaluation: SugarCrepe uses adversarial refinement to remove biases that arise during compositionality-benchmark generation.This applies the adversarial-filtering idea to generated hard negatives.

3 Current compositionality benchmarks and their biases

Existing image-to-text compositionality benchmarks generate hard negatives with rule-based procedures that introduce nonsensical and non-fluent artifacts. These artifacts let blind text-based models achieve state-of-the-art benchmark performance without visual compositionality.

  • Evaluation setup: Most existing benchmarks formulate compositionality evaluation as selecting the correct text for a given image from candidate descriptions.The hard negatives are intentionally similar to positive captions while differing compositionally.
  • Sources of bias: Rule-based substitutions, word swaps, and shuffling create nonsensical and non-fluent hard negatives.The paper measures plausibility with Vera and fluency with a grammar-check model.
  • Consequences: Blind Vera and Grammar models outperform state-of-the-art CLIP models on nearly all existing benchmarks by exploiting these artifacts.Figure 2 identifies the artifacts as the basis for the benchmarks’ hackability.
  • Consequences: Blind models achieve state-of-the-art performance on 9 out of 10 existing benchmark tasks.The comparison covers 17 pretrained CLIP models from OpenAI, LAION, and Datacomp.

4 SUGARCREPE

SUGARCREPE is a COCO-based benchmark designed to evaluate vision-language compositionality while reducing biases in hard negatives. Its pipeline generates fluent negatives with ChatGPT, removes false negatives through human validation, and applies adversarial refinement across seven fine-grained negative types.

  • SUGARCREPE is a COCO-based benchmark that reduces two identified dataset biases and covers seven fine-grained hard-negative types.The benchmark is designed for faithful evaluation of vision-language compositionality.
  • Generation workflow: ChatGPT generates plausible, natural hard negatives using author-written few-shot demonstrations and instructions to avoid commonsense and grammatical errors.This replaces rule-based procedures that often produce nonsensical or non-fluent text.
  • Generation workflow: Human validation filters generated captions that still correctly describe their corresponding images, preventing false negatives from entering SUGARCREPE.A changed concept can remain valid for an image, so such cases are manually removed.
  • Generation workflow: Adversarial refinement subsamples candidates to reduce exploitable artifacts and balance score gaps between positive and negative texts.The procedure targets residual bias after language-model generation.
  • Hard-negative taxonomy: The taxonomy includes REPLACE, SWAP, and ADD forms, with object, attribute, and relation variants where the resulting negatives remain appropriate.SWAP excludes relationship swaps, while ADD includes only object and attribute additions because relationship additions are often highly implausible.
  • Dataset overview: The final evaluation set contains 7512 image-to-text retrieval examples, each pairing an image and positive text with a hard negative; random chance averages 50%.SUGARCREPE omits SHUFFLE and NEGATE negatives because they are implausible or expose irreducible keyword artifacts.

5 Evaluations

SUGARCREPE is evaluated against existing benchmarks, recent hard-negative training methods, and pretrained CLIP models. The evaluation finds reduced benchmark bias, overestimated gains from NEGCLIP, and substantial remaining compositionality gaps.

  • 5.1 SUGARCREPE significantly reduces dataset biases: SUGARCREPE hard negatives have higher commonsense and grammar quality than ARO+CREPE negatives.Human evaluations find strictly higher quality on an average of 35% of examples, while pairwise comparisons favor SUGARCREPE on 86% of examples on average.
  • 5.1 SUGARCREPE significantly reduces dataset biases: Adversarial refinement centers SUGARCREPE’s Vera and Grammar score-gap distributions around zero, preventing identified artifacts from revealing positive captions.The resulting blind commonsense and grammar attacks do not work, and blind models consistently rank last on SUGARCREPE.
  • 5.2 Re-evaluating recent methods for improving compositionality: NEGCLIP gains are much smaller on SUGARCREPE than on ARO+CREPE, where matching training and testing hard-negative types produces the largest improvements.On ARO+CREPE, gains exceed 10% in 11 of 16 cases and can reach 94% against 95% human performance; no SUGARCREPE improvement exceeds 10%.
  • 5.3 Comprehensive evaluations on existing pretrained vision-language models: The best pretrained CLIP models approach human performance on REPLACE-OBJ but remain more than 10% below humans on nearly all other hard-negative types.DataComp small and medium models show 23% and 43% drops from human performance on REPLACE-OBJ, respectively.
  • 5.3 Comprehensive evaluations on existing pretrained vision-language models: All pretrained CLIP models struggle most with SWAP hard negatives, despite humans easily distinguishing them from positive captions.This difficulty persists regardless of pretraining dataset or model size.
  • 5.3 Comprehensive evaluations on existing pretrained vision-language models: Pretrained CLIP performance on SUGARCREPE positively correlates with ImageNet zero-shot accuracy, with Pearson r = 0.78 and r = 0.75 on SWAP-OBJ and ADD-ATT.The correlation is strong on all other hard-negative types, with r > 0.8.

6 Conclusion

The paper identifies severe biases in existing compositionality benchmarks and introduces SUGARCREPE to address them. Reassessment shows that prior compositionality improvements were substantially overestimated, underscoring the need for further innovation.

  • 6 Conclusion: Existing compositionality benchmarks contain significant biases that allow text-only models to outperform vision-language models.SUGARCREPE addresses these biases with large-language-model-generated negatives and adversarial filtering.
  • 6 Conclusion: Reassessing state-of-the-art models and compositionality-inducing mechanisms reveals a significant overestimation of their reported advancements.The conclusion attributes this reassessment to evaluation with the less biased SUGARCREPE benchmark.

A.1 Limitation and future work

The paper’s scope is limited to image-to-text retrieval benchmarks, two human-interpretable bias types, and contrastively learned vision-language models. Future work is proposed for broader task formulations, artifact detection, and generative models.

  • A.1 Limitation and future work: The study focuses on compositionality benchmarks formulated as image-to-text retrieval, leaving text-to-image retrieval for future work.The authors identify text-to-image retrieval as an alternative formulation explored by earlier efforts.
  • A.1 Limitation and future work: The identified nonsensical and non-fluent biases may not cover all dataset artifacts exploitable by models.Future work may use more sophisticated techniques to remove artifacts beyond human comprehension.
  • A.1 Limitation and future work: Evaluations focus on contrastively learned vision-language models rather than modern generative vision-language models.The authors call for future evaluation and characterization of generative models’ compositionality.

B Implementation details

The experiments use specified hardware, publicly released datasets and model weights, and an OpenCLIP-based evaluation framework. Training details are provided for the internally trained and fine-tuned models.

  • B Implementation details: Experiments run on an Intel Xeon E5-2678 v3 machine with 512G memory and two 48G NVIDIA RTX A6000 GPUs.
  • B Implementation details: Existing datasets are obtained from their original sources, including CREPE, ARO, VL-CheckList, and COCO.The authors refer readers to the original sources for dataset licenses.
  • B Implementation details: Pretrained Vera, Grammar, CLIP, and NEGCLIP weights are obtained from author releases or OpenCLIP.OpenCLIP supplies the pretrained CLIP weights, while Vera, Grammar, and NEGCLIP weights come from their authors’ releases.
  • B Implementation details: RN50 and ViT-B/32 training uses 1000 warmup steps, batch size 256, learning rate 1e-4, weight decay 0.1, and 30 epochs.Training augments the original CLIP loss with hard-negative captions following NEGCLIP.
  • B Implementation details: Evaluations are based on OpenCLIP and use its default evaluation hyperparameters.

C Vision-language compositionality benchmarks

Existing vision-language compositionality benchmarks mainly use image-to-text retrieval with procedurally generated hard negatives, while SUGARCREPE expands and refines this setup. It covers multiple hard-negative forms and uses language-model generation plus adversarial refinement to reduce textual biases.

  • Existing benchmarks: Most existing benchmarks formulate compositionality evaluation as image-to-text retrieval with rule-based hard-negative generation.The benchmarks differ in which hard-negative types they construct.
  • Existing benchmarks: VL-CheckList uses REPLACE negatives, whereas ARO uses SWAP and SHUFFLE negatives to test altered atomic concepts and word order.VL-CheckList further categorizes replacements by objects, attributes, and relationships; ARO categorizes swaps by attributes and relationships.
  • Existing benchmarks: CREPE includes REPLACE, SWAP, and NEGATE negatives, with NEGATE formed by adding words such as “not” or “no” to positive captions.Its REPLACE and SWAP constructions follow the procedures used in VL-CheckList and ARO.
  • Text-to-image evaluation: Text-to-image compositionality benchmarks remain smaller because hard-negative images are more difficult to obtain than hard-negative texts.Winoground contains two images and two captions with identical words in different orders, while Cola contains 210 examples mined from existing datasets.
  • SUGARCREPE: SUGARCREPE organizes hard negatives into REPLACE, SWAP, and ADD forms, subdivided by altered objects, attributes, or relations, for seven fine-grained types.Its image-to-text formulation enables a larger-scale evaluation set, while SHUFFLE and NEGATE are omitted because they introduce inevitable biases.
  • SUGARCREPE: SUGARCREPE generates hard negatives with ChatGPT prompts and applies adversarial refinement so grammar and plausibility models cannot identify positives above random-guess expectations.REPLACE and ADD use three-step prompting, while SWAP uses a single prompt when same-category concepts can be exchanged; refinement balances symmetric score-difference regions.

D.4 Dataset information

SUGARCREPE is documented as an image-to-text benchmark in which models choose a correct positive caption over a minimally compositional hard negative. Its dataset records image and caption fields, and its evaluation exposes severe weaknesses in existing benchmarks.

  • Dataset documentation: SUGARCREPE requires selecting the positive text describing an image against a hard-negative text differing through small compositional changes.Each example contains an image identifier, a positive caption, and an incorrect negative caption.
  • Dataset documentation: The dataset documentation defines filename as the image identifier, caption as the correct positive text, and negative_caption as the incorrect hard-negative text.
  • Maintenance and licensing: The maintainers state that they monitor repository issues and will address technical problems.
  • Maintenance and licensing: The work uses the MIT License, and its source data comes from publicly released prior work.
  • Author statement: The authors state that they bear responsibility for rights violations.
  • Evaluation context: Blind text-only models outperform most pretrained CLIP models on many existing benchmark tasks, indicating that those benchmarks do not faithfully measure vision-language compositionality.The comparison covers 17 pretrained CLIP models and two text-only models.

E.2 SUGARCREPE human evaluation

The human evaluation compares hard negatives from ARO+CREPE with SUGARCREPE on sampled examples. It records which negative is better or whether the pair ties according to commonsense and grammatical correctness.

  • Evaluation design: The evaluation randomly samples 100 examples for each hard-negative type: REPLACE, SWAP, and NEGATE / ADD.Each comparison includes the original positive text and hard negatives from ARO+CREPE and SUGARCREPE.
  • Evaluation criteria: Human judgments count which hard negative is better or tied with respect to commonsense and grammatical correctness.Table 9 reports the counts out of 100 sampled examples.
Loading 2306.14610v1…