PaperlayerView exact revision on arXiv

Source-linked AI summary

See, Say, and Segment: Teaching LMMs to Overcome False Premises

Tsung-Han Wu, Giscard Biamby, David Chan, Lisa Dunlap, Ritwik Gupta, Xudong Wang, Joseph E. Gonzalez, Trevor Darrell

arXiv:2312.08366v1cs.CV

TL;DR

False-premise queries can make reasoning-segmentation models produce masks for objects absent from an image, while existing pipelines may not provide query-directed feedback. The paper introduces FP-RefCOCO(+/g) and cascading and joint-training approaches; reported results improve detection, feedback, and segmentation under false-premise conditions.

  • Problem

    Reasoning-segmentation models often fail to account for false-premise queries, hallucinating masks, while existing detection-and-segmentation pipelines cannot provide task-directed dialogue or correct erroneous queries.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces FP-RefCOCO(+/g) and develops cascading and jointly trained LMM approaches that detect object presence, provide feedback or corrections, and segment present objects.

    Source: §1 · p. 2

  • Results

    Relative accuracy for detecting false-premise queries improves by up to 55.45%, relative cIoU improves by up to 31.65% over baselines, and feedback is judged helpful up to 67% of the time.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The task treats segmentation as conditional on object presence, with a response or query correction when the object is absent.

    Source: §1 · p. 1

Abstract

from arXiv · show

Current open-source Large Multimodal Models (LMMs) excel at tasks such as open-vocabulary language grounding and segmentation but can suffer under false premises when queries imply the existence of something that is not actually present in the image. We observe that existing methods that fine-tune an LMM to segment images significantly degrade their ability to reliably determine ("see") if an object is present and to interact naturally with humans ("say"), a form of catastrophic forgetting. In this work, we propose a cascading and joint training approach for LMMs to solve this task, avoiding catastrophic forgetting of previous skills. Our resulting model can "see" by detecting whether objects are present in an image, "say" by telling the user if they are not, proposing alternative queries or correcting semantic errors in the query, and finally "segment" by outputting the mask of the desired objects if they exist. Additionally, we introduce a novel False Premise Correction benchmark dataset, an extension of existing RefCOCO(+/g) referring segmentation datasets (which we call FP-RefCOCO(+/g)). The results show that our method not only detects false premises up to 55% better than existing approaches, but under false premise conditions produces relative cIOU improvements of more than 31% over baselines, and produces natural language feedback judged helpful up to 67% of the time.

1. Introduction

False-premise queries expose a gap in reasoning segmentation: models may hallucinate masks for absent objects, while existing pipelines cannot provide useful query-directed feedback. The paper introduces false-premise benchmarks and cascading and jointly trained methods, reporting gains in detection, feedback, and segmentation.

1. Introduction

  • False-premise queries can prompt hallucinated masks, so robust systems should detect object presence, respond when it is absent, and segment only if the request continues.The response can suggest alternatives or correct semantic errors in the query.Source: §1 · p. 1
  • Existing false-premise approaches use cascaded detection and segmentation but cannot interpret references through task-directed dialogue or correct users’ erroneous queries.These pipelines leave users with incorrect assumptions and queries.Source: §1 · pp. 1–2
  • The paper introduces FP-RefCOCO, FP-RefCOCO+, and FP-RefCOCOg, extensions of RefCOCO(+/g) for training and evaluating “see,” “say,” and “segment.”The authors report that instruction fine-tuning can cause catastrophic forgetting of “see” and “say” in open-source LMM-based referring segmentation approaches.Source: §1 · p. 2
  • The proposed methods combine model cascading and joint training so LMMs can detect and segment objects while rejecting or correcting false-premise queries.The integrated model is SESAME (SEe, SAy, segMEnt).Source: §1 · p. 2
  • Cascading models and joint data fine-tuning improve false-premise detection relative accuracy up to 55.45% over a baseline; feedback is judged helpful up to 67%, and robust false-premise training improves relative cIoU up to 31.65% over baselines.Source: §1 · p. 2

2. Related Work

Reasoning segmentation adds reasoning to localized referring segmentation, but existing approaches differ in pixel-level output, false-premise handling, and conversational feedback. The paper positions its approach as enabling false-premise detection and language feedback in open-domain reasoning segmentation.

2. Related Work

  • Reasoning segmentation extends referring segmentation with complex reasoning, localizes object pixels, and is more fine-grained than bounding-box grounding approaches.It differs from semantic, instance, and panoptic segmentation, which address broader pixel-labeling or instance tasks.Source: §2 · p. 2
  • LISA can reason and output segmentation tokens but is trained to produce region outputs even for false-premise queries; X-Decoder and SEEM also struggle with complex reasoning segmentation.X-Decoder and SEEM support pixel-level segmentation and language tokens but focus on multitask performance.Source: §2 · p. 2
  • Prior false-premise work spans visual question answering, image-text matching, image-grounded conversation, tool use, and hallucination detection.Question-answering approaches commonly cascade an answerability detector with a standard answering model.Source: §2 · pp. 2–3
  • Pretrained vision-language measures can assess image-text relevance, but cannot segment objects or provide language feedback.The Figure 3 caption describes an LLM-augmented dataset using similar objects, attributes, and relations to create specific correction ground truth.Source: §2 · p. 3, Figure 3 · p. 3
  • Wang et al. use entity detection, expression parsing, and WordNet-based matching, but their method is restricted to closed-domain tasks with fixed vocabularies.This paper’s approach enables false-premise detection and language feedback in open-domain reasoning segmentation tasks.Source: §2 · p. 3

3. A New Dataset and Benchmark for False Premise Correction

False-premise correction requires contextually plausible negative queries paired with suitable alternatives, whereas randomly sampled negatives may not support that task. FP-RefCOCO(+/g) adds such paired queries to RefCOCO(+/g) and supports evaluation and training across “see,” “say,” and “segment.”

3. A New Dataset and Benchmark for False Premise Correction

  • False-premise strategies that place a classifier before segmentation can restrict LMMs’ ability to engage in diverse, open-domain conversational scenarios.The new task expects models to suggest a more closely matching referring expression when a query describes a missing object.Source: §3 · p. 3
  • FP-RefCOCO(+/g) adds contextually related false-premise queries to original positive expressions and pairs each with a corresponding corrected sentence.The dataset splits mirror RefCOCO, RefCOCO+, and RefCOCOg, with approximately equal numbers of added negative expressions.Source: Table 1 · p. 4, §3 · p. 4
  • Randomly sampled negatives may be implausible in context and lack a direct link to the correct alternative needed for false-premise correction.For a cat on a chair, contextually valid negatives include “a cat under the chair” or “a dog on the chair.”Source: §3 · pp. 3–4
  • The augmentation changes one object, adjective, or relation in a positive expression, using GPT-3.5-turbo to generate context-aligned negatives.After basic cleaning, the dataset has a nearly 1:1 positive-to-negative ratio and retains the original train, validation, and test splits.Source: §3 · p. 4
  • LISA has 0% recall on false-premise queries and predicts masks for all such sentences, reducing segmentation cIoU by approximately 30% versus the original dataset without false premises.The benchmark evaluates “see,” “say,” and “segment” capabilities.Source: §3 · p. 4

4. An LMM that can See, Say, and Segment

The paper introduces a cascading approach and SESAME, a unified model for seeing, saying, and segmenting, to handle false-premise queries while retaining capabilities that segmentation fine-tuning can degrade.

4. An LMM that can See, Say, and Segment

  • The authors identify catastrophic forgetting as a problem: segmentation fine-tuning degrades the seeing and saying abilities present in the original LLaVA model.Segmentation-specialized LISA can produce masks for nonexistent objects and generic responses rather than relevant feedback.Source: §4 · pp. 4–5
  • The cascade checks whether an object is present and responds to the user before passing confirmed queries to a segmentation-focused model.Prompt chaining coordinates the first model’s detection and dialogue with the second model’s segmentation.Source: §4 · p. 5
  • SESAME jointly trains an LMM for seeing and saying with a segmentation backbone, using FP-RefCOCO(+/g), LLaVA VQA, and R-RefCOCO(+/g) data.R-RefCOCO(+/g) trains rejection of nonexistent objects, while LLaVA VQA data helps retain visual-question-answering competence.Source: §4 · p. 5
  • SESAME integrates seeing and saying into the LMM while adding segmentation, rather than relying on an auxiliary branch for true/false-premise responses.The authors report that this joint approach preserves seeing and saying while acquiring segmentation skills.Source: §4 · p. 5
  • Figure 4 contrasts SESAME with LISA [34] on conditional instructions and cases where the user requests no segmentation output.SESAME can handle more complex conditional reasoning and omit a mask when one is not requested.Source: Figure 4 · p. 6

5. Experimental Results

On FP-RefCOCO(+/g), SESAME outperforms the cascade and LISA-7B in detection and correction quality, and achieves the highest reported segmentation cIoU. The experiments also examine false-premise rates, complex instructions, rejection training, and conventional positive-query benchmarks.

5. Experimental Results

  • SESAME’s LLaVA-v1.5-7B backbone and SAM decoder are fine-tuned for 10 epochs using LoRA and the unified training dataset.Training took approximately 24 hours on one DGX A100 80GB GPU; overlapping test and validation images were excluded from training.Source: §5 · p. 6

5.1. Results

  • 79.84% detection accuracy for SESAME exceeds 75.59% for the cascading method and 51.4% for LISA-7B.The authors attribute SESAME’s advantage over the cascade to fine-tuning with a more extensive, balanced collection of positive and negative referring expressions.Source: §5.1 · p. 6
  • CLAIR scores are 0.63, 0.61, and 0.67 for SESAME versus 0.35, 0.42, and 0.55 for the cascading method across the three datasets.The modified CLAIR metric compares a suggested correction with positive referring expressions for the same image referent.Source: §5.1 · p. 6
  • SESAME achieves the highest cIoU scores by skipping masks for false-premise queries, while LISA-7B records the lowest after producing masks for all such queries.The study measures segmentation performance with cIoU.Source: §5.1 · p. 6

5.2. Discussions

  • Across tests varying false-premise query proportions (0% equivalent to RefCOCOg; 50% mirroring FP-RefCOCOg), SESAME’s highest false-premise-query recall was only 67.89%, and false-premise detection remains an improvement opportunity.Increasing false-premise proportions directly impacts downstream segmentation cIoU.Source: §5.2 · p. 8
  • SESAME handles conditional alternative-object segmentation and VQA-only prompts without a mask; it can also suggest and segment a similar instance when the queried concept is absent.The authors suggest this capability could extend to multi-round interactions.Source: Figure 5 · p. 7, §5.2 · p. 8
  • Without R-RefCOCO(+/g), models may make superficial word changes instead of interpreting image context, worsening hallucination and “Say” scores.The ablation describes a false “right bookcase” correction; adding R-RefCOCO(+/g) allows a simple rejection instead of requiring an alternative expression.Source: §5.2 · p. 8, Figure 6 · p. 8
  • On traditional referring-segmentation benchmarks with positive queries, SESAME remains on par with prior methods.The paper reports that joint training does not compromise segmentation ability in this setting.Source: §5.2 · p. 8, Table 4 · p. 8

6. Conclusion

The paper frames false-premise segmentation as a task requiring models to see, say, and segment, and presents FP-RefCOCO(+/g) with cascading and joint-training methods.

6. Conclusion

  • False-premise segmentation requires models to “See,” “Say,” and “Segment” rather than merely produce a mask.The paper names this task False Premise Correction.Source: §6 · p. 8
  • FP-RefCOCO(+/g) is introduced as a dataset for evaluating these skills, alongside cascading and joint-training techniques.SESAME is the paper’s integrated LMM.Source: §6 · p. 8
  • SESAME is reported to improve object-presence detection, user feedback about absent objects or corrected queries, and segmentation of present objects.These are the paper’s stated outcomes for its see, say, and segment capabilities.Source: §6 · p. 8

A. Extension to Reasoning Segmentation Tasks

The extension applies SESAME to reasoning segmentation, where complex queries require reasoning about scene context. Evaluation reports gains in detection and segmentation, with qualitative examples illustrating query correction and a remaining risk of false corrections.

A. Extension to Reasoning Segmentation Tasks

  • The reasoning-segmentation setup pairs randomly selected queries from other images with original image-query pairs, then uses LLaVA to generate plausible question-answer pairs.Training combines reasoning and unified datasets with semantic-segmentation data from ADE20K, COCO-stuff, and LVIS-PACO; semantic false-premise pairs use templates.Source: A. Extension to Reasoning Segmentation… · p. 13, Figure 7 · p. 14
  • The training mixture uses a 10:3:1 allocation across semantic, unified, and augmented reasoning-segmentation data, with true-premise queries sampled nine times more often than false-premise queries.The higher true-premise sampling rate is specified to support segmentation training.Source: A. Extension to Reasoning Segmentation… · p. 13
  • Quantitative evaluation measures existence recall for true and false premises and segmentation cIoU, but excludes “Say” because CLAIR is ill-suited to implicit references.The reported evaluation compares SESAME with LISA and the chained method.Source: A. Extension to Reasoning Segmentation… · p. 13
  • 12 times increase in object detection (See) scores for false premise queries; SESAME’s cIoU exceeds the chained method and LISA baseline by over 7%.The reported comparison concerns reasoning segmentation tasks.Source: A. Extension to Reasoning Segmentation… · p. 13, Table 5 · p. 13
  • Qualitative examples show SESAME denying absent objects, suggesting alternatives when appropriate, and rejecting an irrelevant query without an unnecessary suggestion.The LISA baseline tends to produce masks for false-premise queries and cannot offer alternatives in all cases.Source: A. Extension to Reasoning Segmentation… · pp. 13–14
  • SESAME occasionally produces a corrected premise that is still false, leaving room to improve the accuracy and reliability of its corrections.This is identified as a focus for future work.Source: A. Extension to Reasoning Segmentation… · p. 14

B. More Qualitative Results

Visual comparisons illustrate SESAME’s handling of false assumptions, including query rejection and correction when the actual object is relevant to the presumed one.

B. More Qualitative Results

  • SESAME can reject and, when appropriate, correct queries founded on erroneous assumptions.The described strength is shown in additional visual comparisons for referring segmentation.Source: B. More Qualitative Results · p. 14
  • Correction is especially relevant when the actual object bears some relation to the falsely presumed item in the query.The passage identifies this as a notable setting for the model’s capability.Source: B. More Qualitative Results · p. 14
  • The stated forms of relevance include similarities in objects, attributes, or actions.Source: B. More Qualitative Results · p. 14

C. Our Prompts to the LLM and LMM

The section describes separate prompts for eliciting “see and say” responses and generating false-premise queries. Their examples show SESAME correcting complex queries and handling absent or related instances.

C. Our Prompts to the LLM and LMM

  • The LLaVA-v1.5 prompt elicits “see and say” responses, treating outputs beginning “No, there is no...” as non-existent predictions for scoring.The prompt uses chain-of-thought and was more effective than a simpler segmentation request.Source: C. Our Prompts to the LLM and LMM · p. 14, Table 6 · p. 17
  • The GPT-4 prompt uses chain-of-thought examples to reduce false-premise sentences that stray from the input object or refer to an existing one.The full prompt is identified in Table 7.Source: C. Our Prompts to the LLM and LMM · p. 14, Table 7 · p. 18
  • Qualitative figures show SESAME correcting complex queries, declining unnecessary segmentation, and suggesting masks for similar relevant instances.Figure 8 contrasts its query handling with LISA; Figure 9 illustrates rejection and commonsense alternatives.Source: Figure 8 · p. 15, Figure 9 · p. 16
Loading 2312.08366v1…