PaperlayerView exact revision on arXiv

Source-linked AI summary

Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation

Jiazhen Liu, Mingkuan Feng, Long Chen

arXiv:2608.02791v1cs.CV

TL;DR

MLLM-based segmentation must balance segmentation quality, dialogue ability, and inference efficiency, while broader tasks require predictions that preserve target identities. This paper introduces Structured All-Mask Prediction and STAMPlus, which jointly predicts identified targets in one pass, maintaining strong segmentation and dialogue performance across heterogeneous settings with efficient mask prediction.

  • Problem

    MLLM segmentation faces a trilemma among segmentation performance, dialogue generality, and inference efficiency, while multi-target tasks require explicit semantic or instance identities.

  • Method

    STAMPlus decouples autoregressive dialogue from non-autoregressive mask prediction by binding generated target identities to a shared structured multi-class mask space.

  • Results

    Across referring, reasoning, remote-sensing small-target, open-vocabulary semantic, and instance-aware segmentation, STAMPlus maintains strong segmentation and dialogue performance with efficient single-pass prediction.

  • Takeaways & Limitations

    STAMPlus extends trilemma resolution from single-target segmentation to broader heterogeneous segmentation settings using one unified checkpoint.

Abstract

from arXiv · show

MLLM-based segmentation faces a core segmentation trilemma: high segmentation performance, preserved dialogue ability, and fast inference. Embedding-prediction methods may disrupt language modeling through pixel-level objectives, whereas next-token generation is inefficient for dense masks. We propose All-Mask Prediction, decoupling autoregressive dialogue from non-autoregressive mask prediction. Its binary instantiation, STAMP (Simultaneous Textual All-Mask Prediction), emits an in-vocabulary <SEG> trigger, fuses image-aligned mask tokens with corresponding patch features, and uses hybrid attention to classify all tokens as foreground or background in one pass. It thereby combines strong referring and reasoning segmentation with preserved multimodal ability and efficient inference. However, binary masks cannot retain multiple semantic or instance identities without repeated target-specific predictions. We therefore propose Structured All-Mask Prediction and develop STAMPlus. It generates a target list with explicit IDs and optional boxes, binds these IDs to a shared multi-class mask space, and jointly predicts all targets in one non-autoregressive pass. A single unified checkpoint retains STAMP's referring and reasoning capabilities while extending to open-vocabulary semantic, instance-aware, and remote-sensing small-target segmentation, where high-resolution mask-token scaling preserves finer spatial evidence. Across these settings, STAMPlus achieves state-of-the-art segmentation performance, preserves general multimodal instruction following, and reduces 12-category latency from 13.50s for repeated STAMP inference to 5.16s. Further analyses show that accurate target cues improve segmentation and learned spatial grounding benefits look-twice reasoning. Overall, STAMPlus resolves the trilemma beyond single-target prediction.

I. INTRODUCTION

The introduction frames MLLM segmentation as a trilemma among segmentation performance, dialogue ability, and inference speed, then presents Structured All-Mask Prediction and STAMPlus as a unified extension beyond binary single-target masks. STAMPlus supports broader structured segmentation settings while retaining efficient mask prediction and multimodal capabilities.

  • Motivation: MLLM segmentation must balance high performance, preserved dialogue ability, and fast inference, but existing paradigms compromise at least one of these goals.Embedding prediction may harm dialogue, while next-token prediction trades segmentation quality against autoregressive inefficiency.
  • All-Mask Prediction: All-Mask Prediction decouples autoregressive dialogue generation from non-autoregressive mask generation to address this trilemma.The model first generates dialogue and then predicts image-aligned mask placeholders in a single forward pass.
  • STAMP: STAMP resolves the trilemma for single-target referring and reasoning segmentation through binary foreground/background prediction after a textual <SEG> trigger.Its mask phase directly processes prefilled image-aligned [MASK] tokens and classifies them with a binary head.
  • Limitation of Binary Masks: Binary-mask methods remain ill-suited to multi-target tasks because repeated target-specific predictions cannot preserve multiple semantic categories or instance identities in a shared output.Broader settings include open-vocabulary semantic, instance, and panoptic segmentation.
  • STAMPlus: STAMPlus generalizes All-Mask Prediction to structured multi-target masks and supports referring, reasoning, open-vocabulary semantic, instance-aware, and remote-sensing small-target segmentation with one unified checkpoint.It retains efficient single-pass mask prediction, general dialogue ability, and strong segmentation performance without task-specific fine-tuning.

II. RELATED WORK · A. Multimodal Large Language Models (MLLMs)

MLLMs advance instruction-driven vision through language models with strong reasoning and instruction-following abilities, typically connected to visual encoders through lightweight modality interfaces. MLLM-based segmentation must convert spatially aware understanding into accurate dense masks while preserving dialogue ability and efficient inference.

  • A. Multimodal Large Language Models (MLLMs): MLLMs are powered by the advanced reasoning and instruction-following abilities of their underlying language models.
  • A. Multimodal Large Language Models (MLLMs): Most MLLMs connect a visual encoder to a pre-trained language model through lightweight modality adapters, query transformers, or cross-attention modules.
  • A. Multimodal Large Language Models (MLLMs): Flamingo, InstructBLIP, LLaVA, and Qwen-VL are representative architectures for instruction-driven multimodal modeling.
  • A. Multimodal Large Language Models (MLLMs): MLLM-based segmentation must translate high-level, spatially aware understanding into accurate dense masks.
  • A. Multimodal Large Language Models (MLLMs): This translation must preserve general dialogue ability and efficient inference.
  • A. Multimodal Large Language Models (MLLMs): Existing segmentation methods mainly predict task-specific embeddings for an external mask decoder or follow another strategy described in the related work.The decoder-oriented approach treats the MLLM as a powerful vision-language encoder and uses task-specific embeddings to control the external decoder.

B. MLLM-Based Segmentation · C. Perception for Multimodal Understanding · III. METHOD

The paper contrasts embedding- and next-token-based MLLM segmentation paradigms, then introduces STAMPlus as a native dense-perception interface that extends STAMP’s single-target binary prediction to structured multi-target segmentation. Its two-phase design generates target structure autoregressively and predicts all mask labels jointly in one non-autoregressive pass, while investigating whether learned spatial grounding supports multimodal reasoning.

  • B. MLLM-Based Segmentation: MLLM segmentation methods primarily differ in whether they output continuous embeddings for an external decoder or discrete tokens defining the mask.Embedding Prediction uses an [SEG]-prompted external SAM-like decoder, whereas Next-Token Prediction keeps segmentation within the language-modeling interface.
  • B. MLLM-Based Segmentation: Embedding Prediction uses an [SEG] embedding to localize the target semantically while an external decoder produces the dense mask in O(1) time for one target.This paradigm was pioneered by LISA and subsequently refined by other works.
  • B. MLLM-Based Segmentation: Next-Token Prediction avoids objective conflicts and decoder dependence by generating discrete mask tokens, but sparse polygon representations can be fragile to coordinate or ordering errors.The supplied passage identifies VisionLLM as an early polygon-coordinate approach and notes later CoT-based efforts to improve robustness.
  • C. Perception for Multimodal Understanding: Preliminary perception can provide intermediate evidence for multimodal understanding by grounding visual entities with boxes or spatial coordinates for language reasoning.The passage describes discrete perceptual supervision as a way to localize visual entities and reason over them in language.
  • C. Perception for Multimodal Understanding: STAMPlus treats segmentation as a native dense-perception interface, supporting joint multi-target masks while testing whether structured spatial grounding benefits downstream multimodal understanding.It produces unified semantic- or instance-aware mask maps in a single pass and examines reuse of segmentation-induced spatial representations in reasoning.
  • III. METHOD: STAMP separates dialogue generation from segmentation: after autoregressively emitting <SEG>, it predicts every image-aligned mask token’s binary foreground/background label in one non-autoregressive pass.Each query-defined target is represented by one binary foreground/background mask.
  • III. METHOD: STAMPlus retains STAMP’s <SEG> trigger, mask-token prefilling, hybrid attention, and single-pass prediction while adding explicit target IDs and dynamic binding to a shared multi-class label space.The structured target list is generated in Phase 1, and the IDs are bound to the shared mask space in Phase 2.

A. STAMP: Binary All-Mask Prediction … B. STAMPlus: Structured All-Mask Prediction

STAMP separates autoregressive dialogue and target identification from non-autoregressive binary mask prediction, using cached context, spatially aligned mask tokens, and hybrid attention. STAMPlus extends this mechanism to instruction-defined structured label maps for simultaneous multi-target segmentation.

  • A. STAMP: Binary All-Mask Prediction: STAMP autoregressively generates a contextual response and can emit an in-vocabulary <SEG> token identifying segmentation targets.A ViT extracts patch features, which are prepended to text embeddings before multimodal processing.
  • A. STAMP: Binary All-Mask Prediction: STAMP caches dialogue-history key-value states for each <SEG> token, avoiding redundant computation when starting mask prediction.Each cache is paired with the token’s context-specific dialogue history.
  • 2) All-Mask Prediction (Phase 2):: After <SEG> emission, STAMP appends one [MASK] placeholder per image patch and predicts the entire mask in a single non-autoregressive forward pass.Each placeholder is initialized with its corresponding patch feature and positional embedding.
  • 2) All-Mask Prediction (Phase 2):: Hybrid attention preserves causal attention for dialogue history while allowing mask placeholders to attend bidirectionally across the full context.This lets all mask tokens interact with the dialogue and one another during prediction.
  • 1) Dialogue Generation (Phase 1):: STAMP classifies each patch as foreground or background from mask-token hidden states after loading cached dialogue KV states in one forward pass.The output can optionally be refined into a high-resolution mask using sampled keypoints and a frozen SAM decoder.
  • B. STAMPlus: Structured All-Mask Prediction: STAMPlus addresses simultaneous multi-target segmentation by replacing binary foreground/background outputs with an instruction-defined structured label map.This avoids repeatedly invoking binary mask prediction for different targets that must be distinguished.
  • B. STAMPlus: Structured All-Mask Prediction: STAMPlus inherits STAMP’s core all-mask mechanism, including the <SEG> trigger and mask-token prefill, while extending the output space beyond binary masks.The supplied passage frames this extension as the basis for structured multi-target prediction.

1) Structured Target Generation (Phase 1): · 2) Structured All-Mask Prediction (Phase 2): · 3) High-Resolution Mask-Token Scaling:

STAMPlus structures Phase 1 as an instruction-conditioned ID-to-target mapping and uses that mapping in Phase 2 for dynamic multi-class, single-pass mask prediction. High-resolution scaling increases aligned mask-token budgets for remote-sensing small targets to preserve finer spatial evidence.

  • 1) Structured Target Generation (Phase 1):: The target list adapts to requested granularity, supporting single-target, multi-category, and multi-instance segmentation with IDs and optional boxes for disambiguation.The list uses fixed labels for single-target segmentation and different cls values for distinct categories or instances.
  • 1) Structured Target Generation (Phase 1):: STAMPlus autoregressively emits <SEG> after generating a compact, machine-readable JSON-style target list before Phase 2.Each entry maps a numerical cls ID to a textual target name, with class 0 reserved for background and positive IDs identifying generated targets.
  • 1) Structured Target Generation (Phase 1):: This design preserves STAMP’s autoregressive generation and <SEG> trigger while replacing free-form binary specifications with instruction-conditioned, parseable ID-to-target mappings.The mapping dynamically defines the label space used by Phase 2 and enables broader segmentation within one framework.
  • 2) Structured All-Mask Prediction (Phase 2):: Phase 2 preserves STAMP’s mask-token prefilling, visual fusion, KV-cache reuse, hybrid attention, and one-pass non-autoregressive prediction, but uses a multi-class head with capacity 200.Samples containing more than 199 target entries exceed the stated implementation capacity.
  • 2) Structured All-Mask Prediction (Phase 2):: The generated ID-to-target mapping remains in context through shared dialogue history and KV-cache reuse, conditioning mask tokens on the instruction-specific label space.Mask tokens assigned the same ID are grouped into the corresponding target region, producing a mask map aligned with the Phase 1 list.
  • 2) Structured All-Mask Prediction (Phase 2):: STAMPlus makes Phase 1 an active guide for Phase 2: its structured target list defines the label space used to produce the structured mask map.Unlike STAMP’s primarily trigger-and-context role, the target list is an important prediction context rather than merely auxiliary text.
  • 3) High-Resolution Mask-Token Scaling:: High-resolution mask-token scaling addresses small-target degradation caused by fixed-grid resizing and aggressive downsampling that can leave targets represented by very few patches.The method scales input resolution and the aligned token budget together to retain finer spatial evidence.
  • 3) High-Resolution Mask-Token Scaling:: For ordinary natural images, STAMPlus uses N = 1024–1280 image-aligned mask tokens, while remote-sensing small-target inputs use N = 2560–3200.The higher resolution retains more patch-level evidence, and the larger aligned token budget preserves finer sampling in the predicted mask grid.

C. Training

STAMPlus and STAMP are trained with a unified text-generation and mask-prediction objective. Structured target lists establish the ID-to-target mapping, while mask losses align predicted tokens with binary or structured labels.

  • Unified objective: Both models jointly optimize autoregressive text generation and mask prediction losses.The text objective uses standard cross-entropy language modeling, while the mask objective supervises mask-token logits.
  • Structured target training: STAMPlus uses JSON-style target lists before <SEG> without additional structured-generation supervision.The same autoregressive text loss trains parseable outputs and instruction-specific ID-to-target mappings for Phase 2.
  • Mask supervision: STAMP supervises each mask token with binary foreground/background classification using Binary Cross-Entropy and Dice losses.Each token predicts a foreground probability against a binary ground-truth label.
  • Mask supervision: STAMPlus converts ground-truth masks into ID-based patch labels and trains mask tokens to predict background or instruction-defined foreground IDs.Its structured mask objective combines multi-class cross-entropy with class-wise Dice loss across segmentation channels, including background.
  • Unified formulation: STAMP is a binary special case of STAMPlus, and joint optimization aligns Phase 1 target specifications with Phase 2 structured mask maps.The shared training framework teaches the model when to invoke mask prediction and how to map generated targets to predicted segmentation channels.

IV. EXPERIMENTS

The experiments evaluate STAMP and STAMPlus progressively across single-target, multi-target, interaction, general-ability, efficiency, and ablation settings. Together, these tasks assess segmentation performance, dialogue compatibility, and inference efficiency across broader capabilities.

  • General Ability, Efficiency, and Ablations: General multimodal ability, inference efficiency, design choices, and All-Mask Prediction generalization are evaluated to mirror the segmentation trilemma.The task-specific experiments assess broader segmentation, multimodal benchmarks assess dialogue compatibility, and latency studies assess efficiency.
  • Single-Target Referring and Reasoning Segmentation: STAMP and STAMPlus are evaluated first on single-target referring and reasoning segmentation, including natural-image benchmarks and remote-sensing images with small targets.The remote-sensing evaluation is presented in Section IV-B.
  • Open-Vocabulary Semantic and Instance-Aware Segmentation: STAMPlus is evaluated for distinguishing multiple semantic categories or object instances in a single prediction.This open-vocabulary semantic and instance-aware evaluation appears in Section IV-C.
  • Segmentation–Understanding Interaction: The study tests whether better Phase 1 target descriptions improve Phase 2 segmentation and whether segmentation-learned spatial grounding supports downstream visual understanding.These interaction analyses are described in Section IV-D.

A. Common Setup · B. Single-Target Referring and Reasoning Segmentation · 1) Natural Images:

The paper establishes shared STAMP/STAMPlus implementation choices and evaluates whether STAMPlus preserves STAMP’s single-target referring and reasoning segmentation capabilities. Natural-image experiments cover standard, generalized, and reasoning-based language-guided segmentation under binary-mask protocols.

  • A. Common Setup: STAMP and STAMPlus use Qwen2-VL-based 2B and 7B variants, with an additional LLaVA variant for backbone-transfer analysis.Both models share mask tokens, hybrid attention, optional SAM-H refinement, and a linear mask head.
  • A. Common Setup: STAMP classifies mask-token states into foreground and background, whereas STAMPlus maps them to 200 structured classes.This shared linear mask-head design distinguishes the binary and structured prediction spaces.
  • A. Common Setup: Natural-image experiments use 1024–1280 image-aligned [MASK] tokens, while remote-sensing experiments assess high-resolution mask-token scaling for small targets.The supplied implementation passage states the natural-image token range and the evaluation passage identifies the remote-sensing purpose.
  • B. Single-Target Referring and Reasoning Segmentation: The evaluation first tests whether STAMPlus retains the tasks for which STAMP was originally developed.These tasks use binary target masks, merging multiple referred regions into one foreground class.
  • B. Single-Target Referring and Reasoning Segmentation: The single-target evaluation includes natural images and remote-sensing images with small targets.Remote-sensing cases specifically evaluate the proposed high-resolution mask-token scaling.
  • 1) Natural Images:: Natural-image evaluation spans RefCOCO-family referring segmentation, gRefCOCO generalized referring segmentation, and ReasonSeg reasoning segmentation.All datasets are evaluated using their standard protocols.
  • 1) Natural Images:: RefCOCO, RefCOCO+, and RefCOCOg provide language-guided target-segmentation benchmarks with progressively different expression characteristics.RefCOCO+ removes location words, while RefCOCOg uses longer, more descriptive expressions.
  • 1) Natural Images:: ReasonSeg requires inferring the intended target from reasoning-intensive language rather than a direct referring expression.Following LISA, the STAMP-2B RES checkpoint is further trained for one epoch on RefCOCO and 239 ReasonSeg training images.

2) Remote-Sensing Small-Target Segmentation: · C. Open-Vocabulary and Instance-Aware Segmentation · 1) Open-Vocabulary Semantic Segmentation:

STAMPlus extends All-Mask Prediction to demanding remote-sensing small-target segmentation and open-vocabulary semantic segmentation. High-resolution mask-token scaling preserves patch-level evidence, while unified STAMPlus checkpoints support distinguishable semantic targets and achieve average mIoUs of 61.3 and 63.7 on open-vocabulary benchmarks.

  • 2) Remote-Sensing Small-Target Segmentation:: Remote-sensing imagery stresses segmentation because large spatial coverage makes referred objects occupy only a tiny fraction of pixels.Resizing natural-image inputs may collapse small objects to very few visual patches.
  • 2) Remote-Sensing Small-Target Segmentation:: High-resolution mask-token scaling retains more patch-level evidence around small objects by pairing each retained image patch with an image-aligned mask token.The enlarged patch grid therefore receives a larger mask-token budget.
  • C. Open-Vocabulary and Instance-Aware Segmentation: STAMPlus enables semantic categories and object instances to remain distinguishable within a single prediction, unlike merged single-foreground evaluations.The same unified STAMPlus checkpoints are used for these new capabilities and the preceding experiments.
  • 1) Open-Vocabulary Semantic Segmentation:: STAMPlus is trained with dense semantic supervision from COCO-Stuff and evaluated for open-vocabulary transfer on ADE20K-150, Pascal Context-59, and Pascal VOC-20.Performance is measured by mIoU on each benchmark, comparing specialized segmentors and MLLM-based segmentation methods.
  • 1) Open-Vocabulary Semantic Segmentation:: 61.3 and 63.7 are the average mIoUs obtained by STAMPlus-2B and STAMPlus-7B, respectively, across the reported open-vocabulary benchmarks.These benchmarks vary in semantic vocabularies and scene distributions beyond the COCO-Stuff training taxonomy.
  • 1) Open-Vocabulary Semantic Segmentation:: The 7B model achieves the strongest results on ADE20K-150 and PaThe supplied passage ends mid-sentence after identifying the ADE20K-150 result.

2) Instance-Aware Segmentation: · D. Segmentation–Understanding Interaction · 1) Phase 1-Assisted Segmentation:

STAMPlus uses structured target IDs to support reasoning-intensive instance-aware segmentation, while its two-phase interface exposes editable target descriptions whose accuracy strongly affects Phase 2 mask prediction. Human-provided target boxes raise STAMPlus-2B average cIoU from 77.4 to 89.1, demonstrating the benefit of more precise spatial cues.

  • 2) Instance-Aware Segmentation:: Instance-aware segmentation requires distinguishing individual objects, including same-category instances, and is evaluated on MUSE using reasoning-intensive instructions.The evaluation tests whether structured target IDs support instance-level reasoning and separate prediction.
  • D. Segmentation–Understanding Interaction: The interaction analyses test whether precise Phase 1 target descriptions improve Phase 2 segmentation and whether spatial grounding supports downstream visual understanding.These experiments examine the broader value of extending STAMP from single-target binary masks to structured multi-target prediction.
  • D. Segmentation–Understanding Interaction: During Phase 1, STAMPlus generates each target’s textual label and ID, with an optional bounding box indicating its location.The model-generated boxes were used in preceding segmentation experiments.
  • 1) Phase 1-Assisted Segmentation:: The two-phase interface permits a human-provided bounding box to augment the Phase 1 target specification before Phase 2, analogous to a SAM box prompt.This controlled setup isolates how spatial target-description accuracy affects subsequent mask prediction.
  • 1) Phase 1-Assisted Segmentation:: 89.1 average cIoU is achieved by STAMPlus-2B with target boxes, versus 77.4 without them.Table VIII(a) reports the improvement under human-provided target cues.
  • 1) Phase 1-Assisted Segmentation:: The substantial cIoU gain shows that Phase 1 description quality has a major impact on Phase 2 segmentation.The structured interface also enables future reinforcement-learning optimization or specialized localization assistance before mask prediction.
  • 1) Phase 1-Assisted Segmentation:: STAMPlus exposes an explicit, editable intermediate target description, enabling modular improvements whose spatial cues can directly benefit Phase 2.The proposed enhancements include reinforcement-learning-based optimization and assistance from specialized localization tools.

2) Segmentation-Assisted Understanding: … 2) Efficiency Analysis:

STAMPlus extends segmentation capabilities while preserving multimodal understanding and improving efficiency. Segmentation-trained grounding benefits look-twice reasoning, mixed training retains dialogue performance, and shared multi-target prediction scales better than repeated target-specific inference.

  • 2) Segmentation-Assisted Understanding:: Segmentation-trained grounding is tested through attention-guided second-pass answering rather than feeding discrete predicted masks into visual question answering.The protocol uses first-pass attention to derive question-relevant regions and preserve continuous spatial evidence.
  • 2) Segmentation-Assisted Understanding:: 71.1 average score is achieved by STAMPlus-2B across four multimodal benchmarks under look-twice evaluation.This exceeds Qwen2-VL-2B by 2.6 points, while Qwen2-VL-2B itself improves from 66.4 to 68.5 with look-twice reasoning.
  • E. General Properties and Analysis: The evaluation jointly examines segmentation accuracy, task coverage, multimodal ability, and efficiency across STAMP and STAMPlus.Covered settings range from referring and reasoning segmentation to remote-sensing small-target, open-vocabulary, and instance-aware prediction.
  • 1) General Multimodal Ability:: STAMP-2B retains multimodal performance close to Qwen2-VL-2B when segmentation and visual-instruction data are mixed during training.The evaluation combines RefCOCO-family segmentation with MMMU, MMBench, MMStar, ScienceQA, TextVQA, and VizWiz.
  • E. General Properties and Analysis: Table VIII frames the segmentation–understanding interaction through human-provided boxes for referring segmentation and segmentation-derived spatial grounding for look-twice reasoning.These analyses connect target specification and spatial grounding with segmentation and multimodal understanding.
  • 1) General Multimodal Ability:: STAMPlus-2B maintains strong multimodal results while improving average RES performance, extending STAMP without sacrificing dialogue compatibility.The structured multi-target extension therefore preserves general multimodal instruction following.
  • 2) Efficiency Analysis:: STAMP offers a favorable accuracy–latency trade-off, with mask-generation latency comparable to efficient embedding-prediction methods and below autoregressive next-token approaches.Its resolution can also be reduced to lower latency with only a small accuracy decrease and without retraining.
  • 2) Efficiency Analysis:: STAMPlus generates multiple target identities in one shared Phase 2 forward pass, scaling substantially better than repeated target-specific prediction.This establishes the efficiency component of the segmentation trilemma for structured multi-target prediction.

3) Ablation and Generalization Analysis: · V. CONCLUSION

The analysis shows that All-Mask components, resolution-aware scaling, and structured prediction improve dense segmentation and efficiency. STAMPlus extends STAMP’s single-target paradigm through dynamically bound target identities while preserving the shared foundation.

  • 3) Ablation and Generalization Analysis:: Removing either visually enhanced mask embeddings or hybrid attention causes a clear accuracy drop, confirming both components’ contribution to dense prediction.The ablation attributes gains to patch-aligned visual initialization and bidirectional interaction among mask tokens.
  • 3) Ablation and Generalization Analysis:: STAMP and LISA incur steep latency growth because they repeat separate predictions for each target category, whereas STAMPlus shares one Phase 2 mask prediction.STAMPlus concentrates additional cost in Phase 1 autoregressive target-list generation, producing a smaller latency slope as categories increase.
  • 3) Ablation and Generalization Analysis:: Remote-sensing experiments increase the mask-token budget from 1024–1280 to 2560–3200 alongside higher-resolution inputs.These budgets correspond to standard- and high-resolution inputs in the controlled scaling study.
  • 3) Ablation and Generalization Analysis:: 5.5 points on RRSIS-D and 4.9 points on EarthReason: jointly scaling input resolution and mask-token budget improves gIoU.The denser patch–token grid preserves fine small-object evidence, and gIoU is especially sensitive to small-target quality.
  • V. CONCLUSION: All-Mask Prediction decouples autoregressive dialogue generation from non-autoregressive dense mask prediction within the native MLLM token interface.This shared foundation underlies both binary and structured variants.
  • V. CONCLUSION: STAMP implements Binary All-Mask Prediction and resolves the segmentation trilemma for task-specific single-target segmentation.The conclusion distinguishes this binary instantiation from the later structured generalization.
  • V. CONCLUSION: STAMPlus generalizes the paradigm through Structured All-Mask Prediction, dynamically binding target identities generated in Phase 1 to the mask-prediction process.The supplied conclusion passage introduces this structured extension but is truncated after the binding mechanism.
Loading 2608.02791v1…