PaperlayerView exact revision on arXiv

Source-linked AI summary

First Things First: Teaching LLM-Based Agents to Prioritize Must-Haves before Nice-to-Haves

Tianjie Ju, Xinyue Xu, Wanxuan Sun, Lingxiao Diao, Gongshen Liu, Zhuosheng Zhang, Cheng Yang

arXiv:2609.05224v1cs.CV

TL;DR

MLLM agents struggle to satisfy structured requests when must-have and nice-to-have requirements conflict or when no feasible answer exists. This paper introduces FTF-BENCH and FTF-RL to evaluate and train requirement-aware reasoning, finding substantial improvements and broader reasoning gains, while noting that broader domains and larger-scale training remain to be tested.

  • Problem

    Existing multimodal benchmarks rarely test whether agents identify and prioritize must-have requirements before nice-to-have requirements in complex service requests.

  • Method

    The paper constructs FTF-BENCH and introduces FTF-RL, a multiobjective reinforcement-learning framework that rewards requirement identification, priority ordering, and answer validity.

  • Results

    FTF-RL makes MLLMs perform substantially better on FTF-BENCH and produces consistent gains on other complicated reasoning tasks.

  • Takeaways & Limitations

    Requirement-aware reasoning is a potentially important component of reliable MLLM reasoning and broader generalization.

  • Takeaways & Limitations

    FTF-BENCH does not exhaust real-world requirement-aware decision-making, and the training experiments use the current benchmark scale.

Abstract

from arXiv · show

Recent progress in multimodal large language models (MLLMs) has fueled significant enthusiasm in their potential to act as autonomous agents for real-world tasks. However, scenarios requiring agents to fulfill users' complex, structured requirements remain largely underexplored. In this work, we examine reasoning tasks under three distinct requirement scenarios: (i) Must-have requirements uniquely determine a unique feasible solution; (ii) Multiple answers satisfy the must-have requirements and are prioritized via the nice-to-have requirements; and (iii) No candidate solution satisfies the must-have requirements, in which case the agent should abstain from generating a response. We evaluate state-of-the-art MLLMs on 3,649 carefully constructed problems that reflect realistic service scenarios, including e-commerce, booking, and map-based or ride-hailing. Our evaluation reveals that existing MLLMs exhibit catastrophic failures in all scenarios. They frequently misinterpret task requirements, violate must-have requirements, and produce invalid solutions. To address this critical gap, we propose First Things First Reinforcement Learning FTF-rl that explicitly optimizes reasoning over multi-priority user requirements. Experimental results show that our method substantially improves the task success rate compared to strong baselines. Moreover, FTF-rl yields general effectiveness on popular logical and mathematical reasoning tasks, including LogicVista, MathVision, and InfoQA. Our findings suggest that enhancing requirement-aware reasoning capability provides a simple yet effective pathway to improve generalization of MLLM agents. Code and dataset are available at https://github.com/claire62/FTF-RL.

1 Introduction

Real-world service requests combine must-have and nice-to-have requirements, but current MLLM agents often fail to prioritize them correctly. The paper introduces FTF-BENCH and FTF-RL to evaluate and improve requirement-aware reasoning.

  • Motivation: Existing multimodal agents are usually evaluated on clear instructions with minimal ambiguity, unlike multifaceted real-world service requests.These requests may combine hard constraints with preferences and trade-offs.
  • Motivation: Agents often violate must-have requirements by optimizing for nice-to-haves, producing infeasible or incorrect solutions.They may also mishandle conflicting requirements and fail to abstain when no candidate is feasible.
  • Benchmark and method: FTF-BENCH contains 3,649 image–requirement pairs spanning e-commerce, booking, and maps or ride-hailing scenarios.Its tasks cover unique-answer, multiple-answer, and unanswerable settings.
  • Benchmark and method: FTF-RL trains MLLMs with a multiobjective reward for well-formed outputs, answer correctness, and must-have versus nice-to-have classification.The approach also encourages intermediate reasoning.
  • Findings: Current MLLMs exhibit widespread catastrophic failures, especially when multiple candidates require trade-off resolution or when abstention is necessary.Providing gold requirement labels substantially improves accuracy, highlighting requirement comprehension as a key bottleneck.
  • Findings: FTF-RL consistently improves performance across FTF-BENCH scenarios and transfers gains to broader reasoning benchmarks.The reported transfer suggests that parsing and prioritizing requirements may support broader reasoning skills.

2 First Things First Benchmark

FTF-BENCH evaluates whether MLLMs can identify mandatory requirements, use preferences to rank feasible options, and abstain when no valid option exists. It combines realistic service images with structured task construction and human verification.

  • Benchmark overview: FTF-BENCH targets a gap in multimodal benchmarks, which rarely test prioritization of must-have requirements before nice-to-have requirements.The benchmark is designed for service requests involving multiple requirements.
  • Benchmark overview: The benchmark covers e-commerce, booking, and maps or ride-hailing applications with diverse interface layouts and interaction contexts.These domains represent realistic service-oriented scenarios.
  • Task settings: Single-answer tasks contain one candidate satisfying all mandatory conditions, whereas multi-answer tasks require ranking several valid candidates using optional requirements.Unanswerable tasks contain no valid candidate and test whether models abstain rather than hallucinate an option.
  • Task formulation: The formal task representation treats candidates as objects and requirements as logical predicates whose satisfaction is binary.Must-have and nice-to-have requirements are represented separately as R+ and R−.
  • Task formulation: The candidate set is filtered first by all must-have requirements, then by priority-ordered nice-to-have requirements when multiple candidates remain.The resulting set determines the final answer.
  • Task formulation: Unanswerable cases are defined by an empty must-have-feasible set, and predicting any candidate is counted as an error.Single-answer cases have one feasible candidate; multiple-answer cases have at least two.
  • Dataset construction: The dataset pipeline collects images, generates grounded requirements and answers, rewrites them into colloquial queries, and applies human verification.Four annotators validate requirement correctness, answer consistency, and mandatory–optional separation.

3 First Things First Reinforcement Learning

FTF-RL trains MLLMs with a multi-objective reward that evaluates output format, answer correctness, and requirement classification. Its framework samples candidate outputs, scores them, and updates the policy while constraining divergence from a reference policy.

  • Multi-objective reward design: FTF-RL uses a rule-based, multi-objective reward to assess format compliance, final-answer correctness, and must-have versus nice-to-have classification.The design targets requirement understanding and visual reasoning capabilities.
  • Format reward: Format reward checks XML-style tags, valid JSON requirement fields, inclusion of reasoning, and a tagged final answer.The required structures include <requirements>, <think>, and <answer> components.
  • Accuracy reward: Accuracy reward compares the generated answer with the ground-truth answer using an MLLM judge that recognizes semantic equivalence.This reduces errors caused by surface-form differences such as synonyms or equivalent expressions.
  • Optimization framework: FTF-RL samples candidate outputs, assigns each a reward, and optimizes the current policy while KL-regularizing it toward a reference policy.The framework computes an advantage-like objective for candidate outputs during policy optimization.
  • Requirement reward: Requirement reward uses Macro-F1 across must_have and nice_to_have labels to penalize both over-constraining and ignored essential requirements.The reward treats the two requirement categories as separate classes and averages their class-specific F1 scores.

4 Experiments

Experiments show that MLLMs struggle to parse and prioritize multi-level requirements, while FTF-RL improves requirement-aware reasoning and transfers gains to broader reasoning benchmarks.

  • 4.2 Evaluation results on FTF-BENCH: Seven MLLMs, including proprietary and open-source models, were evaluated on FTF-BENCH across Direct and Upper input settings.Direct requires models to infer requirement hierarchy and decisions from colloquial requests; Upper provides gold requirement labels as structured guidance.
  • 4.2 Evaluation results on FTF-BENCH: Upper exceeds Direct across most scenarios, indicating that parsing and prioritizing requirements from natural language is a primary error source.The gap is particularly severe in multiple-answer and unanswerable scenarios.
  • 4.2 Evaluation results on FTF-BENCH: Models frequently fail in multiple-answer and unanswerable settings by mishandling nice-to-have trade-offs or answering when abstention is required.These failures reflect over-accommodation of prompts and difficulty resolving requirement priorities.
  • 4.2 Evaluation results on FTF-BENCH: Scaling effects are not monotonic: Qwen2.5-VL-7B and 72B fail catastrophically in direct reasoning, whereas the 32B model performs better.The authors associate the smaller model with weak requirement distinction and the largest model with overfitting to all expressed conditions.
  • 4.3.1 Main results: Qwen2.5-VL-7B-Instruct improves by more than 26% in the multiple-answer scenario after FTF-RL, with consistent gains also observed for single-answer and unanswerable settings.Both the 7B and 3B variants show strong gains on the held-out evaluation subset.
  • 4.3.2 Generalization across Reasoning Benchmarks: FTF-RL produces consistent improvements on LogicVista, MathVision, and InfoQA despite no explicit training on those benchmarks.Removing individual reward components causes performance drops across datasets, supporting the contribution of the multi-objective reward design.

5 Related Work

Prior work advances multimodal reasoning, reinforcement learning, and instruction-following benchmarks, but existing evaluations generally treat directives as equally important. Recent methods add procedural structure or rewards, yet requirement-priority reasoning remains insufficiently measured.

  • Multimodal Reasoning: Multimodal chain-of-thought methods decompose reasoning into stages, while later approaches use more flexible perception, grounding, and task-specific decompositions.Rigid pipelines can limit adaptability across diverse tasks.
  • Reinforcement Learning: Reinforcement learning frames reasoning as reward-optimized trajectories, with verifiable, spatial-consistency, and intermediate rewards improving specialized reasoning.These methods extend reward supervision beyond final-answer correctness.
  • Benchmarking: MLLM benchmarks assess knowledge, documents, perception, multi-image comprehension, and instruction following across multiple facets.The surveyed benchmarks cover broad multimodal capabilities but not necessarily requirement priority.
  • Instruction Following: Instruction-following benchmarks include automatically verifiable tasks, decomposed sub-instruction scoring, and model-assisted generation or evaluation.InfoBench, for example, computes partial credit over micro-requirements.
  • Research Gap: Existing instruction-following benchmarks implicitly treat all directives as equally important, leaving necessity-aware prioritization underexamined.This limitation motivates evaluating must-have and nice-to-have requirements separately.

6 Conclusion

The paper introduces FTF-BENCH and FTF-RL to study and improve requirement-aware reasoning in realistic service settings. FTF-RL substantially improves performance and also yields consistent gains on other complex reasoning tasks.

  • Conclusion: FTF-BENCH evaluates whether MLLMs understand and prioritize complex, multi-priority requirements in realistic service settings.The benchmark targets failures involving misread requirements, violated hard constraints, and invalid solutions.
  • Conclusion: FTF-RL rewards correct requirement identification, proper ordering of requirement satisfaction, and answer validity.The framework is designed to improve requirement-aware reasoning rather than general compliance alone.
  • Conclusion: After training, MLLMs perform substantially better and narrow the gap toward the upper bound obtained with requirement labels.The conclusion reports improvement on the benchmark without specifying a single aggregate value here.
  • Conclusion: FTF-RL also produces consistent gains on other tasks requiring complicated reasoning, suggesting broader generalization from strengthened requirement comprehension.The paper calls for greater attention to requirement-aware reasoning in improving MLLM reasoning reliability.

Limitations

The benchmark covers representative service domains but does not exhaust real-world requirement-aware decision making. Larger training datasets and broader domains are needed to characterize transfer and its upper limits.

  • Scope: FTF-BENCH covers e-commerce, booking, and map-based applications but not the full space of real-world requirement-aware decision making.Future extensions could include more diverse domains with requirements that evolve over time.
  • Training Scale: FTF-RL experiments use the current scale of FTF-BENCH, limiting characterization of the upper limit of training-data effects.The authors propose scaling training data to test transfer across broader multimodal reasoning tasks.

Ethical Considerations

The paper describes benchmark construction and evaluation procedures designed to ground requirements in application images and distinguish mandatory from optional conditions. Its ethical statement reports human verification, compensation, and the absence of personally identifiable or sensitive information.

  • Data and Verification: The study uses screenshots of publicly available applications, automatically generated requirements, and trained human annotators for verification.The authors report compensation of 1 RMB per data sample.
  • Task Rules: FTF-BENCH tasks distinguish must-have requirements from nice-to-have requirements, with mandatory conditions required simultaneously.A candidate failing any must-have requirement is ineligible for further comparison.
  • Task Rules: Nice-to-have requirements are used only when multiple candidates satisfy all must-have requirements, then compared in priority order.The prompt applies sequential comparison to select a unique best product.
  • Task Rules: If no candidate satisfies all must-have requirements, the prescribed output is an explicit refusal rather than a selected product.The prompt specifies the refusal text and a strict JSON answer format.
  • Benchmark Construction: Requirement-generation prompts ground questions and answers in visible image information while constructing candidate pools, distractors, and prioritized optional requirements.The benchmark also designs confusing options that may satisfy more high-priority nice-to-haves than the correct answer.
  • Benchmark Construction: Single-answer tasks use indispensable must-have requirements, while nice-to-have requirements are optional and ordered from highest to lowest priority.The construction removes redundant conditions and encourages distractors to overemphasize optional requirements.
  • Benchmark Construction: The benchmark explicitly tests whether models resist overemphasizing nice-to-have requirements when must-have requirements are decisive.Its output schema records answers, confusing options, and both requirement categories.
  • Verification Findings: Human verification found failures involving ignored must-have requirements, optional requirements treated as mandatory, and incorrectly ordered preferences.These error types reflect difficulty understanding and prioritizing user preferences.

D Model-Dependent Bias in Data Creation and Evaluation

The data pipeline combines model-generated requirements and answers with human verification, while cross-model checks assess whether evaluation labels depend on the judger model.

  • Data creation and human verification: Human annotators verify requirement realism and final-answer correctness after Doubao-Seed-1.6-250615 generates initial requirements, candidates, and judgments.
  • Data creation and human verification: Most discrepancies involve unrealistic prompts, while most retained items have reasonable requirements and correct answers.The human pass is reported to reduce residual errors and potential same-source bias.
  • Cross-model evaluation: Replacing the judger with GPT-5 produces very high agreement across all three scenarios, suggesting labels do not depend on one model family.
  • Data creation and human verification: The annotation interface evaluates question reasonableness, answer accuracy, requirement categorization, and description correctness.
  • Error analysis: Human verification identifies frequent must-have violations, occasional treatment of nice-to-have requirements as mandatory, and occasional misordering.
  • Reward analysis: Format, requirement-classification, answer-correctness, and overall rewards all improve monotonically during FTF-RL training.The overall reward rises steadily across training steps.

F Qualitative Case Studies on General Reasoning Benchmarks

Qualitative cases show FTF-RL-trained Qwen2.5-VL-7B-Instruct applying explicit requirement extraction and stepwise execution on MathVision and LogicVista without additional supervision.

  • Evaluation setup: In both case studies, the model receives only the original benchmark input and no additional supervision.
  • MathVision: For the MathVision geometry problem, the model states that grey tiles must form one connected region with a smooth boundary from a single convex circle.
  • MathVision: The model checks each candidate against that requirement and rejects the option with a disconnected grey region.
  • Cross-case pattern: The cases illustrate organization of reasoning into requirement extraction followed by execution rather than local pattern matching.
  • LogicVista: For the LogicVista problem, the model restates subgoals, infers symbolic-operation rules, applies them step by step, and selects the correct option.

G Additional Experiments on Complex Agent Tasks

Additional evaluations on ScienceQA and AndroidControl test whether requirement-aware reasoning transfers beyond the paper’s three service-oriented domains and report consistent improvements after FTF-RL.

  • Benchmarks: The additional evaluation uses AndroidControl for mobile GUI control and ScienceQA for multi-step scientific reasoning.AndroidControl reports Task Match Rate and Action Match Rate, while ScienceQA uses accuracy.
  • Results: Across ScienceQA and AndroidControl, performance consistently improves after FTF-RL training.The authors report that these gains mirror improvements on LogicVista, MathVision, and MathVista.
  • Evaluation design: The evaluation compares models before and after FTF-RL training on both additional benchmarks.
  • Evaluation design: The RL experiments train on 90% of FTF-Bench and evaluate on a shared held-out 10% subset across models.

I Additional Analysis on Decoding Temperature

Temperature analyses examine whether refusal behavior and error patterns are decoding-sensitive, finding moderate accuracy variation and high instance-level consistency across temperatures.

  • Temperature sweep: DOUBAO-1.6-SEED is evaluated on FTF-BENCH at decoding temperatures from 0.0 to 1.0, including deterministic decoding.
  • Temperature sweep: Overall accuracy varies only moderately across temperatures, indicating stable refusal tendencies.
  • Consistency analysis: Instance-level agreement between temperature 0 and higher temperatures is measured to assess output consistency.
  • Consistency analysis: High consistency across temperatures indicates that refusal behavior and analyzed error patterns are stable.
Loading 2609.05224v1…