Source-linked AI summary
Task-Utility IQA: Rethinking Image Quality Assessment in Embodied Tasks
Jirong Zha, Yanwei Liao, Yemin Wang, Xiangmin Yi, Siqi Peng, Yingfeng Chen, Chen Gao, Xinlei Chen
TL;DR
Embodied IQA needs to assess whether an image supports a specific task, because task-agnostic ratings do not encode task goals and context. The paper proposes an MLLM-based, goal-conditioned framework and finds task-conditioned utility better aligned with downstream task success than generic IQA scores. Its evidence is a compact VisDrone study, not a saturated benchmark.
Problem
Existing IQA largely aligns task-agnostic scores with human ratings, while embodied image adequacy depends on task goals, context, and action requirements.
Source: §1 · p. 1, §2.2 · p. 4, §3.2 · p. 5Method
The paper proposes an MLLM-based embodied IQA framework that conditions quality judgment on task and context and evaluates it by expected task utility.
Source: §1 · p. 2, §3.2 · p. 5Results
Takeaways & Limitations
The findings support evaluating embodied IQA by whether visual input is useful for a specified downstream task rather than by task-agnostic perceptual fidelity alone.
Source: §8 · pp. 8–9The compact study focuses on UAV-view VQA-style tasks derived from detection annotations and does not cover the full range of embodied settings or closed-loop decisions.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Image quality assessment (IQA) underpins embodied imaging pipelines by judging whether visual quality satisfies downstream tasks, yet most existing methods learn task-agnostic scores aligned with generic human ratings on static benchmarks. This objective mismatches embodied and interactive settings, where image adequacy depends on task goals, context, and action requirements that shape an agent's decisions. We argue that IQA should shift from score regression to goal-conditioned judgment defined by embodied task utility. To this end, we propose an MLLM-based embodied IQA agent framework that reasons about task requirements, grounds quality evidence, and produces decision-facing assessments. A compact VisDrone evidence study shows that conventional perceptual metrics and generic MLLM-based IQA scores are weak proxies for downstream VLM task success, while a task-conditioned utility score better predicts whether visual input supports the target task. These findings support a task-utility-oriented reframing of IQA for embodied, explainable, and decision-aware visual perception.
1 Introduction
Embodied settings make image adequacy depend on task goals, context, and action requirements, which task-agnostic perceptual scores do not capture. The paper proposes goal-conditioned, explainable IQA and reports compact VisDrone evidence that task-conditioned utility aligns better with downstream task success.
1 Introduction
- In embodied settings, image quality should reflect downstream usability, interpretability, and decision relevance rather than perceptual fidelity or generic human-rating alignment alone.Existing protocols evaluate alignment with subjective judgments on static benchmarks.Source: §1 · p. 1
- Task-agnostic scores miss varying task requirements, relevant image regions, and cases where semantic coverage matters more than pixel-level fidelity.Blur may be irrelevant in the background but make an inspected surface unusable.Source: §1 · pp. 1–2
- Embodied agents and MLLMs create demand and capability for context-aware judgments that connect specific perception tasks to actionable decisions.Existing IQA objectives and benchmarks provide little support for explanation, interaction, or policy choices such as retake, reposition, or proceed.Source: §1 · p. 2
- The paper proposes an MLLM-based embodied IQA framework that replaces task-agnostic score regression with goal-conditioned, explainable, utility-aware judgment.Its position is to evaluate judgment by task utility rather than correlation with task-agnostic scores.Source: §1 · p. 2
- Task-agnostic perceptual and generic MLLM-based IQA scores are weak proxies for downstream task success, while task-conditioned utility better aligns with task utility.The compact evidence study uses VisDrone.Source: §1 · p. 2
2 Background: MLLM-based IQA and Related Work
MLLM-based IQA has expanded beyond scalar scores through instruction-tuned, comparative, and spatially grounded approaches, but these remain short of task-conditioned utility alignment. The paper distinguishes its focus on predicting downstream task success from robot-oriented quality scoring and database construction.
2 Background: MLLM-based IQA and Related Work
- Current MLLM-based IQA methods improve interpretability and flexibility, yet rarely model the task-conditioned quality required by embodied agents.The paper organizes this work into single-image assessment, comparative evaluation, and grounded quality analysis.Source: §2 · p. 2
2.1 Single-Image IQA with MLLMs
- Single-image methods span zero-shot CLIP-IQA, instruction-tuned Q-Instruct, and text-defined quality levels in Q-ALIGN, but remain detached from task context and embodied decisions.Q-ALIGN frames IQA as classification and reports state-of-the-art results on KonIQ-10K.Source: §2.1 · pp. 2–3
2.2 Comparative and Relational IQA
- Comparative methods enable richer judgments than scalar scores, but remain detached from task context; embodied comparisons should assess support for a task such as obstacle avoidance.Compare2Score, Q-Insight, and Co-Instruct are examples of comparative or relational methods.Source: §2.2 · p. 4
2.3 Grounded Quality Assessment
- Spatially grounded methods locate or describe quality issues, but their explanations do not establish why a degradation matters for a specific task.The paper’s focus differs from robot-oriented quality databases and dedicated no-reference predictors: it asks whether a quality signal predicts task success in context.Source: §2.3 · p. 4
- The proposed extension is to relate quality explanations to task utility, such as explaining how motion blur obscures structure needed for navigation.This moves beyond describing a degradation’s location or appearance.Source: §2.3 · p. 4
3 The Necessity of Embodied IQA
Examples from robotics, UAV inspection, and AR assembly illustrate that visual adequacy depends on the task and its constraints, not a global quality score. The paper formalizes this as quality conditioned on task and context, with expected task utility as the objective.
3.1 Empirical Motivation: Typical Embodied Scenarios
- In robotics, noise or compression may suit coarse navigation but harm fine manipulation, where preserved edges, keypoints, and depth cues matter.A noisier image with visible target edges may support grasp planning better than a cleaner image with poor edge lighting.Source: §3.1 · pp. 4–5
- UAV inspection requires coverage and localized visibility: blur in non-critical regions can be acceptable, while a sharp image that misses the target surface is unusable.Generic no-reference scores may correlate weakly with inspection outcomes because they do not encode coverage or defect visibility.Source: §3.1 · p. 5
- AR assembly quality must balance detail, contrast, and latency for the task, with overlay alignment accuracy relevant in the regions where instructions appear.The specific relationship between quality metrics and alignment precision requires further empirical investigation.Source: §3.1 · p. 5
- Across scenarios, quality depends on task, context, and action; useful assessment should explain what limits utility and what can be done next.The cited examples include varying task requirements, semantic coverage, and real-time multi-objective trade-offs.Source: §3.1 · p. 5
3.2 Theoretical Foundation: A Task-utility Conditional View of Quality
- Traditional IQA learns an unconditional Q(I) from human opinion scores, whereas embodied IQA conditions judgment on task T and context C to maximize expected task utility.The framework’s objective is utility for downstream performance and decision-making.Source: §3.2 · p. 5
- When task-relevant features differ, access to task and context can yield higher expected task utility than an unconditional assessment trained on task-agnostic ratings.A single predictor must compromise between conflicting task requirements; conditional judgments can retain task-specific decision rules.Source: §3.2 · p. 5
4 Empirical Evidence: Task Utility Beyond Perceptual Quality
The VisDrone study tests whether image-quality signals predict downstream VLM task success, rather than merely reflect task-agnostic perceptual fidelity. Ours-TCUS aligns better with task success than generic IQA scores and cognition-consistency.
4 Empirical Evidence: Task Utility Beyond Perceptual Quality
- The compact study uses VisDrone2019-DET UAV imagery, converting detection annotations into VQA-style existence and counting queries.The authors frame this as evidence of mismatch, not a saturated benchmark.Source: §4 · p. 6
- For each image–question pair, a normalized target-VLM answer is matched to the gold outcome to label task success, which quality indicators are tested against predicting.Ours-TCUS combines perceptual evidence with degradation, question-type, and prediction-consistency signals; AUROC and AP measure ranking of successful versus failed instances.Source: §4 · p. 6, Table 2 · p. 6
- Q-SiT, Q-Insight, and DeQA-Score-Mix3 remain close to chance-level AUROC, while cognition-consistency improves alignment but trails task-conditioned utility.The figure includes baseline comparisons and cross-model evidence through AP heatmaps, AUROC distributions, per-model gains, and AUROC–AP trade-offs.Source: §4 · p. 6, Figure 4 · p. 7
- AUROC=0.925, AP=0.948, and Spearman r = 0.710 for Ours-TCUS, which outperforms generic IQA scores and cognition-consistency across multiple target VLM families.The study evaluates each score by its ability to predict downstream VLM task success.Source: §4 · p. 6, Figure 4 · p. 7
5 Framework for Embodied IQA Agent
The framework casts embodied IQA as a goal-driven, knowledge-augmented reasoning pipeline coordinated by an MLLM agent. It combines task decomposition, grounded quality judgments, and tool-assisted refinement to produce utility-aware decisions and actionable feedback.
5 Framework for Embodied IQA Agent
- The agent interprets the downstream goal, decomposes it into sequential subtasks, and retrieves expert knowledge through RAG to ground task-specific assessment criteria.This replaces generic perceptual heuristics with criteria tied to task requirements.Source: §5 · p. 7
- Conditioned on task requirements and retrieved knowledge, the agent produces quality rankings or usability decisions with textual rationales grounded in task-relevant image regions.Region-level analysis explains how specific quality factors influence task performance.Source: §5 · p. 7
- When it detects deficiencies, the agent can invoke enhancement tools, estimate confidence, flag borderline cases, and issue utility-aware decisions with actionable feedback.Examples of tools include denoising, contrast adjustment, and cropping task-relevant areas.Source: §5 · p. 7
6 Implementation of Embodied IQA Agent
Implementation requires coordinated datasets, instruction-based training, grounded evaluation, and deployment. The proposed components connect task contexts and utility signals to goal-conditioned judgments assessed for downstream support and used within real decision pipelines.
6 Implementation of Embodied IQA Agent
- The implementation plan coordinates embodied datasets, instruction-based training, grounded evaluation, and edge deployment as components of a task-aware, interpretable assessment loop.The loop links task demands to practical, context-aware quality assessment in embodied systems.Source: §6 · pp. 7–8
6.1 Embodied Datasets
- Embodied IQA datasets should pair images with task specifications, environmental context, and goal-driven quality annotations, including explanations and task-performance outcomes.Existing embodied datasets provide platform imagery but lack quality annotations; extending them with task-aware labels is proposed.Source: §6.1 · p. 8
6.2 Instruction-based Training
- Instruction-based training should teach task-specific judgments using pairwise comparisons, soft-label regression, and human feedback aligned with task utility.The proposed objective can combine utility, explanation, and consistency losses, with DPO and human-feedback techniques as possible alignment approaches.Source: §6.2 · p. 8
6.3 Grounded Evaluation and Deployment
- Evaluation should measure usefulness, explanation fidelity, and cross-context consistency rather than only PLCC or SRCC, while deployment requires lightweight real-time integration.Distillation and quantization are suggested to support edge-device deployment.Source: §6.3 · p. 8
7 Alternative Views
The paper acknowledges that MLLM agents are not always necessary: cheaper task-specific predictors may suit well-specified settings. It argues for an embodied IQA layer when judgments must account for task semantics and context and provide actionable explanations.
7 Alternative Views
- Task-specific metrics or lightweight predictors may be preferable when labels are available and explanations are unnecessary, because MLLMs are costly, can hallucinate, and are difficult to deploy in real time.The authors state that embodied IQA should not replace all IQA metrics.Source: §7 · p. 8
- Scalar perceptual scores are insufficient when quality depends on task semantics, contextual constraints, target regions, and actionable explanations.The proposed embodied agent adds task grounding and a decision-facing layer; prompted MLLM reasoning alone does not ensure task-grounded explanations, context adaptation, or embodied integration.Source: §7 · p. 8
- Machine-specific perception trades off against human-aligned interpretability, which the authors retain for safety and oversight while recognizing machine consistency can matter for scale or latency.The key criterion remains whether the judgment supports the task under the current context.Source: §7 · p. 8
8 Conclusion
The paper argues that embodied IQA should judge whether images support specific downstream decisions, rather than relying on task-agnostic perceptual fidelity alone. Its VisDrone evidence favors task-conditioned utility signals as predictors of downstream task success.
8 Conclusion
- Conventional perceptual metrics and recent MLLM-based IQA baselines are weak predictors of downstream task success, while task-conditioned utility signals better align with embodied visual usefulness.The paper frames this as evidence for evaluating image quality by task utility and motivates benchmarks, models, and protocols for task-grounded embodied IQA.Source: §8 · pp. 8–9
Limitations
The evidence is a compact VisDrone study, not a saturated benchmark, and focuses on UAV-view VQA-style perception tasks derived from detection annotations. The authors also identify limitations of current MLLMs and call for broader validation and more efficient, uncertainty-aware approaches.
Limitations
- The evidence covers UAV-view VQA-style tasks derived from detection annotations, not the full range of embodied settings, closed-loop policies, or human-agent decision scenarios.Ours-TCUS is an empirical proxy rather than a final metric, and the authors call for validation across broader tasks, sensors, and deployment conditions.Source: Limitations · p. 9
- Current MLLMs may hallucinate, misground rationales, miss fine-grained visual evidence, or produce unstable judgments under distribution shift; their computational cost may preclude real-time use.Proposed future work includes richer task-performance labels, uncertainty-aware and lightweight utility predictors, and closed-loop evaluation.Source: Limitations · p. 9
Ethical Considerations
The material combines ethical cautions about task-conditioned judgments with proposed training, evaluation, and deployment approaches, alongside related model and metric descriptions. It emphasizes that embodied IQA should account for task utility and responsible use.
Ethical Considerations
- Task-conditioned judgments may be overtrusted in safety-critical settings, so responsible deployment calls for uncertainty reporting, domain-specific validation, and human oversight under distribution shift.The paper names UAV inspection, autonomous driving, and robotic operation as examples.Source: Ethical Considerations · p. 9
A Literature Review
- Figure 5 organizes MLLM-based IQA into model enhancements, scoring shift, and data-centric assessment, distinguishing comparative multi-image methods from single-image methods.The caption marks comparative multi-image methods with an asterisk.Source: Figure 5 · p. 13
B.1 Training Objective
- The proposed training objective uses task performance to measure utility alignment, expert annotations for explanation similarity, and a consistency term that permits task-dependent quality criteria.The dataset pairs images and tasks with context, task-performance outcomes, and expert explanations.Source: §B.1 · p. 13
B.2 Direct Preference Optimization
- Preference-based training compares image pairs by task success, preferring the more successful image even when it scores lower on traditional metrics.The DPO objective uses quality judgments, an optimized policy, a reference policy, and a coefficient controlling preference-signal strength.Source: §B.2 · p. 13
B.3 Training Algorithm
- Training conditions multimodal features on image, task, and context; utility aligns with task performance, explanations with expert annotations, and DPO preferences with task outcomes.The procedure initializes from a pretrained MLLM and updates it using task-performance-ranked preference pairs.Source: §B.3 · pp. 13–14
B.4 Evaluation Protocols
- Evaluation additions include decision-centric tests for retakes, repositioning, or abstention, plus region-level grounding and expert verification of explanation fidelity.Samples can pair image, task, and context with outcomes and expert rationales.Source: §B.4 · p. 14
B.5 Deployment Strategies
- Deployment strategies include teacher-student distillation, quantization, and hardware-aware architecture search to reduce computational cost for real-time edge inference.The proposed use includes low-latency feedback and interaction within constrained UAV, AR, and robot systems, with IQA potentially informing image retakes or flight adjustments.Source: §B.5 · p. 14
C.1 Baselines; C.2 Metrics; C.2.1 Full-Reference (FR) IQA Metrics; C.2.2 No-Reference (NR) IQA Metrics
- The experiments compare embodied IQA with MLLM-based baselines and traditional signal-fidelity and perceptual metrics, including full-reference and no-reference measures.Full-reference metrics compare distorted and pristine images; the described metrics include MSE, PSNR, SSIM, LPIPS, and DISTS, while no-reference examples include NIQE, BRISQUE, MANIQA, and HyperIQA.Source: §C.1 · p. 14, §C.2 · p. 15, §C.2.1 · p. 15
C.3 Task Scenarios
This section examines task-conditioned image quality assessment across six scenarios, VisDrone evidence, and a UAV inspection case study, relating image adequacy to task-specific actionability and downstream task success. It also formalizes task-aware assessment and outlines a dataset-to-model implementation roadmap.
C.3 Task Scenarios
- An Embodied Utility Score of 5.0 is reported; robotic grasping targets 6D-pose and target-coordinate resolution, while aerial detection targets locating and bounding a red vehicle.Source: §C.3 · pp. 16, 17, 19
- The driving scenario scores 1.0 because a clear image reveals wrong-way driving, prompting an emergency alert and human handover or safe stop.The task-grounded assessment identifies the vehicle in opposing traffic and treats the violation as a critical safety state.Source: §C.3 · p. 20
- The authenticity-check scenario scores 1.0 after detecting a floating, unsupported synthetic insulator, so the frame is discarded as invalid inspection data.The assessment identifies missing physical support and shadowing as evidence of a phantom composite.Source: §C.3 · pp. 21–22
- The real-world defect-detection scenario scores 3.0: a bird nest near high-voltage components is identified as a hazard, triggering a defect alert.The image is judged physically valid and sufficiently clear for maintenance action.Source: §C.3 · p. 23
- Indoor navigation scores 3.5 because noisy, dim imagery still exposes actionable boundaries and obstacles, though the robot should reduce speed and use temporal filtering.The visible traversable path supports navigation, while blur and low resolution increase depth-estimation uncertainty.Source: §C.3 · pp. 24–25
C.4 Evaluation Results
- Table 3 compares task-oriented IQA methods, and its accompanying note says low scores indicate correct detection of critical safety issues under Safety Veto.The caption notes that existing MLLM-based IQAs can miss safety hazards despite high aesthetic scores.Source: Table 3 · p. 25, §C.4 · p. 25
- In three scenarios, the Embodied IQA Agent vetoes wrong-way driving (1.00), rejects physics-violating synthetic data, and rates noisy but navigable imagery actionable (3.50).For comparison, Q-SIT scores the driving image 4.81 and the navigation image 0.59; authenticity baselines score 3.61–4.06.Source: §C.4 · p. 25
C.5 Additional Empirical Evidence on VisDrone
- Ours-TCUS aligns more strongly with VisDrone downstream task success than Q-SiT, Q-Insight, and DeQA-Score-Mix3 across AUROC, average precision, and Spearman correlation.The additional evidence is presented as support for the task-utility hypothesis, not as a fully saturated benchmark.Source: §C.5 · p. 26, Table 4 · p. 26
- VisDrone cases show that clean UAV images can fail counting or small-object reasoning, while degraded images can retain cues for existence judgments or coarse reasoning.These examples support evaluating task-relevant information preservation rather than global perceptual quality alone.Source: §C.5 · p. 26, Figure 12 · p. 27, Figure 13 · p. 28
D.2 Traditional IQA Failure Modes
- Bridge inspections vary in lighting, wind, and altitude, and traditional IQA can penalize irrelevant blur, miss inadequate bridge-surface visibility, and offer no explanation.Generic no-reference IQA scores may correlate weakly with inspection outcomes because they do not encode inspected-surface coverage or defect visibility.Source: §D.2 · p. 26, §D.1 · p. 26
D.3 Embodied IQA Approach
- For crack inspection, an embodied agent evaluates bridge-surface visibility, critical-region blur, and lighting against task needs, then explains how they affect detection reliability.Its judgment is intended to align with inspection needs rather than abstract quality metrics.Source: §D.3 · p. 26
E.1 Information-Theoretic Analysis
- Task-aware IQA formalizes quality as information about task success beyond the image, whereas traditional IQA targets information shared between human ratings and quality scores (Zhang et al., 2024a).The conditional assessment is Q(I|T, C); the traditional assessment is Q(I).Source: §E.1 · p. 27
E.2 Generalization Bounds
- Conditioning IQA on task specifications treats tasks as separate domains and is proposed to reduce domain shift and improve generalization across tasks.The passage contrasts this with general-quality methods that assume a single domain.Source: §E.2 · pp. 27–28
F.1 Phase 1: Dataset Construction
- Phase 1 constructs embodied IQA datasets with platform images, task specifications, task-relative judgments and explanations, and measured task outcomes.Dataset scale depends on task diversity, annotation granularity, and desired generalization.Source: §F.1 · p. 28, F Implementation Roadmap · p. 28
F.3 Phase 3: Evaluation and Benchmarking
The roadmap calls for task-usefulness, explanation-fidelity, and cross-context-consistency benchmarks, followed by real-world integration and validation. It also identifies grounding, specialization, latency, and compute as practical risks.
F.3 Phase 3: Evaluation and Benchmarking
- Evaluation benchmarks should measure task usefulness, explanation fidelity, and cross-context consistency, using datasets with ground-truth task performance and expert explanations.The proposed phase includes evaluating models on these benchmarks and notes challenges in metric definition and benchmark quality.Source: §F.3 · p. 29
F.4 Phase 4: Deployment and Integration
- Deployment should optimize efficiency through distillation and quantization, integrate with UAVs, robots, and AR, and validate performance in operational settings.The phase identifies latency, robustness, and operator trust as challenges, and calls for industry partnerships and field testing.Source: §F.4 · p. 29
G Practical Risks
- Implementation risks include ungrounded justifications, over-specialization to narrow task prompts, and deployment constraints on latency and compute.The paper points to grounding, efficiency, and benchmark design that rewards safe behavior under uncertainty as areas requiring progress.Source: G Practical Risks · p. 29
H Glossary of Key Concepts
- Embodied IQA evaluates images from an agent’s first-person perspective using task utility, environmental context, and goal-driven reasoning rather than abstract distortion metrics.Task-aware assessment considers why an image is useful for a specific goal and its relationship to downstream performance.Source: H Glossary of Key Concepts · p. 29
- Conditional quality judgment makes image quality depend on task specification and context, while evaluation misalignment describes objectives that diverge from desired outcomes such as task performance.The glossary contrasts Q(I | T, C) with unconditional Q(I).Source: H Glossary of Key Concepts · p. 29