PaperlayerView exact revision on arXiv

Source-linked AI summary

TaskGuard: Task-Conditioned Restoration Utility for Risk-Aware Object Detection

Vung Pham

arXiv:2609.08011v1cs.CV

TL;DR

Restoration can improve average detection yet worsen individual observations, making the utility of a particular intervention important to assess. TaskGuard predicts whether to accept a candidate restoration by combining its realized residual with detector sensitivity in frozen pipelines. On unseen controlled degradation families, it reduces loss-negative interventions while retaining most Always-Restore COCO AP.

  • Problem

    Restoration may improve aggregate detection while making individual observations worse, and visual clarity or detector confidence alone may not indicate whether a specific intervention improves downstream correctness.

    Source: §1 · p. 1

  • Method

    TaskGuard is a post-hoc controller for frozen restoration and detection pipelines that predicts candidate utility from the realized residual’s interaction with detector sensitivity.

    Source: §1 · p. 1

  • Results

    Across three unseen degradation families, TaskGuard reduces loss-negative interventions by 54.2% while retaining 98.8% of Always-Restore COCO AP.

    Source: §5.2 · p. 5

  • Takeaways & Limitations

    TaskGuard controls intervention risk rather than maximizing aggregate AP, since Always Restore remains best in aggregate AP.

    Source: §5.2 · p. 5

    TaskGuard must generate the candidate before judging it, adds nontrivial gradient-computation overhead, and is not shown to transfer arbitrarily across restorers, detectors, datasets, or tasks.

    Source: §6 · p. 7

Abstract

from arXiv · show

Image restoration is commonly applied before object detection under adverse conditions, yet a visually improved image need not improve the downstream task. We study this mismatch as restoration utility prediction: given a degraded image and its candidate restoration, should the restoration be used or should the original observation be preserved? We introduce TaskGuard, a post-hoc controller for frozen restoration and detection pipelines. TaskGuard characterizes the realized restoration residual through its interaction with detector sensitivity and predicts whether the intervention is task-beneficial. Exact regional counterfactuals reveal substantial within-image utility heterogeneity, while a deployable pseudo-gradient preserves statistically reliable directional information. Feature-group ablation further shows that task-conditioned evidence contributes information beyond detector-response and residual statistics. The TaskGuard utility predictor is trained only on Gaussian degradation and frozen before final evaluation, then transferred to unseen motion blur, rain, and defocus. Across these unseen families, TaskGuard reduces lossnegative interventions by 54.2% (family macro) and practical per-image detection deteriorations by 37.0% (pooled), while preserving 98.8% of the Always-Restore COCO AP. On natural-rain DAWN, it reduces loss-negative interventions by 97.9% while retaining 77.8% of the AP improvement obtained by deraining. These results support restoration utility as a task-conditioned property of the specific intervention rather than image appearance alone.

1. Introduction

Restoration can improve aggregate detection while worsening individual observations, so TaskGuard predicts whether to accept each candidate restoration or preserve the degraded input. It uses the realized image change and detector sensitivity to estimate task utility within frozen restoration and detection pipelines.

1. Introduction

  • Unconditional restoration can improve average accuracy while making individual observations worse, because perceptual clarity or detector confidence can rise as correctness deteriorates.The examples include missed objects recovered by restoration, a correct cow changed to an incorrect horse, and a false-positive car.Source: §1 · p. 1, Figure 1 · p. 2
  • TaskGuard frames restoration utility prediction as a post-hoc decision about accepting a particular candidate from a frozen restorer or preserving the degraded input.Unlike approaches that modify enhancement using downstream supervision, it leaves both the restorer and detector unchanged.Source: §1 · p. 1
  • TaskGuard couples the realized restoration residual with detector sensitivity, and validates the directional mechanism using exact regional counterfactuals and an annotation-free pseudo-gradient.Its task-conditioned evidence adds information beyond detector-response and residual statistics.Source: §1 · pp. 1–2
  • The utility predictor is trained only on Gaussian degradation, frozen before evaluation on unseen degradation families and natural-rain DAWN, and also tested with RT-DETR-L.The policy uses a TRAIN-only risk–utility threshold and evaluates transfer across motion blur, rain, and defocus.Source: §1 · pp. 1–2

2. Related Work

Prior work couples enhancement or restoration to downstream recognition, while TaskGuard instead controls acceptance of an already-generated candidate in frozen pipelines. It also differs from appearance-based enhancement selection by conditioning utility prediction on the interaction between the restoration residual and detector sensitivity.

2. Related Work

  • Task-aware enhancement methods couple image processing with downstream objectives, including differentiable processing parameters in IA-YOLO [13] and gated processing operators in GDIP [5].Detection-Friendly Dehazing [10] explicitly couples dehazing with detection.Source: §2 · p. 2
  • Task-driven restoration methods use recognition supervision to shape restoration, whereas TaskGuard assumes the restorer and detector are frozen and decides whether to accept the candidate.Related approaches include SR4IR [7], UniRestore [1], EDTR [8], and TaskTok [9].Source: §2 · p. 2
  • LQTARanker predicts detection-performance gain for per-image enhance-or-not decisions, while TaskGuard uses the actual restoration residual–detector-sensitivity interaction and validates it against exact regional counterfactuals.TaskGuard also freezes its predictor after Gaussian-only training for evaluation across unseen degradation families.Source: §2 · p. 2

3. Method

TaskGuard evaluates the realized restoration intervention by its task utility, using a detector-sensitivity signal to summarize candidate changes. A frozen utility predictor and TRAIN-only threshold then decide whether to restore or preserve the input.

3. Method

  • TaskGuard first forms a candidate restoration and characterizes the realized image change using task-conditioned, detector-response, and residual evidence to predict whether to accept it.The realized intervention is denoted R.Source: §3 · p. 3

3.1. Restoration Utility

  • Bfull measures whole-image restoration utility through the change in detector loss; positive values favor restoration, while Bfull < 0 marks a loss-negative intervention.Detector loss is a differentiable per-image utility signal, not an equivalent of AP or per-image F1.Source: §3.1 · p. 3

3.2. Deployable Task Signal

  • Because ground truth is unavailable at deployment, TaskGuard uses pseudo-labels from the restored candidate and evaluates the corresponding gradient at the degraded input.The residual gives the actual image-change direction and the gradient gives local task sensitivity; the first-order estimate is directional evidence, not an exact counterfactual estimator.Source: §3.2 · p. 3, §3.1 · p. 3

3.3. Intervention Representation

  • The 43-D representation combines 29 task features, 11 detector-response features, and 3 residual statistics, without supplying degradation family, severity, or restorer identity.Task features summarize 64 regional scores; detector-response features describe detection counts and confidence, while residual features summarize absolute-residual magnitude.Source: §3.3 · p. 3

3.4. Utility Regression and Frozen Decision

  • A standardized Ridge regressor predicts Bfull from Gaussian TRAIN data, and the frozen policy restores only above a threshold selected using TRAIN out-of-fold predictions.The threshold-selection procedure retains at least 98% of Always-Restore mean utility while minimizing the loss-negative rate; validation and final-benchmark data do not retune it.Source: §3.4 · pp. 3–4

4. Experimental Setup

The evaluation freezes the policy before testing on source-disjoint controlled degradations and natural-rain images, using frozen restoration and detection models. It compares fixed and learned selection baselines and reports complementary utility, risk, and detection metrics.

4. Experimental Setup

  • After policy freezing, evaluation uses 200 untouched, source-disjoint COCO val2017 images across Gaussian noise, motion blur, synthetic rain, and defocus at three severities, yielding 2400 pairs.Development uses 500 COCO sources with Gaussian noise, split by source into TRAIN, VAL, and development-TEST sets.Source: §4 · p. 4
  • Controlled restorations use frozen Restormer specialists and YOLO11s, while transfer tests include RT-DETR-L on fresh Gaussian pairs and 198 rainy DAWN images without DAWN fitting or tuning.TaskGuard receives the degraded/candidate pair without degradation-family, severity, or restorer-identity inputs.Source: §4 · p. 4
  • Baselines include Always Preserve, Always Restore, and learned Image-only and Visual-pair predictors, with learned baselines following the same Gaussian-only freezing protocol.Visual-pair sees both images but does not explicitly measure agreement between the restoration change and detector sensitivity.Source: §4 · p. 4
  • Evaluation reports utility ranking, benefit retention, loss-negative reduction, preserve rate, and COCO AP50:95, alongside practical-negative status based on per-image F1 decrease or increased missed objects.Statistical resampling is performed at the source-image level.Source: §4 · p. 4

5. Results

TaskGuard’s directional signal is imperfect but informative, and its frozen policy reduces negative restoration interventions across unseen degradations while retaining most restoration benefit. On natural-rain DAWN, it sharply reduces loss-negative interventions while retaining part of the deraining AP gain.

5.1. Directional Utility Approximation

  • Among 83,088 active regions, 51.7% have positive utility and 48.3% negative utility, with both signs in every image–condition pair; the privileged directional estimate has Pearson r = 0.314 and AUROC 0.608 against exact utility.The deployable pseudo-gradient has Pearson r = 0.261 and AUROC 0.566 against exact utility, while its alignment with the privileged signal is r = 0.767 and sign agreement is 71.8%.Source: §5.1 · p. 5

5.2. Fresh Controlled Evaluation

  • Across unseen controlled families, TaskGuard retains 93.3% of loss-defined benefit and 98.8% of Always-Restore COCO AP while reducing loss-negative interventions by 54.2%.Always Restore remains best in aggregate AP; the policy controls intervention risk rather than maximizing AP.Source: §5.2 · p. 5, Table 1 · p. 6

5.3. What Information Drives Utility Prediction?

  • In a post-freeze feature-group ablation, task-gradient features alone achieved ρs = 0.631, versus 0.480 for detector-response and 0.240 for residual-only, and raised Detector+Residual from 0.470 to 0.622.Benefit retention increased by 6.6 points ([2.5, 10.6]); loss-negative reduction increased by 6.9 points, with 95% CI [−0.2, 14.3] including zero.Source: §5.3 · p. 5

5.4. Frozen Threshold and Risk–Utility Tradeoff

  • The TRAIN-only 98%-retention threshold gives unseen-family benefit-retention/risk-reduction pairs of 93.3/54.2%, between the reported 99% and 97% operating points.The 99%, 98%, and 97% constraints yield unseen-family pairs of 93.9/45.5%, 93.3/54.2%, and 91.0/60.3%, respectively.Source: §5.4 · p. 5

5.5. Practical and Cross-Detector Risk

  • Across unseen families, TaskGuard reduces practical-negative restorations from 18.3% to 11.6%, a 37.0% relative reduction; on Gaussian data, frozen decisions retain 93.9% of RT-DETR-L restoration benefit while reducing its loss-negative rate from 28.5% to 19.8%.The lower utility-score correlation under RT-DETR-L than YOLO11s (ρs = 0.305 versus 0.726) indicates partial safety transfer, not detector invariance.Source: §5.5 · pp. 5–6

5.6. Natural-Rain Transfer

  • On DAWN, TaskGuard retains 77.8% of the Always-Derain AP gain while reducing loss-negative interventions by 97.9%.The paired TaskGuard–Always-Derain AP difference is −0.0053 (95% CI [−0.0133, 0.0021]), so no equivalence or non-inferiority claim is made; practical-risk results are descriptive.Source: §5.6 · p. 6

B. TaskGuard gain over Visual-pair

TaskGuard’s gains over Visual-pair are reported across benefit retention and both negative-intervention reductions. The supplied figure caption describes a separate threshold-selection curve and unseen-family sensitivity analysis.

B. TaskGuard gain over Visual-pair

  • Reported differences are +19.50 pp for benefit retention, +33.59 pp for loss-negative reduction, and +15.15 pp for practical-negative reduction.Benefit retention and loss-negative reduction are unweighted family macros; practical-negative reduction is pooled over all 1,800 zero-shot pairs.Source: B. TaskGuard gain over Visual-pair · p. 7
  • Figure 4 contrasts Gaussian TRAIN OOF threshold selection under a predefined 98% benefit-retention constraint with descriptive post-freeze sensitivity on unseen degradations.Source: Figure 4 · p. 7
  • The contextual Restore+Detector and Restore+TaskGuard pipelines require approximately 667 and 803 ms/image, respectively.Source: B. TaskGuard gain over Visual-pair · p. 7

6. Discussion

TaskGuard prioritizes controlling restoration risk over maximizing aggregate AP, using a TRAIN-only threshold that can be adjusted for intervention costs. Its directional evidence is informative but imperfect, and its evaluation distinguishes loss-defined utility from practical detection outcomes.

6. Discussion

  • TaskGuard deliberately trades a small amount of aggregate AP for fewer negative interventions, with its threshold fixed by a TRAIN-only utility-retention constraint.Applications with different intervention costs can choose another operating point using development data.Source: §6 · p. 7
  • The pseudo-gradient has only modest direct agreement with exact regional counterfactual utility, but remains aligned with its privileged counterpart and improves zero-shot whole-image utility prediction.Confident restoration-induced prediction errors can also corrupt pseudo-labels obtained from restored-image detections.Source: §6 · p. 7
  • TaskGuard must generate the candidate restoration before judging it, adds gradient-computation overhead, and is not shown to transfer arbitrarily across restorers, detectors, datasets, or tasks.The zero-shot claim is limited to degradation-family transfer from Gaussian training to motion, rain, and defocus; DAWN is an additional natural-domain test.Source: §6 · p. 7
  • Detector-loss utility is not equivalent to AP or per-image F1, so both loss-defined and practical-negative evaluations are needed.The practical-negative analysis finds a meaningful association with concrete detection deterioration, but the relationship is not one-to-one.Source: §6 · p. 8

7. Conclusion

TaskGuard treats restoration as a task-conditioned intervention, using the realized image change and frozen-detector sensitivity to decide whether to accept a candidate. Its supplementary material details the representation and frozen protocol, including TRAIN-only threshold selection.

7. Conclusion

  • TaskGuard couples the actual restoration residual with frozen-detector sensitivity to decide whether a candidate should be accepted, and transfers from Gaussian training to unseen controlled degradations and natural rain.Exact regional counterfactuals, pseudo-gradient validation, and feature-group ablation support the directional mechanism as informative, though not exact; the policy retains most of unconditional restoration's benefit while reducing negative interventions.Source: §7 · p. 8

A. Supplement Overview

  • The supplement provides implementation details and analyses on the 43-D representation, frozen inference ensemble, feature-group ablation, pseudo-gradient validation, threshold sensitivity, and additional transfer and runtime results.Source: A. Supplement Overview · p. 10

B. Exact TaskGuard Representation

  • The TaskGuard representation comprises 29 task-gradient, 11 detector-response, and 3 residual features, with no degradation family, severity, or restorer identity supplied.It uses 64 spatial masks on an 8 × 8 grid; the masks are feathered rather than strictly binary.Source: B. Exact TaskGuard Representation · p. 10, Table 1 · p. 11

C. Frozen Training and Inference Protocol

  • The inference predictor uses 50 fold-specific regressors from repeated five-fold, source-grouped stratified splits rather than a single refit on all TRAIN samples.The candidate is ridge regression, and the Gaussian TRAIN set contains 350 source images and 1050 degraded/restored pairs.Source: C. Frozen Training and Inference… · p. 10

C.1. TRAIN-only threshold selection

  • The threshold is selected using Gaussian TRAIN out-of-fold scores to minimize the policy loss-negative intervention rate subject to retention of at least 0.98.Validation confirms rather than retunes the threshold; at the frozen point, utility-ranking ρs = 0.597, benefit retention 95.9%, loss-negative rate 10.0%, and relative loss-negative reduction 46.4%.Source: §C.1 · p. 11

D. Complete Feature-Group Ablation

Feature ablations and diagnostics show that task-gradient evidence contributes distinct utility information, while the deployable pseudo-gradient preserves directional signal imperfectly. On unseen controlled degradations, TaskGuard outperforms appearance-based baselines.

D. Complete Feature-Group Ablation

  • Task-gradient features were the strongest individual utility-ranking signal, while the full 43-D model led practical-negative reduction without maximizing every metric.Other subsets slightly exceeded the full representation on individual metrics.Source: D. Complete Feature-Group Ablation · p. 11

D.1. Paired source-bootstrap comparisons

  • Task-gradient features add information beyond appearance-scale residual statistics or detector response alone, while residual and detector-response features complement the final risk-aware policy.Source: §D.1 · p. 11

E. Pseudo-Gradient Versus Exact Counterfactual Utility

  • Pseudo-gradient agreement with exact counterfactual utility was weaker than privileged-gradient agreement, but source-bootstrap intervals remained above null or chance baselines.The pseudo-gradient also remained substantially aligned with the privileged gradient, supporting attenuation rather than removal of the directional task signal.Source: E. Pseudo-Gradient Versus Exact… · p. 12

E.1. Gaussian severity diagnostic

  • The pseudo-gradient association strengthened at the highest Gaussian severity, but this TRAIN-only pattern is not a primary claim.Source: §E.1 · p. 12

F. Threshold Operating-Point Sensitivity

G. Appearance-Baseline Details

  • On unseen motion/rain/defocus, TaskGuard reached ρs = 0.622, 93.3% benefit retention, and 54.2% loss-negative reduction, versus Visual-pair's ρs = 0.211, 79.6% benefit retention, and 30.3% loss-negative reduction.Source: G. Appearance-Baseline Details · p. 13

H. Natural-Rain DAWN Details

On natural-rain DAWN, TaskGuard reduced loss-negative interventions while retaining much of the deraining AP gain. The paper also reports partial transfer to another detector, practical detection-risk results, runtime costs, and a locked evaluation protocol.

H. Natural-Rain DAWN Details

  • On DAWN, TaskGuard reached AP 0.376466, retained 77.8% of the Always-Derain AP gain, and reduced loss-negative interventions by 97.9%.Only six Always-Derain cases were practical-negative, so practical-risk comparisons are descriptive.Source: H. Natural-Rain DAWN Details · p. 13

I. Cross-Detector Safety Transfer

  • On 600 fresh Gaussian pairs, frozen TaskGuard decisions retained 93.92% of RT-DETR-L loss-defined benefit and reduced its loss-negative rate from 28.50% to 19.83%, a 30.41% relative reduction.The weaker RT-DETR-L result motivates describing this as partial cross-detector safety transfer, not detector invariance.Source: §I · p. 13

J. Practical Detection-Utility Analysis

  • On pooled unseen families, practical-negative pairs fell from 18.33% under Always Restore to 11.56% under TaskGuard, a 36.97% relative reduction.A practical deterioration occurred in 34.32% of loss-negative restorations versus 13.73% of non-loss-negative restorations; detector loss is a useful proxy but not identical to detection quality.Source: J. Practical Detection-Utility Analysis · p. 14

K. Runtime Details

  • The post-restoration TaskGuard stage requires 162.865 ms/image, adding 136.134 ms/image; its increment is approximately 20.4% relative to Restore+Detector.Contextual pipeline timings include Restormer process-startup and model-initialization overhead, so they are not optimized persistent-model latency.Source: K. Runtime Details · p. 14

L. Reproducibility Checklist

  • The policy was fitted and thresholded using Gaussian TRAIN only, then evaluated on 200 untouched, source-disjoint sources and 2400 pairs without final-set retuning.Motion blur, rain, and defocus were unseen during fitting.Source: L. Reproducibility Checklist · p. 15

Related papers

Loading 2609.08011v1…