PaperlayerView exact revision on arXiv

Source-linked AI summary

DLC: A Metric-Guided Dynamic Loss Controller for Multi-Objective Training

Jaewan Ko, Janghoon Choi

arXiv:2610.11433v1cs.CV

TL;DR

Restoration training typically fixes loss weights even though fidelity and perceptual quality can conflict. DLC uses an LLM-based controller to adjust existing losses from quality-metric feedback during training. Across restoration domains, it improves key fidelity or perceptual metrics while leaving trade-offs explicit.

  • Problem

    Fixed loss weights assume one balance among fidelity, perceptual quality, and other objectives remains appropriate throughout optimization, although these metrics can conflict.

    Source: §1 · p. 2

  • Method

    DLC periodically evaluates a fixed feedback subset and uses metric profiles, recent changes, and loss-weight history to update existing loss weights through an LLM-based controller.

    Source: §3.3 · p. 7, §5 · p. 14

  • Results

    Across low-light enhancement, deraining, and real-world super-resolution, DLC improves key fidelity or perceptual metrics while making remaining trade-offs explicit.

    Source: §1 · p. 3, §5 · p. 14

  • Takeaways & Limitations

    DLC can steer existing restoration training toward useful operating points while keeping the remaining metric trade-offs visible.

    Source: §5 · p. 14

Abstract

from arXiv · show

In this paper, we introduce a metric-guided dynamic loss controller (DLC) for multi-objective image restoration. Conventional image restoration pipelines usually train with a fixed weighted combination of multiple losses, without changing the relative importance of fidelity, perceptual similarity, and no-reference quality during optimization. DLC is an architecture- and loss-term-agnostic training-time controller: it does not modify the restoration architecture or introduce new differentiable loss terms, but dynamically reweights the existing training losses. During training, DLC periodically evaluates the current model on a small fixed feedback subset and uses the resulting quality metrics to update the loss-weight vector through an LLM-based controller. Because DLC operates on existing loss terms rather than task-specific architectures, the same controller formulation can be instantiated across diverse image restoration training pipelines. We evaluate DLC on three restoration domains: low-light image enhancement, deraining, and real-world super-resolution, using both reference-based and no-reference quality metrics. Across these settings, DLC considers metric-dependent trade-offs during optimization and guides training toward balanced operating points across fidelity and perceptual quality. The results show that DLC can move models toward more favorable operating points across different restoration domains, supporting its role as a practical plug-in controller for multi-objective image restoration.

1 Introduction

DLC addresses fixed loss weights and the tension between fidelity and perceptual quality by using quality-metric feedback to adjust existing losses during training. Its evaluations across restoration domains show metric improvements while keeping trade-offs explicit.

1 Introduction

  • Fixed loss weights assume one balance among objectives remains appropriate throughout training, despite fidelity and perceptual metrics sometimes favoring conflicting outcomes.Restoration benchmarks also use heterogeneous evaluation metrics that may differ from the signals used to tune training objectives.Source: §1 · p. 2
  • DLC periodically evaluates the current model on a fixed feedback subset and uses metric profiles, recent changes, and loss-weight history to update weights for the next training interval.It preserves the original model, optimizer, data pipeline, and training losses.Source: §1 · p. 3
  • DLC instantiates the same feedback-to-action formulation in low-light enhancement, deraining, and real-world super-resolution despite differences in architecture, supervision, and loss design.The adjustable losses differ across these pipelines.Source: §1 · p. 3
  • DLC improves key fidelity or perceptual metrics across restoration evaluations while making the remaining trade-offs among metrics explicit.The results do not show that one method improves every metric simultaneously.Source: §1 · pp. 3–4

2 Related Work

Prior work motivates dynamic loss weighting but often relies on training signals that differ from restoration evaluation metrics. DLC connects those metrics to an LLM-based controller that adjusts existing loss weights.

2.1 Composite Losses in Image Restoration

  • Composite restoration losses combine objectives such as fidelity, structural consistency, and perceptual similarity, but their weights are typically fixed throughout optimization.DLC instead dynamically adjusts the weights of the same training losses according to metric feedback.Source: §2.1 · p. 4

2.2 Dynamic Loss Weighting for Multi-Objective Training

  • Existing dynamic weighting methods use signals such as uncertainty, training-loss changes, gradients, or multi-objective gradient information rather than restoration quality metrics.DLC uses heterogeneous evaluation metrics as training-time feedback for loss-weight control.Source: §2.2 · p. 4

2.3 Quality Metrics for Image Restoration Feedback

  • Restoration metrics cover fidelity, reference-based perception, no-reference quality, and realism, and their rankings can conflict.DLC preserves the multi-metric profile instead of collapsing it into one scalar reward.Source: §2.3 · p. 5

2.4 LLM-Based Optimization and Loss Control

  • DLC gives the LLM a narrow operational role: it maps structured metric feedback and training history to new weights within the existing loss set.The LLM does not generate images, synthesize labels, or design a new architecture.Source: §2.4 · p. 5

3 Method

DLC periodically evaluates restoration quality and uses metric feedback to update existing loss weights, while leaving the model architecture and training pipeline unchanged. Its controller turns a structured metric state into weights for the next training interval.

3.1 Problem Formulation

  • DLC replaces fixed loss weights with a vector that remains constant within each training interval and changes only at discrete intervention steps.The objective combines existing loss terms, whose weights can represent fidelity, structural consistency, perceptual similarity, realism, or task-specific constraints.Source: §3.1 · pp. 5–6

3.2 DLC Feedback Loop

  • At each intervention, DLC evaluates the model on an unchanged feedback subset, then uses quality metrics and recent changes to select the next loss weights.The subset is used only for controller feedback; final performance is measured on standard benchmark test sets, and the metrics are not optimized as differentiable losses.Source: §3.2 · pp. 6–7

3.3 Feedback State and Controller Output

  • The structured controller state preserves metric values and optimization directions alongside recent changes, current weights, loss terms, history, and a task instruction.The controller returns numeric loss weights; any textual rationale is for logging and analysis, not an optimization signal.Source: §3.3 · p. 7

3.4 Domain Instantiations

  • The same feedback-to-action formulation controls existing, task-dependent losses across low-light enhancement, deraining, and real-world super-resolution.DLC inherits each pipeline’s loss terms and adds no task-specific architectural changes.Source: §3.4 · p. 8

3.5 Training Procedure

  • The training procedure alternates between training with the current weights, evaluating on the feedback subset, constructing a feedback state, and querying the controller for updated weights.The model, optimizer, data pipeline, and training budget remain unchanged; only weights change at intervention steps.Source: §3.5 · p. 8

4 Experiments

Across three restoration domains, DLC is evaluated against fixed-loss baselines with unchanged backbones and training losses. The experiments examine metric trade-offs, weighting policies, controller choices, weight trajectories, and training cost.

4.1 Experimental Setup

  • The baseline and DLC share the backbone, data, optimizer, and losses; DLC changes only the loss-weight policy and uses a fixed 10-sample feedback subset.Low-light enhancement and deraining report PSNR, SSIM, LPIPS, NIQE, CLIP-IQA, and BRISQUE; super-resolution uses a broader metric set.Source: §4.1 · p. 9

4.2 Main Quantitative Results

  • Across low-light enhancement, deraining, and real-world super-resolution, DLC improves selected fidelity or perceptual metrics while retaining dataset- and metric-dependent trade-offs.Low-light results include gains across multiple metrics; deraining outcomes vary by dataset; super-resolution improves key fidelity and reference-based perceptual metrics on both datasets.Source: Table 2 · p. 9, §4.2 · pp. 9–10, Table 3 · p. 10

4.3 Qualitative Results

  • Across domains, DLC often produces clearer local structures or more favorable perceptual details in zoomed crops, though improvements are not uniform across regions.The crops illustrate visual shifts that accompany the metric-level trends without changing the underlying restoration architecture.Source: §4.3 · pp. 10–11, Figure 3 · p. 11

4.4 Comparison of Loss-Weighting Policies

  • On LOL-v1, DLC achieves the highest PSNR while remaining competitive on SSIM and perceptual quality; other policies lead on selected perceptual or no-reference metrics.The comparison holds the backbone, dataset, losses, optimizer, and training budget constant while varying the loss-weight decision policy.Source: §4.4 · pp. 11–12, Table 5 · p. 12

4.5 Controller LLM Sensitivity

  • Qwen2.5-7B, Mistral-7B, and LLaMA-3 8B controllers show comparable performance, indicating DLC is not tied to one LLM backbone.Source: §4.5 · p. 12, Table 6 · p. 13

4.6 Loss-Weight Trajectory Analysis

  • On LOL-v1, pixel and perceptual weights gradually increase while the structural weight stays relatively stable; when fidelity improves but perceptual feedback degrades, DLC raises the next perceptual weight.The trajectory illustrates adaptation to observed metric trade-offs rather than a fixed update heuristic.Source: Figure 5 · p. 13, §4.6 · p. 13

4.7 Training-Time and Test-Time Cost

  • DLC adds training-time overhead only: relative overhead is below 0.2% in low-light enhancement and deraining, and 1.69% in measured super-resolution runs.The final model has the same parameter count and test-time cost as the baseline.Source: Table 7 · p. 14, §4.7 · p. 14

5 Conclusion

DLC updates existing loss weights at discrete training intervals using metric feedback and an LLM-based controller. Across three restoration domains, it improves selected fidelity or perceptual metrics while making trade-offs explicit, with no added test-time cost.

  • DLC preserves the model, optimizer, data pipeline, and training losses, updating only the loss-weight vector from metric feedback, recent metric changes, and weight history.Source: §5 · p. 14
  • Across low-light enhancement, deraining, and real-world super-resolution, DLC improves key fidelity or perceptual metrics while making remaining trade-offs explicit.The policy ablation indicates that metric feedback guides the update direction, rather than benefit coming merely from changing weights.Source: §5 · p. 14
  • DLC is a training-time plug-in with no additional test-time cost; future work includes feedback-subset selection, intervention frequency, and controller design.Source: §5 · p. 14
Loading 2610.11433v1…