PaperlayerView exact revision on arXiv

Source-linked AI summary

MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models

Dohwan Ko, Joonmyung Choi, Hyeong Kyu Choi, Kyoung-Woon On, Byungseok Roh, Hyunwoo J. Kim

arXiv:2303.13009v1cs.CV

TL;DR

Fine-tuning video foundation models often minimizes only a task-specific loss, leaving potentially useful pretext losses unused. MELTR learns to combine losses through auxiliary learning and bi-level optimization. Across multiple models and downstream tasks, it outperforms single-task and multi-task baselines.

  • Problem

    Fine-tuning often minimizes a single target-task loss, while other potentially beneficial pretext-task losses remain unexplored.

    Source: §1 · p. 1

  • Method

    MELTR is a plug-in module that non-linearly combines auxiliary losses into a unified loss, formulated as bi-level optimization and trained with an efficient AID-based procedure.

    Source: §1 · pp. 1–2

  • Results

    MELTR significantly outperforms single-task and multi-task fine-tuning baselines across three video foundation models, four downstream tasks, and five benchmark datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MELTR learns to adaptively re-weight and non-linearly combine pretext-task losses for the target downstream task.

    Source: §1 · p. 2

Abstract

from arXiv · show

Foundation models have shown outstanding performance and generalization capabilities across domains. Since most studies on foundation models mainly focus on the pretraining phase, a naive strategy to minimize a single task-specific loss is adopted for fine-tuning. However, such fine-tuning methods do not fully leverage other losses that are potentially beneficial for the target task. Therefore, we propose MEta Loss TRansformer (MELTR), a plug-in module that automatically and non-linearly combines various loss functions to aid learning the target task via auxiliary learning. We formulate the auxiliary learning as a bi-level optimization problem and present an efficient optimization algorithm based on Approximate Implicit Differentiation (AID). For evaluation, we apply our framework to various video foundation models (UniVL, Violet and All-in-one), and show significant performance gain on all four downstream tasks: text-to-video retrieval, video question answering, video captioning, and multi-modal sentiment analysis. Our qualitative analyses demonstrate that MELTR adequately `transforms' individual loss functions and `melts' them into an effective unified loss. Code is available at https://github.com/mlvlab/MELTR.

1. Introduction

Fine-tuning often minimizes only a target-task loss, leaving potentially useful pretext losses unused. MELTR learns a unified auxiliary loss for fine-tuning and is evaluated across video foundation models and downstream tasks.

1. Introduction

  • Fine-tuning video foundation models commonly minimizes a single target loss, leaving other potentially beneficial pretext-task losses unexplored.For example, UniVL is pretrained with several pretext tasks but fine-tuned by minimizing a single target loss.Source: §1 · p. 1
  • Auxiliary learning focuses on the primary task while using auxiliary tasks, and meta-learning can automate how those tasks are leveraged.Prior frameworks often selected auxiliary tasks manually, requiring domain knowledge and not guaranteeing they would benefit the primary task.Source: §1 · p. 1
  • MELTR is a plug-in module that automatically and non-linearly transforms auxiliary losses into a unified loss for the target downstream task.It takes target-task and pretext-task losses as input and learns their relationship via self-attention.Source: §1 · p. 1
  • MELTR formulates auxiliary learning as a bi-level optimization problem and presents an efficient training scheme that approximates the inverse Hessian.The approximation addresses the computational cost associated with second-order derivatives and inverse-Hessian computation.Source: §1 · pp. 1–2
  • MELTR significantly outperforms single-task and multi-task fine-tuning baselines across three video foundation models, four downstream tasks, and five benchmark datasets.The evaluated tasks include retrieval, question answering, captioning, and multi-modal sentiment analysis.Source: §1 · p. 2
  • Qualitative analyses examine how MELTR non-linearly transforms individual loss functions and combines them into a unified loss.The analyses concern the loss transformation for the target downstream task.Source: §1 · p. 2

2. Related work

Video foundation models use multimodal and contrastive pretraining approaches, while their pretext losses are often not used during fine-tuning. Related auxiliary-learning methods include multi-task training and meta-learning approaches that select auxiliary tasks.

Video foundation models

  • Video foundation models address multimodal data with cross-modal attention and use contrastive learning for representations or video-text alignment.The cited work also studies self-supervised alignment where labels or descriptions are lacking.Source: §2 · p. 2
  • Video foundation models such as MERLOT, VATT, and UniVL have been applied to varied video reasoning, vision, retrieval, and language tasks.UniVL uses auxiliary losses in pretraining and reports performance across several downstream tasks after fine-tuning.Source: §2 · p. 2

Auxiliary learning

  • Pretext losses used to pretrain foundation models are often ignored during fine-tuning, which instead minimizes only the primary task loss.Multi-task learning trains one model with multiple task losses and is commonly used for self-supervised foundation-model pretraining.Source: §2 · p. 2
  • Meta-learning auxiliary-learning methods learn auxiliary tasks to generalize well to unseen data, often using held-out primary-task data and other data for auxiliary tasks.Related applications include semantic segmentation, reinforcement-learning navigation, and graph self-supervised learning.Source: §2 · p. 2

3. Preliminaries

The preliminaries introduce UniVL and bi-level optimization schemes used in meta-learning. They establish that pretext losses may be unused in fine-tuning and describe ITD and AID approaches to hypergradient computation.

3.1. UniVL

  • UniVL is a video-and-language foundation model pretrained on HowTo100M with five pretext tasks whose losses are summed.Its tasks include joint video-text learning, conditioned masked language and frame modeling, video-text alignment, and language generation.Source: §3.1 · p. 3
  • UniVL fine-tuning minimizes only one target-task loss, leaving other pretraining losses unused even when they may help the downstream task.The section presents this as motivation for learning how to combine multiple losses through meta-learning.Source: §3.1 · p. 3

3.2. Bi-level optimization and hypergradient approximation

  • Bi-level optimization separates an upper-level objective from a lower-level decision vector; in hyperparameter optimization, these correspond to validation loss and model parameters optimized on training loss.The upper-level gradient, or hypergradient, updates the upper-level decision vector.Source: §3.2 · p. 3
  • ITD approximates the hypergradient using a fixed-point parameter, whereas AID factorizes it via the implicit function theorem and approximates inverse-Hessian computation.The figure contrasts ITD's fixed-point parameter with AID's two-step scheme.Source: §3.2 · p. 3, Figure 1 · p. 3

4. Method

MELTR learns to fine-tune foundation models by adaptively combining primary and auxiliary task losses, with a bi-level objective optimized using an efficient AID procedure. Its transformer maps task losses to embeddings, attends across them, and produces a unified loss; regularization keeps that loss within a reasonable range.

4. Method

  • MELTR combines primary and auxiliary losses into a learned unified loss to assist fine-tuning, framing this as a bi-level optimization problem.The framework uses an efficient training procedure with Approximate Implicit Differentiation (AID) and regularizes the loss combination to alleviate meta-overfitting.Source: §4 · p. 3, §4.1 · p. 3

4.1. Meta Loss Transformer

  • Scalar losses are projected through an MLP and paired with learnable task embeddings; self-attention uses both loss-scale and task information to produce a scalar combined loss.The target-task loss is included among the input losses, and the Transformer is the module that combines them.Source: §4.1 · p. 4, Figure 2 · p. 4
  • When meta-data is small, meta-parameters may overfit primary-task performance, so MELTR regularizes its learned loss to remain within a reasonable range.The primary loss adds a regularization term weighted by γ; L0 is the original supervised target-task loss.Source: §4.1 · p. 4

4.2. Objective function and optimization

  • The bi-level objective optimizes backbone parameters using the unified auxiliary loss, then updates MELTR’s meta-parameters against the primary loss using an AID hypergradient.Approximating the Hessian as the identity removes inverse-Hessian computation, otherwise O(n^3); experiments report no significant fine-tuned-model performance degradation.Source: §4.2 · p. 5

5. Experiments

MELTR is evaluated across video foundation models and downstream tasks, then analyzed for how it combines losses and how efficiently it can be optimized.

5. Experiments

  • The experiments apply MELTR to UniVL, Violet, and All-in-one across four downstream tasks and five benchmark datasets.The evaluation includes text-to-video retrieval, video question answering, video captioning, and multi-modal sentiment analysis.Source: §5 · p. 5

5.1. Evaluation on downstream tasks

  • 4.8% higher R@1 than UniVL and 1.5% higher than VideoCLIP on YouCook2 retrieval; the regularized MELTR improves all metrics over MELTR−.The table compares UniVL-Joint, UniVL-Align, MELTR, and MELTR without Lreg.Source: §5.1 · p. 6, Table 1 · p. 6
  • MELTR outperforms baselines across retrieval, question answering, and captioning, including on MSRVTT retrieval and TGIF-QA/MSVD-QA.For multi-modal sentiment analysis on CMU-MOSI, it also surpasses all baselines.Source: §5.1 · pp. 6–7, Table 2 · p. 6, Table 3 · p. 6

5.2. Analysis on MELTR

  • MELTR’s loss transformation is nonlinear, and during captioning training its gradients increasingly favor Decoder and M-Decoder over CMFM.The loss sensitivity peaks around ℓt = 0.5 and decreases for larger losses, while the epoch-wise gradients begin at similar scales.Source: §5.2 · p. 7, Figure 3 · p. 7, Figure 4 · p. 7
  • Manual multi-task loss combinations can hinder fine-tuning when they include harmful losses, whereas MELTR surpasses the compared schemes.Dropping CMFM slightly improves one manual scheme, and reweighting losses improves another.Source: §5.2 · pp. 7–8, Table 7 · p. 8

5.3. Efficient optimization algorithm

  • MELTR + AID-FP-Lite† adds 4.9% training-time overhead over multi-task learning while improving R@1 by 2.4%.The identity-matrix approximation is the fastest bi-level scheme and achieves strong R@1 performance.Source: §5.3 · p. 8, Table 8 · p. 8

6. Conclusion

The paper concludes that MELTR adaptively integrates pretext losses for video foundation-model fine-tuning and reports strong downstream results with an efficient training procedure.

6. Conclusion

  • MELTR integrates pretext losses into one loss for the target task and outperforms state-of-the-art video foundation and task-specific models across downstream tasks.Its AID-FP-Lite procedure uses a simple inverse-Hessian approximation without significant performance loss.Source: §6 · p. 8

A.1. Backbone Foundation Models

  • UniVL, Violet, and All-in-one use different pretraining datasets and task-loss configurations in the implementation.For UniVL, the primary losses include LAlign for retrieval and sentiment analysis and LDecoder for captioning.Source: §A.1 · p. 9

A.2. Evaluation metrics

The supplement specifies evaluation metrics, implementation settings, benchmark datasets, and an ablation of MELTR’s regularization strength.

A.2. Evaluation metrics

  • Retrieval uses R@K (K=1,5,10) and MedR, question answering uses accuracy, and video captioning uses BLEU, METEOR, ROUGE-L, and CIDEr.Source: §A.2 · p. 9

A.3. MELTR Details.

  • MELTR uses one 8-head, 512-dimensional Transformer encoder layer, with γ searched over {0.1, 0.3, 0.5} and K = 3.Training runs for 40 epochs on retrieval and 20 epochs each on question answering and captioning, using 8 Tesla A100 GPUs.Source: §A.3 · p. 9

C. Effectiveness of the Regularization Term

D. Effectiveness of transformer architecture

On MSRVTT text-to-video retrieval, the Transformer architecture improves MELTR over a linear layer, with self-attention identified as the advantage.

D. Effectiveness of transformer architecture

E. Effectiveness of input type

MELTR’s combined scale and task embeddings support loss weighting that reflects both task identity and loss scale; its learned transformation also outperforms linear combinations on MSRVTT retrieval.

F. Non-linearity of MELTR

  • Figure 7 compares MELTR and MTL outputs as functions of LDecoder and LCMFM to illustrate their transformation behavior.Source: Figure 7 · p. 11

E. Effectiveness of input type

  • With scale and task embeddings together, MELTR considers task-advantageous losses while focusing on reasonably challenging samples.Scale-only inputs yield loss-range-based groupings, while task-only inputs make gradients invariant to loss scale; Figure 6 shows CMLM/CMFM losses have distinct ranges from the others.Source: E. Effectiveness of input type · pp. 10–11, Figure 6 · p. 11

F. Non-linearity of MELTR

G. Effectiveness of advanced loss of UniVL

For YouCook2 video captioning, adding UniVL losses improves average performance, with LM-Decoder the most effective individual addition among those tested.

G. Effectiveness of advanced loss of UniVL

H. Adaptation to a new baseline and tasks

MELTR is evaluated with a new video baseline, a video-only task, and an image-classification task, with improved performance reported in each setting.

H. Adaptation to a new baseline and tasks

Related papers

Loading 2303.13009v1…