PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantifying Cross-Modality Memorization in Vision-Language Models

Yuxin Wen, Yangsibo Huang, Tom Goldstein, Ravi Kumar, Badih Ghazi, Chiyuan Zhang

arXiv:2506.05198v1cs.CVcs.LG

TL;DR

Prior memorization research has focused mainly on individual modalities, while practical multimodal models make cross-modal factual recall important to study. This work uses synthetic personas and controlled single-modality training to measure transfer in vision-language models. It finds facts transfer across modalities but with a significant and asymmetric gap, and reports that in-distribution synthetic image-text augmentation can help bridge it.

  • Problem

    Prior studies primarily investigate memorization within a single modality, while multimodal models increasingly process and integrate diverse media.

    Source: §1 · p. 2

  • Method

    The study introduces synthetic persona images and descriptions, trains vision-language models on one modality, and evaluates factual recall in the other.

    Source: §1 · p. 2

  • Results

    Facts learned in one modality transfer to the other, but target-modality recall lags source-modality recall, with image-to-text transfer more effective than text-to-image transfer.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    In-distribution synthetic image-text augmentation can bridge the modality gap, whereas out-of-distribution data is less beneficial.

    Source: §1 · p. 3

    The study focuses on text and image modalities; exploring additional modalities remains future work.

    Source: §5 · p. 10

Abstract

from arXiv · show

Understanding what and how neural networks memorize during training is crucial, both from the perspective of unintentional memorization of potentially sensitive information and from the standpoint of effective knowledge acquisition for real-world, knowledge-intensive tasks. While previous studies primarily investigate memorization within a single modality, such as text memorization in large language models or image memorization in diffusion models, unified multimodal models are becoming increasingly prevalent in practical applications. In this work, we focus on the unique characteristics of cross-modality memorization and conduct a systematic study centered on vision-language models. To facilitate controlled experiments, we first introduce a synthetic persona dataset comprising diverse synthetic person images and textual descriptions. We quantify factual knowledge memorization and cross-modal transferability by training models on a single modality and evaluating their performance in the other. Our results reveal that facts learned in one modality transfer to the other, but a significant gap exists between recalling information in the source and target modalities. Furthermore, we observe that this gap exists across various scenarios, including more capable models, machine unlearning, and the multi-hop case. At the end, we propose a baseline method to mitigate this challenge. We hope our study can inspire future research on developing more robust multimodal learning techniques to enhance cross-modal transferability.

1 Introduction

The introduction extends memorization research from individual modalities to cross-modal factual recall in vision-language models. Using synthetic personas and controlled training, it finds that facts transfer across modalities but with an asymmetric source–target performance gap, and tests strategies to mitigate it.

1 Introduction

  • The study systematically quantifies how knowledge learned in one modality transfers to inference in another under controlled training conditions.It compares accuracy between the training and testing modalities.Source: §1 · p. 2
  • The focus on factual knowledge across text and images reflects personal-assistant settings that use user-provided data such as emails, calendars, photos, and screenshots.The introduction frames understanding cross-modal factual transfer as a step toward robust and trustworthy multimodal personal agents.Source: §1 · p. 2
  • The synthetic persona dataset pairs diverse persona profiles with textual descriptions and visual representations, including synthetic photographs.It is designed to investigate cross-modal factual memorization.Source: §1 · p. 2
  • Facts transfer across modalities, but recall is weaker in the target modality, with image-to-text transfer more effective than text-to-image transfer.The asymmetry persists even in larger, more capable models.Source: §1 · pp. 2–3
  • Cross-modal inconsistency also appears in unlearning and multi-hop inference, where models struggle to unlearn facts across modalities or infer complex relations.The multi-hop difficulty occurs despite successful learning of simpler single-hop associations.Source: §1 · p. 3
  • In-distribution synthetic image-text augmentation can bridge the modality gap, while diverse data and larger models do not improve transferability and out-of-distribution augmentation is less beneficial.The study examines these strategies as potential ways to improve cross-modal transfer.Source: §1 · p. 3

2 Related Work

Prior memorization research has largely examined text and images separately, studying both verbatim reproduction and factual knowledge in generative models.

2 Related Work

  • Language-model research examines regurgitation of training text and finds memorization related to model size, canary repetition frequency, and sequence length [Carlini et al., 2019, 2022; Kandpal et al., 2022].Factual knowledge memorization has also been studied as recall prompted by queries that differ from the training instances [Allen-Zhu and Li, 2023].Source: §2 · p. 3
  • Diffusion models can reproduce training images, with memorization extent depending on model and data sizes [Somepalli et al., 2023b; Carlini et al., 2023].Related image-generation work also studies memorized styles and copyrighted characters [Somepalli et al., 2024; He et al., 2024].Source: §2 · p. 3

3 Cross-Modality Factual Knowledge Memorization

The study uses synthetic personas to test how factual knowledge transfers between image and text inputs in vision-language models. Transfer remains weaker than source-modality recall across training variations, unlearning, and multi-hop tasks; image-caption augmentation improves transfer when it uses in-distribution data.

3.1 Synthetic Persona Data

  • The synthetic persona dataset contains 100 unique personas, each with 100 image variants for training and one distinct test image, plus 100 textual-description variants for training and one distinct test description.Source: §3.1 · p. 4

3.2 Experimental Setup

  • Models are trained on descriptions or images and tested on both modalities using held-out inputs for personas seen during training.Recognition accuracy is compared between the training and testing modalities.Source: §3.2 · p. 5

3.3 Memorization Results

  • Factual knowledge transfers across modalities, but recall remains significantly lower in the target modality than in the training modality.The gap persists with diverse inputs, which mitigate overfitting, and larger models, which raise accuracy without materially changing the transfer rate.Source: §3.3 · p. 6
  • Training on varied inputs improves accuracy and mitigates image-training overfitting, while larger models improve accuracy but leave the cross-modal transfer rate nearly unchanged.With image training, performance falls after 25 epochs when a single input is repeated; using new inputs per epoch avoids this early overfitting, but does not improve transferability.Source: §3.3 · pp. 5–7

3.4 Mitigation with Image-Caption Augmentation

  • In-distribution synthetic image-caption augmentation improves cross-modal transfer, whereas out-of-distribution COCO augmentation provides no similar benefit.The synthetic pairs are generated by producing images from text and captions from images; the in-distribution additions are held-out persona data.Source: §3.4 · p. 7

3.5 Cross-Modal Unlearning

  • Unlearning effects transfer unevenly across modalities when the unlearning data differ from the modality in which the fact was learned.The experiments replace each profile’s name and continue training; same-modality learning and unlearning produce more similar drops across both test modalities.Source: §3.5 · p. 8

3.6 Cross-Modal Multi-Hop Learning

  • In multi-hop tests, perfect accuracy on individual tasks can coexist with only 11% accuracy when retrieving a favorite number from a description.Image-based training yields over 50% accuracy for Image⇒FN, and higher Desc⇒FN accuracy than description-based training.Source: §3.6 · p. 9
  • Membership inference detects unintended memorization of irrelevant image-injected favorite numbers, with AUC reaching around 70% after 100 epochs.Direct extraction success is near zero, while the membership-inference signal grows during training.Source: §3.6 · pp. 9–10

5 Conclusion and Future Work

The study finds asymmetric cross-modal transfer, alongside challenges in cross-modal unlearning and multi-hop inference. In-distribution synthetic image-text augmentation helps bridge the transfer gap, while out-of-distribution data is less effective.

5 Conclusion and Future Work

  • In-distribution synthetic image-text augmentation helps bridge the cross-modal transfer gap, while out-of-distribution data is less effective.Source: §5 · p. 10
  • Knowledge transfers between modalities asymmetrically, with image-to-text transfer more effective than text-to-image transfer.Source: §5 · p. 10
  • Unlearning facts in one modality does not fully translate to the other, and models struggle with complex cross-modal relations despite learning simpler single-hop associations.Source: §5 · p. 10
  • The study focuses on text and image modalities; future work should explore additional modalities, improved unlearning techniques, and approaches to multi-hop reasoning.Source: §5 · p. 10

A Broader Impacts

The paper reports knowledge transfer between vision and text despite a significant, asymmetric gap. It notes potential privacy implications and recommends evaluating cross-modal interactions in applications handling sensitive or personal data.

A Broader Impacts

  • Knowledge transfers between vision and text, but a significant gap remains and transfer is asymmetric.Source: A Broader Impacts · p. 15
  • Cross-modal knowledge interactions may allow sensitive information learned from images to influence text outputs, or vice versa.Source: A Broader Impacts · p. 15
  • Researchers and practitioners are encouraged to evaluate cross-modal interactions and investigate techniques that protect privacy while maintaining performance.Source: A Broader Impacts · p. 15

B Attribute Pool

The attribute pool spans demographic identity and age, appearance and accessories, and facial expression and background.

B Attribute Pool

  • Demographic identity covers eight listed groups and nationalities, while age ranges are adult, middle aged, and senior.Source: B Attribute Pool · p. 15
  • Appearance attributes include clothing type and color, accessories, hairstyle, and hair color.Source: B Attribute Pool · p. 15
  • The pool also includes seven facial expressions and twelve background options.Source: B Attribute Pool · p. 15

C Data Generation Prompts

The image prompt specifies a photorealistic person through demographic, appearance, expression, and background attributes. The description prompt asks for a natural, detailed portrayal using those characteristics.

C Data Generation Prompts

  • The Imagen 3 prompt requests a highly detailed, photorealistic person image with specified demographic, clothing, accessory, hair, expression, and background attributes.Source: C Data Generation Prompts · p. 15
  • The Gemini 2.0 prompt asks for a detailed, natural description that makes the person recognizable and incorporates the specified attributes.Source: C Data Generation Prompts · p. 16
  • The description should read like an account of a friend and be provided directly, without extra formatting or instructions.Source: C Data Generation Prompts · p. 16

D Training and Testing Prompts

The training and testing prompts frame name recall as a question grounded in either image data or description data. The description prompt includes a sample person description.

D Training and Testing Prompts

E Test Prompt Engineering

The authors tested appending prompts intended to encourage cross-domain recall during inference. Neither prompt significantly changed transferability.

E Test Prompt Engineering

Loading 2506.05198v1…