PaperlayerView exact revision on arXiv

Source-linked AI summary

Persistent Identity Preservation in Generative Image Models: A Benchmark and Evaluation System

Mengwei Ren, Xuaner Zhang, Zhihao Xia

arXiv:2609.04151v2cs.CV

TL;DR

Generative image models can perform complex visual tasks but still struggle to preserve a particular subject’s identity through changes and edits. This paper benchmarks in-context, model-specific, and persistent identity across generation, editing, restoration, and multi-subject settings. Persistent identity improves identity preservation across different foundation models while maintaining comparable instruction adherence and perceptual quality.

  • Problem

    Generative image models can produce high-quality images and follow instructions but still struggle to preserve a particular subject’s identity as appearance, viewpoint, or scene changes.

    Source: §1 · p. 1

  • Method

    The paper benchmarks in-context, model-specific, and persistent identity paradigms across subject-driven generation, editing, restoration, and multi-subject settings.

    Source: §6 · p. 18

  • Results

    Persistent identity provides stronger and more robust identity preservation across different foundation models and task settings without sacrificing task performance, with gains reflected in preferences from familiar raters.

    Source: §6 · p. 18

  • Takeaways & Limitations

    The results suggest identity can be represented as persistent subject knowledge composed independently with the underlying generative model.

    Source: §6 · p. 18

    The quantitative evaluation of human identity focuses primarily on facial identity, while perceived identity can also depend on body shape, age-related characteristics, posture, and other cues.

    Source: §5 · p. 18

Abstract

from arXiv · show

Generative image models can now produce high-quality images, follow complex instructions, and support precise edits, but they still struggle to preserve who or what is being depicted. When generating or editing images of a specific subject, identity may drift as the pose, expression, appearance, viewpoint, or surrounding scene changes. Existing subject-driven methods make fundamentally different choices about where identity is represented: through the input context (GPT-Image-2, NB2), as trainable subject-specific model parameters (LoRA), or as a persistent identity layer (PHOTA IDENTITY) reusable across generations and edits. We systematically benchmark these paradigms across subject-driven generation, editing, restoration, and multi-subject settings, with tasks designed to increasingly stress identity preservation. Our results show that identity preservation remains a distinct limitation of current generative foundation models: strong image quality and instruction following do not necessarily imply strong identity fidelity, and identity degradation becomes more pronounced under iterative edits, small subject scales, severe image degradation, and multi-subject composition. Persistent identity substantially reduces this degradation across generation, editing, and restoration, consistently improving identity preservation when applied to different foundation models while maintaining comparable instruction adherence and perceptual image quality. These results suggest that identity does not simply emerge from increasingly capable generative models, but can instead be represented as persistent subject knowledge that is composed independently with the underlying generative model.

1 Introduction

Subject-driven generation and editing require preserving a particular subject’s identity through substantial visual changes, a requirement that matters in persistent creative workflows. The paper distinguishes three ways of representing identity and proposes a unified benchmark, with familiarity-aware judgments complementing automated evaluation.

1 Introduction

  • Identity must remain consistent as pose, expression, clothing, viewpoint, illumination, scene, or style changes, including across repeated edits, restoration, scale changes, and multi-subject composition.The image may change substantially while the depicted subject remains the same.Source: §1 · pp. 1–2

1.1 Three paradigms for subject identity

  • The three paradigms represent identity in model-specific parameters, infer it from input context, or maintain it in a persistent, model-independent profile reusable across requests and backends.Per-subject LoRA instantiates model-specific identity; in-context methods reconstruct identity from reference images for each request.Source: §1.1 · pp. 2–3
  • Because approaches are evaluated on different subjects, tasks, protocols, and identity metrics, the paper establishes a unified benchmark to compare their robustness alongside image quality and instruction following.Its generation and editing tasks progressively stress identity preservation through increasingly challenging transformations.Source: §1.1 · p. 3

1.2 Evaluation of subject-driven image generation and editing

  • Human identity is less well defined than task correctness or perceptual quality, and face-embedding similarity may not match perceived likeness.An unfamiliar rater may miss subtle characteristics of a particular person that are apparent to the subject or people who know them.Source: §1.2 · p. 3
  • The benchmark complements automated identity metrics with blinded pairwise judgments by the subject or people personally familiar with them, alongside task adherence and perceptual quality.Familiarity-aware judgments assess likeness using prior knowledge of the subject rather than only a reference image.Source: §1.2 · p. 3

2 Benchmark Design

The benchmark compares in-context, model-specific, and persistent identity methods on consistent subjects and task specifications, using each method’s native or common personalization mechanism. Its task suite stresses identity preservation across generation, editing, restoration, and multi-subject settings.

2.1 Profiles and benchmark methods

  • The benchmark compares three identity paradigms using the same target subjects and task specifications as closely as each model interface permits, evaluating approximately 300 subject profiles under common protocols and metrics.LoRA is trained separately for each open-weight base model; in-context methods use visual references without subject-specific training; PHOTA profiles are created once and reused across tasks and backends.Source: §2.1 · pp. 3–4

2.2 Task suite and use-cases

  • The task suite spans generation, editing, and restoration in single- and multi-subject settings, stressing identity under different degrees and forms of visual transformation.It includes new contexts, attribute and scene changes, recovery from degraded evidence, and simultaneous preservation of multiple identities.Source: §2.2 · p. 4
  • Generation tasks range from everyday and professional photography to imaginative styles such as paintings, sketches, illustrations, and storyboards, requiring identity preservation across appearance and domain shifts.The suite uses around 160 prompt candidates.Source: §2.2 · p. 4
  • Editing tasks cover appearance, stylization, and geometry changes, with iterative edits testing identity stability as transformations accumulate across turns.Larger pose and viewpoint changes can require synthesizing identity-specific features not visible in the input; around 200 editing instructions are sampled for testing.Source: §2.2 · pp. 4–5
  • Restoration tests deblurring, upscaling, and low-light recovery at mild, moderate, and severe degradation levels, using paired clean images to evaluate identity recovery.Increasing degradation obscures more identity-relevant information in the input.Source: §2.2 · p. 5
  • Multi-subject tasks test generation and editing with instructions targeting one subject, multiple subjects, or the surrounding scene, measuring each identity independently.The setting challenges in-context methods to associate evidence with each subject and model-specific methods to compose multiple adaptations without identity or attribute mixing.Source: §2.2 · p. 5

3 Evaluation Metrics

The evaluation separates identity preservation, instruction adherence, and image quality, combining automated measures with judgments from people familiar with the depicted subjects. Identity is measured using enrollment-based face similarity and a threshold-based match rate, with a paired-clean-image comparison for restoration.

3 Evaluation Metrics

  • The evaluation measures identity preservation, instruction adherence, and image quality as complementary dimensions rather than treating task completion or visual quality as evidence of identity fidelity.Automated metrics evaluate the dimensions independently, while a familiar-identity preference study assesses joint user preference.Source: §3 · p. 5, §3.2 · p. 6, §3.3 · p. 7

3.1.1 Human (face) identity

  • For generation and editing, sid measures continuous similarity between an output face and an enrollment-based identity anchor built from multiple reference images.Aggregating enrollment images reduces dependence on transient pose, expression, or illumination in any one image.Source: §3.1.1 · pp. 5–6
  • For restoration, identity similarity is computed against the paired clean image to measure how faithfully the original subject is recovered as degradation increases.The clean-image face embedding is the comparison target.Source: §3.1.1 · p. 6
  • The threshold-based identity match rate reports the fraction of outputs meeting a similarity threshold, with a curve across thresholds complementing mean sid.This characterizes identity retention under increasingly stringent matching criteria.Source: §3.1.1 · p. 6

3.2 Instruction adherence and image quality

  • Separate VLM judges score instruction adherence and visual quality from 0–10, with identity excluded from the quality judgment.For editing, the adherence judge sees both input and output and can penalize undesired changes as well as missing edits.Source: §3.2 · pp. 6–7

3.3 Familiar-identity preference study

  • The preference study uses blinded pairwise comparisons judged only by the subject or people who know them personally, combining task adherence, image quality, and recognizable identity into Elo ratings.It complements component-wise automated evaluation with judgments informed by familiarity with the subject.Source: §3.3 · p. 7

4 Results

Across generation, editing, and restoration, PHOTA’s persistent identity conditioning shifts identity similarity higher than in-context and per-subject LoRA approaches. The results also examine task performance and increasingly demanding conditions, including iterative edits, degradation, and multiple subjects.

4.1 Quantitative results

  • Across generation, editing, and restoration, PHOTA shifts mean identity similarity higher than in-context foundation models and per-subject LoRA approaches.PHOTA also produces a greater concentration of outputs in the high-similarity regime.Source: Figure 1 · p. 7, §4.1 · p. 8
  • Adding PHOTA increases GPT-IMAGE-2 mean sid from 0.64 to 0.78 for generation, 0.68 to 0.79 for editing, and 0.67 to 0.81 for restoration.For NB2, mean sid increases from 0.50 to 0.75, 0.62 to 0.76, and 0.71 to 0.81, respectively.Source: §4.1 · p. 8
  • With smaller faces, base-model identity similarity declines, while PHOTA remains more robust across face scales.PHOTA+GPT2.5 maintains mean sid of approximately 0.81, 0.82, and 0.81 for large, medium, and small faces; PHOTA+NB2 reaches approximately 0.79, 0.80, and 0.77.Source: §4.1 · p. 9, Figure 4 · p. 9
  • PHOTA achieves higher identity fidelity while maintaining comparable instruction adherence and image quality to in-context models, and exceeds per-subject LoRA alternatives across all three dimensions.In the familiar-identity study, PHOTA+GPT2 and PHOTA+NB2 reach Elo scores of 1418 and 1355, respectively.Source: §4.1 · pp. 10–11, Figure 5 · p. 10, Figure 6 · p. 10

4.2 Subject-driven Generation

  • In subject-driven generation, in-context models often retain visual quality but drift in identity, while LoRA can improve resemblance at a cost to image quality; PHOTA more consistently preserves identity and quality across varied scenes.The comparison spans photography, digital avatars, and commercial photography.Source: §4.2 · pp. 11–12, Figure 8 · p. 12

4.3 Subject-driven Editing

  • For single-step edits, identity drift is pronounced with expression and pose changes; PHOTA more consistently preserves recognizable traits while carrying out identity-sensitive transformations.Across iterative edits, GPT-IMAGE-2 falls from approximately 0.71 to 0.53 and NB2 from 0.65 to 0.5 after four edits; PHOTA+GPT2 falls from approximately 0.79 to 0.7 and PHOTA+NB2 from approximately 0.77 to 0.67.Source: §4.3 · pp. 12–14, Figure 11 · p. 15

4.4 Subject-driven Restoration

  • As restoration degradation increases and identity-relevant evidence is obscured, base models lose identity similarity faster than PHOTA.The comparisons include deblurring, upscaling, and low-light restoration at mild, moderate, and severe levels.Source: §4.4 · p. 15, Figure 12 · p. 15

4.5 Multi-subject generation and editing

  • As more subjects are introduced, baseline identity similarity declines, especially for LoRA; PHOTA more consistently preserves distinct identities, instruction adherence, and visual quality.LoRA adapter stacking can also substantially degrade image quality and identity fidelity in some cases.Source: §4.5 · pp. 16–17, Figure 14 · p. 16

5 Discussion

The discussion notes that evaluation focuses mainly on facial identity, although perceived identity also includes other subject-specific cues. It also identifies privacy and identity ownership as considerations for reusable persistent identities.

Beyond facial identity

  • Identity evaluation focuses primarily on faces, while perceived identity can also depend on body shape, proportions, age-related characteristics, posture, and other subject-specific cues.Established recognition models provide a scalable, well-defined measurement, but do not capture this broader notion of identity.Source: §5 · p. 18
  • Full-body, small-subject, and stylized imagery can limit or substantially transform facial information.Source: §5 · p. 18
  • Extending evaluation beyond the face toward a more holistic notion of subject identity remains future work.Source: §5 · p. 18

Privacy and identity ownership

  • Reusable identity models raise privacy considerations beyond inference-time reference conditioning because they can be reused without repeatedly providing original reference images.Source: §5 · p. 18
  • Practical systems should consider consent, authorization, access control, revocation, deletion, and subject control over identity storage and use.Source: §5 · p. 18

6 Conclusion

The paper introduces a common framework for comparing model-specific, in-context, and persistent identity across subject-centered tasks. It finds that persistent identity strengthens preservation across models and challenging transformations without sacrificing task performance, supporting identity as persistent state independent of a generative model.

6 Conclusion

  • The paper organizes subject identity into model-specific, in-context, and persistent paradigms and benchmarks them across generation, editing, restoration, and multi-subject settings.The benchmark evaluates identity preservation alongside instruction adherence and image quality.Source: §6 · p. 18
  • Persistent identity provides stronger, more robust preservation across foundation models, especially under challenging transformations or limited direct visual evidence, without sacrificing task performance.The gains also appear in holistic preferences from raters familiar with the evaluated subjects.Source: §6 · p. 18
  • The results suggest that persistent identity state independent of an individual generative model is a promising foundation for subject-driven creative workflows.Source: §6 · p. 18
Loading 2609.04151v2…