PaperlayerView exact revision on arXiv

Source-linked AI summary

WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

Hengyuan Xu, Qixun Wang, Yiji Cheng, Miles Yang, Zhao Zhong, Wei Cheng, Xingjun Ma, Yu-gang Jiang

arXiv:2608.20336v1cs.CV

TL;DR

Generating images with many specified people requires preserving each identity while assigning distinct people to locations and handling noisy correspondence during training. WithEveryone addresses this with addressed identity tokens, structured identity–layout planning, and layout-grounded identity supervision. On an identity-disjoint benchmark, it reports higher target-context identity similarity and fewer copy-paste artifacts than GPT-Image 2.

  • Problem

    As group size grows, identity signals are diluted, identities can be duplicated or merged, and existing face-matching identity losses become unstable for many predicted faces.

  • Method

    WithEveryone binds references to addressed ID tokens, predicts an identity-aware Layout CoT, and supervises identities using annotated face regions instead of generated-face embedding matching.

  • Results

    0.499 Sim(Tgt) versus 0.462 for GPT-Image 2, with copy-paste artifacts of 0.055 versus 0.169 on a five-to-ten-person identity-disjoint benchmark.

  • Takeaways & Limitations

    Explicit spatial addresses provide a basis for supervising each identity independently as group size increases.

  • Takeaways & Limitations

    The conclusions come from 210 examples on one benchmark, with the largest-reference groups smallest and identity metrics dependent on face detectors and recognizers.

Abstract

from arXiv · show

Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, the model must bind every reference to a distinct person and location, while training-time identity losses must establish correspondence among several noisy predicted faces. We introduce WithEveryone, a unified framework for generating group images up to ten reference identities. WithEveryone injects each selected identity as an addressed token, predicts a structured identity--layout plan, and renders the plan as a visual condition. Its key objective, Layout-Grounded ID Loss, uses annotated face regions to supervise the intended identities directly, avoiding unstable embedding-based face matching; ID Representation Forcing additionally trains a prediction for each identity before image synthesis. On an identity-disjoint benchmark, WithEveryone achieves the highest target-context identity similarity, improving face similarity from 0.462 for GPT-Image-2 to 0.499, while reducing copy-paste artifacts from 0.169 to 0.055. It further covers 97.3\% of the requested identities with a duplicate rate of only 2.8\%. These results show that explicit identity--layout grounding enables identity-preserving generation to scale to larger groups without relying on direct reference-face copying.

WITHEVERYONE: UNIFIED PLANNING AND IDENTITY

WithEveryone is presented as a unified framework for group image generation. Its overview emphasizes joint reasoning about identities, layout, and composition, while its examples span varied group sizes, scenes, and styles.

  • Framework overview: WithEveryone unifies identity and composition decisions in one model context rather than separate modules.The model plans identity–layout bindings, person regions, and poses before generating the image.
  • Generated examples: The generated samples cover different group sizes, scene types, and visual styles.

1 INTRODUCTION

Group image generation becomes harder as more specified people must be retained, distinguished, and spatially organized. WithEveryone addresses these coupled challenges through addressed identity tokens, explicit layout planning, and layout-grounded supervision, achieving stronger identity similarity and fewer copy-paste artifacts than GPT-Image 2.

  • Motivation: Existing methods mainly support one to five reference identities, while real group-image creation requires jointly generating more specified people.
  • Motivation: Increasing reference counts can dilute identity signals, increase cross-identity interference, and cause duplicated or merged faces.
  • Motivation: Identity losses for larger groups become unstable because noisy face-embedding matching can assign generated faces to the wrong references.
  • Method: WithEveryone binds each selected reference to an addressed ID token and uses ID Representation Forcing before synthesis.
  • Method: Its structured Layout CoT predicts identity–layout bindings, person regions, and poses, then renders the plan as a visual condition.
  • Results: 0.499 Sim(Tgt) versus 0.462 for GPT-Image 2, with copy-paste artifacts of 0.055 versus 0.169.

2 RELATED WORK

Related work spans identity-preserving generation, reference-conditioned and unified multimodal models, and layout planning. WithEveryone extends unified planning to identity-conditioned group generation by binding planned people to reference identities.

  • Identity-preserving generation: Identity-preserving generation has progressed from single-person methods to multi-person customization, but existing methods remain limited to small groups.
  • Reference-conditioned and unified models: Reference-conditioned generation is supported by open and proprietary models, while unified architectures enable in-context planning before synthesis.
  • Layout planning and generation: Most layout-conditioned methods separate planning from image generation, whereas unified methods combine them in one contextual sequence.

3 METHOD

WithEveryone unifies addressed identity conditioning, structured identity-aware layout planning, and visual layout rendering before flow-based image synthesis. Its training objectives ground each identity to annotated face regions while representation forcing preserves identity information in the shared causal context.

  • Overview: The model handles five-to-ten-person group-image generation with a transfusion-style mixture-of-transformers architecture combining autoregressive reasoning and flow matching.Text and structured reasoning are predicted autoregressively, while target-image latents are learned through flow matching.
  • Overview: WithEveryone selects participating references, loads each as an addressed ID token, predicts identity–layout bindings, renders the layout, and predicts target identity representations before synthesis.The rendered layout condition and representation scaffold jointly condition flow-based image generation.
  • Identity conditioning: ID Representation Forcing predicts an identity representation from the preceding prompt, references, ID tokens, and layout reasoning so the resulting hidden state remains available to later image tokens.The prediction is aligned to the corresponding target identity in ArcFace space, forming an identity scaffold for generation.
  • Layout planning: Structured Layout CoT autoregressively predicts identity–layout bindings, person and face regions, body extents, and pose keypoints using a fixed causal order.The coordinate vocabulary contains 2,002 tokens corresponding to 1,001 positions on each axis.
  • Training objectives: Layout-Grounded ID Loss uses each identity’s annotated target face region to crop predicted and target images, eliminating generated-face matching during multi-person supervision.The crops are encoded by ArcFace, and the loss applies only below a timestep threshold where the one-step estimate contains a recognizable face.
  • Training objectives: The joint objective applies next-token prediction to text and layout sequences, flow matching to image latents, and cosine alignment to identity predictions.At inference, identity selection, Layout CoT, and recaption are predicted autoregressively; target identity embeddings serve only as training targets for representation forcing.

4 EXPERIMENTS

WithEveryone is evaluated on an identity-disjoint 5–10-person benchmark using shared metrics, cross-model comparisons, scaling analyses, and ablations of layout, identity addressing, and layout-grounded supervision.

  • Experimental setup: The benchmark contains 210 identity-disjoint group-image examples with five to ten reference identities, and every model is evaluated on all examples.WithEveryone uses 2K resolution for the main Table 1 comparison and 1K in other reported settings; some methods receive ground-truth layouts because they cannot plan autonomously.
  • Experimental setup: Cross-model evaluation averages Sim(Ref) and Sim(Tgt) over ArcFace, FaceNet, and AdaFace, while Coverage and Dup measure identity inclusion and duplication.The main protocol assigns generated faces to references and targets on the clean final image.
  • Cross-model comparison: 0.973 Coverage and 0.028 duplicate rate are WithEveryone’s best identity-coverage results, ahead of GPT-Image 2 at 0.905 and 0.075.Open-source general-purpose models mostly remain below 0.42 coverage, while Figure 6 illustrates corresponding failure modes.
  • Scaling to more reference identities: 0.629 to 0.571 is WithEveryone’s ArcFace-similarity decline from five to ten references, versus 0.593 to 0.496 for GPT-Image 2.WithEveryone ranks first across the five-to-ten range and shows a flatter degradation trend with more concentrated distributions.
  • Ablation studies: The LG-ID Loss alone raises Sim(Ref) from 0.339 to 0.506, Sim(Tgt) from 0.304 to 0.435, Count from 0.771 to 0.845, and Coverage from 0.741 to 0.947.The ablation attributes these gains to supervising each identity inside its annotated region; the complete configuration changes several components and the training corpus together.

5 CONCLUSION

WithEveryone unifies addressed identity tokens, identity-specific supervision, and structured layout planning to generate coherent images containing five to ten specified people. On an identity-disjoint benchmark, it achieves the strongest reported identity-context similarity while reducing copy-paste artifacts and improving identity coverage and collision control.

  • WithEveryone generates coherent group images from five to ten reference identities while addressing identity retention, identity distinction, and spatial organization.
  • The framework binds references to addressed ID tokens, predicts per-identity representations before synthesis, and applies layout-grounded supervision to generated faces.
  • Structured Layout CoT moves multi-person planning to the understanding side and renders the predicted plan as a visual condition for image generation.
  • 0.499 Sim(Tgt) versus 0.462 for GPT-Image 2, with Copy-Paste 0.055 versus 0.169, gives WithEveryone the highest target-context identity similarity and fewer artifacts.
  • The benchmark results show broader requested-identity coverage and fewer collisions than any compared system.

A ADDITIONAL QUALITATIVE RESULTS

The statistical analysis evaluates the benchmark means over 210 examples and tests whether reported differences are reliable across paired examples. It confirms the Sim(Tgt) advantage over GPT-Image 2 and simultaneous identity and artifact improvements over Seedream 5.0 Pro, while identifying a near-tie on CLIP-I.

  • Table 1 reports means over 210 examples, while per-example recomputation and paired comparisons assess dispersion and reliability.
  • +0.038 Sim(Tgt) over GPT-Image 2 has a 20,000-sample bootstrap interval of [+0.027, +0.048], Wilcoxon p 3.8×10−11, and a higher score on 73% of examples.
  • +0.001 CLIP-I over Nano Banana 2 has interval [−0.006, +0.009], p 0.98, and a higher score on 48% of examples, indicating statistical indistinguishability.
  • Against Seedream 5.0 Pro, Sim(Ref) improves by +0.022 while Copy-Paste falls by 0.062 and ArcFace similarity rises by +0.112.

B.2 FACE SIZE AND SCALING DEGRADATION

Face-size analysis shows that baseline identity similarity degrades as faces become smaller, whereas WithEveryone is comparatively flat at 2K. Its strongest advantage occurs below 200 pixels, although the 1K comparison is not a controlled resolution ablation.

  • Baseline similarity increases by 0.016–0.019 per percentage point of relative face side, while WithEveryone at 2K has slope −0.002 ± 0.002.
  • Growing from five to ten references changes mean relative face side only from 11.9% to 9.4%, so group size produces a narrow face-size range.
  • Controlling for face size removes most group-size degradation for GPT-Image 2 and Seedream 5.0 Pro, but WithEveryone’s slope remains −0.014 with an interval excluding zero.
  • Between 112 and 150 px, WithEveryone reaches 0.611 versus 0.561 for GPT-Image 2, 0.388 for Seedream 5.0 Pro, and 0.383 for Nano Banana 2.
  • The 1K curve is only a reference because it differs from the 2K curve in both resolution and reference-image configuration.

B.3 HIGH-RESOLUTION TRAINING

High-resolution training improves identity similarity and especially layout-plan execution under controlled 1K evaluation, while representation forcing and LG-ID supervision provide additional identity-training mechanisms. The LG-ID loss is applied only when one-step predictions retain usable identity cues.

  • B.3 HIGH-RESOLUTION TRAINING: Under controlled 1K evaluation, 2K training raises Sim(Ref) from 0.546 to 0.555 and Plan IoU from 0.773 to 0.814 without reducing Sim(Tgt) or Layout Score.
  • B.3 HIGH-RESOLUTION TRAINING: High-resolution training mainly improves how faithfully generation realizes the spatial plan, with a smaller identity benefit.
  • Representation Forcing: Representation Forcing predictions become more similar to target and reference identity embeddings during training, but remain below the approximately 0.40 reference–target baseline.
  • LG-ID Loss: The LG-ID Loss supervises identities on a differentiable clean-image estimate, using annotated face regions and target correspondences rather than embedding-based reordering.
  • LG-ID Loss: LG-ID supervision is computed only for t ≤0.85 because identity cues degrade rapidly at larger sampled timesteps.
  • LG-ID Loss ablation: Increasing λID improves Sim(Ref) while Layout Score and RLS remain in similar ranges, though larger weights can slightly slow early layout improvement.

B.6 EFFECT OF THE LG-ID LOSS

The LG-ID Loss provides direct identity supervision while preserving or improving layout quality. The complete configuration combines this loss with identity tokens, representation forcing, Layout CoT, and additional text-to-layout supervision.

  • Sim(Ref) rises from 0.339 to 0.506 and Sim(Tgt) from 0.304 to 0.435 with the LG-ID Loss alone.Layout Score increases from 0.690 to 0.700, Count from 0.771 to 0.845, and Coverage from 0.741 to 0.947.
  • The LG-ID Loss yields the largest individual identity gain while preserving or improving all reported layout measures.
  • The complete configuration reaches Sim(Ref) 0.555, Sim(Tgt) 0.461, and Layout Score 0.759.It combines LG-ID Loss with ID tokens, Representation Forcing, Layout CoT, and additional text-to-layout supervision.
  • The complete configuration’s additional gain is not isolated because several components and the training corpus change together.

C.2 TRAINING CONFIGURATION

The evaluation configuration measures identity, scene, prompt, count, coverage, uniqueness, distinctness, spatial validity, and plan adherence using standardized detection, matching, and aggregation procedures. Relative layout evaluation separately measures whether the planned arrangement matches ground-truth relationships, while Plan IoU measures execution of the supplied plan.

  • All models are scored on the same 210 benchmark examples with identical detection, matching, and aggregation code, without excluding examples.Each example contains one generated image, 5–10 reference identities, a target group image, and identity correspondences.
  • Sim(Ref) and Sim(Tgt) are means from maximum-similarity Hungarian assignments using ArcFace, FaceNet, and AdaFace.Sim(Ref) matches generated faces to references, while Sim(Tgt) matches them to target faces.
  • CLIP-I, DINO-I, and CLIP-T measure generated-to-target image similarity or generated-to-prompt similarity, but do not identify which person appears where.They are therefore interpreted alongside identity metrics rather than alone.
  • Layout Score aggregates seven validity sub-scores in [0, 1], renormalizing remaining weights when a component cannot be computed.Count penalizes both missing and extra people relative to the target group size.
  • Coverage, uniqueness, and distinctness use cosine similarities and one-to-one Hungarian matching to assess identity coverage and collisions.A reference counts as covered at matched similarity ≥ 0.20; uniqueness penalizes references over-claimed by multiple generated faces.
  • Relative Layout Score evaluates planned relative count, position, and size, while Plan IoU evaluates whether generation executes the supplied plan.Relative Layout Score is insensitive to global translation, rotation, and scaling, whereas Layout Score evaluates final-image validity.

E LIMITATIONS, FUTURE WORK, AND RESPONSIBLE USE

The paper identifies limitations in layout evaluation, benchmark scope, metric dependence, and responsible deployment. Future evaluation must account for multiple valid layouts, while deployment should address consent and provenance risks.

  • When prompts are underspecified, many layouts may be equally valid, making comparison against a single reference difficult.The paper leaves this evaluation problem to future work.
  • The conclusions are based on 210 examples from a single benchmark, with the largest-reference groups being the smallest and per-group results therefore trends rather than precise estimates.
  • Identity measures inherit face-detector and recognizer behavior that varies across demographic groups, while systems differ in output resolution and layout-planning ability.
  • Identity-conditioned generation can enable non-consensual depictions, fabricated scenes, and impersonation despite compatible training-data licences.The paper recommends consent for referenced people and provenance signalling.
Loading 2608.20336v1…