PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-aware Design Generation: Adding 3D Humans into Graphic Designs

Zijin Hou, Ying Cao

arXiv:2609.17689v1cs.GR

TL;DR

Graphic-design generation had not explicitly addressed how human representations contribute to designs, despite their role in visual attention and evoked feelings. This paper adds controllable 3D humans to partial designs by predicting pose, framing, and image layout. Its model outperforms baselines, generating higher-quality designs with human elements better aligned to the design context.

  • Problem

    Graphic-design generation had not explicitly modeled human representations, although human pose and framing affect viewers' visual exploration and evoked feelings.

    Source: §1 · pp. 1–2

  • Method

    The model adds 3D humans to a partial design by predicting their 3D poses, 2D framing, and the human image's layout in the design.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    The model outperforms alternative baselines, producing higher-quality complete designs with poses, framing, and image layout better aligned to the input design context.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Explicit 3D representations allow users to change human poses and image composition by manipulating the generated representations, supporting control in design workflows.

    Source: §1 · p. 2

Abstract

from arXiv · show

Graphic designs nowadays predominantly contain human images, where people are posed and framed in a delicate way to guide viewers' visual exploration and evoke desired feelings. Despite recent progress in automatic graphic design generation, there has been no work on investigating the role of the human representations in design creation. In this paper, we seek to emphasize the importance of visual human representations in graphic design generation, by studying a novel task, dubbed as design-conditioned 3D human adding. To solve this task, we propose a model that adds 3D human representations into an existing partial design, by predicting their 3D poses, their 2D composition in the image containing them, as well as the spatial arrangement of the image in the input design, to generate a complete design. Our experiments show that our model outperforms baselines by predicting the 3D human poses, 2D framing and human image layout that better harmonize with the other existing design elements, and generating higher-quality holistic designs. Our work is the first to explore connections between 3D human generation and 2D graphic design generation, which would inspire future research in leveraging 3D information to build more powerful generative models for graphic designs in the 2D domain. The code is available at https://github.com/GreyNails/Human-aware-Design-Generation-Adding-3D-Humans-into-Graphic-Designs.

1 Introduction

Human pose and framing shape visual attention and emotional cues in graphic designs, yet generative models had not explicitly addressed human representations. The paper introduces design-conditioned 3D human adding and POFRAY, reporting better-aligned, higher-quality designs than alternative baselines.

1 Introduction

  • Human poses can direct viewers' visual flow, while framing can convey feelings such as strength and confidence.The introduction illustrates attention moving from a woman's face and hand toward a plant in the direction of her finger.Source: §1 · pp. 1–2
  • The paper frames the task as an initial bridge between 3D human generation and 2D graphic design generation, where human representations had not been explicitly considered.Its stated contributions also include POFRAY and the 3DHumanInDesign benchmark.Source: §1 · p. 2
  • Design-conditioned 3D human adding places 3D human representations into a partial design to form a cohesive complete design.The generated 3D representations can be rendered into realistic human images.Source: §1 · p. 2
  • POFRAY conditions pose-and-framing and layout generators on multimodal features from a partial layered design, generating human poses, image framing, and design layout.A causal Transformer generates pose and framing tokens, while a bidirectional Transformer predicts the image layout.Source: §1 · p. 2
  • The 3DHumanInDesign benchmark contains approximately 10k layered graphic designs, with human images removed to form inputs and their pose, framing, and layout used as targets.The estimated pose and framing parameters come from the original human images.Source: §1 · p. 2

2 Relate Work

Prior graphic-design work covers design generation and element layout, while 3D human-generation work conditions people on modalities such as text, scenes, or objects. The paper identifies modeling 3D human representations in 2D graphic designs as an unexplored connection.

2.1 Graphic Design Generation

  • Graphic-design generation includes vector-design models, text-to-design methods, masked-field prediction, and automatic composition of input elements.Layout-generation work applies Transformers to model element relationships or generate layout tokens in parallel.Source: §2.1 · pp. 2–3
  • The paper states that prior work had not modeled 3D human representations in 2D graphic designs.Source: §2.1 · p. 3

2.2 Human Pose Estimation and Generation

  • 3D human-pose research includes estimation of whole-body meshes and conditional generation from text, scenes, or objects; this work instead focuses on graphic-design context.Source: §2.2 · p. 3

3 Method

The method formulates adding 3D humans to a human-free design as predicting their poses, framing, and image layout. Its two generators use encoded design context to produce these outputs, which are composed into the design.

3.1 Problem Formulation

  • The task predicts 3D human poses, their framing within an image, and that image’s position, size, and z-order in a human-free design.The poses are rendered using the framing parameters and placed according to the predicted image layout.Source: §3.1 · p. 3

3.2 Our Model

  • The model has a design encoder, a pose-and-framing generator, and a layout generator that predict the human image’s spatial arrangement.The two generators are conditioned on design context representations.Source: §3.2 · p. 3

3.2.1 Design Encoder.

  • The design encoder combines appearance, content, and layout embeddings for each element, then uses self-attention to produce contextualized design representations.Appearance attributes include font, color, and opacity; content is encoded from images or text.Source: §3.2.1 · p. 3

3.2.2 Pose and Framing Generator.

  • A context-conditioned autoregressive Transformer generates pose and framing tokens, which are decoded into SMPL-H poses and continuous framing parameters.Pose tokens are decoded by a pretrained VQ-VAE decoder, while framing tokens map to bin-center values.Source: §3.2.2 · pp. 3–4

3.2.3 Layout Generator.

  • A learnable query attends to design-element embeddings, and an MLP decodes its representation into the human image’s position, size, and z-order.The predicted pose and framing are used to create a human image that is composed into the design according to the predicted layout.Source: §3.2.3 · p. 4, Figure 3 · p. 4, §3.2.4 · p. 5

3.3 Training

  • Training uses next-token cross-entropy for pose and framing, L1 plus GIoU for layout, separate generator pretraining, and joint fine-tuning.The joint fine-tuning stage is intended to produce mutually harmonious pose, framing, and layout outputs.Source: §3.3 · p. 5

4 Experiments

The experiments evaluate POFRAY on a constructed design benchmark against two baselines, using complementary metrics, qualitative comparisons, and a professional-designer study.

4.1 Dataset

  • The benchmark contains about 10,000 layered designs from Crello and Freepik, with human images removed for input and an 80:20 train-test split.The collection includes 4,479 Crello samples and 5,460 magazine covers and posters from Freepik.Source: §4.1 · p. 5

4.2 Implementation Details

  • Training uses AdamW with a 1×10^-4 base learning rate, warm-up followed by cosine decay, and top-p sampling at inference.Inference uses temperature 0.8 and p=0.8.Source: §4.2 · p. 5

4.3 Baselines

  • POFRAY is compared with FlexDM adapted to image retrieval and layout generation, and an MLLM pipeline that generates a human description and bounding box.Both baselines use an off-the-shelf estimator for pose and framing; the MLLM pipeline generates a realistic human image from its description.Source: §4.3 · pp. 5–6

4.4 Evaluation Metrics

  • Evaluation covers human-design coherence, holistic design quality, 3D pose accuracy, and human-image layout quality.The metrics are HD-Align, D-FID, MPJPE, Alignment, and Overlap.Source: §4.4 · p. 6

4.5 Quantitative Results

  • POFRAY beats baselines by significant margins in HD-Align and MPJPE, improves D-FID, leads in Alignment, and ranks second in Overlap.Table 1 reports the quantitative comparison; POFRAY's Overlap score of 209.11 is close to real data's 210.48.Source: §4.5 · p. 6, Table 1 · p. 6

4.6 Qualitative Results

  • Qualitatively, POFRAY better matches pose, framing, and layout to the design; MLLM sometimes places humans where text occludes them.Examples include matching the ground-truth medium-full-shot crop and generating multiple humans with similar body orientations.Source: §4.6 · p. 6, Figure 4 · p. 7

4.7 User Study

  • The user study found POFRAY significantly preferred over FlexDM and MLLM for human-design alignment and aesthetic quality.Twenty-three professional designers judged designs generated for 100 test inputs; Table 2 reports the percentage of times each method was chosen as best.Source: §4.7 · pp. 6, 8, Table 2 · p. 6

5 Conclusion

The conclusion presents design-conditioned 3D human adding as a first step toward bringing 3D human modeling into 2D graphic design, with experiments favoring the proposed model over baselines.

5 Conclusion

  • The paper formulates design-conditioned 3D human adding to bring 3D human representations into partial 2D graphic designs.It describes this as a first step in bringing 3D human modeling into 2D graphic-design creation.Source: §5 · p. 8
  • Compared with several baselines, the model generates poses, 2D framing, and human-image layouts that better fit the remaining design elements.Source: §5 · p. 8
  • The authors suggest this work may inspire future 3D-aware generative models for graphic design.Source: §5 · p. 8

Related papers

Loading 2609.17689v1…