PaperlayerView exact revision on arXiv

Source-linked AI summary

Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design

Hongyang Du, Lan Yan, Christian Flores, Asim Kadav

arXiv:2609.22086v1cs.AIcs.CV

TL;DR

Professional graphic design requires editable, structured artifacts built through many actions, but creative briefs lack reliable, verifiable rewards. Designer-RSI evolves an external library of reusable natural-language skills around a frozen design agent. Across multiple benchmarks, the evolved agent improves performance; on Claude-Sonnet-4, GenEval2 execution success rises from 72.7% to 99.3%.

  • Problem

    Professional design requires editable, structured artifacts built through sequential actions, while creative briefs lack reliable, verifiable rewards.

    Source: §1 · p. 1, §A.7 · p. 19

  • Method

    The framework evolves an external natural-language skill library by adding procedures for uncovered subtasks and revising existing ones around a frozen design agent.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    Across multiple frozen backbones and benchmarks, skill evolution improves performance; on Claude-Sonnet-4, GenEval2 execution success rises from 72.7% to 99.3%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    External procedural memory can support adaptation of graphic-design agents without updating foundation-model weights.

    Source: §6 · p. 9

    The replay gate rejects regressions only on evaluated cases and does not guarantee monotonic improvement across the full user-traffic distribution.

    Source: §6 · p. 9

Abstract

from arXiv · show

Professional graphic design is a long-horizon agentic task in which structured, editable artifacts emerge from many interdependent actions, yet outcomes admit no reliable programmatic oracle. We introduce a continual adaptation framework in which a frozen frontier model operates professional design software through more than 230 tools, while an external procedural memory of natural-language skills accumulates and refines reusable design procedures from experience. The memory widens by acquiring procedures for recurring uncovered subtasks and deepens by revising existing procedures against their own successful and failed executions, while a matched replay gate admits only changes that repair failures without regressing observed successes. Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality), with 61.8% and 67.6% win rates against the no-skill agent across four specialized design benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6. We further show the two mechanisms are effective in combination: on 200 held-out briefs from user-traffic benchmark, widening or deepening alone reaches a 49.4% / 48.6% win rate over the no-skill agent, while their combination reaches 58.5% (p = 0.025). Procedural memory offers a practical route to continual adaptation of agents under noisy, unverifiable feedback.

1 INTRODUCTION

Professional graphic design is a long-horizon sequence of interdependent actions with outcomes that lack a reliable success oracle. Designer-RSI adapts a frozen model by evolving an external library of reusable natural-language procedures from user traffic.

Motivation

  • Design requires sequences of interdependent edits, combines concrete requirements with subjective criteria, and has no success oracle comparable to executable code tests.Terminal feedback therefore weakly identifies which decisions caused success or failure.Source: §1 · p. 1

Procedural memory

  • Each skill is an adaptable natural-language guide between an atomic tool call and a full trajectory, which the model can reorder or partially apply.The paper contrasts this flexibility with fixed-sequence routines such as macros and Photoshop actions.Source: §1 · pp. 1–2

Approach

  • A frozen frontier model operates design software through more than 230 tools, while widening adds skills for uncovered recurring subtasks and deepening revises existing skills linked to failures.The foundation models, tools, renderer, evaluator, and evolution roles remain fixed; only the skill library changes.Source: §1 · p. 2

Replay gate

  • A matched replay gate admits a proposed skill change only if it beats the incumbent on at least one replayed case without a detected regression.The gate holds upstream context fixed when comparing the candidate with the incumbent.Source: §1 · p. 2

Results

  • GenEval2 execution success on Claude-Sonnet-4 rises from 72.7% to 99.3% across five rounds, while acquiring and revising procedures together yields larger task-completeness gains than either alone.The evaluation uses no model-weight updates or human reward labels.Source: §1 · p. 2

2 BACKGROUND: AGENTIC SYSTEM FOR GRAPHIC DESIGN

The study evaluates skill evolution in a professional graphic-design agent that creates editable artifacts through design software tools, with reusable playbooks supplied through an external skill bank.

2 BACKGROUND: AGENTIC SYSTEM FOR GRAPHIC DESIGN

  • The agent uses a frozen frontier language model to control equivalents of Photoshop, Illustrator, and InDesign through more than 230 tools.Requests run through iterative tool-calling loops to construct structured, editable artifacts.Source: §2 · p. 3
  • The agent retrieves real assets, renders intermediate document states for multimodal inspection, and supports deterministic offline rendering and evaluation.These capabilities allow completed trajectories to be evaluated without modifying the agent's tools or renderer.Source: §2 · p. 3
  • Skill retrieval can inject reusable SKILL.md workflows and a reduced tool set, while disabling retrieval recovers the original agent as a control.Neither model weights nor the underlying tools and renderer are modified.Source: §2 · p. 3

3 EVOLVING LOOP

The framework evolves an external procedural skill bank while keeping model weights fixed, widening coverage for uncovered subtasks and deepening existing skills.

3 EVOLVING LOOP

  • The offline loop keeps model weights fixed and changes only SKILL.md files, with a Prompter, Solver, Grader, and Reflector coordinating the process.The Grader scores rendered images and explains unmet requirements; the Reflector turns failures into targeted skill edits.Source: §3 · p. 3

3.1 WIDENING: MINTING NEW SKILLS FROM RECURRING UNCOVERED SUBTASKS

  • Widening pools recurring uncovered subtasks, distills a candidate after three occurrences, and admits it only if it passes replay against the no-skill baseline.Rejected candidates are discarded while their occurrences remain in the pool for further evidence.Source: §3.1 · p. 3

3.2 DEEPENING: HARDENING SKILLS THAT ALREADY EXIST

  • Deepening prioritizes skills with at least two failures (trajectory score <0.6) and revises them by contrasting failure rationales with successful histories.Repeatedly rejected rewrites can trigger a major rewrite or an optional exploration cycle that leads to updating, deleting, or keeping the skill; kept failures return to the coverage pool.Source: §3.2 · p. 4

3.3 REPLAY GATE

  • The replay gate freezes matched contexts and compares candidates with incumbents for rewrites or the no-skill agent for new skills, shipping only prompt-level majority wins.It avoids absolute grader scores and requires rewrites to repair failures without regressing existing successes.Source: §3.3 · pp. 4–5

4 EXPERIMENTS

EVOLVE improves specialized graphic-design win rates and general T2I quality and reliability, while its procedural memory expands through gated widening and deepening. Across five rounds, the bank grows from 76 to 139 skills, and qualitative examples show procedures shaping both editing and asset selection.

4 EXPERIMENTS

  • Experiments compare EVOLVE with a no-skill BASE agent across three foundation models and evaluate general T2I and specialized design tasks using distinct metrics.The internal benchmark uses approximately 300 traffic-derived briefs per evolution round and a fixed held-out set of 200 human-authored briefs. External evaluations sample 300 prompts per benchmark; general T2I uses Soft-TIFA or VQAScore, while design benchmarks use blind pairwise EVOLVE-vs-BASE win rates.Source: §4 · p. 5

4.1 EVOLUTION DYNAMICS

  • On 200 fixed, held-out human-authored briefs, R5 improves completeness versus no skill at essentially every threshold: 86% to 93% at ≥0.5, 43% to 56% at ≥0.9, and 24% to 32% at 1.0; the largest gain is +13 pp at ≥0.9.The briefs were disjoint from the 1,406 evolution briefs and fixed across all five rounds.Source: §4 · p. 5, §4.1 · p. 6, Figure 5 · p. 6
  • Across five rounds, gated widening and deepening expand the bank from 76 documentation-derived skills to 139, using 1,869 graded trajectories from 1,406 briefs without human labels.The loop widens when failures recur and shifts toward deepening as coverage saturates; R5 commits 21 rewrites and 4 mints, reversing the mix after R3–R4 minting leaves unrevised skills that can misfire. Across rounds, the gate commits 131 rewrites and 69 mints, while rejecting 100 rewrite proposals and 67 mint candidates; minted skills are more distant from the cold-start bank than its seed skills are from one another (median 0.215 vs. 0.158; p < 10^-8).Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • EVOLVE raises Claude-Sonnet-4 success from 72.7% to 99.3% on GenEval2 and from 82.7% to 100% on DPG-Bench.Average generation quality increases by +3.83 for Claude-Opus-4.6, +7.70 for Claude-Sonnet-4, and +9.67 for Qwen3.6-27B; latency overhead is 3.4%–6.2% across backbones, with occasional reductions.Source: §4.2 · p. 7, Table 1 · p. 7, Figure 6 · p. 7
  • EVOLVE achieves overall win rates of 67.6% for Claude-Opus-4.6 and 61.8% for Claude-Sonnet-4 against BASE on specialized graphic-design tasks.Opus peaks at a 71.7% win rate on CreatiDesign. Win rates are judged by GPT-5.4 in pairwise comparisons.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 QUALITATIVE RESULTS

  • Qualitative examples show EVOLVE transferring editing procedures across subjects and selecting assets that make requested downstream edits and spatial relations feasible.For double exposure, EVOLVE extracts and blends both a logo subject and a soccer-player silhouette, while BASE leaves one blend muddied and never extracts the other. In other examples, EVOLVE selects a glass tunnel or beach scene suited to the requested composition, unlike BASE’s unsuitable asset choices.Source: §4.3 · pp. 8–9, Figure 7 · p. 8

5 ABLATION STUDY

Deepening and Widening together outperform either mechanism alone in the ablation, with a superadditive completeness gain and a higher win rate against BASE. Gains concentrate in completeness, while aesthetics and critique change little.

5 ABLATION STUDY

  • Rewriting alone reaches 69.02 completeness and a 48.6% win rate, while adding new skills alone reaches 69.79 completeness and a 49.4% win rate.The documentation-derived cold start scores 68.62 completeness versus BASE at 69.08, with a 46.4% win rate; neither individual mechanism suffices.Source: §5 · p. 9
  • Aesthetics changes from 65.92 to 66.53 and critique remains largely unchanged, while EVOLVE uses 436.6k/5167 prompt/output tokens versus cold start at 445.2k/5298.Skill retrieval adds ∼28% prompt tokens over BASE, but evolution adds no marginal token cost; token comparison uses top-k (k = 3) matching.Source: §5 · p. 9
  • 74.04 completeness (+5.42 over cold start) and a 58.5% win rate against BASE (p = 0.025) result from combining Deepening and Widening.The combined gain is superadditive (+3.85 completeness): minted skills need refined retrieval descriptions to surface, while rewriting can route unfixable failures to the gap store for minting. Evaluated on 200 unseen prompts in one batch.Source: Table 3 · p. 9, §5 · p. 9

6 LIMITATION & CONCLUSION … C.1 SKILL REPRESENTATION

The paper presents procedural memory as an external skill bank that widens with procedures for uncovered subtasks and deepens through revisions checked by matched replay. It reports gains from combining both mechanisms while identifying limits of natural-language guidance and automated evaluation.

6 LIMITATION & CONCLUSION

  • Across five rounds and 1,406 briefs, the bank grew from 76 to 139 skills; widening and deepening together improved three backbones without weight updates or human reward labels.Widening adds procedures for recurring uncovered subtasks; deepening revises existing ones using successful and failed executions, with candidate changes passing matched replay before deployment. Neither axis alone suffices, and the replay gate only protects against regressions on evaluated cases, not the full user-traffic distribution.Source: §6 · p. 9

A.1 SELF-IMPROVING AGENTS AND EXPERIENCE-BASED ADAPTATION / A.2 AGENT MEMORY, PROCEDURAL SKILLS, AND REUSABLE TOOL-USE KNOWLEDGE

  • Experience-based agents retain feedback or insights across trials, while memory and skill systems distinguish episodic records from reusable procedures (Reflexion (Shinn et al., 2023); Voyager (Wang et al., 2024a)).Self-Refine (Madaan et al., 2023) focuses on within-task refinement, whereas Reflexion, ExpeL (Zhao et al., 2024), and Generative Agents (Park et al., 2023) retain experience across trials or tasks; the cited work does not study a shared, regression-gated skill bank evolved from multi-user deployment traffic. Episodic memory records what happened, while procedural memory captures how a class of tasks can be accomplished.Source: §A.1 · p. 16, §A.2 · pp. 16–17

A.3 CONTINUAL SKILL ACQUISITION AND REFINEMENT / A.5 SKILL RETRIEVAL, TOOL-SURFACE REDUCTION, AND COST-AWARE AGENT DESIGN / A.6 CO-EVOLVING CURRICULA AND ENVIRONMENTS

  • Related work frames continual procedural learning as acquiring missing capabilities and refining existing knowledge; retrieval research also addresses skill-bundle relevance and context cost.ACE (Zhang et al., 2026b) uses a grow-and-refine strategy, while Graph of Skills (Liu et al., 2026) retrieves bounded, dependency-aware bundles to address context saturation and semantic retrieval that misses prerequisites. Co-evolving curricula and environments instead adapt model weights or reshape training environments (Huang et al., 2026c; Li et al., 2026a; He et al., 2025; Huang et al., 2026b).Source: §A.3 · p. 17, §A.5 · p. 18, §A.6 · p. 18

A.3 CONTINUAL SKILL ACQUISITION AND REFINEMENT / A.4 RELIABLE IMPROVEMENT UNDER AUTOMATED FEEDBACK

  • MACLA, Skill-Pro, and SkillAudit study acquiring and refining procedural knowledge, while automated-feedback work highlights judge noise, proxy limits, and conservative comparison with deployed systems.G-Zero (Huang et al., 2026a) warns that proxy judges can impose capability ceilings and invite reward hacking; safe-policy-improvement work motivates comparing proposed updates with the incumbent.Source: §A.3 · p. 17, §A.4 · pp. 17–18

A.7 LLM AGENTS FOR GRAPHIC DESIGN

  • Graphic design combines long, interdependent tool workflows with subjective quality judgments, leaving no deterministic oracle for deciding whether a professional design satisfies its brief.Even a compact action space can fail because of spatial reasoning, dependencies across steps, and incorrect action selection; professional workflows compose many lower-level operations into coherent artifacts.Source: §A.7 · p. 19

B ADDITIONAL GRAPHIC DESIGN AGENT DETAILS

  • The graphic-design agent uses an iterative tool-calling runtime, asset-grounded rendering, and offline evaluation to support editable designs and inspect execution outcomes.The runtime supports parallel tools, validation and repair, failure recovery, previews, and context compression. The renderer supports typography, compositing, masks, effects, multilingual layout, and editable exports; verification combines multimodal inspection with deterministic checks, and completed trajectories can be rendered and evaluated offline.Source: B ADDITIONAL GRAPHIC DESIGN AGENT… · p. 19

C THE SKILL BANK / C.1 SKILL REPRESENTATION

  • The skill bank stores reusable procedures as versioned natural-language skills, partitioned by app mode and retrieved using trigger descriptions rather than body text.Each skill is a Markdown file with YAML front matter; its description is 200–600 characters and is the sole retrieval signal. Metadata includes app mode, a 5–10-tool critical path, optional prerequisites, status, and version; the body has five required sections, while per-skill statistics are stored separately.Source: C THE SKILL BANK · p. 20, §C.1 · p. 20

C.2 RETRIEVAL · C.3 INJECTION: PROGRESSIVE DISCLOSURE VIA LOAD SKILL · C.4 TRAJECTORY RECORDING

The system retrieves operation-matched skills, discloses their full procedures only when the agent loads them, and records each turn for later credit assignment and coverage-gap detection.

C.2 RETRIEVAL

  • A frozen LLM distills each brief into at most six operation keywords, ignoring subject matter, adjectives, and colors so retrieval matches operation-centric skill descriptions.Source: §C.2 · p. 20
  • Token-overlap scores rank skills independently for each operation, with name-match bonuses and app-mode scope; 2–4 likely matches are flagged without restricting what the agent can see.Matches must clear τ = 0.2 and lie within 0.6× the top score; the list is padded to two when fewer qualify.Source: §C.2 · p. 20
  • Token overlap aligns runtime retrieval with cold-start seeding, ensuring trigger phrases written into skill descriptions remain retrievable without guessing an embedding space.Source: §C.2 · p. 20

C.3 INJECTION: PROGRESSIVE DISCLOSURE VIA LOAD SKILL

  • Each turn, the injector presents a token-bounded catalog of all skills, with ranked matches starred and floated to the top; the agent loads relevant skill bodies only when it chooses to read them.The catalog is app-agnostic and tags each skill’s app mode, while execution tools are scoped to the active document’s app.Source: §C.3 · p. 21
  • Optionally, loaded skills’ tool sequences plus an always-on core can narrow the 230-tool catalog to task-relevant tools; this was off by default, and turns with no starred skills mark coverage gaps.Source: §C.3 · p. 21

C.4 TRAJECTORY RECORDING

  • When enabled, a recorder logs each turn’s distilled query, retrieved skills and scores, tool calls, and XML snapshot pointer, tagging it covered or uncovered for credit assignment and gap detection.Scores are back-filled by the offline grader; the record identifies active skills in failures and requests with no skill or misused skills.Source: §C.4 · p. 21

D EVALUATION KIT: THE GRAPHIC-EVAL RUBRIC · E SCORING AND OUTCOME BINNING

The section defines a frozen, image-based judge for evaluating design quality and producing the outcomes used to guide skill revision. It specifies how rubric axes, scoring, outcome bins, and rollout costs are recorded.

D EVALUATION KIT: THE GRAPHIC-EVAL RUBRIC

  • Graphic-Eval is the same frozen judge used to reward self-evolution and report progress, scoring rendered designs on completeness, aesthetics, and critique.Each axis is implemented as a constrained JSON-only judge call.Source: D EVALUATION KIT: THE GRAPHIC-EVAL… · p. 21

D.1 RENDERING

  • Grading evaluates a rendered PNG rather than a document model, and production turns are judged against the initial brief plus all refinements so far.The XML snapshot is rendered at a fixed scale.Source: §D.1 · p. 21

D.2 COMPLETENESS (DECOMPOSE THEN CHECK)

  • Completeness is the fraction of brief requirements marked met, partial, or not met after one call extracts 3–10 outcome-based requirements and another checks them against the image.Requirements exclude tool or implementation choices; unmet items receive reasons, which supply the reflector’s “why-bad” signal.Source: §D.2 · pp. 21–22

D.3 AESTHETICS (ANCHORED 1–10)

  • Aesthetics rates visual quality independently of brief adherence on a 1–10 scale, normalized by dividing the score by 10.Anchors include 6 for passable but unremarkable, 8 for professional and polished, 9–10 for exceptional, and 3 or below for weak.Source: §D.3 · p. 22

D.4 CRITIQUE (COUNTED PROBLEMS)

  • Critique counts visible major and minor problems across eight design dimensions, providing localized diagnostics rather than another scalar score.The judge skips dimensions with no problems and suggests no fixes.Source: §D.4 · p. 22

D.5 COMBINING AXES AND JUDGE BACKENDS

  • The grader can use Azure-hosted GPT or Bedrock-hosted Claude backends, while pairwise comparison mitigates risk from a grader over-rewarding a same-family solver.If the estimated pass rate skews uniformly high, the grader can be switched to the cross-family backend.Source: §D.5 · p. 22

D.6 COST SIGNALS

  • Each rollout records tool-call count, wall-clock latency, and input/output token counts as first-class cost outcomes.Token counts are parsed from the agent’s completion marker without a telemetry backend.Source: §D.6 · p. 22

E SCORING AND OUTCOME BINNING

  • sj = 0.7 × completenessj + 0.3 × aestheticj; trajectories scoring at least 0.6 are successes, and failures count against every retrieved skill.An evolve round fires after 50 new graded records; skills with at least 2 failures are revised, ordered by failure count and capped at 50 per round, without consulting success counts. The per-skill quality average is for observability, not selection or routing.Source: E SCORING AND OUTCOME BINNING · pp. 22–23

F REPLAY GATE DETAILS · G COLD START FROM DOCUMENTATION

The paper seeds a fresh deployment with documentation-derived skills, then uses a conservative replay gate to refine and expand the bank from experience. The gate compares candidate and baseline executions under matched conditions, while cold start supplies an initial set of reusable workflows before user traffic arrives.

F REPLAY GATE DETAILS

  • The replay gate reruns candidate and baseline arms in the same batch, using pairwise comparisons to cancel batch-level judge drift and avoid relying on noisy absolute scores.The solver remains stochastic, but fresh matched runs and relative judging remove two of the three stated sources of variation.Source: F REPLAY GATE DETAILS · p. 23
  • For rewrites, replay samples two successful and two failed retrieved-skill trajectories; for new skills, it samples the triggering cluster without score conditioning and compares against no retrieved skill.The successful rewrite cases test preservation, while failed cases test whether the revision repairs its motivating failures.Source: F REPLAY GATE DETAILS · p. 23
  • Pairwise votes with swapped presentation order yield a per-record candidate win rate; a δ tie band classifies records as improvements, regressions, or ties around 0.5.Improvement requires r > 0.5 + δ, regression requires r < 0.5 − δ, and all other records are ties.Source: F REPLAY GATE DETAILS · p. 23
  • The gate rejects candidates that tie everywhere and never trades a regression on one case for a gain on another.Rejected rewrites are discarded and increment the consecutive-failure counter; rejected mints are discarded while their occurrences remain eligible for a later attempt.Source: F REPLAY GATE DETAILS · p. 23

G COLD START FROM DOCUMENTATION

  • Cold start uses a four-phase documentation pipeline, with LLM calls reusing the frozen model and two phases requiring no LLM.The process begins by filtering, deduplicating, and organizing English product documentation into topics, then clusters topics into 20–25 candidate themes.Source: G COLD START FROM DOCUMENTATION · p. 24
  • Page assignment scores documentation pages against themes with the runtime retriever’s token-overlap tokenizer, so the same matching method seeds skills and later retrieves them.The top-scoring pages are attached to each theme.Source: G COLD START FROM DOCUMENTATION · p. 24
  • Skill generation creates a SKILL.md per theme with trigger phrases, documentation sources, catalog-derived tool whitelists, and bank-based deduplication; validation checks tools, duplicates, and completeness before saving.Themes limited to UI chrome can emit no skill, and saved skills receive neutral initial statistics.Source: G COLD START FROM DOCUMENTATION · p. 24
  • The documentation pipeline produced an initial bank of 76 skills, providing a recall floor before production and prompter traffic drive later refinement.The bank included 26 Photoshop-, 25 Illustrator-, and 25 InDesign-equivalent or cross-app skills; human review checkpoints followed clustering and raw-skill generation.Source: G COLD START FROM DOCUMENTATION · p. 24

H SUCCESS RATE ON SPECIALIZED DESIGN TASKS

EVOLVE raises hosted Claude agents’ specialized-task completion to near-perfect levels, with its largest reported reliability gain on Claude-Sonnet-4’s complex BannerRequest400 benchmark. Qwen3.6-27B results are confounded by local deployment instability.

H SUCCESS RATE ON SPECIALIZED DESIGN TASKS

  • 99.2% average completion: EVOLVE reaches near-perfect success across specialized tasks on hosted Claude backbones.Table 4 reports success as the percentage of prompts completed with a valid design within 900 seconds.Source: Table 4 · p. 24, H SUCCESS RATE ON SPECIALIZED DESIGN… · p. 25
  • 100% success on BannerRequest400: Claude-Sonnet-4 with EVOLVE improves from BASE’s 74.7%, an absolute gain of +25.3%.The passage attributes the gain to pre-verified workflows mitigating tool-use failures on a highly complex benchmark.Source: H SUCCESS RATE ON SPECIALIZED DESIGN… · p. 25
  • Qwen3.6-27B success-rate fluctuations, including +22.7% on GraphicBench, are largely confounded by timeouts and connection drops.The passage attributes these fluctuations primarily to local deployment stability rather than reasoning or tool-use failures.Source: H SUCCESS RATE ON SPECIALIZED DESIGN… · p. 25

I PERSONALIZE: WHERE RETRIEVAL-AND-INJECTION IS NOT ENOUGH · J GALLERY ON THE HELD-OUT PROMPT SET

The personalization cases show that retrieval and skill injection can reproduce a user’s exact target for a short procedure, but do not by themselves resolve default-method, precision, or long-construction limits. The held-out gallery presents outputs across the full prompt split rather than a handpicked subset.

I PERSONALIZE: WHERE RETRIEVAL-AND-INJECTION IS NOT ENOUGH

  • For the snow-effect preference, skill injection alone leaves the model using its default particle method; user-conditioned retrieval and tool scoping make it reproduce the user’s layered noise technique.The scoped tool set removes the particle-generator shortcut, while retrieval encodes the user’s preference for the noise-based method.Source: I PERSONALIZE: WHERE… · p. 25, Figure 8 · p. 26
  • For recursive Droste placement, a personalized skill yields only a modest, unstable improvement: numeric verification catches some misalignment, but pixel-exact execution remains unreliable and requires human oversight.The limitation is fine-detail perception and verification: a deterministic placement primitive does not fix the overall composition, and localization worsens as recursion deepens.Source: I PERSONALIZE: WHERE… · p. 25, Figure 8 · p. 26
  • The personalized tile skill reconstructs the requested De Nigris unit exactly in one pass as editable vector paths, arcs, and fills that tessellate into the target pattern.The BASE agent instead produces a look-alike with different structure that must be rebuilt; the successful skill combines the user’s hand-guided construction with deterministic primitives.Source: I PERSONALIZE: WHERE… · pp. 25–26, Figure 8 · p. 26
  • Long, many-step skills lose fidelity during annotation, distillation, and execution: the full pattern’s layout survives, but its three-tone fills and fine structure wash out.Hand-guiding requires correction at each added step, noisy trajectories are difficult to compress, and the frozen model may stop following the skill; the authors suggest deterministic macros, plan-level orchestration, or model updating.Source: I PERSONALIZE: WHERE… · pp. 26–27, Figure 9 · p. 27
  • The authors frame personalization as a distinct, only partly addressed axis of continual adaptation, requiring preference-aware retrieval, stronger verification, and plan-level enforcement beyond widening and deepening the skill bank.They identify method defaults, perception and verification limits, and scale-limited distillation as recurring constraints; precision-critical detail currently falls back to human-in-the-loop verification.Source: I PERSONALIZE: WHERE… · p. 27

J GALLERY ON THE HELD-OUT PROMPT SET

  • Figures 10 and 11 show round-5 skill-bank designs for the same 193 held-out prompts on Opus 4.6 and Sonnet 4, respectively, both run with low thinking budgets.The figures omit the corresponding no-skill baselines and side-by-side pairs; prompts span posters, flyers, invitations, logos, and brand and UI layouts.Source: J GALLERY ON THE HELD-OUT PROMPT SET · p. 27
  • Both galleries include every prompt in a fixed random order, allowing assessment of typical output quality, layout, and stylistic variety rather than a handpicked best-result selection.The test set is internal and prompt text cannot be released; panels preserve each prompt’s canvas aspect ratio by uniform scaling and tiling without cropping or stretching.Source: J GALLERY ON THE HELD-OUT PROMPT SET · p. 28

K ETHICAL STATEMENT

The work uses pre-existing user traffic in an offline evolution pipeline, without recruiting users or collecting study-specific data or annotations. It addresses privacy and release limits, third-party data handling, and risks from graphic-design automation.

K ETHICAL STATEMENT

  • The study uses pre-existing traffic and automated evaluation; it recruits no users or collects new data or annotations, and plans a small-scale human-perspective study.The human-authored internal benchmark and main experiments use no human reward labels; personalization examples are evaluated separately and excluded from the shared skill pool and main experiments.Source: K ETHICAL STATEMENT · p. 28
  • User-derived inputs are filtered under organizational governance, and prompts and trajectories are withheld; the gallery shows 193 held-out outputs without prompt text.The gallery panels are presented in random order, preserve aspect ratios, and are not selected for quality.Source: K ETHICAL STATEMENT · p. 28, Figure 10 · p. 29, Figure 11 · p. 30
  • User-derived data may be processed by hosted models under enterprise agreements prohibiting provider training, while third-party assets and user materials are released only when authorized and licensed.The system uses Amazon Bedrock and Azure hosted services, locally served models, and assets retrieved from Adobe Stock.Source: K ETHICAL STATEMENT · p. 28
  • Because design automation could produce misleading, deceptive, or rights-infringing content, deployment should retain content safeguards, access controls, and human oversight.Source: K ETHICAL STATEMENT · p. 28

Related papers

Loading 2609.22086v1…