Source-linked AI summary
Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching
Huai-Hsun Cheng, Siang-Ling Zhang, Yu-Lun Liu
TL;DR
Progressive Semantic Illusions ask how one vector sketch can support distinct meanings as strokes are added, rather than through spatial transformations. Stroke of Surprise jointly optimizes prefix and delta strokes with dual-branch SDS and Overlay Loss to discover shared structure and prevent occlusion. The method outperforms baselines in recognizability and illusion strength, with reported user selections of 67.7% and 87.1% in two ranking settings.
Problem
Progressive illusions require prefix strokes to depict one object while simultaneously forming the structural basis for a second object.
Method
Stroke of Surprise jointly optimizes shared stroke parameters with dual-branch SDS and Overlay Loss for complementary, structurally integrated additions.
Results
The method significantly outperforms baselines in recognizability and illusion strength, while users selected it in 67.7% of GPT-ranking and 87.1% of Metric-ranking cases.
Takeaways & Limitations
The framework extends visual anagrams from spatial transformations to temporal, additive semantic re-contextualization in vector sketches.
Takeaways & Limitations
Weak SDS guidance for complex structures such as scissors can cause optimization failure.
Abstract
from arXiv · showhide
Visual illusions traditionally rely on spatial manipulations such as multi-view consistency. In this work, we introduce Progressive Semantic Illusions, a novel vector sketching task where a single sketch undergoes a dramatic semantic transformation through the sequential addition of strokes. We present Stroke of Surprise, a generative framework that optimizes vector strokes to satisfy distinct semantic interpretations at different drawing stages. The core challenge lies in the "dual-constraint": initial prefix strokes must form a coherent object (e.g., a duck) while simultaneously serving as the structural foundation for a second concept (e.g., a sheep) upon adding delta strokes. To address this, we propose a sequence-aware joint optimization framework driven by a dual-branch Score Distillation Sampling (SDS) mechanism. Unlike sequential approaches that freeze the initial state, our method dynamically adjusts prefix strokes to discover a "common structural subspace" valid for both targets. Furthermore, we introduce a novel Overlay Loss that enforces spatial complementarity, ensuring structural integration rather than occlusion. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art baselines in recognizability and illusion strength, successfully expanding visual anagrams from the spatial to the temporal dimension. Project page: https://stroke-of-surprise.github.io/
1 Introduction
The paper introduces Progressive Semantic Illusions, where sequentially added strokes transform one recognizable vector sketch into another concept. Stroke of Surprise jointly optimizes prefix and delta strokes to satisfy both interpretations through a shared structural subspace and spatially integrated additions.
- 1 Introduction: Progressive Semantic Illusions shift visual illusions from spatial viewpoint changes to semantic transformations driven by sequential stroke accumulation.Sparse line drawings support this shift because their incompleteness invites perceptual reinterpretation as new strokes arrive.
- 1 Introduction: The task imposes a dual constraint: prefix strokes must depict object A while also forming the structural foundation for object B.Existing raster and greedy sequential approaches do not address this additive requirement effectively.
- 1 Introduction: Stroke of Surprise uses sequence-aware joint optimization to discover a common structural subspace valid for both semantic interpretations.Prefix and full-phase objectives are optimized together rather than freezing the initial state.
- 1 Introduction: A dual-branch SDS mechanism keeps prefix strokes recognizable while priming them for reinterpretation, and Overlay Loss enforces complementarity instead of occlusion.The resulting delta strokes can structurally re-contextualize existing forms, such as pig ears becoming angel wings.
- 1 Introduction: The framework supports K-phase illusions and alternative representations including B-splines, colored strokes, and general vector graphics.Experiments and user studies report improved recognizability and illusion strength over baselines.
2 Related Work
Prior work covers vector sketch synthesis, sequential stroke generation, and spatial visual illusions, but these paradigms generally target static drawings or viewpoint-dependent transformations. Stroke of Surprise instead combines vector stroke optimization with multi-stage semantic guidance for additive temporal illusions.
- 2 Related Work: Vector graphic synthesis methods optimize Bézier curves through differentiable rasterization using CLIP or diffusion-based guidance, typically for a single static target.These approaches do not require the same strokes to satisfy different interpretations at different completion stages.
- 2 Related Work: The pipeline separates learnable strokes into prefix and delta subsets and applies parallel semantic guidance to partial and complete renderings.The prefix branch renders only prefix strokes for one prompt, while the full branch renders all strokes for the second prompt.
- 2 Related Work: The total SDS guidance combines branch losses, and its gradients update all learnable stroke parameters.This shared optimization supports both semantic objectives rather than treating the prefix as fixed.
- 2 Related Work: Sequential sketch methods model drawing as a temporal stroke process using recurrent, transformer, or autoregressive geometric representations.Their focus on generating stroke sequences differs from jointly satisfying two semantic interpretations with shared geometry.
- 2 Related Work: Most computational illusions use symmetric spatial transformations such as flips, rotations, reprojections, or viewpoint changes to exchange complete images.Progressive illusions are asymmetric and additive because the prefix is a strict geometric subset of the final drawing.
3 Method
Stroke of Surprise jointly optimizes prefix and full sketches so early strokes satisfy both semantic interpretations, while an overlay loss enforces spatial complementarity. The framework also supports multi-phase illusions and uses VLM assessment and quantitative ranking to select candidates.
- Joint Optimization Pipeline: Prefix and delta strokes are jointly optimized through parallel SDS branches, allowing prefix strokes to satisfy both target concepts.The prefix branch is conditioned on p1, while the full branch is conditioned on p2; shared parameters receive gradients from both objectives.
- Overlay Loss for Spatial Coordination: The overlay loss penalizes spatial overlap between separately rendered, Gaussian-blurred stroke maps to promote structural integration.Gaussian blurring creates soft spatial buffers, and the normalized inner product measures overlap across pixel space.
- Overlay Loss for Spatial Coordination: The final objective combines SDS guidance with a weighted overlay penalty, with gradients backpropagated through differentiable rasterization.The overlay weight controls the contribution of the spatial penalty.
- Filtering and Ranking: GPT-4o evaluates phase recognizability, single-object integrity, illusion quality, and sketch quality, filtering candidates that fail minimum thresholds.Illusion quality compares the full sketch against delta strokes alone to assess the prefix’s structural contribution.
- Filtering and Ranking: Candidates are ranked either by GPT-based semantic scores or by a product of CLIP, IR, and HPS scores.The metric-based score is defined as R = S_CLIP · S_IR · S_HPS.
- Extension to Multi-Phase Illusions: The framework scales to K-phase illusions by jointly optimizing cumulative stroke subsets, giving early strokes gradients from subsequent branches.Each cumulative sketch is conditioned on its corresponding prompt, and overlay penalties are extended between successive subsets.
4 Experiments
Experiments evaluate progressive illusion quality through baseline comparisons, user studies, ablations, and extensions. Results show that joint optimization, spatial complementarity, suitable initialization, and concept-dependent stroke budgets improve illusion quality and versatility.
- Experimental Setup: The evaluation uses text-to-illusion and ours-to-illusion protocols across 64 common objects, with multiple optimization runs followed by filtering and ranking.Baselines include raster, vector, and image-based methods; image-based methods receive matched SDXL-generated references.
- Experimental Setup: CLIP, structural concealment, and semantic concealment measure semantic alignment, prefix contribution, and phase-specific semantics.Structural concealment is defined as M_full − M_delta, with higher values indicating retained prefix structure.
- Results and Analysis: 100% coverage versus Nano Banana Pro’s 34.9% demonstrates the strongest reported quantitative comparison for the proposed method.The method also substantially outperforms baselines in CLIP and concealment scores.
- Results and Analysis: With fixed prefixes, baselines improve, but ours retains the highest success rate and structural consistency because delta strokes are jointly optimized with the prefix.The result indicates that optimized prefixes embed structural cues for the second concept.
- Results and Analysis: Participants selected the method in 67.7% of GPT-ranking cases and 87.1% of Metric-ranking cases, while ranking outputs exceeded 98% overall satisfaction.The studies involved 143 participants and evaluated both preference and ranking reliability.
- Ablation Studies: Joint optimization outperforms sequential optimization by coordinating prefix and delta strokes within a common structural subspace.Ablations also show that centered initialization, Overlay Loss, and concept-dependent stroke budgets support coherent transitions and recognizability.
- Applications: The framework extends to three-phase illusions and alternative representations including B-splines, vector graphics, and colored sketches.Two-phase results remain robust across concept pairs ranging from structurally similar to semantically distant.
5 Conclusion
Stroke of Surprise frames semantic illusion as temporal re-contextualization through progressive stroke addition. The conclusion reports semantic accuracy and perceptual surprise, while noting failures inherited from diffusion priors.
- Conclusion: Stroke of Surprise presents progressive semantic illusions in vector sketching, shifting visual illusion from spatial ambiguity to temporal transformation.The framework enables semantic re-contextualization as strokes are added.
- Conclusion: The method uses jointly optimized prefix and delta strokes, with Overlay Loss promoting structural integration without obfuscation.Its evaluation is described as semantically accurate and perceptually surprising.
- Limitations: The method inherits limitations from pretrained diffusion priors, and weak SDS guidance for complex structures such as scissors can cause optimization failure.Visual examples are provided in the supplementary material.
- Additional Results: The supplementary material provides additional two-phase and three-phase results, qualitative comparisons, rankings, and representation extensions.These include vector graphs, variable-width B-splines, and colored strokes.
- Evaluation Pipeline: The Phase 1 prompt scores prefix-only sketches by likeness, recognizability, and single-object integrity.It outputs one score from 0 to 10 for the target phase.
- Evaluation Pipeline: The Phase 2 prompt evaluates the full sketch against the second target and checks whether it is more complete than the delta alone.The integration check penalizes cases where the delta already appears as complete as the full sketch.
- User Studies: The user studies involved 143 participants who completed both studies sequentially.Study materials and evaluation prompts are described alongside the ranking pipeline.
B.1 Study 1: Method Comparison
Study 1 compares four methods on progressive illusion sketches using participant preferences. Participants assess phase semantics, transition smoothness, and perceptual reversal.
- Study 1: Method Comparison: Participants compare SketchDreamer, Ours, SketchAgent, and Nano Banana Pro across four candidate illusion sketches.The representative survey presents methods as options A through D.
- Study 1: Method Comparison: Each sketch is shown as Phase 1, Phase 2, and an animated toggle between phases.This presentation lets participants directly perceive the structural transition.
- Study 1: Method Comparison: Participants judge clear semantics at each phase, smooth structural transition, and perceptual reversal rather than mere stroke accumulation.They select one method or “Other” if none is satisfactory.
B.2 Study 2: Ranking Pipeline Validation
Study 2 evaluates whether the ranking pipeline reliably surfaces successful illusions from the method’s top-ranked outputs. Participants may select multiple satisfactory sketches or none.
- Study 2: Ranking Pipeline Validation: Participants review four questions containing the method’s top-4 ranked outputs for the same prompt pair.The study measures ranking reliability rather than direct baseline preference.
- Study 2: Ranking Pipeline Validation: Participants select all sketches they consider successful, using clear phase semantics, smooth transitions, and perceptual reversal as guidance.Selecting none is allowed.
C Initialization and Implementation Details
The framework represents each stroke as a cubic Bézier curve and optimizes initialized stroke sets under fixed rendering, guidance, and overlay-loss settings. It also reports runtime and repeatability-related implementation details.
- Stroke representation: Each stroke is a single-segment cubic Bézier curve with four control points, represented by an N×4×2 learnable parameter tensor.Optimization uses 512×512 resolution with 2.5 px strokes; exported SVGs use 1024×1024 resolution with 5 px strokes.
- Stroke initialization: Strokes are initialized near the canvas center using a gathered strategy, with a sampled anchor and uniformly perturbed subsequent control points.The anchor is sampled from U([0.3, 0.7]^2), and each perturbation is drawn from U([-0.025, 0.025]^2).
- Repeatability: Five runs with a fixed random seed produce recognizable, structurally coherent rabbit-to-elephant illusions despite minor CUDA-induced stroke variations.
- Optimization: Optimization uses Adam for 2,000 iterations, with shared SDS guidance scale 100, a frozen Stable Diffusion v1.5 backbone, and overlay-loss weight λ_overlay = 0.1.The default configuration uses k=16 prefix strokes and N=32 total strokes, yielding 16 delta strokes.
- Overlay loss: The overlay buffer is computed by Gaussian-blurring separately rasterized prefix and delta maps before their normalized inner product.The blur uses σ=2.0 with a 15×15 kernel.
- Runtime: Experiments run on a single NVIDIA RTX 4090, requiring approximately 13 minutes for two-phase and 15 minutes for three-phase illusions.
D Reproducibility and Variance Analysis
A fixed-seed robustness test varied only CUDA execution order across five runs of the same rabbit-to-elephant prompt pair. All outputs remained recognizable at both phases and showed a clear structural transition despite minor geometric variation.
- Robustness under CUDA nondeterminism: Five fixed-seed runs of the rabbit-to-elephant prompt pair remained recognizable at both phases despite minor geometric variations from CUDA nondeterminism.All outputs also exhibited a clear structural transition.
E Quantitative Ablation Studies
The ablations evaluate initialization, optimization strategy, and overlay loss using matched prompt, stroke-count, and stroke-width settings. Joint optimization, gathered initialization, and overlay loss each improve concealment quality, with overlay loss producing the largest reported drop when removed.
- Study design: The ablation study evaluates three design choices across five prompt pairs and 30 illusions per setting: initialization, optimization strategy, and overlay loss.All settings use identical prompts, stroke counts, and stroke widths.
- Optimization strategy: Joint optimization discovers a common structural subspace by coordinating competing gradient fields rather than freezing prefix strokes.
- Stroke initialization: 3.759 versus 5.723 on concealment CLIP: scattered initialization scores lower than centered gathered initialization, with IR and HPS corroborating the gap.The comparison is Abl. 1 versus Ours.
- Overlay loss: 5.723 to 2.421 on concealment CLIP: removing overlay loss causes the largest reported degradation, with similarly pronounced declines in IR and HPS.Without the constraint, delta strokes clutter the prefix rather than structurally integrating with it.
F Applications: Additive and Subtractive Modes.
The framework supports additive, subtractive, and mixed interaction paradigms using the same optimized stroke sets. These modes change which strokes are rendered or removed, enabling multi-stage semantic transitions without re-optimization in subtractive mode.
- Interaction paradigms: Additive mode reveals concepts through sequential stroke accumulation, while subtractive mode starts from the full sketch and removes delta strokes in reverse order.Subtractive transitions require no re-optimization, only reversal of rendering order.
- Multi-stage transitions: For apple→angel→chicken, rendering selected delta-stroke subsets can produce the intermediate angel concept before subsequent transitions.The framework controls the sequence through rendered subsets and their order.
- Interaction paradigms: Mixed mode interleaves stroke addition and removal within a single sequence.