PaperlayerView exact revision on arXiv

Source-linked AI summary

ControlTrace: Recovering Control Fields for Hidden-Content Recognition

Zijian Liu, Yaoguang Chen, Liwei Liu, Weixi Wu, Hanming Zhang, Jiashui Wang, Na Ruan

arXiv:2609.34807v1cs.CV

TL;DR

Spatially conditioned diffusion can hide words and contours in natural-looking images that VLMs may fail to recognize. ControlTrace recovers the generation control field from the carrier for VLM reading, achieving higher contour recognition than the evaluated prior methods on FreqBlind.

  • Problem

    VLMs may fail to identify words and symbols embedded in natural-looking images, complicating inspection of hidden content.

    Source: §1 · p. 1

  • Method

    ControlTrace learns to recover the grayscale control field from a carrier image and passes the recovered field to a VLM for recognition.

    Source: §1 · pp. 1–2

  • Results

    60.2% open-ended contour accuracy across three conditioning strengths exceeded SMSP, the best evaluated prior method, by 26.9 percentage points.

    Source: §1 · p. 2

  • Takeaways & Limitations

    On FreqBlind, control-field recovery improves contour recognition with a small measured increase in complete inference latency.

    Source: §7 · p. 9

    ControlTrace’s advantage is concentrated in contours; SMSP performs better on strongly conditioned words, and cross-generator transfer depends on the control encoder.

    Source: §6 · p. 9

Abstract

from arXiv · show

Spatially conditioned diffusion models can embed words and contours in natural-looking images, but vision-language models (VLMs) may fail to recognize the hidden content. Transformation-based recovery depends on parameter and view selection. To evaluate hidden-content recovery and recognition, we construct FreqBlind, a 6,000-image benchmark spanning contours, real words and non-words across three conditioning strengths. The evaluated transformation-based methods show limited recognition of contour patterns and weakly conditioned hidden content. To address this limitation, we propose ControlTrace to recover the grayscale control field used during generation. An 8.4M-parameter U-Net predicts this field from the carrier image, and a VLM then identifies its content. With Qwen2.5-VL-7B-Instruct, ControlTrace achieves 60.2% open-ended contour recognition accuracy across the three conditioning strengths, exceeding the best of the three evaluated prior methods by 26.9 percentage points. On an A100 GPU, the complete pipeline adds only 7.4 ms (5.3%) to direct VLM inference. Recovered fields have lower pixel errors and higher structural similarity than the evaluated transformation views. Across four evaluated VLMs, ControlTrace retains its overall contour recognition advantage. Recognition remains stable under the tested JPEG compression, Gaussian noise and downsampling. These results support control-field recovery for hidden-content recognition in the evaluated setting.

1 INTRODUCTION

VLMs may miss words and symbols embedded in natural-looking images, while transformation-based recovery depends on choices of resolution, filtering, and views. ControlTrace instead recovers the generation control field; on FreqBlind, it improves contour recognition over the evaluated prior methods.

1 INTRODUCTION

  • Hidden words and symbols in natural-looking images may evade VLM recognition and moderation, while transformation-based methods depend on resolution, filtering, and view construction.The evaluated approaches include SemVink, SMSP, and AVR.Source: §1 · p. 1
  • 60.2% open-ended contour accuracy across three conditioning strengths exceeded SMSP, the best evaluated prior method, by 26.9 percentage points.ControlTrace also exceeded AVR any-view coverage by 15.1 points.Source: §1 · p. 2
  • FreqBlind contains contour targets and rendered words and non-words across three conditioning strengths.The benchmark supports evaluation of hidden-content recovery and recognition across target types and strengths.Source: §1 · p. 2
  • ControlTrace learns to recover the grayscale control field from a carrier, then passes the recovered field to a VLM for open-ended recognition.The field specifies hidden structure without the generated scene texture, and recovery at inference requires only the carrier image.Source: §1 · pp. 1–2
  • Experiments assess recovery quality, recognition, efficiency, ablations, reader and generator generalization, and image-perturbation robustness, while identifying limits in word recognition and cross-generator transfer.The reported contour-recognition improvement is especially pronounced under weak conditioning.Source: §1 · p. 2

2 RELATED WORK

Prior work studies hidden-content recognition through benchmarks and carrier transformations, while learned recovery of visual encodings offers a related approach. ControlTrace differs in predicting the grayscale control field from a ControlNet-generated carrier.

2 RELATED WORK

  • Hidden-content benchmarks study recognition of text, objects, or characters in images, establishing it as a distinct visual-understanding challenge.Examples include Hateful Illusion, HC-Bench, and IlluChar (Tu et al., 2026).Source: §2 · p. 2
  • SemVink, SMSP, and AVR expose hidden structure through downsampling, multiscale low-pass views, or complementary views, respectively.The evaluated AVR variant uses independent open-ended questions without its template bank or learned retrieval gate.Source: §2 · p. 3
  • NeuralMagicEye recovers depth encoded by texture disparities, whereas ControlTrace predicts the grayscale control field from a ControlNet-generated carrier for VLM recognition.Both approaches use paired synthetic data to train a recovery network, but their encoding mechanisms and reconstruction targets differ.Source: §2 · p. 3

3 PROBLEM SETTING

The task is to identify content represented by a grayscale control field when only its natural-looking generated carrier is available. FreqBlind tests this setting across target types, scene prompts, and conditioning strengths.

3 PROBLEM SETTING

  • Given a carrier image, the task is to identify the hidden content represented by its control field through an open-ended response, without a candidate list or target template.Conditioning strength controls the field’s influence during generation.Source: §3 · p. 3
  • At test time, the control field and generation prompt are unavailable; inference requires only the carrier and a recognition prompt, with no generator call.Training retains the fields used to generate its carriers.Source: §3 · p. 3
  • FreqBlind contains 6,000 carriers from 50 contour targets and 50 word targets, each rendered with 20 scene prompts at three conditioning strengths.The word targets comprise 25 real four-letter words and 25 pronounceable non-words; word-domain scores combine these equal halves.Source: §3 · p. 3
  • Carriers use Realistic Vision V5.1, a Stable Diffusion 1.5 checkpoint, with QR Code Monster ControlNet and shared indoor and outdoor scene prompts.Conditioning strength scales the control branch’s contribution during generation.Source: §3 · p. 3

4 CONTROLTRACE

ControlTrace predicts the generator’s grayscale control field from a carrier, separating spatial structure from scene-dependent texture. Its training objective combines field reconstruction, gradient consistency, and auxiliary edge supervision.

4.1 CONTROL-FIELD RECONSTRUCTION

  • ControlTrace predicts the generating control field because different scene prompts can yield different carrier appearances while retaining the same spatial target.Paired carrier–control examples provide supervision without requiring recovery of scene-dependent textures; the four-level U-Net has 8.37M parameters.Source: §4.1 · p. 3
  • At inference, the carrier is resized from 512 × 512 to 256 × 256, and a VLM reads the recovered grayscale field with a task prompt.Auxiliary edge predictions are discarded at inference; the workflow figure distinguishes reconstruction and gradient supervision from training-only edge loss.Source: §4.1 · p. 4, Figure 2 · p. 4

4.2 TRAINING OBJECTIVE

  • The reconstruction objective combines L1 grayscale error with cosine alignment across corresponding pixels, while gradient consistency compares clipped Sobel magnitudes in recovered and target fields.Gradient consistency complements reconstruction because gradient magnitudes alone do not determine the grayscale field.Source: §4.2 · pp. 4–5
  • An auxiliary edge head supervises edge locations from shared decoder features, separately from the gradient loss applied directly to the grayscale output.Losses are averaged over the batch.Source: §4.2 · pp. 4–5

5 EXPERIMENTS

The experiments evaluate recognition, field recovery, generalization, robustness, specificity, and efficiency on FreqBlind. ControlTrace leads on contour recognition overall, while its transfer and vulnerability vary across tested settings.

5.1 EXPERIMENTAL SETUP

  • Tests used Qwen2.5-VL-7B-Instruct across three strengths, with 1,000 carriers per domain–strength cell; trained recognition and fidelity results averaged three seeds.Training used source-control splits, and FreqBlind controls and scene prompts were disjoint from training.Source: §5.1 · p. 5, Table 1 · p. 6

5.2 RECOGNITION PERFORMANCE

  • 60.21% contour accuracy versus 19.43% SemVink, 33.33% SMSP, and 19.37% AVR plurality; ControlTrace exceeded SMSP by 26.88 points and AVR any-view coverage by 15.11 points.The comparison spans all three conditioning strengths.Source: §5.2 · pp. 5–6, Table 1 · p. 6

5.3 RECOVERY QUALITY

  • On contours, ControlTrace achieved MSE 0.0327 and SSIM 0.8312 versus SMSP’s view mean at 0.1172 and 0.1193; paired MSE reduction was 0.0846.ControlTrace also had lower MSE and MAE and higher SSIM than transformation summaries in both domains, while SMSP retained higher aggregate word recognition.Source: §5.3 · p. 6, Table 2 · p. 7
  • 45.40% weak-contour accuracy came from control-field supervision versus 13.27% scene supervision; focusing training at s = 1.0 added 6.60 points, while removing gradient loss cost 2.20.At matched approximate capacity, NAFNet reached 56.51% aggregate contour recognition, 3.70 points below U-Net; removing the edge head changed accuracy by 0.27 points, and greater depth gave no consistent benefit.Source: §5.3 · pp. 6–7, Table 3 · p. 7

5.4 GENERALIZATION AND ROBUSTNESS

  • At weak conditioning, ControlTrace exceeded SMSP and AVR plurality on contours for all four readers, but without adaptation it exceeded SMSP on only three of five QR-based generator configurations.Both QR Code configurations reversed the generator comparison; transfer depended on the control encoder.Source: §5.4 · p. 7, Figure 3 · p. 8
  • JPEG compression, Gaussian noise, and downsampling changed weak-contour accuracy by at most 2.0 points, whereas 20-step PGD reduced it from 43.33% to 9.00% at ε = 8/255.PGD impaired recovery at every strength and targeted the recovery network without differentiating through the VLM.Source: §5.4 · pp. 7–8, Figure 4 · p. 8

5.5 SPECIFICITY

  • 36.3% of targets were recognized, with false-positive rates of 2.9–5.3%; those rates were lower than SemVink and SMSP on COCO but higher on other negatives.The evaluation used contours at s = 1.0 and a shared none-capable prompt.Source: §5.5 · p. 8, Table 4 · p. 9

5.6 INFERENCE EFFICIENCY

  • 146.0 ms per carrier was required, versus 138.6 ms for direct VLM reading, a 7.4 ms increase; ControlTrace used 72.4% and 85.7% fewer total tokens than SMSP and AVR.ControlTrace retained one VLM call and the same average input token count as direct reading.Source: §5.6 · p. 9, Table 5 · p. 9

6 LIMITATIONS

The reported advantages are scoped to contours and tested settings; several evaluation choices and vulnerabilities limit broader conclusions.

6 LIMITATIONS

  • ControlTrace’s advantage is concentrated in contours; SMSP performs better on strongly conditioned words, and transfer depends on the control encoder.Paired carrier–field data are required; multi-generator training and adaptation remain untested, and enhanced-scene supervision reuses losses developed for control fields.Source: §6 · p. 9
  • AVR was evaluated with fixed seven-view queries and plurality selection, excluding learned retrieval and parameter optimization.Transformation and learned configurations were selected by downstream accuracy without a held-out split, so reported intervals do not include selection uncertainty.Source: §6 · p. 9
  • PGD revealed recovery-network vulnerability, but post-attack human legibility and adversarial training remain untested.Specificity results also do not estimate precision at unknown deployment prevalence.Source: §6 · p. 9

7 CONCLUSION

ControlTrace recovers generator control fields to improve hidden-content recognition on FreqBlind, while its applicability is bounded by deployment and attack considerations. The appendix documents benchmark generation, reader prompts, and reproducibility materials.

7 CONCLUSION

  • ControlTrace improves contour recognition on FreqBlind with a small measured latency increase, while dependence on control encoders and adapted attacks bounds applicability.The authors also note that releasing recovery weights enables white-box evasion.Source: §7 · p. 9
  • Deployment-prevalence precision is not established, so recovered content requires independent verification; releasing recovery weights also enables white-box evasion.The benchmark contains neutral objects and words, not hateful, personal, or otherwise sensitive content.Source: ETHICS STATEMENT · p. 10
  • Training and evaluation code, three checkpoints, manifests, raw records, and analysis scripts are available, but complete image datasets are not included.Retained images and hashes address the fact that identical seeds do not guarantee bitwise reproduction across software and hardware environments.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 GENERATION SETTINGS

  • FreqBlind carriers use Realistic Vision V5.1 and QR Code Monster, while training pairs use SD1.5 with the same control encoder but different base weights and sampling budgets.The retained generation manifests record prompts, controls, strengths, and seeds.Source: §A.1 · p. 12

A.2 READER PROMPTS

  • Reader prompts ask for the hidden animal, object, or word without supplying the correct target, candidate list, or answer-informed follow-up.HC-Bench is evaluated under both its quoted questions and the FreqBlind task prompts.Source: §A.2 · pp. 12–13

A.3 BASELINE IMPLEMENTATIONS

The appendix specifies transformation and scoring implementations, then tests recognition at intermediate conditioning strengths. ControlTrace remains ahead of the evaluated baselines across those added strengths.

A.3 BASELINE IMPLEMENTATIONS

  • SemVink uses a 40-pixel-wide carrier view, SMSP supplies the original plus three low-pass views, and the evaluated AVR variant independently queries seven views.The AVR implementation omits the template bank, learned retrieval gate, and related components; fixed parameters are not claimed to reproduce the authors’ settings or optimum.Source: §A.3 · p. 13
  • Plurality voting normalizes complete answers, selects the most frequent response, and breaks ties by view order; the main contour evaluation selects the original view in 58.8% of cases.Winning answers tie in 26.3% of cases, illustrating limits of exact-answer voting when views elicit different descriptions.Source: §A.3 · p. 14

A.4 SCORING SENSITIVITY

  • Changing the response-matching rule can raise or lower scores, because complete-token matching excludes embedded substrings while final-token matching may accept less specific responses.The sensitivity test changes automatic scoring, not images, reader outputs, or AVR answer selection.Source: §A.4 · p. 14, Table 7 · p. 15

B.1 INTERMEDIATE CONDITIONING STRENGTHS

  • 56.8%, 60.1%, and 63.0%: ControlTrace contour accuracy across added strengths s = 1.2, 1.3, and 1.4, versus 26.2%, 32.5%, and 34.7% for SMSP.The experiment adds 3,000 carriers; results do not isolate why this differs from SemVink’s published performance.Source: §B.1 · p. 15, Figure 5 · p. 15
  • SemVink’s reported 91–100% HC-Bench accuracy is not a directly comparable FreqBlind reference because the dataset, reader, and judgement protocol differ.The added-strength evaluation uses FreqBlind’s 50 targets and 20 scenes per target.Source: §B.1 · p. 15

B.2 HC-BENCH EVALUATION

The HC-Bench checks show that prompt effects vary by method and target domain, while paired lexical controls show accuracy drops when real words are replaced by anagrams. Comparisons to published results remain constrained by protocol differences.

B.2 HC-BENCH EVALUATION

  • 10.7% to 42.9%: SMSP’s HC-Bench text score rises when its prompt changes to the FreqBlind task question; downsampling scores shift by at most 7.2 points.The prompt effect depends on target domain and method; the comparison uses 56 object and 28 Latin-text images under both prompt regimes.Source: §B.2 · p. 16, Table 8 · p. 16
  • The baseline comparison is implementation-level rather than a reproduction of published headline scores, because targets, readers, prompts, and scoring differ.HC-Bench also includes targets requiring more specific names than FreqBlind’s common-noun contours.Source: §B.2 · p. 16

B.3 LEXICAL CONTROLS

  • 31.8 points: ControlTrace’s aggregate paired single-answer accuracy drop on exact-anagram words, while the simultaneous spatial rearrangement prevents attributing the full effect to lexical completion.The experiment preserves each word’s letters and rendering procedure while changing letter order and spatial arrangement.Source: §B.3 · p. 16, Figure 6 · p. 17

C RECOVERY QUALITY AND TRAINING DESIGN

ControlTrace predicts grayscale control fields from carrier images, and the appendix evaluates reconstruction fidelity, supervision and backbone choices, training configuration, and model-selection scope. The reported controls favor control-field supervision and the selected U-Net at s = 1.0.

C RECOVERY QUALITY AND TRAINING DESIGN

C.1 CONTROL-FIELD FIDELITY

  • Fidelity evaluation compares predictions against generating controls across all 6,000 carriers; lower MSE and MAE indicate smaller pixel errors, while higher SSIM indicates greater local structural similarity.Views and trained-model outputs are aggregated as scores, and fidelity metrics do not replace open-ended recognition.Source: §C.1 · p. 18
  • ControlTrace improved all three fidelity metrics over an all-zero field in both domains, but high SSIM can reflect large shared backgrounds rather than recovered content.The authors advise interpreting SSIM alongside pixel errors and recognition.Source: §C.1 · p. 18

C.2 SUPERVISION AND BACKBONE CONTROLS

  • 32.13 points: at s = 1.0, control-field supervision increased accuracy over a stronger-conditioning scene target; the U-Net increased accuracy over NAFNet by 7.27 points.Both comparisons average three training seeds on the same 1,000 carriers.Source: §C.2 · p. 20, Table 12 · p. 20

C.3 TRAINING CONFIGURATIONS

  • The reference model trains on 12,000 pairs at s = 1.0, with separate FreqBlind controls and scene prompts absent from the training pool.The full pool contains 36,000 carrier-control pairs across three strengths.Source: §C.3 · p. 20

C.4 MODEL SELECTION

  • FreqBlind recognition informed training-configuration selection, with no separate selection subset; item-bootstrap intervals do not account for this selection process.The fixed-operation sweeps’ best observed settings were identified on evaluated data.Source: §C.4 · p. 20

D TRANSFORMATION SENSITIVITY

  • The transformation sweeps examine how Gaussian blur and downsampling trade scene-texture suppression against retention of hidden structure.These fixed-operation sweeps complement the comparison of complete recognition pipelines.Source: D TRANSFORMATION SENSITIVITY · p. 21

D.1 FIXED-OPERATION SWEEP

ControlTrace’s weak-contour advantage persists across four VLM readers, while word comparisons vary by reader. The fixed-operation sweep finds modest peaks within tested settings, and clean-field scores provide reader references.

D.1 FIXED-OPERATION SWEEP

  • On weak contours, Gaussian blur reaches 15.00% at σ = 16 versus 11.00% at σ = 8, but their intervals do not resolve which setting is better.The peak tested scores occur away from the endpoints; the figure’s lines connect tested settings rather than an exhaustive search.Source: §D.1 · p. 21, Figure 7 · p. 21
  • The sweep covers tested grids, not an upper bound on transformations, and both fixed and learned configurations were selected using downstream recognition accuracy.The reported intervals do not include configuration-selection uncertainty.Source: §D.1 · p. 21

E.1 READER GENERALIZATION

  • Across strengths, word differences from SMSP are −0.89, −9.44, +11.33, and +11.00 points for the four readers, respectively; the latter three intervals exclude zero.The word domain combines real words and non-words, and all readers use the same recovery checkpoint.Source: §E.1 · pp. 22–23, Table 17 · p. 23
  • At s = 2.0 with InternVL3-8B, AVR any-view coverage reaches 82.67%, above ControlTrace’s 80.33%, despite ControlTrace’s higher aggregate word accuracy across strengths for all four readers.Across strengths, ControlTrace’s word accuracy exceeds AVR’s selected-answer accuracy and separate seven-answer coverage for every reader.Source: §E.1 · pp. 22–23, Table 17 · p. 23
  • Weak-contour gains over SMSP are 28.00, 15.00, 32.00, and 39.33 points for Qwen2.5-VL-7B, InternVL3-8B, Qwen2.5-VL-32B, and LLaVA-OneVision-7B, respectively.All four paired intervals exclude zero.Source: §E.1 · p. 23, Table 16 · p. 23

E.2 CLEAN-FIELD RECOGNITION

  • Clean-field word accuracy exceeds contour accuracy by 2–38 points across the four readers, providing reader references rather than a direct test of lexical content.The paired anagram comparison examines lexical content more directly.Source: §E.2 · p. 23, Table 18 · p. 24

E.3 GENERATOR TRANSFER

Transfer without adaptation varies across generator configurations: ControlTrace leads on several QR-based settings but underperforms on both QR Code configurations and the auxiliary Canny setting.

E.3 GENERATOR TRANSFER

  • Across five QR-based configurations, ControlTrace has higher aggregate contour accuracy than SMSP in three and AVR vote in four; on SD1.5 with QR Code Monster, scores are 63.78%, 37.56%, and 22.56%, respectively.The seven-answer diagnostic reaches 47.33% in that configuration, but exceeds ControlTrace at s = 2.0 on SDXL and QR Pattern.Source: §E.3 · p. 24
  • On SD1.5 QR Code, ControlTrace reaches 4.78% across strengths, below SMSP’s 22.56%, AVR plurality’s 11.56%, and AVR coverage’s 27.89%.On SD2.1 QR Code, ControlTrace’s 27.56% is also below SMSP’s 30.11% and AVR coverage’s 37.44%.Source: §E.3 · p. 24
  • The transfer results support sensitivity to the encoding process, rather than a general transfer advantage for learned recovery.Across strengths, AVR any-view coverage exceeds ControlTrace on QR Pattern and both QR Code configurations.Source: §E.3 · p. 24
  • In the auxiliary Canny configuration, ControlTrace’s contour accuracy is 16.00%, below direct reading at 34.33%, SMSP at 38.89%, and AVR vote at 35.33%.This edge-control reference is separate from the QR-based hidden-content comparison.Source: §E.3 · p. 24

F ROBUSTNESS

ControlTrace remains close to its unperturbed performance under the tested common image changes, but targeted PGD attacks substantially reduce weak-contour accuracy. The appendix also reports negative-control statistics and qualitative recovery examples, including imperfect shape recovery.

F.1 COMMON IMAGE PERTURBATIONS

  • Under JPEG compression, Gaussian noise, and downsampling, weak-contour accuracy is 41.3–44.0%, with a largest absolute change of 2.33 points and paired intervals including zero.Each method receives the same perturbed carrier; the contour table reports 300 carriers per strength.Source: §F.1 · p. 26, Table 21 · p. 26
  • Across strengths, ControlTrace stays within one point of its unperturbed score in both domains, while downsampling raises direct word recognition from 2.56% to 6.44%.AVR’s single-answer word accuracy also rises from 22.1% to 27.9% under downsampling.Source: §F.1 · p. 26, Table 21 · p. 26, Table 22 · p. 27

F.2 ADVERSARIAL EVASION

  • Low-, mid-, and high-band noise change weak-contour accuracy by −1.33, +0.33, and −0.67 points, while PGD reduces it by 23.00 points at 4/255 and 34.33 points at 8/255.The PGD reductions and the 33.67-point reduction after JPEG recompression have paired intervals excluding zero.Source: §F.2 · pp. 26–27, Table 23 · p. 27
  • Across strengths after the 8/255 attack, contour accuracy is 31.78% for ControlTrace versus AVR coverage of 42.33%; word accuracy is 19.56% for ControlTrace versus SMSP’s 59.00%.After JPEG recompression, the corresponding word scores are 20.78%, 23.00%, 39.22%, and 58.33% for ControlTrace, AVR plurality, AVR coverage, and SMSP.Source: §F.2 · pp. 27–28, Table 23 · p. 27, Table 24 · p. 28

G.1 NEGATIVE CONTROLS

  • Figure 8 compares recovered fields from six configuration-matched positive and negative generations; the pairs share settings but not pixels because removing control residuals changes the generation trajectory.The examples use one scene per target sampled from 1,000 matched pairs.Source: Figure 8 · p. 29

G.2 RECOVERED-FIELD STATISTICS

  • Recovered-field gradient energy gives AUC 0.9947 against matched control-off images and 0.8070 against non-semantic conditioning, but this diagnostic is not content recognition or deployment precision.The other reported AUCs are 0.9792 for dense-texture scenes and 0.9895 for COCO photographs.Source: §G.2 · pp. 29–30, Figure 9 · p. 30

G.3 QUALITATIVE RECOVERY

  • Across the eight sampled targets, Figure 10 shows imperfect recoveries: Diamond lacks facets, and Snake is recovered as a different shape.The figure expands the sampled comparison underlying Figure 1, including weaker recoveries.Source: Figure 10 · p. 31, §G.3 · p. 31

Related papers

Loading 2609.34807v1…