PaperlayerView exact revision on arXiv

Source-linked AI summary

IDSPACE: A Novel Document Generator for Reliable Evaluation of Digital Identity Verification Systems [Extended Technical Report]

Lulu Xie, Yancheng Wang, Kanchan Chowdhury, Rolando Garcia, Yingzhen Yang, Jia Zou

arXiv:2609.03052v1cs.CVcs.LG

TL;DR

Sensitive identity documents are scarce, limiting comprehensive evaluation of remote identity verification systems. IDSPACE uses model-guided few-shot generation with declarative metadata and automatically tuned rendering controls, extending support to scanned and mobile-captured documents. With as few as two real samples, it improves evaluation consistency over baselines and releases a large multi-modal synthetic dataset.

  • Problem

    Sensitive personal information makes comprehensive real-world datasets scarce, complicating flexible and reliable evaluation of identity verification systems.

    Source: §I · p. 1

  • Method

    IDSPACE uses model-guided Bayesian optimization to tune generation parameters for visual similarity and prediction consistency, separating user-specified metadata from rendering controls.

    Source: §I · p. 2, §IV · p. 5

  • Results

    With as few as two real samples, IDSPACE improves evaluation consistency by 15–45% over CycleGAN, diffusion inpainting, and non-guided optimization, and training accuracy by up to 9% over IDNET.

    Source: §I · p. 2

  • Takeaways & Limitations

    IDSPACE provides 359,240 synthetic documents across ten European ID types and templated, scanned, and mobile-captured modalities, while reducing dependence on sensitive real documents for evaluation and training.

    Source: §I · p. 2, §VIII · p. 9

    Under few-shot and limited-data regimes, StyleGAN, diffusion models, and large generative models struggle to preserve the precise document structure and layout fidelity required.

    Source: 2) Limitations of Generative AI Models · p. 8

Abstract

from arXiv · show

As services move online, trust institutions such as banks, lenders, and governments must verify the identity of remote users. Fraud detection tools are widely available, but evaluating and fine-tuning them remains difficult because identity documents are sensitive and therefore scarce. Synthetic data generation offers a path forward, and demand is clear: our prior work in this area has been downloaded over $11{,}000$ times (aggregated from eight parts). We introduce IDSpace, extending this line of research in three directions. First, we propose model-guided Bayesian optimization, which tunes generation parameters to maximize both visual similarity and prediction consistency with target-domain models given only a few samples from a target domain. Second, we decouple user-specified metadata (demographics, fraud patterns, capture device) from automatically tuned control parameters (font styles, noise levels, image quality), allowing users to configure evaluations without low-level expertise. Third, we expand beyond template images to support scanned and mobile-captured documents. Experiments show IDSpace improves evaluation consistency by $15-45\%$ over baselines including CycleGAN, diffusion inpainting, and non-guided optimization, using only a few real samples, while improving training accuracy by up to $9\%$ and SSIM similarity with the target domain by $10\%$. We also released a new dataset consisting of $359{,}240$ high-quality synthetic documents across ten European ID types.

I. INTRODUCTION

Sensitive identity documents are scarce, making comprehensive evaluation of remote identity verification systems difficult. IDSPACE addresses this with configurable synthetic-document generation that targets both visual similarity and prediction consistency.

I. INTRODUCTION

  • Sensitive personal information limits real-world dataset collection, while recruitment costs, uncontrolled image quality, and demographic imbalance complicate evaluations across groups and document types.The motivating GSA study recruited 3,991 participants across five racial and ethnic groups using mobile devices.Source: §I · p. 1

II. RELATED WORK

  • Existing synthetic approaches trade off scale, data requirements, and target-domain fidelity; few-shot methods reduce data needs but can diverge from the target distribution and mislead fraud-detection evaluation.Manual datasets are costly to produce, while training-based approaches may require many labeled real documents and generalize poorly across regions.Source: §I · p. 1

I. INTRODUCTION

  • IDSPACE uses model-guided Bayesian optimization to tune rendering parameters for visual similarity and prediction consistency, while separating user-specified metadata from low-level rendering controls.Users can specify metadata declaratively; rendering controls are optimized internally for reproducible, controlled benchmarking.Source: §I · p. 2
  • With as few as two real samples, IDSPACE improves evaluation consistency by 15–45% over CycleGAN, diffusion inpainting, and non-guided optimization, with training accuracy up to 9% higher than IDNET.Generated documents also have consistently better SSIM similarity to the target domain than IDNET.Source: §I · p. 2
  • The released dataset contains 359,240 synthetic documents across ten European ID types and spans templated, scanned, and mobile-captured modalities.The paper reports the dataset is publicly available under CC-BY 4.0, with source code under Apache 2.0.Source: §I · p. 2

II. RELATED WORK

  • Prior IDNET work focused on template images and visual similarity; IDSPACE adds prediction-consistency guidance and supports scanned and mobile-captured documents.The introduction positions this extension as a way to support more reliable evaluation under data scarcity.Source: §II · p. 3

III. A NOVEL PROBLEM ABSTRACTION

IDSPACE frames document generation as filling user-specified metadata into a document template using control parameters. Its abstraction separates configurable document content from rendering controls that can be automatically tuned.

Problem abstraction

  • The framework separates user-specified metadata from parameters that can be automatically adjusted, while allowing advanced users to redefine that boundary based on available tools and expertise.Background color may be inferred with standard tools, whereas text color over complex backgrounds and proprietary font styles may require autotuning.Source: §III · p. 3
  • The default portrait source is a dataset of 10,000 generated photos, of which 5,979 are deemed suitable for identity documents, with metadata used to match portraits to specified demographics.Advanced users can substitute their own portrait-photo database.Source: §III · p. 4
  • The current implementation supports inpaint-and-rewrite and crop-and-move fraud patterns, with fields selected randomly and a 95% probability of matching the field across the two IDs.Photo-based fraud patterns are excluded from evaluation because established face-morphing benchmarks exist and inserting altered portraits does not add document-level generation challenges.Source: §III · pp. 4–5
  • Users specify document metadata, including fraud patterns, capture settings, and personal information; control parameters govern how that metadata is rendered into the template.The generated image is represented as xi = Gθ(xi_meta, T), with θ controlling the metadata-filling process.Source: §III · p. 5
  • Given a few real samples and a target-domain fraud-detection model, the objective is to learn rendering parameters so generated documents preserve the model’s predictions on real documents.The formalization also seeks generated images that correspond to the real samples through the same template and metadata.Source: §III · p. 5

IV. MODEL GUIDED CONTROL PARAMETER TUNING

IDSPACE tunes generation controls using a weighted objective that combines visual similarity with prediction consistency on a small target-domain sample. It then applies user-specified metadata and the tuned controls to generate templated, scanned, and mobile-captured documents.

IV. MODEL GUIDED CONTROL PARAMETER TUNING

  • The objective combines similarity between real and generated documents with consistency between guiding-model predictions on those documents.It is a weighted sum, evaluated on a small target-domain sample; the guiding models are trained in the target domain but need not be the models being evaluated.Source: §IV · p. 5
  • Bayesian optimization learns how control parameters relate to the objective, iteratively tests candidate settings, and updates its surrogate model from measured scores.The search strategy could be replaced by alternatives such as Hyperband.Source: §IV · p. 5

V. DOCUMENT GENERATION

  • Users specify document metadata separately from tuned controls, using a CSV of per-document values or YAML distributions to target evaluation groups.Uniform distributions can include minority demographic groups when evaluating fairness across age, gender, or ethnicity.Source: §V · p. 5
  • Generation applies the tuned controls to fill user-specified metadata into templates, and outputs JSON annotations including typography, coordinates, masks, and bounding boxes.These annotations facilitate downstream construction of scanned or mobile documents.Source: §V · pp. 5–6
  • For scanned documents, the pipeline fills a customized template, then combines it with a scanner background using specified position and rotation.The background is generated as part of the pipeline.Source: §V · p. 6
  • For mobile documents, users can specify backgrounds and metadata distributions; Grounding DINO localizes the document and SAM segments it for geometric alignment.The pipeline can replace a document in an existing mobile-image background with a generated template document.Source: §V · p. 6
  • Mobile-image blending uses Deep Image Blending losses, augmented with differentiable SSIM loss to improve local and global consistency with the background.The losses include Poisson gradient, content, style, histogram, and total variation terms.Source: §V · p. 6

VI. IDSPACE DATASET

IDSPACE’s dataset combines templated, scanned, and mobile documents generated across ten European identity-document types. The evaluation uses target-domain data splits for model training, testing, and control-parameter tuning, with scanned-document results placed in the appendix.

VI. IDSPACE DATASET

  • Ten templates cover identity documents from Albania, Azerbaijan, Estonia, Finland, Greece, Latvia, Russia, Serbia, Slovakia, and Spain.The templates include ID cards and passports.Source: §VI · p. 6
  • 5,979 entities supplied metadata and photos; for each, the authors generated non-fraud and two fraud-pattern documents for each of the ten document types.The fraud patterns were crop-and-move and inpaint-and-replacement; tuning used 40 balanced target-domain documents and ResNet50 guiding models.Source: §VI · p. 6
  • Each generated document was also rendered as a scanned document with randomly selected position and rotation, while 50 mobile documents were generated per document type.Mobile backgrounds were sampled from MIDV and paired with generated templates.Source: §VI · p. 6
  • 359,240 images make up the dataset: 179,370 templated, 179,370 scanned, and 500 mobile documents.The dataset was published on HuggingFace for public access.Source: §VI · p. 6

VII. EVALUATION OF IDSPACE DATA GENERATOR

  • The experiments examined prediction consistency against alternatives, benefits to learning, and the parameter-tuning approach.The target datasets contain ten European document types, with 100 templated and around 100 fraud templated documents of each type.Source: §VII · pp. 6–7
  • Evaluation used MIDV and SIDTD target-domain data, split into 50% training, 30% testing, and 20% tuning samples.The training split supplied guiding and target models; SSIM measured similarity, and scanned-document results were reported in the appendix because the main evaluation focused on customized templates.Source: §VII · pp. 6–7

A. R1. Prediction Consistency Comparison

The section compares model-guided Bayesian optimization with alternative generation and tuning approaches for prediction consistency, and examines target-domain sample counts, SSIM, and parameter variation.

A. R1. Prediction Consistency Comparison

  • The approach generalizes consistency gains beyond guiding models, and adding guiding models improves consistency in most cases, with mean consistency of 0.9354±0.003% across architectures.Source: 1) Using Models Independently… · p. 7
  • Even with two target samples, IDSPACE significantly improves prediction consistency over baselines; the study compares settings with 2, 20, and 40 samples.The testing set is disjoint from the tuning set.Source: 1) Using Models Independently… · p. 7, Table III · p. 8
  • IDSPACE consistently exceeds IDNET in SSIM across regions, while increasing guidance from 1 to 10 or 20 samples does not consistently improve SSIM.The comparison covers multiple regions and target-domain sample counts.Source: 1) Using Models Independently… · p. 7, Figure 5 · p. 8
  • Consistency scores improve 15.62%–30.55% over BO with an SSIM-only objective, 27.40%–42.33% over CycleGAN, and 30.60%–45.51% over diffusion-based inpainting.The comparison evaluates prediction consistency across models.Source: 1) Using Models Independently… · p. 7, Table II · p. 8
  • Jointly optimized models exhibit less parameter variance than single-model counterparts; RGB channels, font style and size, and stroke width vary substantially for most models.Source: 1) Using Models Independently… · p. 8

2) Limitations of Generative AI Models:

The section reports IDSPACE’s benchmark and training results, compares tuning algorithms, and describes limitations of other generators under data scarcity and document-structure requirements.

2) Limitations of Generative AI Models:

  • Under few-shot and limited-data regimes, StyleGAN, diffusion generators, GPT-4o, and GPT-image-1 struggle to preserve document layout, typography, and field alignment.This limits their suitability for evaluation requiring precise document structure and layout fidelity.Source: 2) Limitations of Generative AI Models · p. 8

3) Can LLM detect IDSPACE documents as generated or synthetic?:

B. R2. Does the dataset generated by IDSPACE also benefit the learning process?

C. R3. Tuning Algorithm Comparison

VIII. OUR IDNET DATASET AND CONCLUSIONS

  • IDSPACE improves evaluation consistency by 15–45% over CycleGAN, diffusion inpainting, and non-guided optimization such as IDNET, using 2–40 real samples, while improving training accuracy up to 9% and target-domain SSIM by 10% versus IDNET.Source: §VIII · p. 9

APPENDIX A. More Document Examples.

Examples span multiple document types, fraud patterns, and capture conditions. Model-guided Bayesian optimization improves consistency on customized templates and scanned documents.

APPENDIX A. More Document Examples.

  • Examples cover ten countries and show non-fraud templates, two fraud patterns, and simulated scanned variants alongside real scans.The appendix describes the generated dataset examples as diverse and realistic.Source: Appendix A · p. 11

B. Consistency Evaluation Results for More Types of Customized Template Documents.

  • 22.00%–34.86% relative consistency-score improvements over the SSIM-only baseline were observed for customized Finnish templates.The experiments used the same settings as Section VII and compared combinations of guiding models.Source: B. Consistency Evaluation Results for… · p. 11

C. Consistency Evaluation Results for Scanned Documents.

  • The scanned-document evaluation trained models on generated data and compared BO with and without model guidance across Albanian and Finnish documents.Consistency scores were reported in Tables VII and VIII.Source: C. Consistency Evaluation Results for… · p. 11
  • On scanned Albanian and Finnish documents, model-guided BO generally achieved higher consistency scores than on template images.The authors attribute this to template regions occupying around 20% of scanned images, making SSIM-only optimization less sensitive to subtle manipulations.Source: C. Consistency Evaluation Results for… · p. 11

D. Scaling to New Document Types.

IDSpace is evaluated on additional document types, alongside comparisons with generative baselines. The results and experiments expose both the approach’s scope and the challenges of generating faithful structured identity documents.

D. Scaling to New Document Types.

  • The released dataset covers ten European document types, each with around 100 high-quality documents per non-fraud and fraud class, spanning security features, schemas, and languages.The ten types comprise five passports and five ID cards.Source: D. Scaling to New Document Types · p. 11
  • The West Virginia license experiment reports good generalization on test sets of 300 non-fraud and 300 fraud documents.EfficientNet-b3, ResNet50, and ViT-Large were trained, with and without guiding models for parameter search.Source: D. Scaling to New Document Types · p. 11
  • Adding a template requires a one-time engineering pass to specify its segments and parameter search spaces, a cost described as modest and orders of magnitude below collecting and annotating real data.The required work is said to amortize quickly through generation scalability.Source: D. Scaling to New Document Types · p. 11

E. StyleGAN Results.

  • With 40 Albanian identity documents, StyleGAN outputs struggled to be semantically meaningful or identity-consistent, which the authors attribute mainly to limited training data.The authors note that GANs generally require hundreds to thousands of high-quality images for robust, high-fidelity representations.Source: E. StyleGAN Results · p. 12

F. Diffusion-Based Image Inpainting.

  • At sufficiently high noise strength, Stable Diffusion changed masked target segments but often corrupted the target text; higher guidance or more denoising steps did not reliably restore textual correctness.The evaluation used paired examples created by replacing a target segment with another document’s segment while keeping surrounding content unchanged.Source: F. Diffusion-Based Image Inpainting · p. 13
  • Compared with text-to-image generation, inpainting improved structural fidelity through conditioning on the original layout, but residual artifacts and background drift remained.The authors report that image-conditioned diffusion still struggles with identity documents’ precise geometry and layout constraints.Source: F. Diffusion-Based Image Inpainting · p. 14

G. Text-to-Image Diffusion Generation.

Text-to-image diffusion and general-purpose image models were tested for identity-document generation. Their outputs did not reliably preserve the precise content and structure needed for realistic ID templates.

G. Text-to-Image Diffusion Generation.

  • The text-to-image experiment used LoRA adaptation and generated five samples per checkpoint from a fixed structured prompt.The model was trained for 5000 epochs with LoRA ranks of 64 and 128.Source: G. Text-to-Image Diffusion Generation · p. 14
  • Despite fine-tuning, text-to-image diffusion failed to reproduce identity documents’ rigid layout, typographic consistency, and fine-grained security features.The Stable Diffusion model was adapted on 40 Albanian ID cards, using prompts compressed to fit the text encoder’s 77-token limit.Source: G. Text-to-Image Diffusion Generation · pp. 14–15

H. Generative Results.

  • GPT-4o replicated some background elements but left some personal details unchanged and inconsistently altered the template layout.The expiry date remained unchanged, and fields including issuing authority, personal number, and date of issue were affected.Source: H. Generative Results · p. 15
  • GPT-Image-1 modified all personal information as instructed, unlike GPT-4o, but changed the ID background; both models missed font fidelity and produced dimensions different from the input.The paper concludes that both models lack the precision needed for structural and semantic fidelity in realistic ID-template generation.Source: H. Generative Results · pp. 15–16

I. Ablation Study: Tuning of λi.

The ablation varies the consistency weight while holding the SSIM weight and guiding model fixed. Consistency rises up to a weight of 2, then declines slightly, supporting a balance between the two objectives.

I. Ablation Study: Tuning of λi.

  • The ablation tested λ1 values of 0, 0.2, 0.5, 1, 1.5, 2, 5, and 10 while fixing λ0 at 1 and using ResNet50 guidance.The experiments used the Albanian region of SIDTD.Source: §I · p. 16
  • The consistency score increased as λ1 rose to 2, then began to decline slightly.The experiment used the Albanian SIDTD dataset, ResNet50 as guiding model, and λ0 = 1.Source: §I · p. 16
  • The authors describe SSIM as fundamental to visual quality and the consistency term as supporting semantic alignment, making a balanced combination important for optimal performance.Source: §I · p. 16

J. Identity Document Fraud Detection Models used in Academic and Industry

The fraud-detection models are evaluated within a scope limited to white-light document capture and binary classification of specific fraud patterns. The study uses six open-source vision architectures as surrogates for fraud-detection behavior.

K. Mobile Document Generation

IDSPACE extends document generation to mobile images by replacing documents in existing backgrounds and blending them into the scene. Users can specify generation distributions, and the section illustrates the approach across ten European document types while noting unresolved background-matching issues.

Loading 2609.03052v1…