PaperlayerView exact revision on arXiv

Source-linked AI summary

RealCAD: Towards Real-World Image-to-CAD Reconstruction under Domain Shift and Parameter Bias

Yihe Sun, Ziyu Lu, Kaihua Tang, Xian-Sheng Hua

arXiv:2608.30617v1cs.CV

TL;DR

Image-to-CAD reconstruction must overcome both the synthetic-to-real appearance gap and DeepCAD parameter bias, which can make parameter accuracy reflect frequent-value priors rather than image-conditioned geometry. RealCAD revises the representation, adapts images with geometry constraints, aligns cross-view and cross-domain features, and introduces OpenRealCAD; experiments reduce prior-based accuracy and improve real-domain command and parameter accuracy, while geometric-quality and validity gains remain limited.

  • Problem

    Existing image-to-CAD methods rely mainly on synthetic renderings and a biased DeepCAD parameterization, limiting reliable evaluation and real-world reconstruction.

  • Method

    RealCAD redistributes DeepCAD scale information, uses geometry-constrained synthetic-to-real translation, and applies multi-positive contrastive learning across viewpoints and image domains.

  • Results

    RealCAD improves command and parameter accuracy on real images, while the revised representation reduces accuracy attainable from marginal frequency priors and retains competitive synthetic-domain performance.

  • Takeaways & Limitations

    The revised representation makes parameter accuracy a more reliable measure of image-conditioned geometric inference, while OpenRealCAD enables reproducible real-image evaluation.

  • Takeaways & Limitations

    RealCAD remains limited by intrinsically long-tailed CAD parameters and token-level optimization that does not explicitly model grammar, dependencies, or geometric feasibility.

Abstract

from arXiv · show

Reconstructing editable Computer-Aided Design (CAD) models from images is essential for downstream modification, manufacturing, and design reuse. However, existing image-to-CAD methods are developed predominantly on synthetic renderings and face two coupled obstacles: a substantial appearance domain gap between synthetic and real images, and a previously overlooked parameter bias in widely used CAD data. We show that the local normalization adopted by DeepCAD concentrates several geometric parameters around a few discrete values while encoding substantial information in a single scale factor. Consequently, a model can achieve deceptively high parameter accuracy by exploiting these frequent values rather than inferring geometry from the input image. In this paper, we propose RealCAD, a unified framework that addresses these limitations at the representation, image, and feature levels. At the representation level, we redistribute scale information to the corresponding geometric parameters, producing less concentrated parameter distributions in a shared scale space. At the image level, geometry-constrained translation converts synthetic renderings toward the real-image domain while conditioning on object contours. At the feature level, a multi-positive contrastive objective aligns representations of the same CAD model across viewpoints and image domains, enabling CAD sequence prediction from each individual view. We further introduce OpenRealCAD, comprising four-view photographs of 392 3D-printed objects paired with ground-truth command sequences. Experiments show that the revised representation substantially reduces the accuracy attainable from parameter-frequency priors, making parameter accuracy a more reliable measure of image-conditioned geometric inference. RealCAD further improves real-domain command and parameter accuracy, while retaining competitive synthetic-domain performance.

Introduction

Practical image-to-CAD reconstruction is hindered by both synthetic-to-real appearance differences and bias in the DeepCAD parameterization. RealCAD addresses these issues through revised parameters, a real-image benchmark, geometry-constrained translation, and feature alignment.

  • Motivation: Images are a practical input modality for CAD reconstruction because they are easier to acquire than dense, relatively complete point-cloud observations.Text often lacks fine-grained geometry and spatial relationships, whereas point-cloud methods typically require substantial 3D observations.
  • Challenges: Synthetic-to-real domain shift substantially degrades image-to-CAD performance on real photographs, while systematic adaptation remains underexplored.Existing methods are commonly trained on synthetic renderings, and CADCrafter lacks a publicly available real-image evaluation set and explicit Sim2Real adaptation mechanism.
  • Challenges: DeepCAD’s local normalization concentrates geometric parameters by assigning substantial scale information to a separate factor S, encouraging statistical shortcuts.This imbalance can let models exploit frequent parameter values instead of inferring geometry from visual evidence.
  • Contributions: RealCAD redistributes DeepCAD’s scale information to corresponding geometric parameters to reduce parameter concentration and frequency-based shortcut learning.The revised representation is designed to address the representation-induced source of unreliable parameter inference.
  • Contributions: OpenRealCAD provides four-view photographs of 392 3D-printed CAD objects paired with ground-truth command sequences for reproducible Sim2Real evaluation.The benchmark complements abundant synthetic data with limited real images and annotations.
  • Contributions: RealCAD combines geometry-constrained synthetic-to-real translation with multi-positive contrastive learning to reduce discrepancies at image and feature levels.The image-level component adapts appearance, while the feature-level component aligns representations across viewpoints and image domains.

Related Work

Prior image-to-CAD work builds on parametric CAD sequences but has largely focused on synthetic data and has paid limited attention to DeepCAD’s parameter statistics. Related approaches address real-image generalization through invariant features or intermediate representations.

  • Image-to-CAD reconstruction: Image-based 3D reconstruction methods recover geometry or scene representations from images, but they primarily represent geometry or appearance rather than editable CAD construction histories.Representations include point clouds, meshes, voxels, neural fields, and 3D Gaussians.
  • Parametric CAD reconstruction: DeepCAD represents CAD models as sketch and extrusion command sequences and supplies a large-scale dataset paired with construction histories.Subsequent methods use this representation to reconstruct CAD sequences from point clouds, text, and images.
  • Parameterization: The statistical properties and learning effects of the widely used DeepCAD parameterization have received limited attention.RealCAD identifies a representation-induced bias associated with DeepCAD’s local normalization.
  • Sim2Real generalization: Existing image-to-CAD methods typically train on synthetic renderings because paired real images and CAD command sequences are costly to collect.The synthetic-to-real appearance gap includes differences in materials, textures, illumination, and imaging conditions.
  • Sim2Real generalization: CADCrafter extracts texture-invariant geometric features, while Img2CADSeq introduces an intermediate point-cloud representation and contrastive alignment for synthetic-to-real generalization.These approaches provide alternatives to directly adapting synthetic image appearance.

Benchmark Construction

The paper constructs a revised CAD command-sequence representation and the controlled OpenRealCAD benchmark to study image-to-CAD reconstruction under Sim2Real conditions. The revision reduces parameter concentration by redistributing scale information, while the benchmark pairs four-view photographs of 392 printed models with ground-truth sequences.

  • Preliminaries: CAD Command Sequence Encoding: CAD models are encoded as ordered sequences of sketch, extrusion, and Boolean-operation commands with associated parameters.The neural representation uses 17-dimensional command vectors, including special start-of-loop and end-of-sequence tokens.
  • Investigating Parameter Bias in DeepCAD: 61.62% of x and y observations fall on values 128, 176, and 223, while radius values 47 and 48 account for 55.86% of samples.The concentration is strongest for sketch coordinates and radii; orthogonal rotation frequencies largely reflect the underlying discrete orientations.
  • Investigating Parameter Bias in DeepCAD: DeepCAD’s local normalization concentrates geometric parameters because relative size information is transferred from coordinates and radii to the unit-level scale factor s.Geometrically distinct endpoints can therefore receive identical quantized coordinates, while s carries substantial dimensional information.
  • Investigating Parameter Bias in DeepCAD: Frequency shortcuts can inflate parameter accuracy, and uniform averaging can understate errors in s even though s controls an entire sketch–extrusion unit.Thus, similar parameter accuracies do not necessarily imply comparable geometric fidelity.
  • Investigating Parameter Bias in DeepCAD: The revised representation absorbs s into affected geometric parameters in a shared scale space and removes the explicit scale parameter while preserving relative dimensions and spatial relationships.After reparameterization, no individual coordinate value exceeds 3.5% of its distribution and the maximum radius frequency is 14.6%.
  • Details of OpenRealCAD: OpenRealCAD re-encodes 392 DeepCAD models, fabricates them by 3D printing, and captures each from four predefined viewpoints with paired command sequences.The benchmark uses controlled backgrounds and illumination, separates adaptation and evaluation subsets at the CAD-instance level, and excludes synthetic renderings of real-test models from synthetic training.

Method

RealCAD combines geometry-preserving image translation, cross-view and cross-domain feature alignment, and Transformer-based CAD sequence decoding to support reconstruction from individual images.

  • Image-level adaptation: Geometry-constrained synthetic-to-real translation reduces appearance differences while preserving object structure through edge-conditioned ControlNet.The edge condition preserves contours while adapting materials, textures, illumination, and shadows.
  • Feature learning: Multi-view synthetic and real images of each CAD model are encoded with frozen DINOv3 features and projected into a shared latent space.The projected representations support both contrastive alignment and Transformer decoding.
  • Feature learning: Samples from the same CAD model across viewpoints and domains serve as multiple positives, while images from different models serve as negatives.The objective promotes cross-view and cross-domain consistency while separating different CAD models.
  • Feature learning: The similarity logit uses l2-normalized representations with learnable log-scale and bias parameters, with the scale constrained positive.The parameterization defines similarity as s_ij = α z_i^T z_j + b with α = exp(t).
  • CAD decoding: A Transformer decoder independently reconstructs the complete CAD command sequence from each latent representation.It predicts command types and discretized parameters, optimized with weighted command and parameter classification losses alongside contrastive learning.

Experiments

The experiments evaluate RealCAD across synthetic and real domains, test whether revised parameterization reduces frequency-based shortcuts, and analyze the effects and limitations of its adaptation modules.

  • Experimental Setup: The evaluation uses synthetic renderings, OpenRealCAD images, shared protocols, and metrics for command accuracy, parameter accuracy, geometric discrepancy, and invalid predictions.OpenRealCAD contains multi-view photographs of 3D-printed CAD models; synthetic data use fixed viewpoints and instance-level leakage prevention.
  • Representation Refinement: The frequency-only baseline attains nontrivial parameter accuracy under the original DeepCAD representation, especially for Line, Arc, and Circle parameters.This demonstrates that parameter accuracy can partly arise without visual evidence.
  • Representation Refinement: Revised representation priors decrease for Line, Circle, and Translation from 16.77%, 26.13%, and 7.15% to 3.14%, 6.41%, and 1.64%, respectively.Rotation and Extrusion remain nearly unchanged because scale redistribution does not directly affect their distributions.
  • Main Results: RealCAD improves real-domain command and parameter accuracy over CADCrafter by 7.49 and 14.73 percentage points, respectively.The result supports combining geometry-constrained image translation with feature-level multi-positive contrastive learning.
  • Limitations: Token-level gains do not translate into comparable mCD improvements and coincide with a higher invalid ratio, because geometry quality depends on sensitive parameters, dependencies, and executability.The complete model also modestly decreases synthetic-domain command accuracy while maintaining comparable parameter accuracy.
  • Ablation Studies: Combining ST and CL yields real-domain command and parameter accuracies of 64.85% and 52.03%, improving over baseline by 4.67 and 2.05 percentage points.ST narrows the appearance gap, while CL promotes cross-view and cross-domain feature consistency.

Conclusion

The paper concludes that revised parameterization reduces frequency-prior dependence and RealCAD improves real-image command and parameter accuracy, while geometric quality and validity gains remain limited.

  • Conclusion: The revised representation reduces geometric parameter concentration and frequency-based shortcuts, while RealCAD improves command and parameter accuracy on real images.RealCAD combines geometry-constrained image translation with multi-positive contrastive learning for cross-domain and cross-view consistency.
  • Conclusion: Limited gains in geometric quality and program validity indicate that token-level optimization alone is insufficient.The conclusion motivates geometry- and validity-aware learning.

Supplementary Material

The supplementary material analyzes DeepCAD parameter-distribution bias and presents qualitative reconstructions for synthetic and real-world image inputs.

  • Supplementary Material: The supplement provides detailed statistics on DeepCAD parameter distributions and qualitative reconstruction results under synthetic and real-world images.It complements the main paper's analysis and qualitative evaluation.

Detailed Analysis of the DeepCAD Dataset

The DeepCAD representation exhibits strong, uneven biases across coordinate, radius, rotation, and translation parameters. These concentrations can enable frequency-based shortcuts instead of image-grounded geometric inference, motivating revised parameterization.

  • Distributional Overview: DeepCAD parameter distributions are substantially biased across coordinates, geometric sizes, rotations, and translations.The reported analysis covers multiple parameter families rather than a single isolated bias.
  • Coordinate Parameters: 32.27% of x occurrences and 43.77% of y occurrences use the coordinate value 128.Values 176 and 223 also occur frequently, indicating nonuniform spatial coverage.
  • Radius Parameter: 44.25% of radius samples use r=48, while r=47 contributes 11.62%.Together, these two values account for more than half of the observed radius distribution and may limit generalization to uncommon sizes.
  • Rotation Parameters: 97.36% of θ occurrences use 128 or 192, while the corresponding concentration for γ is 97.57%.For ϕ, values 64 and 128 account for 39.60% and 58.18%, respectively, reflecting limited orientation diversity.
  • Translation Parameters: 51.87% of py samples and 56.24% of pz samples use 128, whereas px has 67.41% in the “Others” category.Vertical and depth translations are therefore more concentrated than horizontal translations.
  • Implications: These imbalances may let models exploit frequent parameter patterns instead of learning generalizable geometric reasoning from images.The analysis identifies addressing parameter-distribution bias as important for robustness and generalization.

Qualitative Results

RealCAD reconstructs synthetic examples accurately and shows stronger dimensional accuracy and robustness than existing approaches on real-world images, especially for complex geometry.

  • Synthetic Inputs: RealCAD successfully reconstructs all provided synthetic examples with accurate geometric structures.DeepCAD and Img2CAD achieve comparable quality on synthetic inputs, while CadCrafter sometimes predicts inaccurate dimensions and spatial locations.
  • Real-World Inputs: On real-world images, RealCAD predicts more accurate geometric dimensions than DeepCAD, Img2CAD, and CadCrafter.The comparison concerns qualitative reconstruction results under captured-image conditions.
  • Complex Objects: RealCAD is the only compared method that accurately reconstructs the triangular prism example.The passage presents this as evidence of stronger robustness for complex structures and realistic variations.
Loading 2608.30617v1…