Source-linked AI summary
HI3D 3.0 (Twinkle3D): Object-specific 3D Asset Generation with High Resolution
Ziying Li, Shengchu Zhao, Huiang He, Yiyang Chen, Jianwen Huang, Bailin Li, Changhao Li, Jianhui Li, Jie Li, Ruiyang Liu, Yibo Luo, Tengjiao Sun, Pei Tang, Shiwen Wang, Jiaqi Wu, Kang Wu, Kaiqiao Yang, Zherui Yang, Hu Zhang, Xuezhi Zhao, Xinhe Zheng, Yukun Li, Heliang Zheng, Rongfei Jia
TL;DR
Image-to-3D systems can match overall shape yet lose identity-defining details that silhouette- and appearance-based evaluation may not capture. Hi3D 3.0 uses Twinkle3D to generate high-resolution watertight meshes and evaluates fidelity with geometry-focused measures; it outperforms four commercial systems across the reported metrics and recovers more inscribed characters than the strongest competitor.
Problem
Current systems often distort or lose identity-defining details, while silhouette-based measures are largely insensitive to object-specific fidelity.
Source: §1 · p. 2Method
Twinkle3D generates watertight triangle meshes at 2048^3 resolution and strengthens global shape and local detail during initial generation.
Source: §1 · pp. 2–3Results
Hi3D 3.0 outperforms four commercial systems across all four metrics, recovering 82.1% of inscribed characters at 98.2% precision versus 21.7% recall for the strongest competitor.
Source: §1 · p. 3Takeaways & Limitations
The report frames object-specific fidelity as a distinct quality axis and introduces geometry-domain evaluation with an OCR-scored inscription benchmark.
Source: §1 · p. 3A 16.5% miss rate leaves roughly one character in six below legibility, and seven located characters were misread.
Source: §3.8 · p. 12
Abstract
from arXiv · showhide
Image-to-3D generation has become increasingly capable of producing objects that closely resemble the input image, and an outstanding challenge is to reproduce the depicted object itself, including the specific geometry that defines it. Inscriptions, brand marks, and repeated structures are frequently distorted or lost, despite being critical to object identity. We present Hi3D 3.0, an image-to-3D generation system targeting object-specific fidelity, with Twinkle3D as its geometry model for generating watertight triangle meshes at $2048^{3}$ resolution. Twinkle3D advances high-fidelity geometry generation along four dimensions. First, while O-Voxel/FaithC offers high representational precision, it often suffers from poor surface quality and non-watertight geometry. We address both issues while retaining its $2048^{3}$-level precision. Second, we scale diffusion generation to sequences of up to 300K geometric tokens through a redesigned DiT architecture and large-scale distributed training optimizations, reducing training time per step from approximately ten minutes to ten seconds. Third, subsequent refinement cannot fully compensate for errors introduced during initial generation; we therefore strengthen both global shape and local detail in the initial generation stage, and the resulting single-stage model surpasses prior two-stage pipelines with $512^{3}$ refinement. Finally, we introduce a fine-grained image-3D cross-modal interaction mechanism that strengthens correspondence between visual evidence and geometric tokens, improving the recovery of object-specific structures. We evaluate geometric fidelity using alignment metrics derived from silhouettes and normal fields. Hi3D 3.0 outperforms four commercial systems across all reported metrics, recovering 82.1% of inscribed characters at 98.2% precision, compared with 21.7% recall for the strongest competitor.
1 Introduction
Image-to-3D systems can match an input’s overall shape while losing identity-defining details; Hi3D 3.0 introduces Twinkle3D to target object-specific fidelity. The system combines high-resolution watertight mesh generation with geometry-focused evaluation and outperforms four commercial systems on the reported metrics.
1 Introduction
- Twinkle3D, the geometry model of Hi3D 3.0, generates watertight triangle meshes at 2048^3 resolution.The report presents it as the third generation of the image-to-3D series, after Hi3D 1.0 and Hi3D 2.1.Source: §1 · p. 2
- The representation retains O-Voxel/FaithC’s 2048^3-level precision while improving surface quality and ensuring watertight extraction without mesh repair.O-Voxel/FaithC can represent several surface sheets per cell but often produces poor-quality, non-watertight geometry.Source: §1 · p. 3
- A redesigned DiT and distributed-training optimizations handle up to 300K geometric tokens, reducing training-step time at thousand-GPU scale from about ten minutes to ten seconds.The report says this roughly 60× reduction shortens a 105-step schedule from close to two years to days.Source: §1 · p. 3
- Strengthening global shape and local detail in the initial stage lets the single-stage model exceed Sparc3D and TRELLIS 2.0 with 512^3 refinement.Later refinement adds detail relative to the first-stage surface but does not revise its placement, so initial displacement and merged neighboring surfaces can persist.Source: §1 · p. 3
- Fine-grained image–3D interaction strengthens correspondence between local visual evidence and geometric tokens, improving recovery of inscriptions, marks, and repeated structures.These object-specific features occupy little image area and are represented by few image tokens.Source: §1 · p. 3
- Hi3D 3.0 outperforms four commercial systems on all four metrics, recovering 82.1% of inscribed characters at 98.2% precision versus 21.7% recall for the strongest competitor.In blind pairwise comparisons, it is rated equal to or better than competitors in 78.0–89.8% of structure judgements and 88.1–95.5% of geometric-detail judgements.Source: §1 · p. 3
- The report frames object-specific fidelity as a distinct quality axis and introduces geometry-domain evaluation alongside an OCR-scored inscription benchmark.Its protocol uses silhouette and normal-field alignment for geometry, reserving vision-language scoring for multi-view plausibility and shape aesthetics.Source: §1 · p. 3
2 Related Work
Related work spans image-to-3D systems, native 3D shape generation, and direct mesh-generation methods. Twinkle3D is positioned as dense, high-resolution surface generation focused on reproducing the input object, rather than controlling topology or face budgets.
Native Mesh Generation
- Native mesh methods generate vertices and faces autoregressively or generate faces in parallel with discrete diffusion, continuous connectivity representations, diffusion, or flow matching.The cited line of work also explores larger models, conditioning, structured tokenizations, level-of-detail generation, and other mesh formats.Source: §2 · p. 3
3D native shape generation
- Twinkle3D generates a dense surface at 2048^3 and extracts a watertight mesh, targeting whether the surface reproduces the input object rather than artist-like topology or face-budget control.Its related-work positioning also highlights cells crossed by multiple surface sheets and watertight extraction without repair.Source: §2 · p. 4
- Image-to-3D methods progress from lifted 2D priors and feed-forward reconstruction to native 3D latent models and sparse voxel structures.Examples include sparse hierarchies, structured coarse-to-fine latents, sparse attention, and local-block modeling at 2048^3 resolution.Source: §2 · p. 4
3 Experiments
The experiments compare five systems on production-request images and use metrics that separate projected occupancy, fine geometric detail, unseen-side completeness, and appearance aesthetics. Geometry metrics avoid texture, material, and lighting effects, while the detail measure has pose-fit and monocular-reference constraints.
3.1 Setup
- The benchmark contains 33 single-image production requests and compares Hi3D 3.0, Hi3D 2.1, Meshy 7, Tripo 3.1, and Rodin Gen 2.5 at each system’s highest available geometry resolution.Source: §3.1 · p. 4
- Silhouette IoU measures projected occupancy after fitting each system’s camera pose against the input foreground mask; all systems share 35° perspective intrinsics.Separate pose fitting avoids penalizing scale or orientation convention differences.Source: §3.1 · p. 4
- Detail alignment compares rendered mesh normals with a monocular normal estimate using multi-scale bandpasses that penalize both excess and missing detail.It is computed only for high-confidence pose fits, inherits estimator biases, and ranks systems against a shared reference rather than measuring absolute geometric error.Source: §3.1 · p. 4
- Multi-view plausibility uses a vision-language judge to assess whether back views are completed, while shape aesthetics scores front and back renders with a three-level rubric.The completeness score is a presence-or-absence judgement; the aesthetics metric is appearance-dependent by design.Source: §3.1 · pp. 4–5
3.2 Main Results
On the 33-input benchmark, Hi3D 3.0 leads all four reported metrics, while detail alignment separates systems more than silhouette IoU. Completeness varies independently of alignment, and the aesthetics rubric provides little separation among most systems.
3.2 Main Results
- Hi3D 3.0 has the best value on all four metrics; detail alignment spans 0.265 to 0.378, while silhouette IoU spans 0.879 to 0.918.Table 1 reports the main results; the detail metric is computed on the high-confidence-pose subset.Source: Table 1 · p. 5, §3.2 · p. 5
- Multi-view plausibility varies independently of alignment: Hi3D 3.0 and Meshy 7 differ by 0.119 on completeness despite ranking among the top two on detail alignment.Completeness remains below its ceiling for every system, including Hi3D 3.0 at 0.757.Source: §3.2 · p. 5
- Four of five systems fall within one point on shape aesthetics, which the authors attribute to the three-level rubric saturating once outputs reliably reach its middle level.Detail alignment and completeness show the widest spreads among the metrics measuring geometry beyond projected occupancy.Source: §3.2 · p. 5
3.3 Inscription Fidelity
The section measures whether generated geometry preserves readable inscriptions, a discrete component of object-specific fidelity. Hi3D 3.0 recovers substantially more ground-truth characters than the strongest peer while maintaining high precision.
3.3 Inscription Fidelity
- Character recovery is a discrete fidelity measure: an inscription either reads back or it does not, regardless of rendering style.The benchmark covers Chinese and English inscriptions, engravings, and brand marks, aligned character by character against 526 ground-truth characters.Source: §3.3 · p. 5
- 82.1% of 526 ground-truth characters are recovered by Hi3D 3.0 at 98.2% precision, versus 21.7% recall for the best peer, Meshy 7.The benchmark uses PaddleOCR on clay-shaded renderings against shared scene-level ground truth.Source: §3.3 · p. 5, Table 2 · p. 6
- Normal-shaded examples show reproduced pump specification plates and medal rim lettering as geometry, independent of texture or material.The system comparisons also show Hi3D 3.0 reproducing the mask-cabinet name “Shadow,” while competing outputs include faint marks, blank plates, and pseudo-characters.Source: Figure 2 · p. 6, §3.3 · p. 6
- 98.2% of everything PaddleOCR reads from Hi3D 3.0 is present, versus 62.0% for Meshy 7 and 9.8% for Tripo 3.1.Omitting illegible text preserves precision but loses recall; invented text-like relief can lower both, and Hi3D 3.0 produces one invented character across the benchmark.Source: §3.3 · p. 6
3.4 Human Evaluation
In a blind pairwise study, Hi3D 3.0 was rated the same or better more often for geometric detail than for overall structure. Qualitative inscription comparisons show sharper, more legible strokes in its outputs than in peer results.
3.4 Human Evaluation
- Raters compared Hi3D 3.0 with each of four peer systems, judging overall structure and geometric detail quality separately.Each comparison received one of three verdicts: Hi3D 3.0 better, the same, or inferior.Source: §3.4 · p. 6
- In the inscription examples, Hi3D 3.0 forms sharp, deeply cut characters, while Meshy 7’s strokes are shallower and Tripo 3.1’s are pseudo-characters.Two characters outside the enlarged regions in Hi3D 3.0’s result are malformed.Source: Figure 3 · p. 7
- 88.1–95.5% of detail judgements rated Hi3D 3.0 the same or better, compared with 78.0–89.8% of structure judgements.The detail margin exceeded the structure margin against every system, by 5.7 to 13.1 percentage points.Source: §3.4 · p. 6, Table 3 · p. 7
3.5 Autoencoder Reconstruction
The autoencoder comparison tests whether each representation preserves identity-bearing fine structure. Twinkle3D keeps the grille periodic and in phase, while the baselines lose or alter details.
3.5 Autoencoder Reconstruction
- Twinkle3D preserves the grille’s periodic spacing and phase across its full width; Sparc3D loses periodicity and TRELLIS 2.0 misses spacing or thickness.The comparison passes the same input mesh through all three autoencoders.Source: §3.5 · p. 7
- Sparc3D tapers the arrow’s repeated barbs into a cone, whereas TRELLIS 2.0 and Twinkle3D keep them separate.The barbs form a repeated profile along one shaft.Source: §3.5 · p. 7
3.6 One Stage vs. Two Stages
This experiment compares Twinkle3D’s standalone first stage with two-stage baselines. Across three inputs, the single-stage model preserves distinct structures that the refined baselines fuse.
3.6 One Stage vs. Two Stages
- Twinkle3D runs one stage, while both baselines use a first stage followed by refinement.The comparison tests the standalone first stage against the two-stage pipelines.Source: §3.6 · p. 8
- On all three inputs, Twinkle3D’s single stage recovers structures that Sparc3D and TRELLIS 2.0 do not preserve after 512^3 refinement.The examples include separate robe perforations, blades, scrollwork, cords, tassels, saddle, and harness.Source: §3.6 · p. 8
- When nearby boundaries are displaced by amounts comparable to their gap, refinement adds detail without correcting placement, and the gap closes.The resulting fusion explains why the refined baselines lose distinct structures in these examples.Source: §3.6 · p. 8
3.7 Qualitative Results
The qualitative comparisons probe object completeness, thin structures, relief, inscriptions, and identifying details. Across the examples, systems that resemble the same object differ in whether they preserve its specific geometry.
3.7 Qualitative Results
- Across five inputs, the systems differ in scene completeness, thin structures, relief, and facial detail; Rodin Gen 2.5 omits the stag’s surroundings, a failure not captured by the detail metric.The medal combines lettering, mid-scale relief, and fine rigging, so performance can differ across scales within one result.Source: §3.7 · p. 8, Figure 6 · p. 10
- In Figure 5’s three-input comparison, the single-stage result keeps structures distinct where both two-stage baselines fuse them.Examples include separate structures across the robe, cords, tassels, saddle, and harness.Source: Figure 5 · p. 9
- In the medal-and-lighthouse example, Meshy 7 and Hi3D 3.0 form both rim inscriptions and a windowed lighthouse, while the other systems miss or distort parts.The figure also compares foliage, medallions, hair, beard strands, collars, and buttons across five systems.Source: Figure 6 · p. 10
- On the winged figure and qilin, Hi3D 3.0 separates feathers and carves facial and crown edges, and forms nested scroll relief and a spiral volute.The comparison also shows smooth feather lobes, radial grooves, or missing scrollwork in other systems.Source: Figure 7 · p. 11
- On the mask cabinet and fox statue, Hi3D 3.0 alone reproduces the engraved “Jade” word and forms a raised diamond with a recessed centre for the fox’s forehead mark.Other systems produce unreadable strokes or omit these marks.Source: Figure 8 · p. 11
- In close-ups of four objects, systems agree on overall shape but differ on identifying details such as feathers, scrollwork, engraved lettering, and forehead marks.Figures 7 and 8 use the same camera-path frame across systems and enlarge two regions per example.Source: Figure 7 · p. 11, Figure 8 · p. 11, §3.7 · p. 12
- Additional Hi3D 3.0 examples span six categories, including multi-object scenes, high-relief vessels, thin structures, and reflective mechanical objects.Each example pairs an input image with clay-shaded views from complementary viewpoints.Source: §3.7 · p. 12, Figure 9 · p. 12
3.8 Summary
Hi3D 3.0 leads the reported automatic metrics and inscription benchmark rates, with human comparisons rated the same or better in most judgments. The results also identify remaining inscription errors and limits on within-group rankings.
3.8 Summary
- Hi3D 3.0 obtains the best value on all four automatic metrics and all five inscription-benchmark rate columns.It is rated the same or better in the majority of judgments in every pairwise human comparison.Source: §3.8 · p. 12
- At 0.757 multi-view plausibility, close to a quarter of back-view completions fail the criterion requiring only surface completion.The criterion concerns whether the surface was completed.Source: §3.8 · p. 12
- A 16.5% miss rate leaves roughly one character in six below legibility, and seven located characters were misread.The model omits rather than invents, but inscription capability is not saturated.Source: §3.8 · p. 12
- The 33-input detail-alignment comparison supports separation between groups by at least 0.087, but not rankings within a group.The 0.0088 gap between Hi3D 3.0 and Meshy 7 on detail alignment is within the range per-input variance can produce on this set.Source: §3.8 · p. 12