Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items

Motivation

• Existing web-scale image datasets are difficult to label reliably and do not readily generalize to 3D or real-world robotic data.Real-world 3D collection is challenging, and human labelers cannot extract 3D geometric properties from images.Source: §I · p. 1
• Simulation provides safe variation and easily derived semantic labels, but handcrafted models and fixed scanned scenes poorly represent responsive real objects.The paper identifies a need for object models that can be incorporated into physical and visual simulations.Source: §I · p. 1

Contribution

• Over 1000 curated 3D-scanned household items are released for Ignition Gazebo, Bullet, and other tools reading SDF models.The dataset is accompanied by object-collection, scanning, quality-assurance, and publication pipelines.Source: §I · p. 1
• The paper contributes the dataset, its 3D scanning pipeline, its scan curation and publication process, and a review of research impact.Source: §I · p. 1

Simulation use

• Simulation research uses both CAD and scanned datasets, while earlier simulators and newer interactive-navigation benchmarks use injected 3D objects.Interactive Gibson is identified as a benchmark using GSO.Source: §II · pp. 1–2

Construction approaches

• Simulation objects have been created through manual modeling, scanning, CAD conversion, and combinations of these techniques.Neural rendering methods have also shown promise for 3D object scanning.Source: §II · p. 2

Dataset landscape

• 3D object datasets serve shape retrieval, recognition, grasping, manipulation, prosthetics, and rehabilitation, with simulation requiring available watertight meshes and high visual fidelity.Source: §II · p. 2

Alternative sources

• Manufacturers provide models that may include articulated geometry, mechanical properties, or rich textures, while ModelBank offers 1200 licensable models.Source: §II · p. 2

Simulation datasets

• Comparable scanned datasets include YCB with 77 objects, BigBird with 125, and KIT with 145, but these generally omit object bases.Source: §II · pp. 2–3

Dataset landscape

• Computer-vision datasets vary widely, from 800-plus scanned B3DO models and 1500-plus scanned ScanNet scenes to 25K CAD models in PartNet.Other examples include pose-aligned ObjectNet3D shapes and BigBIRD’s calibrated RGB-D object views.Source: §II · p. 3

Origins

• GSO began as Google’s 2011 Cloud Robotics effort to support recognition and grasping with high-fidelity models of common household objects.The project later expanded to physical simulation and 3D visualization uses.Source: §III · p. 3

Pipeline

• The end-to-end effort covered object acquisition, custom scanning hardware, efficient scanning software, QA rendering, viewers, interaction studies, and multi-format publication.Source: §III · p. 3

Scanning

• Calibration patterns aligned cameras, while projected patterns enabled sub-pixel localization and extraction of complete 3D object shape.Source: Figure 3 · p. 3

Quality assurance

• Quality assurance distinguished high-quality closed manifold meshes from deformations sometimes produced by optically uncooperative objects.Source: Figure 4 · p. 3

System design

• Dedicated hardware was designed for scalable scanning because existing systems were labor-intensive and not cost-effective.The target challenge included lighting, camera reliability, color matching, texture rendering, and optically difficult materials.Source: A. 3D Scanning Pipeline · p. 3
• The enclosure uses two machine-vision cameras and a projector for structured-light geometry, plus a DSLR for high-resolution textures.The system was designed to scan objects in 10 minutes with 2M-triangle models and eventually captured over 400 scans per week.Source: A. 3D Scanning Pipeline · p. 3

Calibration and reconstruction

• The software extends camera calibration with a projector generating over 200 patterns, combining gray codes, phase stripes, HDR frames, and aligned DSLR textures.Source: A. 3D Scanning Pipeline · p. 4
• Gray codes provide globally robust projector correspondence, while phase patterns provide sub-pixel line localization but require gray codes for uniqueness.Source: A. 3D Scanning Pipeline · p. 4
• Sub-pixel projector coordinates and camera rays yield 3D points, depth maps, and raw meshes from valid quad-pixel neighborhoods.Source: A. 3D Scanning Pipeline · p. 4

Alignment and QA

• A turntable generates closely aligned views, and parallel ICP registration optimizes 100K to 400K constraints per iteration in less than a minute.Source: A. 3D Scanning Pipeline · p. 4
• Nearly complete raw scans are produced at hardware scanning speed, allowing later processing to focus on concave interiors and human quality assurance.Source: A. 3D Scanning Pipeline · p. 4

• Source models used very high-resolution visuals and formats unsuitable for simulation, with friction absent from metadata.Source: B. Simulation Model Conversion · p. 4
• The conversion pipeline filters invalid, duplicate, test, calibration, and hand-modeled objects before manual quality assurance.Source: B. Simulation Model Conversion · p. 4
• Simulation-ready meshes must be closed manifolds with interiors, exteriors, no holes or self-intersections, and non-zero thickness.Source: B. Simulation Model Conversion · p. 4
• The pipeline calculates physical properties because the visualization-focused source process omitted properties such as density and friction.Source: B. Simulation Model Conversion · p. 4
• Meshes are simplified with edge-collapse algorithms and quadratic error metrics to reach reasonable rendering resolutions.Source: B. Simulation Model Conversion · p. 5

• Ignition Fuel scales model hosting beyond code repositories, which are cumbersome for large binary datasets and model discoverability.Source: C. Ignition Fuel · p. 5
• Each model has version control on a scalable server, while a web application supports upload, download, visualization, and search.Source: C. Ignition Fuel · p. 5
• GSO metadata adds SKUs, manufacturer numbers, brands, compatibility, licensing, tags, versions, annotations, and categories.Source: C. Ignition Fuel · p. 5
• Ignition Fuel Tools supports bulk uploads and downloads through collections that group models such as the GSO dataset.Source: C. Ignition Fuel · p. 5
• Calibrated projected patterns allow GSO scans to capture fine surface details and complex topology.Source: Figure 5 · p. 5

• 1030 scanned objects and associated metadata comprise the dataset, totaling 13Gb under the CC-BY 4.0 License.Source: A. Dataset Composition · p. 5
• Visual meshes use Wavefront OBJ format and average 1.4MB per model, while diffuse PNG textures average 11.2MB per texture.Source: A. Dataset Composition · p. 5
• Table IV.1 lists projects using Google Scanned Objects.Source: Table IV · p. 6

• The automated pipeline generates large numbers of realistic scanned models without manual processing, reducing idealization relative to hand-modeled objects.Source: B. Strengths of the Dataset · p. 6
• The glass platform scans objects from all sides, including bases that opaque-platform scanners and environment-extracted models generally miss.Source: B. Strengths of the Dataset · p. 6
• Projected-pattern reconstruction produces high-fidelity meshes with smooth surfaces and accurate silhouette edges compared with RGB-D-derived meshes.Source: B. Strengths of the Dataset · p. 6

Dataset limitations

• Objects larger than roughly 50 cm, very small objects, and highly specular or transparent objects are not represented faithfully.The scanner is limited to breadbox-sized objects, scan resolution limits small-object fidelity, and generated textures are diffuse.Source: C. Limitations of the Dataset · p. 6

Research usage

• A dozen papers across ten projects used GSO for computer vision, graphics, manipulation, navigation, and 3D shape processing.Hosting on Ignition Fuel made the dataset discoverable, downloadable, and usable inside and outside the robotics community.Source: §V · p. 6
• Controllable synthetic data from GSO either improved overall system performance or increased simulated-environment visual realism.The reported usage spans synthetic data generation and scene population for vision, rendering, navigation, and benchmarking.Source: §V · p. 6
• Three projects used GSO to generate synthetic data for robotic grasping and manipulation, with some showing gains from augmenting real data.These pipelines either relied on GSO to create scenes or improved results by adding synthetic data to real data.Source: §V · p. 6
• Visual realism and randomized visual and physical properties facilitated sim-to-real transfer learning for real-world tasks.This usage relied on an internal version with cleaned-up transparency and specular materials.Source: §V · p. 6
• Because GSO contains scans of real objects, two projects used it as a validation set.Differentiable Stereopsis used 50 categories in one evaluation, alongside two other evaluations using different datasets.Source: §V · p. 6

• Google Scanned Objects is a diverse collection of high-quality 3D household-object scans released in Ignition Gazebo format.Source: §VI · p. 6
• The objects had already proved useful in ten robotics and simulation projects.Source: §VI · p. 6