Source-linked AI summary
Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B. McHugh, Vincent Vanhoucke
TL;DR
High-quality, diverse 3D object models are needed for realistic and varied robotic simulation, but existing image datasets and real-world collection do not readily provide them. The paper presents Google Scanned Objects, a curated scanning and curation pipeline producing simulation-ready household-object models. The resulting dataset contains 1030 scanned objects and has already been used in ten robotics and simulation projects.
Problem
Deep learning needs diverse 3D inputs, but image-based datasets and real-world capture do not readily provide labeled, high-quality 3D object data for simulation.
Method
The paper constructs and publishes a curated dataset using custom 3D scanning, mesh reconstruction, quality assurance, and simulation-model conversion pipelines.
Results
1030 scanned objects totaling 13Gb were released under CC-BY 4.0, and the dataset has already proven useful in ten robotics and simulation projects.
Takeaways & Limitations
GSO provides an openly released collection of high-quality household-object scans for simulation, synthetic perception, and robotic learning research.
Takeaways & Limitations
The dataset excludes objects larger than roughly 50 cm and cannot faithfully represent very small, highly specular, or transparent objects.
Abstract
from arXiv · showhide
Interactive 3D simulations have enabled breakthroughs in robotics and computer vision, but simulating the broad diversity of environments needed for deep learning requires large corpora of photo-realistic 3D object models. To address this need, we present Google Scanned Objects, an open-source collection of over one thousand 3D-scanned household items released under a Creative Commons license; these models are preprocessed for use in Ignition Gazebo and the Bullet simulation platforms, but are easily adaptable to other simulators. We describe our object scanning and curation pipeline, then provide statistics about the contents of the dataset and its usage. We hope that the diversity, quality, and flexibility of Google Scanned Objects will lead to advances in interactive simulation, synthetic perception, and robotic learning.
I. INTRODUCTION
Deep learning and robotics need diverse 3D inputs, but existing image-based collection and real-world capture do not readily provide labeled, high-quality 3D data. Google Scanned Objects addresses this need with a curated collection of over 1000 household-item scans and supporting scanning and curation processes.
- Motivation: Existing web-scale image datasets are difficult to label reliably and do not readily generalize to 3D or real-world robotic data.Real-world 3D collection is challenging, and human labelers cannot extract 3D geometric properties from images.
- Motivation: Simulation provides safe variation and easily derived semantic labels, but handcrafted models and fixed scanned scenes poorly represent responsive real objects.The paper identifies a need for object models that can be incorporated into physical and visual simulations.
- Contribution: Over 1000 curated 3D-scanned household items are released for Ignition Gazebo, Bullet, and other tools reading SDF models.The dataset is accompanied by object-collection, scanning, quality-assurance, and publication pipelines.
- Contribution: The paper contributes the dataset, its 3D scanning pipeline, its scan curation and publication process, and a review of research impact.
II. RELATED WORK
Prior 3D datasets support vision, recognition, grasping, and simulation, spanning scanned, CAD, and hybrid collections. GSO is positioned among simulation-suitable datasets emphasizing freely available, watertight, textured models with high visual fidelity.
- Simulation use: Simulation research uses both CAD and scanned datasets, while earlier simulators and newer interactive-navigation benchmarks use injected 3D objects.Interactive Gibson is identified as a benchmark using GSO.
- Construction approaches: Simulation objects have been created through manual modeling, scanning, CAD conversion, and combinations of these techniques.Neural rendering methods have also shown promise for 3D object scanning.
- Dataset landscape: 3D object datasets serve shape retrieval, recognition, grasping, manipulation, prosthetics, and rehabilitation, with simulation requiring available watertight meshes and high visual fidelity.
- Alternative sources: Manufacturers provide models that may include articulated geometry, mechanical properties, or rich textures, while ModelBank offers 1200 licensable models.
- Simulation datasets: Comparable scanned datasets include YCB with 77 objects, BigBird with 125, and KIT with 145, but these generally omit object bases.
- Dataset landscape: Computer-vision datasets vary widely, from 800-plus scanned B3DO models and 1500-plus scanned ScanNet scenes to 25K CAD models in PartNet.Other examples include pose-aligned ObjectNet3D shapes and BigBIRD’s calibrated RGB-D object views.
III. CREATION OF THE DATASET
GSO grew from a robotics effort into an end-to-end program for collecting, scanning, processing, and publishing household-object models. Its creation combined dedicated hardware, software, calibration, quality assurance, and user-facing data-access pipelines.
- Origins: GSO began as Google’s 2011 Cloud Robotics effort to support recognition and grasping with high-fidelity models of common household objects.The project later expanded to physical simulation and 3D visualization uses.
- Pipeline: The end-to-end effort covered object acquisition, custom scanning hardware, efficient scanning software, QA rendering, viewers, interaction studies, and multi-format publication.
- Scanning: Calibration patterns aligned cameras, while projected patterns enabled sub-pixel localization and extraction of complete 3D object shape.
- Quality assurance: Quality assurance distinguished high-quality closed manifold meshes from deformations sometimes produced by optically uncooperative objects.
A. 3D Scanning Pipeline
The scanning pipeline treats 3D capture as calibrated structured-light reconstruction, producing aligned meshes through projector patterns, stereo geometry, turntable motion, and parallel registration. This hardware-first design leaves later processing to address concavities and human QA.
- System design: Dedicated hardware was designed for scalable scanning because existing systems were labor-intensive and not cost-effective.The target challenge included lighting, camera reliability, color matching, texture rendering, and optically difficult materials.
- System design: The enclosure uses two machine-vision cameras and a projector for structured-light geometry, plus a DSLR for high-resolution textures.The system was designed to scan objects in 10 minutes with 2M-triangle models and eventually captured over 400 scans per week.
- Calibration and reconstruction: The software extends camera calibration with a projector generating over 200 patterns, combining gray codes, phase stripes, HDR frames, and aligned DSLR textures.
- Calibration and reconstruction: Gray codes provide globally robust projector correspondence, while phase patterns provide sub-pixel line localization but require gray codes for uniqueness.
- Calibration and reconstruction: Sub-pixel projector coordinates and camera rays yield 3D points, depth maps, and raw meshes from valid quad-pixel neighborhoods.
- Alignment and QA: A turntable generates closely aligned views, and parallel ICP registration optimizes 100K to 400K constraints per iteration in less than a minute.
- Alignment and QA: Nearly complete raw scans are produced at hardware scanning speed, allowing later processing to focus on concave interiors and human quality assurance.
B. Simulation Model Conversion
GSO converts scanned assets into simulation-ready models through filtering, naming, mesh validation, physical-property calculation, and model simplification. The pipeline addresses source formats and metadata that were unsuitable for simulation.
- Source models used very high-resolution visuals and formats unsuitable for simulation, with friction absent from metadata.
- The conversion pipeline filters invalid, duplicate, test, calibration, and hand-modeled objects before manual quality assurance.
- Simulation-ready meshes must be closed manifolds with interiors, exteriors, no holes or self-intersections, and non-zero thickness.
- The pipeline calculates physical properties because the visualization-focused source process omitted properties such as density and friction.
- Meshes are simplified with edge-collapse algorithms and quadratic error metrics to reach reasonable rendering resolutions.
C. Ignition Fuel
GSO uses Ignition Fuel to host and distribute many simulation models with scalable storage, search, richer metadata, and bulk collection transfers. The platform combines version control with web-based discovery and dataset management.
- Ignition Fuel scales model hosting beyond code repositories, which are cumbersome for large binary datasets and model discoverability.
- Each model has version control on a scalable server, while a web application supports upload, download, visualization, and search.
- GSO metadata adds SKUs, manufacturer numbers, brands, compatibility, licensing, tags, versions, annotations, and categories.
- Ignition Fuel Tools supports bulk uploads and downloads through collections that group models such as the GSO dataset.
- Calibrated projected patterns allow GSO scans to capture fine surface details and complex topology.
A. Dataset Composition
GSO contains over one thousand curated scanned household objects with associated metadata and licensing, alongside model and texture assets. The paper also reports dataset usage through projects using Google Scanned Objects.
- 1030 scanned objects and associated metadata comprise the dataset, totaling 13Gb under the CC-BY 4.0 License.
- Visual meshes use Wavefront OBJ format and average 1.4MB per model, while diffuse PNG textures average 11.2MB per texture.
- Table IV.1 lists projects using Google Scanned Objects.
B. Strengths of the Dataset
The dataset combines scalable automated production with realistic, high-fidelity scans that include otherwise occluded object bases. Its scanner also preserves smooth surfaces, accurate silhouettes, fine details, and complex topology.
- The automated pipeline generates large numbers of realistic scanned models without manual processing, reducing idealization relative to hand-modeled objects.
- The glass platform scans objects from all sides, including bases that opaque-platform scanners and environment-extracted models generally miss.
- Projected-pattern reconstruction produces high-fidelity meshes with smooth surfaces and accurate silhouette edges compared with RGB-D-derived meshes.
C. Limitations of the Dataset
The dataset has physical and material scope limits, but its scanned objects have supported diverse simulation, synthetic-data, manipulation, transfer-learning, and validation uses.
- Dataset limitations: Objects larger than roughly 50 cm, very small objects, and highly specular or transparent objects are not represented faithfully.The scanner is limited to breadbox-sized objects, scan resolution limits small-object fidelity, and generated textures are diffuse.
- Research usage: A dozen papers across ten projects used GSO for computer vision, graphics, manipulation, navigation, and 3D shape processing.Hosting on Ignition Fuel made the dataset discoverable, downloadable, and usable inside and outside the robotics community.
- Research usage: Controllable synthetic data from GSO either improved overall system performance or increased simulated-environment visual realism.The reported usage spans synthetic data generation and scene population for vision, rendering, navigation, and benchmarking.
- Research usage: Three projects used GSO to generate synthetic data for robotic grasping and manipulation, with some showing gains from augmenting real data.These pipelines either relied on GSO to create scenes or improved results by adding synthetic data to real data.
- Research usage: Visual realism and randomized visual and physical properties facilitated sim-to-real transfer learning for real-world tasks.This usage relied on an internal version with cleaned-up transparency and specular materials.
- Research usage: Because GSO contains scans of real objects, two projects used it as a validation set.Differentiable Stereopsis used 50 categories in one evaluation, alongside two other evaluations using different datasets.
VI. CONCLUSIONS
Google Scanned Objects is a diverse collection of high-quality household-object scans released in Ignition Gazebo format. The dataset had already proved useful across ten robotics and simulation projects.
- Google Scanned Objects is a diverse collection of high-quality 3D household-object scans released in Ignition Gazebo format.
- The objects had already proved useful in ten robotics and simulation projects.