Source-linked AI summary
A Large Dataset of Object Scans
Sungjoon Choi, Qian-Yi Zhou, Stephen Miller, Vladlen Koltun
TL;DR
Object reconstruction lacks large-scale data captured with consumer-grade mobile systems in uncontrolled, non-expert settings. This paper builds and releases a public dataset of over 10,000 such scans, along with reconstructed models, using a hybrid geometric-photometric odometry pipeline. The dataset has 44 categories with at least 44 scans each, averages 77 seconds per scan, and includes over 23 million RGB-D images, while the reconstruction pipeline is limited for large objects and handheld scans.
Problem
Object reconstruction needs data reflecting consumer-grade cameras used outside controlled laboratories by users without extensive training or expertise.
Method
The paper constructs a public dataset from mobile RGB-D scans and releases reconstructed models produced with a hybrid ICP and RGB-D odometry pipeline.
Results
The dataset has 44 categories with at least 44 scans each, averages 77 seconds per scan, and contains over 23 million RGB-D images.
Takeaways & Limitations
The released collection provides large-scale object-scan data gathered in conditions intended to resemble broadly deployed reconstruction systems.
Takeaways & Limitations
The reconstruction pipeline does not handle loop closure, is unsuitable for large objects, and was not used for vehicles, sculptures, or handheld scans.
Abstract
from arXiv · showhide
We have created a dataset of more than ten thousand 3D scans of real objects. To create the dataset, we recruited 70 operators, equipped them with consumer-grade mobile 3D scanning setups, and paid them to scan objects in their environments. The operators scanned objects of their choosing, outside the laboratory and without direct supervision by computer vision professionals. The result is a large and diverse collection of object scans: from shoes, mugs, and toys to grand pianos, construction vehicles, and large outdoor sculptures. We worked with an attorney to ensure that data acquisition did not violate privacy constraints. The acquired data was irrevocably placed in the public domain and is available freely at http://redwood-data.org/3dscan .
1 Introduction and Related Work
The dataset contains over 10,000 object scans collected “in the wild” by non-experts using consumer-grade mobile scanning setups. It is substantially larger than prior collections and targets conditions expected in broadly deployed reconstruction systems.
- Over 10,000 dedicated 3D scans were collected from individual objects in deployment-like conditions by non-expert operators.The operators worked outside controlled laboratory settings without direct familiarity with computer vision research.
- The collection reflects mobile consumer-grade scanning by users who may move the camera or object freely during acquisition.These conditions are intended to resemble broad deployment of object reconstruction systems.
- The dataset is one to two orders of magnitude larger than prior collections and provides data likely to occur in broadly deployed reconstruction systems.
2 Data Acquisition
The dataset was acquired with ten lightweight mobile RGB-D scanning setups operated by 70 recruited users after brief instruction. Operators chose objects largely themselves, used handheld or camera-around-object scanning, and submitted screened videos under public-domain and privacy procedures.
- Ten lightweight setups combined a netbook, RGB-D camera, and carrying case into fully mobile scanning units.The components were consumer-oriented and the camera was powered from the netbook’s USB port.
- The custom application provided a live color feed, start/stop recording controls, and a red warning for regions beyond 2 meters.
- Seventy operators were recruited, given a five-minute tutorial and brief in-person instruction, and asked to record one video per object.The project was conducted with IRB approval.
- Operators were paid per recorded minute, while submitted videos were screened to remove partial, repeated, or inadequate scans.
- Object selection was largely left to operators, subject to camera-related exclusions such as very small, unstable, transparent, or highly reflective objects.
- Handheld objects were rotated in front of the camera, whereas larger or heavier objects were scanned by moving the camera around them.Telescoping monopods extended camera reach for tall objects such as cargo vans and outdoor sculptures.
- An attorney prepared privacy instructions and agreements, and all acquired data was assigned irrevocably to the public domain.Operators knew the data would be made public for research and development.
3 Dataset Composition
The dataset spans semantically diverse and shape-based categories, with substantial concentration in vehicles, furniture, and 44 well-populated categories. Scans average 77 seconds and total over 23 million RGB-D images.
- Categories include semantically distinct objects and shape-based groupings because the dataset is not intended to evaluate recognition algorithms.Personal grooming combines shampoo, soap, deodorant, and similar objects, while box and bottle are shape-based categories.
- 44 categories contain at least 44 scans each, defining the dataset’s h-index as 44.
- Vehicles account for about 13% of the dataset, while chairs, tables, and benches together account for another 10%.
- The average scan lasts 77 seconds, yielding over 23 million RGB-D images in total.
- The collection includes mundane objects such as fire hydrants and parking meters alongside rare musical instruments, museum artifacts, sculptures, and a Howitzer.The cited examples include 52 fire-hydrant scans and 16 parking-meter scans.
- Figure 3 displays scan counts for the largest categories, distinguishing stationary scans from handheld scans by color.
4 Reconstructed Models
The paper releases reconstructed 3D models alongside raw RGB-D scans, using hybrid geometric-photometric odometry to improve robustness. Reconstruction was limited by pipeline failures and scope constraints, yielding 398 inspected models from 1,781 processed sequences, plus three high-fidelity large-object reconstructions.
- Release: The paper releases reconstructed 3D models alongside the raw RGB-D scans.The reconstructions were produced by a pipeline performing camera odometry and volumetric integration.
- Odometry: The released reconstruction pipeline combines ICP odometry with RGB-D odometry to use geometric and photometric cues.The hybrid approach combines frame-to-model registration with photometric stability.
- Odometry: The hybrid odometry estimates each frame transformation by minimizing a weighted combination of point-to-plane and photometric errors.EICP uses corresponding point pairs and surface normals, while ERGBD measures photometric error between consecutive frames; λ balances the terms empirically.
- Scope: The standard pipeline excludes vehicles and sculptures because it lacks loop closure and is unsuitable for large objects, and excludes handheld scans because tested pipelines performed unsatisfactorily.These limitations define the scope of the released standard reconstructions.
- Results: 398 models passed qualitative inspection and were released from 1,781 processed sequences across nine object categories.The pipeline lost track or failed on 969 sequences, often due to fast camera motion or odometry drift; 812 other models were inspected before low-quality results were removed.
- Scope: Three selected large objects were reconstructed with a high-fidelity pipeline using loop closure detection and global optimization.That pipeline was not applied to all stationary scans because of its high computational requirements.