Source-linked AI summary
Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect
Hamed Sarbolandi, Damien Lefloch, Andreas Kolb
TL;DR
The paper addresses the need to understand how structured-light and Time-of-Flight Kinect cameras differ in range-data behavior. It compares them using seven experiments designed to isolate sensor effects, reporting several device-specific results while noting that distance errors may remain unflagged.
Problem
Structured-light and Time-of-Flight Kinect cameras use different principles and are subject to varied error sources, motivating an application-relevant comparison.
Method
The paper compares KinectSL and KinectToF through seven generic experimental setups designed to isolate sensor effects and transfer to other range cameras.
Results
The study presents an in-depth comparison of the two Kinect versions, including reported RMSE, linearity-error, and distance-dependent findings.
Takeaways & Limitations
The framework is intended to help researchers assess the specific benefits and potential problems of KinectSL and KinectToF for their application scenarios.
Takeaways & Limitations
Distance-error assessment is constrained because the device may not indicate a pixel’s invalidity even when its distance error is large.
Abstract
from arXiv · showhide
Recently, the new Kinect One has been issued by Microsoft, providing the next generation of real-time range sensing devices based on the Time-of-Flight (ToF) principle. As the first Kinect version was using a structured light approach, one would expect various differences in the characteristics of the range data delivered by both devices. This paper presents a detailed and in-depth comparison between both devices. In order to conduct the comparison, we propose a framework of seven different experimental setups, which is a generic basis for evaluating range cameras such as Kinect. The experiments have been designed with the goal to capture individual effects of the Kinect devices as isolatedly as possible and in a way, that they can also be adopted, in order to apply them to any other range sensing device. The overall goal of this paper is to provide a solid insight into the pros and cons of either device. Thus, scientists that are interested in using Kinect range sensing cameras in their specific application scenario can directly assess the expected, specific benefits and potential problem of either device.
1. Introduction and Related Works
This paper compares the structured-light and Time-of-Flight Kinect cameras through generic experiments designed to isolate sensor effects and clarify application-relevant advantages and drawbacks.
- Motivation: Microsoft’s second-generation KinectToF uses Time-of-Flight, whereas the original Kinect uses structured light, so their range data are subject to different characteristics and error sources.The paper denotes the devices KinectToF and KinectSL.
- Motivation: The study evaluates both Kinect cameras to extract pros and cons relevant to applications incorporating range sensing.It explicitly avoids evaluating the devices only within a fixed set of application scenarios.
- Contribution: Seven experimental setups provide a generic basis for evaluating Kinect and other range cameras.The setups are intended to support application development and comparison beyond these two devices.
- Related Work: Prior studies compared depth precision, sensor characteristics, material classes, or particular ToF and stereo systems, but this work emphasizes a broader set of characteristic sensor effects.Earlier evaluations included comparisons with laser range sensing, linearity and lateral resolution, and material-specific measurements.
- Contribution: The paper aims to identify conditions under which one Kinect version is superior to the other.Its stated motivation is to help researchers assess device-specific benefits and potential problems for their applications.
- Method: The experiments isolate individual sensor effects as far as possible and are designed to be reproducible for other range sensing cameras.This design addresses prior concerns that combined scenes may not separate error sources or transfer well across application scenarios.
2. Devices Principle
KinectSL estimates depth from structured-light pattern disparity, whereas KinectToF estimates distance from phase shifts in modulated NIR light. Their sensing geometries and processing pipelines differ accordingly.
- Structured Light: Structured-light cameras project known patterns and infer depth from their deformation observed by a camera.KinectSL uses an NIR projector and monochrome camera; its fixed dot pattern is matched against the stored projection pattern.
- Structured Light: KinectSL depth is computed from camera focal length, projector-camera baseline, and image disparity.The paper gives d = b·f/m(x,y), with disparity measured in pixel units.
- Structured Light: KinectSL uses a baseline of approximately 7.5cm between its NIR projector and NIR camera.Longer baselines support robust depth measurements at long distances.
- Time of Flight: Time-of-Flight cameras illuminate scenes with periodic, intensity-modulated NIR light and convert measured phase shifts into sensor-object distance.The phase shift results from light’s travel time between the sensor and object and is detected through pixel-wise mixing and correlation.
- Time of Flight: ToF measurements have a unique unambiguous range because the reference signal is periodic, although multiple modulation frequencies can extend the measurement range.KinectToF can obtain depth values beyond 9 meters, but its official driver masks distances beyond approximately 4.5 meters.
- Time of Flight: The KinectToF uses continuous-wave intensity modulation and reconstructs phase from nine of ten correlation images at three modulation frequencies.The nine images use phase shifts of 0°, 120°, and 240°; the purpose of the tenth image remains unclear.
3. General Considerations for Comparing KinectSL and KinectToF
The comparison uses official camera drivers as black boxes because hardware, raw-data processing, and post-processing effects cannot be cleanly separated. The KinectSL’s quantization complicates statistical analysis, while KinectToF filtering affects dynamic-scene artifacts.
- Comparison basis: Official drivers are retained because reversing or neutralizing their processing does not produce fair, equivalent data quality.Processing occurs at raw-data and range-data levels, and reverse-engineered processing usually fails to reproduce the same quality.
- Comparison basis: KinectSL disparity is normalized and quantized to 11-bit values, producing quantized ranges that can compromise or invalidate statistical analysis.The disparity-to-depth conversion requires a more complex mapping function because zero disparity does not represent infinite distance.
- Noise statistics: Per-pixel temporal noise statistics cannot be derived reliably for KinectSL from individual static-scene measurements because of strong quantization.The paper therefore compares temporal statistics primarily for KinectToF.
- Experimental comparison: The experiments compare official and alternative processing paths for planar-wall and rotating-Siemens-star scenes across 200 frames.The tested options include official drivers, KinectSL post-processing, KinectToF OpenKinect processing, and raw KinectToF data.
- Experimental comparison: KinectSL post-processing does not improve quality, whereas KinectToF filtering removes artifacts at the Siemens star’s jumping edges.For KinectToF, disabling internal filters mainly harms the rotating-star result; OpenKinect quality also remains behind the official driver.
4. Experimental Results And Comparison
The paper evaluates KinectSL and KinectToF through reproducible scenarios designed to isolate individual range-sensing effects. It combines calibration, pixel-statistics analysis, controlled scenes, and a summary of comparative strengths and limitations.
- Framework: The framework is designed to isolate individual effects and be reproducible for other range-sensing systems operating over similar depths.The tests primarily address one or two separable effects and cover 500mm to 3000mm.
- Framework: Both devices are evaluated with their respective Microsoft drivers, using a KinectSL near mode and a KinectToF Developer Preview prototype.All evaluations were performed in Matlab.
- Interpretation: The black-box comparison cannot attribute observed effects exclusively to sensing principles because integrated camera post-processing also contributes.Consequently, KinectSL and KinectToF data are not fully comparable across devices.
- Camera parameters and noise: Calibration uses standard techniques with a planar checkerboard, while KinectToF temporal statistics use 5000 frames from center, intermediate, and corner pixels.Gaussian and Poisson distributions are fitted using nonlinear least-squares optimization.
- Camera parameters and noise: Corner pixels have higher variance because illumination amplitude is reduced in corner regions, while Gaussian and Poisson fits achieve the same fitting quality.The fitting parameters and corresponding RMSE are reported for the three KinectToF pixel locations.
4.2. Ambient Background Light
The ambient-light experiment measures how incident background radiance affects depth measurements and whether that effect varies across image regions. Radiance is calibrated with filtered optical measurements and controlled illumination.
- Purpose: The experiment relates incident background radiance to delivered depth range for center, intermediate, and corner image regions.It also examines spatial variation caused by nonuniform active illumination.
- Measurement setup: Radiance is measured directionally with a powermeter calibrated to 850nm, matching the Kinect laser diode wavelength.A laser pointer aligns the powermeter with selected wall points, while halogen lamps control illumination.
- Measurement setup: The powermeter and astronomy-camera measurements show a proper linear relation for radiance verification.The astronomy camera measures radiant emittance in relation to photons received per pixel.
- Radiance calibration: Daylight reference measurements yield 11mW/cm2 radiant flux density and an estimated sun-reference incident radiance of about 10µW/cm2.The estimate is derived by relating direct-sunlight and diffuse-paper measurements.
- Measurement setup: The setup acquires 200 frames under light conditions up to 20µW/cm2 at three positions along the image half-diagonal.Measurements are made with the Kinect camera turned off to prevent interference from its illumination.
Evaluation and Results.
Ambient light affects the two cameras differently: KinectSL preserves depth variation while valid data remain available, whereas KinectToF becomes noisier but continues returning range data across the tested light range. KinectToF’s corner performance degrades most strongly.
- Ambient background light: KinectSL cannot handle background light beyond 1µW, while KinectToF delivers range data throughout the experiment’s full ambient-light range.This creates a trade-off between KinectSL’s lower variation within its valid range and KinectToF’s broader operating range.
- KinectSL: KinectSL depth variation is hardly affected by ambient light while valid data are delivered, although variation increases toward image-vicinity pixels.Observed pixels remain below 6mm SD, with a maximum median deviation of 25mm for the corner pixel.
- KinectToF: KinectToF depth variation rises with ambient light because of reduced SNR, with center and intermediate SD below 4mm below 6µW.Its boxplots show more outliers than KinectSL, and corner statistics are worse than KinectSL’s.
- KinectToF: Beyond 10µW ambient light, KinectToF variation reaches about 22mm, 12mm, and 42mm for center, intermediate, and corner pixels.The center becoming worse than the intermediate pixel may result from oversaturation caused by active illumination.
- Multi-device interference: The multi-device experiment investigates active-illumination interference by running multiple same-type Kinect cameras in parallel.It measures range effects and the temporal and spatial distribution of resulting artifacts.
Evaluation and Results.
The interference tests show stronger typical interference for KinectToF, but KinectSL also exhibits occasional prominent errors and persistent invalid pixels. KinectToF usually avoids invalid measurements outside interference, yet produces severe periodic interference episodes.
- Multi-device interference: KinectToF has stronger interference than KinectSL for most interfering-camera poses.KinectSL averages RMSE <5.6mm, aside from prominent poses with RMSE up to <9.4mm.
- Multi-device interference: KinectToF shows RMSE <5mm in most frames but reaches 19.3mm in roughly 25% of frames during periodic interference.The repetition is attributed most likely to asynchronous device operation and signal drift.
- Multi-device interference: KinectSL’s interference median remains stable, whereas KinectToF’s median drifts by up to 5mm and varies more.The KinectSL result is linked to strong quantization in its disparity maps.
- Multi-device interference: KinectSL has about 1.5% invalid pixels initially and up to 16.3% after pose changes; KinectToF reaches up to 22.7% during interference.KinectToF has no invalid pixels during non-interfered periods.
- Warm-up: KinectSL shows strong variation during its first 400 frames, with RMSE up to 6.7mm without interference and 7.7mm with interference.The paper advises excluding this initial 13-second sequence.
Evaluation and Results.
Warm-up, geometric accuracy, and planarity tests reveal complementary strengths: KinectToF generally has lower distance error and smoother planes, while KinectSL can have lower stabilized variation but suffers quantization and peripheral-distance errors.
- Device warm-up: KinectToF generally has lower warm-up error, with nearly constant SDA and RMSE over time.KinectSL has strong error and variation fluctuations during its first 10 minutes.
- Device warm-up: After stabilization, KinectSL’s distance SDA stays within 1mm versus 1.5mm for KinectToF, but its RMSE remains higher: <7.1mm versus <5.3mm.KinectSL box plots show less variation, partly because of heavy quantization.
- Device warm-up: At minute 60, KinectToF produces smoother depth with fewer out-of-plane errors than KinectSL.Both devices are treated as temperature-stable at this time.
- Linearity and planarity: KinectToF is more precise than KinectSL when the corner pixel is excluded, while the corner pixel delivers invalid data.KinectSL’s increasing-distance fluctuations are linked to decreasing depth resolution and quantization.
- Linearity and planarity: KinectToF’s planarity standard deviation stays below 1.65mm across the 4m range, whereas KinectSL has much stronger out-of-plane errors.The comparison uses standard deviation within the evaluated region of interest.
4.6. Semitransparent Liquid
The semitransparent-liquid experiment evaluates how translucent and scattering materials affect acquired range quality.
- Semitransparent liquids: The test examines range-data quality for translucent, semitransparent, and scattering materials.It is designed to isolate effects caused by material transparency and scattering.
Goal.
The liquid experiments vary milk concentration and compare depth error with invalid-pixel rates. KinectSL performs accurately above 3.12% milk but fails for thinner samples, while KinectToF retains valid pixels at the cost of large positive errors.
- Experimental design: The setup uses diluted-milk cylinders to test semitransparent liquids across concentrations from 100% to 0.19%.Each setup is acquired for 200 frames from a frontal view.
- KinectSL: KinectSL performs well above 3.12% milk, with almost no invalid pixels and signed error between [1, 1.5]mm.The error is approximately the thickness of the plastic cylinder.
- KinectSL: Below 3.12% milk, KinectSL’s invalid-pixel rate rises above 90%.The paper characterizes this behavior as failure for thinner samples.
- KinectToF: KinectToF reports positive distance errors from 12 to 378mm while marking almost no measurements invalid.Thus, valid-pixel status does not indicate accurate depth for these samples.
- KinectToF: For 0.2% milk, KinectToF still returns valid pixels but its error rises to 400mm.For samples thinner than 3.1%, KinectSL exceeds 90% invalid pixels while KinectToF continues delivering valid pixels.
- Comparison: KinectSL indicates failure for thinner liquids, whereas KinectToF is harder to interpret because it does not flag invalid pixels despite large distance errors.This is the principal practical distinction reported for the liquid test.
4.7. Reflective Board
The reflective-board test examines depth errors and invalid pixels caused by reflective surfaces, especially under multi-path illumination and varying incidence angles. KinectSL is generally more robust, while KinectToF shows larger low-angle errors and invalid regions.
- Setup: The test varies a reflective board’s incidence angle and compares measured depth against a non-multi-path reference.The board rotates from 0° to 90° in 0°15′ steps, with 20 acquisitions per step.
- Metrics: The evaluation measures RMSE, standard deviation, and the number of invalid pixels as functions of incidence angle.A vertical 4 × 100 pixel region around the rotation pivot is used under a constant multi-path situation.
- Interpretation: Structured light is less affected by indirect illumination because diffuse scattered light does not confuse its measurement principle.Nevertheless, KinectSL also exhibits limitations at low incidence angles.
- KinectSL results: For incidence angles below 10°, up to 100% of KinectSL pixels are marked invalid.Above 15°, KinectSL has nearly no invalid pixels and depth error close to zero.
Experimental setup.
The Siemens-Star experiment evaluates flying pixels in static and dynamic scenes by comparing foreground, background, and invalid-pixel classifications across motion speeds and edge orientations. KinectToF is more stable during motion, while both devices perform worse near vertical edges.
- Evaluation: The analysis samples an outer circular arc and computes minimum, mean, and maximum relative foreground, background, and invalid-pixel counts.The two wheel quarters, S0 and S1, are evaluated separately.
- Static scene: For static S0, foreground classification is 52.5% for KinectSL and 53.2% for KinectToF, while invalid pixels are 7.0% and 10.9%, respectively.These values describe the static flying-pixel condition.
- Dynamic scene: With increasing speed, KinectSL produces fewer invalid pixels and more false foreground pixels, reaching 100% foreground pixels at 100 RPM.KinectToF instead shows increasing invalid pixels and comparable decreases in foreground and background pixels.
- Edge orientation and artifacts: Results are worse for both devices in top segment S1 because shadowing is stronger at vertical edges, although KinectToF shows smaller segment differences.Range profiles also show edge overshooting attributed to motion artifacts in ToF cameras.
5. Conclusion
The paper compares structured-light KinectSL with Time-of-Flight KinectToF using a generic seven-setup evaluation framework covering known range-camera artifacts. It also provides application-oriented failure-ratio guidelines, while cautioning that device choice remains application dependent.
- Contribution: The study presents an in-depth comparison of KinectSL and KinectToF and a framework applicable to Structured Light and Time-of-Flight cameras.The framework contains seven experimental setups covering the full range of known artifacts for these range cameras.
- Evaluation scope: The evaluation covers ambient background light, multi-device interference, temperature drift, linearity, planarity, intensity-related error, and semitransparent media or scattering.These error sources are summarized across the paper’s experiments and characteristics tables.
- Device selection: The paper reports application-specific rules of thumb based on failure ratios for selecting between KinectSL and KinectToF.Users choose relevant error sources, combine their failure ratios, and use the resulting ratio as an initial suggestion.
- Decision guideline: If the final failure ratio is below 1, KinectSL is suggested; otherwise, KinectToF is preferable.The authors describe this as a coarse, quick guideline and recommend examining the most relevant error sources in detail.
- Reproducibility: The evaluation material is made publicly available through documented Matlab scripts and technical setup resources.Listed resources include a Siemens-star CAD file, intensity materials, and a calibration checkerboard.