PaperlayerView exact revision on arXiv

Source-linked AI summary

MilliSonic: Pushing the Limits of Acoustic Motion Tracking

Anran Wang, Shyamnath Gollakota

arXiv:1901.06601v1cs.HC

TL;DR

Existing acoustic tracking is described as lacking millimeter accuracy and requiring large microphone/speaker separation, while concurrent tracking can reduce frame rate. MilliSonic uses FMCW phase to disambiguate multipath for 1D localization and time-shifted FMCW transmissions for concurrent tracking. MilliSonic reports median 1D accuracy of 0.7 mm up to 1 m, 1.7 mm between 1 and 2 m, and median 3D accuracy around 2.6 mm.

  • Problem

    Existing acoustic tracking is described as lacking millimeter accuracy and requiring large microphone/speaker separation, while concurrent tracking can reduce frame rate.

    Source: §1 · p. 1

  • Method

    MilliSonic uses FMCW phase to disambiguate multipath for 1D localization and time-shifted FMCW transmissions for concurrent tracking.

    Source: §1 · pp. 1–2

  • Results

    MilliSonic reports median 1D accuracy of 0.7 mm up to 1 m, 1.7 mm between 1 and 2 m, and median 3D accuracy around 2.6 mm.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Sub-millimeter 1D tracking under multipath is achieved with a single beacon and a 4-microphone array.

    Source: §1 · pp. 1–2

    The authors call for more evaluation of edge cases and rooms with significant multipath, and report support only for simple occlusions such as fabric and paper.

    Source: §8 · p. 9

Abstract

from arXiv · show

Recent years have seen interest in device tracking and localization using acoustic signals. State-of-the-art acoustic motion tracking systems however do not achieve millimeter accuracy and require large separation between microphones and speakers, and as a result, do not meet the requirements for many VR/AR applications. Further, tracking multiple concurrent acoustic transmissions from VR devices today requires sacrificing accuracy or frame rate. We present MilliSonic, a novel system that pushes the limits of acoustic based motion tracking. Our core contribution is a novel localization algorithm that can provably achieve sub-millimeter 1D tracking accuracy in the presence of multipath, while using only a single beacon with a small 4-microphone array.Further, MilliSonic enables concurrent tracking of up to four smartphones without reducing frame rate or accuracy. Our evaluation shows that MilliSonic achieves 0.7mm median 1D accuracy and a 2.6mm median 3D accuracy for smartphones, which is 5x more accurate than state-of-the-art systems. MilliSonic enables two previously infeasible interaction applications: a) 3D tracking of VR headsets using the smartphone as a beacon and b) fine-grained 3D tracking for the Google Cardboard VR system using a small microphone array.

1 INTRODUCTION

Existing acoustic tracking falls short of VR/AR needs in accuracy, microphone/speaker spacing, and concurrent tracking. MilliSonic uses FMCW phase and time-shifted transmissions to pursue accurate tracking with a compact array, and reports sub-millimeter 1D results.

1 INTRODUCTION

  • Existing acoustic systems are described as inaccurate under multipath, reliant on widely separated microphones and speakers, and limited in concurrent tracking.Source: §1 · p. 1
  • Its phase-based FMCW algorithm targets sub-millimeter 1D tracking under multipath using one beacon with a 4-microphone array.Source: §1 · pp. 1–2
  • The scenarios include tracking Google Cardboard with a small array and tracking VR/AR headsets using a smartphone as a beacon.Source: §1 · p. 1, Figure 1 · p. 2
  • Time-shifted FMCW transmissions let multiple devices transmit concurrently without reducing per-device accuracy or frame rate.Source: §1 · p. 2
  • 0.7 mm median 1D accuracy up to 1 m, 1.7 mm from 1–2 m, and 2.6 mm median 3D accuracy are reported.Source: §1 · p. 2
  • MilliSonic tracks up to four smartphones at 40 frames/sec per device without sacrificing accuracy.Source: §1 · p. 2

2 APPLICATION SCENARIOS

MilliSonic is presented for smartphone-based VR tracking, portable beacon use, and concurrent device tracking. The design overview identifies phase-based FMCW tracking and subsequent 3D localization as its core stages.

2 APPLICATION SCENARIOS

  • A small microphone array enables 6DoF tracking for smartphone-based VR headsets without modifying the smartphone.Source: §2 · p. 2
  • A smartphone can serve as a portable VR beacon, while a headset microphone array can track acoustic-enabled devices such as smartwatches.Source: §2 · p. 2
  • One microphone array can track up to four nearby speakers without sacrificing accuracy or frame rate; a single nearby speaker can serve unlimited microphone arrays.Source: §2 · p. 2

3 MILLISONIC DESIGN

  • The design overview proceeds from FMCW background to phase-based sub-millimeter 1D tracking, then 3D tracking from multiple microphones and practical issues.Source: §3 · pp. 2–3

FMCW Background

FMCW tracking uses changes in frequency to separate acoustic paths, but limited smartphone bandwidth constrains resolution for closely spaced paths. The described processing also limits frame rate to the chirp duration.

FMCW Background

  • FMCW chirps change frequency linearly over time and distinguish reflections with different arrival times through their frequency shifts.Source: FMCW Background · p. 3
  • A receiver uses a DFT to find the first peak frequency bin associated with the line-of-sight path and derives distance from that peak.Source: FMCW Background · p. 3
  • With smartphone bandwidth limited to 7 kHz between 17–24 kHz, conventional FMCW has limited resolution for nearby paths and chirp-duration processing limits frame rate.Source: FMCW Background · p. 3

Sub-mm 1D tracking using FMCW phase

MilliSonic estimates distance from FMCW phase after filtering most distant multipath, rather than relying on the first frequency peak. Its phase error is less sensitive to indirect-path separation, and the method’s distance error is reported as around 10× lower than peak estimation.

Our method

  • MilliSonic filters out most multipath with distant time-of-flight, then extracts distance from instantaneous FMCW phase.The remaining indirect paths are near the direct path.Source: Sub-mm 1D tracking using FMCW phase · p. 3

Traditional FMCW approaches

  • When direct and indirect peaks merge because their frequencies differ by less than one bin, peak estimation shifts toward their amplitude-weighted frequency.The resulting distance error increases with frequency separation and with the indirect-to-direct amplitude ratio.Source: Sub-mm 1D tracking using FMCW phase · p. 3

Our method

  • With residual indirect-path amplitude below the direct-path amplitude, phase error is bounded by sin^-1(A2/A1) and does not depend on frequency separation.The error bound is described for the phase of the combined direct and indirect signal.Source: Sub-mm 1D tracking using FMCW phase · p. 3

Intuition

  • Around 10× lower distance error is reported for FMCW phase than peak estimation, whose errors are severely affected by indirect-path time-of-flight.Figure 4 compares distance error as a function of indirect-path amplitude ratio and frequency separation.Source: Sub-mm 1D tracking using FMCW phase · pp. 3–4, Figure 4 · p. 4

3D tracking from 1D locations

MilliSonic triangulates 3D location from distance measurements by four microphones, using the high 1D resolution to keep the array compact. Practical tracking addresses phase ambiguity and clock differences, and detects and recovers from failures caused by occlusion or noise.

3D tracking from 1D locations

  • Four microphones arranged in vertical and horizontal pairs triangulate 3D location from 1D measurements, reducing power and avoiding multiple-speaker multiplexing.The method relies on high 1D resolution to permit smaller microphone separation.Source: 3D tracking from 1D locations · p. 4
  • At larger distances accuracy worsens, while greater microphone separation improves triangulation; averaging each 10 ms yields 40 frames per second after filter latency.The system is described as supporting good 3D tracking up to 2 m with compact microphone separation.Source: 3D tracking from 1D locations · p. 4

Addressing practical issues

  • The initial 2Nπ phase offset is inferred from the previous chirp’s ending distance and speed; the acceleration constraint is 25 m/s^2.Speed is estimated by least-squares regression over distance measurements in a 10 ms window; the bound is based on a 5 ms chirp gap and 15 ms filter delay.Source: Addressing practical issues · p. 4
  • Clock calibration uses a five-second recording with the phone speaker touching a microphone to estimate the initial phase offset, then removes measured linear drift.The setup uses autocorrelation to determine chirp start time and peak-FFT distance at zero range to estimate the phase offset.Source: Addressing practical issues · pp. 4–5
  • Occlusion and noise can cause tracking failures; four-microphone outliers exceeding 2 cm trigger correction, with peak estimation as fallback for sustained failures.The method assumes the direct path dominates the filtered signal; some occlusions create dense, attenuated multipath that violates this premise.Source: Addressing practical issues · p. 5

4 TRACKING MULTIPLE DEVICES

MilliSonic supports concurrent tracking by separating transmissions through their time-of-arrival offsets. A single smartphone speaker can serve multiple microphone arrays, while a single array can track multiple smartphone speakers without sacrificing accuracy or frame rate.

4 TRACKING MULTIPLE DEVICES

  • One smartphone speaker can support an unlimited number of nearby microphone arrays because the acoustic tracking algorithm is unidirectional.The arrays can be integrated into multiple VR headsets.Source: §4 · p. 5
  • Time-division multiplexing lowers refresh rate as smartphone count grows, while limited inaudible bandwidth makes frequency division challenging.The passage notes that accuracy depends on bandwidth.Source: §4 · p. 5
  • Virtual time-of-arrival offsets separate concurrent smartphone chirps into FFT bins, allowing each transmission to be tracked without reducing accuracy or frame rate.The receiver assigns separated offsets, filters other transmissions, tracks each phase, then subtracts its offset from the measured time-of-arrival.Source: §4 · pp. 5–6
  • When motion brings transmission peaks close together, MilliSonic updates virtual delays, adding at most one chirp duration divided by transmitter count.The update is described as infrequent.Source: §4 · p. 6

5 IMPLEMENTATION

The implementation uses Android smartphones to emit FMCW chirps and a four-microphone array connected to processing hardware. Its software runs on Raspberry Pi or laptop platforms and supports real-time tracking.

5 IMPLEMENTATION

  • Android phones emit 45 ms FMCW chirps spanning 17.5–23.5 kHz, tested on Samsung Galaxy S6, S7, and S9 devices.The app uses the smartphone speaker.Source: §5 · p. 6
  • The microphone array uses four MAX9814 amplifiers connected to an Arduino Due, with microphones mounted on a 15 cm square or smaller array.The smaller array measures 6 cm × 5.35 cm × 3 cm.Source: §5 · p. 6
  • Scala software processes chirps on Raspberry Pi or laptop; a 45 ms chirp takes 40 ms on Raspberry Pi and 9 ms on PC.The implementation uses multithreading and is reported to support real-time tracking on both platforms.Source: §5 · p. 6

6 EVALUATION

MilliSonic is evaluated in controlled 1D and 3D tracking tests, environmental conditions, participant drawing, and concurrent-transmission experiments. It reports higher accuracy than CAT and SoundTrak in the tested comparisons, while retaining accuracy with a compact array and up to four concurrent phones.

1D localization accuracy

  • 0.7 mm median accuracy up to 1 m for MilliSonic, versus 4 mm for CAT and 4.8 mm for SoundTrak.At 1–2 m, median accuracy was 1.74 mm, 6.89 mm, and 5.68 mm for MilliSonic, CAT, and SoundTrak, respectively; the paper attributes the decrease to reduced signal-to-noise ratio.Source: 1D Localization Accuracy · p. 6

Environmental robustness

  • MilliSonic remained resilient to random environmental motion and nearby music, while its drift after 10 minutes was a few millimeters and better than the compared acoustic systems.The authors attribute CAT’s larger drift to detecting peak-frequency drift, whereas MilliSonic and SoundTrak use phase to estimate clock difference.Source: Figure 10 · p. 6, Effect of environmental motion and noise · p. 7, Distance drift over time · p. 7
  • At 0.6 m, median errors were 0.75 mm in an anechoic chamber, 1.11 mm in a lobby, and 0.94 mm outdoors; cloth-present and cloth-absent medians were 0.74 mm and 0.95 mm.The cloth test supports tracking through fabric; the tested environments were an anechoic chamber, a 200 m2 lobby, and an outdoor balcony.Source: Figure 8 · p. 6

3D localization accuracy

  • 2.6 mm median 3D accuracy for MilliSonic versus 10.6 mm for CAT across tested locations.The comparison used 15 cm microphone separation for CAT; the authors note CAT is designed for 90 cm separation.Source: Figure 9 · p. 6, 3D Localization Accuracy · p. 7
  • 5.35 cm microphone separation produced little accuracy degradation, with an array prototype measuring approximately 6 cm × 6 cm × 3 cm.The authors identify this compact design as suitable for portable Google Cardboard tracking and headset tracking using a smartphone beacon.Source: Figure 9 · p. 6, 3D Localization Accuracy · p. 7

Free-motion tracking with participants

Concurrent transmissions

  • Up to four concurrent smartphone transmissions did not affect accuracy; five phones caused slightly worse accuracy as nearby peaks interfered.The test used one microphone array to track five phones across a 0–1 m range.Source: Enabling concurrent transmissions · p. 8

7 RELATED WORK

The related work spans inertial, optical, camera-based, and acoustic tracking. MilliSonic is positioned against acoustic systems that lack some combination of 3D localization, sub-millimeter 1D accuracy, compact separation, or concurrent transmission.

Tracking using IMUs

  • IMUs provide acceleration, rotational rate, and heading, but position requires double integration of acceleration, which introduces large drift error.Gaming controllers and some low-end VR systems use IMUs for motion tracking.Source: §7 · p. 8

Tracking using IR/visible light

  • Optical tracking can be accurate enough for VR/AR but requires dedicated beacon hardware; bright environments also impair these techniques.The section describes Lighthouse, PlayStation VR, and Oculus Rift implementations.Source: §7 · p. 8

Tracking using cameras

  • SLAM tracks from camera-captured environments without beacon infrastructure, but performance depends on lighting and visual features, and it can require specialized accelerators.The authors say this makes SLAM unlikely to suit tiny controllers.Source: §7 · pp. 8–9

Device tracking using acoustic signals

  • Prior acoustic systems include 1D-only phone tracking, 2D tracking assuming limited multipath, and centimeter-level systems; CAT requires 0.9 m and 0.7 m speaker-pair separations.SoundTrak reports 1.3 cm average 3D error in a limited movement area; MilliSonic is presented as adding sub-millimeter 1D resolution, compact separation, and concurrent transmission.Source: §7 · p. 9

8 CONCLUSION AND DISCUSSION

MilliSonic combines phase-based FMCW path disambiguation with time-shifted transmissions to support accurate acoustic tracking. The authors identify evaluation, occlusion, drift, range, and concurrency boundaries for the current design.

8 CONCLUSION AND DISCUSSION

  • MilliSonic uses FMCW phase to disambiguate multipath and supports concurrent smartphone transmissions without sacrificing accuracy or frame rate.The conclusion characterizes sub-millimeter 1D tracking as a first for acoustic smartphone tracking.Source: §8 · p. 9
  • More evaluation is needed for edge cases and rooms with significant multipath, which can adversely affect accuracy.The paper reports benchmarks, user studies, and indoor and outdoor evaluation, but says these do not yet establish behavior across those conditions.Source: §8 · p. 9
  • The system supports fabric and paper occlusions, but not human limbs or the device itself; further work is also needed to make drift comparable to optical systems.The authors suggest sensor fusion with IMU data as one possible approach.Source: §8 · p. 9
  • The current range is limited to 2 m, and longer chirps for additional concurrent devices would reduce per-device frame rate.The range limitation is attributed to prototype microphones not optimized for response at 17.5–23.5 kHz.Source: §8 · pp. 9–10

9 APPENDIX: 1D TRACKING DETAILS

The 1D tracker combines adaptive filtering with phase-based distance extraction to handle residual multipath. Its analysis bounds distance error, while estimates from one chirp guide tracking on the next.

9 APPENDIX: 1D TRACKING DETAILS

  • The adaptive FIR filter centers on the first DFT peak, uses a 15 ms delay above 10 dB SNR or 30 ms otherwise, and widens from 1 Hz to 2 Hz above 1 m/s.Source: §9 · p. 10
  • On later chirps, distance and speed from the previous chirp predict the next peak, avoiding the need to receive a full chirp before processing.The prediction uses the estimated distance at the previous chirp's end to infer distance at the current chirp's beginning.Source: §9 · p. 10
  • After distant paths are filtered, residual indirect paths are assumed lower in amplitude than the direct path when unobstructed, and phase is used to extract distance.Source: §9 · p. 10
  • The quadratic in time-of-arrival is solved using its in-chirp root [0,T], which yields distance as ctd.The phase function is shown to be convex with respect to time-of-arrival.Source: §9 · p. 10
  • The distance-estimation error is bounded by c sin^-1(A2/A1)/[2π(f0−B/2)] when residual phase error is bounded by sin^-1(A2/A1).The approximation assumes dynamic filtering has removed most paths with much larger time-of-arrival distances than the direct path.Source: §9 · p. 10
Loading 1901.06601v1…