LARK: A Low-Cost, Accurate, Occlusion-Resilient, Kalman Filter-Assisted Tracking System for Image-Guided Surgery

1. Introduction

• Stereo infrared trackers can lose tracking under occlusion, occupy workspace, and cost $8,000–$25,000 USD for the tracking module alone.Electromagnetic systems avoid direct line-of-sight requirements but can be affected by metal objects in the operating room.Source: §1 · p. 1
• LARK addresses camera-level occlusion with multiple sightlines and marker-level occlusion with two pose-estimation methods.The methods are multi-view monocular pose fusion and multi-view triangulation, both evaluated on a precision-machined grid and anatomical head phantom.Source: §1 · pp. 1–2
• LARK combines commodity RGB cameras, adaptive filtering, pose fusion, and triangulation in a surgical-light-mounted system costing under $1,000 USD.With five cameras on the machined grid, it reports 0.64 mm point localization with triangulation and 0.73 mm trajectory tracking with pose fusion.Source: §1 · pp. 1–2
• The work releases hardware designs, a bill of materials, software, and 5,000 multi-view tracking videos as an open benchmark for surgical tracking research.Evaluation includes camera subsets representing different levels and patterns of camera unavailability.Source: §1 · p. 2

2.1. State-of-the-Art Optical Tracking Systems

• Single stereo infrared cameras provide surgical tracking but are expensive, large, and prone to occlusion; prior multi-camera expansions also remain costly.A cited Polaris system achieved sub-millimeter accuracy with latency below 20 milliseconds at distances up to 2.4 meters.Source: §2.1 · p. 2

2.2. Optical Tracking Using Planar Fiducial Markers

• Single planar markers on off-the-shelf stereo RGB-Depth cameras have not provided accuracy required for surgical navigation, motivating multiple markers or cameras.ArUco and AprilTag are identified as popular planar-fiducial options because of their accessibility and ease of use.Source: §2.2 · p. 2

2.2.1. Multi-Marker Instrument Tracking

• Dodecahedral multi-marker systems improve tracking from different perspectives, but cited evaluations use short camera distances or a single viewpoint and do not address camera-level occlusion.Moviglia and Stallkamp (2026) report 0.44 mm translational accuracy at 30–70 cm, from static position estimates at short range and a single viewpoint.Source: §2.2.1 · p. 2

2.2.2. Multi-View RGB Tracking

• Multi-view RGB tracking relies on calibration and fusion, while prior approaches have limits in calibration scope, dynamics, simulation, or OR-like evaluation.Tsui et al. (2024) report 2 mm root-mean-square error, but evaluate planar motion at distances typically below one meter.Source: §2.2.2 · pp. 2–3

2.2.3. Hybrid Multi-Camera, Multi-Marker Systems

• A prior surgical-light-mounted RGB system reported mean point and trajectory errors of 2.01 mm and 2.37 mm, respectively, and did not release its design or software.LARK targets sub-millimeter localization with commodity hardware, tests camera-view loss, and releases its fixture, software, and bill of materials.Source: §2.2.3 · p. 3

2.3. Marker-Less Instrument Tracking

• Marker-less methods avoid instrument modification, but the cited evaluation reports 1.01 mm instrument position error under optimal in-domain conditions and requires instrument-specific models and training data.The cited error is measured against a marker-based tracker rather than as registration-based target error in a patient reference frame.Source: §2.3 · p. 3
• LARK compares pose fusion and triangulation on shared hardware and recordings under controlled camera-subset removal, releasing the benchmark for further evaluation.The release includes hardware designs, software, and data.Source: §2.3 · p. 3

3.1. System Overview

• The system tracks a pointer relative to a static reference using planar fiducial markers, with multi-view monocular pose fusion or multi-view triangulation and adaptive Kalman filtering.IBIS displays the tracked pointer over patient imaging data.Source: §3.1 · p. 3

3.2.1. Multi-Camera Fixture

• The fixture uses five cameras in a ring attached to a surgical light, with adjustable hinges directing overlapping views toward a point roughly one meter below the light.Its segments are 3D-printed and fastened together, and toggle clamps allow quick attachment or removal from the lamp.Source: §3.2.1 · p. 3, Figure 1 · p. 4

3.2.2. Surgical Pointer and Static Reference

• Dodecahedral ArUco marker assemblies are attached to the pointer and reference; the marker geometry keeps at least three faces visible from most viewing angles.A rigid instrument with a well-defined working point can be tracked after one-time pivot calibration, without a per-instrument model or training.Source: §3.1 · p. 3, §3.2.2 · p. 4

3.2.3. Electronics and Optics

• The selected camera setup uses a 6 mm lens to cover the head and 220 mm pointer at a one-meter working distance, with an approximately 837 mm horizontal field width.The cameras use Raspberry Pi Global Shutter modules paired with Raspberry Pi boards.Source: §3.2.3 · p. 5

3.3.1. Multi-Camera Calibration

• Multi-camera calibration estimates camera intrinsics and extrinsics from ChArUco-board observations, using pairwise initialization followed by global bundle adjustment.The global adjustment refines camera parameters using triangulated board corners and PnP board poses across observed frames and cameras.Source: §3.3.1 · p. 5, §3.3.2 · p. 6

3.3.2. Instrument Calibration

• Instrument calibration refines marker-corner coordinates by fitting their observations across video frames, avoiding dependence on precise manufacturing.For pointer calibration, the tip is placed at the ChArUco-board origin and the shaft is pivoted and rotated while synchronized video is recorded.Source: §3.3.2 · p. 5

3.4.1. Multi-View Monocular Pose Fusion

• A rolling window of up to 30 frames updates pose-related process and measurement covariances from recent variability, adapting to noise changes.Missing poses are omitted and available history is used during startup.Source: §3.4.1 · pp. 7–8
• Each active camera independently estimates pointer-to-reference pose with PnP, and an adaptive Kalman filter fuses available poses frame by frame.A camera is active when it detects both the pointer and static reference.Source: §3.4.1 · pp. 6, 8, Figure 4 · p. 7

3.4.2. Multi-View Triangulation

• Corners observed by at least two cameras are triangulated in camera-1 coordinates; reprojection outliers are discarded before remaining points are registered to estimate relative pose.A marker is discarded when any corner’s reprojection error exceeds 8 pixels in an observing camera.Source: §3.4.2 · p. 9
• Triangulation filtering smooths a single triangulated pose per frame using recent pose history for covariance, whereas pose fusion combines camera-specific measurements and covariances.The triangulation filter keeps its velocity-noise block fixed at 0.001 I6.Source: §3.4.2 · p. 9
• The triangulation method cannot selectively suppress cameras with poor views.Source: §3.4.2 · p. 9

4.1. Experimental Setups

• Both setups mount the multi-camera fixture on a surgical-light stand-in and use a gold-standard fusionTrack 500 for simultaneous measurements.The grid establishes baseline accuracy, while the SLA-printed head phantom provides neurosurgery-representative validation.Source: §4.1 · pp. 9–10, Figure 5 · p. 10, Figure 6 · p. 11

4.1.1. Grid Experiment

• Across 20 trials, the grid experiment measures TRE at 16 target points after registering nine fiducials and also evaluates pointer trajectories along machined S-shaped grooves.The 400 mm × 400 mm grid has 25 uniformly spaced points; the pointer holder and 2 mm holes reduce motion-related measurement noise.Source: §4.1.1 · p. 10

4.1.2. Head Phantom Experiment

• Across 20 trials, the head phantom protocol measures point TRE at 10 targets and trajectory TRE along two grooves after registration from nine anatomical fiducials.The SLA-printed phantom derives from a 3T MRI scan and includes 21 landmarks.Source: §4.1.2 · p. 10

4.2.1. Camera Count

• Camera-level occlusion is tested across all 31 nonempty subsets of five cameras on the grid and across three head-phantom obstruction directions at five severity levels.The phantom severity scale ranges from one active camera to all cameras active.Source: §4.2.1 · p. 10, Figure 7 · p. 12

4.2.2. Filtering Method

• The filtering ablation compares adaptive Kalman filtering, a non-adaptive Kalman filter, and no filtering, with unfiltered pose-fusion estimates averaged across cameras.Source: §4.2.2 · p. 10

5. Results

• Point TRE distributions contain individual target-point errors, whereas trajectory TRE distributions contain one RMSE per measured trajectory and configuration.Statistical comparisons use 20 acquisition trials per phantom as independent units.Source: §5 · p. 12
• Pooled trial-summary errors generally decrease with more cameras, although improvements are not monotonic.Camera-count comparisons pool squared errors across available methods and filters within each trial.Source: §5 · p. 12
• Significance decisions differ between the paired signed-rank and sign tests for some comparisons.The sign tests assess sensitivity to the signed-rank symmetry assumption.Source: §5 · p. 12
• Adaptive Kalman filtering significantly reduces pose-fusion trajectory error at every grid camera count and at three to five head-phantom cameras, and triangulation trajectory error at every grid count.Both paired tests support these filtering comparisons.Source: §5 · p. 12
• In the worst-case head-phantom trial, camera 3 sees only one marker at an oblique angle, and tracking errors accumulate toward the trajectory’s end.Figures 12 and 13 illustrate this trial’s contrasting method behavior.Source: §5 · p. 12

6. Discussion

• Triangulation trajectory TRE is non-monotonic on the grid, reaching 1.11 mm at three cameras before recovering with five, likely because camera angles affect geometric conditioning.Triangulation nevertheless has lower static point TRE than pose fusion on the grid (0.64 mm versus 0.74 mm).Source: §6 · p. 12, Figure 8 · p. 13
• On the head phantom, neither method achieves sub-millimeter median point or trajectory TRE with five cameras and adaptive filtering.The fixed camera positions make this evaluation more demanding than typical clinical use, where cameras can be repositioned; phantom warping and wear are also noted.Source: §6 · p. 14
• With five cameras and adaptive filtering, pose fusion has lower grid trajectory TRE (0.73 mm), while triangulation has lower phantom point TRE (1.13 mm versus 1.35 mm) and trajectory TRE (1.06 mm versus 1.53 mm).Figure 9 compares five-camera performance against fusionTrack 500; Figure 8 shows TRE across camera counts.Source: §6 · pp. 12, 14, Figure 8 · p. 13, Figure 9 · p. 14
• A compromised single-marker view produces large outliers in pose fusion, while triangulation combines corners across cameras and remains stable under the same poor view.A single marker’s four coplanar corners can yield ambiguous or noise-sensitive PnP estimates; camera 3’s few corner measurements are diluted among the other views.Source: §6 · p. 14
• Adaptive filtering reduces pose-fusion trajectory-error spread, but stable erroneous single-marker estimates may retain appreciable weight; its effect on triangulation varies by camera count and phantom.Pose fusion uses per-camera covariance weighting; both methods can be affected by inter-camera timing errors during motion.Source: §6 · pp. 12, 14
• Camera-subset performance depends on both the number and quality of retained views, and the results demonstrate potential for commodity RGB cameras with multiple viewpoints.The illustrated poor single-marker view affects pose fusion more than triangulation, while filtering narrows pose-fusion trajectory-error spread.Source: Figure 8 · p. 13, §6 · p. 14
• Future work should explore view-quality-triggered method switching or camera exclusion, and combining triangulated stereo-pair poses with adaptive Kalman fusion.These strategies are proposed to prevent inaccurate estimates from poor single-marker views while preserving accurate trajectory tracking under favorable conditions.Source: §6 · p. 14

7. Conclusion

• On a machined grid, the system achieves 0.64 mm point TRE with triangulation and 0.73 mm trajectory TRE with pose fusion; on an anatomical head phantom, triangulation yields 1.13 mm point and 1.06 mm trajectory TRE.Source: §7 · pp. 14–15
• Triangulation is affected less than monocular pose fusion by the illustrated poor-view condition, where one camera sees only a single marker at an oblique angle.Figure 12 shows the compromised camera view during the head-phantom trajectory trial with the maximum error.Source: §7 · p. 15, Figure 12 · p. 15
• Camera-subset experiments characterize how tracking accuracy changes as views become unavailable.Figure 10 presents point TRE and per-trajectory RMSE across camera counts for pose fusion and triangulation.Source: §7 · p. 15, Figure 10 · p. 15
• With tracking hardware costing under $1,000 USD, the released designs, software, and datasets provide an accessible, reproducible benchmark for IGS research.The paper states that clinical validation remains the critical next step to confirm translation of laboratory results to improved surgical outcomes.Source: §7 · pp. 15, 17

A. Noise Profiling

• The procedure estimates static measurement covariance for each camera from stationary poses sampled at 30 consecutive frames across workspace locations.The resulting per-camera covariance matrices configure the non-adaptive Kalman filter; their camera-wise average supplies the pose block of its process-noise covariance, with velocity variances fixed as tuning parameters.Source: A. Noise Profiling · p. 17
• The Frobenius norm of each location- and camera-specific measurement covariance provides a scalar noise level for mapping spatial noise variation.Figure A.1 shows the spatial distribution of these noise levels across measured positions in one experimental trial.Source: A. Noise Profiling · p. 17, Figure A.1 · p. 17
• Noise profiling supplies the static-covariance baseline for comparison with adaptive filtering and can inform lighting, camera placement, and calibration adjustments.The authors also suggest it could help technicians assess tracking precision during OR camera setup.Source: A. Noise Profiling · p. 17

B. Landmark Registration

• Orthogonal Procrustes registration finds a rotation and translation that minimize differences between corresponding measured and ground-truth fiducials.The method first centers both point sets before determining the transformation and prevents reflection by negating a column of the singular-vector matrix when det(S) < 0.Source: B. Landmark Registration · p. 18
• The fiducial-derived transformation is validated on a separate set of target points to calculate target registration error.The same transformation is applied to measured trajectories to calculate trajectory TRE.Source: B. Landmark Registration · p. 18
• Trajectory TRE measures nearest-neighbour spatial agreement with the reference trajectory, without measuring tracking latency or along-path lag.This metric does not require temporal alignment.Source: B. Landmark Registration · p. 18

C. Supplementary Data

• Table C.1 reports registration RMSE in millimeters as median ± standard deviation across 20 head-phantom trials for top, right, and left occlusion scenarios.Source: C. Supplementary Data · p. 18, Table C.1 · p. 19
• Figure C.1 displays FRE error distributions by experiment and tracking pipeline, with medians and interquartile ranges shown.The top row is the grid experiment, the bottom row the head phantom; columns separate monocular pose fusion and triangulation.Source: C. Supplementary Data · p. 18, Figure C.1 · p. 20
• Tables C.2–C.4 provide Holm-corrected paired Wilcoxon comparisons for camera-count and filtering-method effects using 20 paired trial summaries.The tables cover pooled camera-count comparisons and filtering-method comparisons for the grid and head phantom.Source: C. Supplementary Data · p. 18, Table C.2 · p. 20, Table C.3 · p. 21

• George Sideris is credited across conceptualization, methodology, software, validation, analysis, investigation, data curation, visualization, writing, and project administration.Source: CRediT authorship contribution statement · p. 23
• Justin Cree shares broad research and data roles with Sideris and is credited with writing the review and editing.Source: CRediT authorship contribution statement · p. 23
• Andrew Stirling and Mamadou Ly contributed to methodology and review editing, while Étienne Léger and D. Louis Collins are credited with supervision and other leadership roles.Léger is credited with conceptualization and investigation; Collins with conceptualization, funding acquisition, and resources.Source: CRediT authorship contribution statement · p. 23