COVINS: Visual-Inertial SLAM for Centralized Collaboration

1 INTRODUCTION

• Collaborative SLAM jointly handles mapping and localization, whereas much prior collaborative-perception work focuses on one or the other.The paper identifies shared AR and search-and-rescue as settings where avoiding pre-mapping or external tracking infrastructure is relevant.Source: §1 · p. 1
• COVINS reports improved accuracy and demonstrates collaborative SLAM with 12 agents contributing simultaneously.The authors describe this as the most populous team demonstrated for collaborative VI SLAM to date.Source: §1 · p. 1

2 RELATED WORK

• Distributed SLAM supports collaboration and scalability, but data consistency and information double-counting are challenges; centralized systems offer more straightforward information management and usually higher accuracy than state-of-the-art distributed approaches.The cited comparison includes [12] as distributed systems and [15] as centralized systems.Source: §2 · p. 2
• Earlier collaborative VI SLAM demonstrated up to four agents, while COVINS is presented as a more flexible and scalable extension of that architecture.The paper notes limitations of CVI-SLAM in interfacing with custom VIO front-ends and scaling to larger teams.Source: §2 · p. 2
• COVINS supports agents joining on the fly, an unknown team size in advance, and a generic interface to different VIO systems.Its map-management, optimization, and redundancy-detection schemes are reported to improve accuracy and scalability.Source: §2 · p. 2

3 PRELIMINARIES

• The world, IMU body, and camera frames are denoted W, S, and C, respectively.A vector x expressed in frame A is written Ax, and a rigid transformation from B to A is TAB.Source: §3.1 · p. 2
• The IMU model assumes additive Gaussian noise and unknown, time-varying sensor bias.Source: §3.1 · p. 2
• The system state includes keyframe poses and landmark positions, as well as linear velocities and per-keyframe bias variables.Source: §3.1 · p. 2
• V and L denote the sets of all keyframes and landmarks, respectively; θ_j denotes an individual state variable when context permits.Source: §3.1 · p. 2

4.1 System Overview

• Each agent runs VIO locally while the server manages global maps, place recognition, map merging, keyframe pruning, and optimization.The server can run locally or remotely, and COVINS exposes an interface for custom keyframe-based VIO systems.Source: §4.1 · pp. 2–3

4.2 Map Structure

• A server map is a SLAM graph of keyframes and landmarks, with IMU-constraint and landmark-observation edges; multiple maps persist until agents are co-localized.IMU factors connect consecutive keyframes from the same agent, while shared landmark observations link agents’ keyframes.Source: §4.2 · p. 3

4.3 Error Residuals Formulation; 4.4 Visual-Inertial Odometry Front-End

• COVINS expresses keyframe-based visual-inertial SLAM as weighted nonlinear least squares using reprojection, relative-pose, and IMU pre-integration residuals.The server interface supports indirect, feature-based VIO; GBA is initialized with the VIO’s metric scale, biases, and velocities.Source: §4.3 · p. 3, §4.4 · p. 3

4.5 Communication; 4.5.1 Agent-to-Server Communication

• Agents send map data through full messages or smaller state-update messages, batching data over short windows at a fixed frequency.Static keyframe and landmark information is not repeatedly transmitted, reducing required network bandwidth.Source: §4.5.1 · p. 3, §4.5 · p. 3

4.5.2 Server-to-Agent Communication (Map Re-Use)

• The server returns its latest keyframe pose so an agent can estimate local odometry drift without modifying its smooth local VIO map.The paper identifies preserving smoothness as important when pose estimates feed a robot-control system.Source: §4.5.2 · p. 3

4.6 Multi-Map Management; 4.7 Place Recognition, Loop Closure & Map Fusion

• The map manager maintains maps and a keyframe database; detected overlap triggers place recognition, then either loop closure with optimization or map fusion.Fusion aligns the maps, combines duplicate landmarks, and merges their associated AR content.Source: §4.6 · pp. 3–4, §4.7 · p. 4

4.8.1 Pose-Graph Optimization; 4.8.2 Global Bundle Adjustment

• PGO follows detected loop closure, while on-demand GBA refines all keyframes and landmarks; robust loss reduces outlier influence and large-residual outliers are removed.GBA uses reprojection and IMU pre-integration residuals, with IMU constraints only between consecutive keyframes from the same agent.Source: §4.8.1 · p. 4, §4.8.2 · p. 4

4.8.3 Redundancy Removal (KF Pruning)

• COVINS scores keyframes by landmark-observation redundancy and removes the most redundant before GBA to improve scalability.Landmarks with fewer than two observations are removed after outlier-observation or keyframe pruning.Source: §4.8.3 · p. 4

5 EXPERIMENTAL RESULTS

• Experiments replayed recorded datasets in real time over a wireless network, with results averaged over five runs unless otherwise stated.The setup used onboard Intel NUC agents and either a Lenovo local server or an AWS cloud server.Source: §5 · pp. 4–5

5.1 Collaborative SLAM Estimation Accuracy

• COVINS achieves high accuracy across the EuRoC collaborative scenarios, with performance generally similar to or better than the state of the art and similar accuracy on a cloud server.The comparison covers ATE and scale error against ORB-SLAM3, VINS-mono, and CVI-SLAM; the five-agent estimate is illustrated in Figure 3.Source: §5.1 · p. 5, Table 1 · p. 5, Figure 3 · p. 5

5.2 Large-Scale Collaborative SLAM with 12 Agents

• With 12 UAVs, COVINS produced an estimate from over 3,200 keyframes and about 200,000 landmarks, with 0.050 m average ATE and 0.44% average scale error.The simulated trajectories covered about 500 m² and 1,750 m total trajectory length.Source: §5.2 · p. 5

5.3 Drift Correction

• Server-informed drift correction moved the agent’s trajectory estimate noticeably closer to ground truth after drift accumulated in onboard VIO.The examples use MH3 and MH5; the onboard trajectory was not globally optimized by the agent itself.Source: §5.3 · p. 5, Figure 4 · p. 5

5.4 Redundancy Detection & Removal

• Even with more than 50% map compression, the mean error increased by only 7 mm, while global bundle adjustment time fell significantly.The redundancy evaluation reduced maps averaging about 1,700 keyframes to 1,250, 1,000, or 750 keyframes.Source: §5.4 · p. 5, Table 2 · p. 5

5.5 Communication Statistics

• Agent-to-server traffic was approximately 400–600 KB/s, while onboard communication consumed less than 1% of estimation time and preserved real-time VIO.Agents sent new or modified map data at 5 Hz, and the server shared information at 2 Hz.Source: §5.5 · p. 6, Table 3 · p. 6

6 CONCLUSION

• COVINS generates collaborative global SLAM estimates online from data contributed simultaneously by multiple agents, without external infrastructure or pre-built maps.Its architecture is designed to process data from many agents simultaneously.Source: §6 · p. 6
• The system demonstrated collaborative SLAM with 12 agents, and its redundancy detection reduced keyframes while maintaining a similar level of accuracy.COVINS can run on a local PC or remote cloud server.Source: §6 · p. 6
• Future work will target scalability to potentially hundreds of agents and shared AR through a mobile-device front-end.Source: §6 · p. 6

• Notation follows, with bold lowercase and uppercase letters denoting vectors and matrices, respectively.Source: §1 · p. 8
• Plain capital letters denote coordinate frames, and a vector x expressed in frame A is written Ax.Source: §1 · p. 8
• A rigid-body transform from frame B to A is denoted TAB, with t and R representing its translation and rotation.Source: §1 · p. 8

• Accelerometer and gyroscope measurements are modeled with additive Gaussian noise and unknown, time-varying sensor bias.Source: §2 · p. 8
• The system state includes KF poses and LM positions, plus linear velocities and bias variables for each KF.Source: §2 · p. 8
• The sets V and L denote, respectively, all keyframes and landmarks.Source: §2 · p. 8

3 ERROR RESIDUALS FORMULATION

• Each residual measures the difference between a state-predicted measurement and the actual measurement, and the collected residuals define a weighted nonlinear least-squares objective.Source: §3 · p. 8
• COVINS uses reprojection, relative-pose, and IMU pre-integration residuals.Source: §3 · p. 8

3.1 Reprojection Residual

• Reprojection residuals relate a landmark’s projected position to its associated 2D image observation in a keyframe.Source: §3.1 · p. 8

3.2 Relative Pose Residual

• Relative-pose residuals constrain two poses using a measured estimate of the transformation between their frames.Source: §3.2 · p. 8
• The relative-pose residual maps rotation differences to the tangent plane using log(R), as described in.Source: §3.2 · p. 8

3.3 IMU pre-integration Residual

• IMU pre-integration integrates accelerometer and gyroscope readings between keyframes once, then applies linearized corrections for bias changes rather than re-integrating after every optimization step.The bias estimate at pre-integration and the current state estimate are distinguished; Δt represents the interval between consecutive keyframes.Source: §3.3 · pp. 8–9
• In the factor graph used for GBA, shared landmark observations connect keyframes across agents, while IMU residuals connect only keyframes from the same agent.Possible loop-closure constraints are not illustrated.Source: Figure 1 · p. 9

• The factor graph induced by the system state and residuals forms the basis of COVINS’s Global Bundle Adjustment scheme.Source: §4 · p. 9
• Shared landmark observations create dependencies between keyframes from multiple agents.Source: §4 · p. 9
• IMU factors connect only consecutive keyframes created by the same agent.Source: §4 · p. 9

• τ(x) rises from 0 at two observations to 1 above five, treating five observations as sufficient for robust 3D landmark estimation.The intermediate values are 0.4 at three observations, 0.7 at four, and 0.9 at five.Source: §5 · p. 9

6.1 Large-Scale Collaborative SLAM with 12 Agents

• Well-aligned landmarks at a trajectory intersection attest to high accuracy in the 12-agent village estimate.The experiment used 12 circular UAV trajectories of 20 m radius.Source: Figure 2 · p. 9, §6.1 · p. 9

6.2 Communication Statistics

• Communication statistics cover all eight EuRoC sequences and report agent-server traffic and onboard communication time averaged over five runs.More challenging sequences generate more onboard keyframes and consequently increasing network traffic.Source: Table 1 · p. 9, §6.2 · p. 9