PaperlayerView exact revision on arXiv

Source-linked AI summary

NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge

Ali Agha, Kyohei Otsu, Benjamin Morrell, David D. Fan, Rohan Thakker, Angel Santamaria-Navarro, Sung-Kyun Kim, Amanda Bouman, Xianmei Lei, Jeffrey Edlund, Muhammad Fadhil Ginting, Kamak Ebadi, Matthew Anderson, Torkom Pailevanian, Edward Terry, Michael Wolf, Andrea Tagliabue, Tiago Stegun Vaquero, Matteo Palieri, Scott Tepsuporn, Yun Chang, Arash Kalantari, Fernando Chavez, Brett Lopez, Nobuhiro Funabiki, Gregory Miles, Thomas Touma, Alessandro Buscicchio, Jesus Tordesillas, Nikhilesh Alatur, Jeremy Nash, William Walsh, Sunggoo Jung, Hanseob Lee, Christoforos Kanellakis, John Mayo, Scott Harper, Marcel Kaufmann, Anushri Dixit, Gustavo Correa, Carlyn Lee, Jay Gao, Gene Merewether, Jairo Maldonado-Contreras, Gautam Salhotra, Maira Saboia Da Silva, Benjamin Ramtoula, Yuki Kubo, Seyed Fakoorian, Alexander Hatteland, Taeyeon Kim, Tara Bartlett, Alex Stephens, Leon Kim, Chuck Bergh, Eric Heiden, Thomas Lew, Abhishek Cauligi, Tristan Heywood, Andrew Kramer, Henry A. Leopold, Chris Choi, Shreyansh Daftry, Olivier Toupet, Inhwan Wee, Abhishek Thakur, Micah Feras, Giovanni Beltrame, George Nikolakopoulos, David Shim, Luca Carlone, Joel Burdick

arXiv:2103.11470v4cs.ROcs.AI

TL;DR

Robotic exploration underground and in other extreme environments requires autonomy that can handle degraded perception, difficult terrain, limited resources, and high operational risk. The paper presents NeBula, an uncertainty-aware and networked autonomy framework, and reports Team CoSTAR’s first-place Urban and second-place Tunnel results in the DARPA Subterranean Challenge. It also identifies ongoing work on perception-aware planning and more principled treatment of localization uncertainty.

  • Problem

    Extreme environments combine perceptual degradation, difficult traversability, constrained resources, and high-risk operations that autonomy systems must address simultaneously.

  • Method

    NeBula combines uncertainty-aware reasoning with modular autonomy components for estimation, mapping, traversability, planning, networking, and mission decision making.

  • Results

    NeBula led Team CoSTAR to second place in the Tunnel competition and first place in the Urban competition, including detection of 25 artifacts during four Urban runs.

  • Takeaways & Limitations

    The paper demonstrates an integrated uncertainty-aware autonomy solution across heterogeneous robots and challenging subterranean environments.

  • Takeaways & Limitations

    Ongoing work remains to incorporate perception models and localization uncertainty into traversability planning in a theoretically satisfying yet computationally tractable way.

Abstract

from arXiv · show

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR's demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including: (i) geometric and semantic environment mapping; (ii) a multi-modal positioning system; (iii) traversability analysis and local planning; (iv) global motion planning and exploration behavior; (i) risk-aware mission planning; (vi) networking and decentralized reasoning; and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g. wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition.

1 Introduction

The paper addresses autonomous exploration in extreme environments, where underground settings combine severe human risk with perceptual, terrain, resource, and failure challenges. It presents NeBula and Team CoSTAR’s contributions to robotic exploration, including competition results and a broad autonomy architecture.

  • Motivation: Underground environments matter for terrestrial rescue, industrial operations, cave missions, and planetary exploration, but remain too hazardous for conventional human access.During the Tham Luang rescue, no technology could autonomously reach people, map the cave, and scan for survivors deep underground.
  • Motivation: Planetary caves are important exploration targets because they may provide radiation shielding, stable environments, volatile traps, and potential habitats.The paper notes more than 200 lunar and 2000 Martian cave-related features.
  • Open Challenges: Extreme environments combine degraded perception, difficult terrain, constrained resources, and high-risk operations in ways that current autonomy must address simultaneously.The paper emphasizes that the combination of these challenges is especially difficult.
  • Contributions: NeBula is presented as Team CoSTAR’s autonomy solution for unknown extreme surface and subsurface environments.The paper discusses its application in the DARPA Subterranean Challenge, where CoSTAR won Urban and ranked second in Tunnel.
  • Contributions: The contributions span uncertainty-aware architecture, state estimation, mapping, semantic understanding, traversability, planning, networking, mission management, and hardware integration.These components cover both autonomy algorithms and the mobility-sensor-computing systems supporting them.

2 DARPA Subterranean Challenge

The DARPA Subterranean Challenge evaluates autonomous robotic systems for rapidly mapping, navigating, and searching unknown underground environments. Its courses combine difficult terrain, degraded sensing, limited communications, endurance constraints, and artifact-reporting requirements under a one-hour mission window.

  • Challenge Overview: The Subterranean Challenge seeks novel approaches to rapidly map, navigate, and search mines, industrial complexes, and natural caves.The systems track requires teams to develop physical systems for autonomous traversal, mapping, and search.
  • Challenge Domains: The challenge spans three subdomains: tunnel systems, urban underground, and cave networks.The final event combines elements from all subdomains to demonstrate solution versatility.
  • Challenge Overview: The competition targets rapid autonomous situational awareness where environments are unknown, may change, and can be too dangerous for personnel.Representative scenarios include planetary cave exploration, mine rescue, urban search and rescue, and cave rescue.
  • Competition Rules: Each team has 1 hour to map the unknown environment, localize artifacts, and communicate situational-awareness updates to a remote base station.Communication is expected to operate as close to real time as possible during the mission.
  • Competition Rules: Artifacts score only when teams reach, detect, recognize, globally localize within less than 5 meters error, and report them within the mission period.There are 20 artifacts and 25 reporting chances, making report selection a constrained resource.
  • Challenge Elements: Core challenge elements include austere navigation, degraded sensing, severe communication constraints, terrain obstacles, and endurance limits.Terrain includes constrained passages, drops, climbs, inclines, steps, ladders, mud, sand, water, clutter, and debris; sensing ranges from lighted areas to complete darkness.

3 Concept of Operations

NeBula’s concept of operations uses heterogeneous robots to explore unknown terrain while extending communications, exchanging information, and dynamically allocating tasks. Robots operate autonomously beyond the mesh network, using complementary mobility, sensing, and computing capabilities.

  • ConOps and Robot Teams: NeBula deploys heterogeneous platforms whose complementary mobility, sensing, and computing capabilities support exploration under time constraints.Robot capability combinations and payload capacity influence the sensors and endurance available for each platform.
  • Mission Sequence: Vanguard robots enter first with capable sensing for frontier mapping and artifact detection.This initiates exploration using robots selected for strong perception.
  • Mission Sequence: Ground robots extend a wireless mesh with communication pucks, while aerial scouts can self-deploy as communication relays or sensing assets.Mission autonomy decides where and how to deploy the communication breadcrumbs.
  • Mission Sequence: Because mesh reach is usually limited near the entrance, robots leave communication range and conduct most of the mission autonomously.The environment’s scale, complexity, and communication denial constrain persistent links to the base station.
  • Mission Sequence: Robots perform search, mapping, and exploration, rendezvousing to exchange information or returning to the mesh to communicate with the base station.Information exchange is therefore supported both within the team and through network contact with the supervisor.
  • Dynamic Task Allocation: Mission planning monitors team health, battery, functionality, world knowledge, mission time, communication state, and acceptable risk before retasking or repositioning robots.The planner can deploy new robots, re-task active robots, or reposition them in the environment.
  • Team Behaviors: Team behaviors include returning to base for battery swaps, returning to the mesh, exploring frontiers, and acting as data mules.Faster or healthier vehicles can carry information from robots unable to return to the network.
  • Heterogeneous Coverage: Heterogeneous coverage may send different robots to the same area to increase mapping and artifact-detection confidence through multimodal observations.The paper gives thermal and radar sensing as examples of complementary information.

4 NeBula Autonomy Architecture

NeBula is a resilient, modular autonomy architecture that reasons over uncertainty and tightly co-designs perception, inference, planning, and execution. Its layered framework supports heterogeneous robots, networked operations, and risk-aware adaptation in challenging environments.

  • Architecture: NeBula’s central principle is reciprocal, tightly co-designed perception and decision-making over joint probability distributions.The architecture uses a plan-to-sense, infer, and act loop to acquire information needed for resilient operation.
  • Modularity and scalability: NeBula’s modular, hardware-agnostic design supports networked teams of heterogeneous wheeled, legged, aerial, hybrid, tracked, and passenger-vehicle platforms.The framework was deployed across several vehicle types in terrestrial and planetary projects.
  • System organization: The architecture combines perception, planning, communication, operations, and belief prediction across robots, communication nodes, and a base station.These modules exchange world-belief information when communication links are available.
  • Planning: Risk-aware traversability analysis evaluates terrain motion risk and frequently replans trajectories while remaining within mission specifications.Motion models are abstracted so the planning stack can support heterogeneous mobility platforms.
  • Architecture: NeBula maintains probabilistic beliefs over robot, environment, communication, team, and health states instead of isolated state estimates.This belief representation supports joint reasoning across system components.

5 State Estimation

NeBula’s state-estimation stack combines heterogeneous sensing and parallel estimators with health-aware resiliency logic for perceptually degraded environments. Its LiDAR-centered components target accurate, robust real-time odometry while adapting to unhealthy measurements and sensor failures.

  • Challenges: NeBula targets reliable state estimation across darkness, obscurants, reflective surfaces, self-similar scenes, and feature-poor terrain.These conditions can cause localization error and drift during extended runs.
  • State representation: The state-estimation pipeline assigns quality measures to multimodal sensor outputs before probabilistic fusion.The represented state includes position, orientation, velocity, and linear and angular accelerations.
  • HeRO: HeRO runs heterogeneous odometry algorithms in parallel and uses resiliency logic to test quality, reinitialize failures, and multiplex healthy estimates.The resulting state-quality measure informs guidance and control decisions.
  • LOCUS: LOCUS combines multi-LiDAR preprocessing, cascaded GICP scan matching, and health-aware integration of additional odometry and IMU inputs.It estimates six-degree-of-freedom motion between consecutive LiDAR acquisitions and can use non-LiDAR initialization to improve convergence.
  • Evaluation: LOCUS achieves highly accurate performance in Tunnel and Urban Circuit evaluations while testing robustness to sensor failures against other LiDAR odometry methods.The evaluation uses absolute position error on subterranean runs with a wheeled robot carrying two Velodyne LiDARs, an IMU, and WIO.

6 Large-Scale Positioning and 3D Mapping

LAMP is NeBula’s large-scale, uncertainty-aware mapping and positioning system, using multimodal factor-graph SLAM to produce consistent geometric and semantic maps for single- and multi-robot exploration. Its evaluations show low drift and strong artifact-localization performance across challenging environments, with UWB improving multi-robot results.

  • System goals: LAMP performs low-drift, multi-robot, multisensor SLAM in perceptually degraded environments while producing global maps with associated covariances.It targets artifact localization errors below 5 m over multiple kilometers of traverse.
  • Architecture: LAMP uses adaptable odometry, multimodal loop closures, and outlier-resilient factor-graph optimization.The graph fuses inputs from LiDAR, vision, semantics, inertial sensing, landmarks, and inter-robot measurements.
  • Multi-robot fusion: Each robot builds a local factor graph, while a base station merges and optimizes graphs into a common multi-robot representation.The main outputs are robot poses and artifact locations.
  • Factors: LAMP incorporates gravity, deployed and environmental landmarks, calibration, and loop-closure factors to constrain pose estimation.UWB beacons can seed loop closures, while artifact observations update global object locations.
  • Mapping: LAMP builds geometric and semantic global maps by projecting sensor measurements through optimized factor-graph pose nodes.Inputs include point clouds and local occupancy grids.
  • Performance: On a three-LiDAR Husky dataset, LAMP achieves error below 0.2% of distance travelled.Across benchmark datasets, it achieves better-than-5 m accuracy except on tunnel and cave datasets affected by motion-distorted LiDAR; UWB improves multi-robot mapping relative to pure LiDAR loop closures.

7 Semantic Understanding and Artifact Detection

NeBula’s semantic layer detects, localizes, reconciles, and visualizes objects and diffuse phenomena across heterogeneous robots and sensor modalities. It separates detection from relative localization and uses confidence, multimodal sensing, and distributed measurements to improve situational awareness.

  • System role: The semantic system addresses object detection, localization, and visualization on heterogeneous robots with different sensor configurations.Semantic mapping and artifact detection support higher levels of autonomy in unknown environments.
  • Detection and localization: The pipeline separates image-based object detection from projective relative localization to combine fast detectors with generic camera types.Bounding boxes are combined with depth from RGB-D cameras, LiDAR, size-based projection, or monocular tracking.
  • Visual detection: Visual artifacts are detected in color and thermal imagery using CNNs, including a YOLO Tiny variant for real-time ground-robot processing.The implementation adapts CNNs to available computational resources.
  • Multi-robot reconciliation: Base-station reconciliation matches artifact observations across robots and visits while rejecting outliers.Reports include class, confidence, images, bounding boxes, and location estimates.
  • Diffuse phenomena: Gas leaks and WiFi sources are localized by treating the robot team as a mobile sensor network and following signal-strength gradients.Measurements augment the semantic map and provide initial source estimates near peak signal strength.
  • Performance: False positives occur when spray paint markings and existing equipment share gross features with target artifacts.This observation identifies a concrete challenge for visual detection in subterranean environments.

8 Risk-aware Traversability and Motion Planning

NeBula’s STEP component enables robots to assess and plan traversal through extreme terrain by quantifying traversability uncertainty and risk. It combines multi-fidelity mapping, risk-aware analysis, hierarchical planning, recovery, and adaptation for challenging environments.

  • Risk-aware traversability and motion planning: STEP quantifies terrain uncertainty and risk to support safer traversal and motion planning in extreme environments.It is designed for difficult geometries, hazards, overhangs, and narrow passages.
  • System architecture: The architecture assesses traversability at multiple fidelity levels, encodes confidence in maps, and plans kinodynamically feasible paths while considering mobility risks.Pointcloud and odometry feed risk analysis, whose map supports hierarchical geometric and kinodynamic planners.
  • Robot agnosticism and adaptation: The approach supports varied ground robots by changing the dynamics model and allows constraints on attitude, obstacle distance, narrow passages, and risky-area speed.Recovery behaviors address non-fatal failures caused by localization, traversability, hardware, or unknown issues, while Gaussian processes adapt dynamics models.
  • Traversability risk analysis: Traversability risk analysis evaluates collision, tip-over, traction loss, and negative-obstacle risks from elevation maps and segmented pointclouds.Semantic cues such as water and stairs can add terrain-specific risk information and influence planner behavior.
  • Risk-aware kinodynamic planning: CVaR-based planning uses a two-stage hierarchy: A* over a 2D grid for a 40 m global plan, followed by shorter-range kinodynamic planning over 8 m.Risk sources and uncertainties are fused into a CVaR costmap used for planning.
  • Ongoing work: NeBula’s ongoing work targets perception-aware planning and more theoretically satisfying incorporation of localization uncertainty through belief-cloud mapping.The goal is to encode pose uncertainty in aggregated pointclouds while retaining computational tractability.

9 Uncertainty-aware Global Planning

NeBula formulates exploration under motion and sensing uncertainty as hierarchical POMDP planning over compact local and global belief representations. PLGRIM combines global guidance with local, risk-aware coverage planning, achieving scalable exploration across environments with different sizes and terrain complexity.

  • Uncertainty-aware planning: NeBula models unknown-environment exploration as a POMDP that jointly considers sequential perception and control outcomes under uncertainty.The formulation represents robot and world states probabilistically and uses receding-horizon planning because full POMDP complexity grows exponentially with planning horizon.
  • Risk-aware objectives: The coverage objective balances information gain against actuation effort and collision risk, allowing mapping and planning to be solved simultaneously.Information gain is expressed as entropy reduction in world coverage, while action cost incorporates effort and risk at the robot’s state.
  • Hierarchical belief representation: PLGRIM scales belief-space coverage planning by maintaining hierarchical Information RoadMaps: a high-fidelity local graph and a sparse global graph encoding large-scale connectivity and frontiers.The local representation captures occupancy, coverage, and traversal risks near the robot, while the global representation supports environments spanning several kilometers.
  • Hierarchical coverage planning: Global Coverage Planning supplies guidance to Local Coverage Planning, which considers information gathering, traversal risk, obstacles, terrain, and robot mobility constraints.LCP either reaches a distant frontier using high-fidelity motion commands or optimizes local coverage when the frontier lies within the local roadmap.
  • Scalability: The Global Coverage Planner’s complexity grows linearly with the number of global-roadmap nodes when each node has a bounded number of nearest neighbors.This bounded-connectivity property supports scaling the planner to large environments.
  • Evaluation: PLGRIM outperforms baseline planners in complex maze and cave environments, where long-horizon, high-resolution planning supports safer exploration through hazardous terrain.In subway environments, NBV becomes less effective as scale grows, while in the cave its deterministic path can drive into rocks; HFE can accumulate locally suboptimal decisions and detours.

10 Multi-Robot Networking

NeBula uses networked, heterogeneous agents and resilient wireless communication to support multi-robot exploration under limited or intermittent connectivity. The system evolved from a ROS 1-based tunnel solution to a ROS 2 inter-agent architecture that improved isolation, bandwidth management, and network stability in the Urban circuit.

  • Architecture and ConOps: NeBula supports static, mobile, and deployable static agents connected through a wireless mesh network.The architecture uses commercial off-the-shelf radios and a hybrid ROS 1/ROS 2 communication middleware.
  • Architecture and ConOps: The subterranean mesh can be extended by deploying communication nodes from robots to build a wireless backbone.This deployment strategy extends communication into environments where direct links to the base station are unavailable.
  • Communication Design: Intra-robot networks use Gigabit Ethernet and are isolated from radio communications through a single bridge computer.That computer runs the ROS 1 core, ROS 1–2 bridge, and inter-agent communication functions.
  • Communication Design: Inter-agent communication uses ROS 2 quality-of-service features to prioritize important data while managing shared, limited bandwidth and communication loss.The system is designed to preserve network stability when robots operate outside radio range.
  • Urban Circuit: ROS 2 inter-agent communication performed better than the previous ROS 1-only system during the Urban competition.Network isolation avoided unintended data flows, kept traffic within the bandwidth budget, and contributed to dynamic-network stability.

11 Mission Planning and Autonomy

NeBula’s mission autonomy coordinates heterogeneous robots through autonomous planning, execution, monitoring, and human assistance. Its architecture supports independent operation during communication gaps while enabling a supervisor to oversee and interact with multiple robots when links permit.

  • Mission Autonomy: NeBula’s mission autonomy plans, reconfigures, and executes multi-robot tasks under unreliable communication, time limits, and resource constraints.The system supports a single human supervisor overseeing more than five heterogeneous robots.
  • Mission Autonomy: The Mission Autonomy architecture combines a Mission Executive, Mission File, autonomy behaviors, IRM Manager, Mission Watchdog, Copilot, and User Interface.These components collectively define mission flow, exchange belief states, monitor communication, assist operators, and expose commands and status.
  • Mission Planning and Scheduling: The Mission Executive steps through the Mission File and triggers robot autonomy behaviors according to mission state.The Mission File specifies the behaviors and their execution flow.
  • Mission Planning and Scheduling: The executive dispatches scheduled tasks, tracks their states, reschedules failed tasks, and relaxes temporal constraints when schedules become infeasible.This makes task execution responsive to failures and changing schedule feasibility.
  • Mission Specification: Mission Files combine high-level robot behaviors into complex mission flows, including parallel execution of five behaviors in the exploration example.The example uses TRACE and BPMN to represent mission structure.
  • Robot Autonomy Behaviors: Autonomy behaviors support frontier movement, communication-node deployment, collision avoidance, heartbeat monitoring, return-to-base, and stair assistance.Comm Drop Autonomy selects a target location to maximize communication coverage while minimizing safety and operational risks.
  • Copilot and Human Interaction: Copilot provides monitoring and assistive capabilities while keeping the human in the loop when communication and cognitive workload allow.Field tests indicated that supervisors could delegate several decisions and focus more on strategically overseeing robot activity and mission progress.

12 Mobility Systems and Hardware Integration

CoSTAR integrates modular payloads and heterogeneous wheeled, legged, and flying robots to match mobility, sensing, computing, endurance, and terrain requirements. The hardware includes sensing, power, diagnostics, and communication-deployment subsystems designed for adaptation across platforms and missions.

  • Robot Platforms: CoSTAR uses heterogeneous robots with complementary mobility, sensing, computing, and endurance capabilities to address varying terrain and mission requirements.The fleet includes wheeled rovers, legged robots, and flying vehicles, each serving different access and mobility needs.
  • Ground and Aerial Roles: Ground robots carry heavier and more capable sensing and processing payloads, while flying robots provide access to vertical shafts, inaccessible areas, and communication-relay opportunities.Ground platforms generally offer longer operational time than flying vehicles because of their battery capacity.
  • Payload Architecture: The NeBula payload comprises the NSP, NPCC, NDB, and NCDS, with modular electronics and software that adapt to platform-specific mechanical and power constraints.This architecture supports reuse across a heterogeneous robotic fleet.
  • Sensing: The NeBula Sensor Package combines selectable LiDARs, cameras, IMUs, encoders, contact sensors, LEDs, radars, gas sensors, UWB, and wireless signal detectors.Custom superstructures provide impact protection for the heterogeneous sensor suite.
  • Communication Deployment: The NCDS lets ground robots carry and autonomously deploy communication radios and static assets to extend the wireless mesh near the environment entrance.Its finite-state-machine activities include startup, calibration, loading, radio deployment, deployment verification, and shutdown.
  • Static Assets: Static comm nodes extend the mesh network, while UWB modules provide auxiliary landmarks and ranging measurements for SLAM and global localization.These assets remain fixed after deployment and support both communication and positioning.
  • Aerial Platforms: A custom 1.5 kg drone provides 12 minutes of flight time while balancing speed, weight, autonomy capability, and sensing.Its payload includes a rotating 2D LiDAR, monocular camera, optical-flow sensor, and single-beam LiDAR.
  • Hybrid Platforms: Rollocopters combine ground rolling with flight over non-rollable terrain and can extend operational lifetime by several folds.The platform is designed to fly when rolling is blocked, then land and resume rolling.

13 Experiments

CoSTAR validated NeBula through simulation, field testing, competition runs, and autonomous cave exploration across heterogeneous robotic platforms. The experiments demonstrated large-scale traversal, mapping, artifact detection, and autonomous operation while exposing localization, sensing, networking, and fault-recovery challenges.

  • Simulation validation: NeBula was validated using multi-fidelity simulations, including Gazebo, flight software-in-the-loop, multi-robot networking, and Monte Carlo dynamics simulators.The setup supported component development, integration testing, and portable local or cloud-based evaluation.
  • Field tests and demonstrations: Heterogeneous platforms demonstrated autonomous exploration across underground mines, tunnels, urban environments, and lava caves.The tested systems included wheeled, tracked, aerial, hybrid, and legged robots operating in challenging terrain.
  • Tunnel Circuit: The Tunnel Circuit team traversed more than 2 km per run, detected 16 artifacts, scored 11 points, and finished second among 11 teams.A longest single-robot drive covered 1.26 km, with a map error under 1% of distance travelled.
  • Lessons learned: Fielding exposed limitations involving featureless-wall localization, communication degradation, poor lighting, restricted sensor field of view, and artifact-detection failures.These failures motivated multimodal sensing, redesigned networking, improved field of view, autonomous fault recovery, and better communication-node configuration.
  • Urban Circuit: The Urban Circuit team detected 25 artifacts, scored 16 points, traversed 2.3 km with four robots, and finished first among 10 teams.The runs included multi-level exploration and multi-robot mapping.
  • Self-organized cave circuit: Cave demonstrations completed fully autonomous runs with zero human intervention, while Spot traversed 400 m on average over four runs.Missions ended after the accessible environment was covered, with some passages limited by low ceilings and cliffs.

14 Lessons Learned

The lessons learned emphasize modular, uncertainty-aware autonomy that integrates heterogeneous sensing, adaptive estimation, scalable mapping, semantic reasoning, and active planning. Resilience depends on anticipating failures, rejecting unreliable measurements, and coordinating perception, action, and communication.

  • Architecture: NeBula’s architecture should be modular and adaptive to heterogeneous robot capabilities and rapidly evolving system requirements.Unified abstractions help integrate differences in mobility, sensing, and computation.
  • State estimation: Predictive failure detection, health checks, and redundant multi-modal estimation can preserve odometry resilience during sensor degradation.HeRO combines redundancy with health monitoring to detect failures and adapt their effects.
  • State estimation: A cascaded estimator rejects anomalous channels before tightly fusing the remaining measurements through probabilistic sensor models.The first layer addresses unknown unknowns; the second handles known unknowns.
  • Positioning and mapping: Complementary sensors and reconfigurable factor-graph SLAM support localization and mapping across perceptually degraded environments and heterogeneous platforms.Loop closures remain important, but computational cost and perceptual aliasing require resilient outlier handling and multiple hypotheses.
  • Positioning and mapping: Distributed computation can improve scalability for large robot teams by allocating workload according to robot capabilities and reducing communication bandwidth.This is presented as an alternative to increasingly expensive centralized multi-robot SLAM.
  • Positioning and mapping: Semantic landmarks such as intersections, stairs, and doors can strengthen loop closures and provide actionable geometric or navigation priors.The paper links semantic augmentation to more robust metric-semantic mapping.
  • Positioning and mapping: Active loop closures guide robot trajectories toward rendezvous or informative locations, coupling perception, action, and communication to reduce localization uncertainty.Trajectory selection substantially affects map reconstruction quality.
  • Traversability: Traversability planning must incorporate perception uncertainty because degraded mapping and state estimation affect risk and cost estimates.Multi-fidelity mapping balances computational constraints and accuracy, while world belief accumulates measurements according to quality.

14.4 Scalable belief-space global planning

Scalable autonomy requires global planning that reasons about uncertain beliefs, sensing value, communication constraints, and coordinated multi-robot missions. The paper also identifies active perception, communication-aware design, and further task-planning research as necessary for robust deployment.

  • 14.4 Scalable belief-space global planning: Global exploration planning must balance belief representation fidelity, planning horizon, computation time, and decision-making under uncertainty.These trade-offs arise in receding-horizon coverage and exploration behaviors.
  • 14.5 Semantic understanding and artifact detection: Artifact detection requires evaluating sensing modalities against heterogeneous platforms’ payload, size, weight, and power constraints.Subterranean artifacts may have visual, thermal, auditory, gas, or radio signatures.
  • 14.5 Semantic understanding and artifact detection: Artifact-detection data remain out-of-distribution because mainstream object-detection datasets do not represent perceptually degraded conditions such as variable lighting and obscurants.This limits direct transfer from standard datasets.
  • 14.5 Semantic understanding and artifact detection: Active perception improves detection confidence by changing viewpoints, selecting robots or sensors, sweeping locally, and adapting camera resolution or input rate.Planning and perception therefore need tight co-design rather than independent optimization.
  • 14.6 Bandwidth-aware communication system design: Bandwidth-aware communication design separates inter-agent and intra-agent traffic, assigns topic-specific QoS, and monitors available bandwidth.The paper calls for continued autonomy in routing and QoS to use communication resources optimally.
  • 14.6 Bandwidth-aware communication system design: Communication-aware planning can deploy relay nodes and use mobile robots to carry data between disconnected network segments.The approach treats networking uncertainty and information value as planning considerations.
  • 14.7 Supervised autonomy to full autonomy: Mission-level autonomy must coordinate large heterogeneous robot teams under strict communication, time, and resource constraints across kilometer-scale environments.The mission objective combines exploration, mapping, and object discovery.
  • 14.7 Supervised autonomy to full autonomy: Future work targets complex task specification, human-machine task distribution, semantic planning, and scheduling under uncertain execution and future events.These areas define an ongoing transition toward fuller autonomy.

15 Conclusion

NeBula is CoSTAR’s uncertainty-aware autonomy solution for resilient decision making in unknown environments. It produced second-place and first-place finishes in the Tunnel and Urban DARPA Subterranean Challenge competitions, respectively, while supporting wheeled, legged, and aerial platforms.

  • 15 Conclusion: NeBula combines uncertainty-aware state estimation, mapping, traversability, planning, communications, and other autonomy modules through synergistic interactions.Its central principle is to quantify and exploit uncertainty throughout the autonomy stack.

16 Glossary: Acronyms

The glossary expands the paper’s major acronyms across autonomy, sensing, mapping, planning, communications, robotics platforms, and system infrastructure.

  • Systems and communications: System and communications terms include ROS, QoS, RF, RSSI, UWB, CHORD, NCDS, and CoSTAR.These terms cover middleware, communication quality, radio technologies, networking systems, and the team name.
  • Autonomy and mapping: Autonomy and mapping terms include NeBula, SLAM, LAMP, LIO, VIO, WIO, HeRO, and related localization and estimation systems.These acronyms cover belief-aware autonomy, simultaneous mapping and localization, and heterogeneous odometry approaches.
  • Planning and decision making: Planning and decision-making terms include POMDP, POMCP, RHC, MPC, RRT, QP, FIRM, GCP, HFE, and STEP.The glossary spans probabilistic planning, receding-horizon control, motion planning, coverage, exploration, and traversability.
Loading 2103.11470v4…