PaperlayerView exact revision on arXiv

Source-linked AI summary

Cognitively-Grounded On-Device Runtime Learning for Ground Robots in Unknown Physical Environments

Yihao Cai, Yanbing Mao, Christian Lebiere

arXiv:2608.28677v1cs.RO

TL;DR

Ground robots need runtime learning in unknown, non-stationary environments where offline training and wireless connectivity are limited, while safety violations can have serious consequences. CogRun combines cognitively grounded learning, a dedicated safety agent, and coordination on edge-AI devices. Experiments on a quadruped robot and an off-road vehicle show safe, efficient runtime learning with improved safety, stability, and task performance.

  • Problem

    Ground robots face unknown, non-stationary environments, limited connectivity, inefficient experience replay, and safety-critical runtime-learning requirements.

  • Method

    CogRun combines cognition-driven experience sampling and safety-aware RL–IBL blending with a non-learning Rational-Agent and Coordinator on edge-AI devices.

  • Results

    Experiments on a quadruped robot and an off-road autonomous vehicle demonstrate safe, efficient runtime learning with enhanced safety assurance, experience efficiency, stability, and task performance.

  • Takeaways & Limitations

    CogRun enables robots to continuously interact with complex, unknown physical environments while learning safely and entirely on-device.

Abstract

from arXiv · show

This paper presents \ul{CogRun}, a framework that enables safety-critical ground robots to perform cognitively-grounded runtime learning entirely on edge-AI devices in unknown physical environments, without prior maps or perceptual knowledge. CogRun consists of three components: a Learning-Agent, a Rational-Agent, and a Coordinator. The Learning-Agent is novel in cognitive-neural learning architecture, which featurs dedicated replay buffers, cognition-driven experience sampling, and a safety-aware action blending of actor-critic reinforcement learning (RL) with instance-based learning (IBL). The Rational-Agent is a non-learning module that complements the Learning-Agent by exclusively handling safety-critical functions, while the Coordinator manages interactions between the two agents to promote safe and efficient runtime learning. CogRun's full autonomy stack (i.e., perception, learning, and control) on edge-AI devices eliminates dependence on wireless communications, enabling broader applications in challenging environments with limited or no connectivity. Experiments on a quadruped robot in real-world wild forests and on an off-road autonomous vehicle in a simulated wild forest demonstrate that CogRun enables safe and efficient runtime learning, allowing robots to safely and continuously interact with the physical world for enhancing task performance in complex, unknown environments.

I. INTRODUCTION

CogRun addresses runtime learning challenges for ground robots in unknown, non-stationary, communication-limited environments. It combines on-device execution, safety-oriented learning, and cognition-driven replay to improve runtime-learning outcomes.

  • Ground robots must operate in unstructured, dynamic, and unpredictable environments beyond the structured settings emphasized by prior AI-robotics advances.
  • Offline train-then-deploy systems remain vulnerable to domain and sim-to-real gaps in non-stationary, unknown environments that cannot be fully characterized before deployment.
  • Cloud–edge learning is impractical when remote environments provide unreliable, intermittent, or unavailable wireless communication and communication latency is consequential.
  • Conventional replay can inefficiently use real-world data because it ignores experience relevance to the robot’s current state, especially in unseen and non-stationary environments.
  • CogRun provides safe runtime adaptation, a complete low-latency edge-AI stack, and cognition-driven prioritization of experiences relevant to current runtime states.
  • Experiments on two representative tasks show that CogRun significantly enhances safety assurance, experience efficiency, and runtime-learning task performance.

III. METHODOLOGY: COGRUN FRAMEWORK

CogRun coordinates a learning component with a non-learning safety component for runtime learning in physical environments. The Coordinator monitors system states and switches control when safety requires intervention.

  • CogRun comprises a Learning-Agent, Rational-Agent, and Coordinator that are tightly coupled for runtime learning.
  • Learning-Agent: The Learning-Agent uses actor–critic reinforcement learning with cognition-driven batch sampling and real-time action blending with instance-based learning.
  • Rational-Agent: The Rational-Agent is a non-learning module dedicated exclusively to safety-critical functions and acts as a safety fallback when learned actions risk violations.
  • Safety conditions: The safety set constrains system states using predetermined conditions covering examples such as lane tracking, velocity regulation, and collision avoidance.
  • Coordinator: The Coordinator monitors real-time states and manages interactions between the two agents according to conditions defining the safety set.
  • Coordinator: When necessary, the Rational-Agent temporarily takes control, shares safety-critical experience, and returns control after the state re-enters the safety set.

B. Rational-Agent Component

The Rational-Agent is a non-learning safety fallback that intervenes during runtime learning, while dedicated replay buffers combine task-oriented and safety-assurance experiences for safer, more efficient learning.

  • B. Rational-Agent Component: The Rational-Agent focuses exclusively on safety-critical functions and intervenes when the Learning-Agent risks violating safety.It uses safety-assured methods and can operate under unknown or uncertain physical conditions.
  • B. Rational-Agent Component: CogRun maintains separate replay buffers for Learning-Agent trial-and-error experiences and Rational-Agent safety-assurance experiences.The Learning-Agent stores task-oriented transitions, whereas the Rational-Agent stores transitions generated during safety interventions.
  • B. Rational-Agent Component: Each runtime-learning mini-batch samples sub-batches from both dedicated buffers.The combined batch incorporates safety-assurance experiences into Learning-Agent updates.
  • B. Rational-Agent Component: Incorporating safety-assurance experiences helps the Learning-Agent learn safe strategies while reducing risky trial-and-error exploration.The paper connects this design to improved sample efficiency during runtime learning.

2) Cognition-Driven Batch Sampling:

CogRun ranks experiences relative to the robot’s current state using similarity, frequency, and recency, then retrieves the highest-ranked samples from dedicated replay buffers.

  • 2) Cognition-Driven Batch Sampling:: Cognition-driven sampling addresses replay inefficiency by selecting experiences according to their relevance to the robot’s current operating context.The method is motivated by context-dependent human memory retrieval.
  • 2) Cognition-Driven Batch Sampling:: Similarity captures contextual resemblance, while frequency and recency prioritize familiar and recently acquired experiences.The paper links these factors to structural recall, repeated exposure, and adaptation in dynamic environments.
  • 2) Cognition-Driven Batch Sampling:: For each dedicated buffer, the retrieval step selects the top-Kσ experiences according to their batch scores.Learning and Rational buffers are sampled independently with desired sub-batch sizes Kσ.
  • 2) Cognition-Driven Batch Sampling:: The ranking score combines similarity, frequency, and recency measures for each stored experience relative to the current experience.Similarity can use inverse Euclidean distance, while frequency and recency reflect repeated occurrence and elapsed time.
  • 2) Cognition-Driven Batch Sampling:: The frequency component increases with the number of experiences in a family, while the recency component increases as elapsed time decreases.The memory decay factor λ modulates the recency contribution.

3) Safety-Aware RL–IBL Blending:

CogRun derives an experience action from rational experiences and blends it with the actor’s action using a state-dependent parameter designed to account for safety.

  • 3) Safety-Aware RL–IBL Blending:: The safety-aware RL–IBL mechanism blends actions from rational experience instances to construct an experience action.This applies instance-based learning to retrieved safety-assurance experiences.
  • 3) Safety-Aware RL–IBL Blending:: The Learning-Agent combines the experience action with the actor-network action through a state-dependent blending parameter η(s).The parameter lies between 0 and 1 and controls the contribution of the two actions.
  • 3) Safety-Aware RL–IBL Blending:: η(s) is designed to be safety-aware so that a safety-assured experience action can strengthen the terminal learning action.Its computation uses the real-time system state and the safety-set matrices C and c.

D. Operation Practice on Edge-AI Devices

CogRun uses edge-AI hardware for local execution, assigning high-frequency safety control to the CPU and lower-frequency learning to the GPU.

  • D. Operation Practice on Edge-AI Devices: Edge-AI devices support lightweight local models with direct real-time interaction, preserving temporal consistency and low latency.The implementation uses an NVIDIA Jetson with an ARM-based CPU and GPU.
  • D. Operation Practice on Edge-AI Devices: The Rational-Agent runs on the ARM-based CPU to improve energy efficiency and thermal performance for mobile autonomous systems.The design targets long-duration on-device runtime learning and safety-critical deployment.
  • D. Operation Practice on Edge-AI Devices: The Coordinator and Rational-Agent operate at 200–1000 Hz on the CPU, while the Learning-Agent runs at 10–20 Hz on the GPU.This decoupled multi-rate design separates learning from safe control while preserving real-time execution.

IV. EXPERIMENT

The experiments evaluate CogRun on a quadruped robot and an off-road autonomous vehicle.

  • CogRun is evaluated on a quadruped robot and an off-road autonomous vehicle.

A. Quadruped Robot in Real Unknown Wild Forests

The quadruped navigation experiment tests runtime learning in an unknown wild forest with explicit safety constraints and multiple performance objectives. CogRun combines onboard hardware, reinforcement learning, and safety-oriented reward components to improve navigation during interaction.

  • Experimental environment: The quadruped must navigate approximately 30 meters through an unknown forest containing dead zones, swales, and leaf-covered holes.The safety set requires at least 1.2 meters of obstacle distance and constrains robot height around 0.3 meters.
  • Learning-Agent: The Learning-Agent uses CoM state and visual features as observations and sends twist commands to a low-level motion controller.
  • Results: CogRun strictly avoids collisions and falls while continuously improving navigation through interaction with the unknown environment.The robot reduces time to goal and achieves stable, efficient navigation within tens of learning episodes.
  • Results: CogRun progressively produces smoother trajectories, higher rewards, and faster goal reaching than the fixed Rational-Agent.The Rational-Agent remains unchanged after deployment, providing a non-learning comparison capability.

2) Runtime Learning for End-to-End Vision-to-Action:

The vision-to-action experiment evaluates CogRun across unknown and changing physical environments while preserving safety during runtime learning. Its reward adds a smoothness penalty, and experiments compare it with fault-tolerant reinforcement-learning frameworks across repeated trips and transferred environments.

  • Reward and control: The vision-to-action task adds a jerk penalty to discourage abrupt changes in consecutive actions.Its associated weight is ξ6 = 0.0002, while other reward components and coupled weights remain unchanged.
  • Comparison study: CogRun is compared with Real-DRL, Runtime Learning Machine, and Model Predictive Shielding in an unknown forest path.The comparison environment contains randomly distributed rounded stones and packed dirt.
  • Results: Over 20 consecutive trips or episodes, CogRun substantially improves safety and learning stability relative to existing fault-tolerant reinforcement-learning frameworks.The task-completion definition requires no collision, no loss of balance, no dead-zone encounter, and reaching the goal.
  • Non-stationary environments: The learned policy transfers from Environment 1 to Environments 2 and 3 after five episodes without reinitialization.The experiments use forest floors with stones, dirt, leaves, branches, twigs, and cobblestones.

B. Autonomous Vehicles in Simulated Unknown Wild Forests

The off-road vehicle study evaluates CogRun across robotic platforms and tests how cognition-driven sampling and safety-aware blending affect runtime navigation learning in randomized unknown forests.

  • Experimental Setup: The experiment evaluates CogRun’s generalizability on a three-wheel swerve-drive vehicle navigating randomized forests with rocks, trees, and varied terrain.The study also includes an ablation of its cognitive mechanisms.
  • Ablation Design: Three configurations compare cognition-driven sampling, cognition plus safety-aware RL-IBL blending, and conventional fault-tolerant RL with random sampling.The blending configuration builds on cognition-driven sampling, so no Random+Blending condition is included.
  • Navigation Task: The navigation safety set requires minimum distance d_min ≥ 0.55 m and pitch and roll rates no greater than 1 rad/s.The Rational-Agent uses Interactive-FAR while the Learning-Agent learns a navigation policy during runtime.
  • Results: Cognition consistently outperforms Random, while Cognition+Blending achieves the highest rewards across the ablation results.The stepwise rewards aggregate five random seeds, each with 60 episodes, and are reported with 95% confidence intervals.

V. CONCLUSION AND DISCUSSION

CogRun combines cognitively grounded learning with on-device execution for ground robots in unknown environments. Experiments on a quadruped robot and an off-road vehicle support its effectiveness and generalizability, while future work targets improved experience management.

  • Contributions: CogRun uses dedicated replay buffers, cognition-driven batch sampling, and safety-aware RL-IBL blending for on-device runtime learning.The framework targets safer and more efficient learning in unknown physical environments.
  • Evaluation: Experiments on a quadruped robot and simulations on an off-road autonomous vehicle demonstrate CogRun’s effectiveness and generalizability.The evaluations cover real-world and simulated robotic platforms.
  • Future Work: Future work will investigate principled experience ranking, memory decay, and improved measures of frequency, recency, and similarity.These directions aim to advance cognitively grounded runtime learning for physical AI robotic systems.
Loading 2608.28677v1…