Source-linked AI summary
Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks
Khadija Shaheen, Muhammad Abdullah Hanif, Osman Hasan, Muhammad Shafique
TL;DR
Real-world autonomous systems face non-stationary data, making frozen models inadequate and continual adaptation vulnerable to catastrophic forgetting. The paper surveys online continual-learning methods, their resource requirements, challenges, and implementations across autonomous systems. It concludes that practical deployment must balance adaptation with limited computation, memory, energy, and data-collection resources.
Problem
Frozen models trained on fixed datasets cannot effectively adapt to non-stationary real-world data, while sequential learning can cause catastrophic forgetting.
Method
The paper surveys online continual-learning algorithms and analyzes their challenges, complexity, resource requirements, and applications in autonomous systems.
Results
The survey organizes current methods and autonomous-system applications around strategies for reducing forgetting under limited supervision and computing resources.
Takeaways & Limitations
Real-world continual learning must account for constrained hardware, energy autonomy, data sampling, and the practical demands of autonomous deployment.
Takeaways & Limitations
Autonomous agents have limited memory and computation, require energy autonomy, and face challenges gathering useful non-redundant training data.
Abstract
from arXiv · showhide
Continual learning is essential for all real-world applications, as frozen pre-trained models cannot effectively deal with non-stationary data distributions. The purpose of this study is to review the state-of-the-art methods that allow continuous learning of computational models over time. We primarily focus on the learning algorithms that perform continuous learning in an online fashion from considerably large (or infinite) sequential data and require substantially low computational and memory resources. We critically analyze the key challenges associated with continual learning for autonomous real-world systems and compare current methods in terms of computations, memory, and network/model complexity. We also briefly describe the implementations of continuous learning algorithms under three main autonomous systems, i.e., self-driving vehicles, unmanned aerial vehicles, and urban robots. The learning methods of these autonomous systems and their strengths and limitations are extensively explored in this article.
1 Introduction
Continual learning addresses the inability of static deep models to adapt to non-stationary real-world data while retaining prior knowledge. This survey reviews online methods and their applicability, resource demands, and challenges in autonomous systems.
- Static deep neural networks struggle with changing environments because they are trained offline on fixed datasets and do not adapt after deployment.
- Continual learning processes sequential streaming data as tasks, but standard networks often suffer catastrophic forgetting when learning new tasks.
- Real-world adaptation requires balancing plasticity for new knowledge with stability for retaining prior knowledge, creating the stability-plasticity dilemma.
- Online continual learning removes task-identity assumptions and supports adaptation from potentially infinite, non-i.i.d. streams with sparse supervision.
- The survey connects continual learning with autonomous systems by comparing algorithms, resource requirements, limitations, and applications in self-driving cars, unmanned aerial vehicles, and urban robots.
2 Continual Learning Strategies
Continual learning strategies address forgetting through replay, regularization, and architectural changes across different task scenarios. These approaches differ in how they preserve prior knowledge, adapt models, and manage task or class structure.
- Replay: Replay methods retain past knowledge by storing and replaying representative samples from previously learned tasks.
- Regularization: Regularization methods protect important previous-task weights with additional loss terms, but excessive regularization can saturate the model after several tasks.
- Architectural approach: Architectural methods isolate task-specific parameters by adding modules for new tasks while keeping previously learned parameters unchanged.
- Class-incremental learning: Class-incremental learning adds new classes while retaining a single-headed output space and requiring the model to infer current and previously observed classes.
- Continual learning scenarios: Task-incremental scenarios use distinct output spaces and may assign separate output heads to different tasks.
3 Online Continual Learning Strategies
Online continual learning methods support task-agnostic adaptation from sequential data while addressing forgetting under computational and memory constraints.
- Online continual learning uses task-agnostic techniques for continuous adaptation from practically unlimited sequential data with reduced resource demands.Unlike conventional task-based continual learning, it does not require task identity to be available.
- Task-free Memory Aware Synapses: Memory Aware Synapses estimates parameter importance online and protects important weights, updating the DNN at detected loss-surface peaks rather than every time step.Stable plateaus are used to compute importance weights, while peaks indicate input-distribution changes.
- Latent Replay: Latent Replay stores intermediate-layer activations instead of raw images, trading a slight accuracy decrease for improved storage and computational efficiency.The method supports incremental class learning without forgetting and enables continual learning on edge-only devices.
- Continual Unsupervised Representation Learning: CURL infers task ambiguity without task labels or boundaries by learning Gaussian-mixture representations, expanding its task-inference head when poorly modeled samples accumulate.Deep generative replay is used before expansion to consolidate new knowledge and address catastrophic forgetting.
- Maximally Interfered Retrieval: MIR retrieves maximally interfered samples rather than random buffer samples, selecting correlated examples that produce effective online training batches.It is evaluated with experience and generative replay, and a hybrid compressed-replay approach is proposed for a challenging CIFAR-10 setting.
- Other online strategies select representative samples through gradient constraints, expand expert networks with distributional variation, or adapt learning rates using parameter importance.The reviewed methods also include meta-learning that searches for generalized parameters balanced across tasks.
4 Continual Learning for Real-World Autonomous Agents
Continual learning enables autonomous agents to adapt to non-stationary environments, but real-world deployment is constrained by hardware, data integrity, sampling, labeling, and stability requirements.
- Real-world continual learning must adapt to changing environments while retaining previously learned knowledge across sequential experiences.Catastrophic forgetting can degrade later learning when prior data are unavailable.
- Hardware limits memory, computation, and energy, while learning failures can damage infrastructure and delay experiments.
- Autonomous agents must gather useful data from unknown environments, while avoiding redundant data during lifelong learning.Reinforcement learning, curiosity, and self-supervision are identified as mechanisms for exploration and data collection.
- Online labeling is tedious, so autonomous systems use external supervision mainly for safety-critical conditions and may exploit rewards or self-supervision.
- Continual learning spans supervised, unsupervised, and reinforcement-learning paradigms distinguished by their supervision signals.Supervised learning uses labels, unsupervised learning models input distributions, and reinforcement learning uses rewards.
5.1 Self-Driving Cars
Self-driving systems use continual learning to adapt perception, control, and decision-making to evolving road conditions. Reviewed approaches include replay, multimodal and reinforcement learning, adaptive control, and cloud-supported training.
- Self-driving vehicles require self-adaptive models because frozen systems cannot cover evolving, safety-critical driving scenarios.The systems map camera, lidar, inertial, and other sensory data to control outputs.
- Lane keeping: CNN lane-keeping uses mini-batch Adam updates and divided buffers storing maneuver-specific data to reduce noise and training bias online.The method adapts to unseen road conditions during human-driven training.
- Lane keeping: Symbiotic learning combines demonstration-based supervised learning, instantaneous reinforcement learning, and self-reinforcement for multimodal lane following.Its architecture maps road images to throttle and steering values through multimodal channel modeling.
- Lane keeping: Reinforcement-learning approaches formulate driving as policy optimization, including actor-critic control and hierarchical meta-policies learned from previously trained policies.The reviewed methods target realistic vehicle control and reduced exploration.
5.1.2 Scene Recognition
Scene-recognition methods incrementally detect novel or abnormal states under changing conditions while controlling model complexity and computational demands.
- Unseen-scene recognition is safety-critical because autonomous vehicles must classify novel conditions reliably without always relying on external supervision.
- Deep Rule-based classification combines a pretrained CNN with self-updating fuzzy rules and uses fewer computational resources with faster convergence than conventional deep networks.The rules represent local data-density peaks across class-specific prototypes.
- Free-energy minimization detects abnormalities from prediction error and creates new dynamic Bayesian models to incorporate detected novelty.The method was evaluated on perimeter-monitoring scenarios including obstacle avoidance and U-turns.
- Gaussian-process regression segments vehicle motion into spatial zones modeled by Kalman filters, using prediction error to measure abnormality.
- Online adaptation is needed because vehicle dynamics change with road conditions and internal uncertainties, requiring neural-network control models to update continuously.LWPR2 uses pseudo-rehearsal to reduce the computational cost of prediction while adapting input-output mappings.
5.1.5 Vehicle Tracking
Vehicle-tracking methods adapt trajectory predictions and decision systems to driver behavior, pedestrians, uncertainty, and unexpected events. The reviewed approaches trade model growth, communication load, and intervention against prediction or control robustness.
- Trajectory prediction must represent uncertain human behavior and adapt generalized models to individual drivers or changing urban interactions.
- Bayesian recurrent networks and PS-GAIL adapt pretrained trajectory models online using particle filtering or recursive least-square parameter updates.PS-GAIL keeps the feature extractor fixed and updates later parameters with a one-step linear method requiring fewer computations.
- SILA represents pedestrian trajectories as motion primitives and merges similar primitives while adding novel ones, slowing model growth during incremental learning.
- Connected Vehicle: Active learning over V2V communication selects diverse labeled data, achieving high accuracy in sample and data modes at the cost of high network load.Vehicles can exchange labels, data, or both through three communication modes.
- Driver-monitoring: Cloud-based driver monitoring triggers cloud retraining after performance drops, allowing computational complexity to be offloaded from the vehicle.
- UAIL estimates output uncertainty at sub-optimal states, transfers control to the driver, and gathers useful data while predicting infractions before many bad actions occur.
5.2 Unmanned Aerial Vehicles
UAV continual-learning methods adapt controllers, classifiers, trackers, and navigation systems to unseen trajectories, changing dynamics, non-stationary streams, and partially observable environments. The surveyed approaches combine online updates with fuzzy rules, evolving architectures, double-state inputs, and online fine-tuning.
- 5.2.1 Trajectory tracking: Online learning enables pre-trained DNN controllers to adapt UAV control to unseen trajectories and varied operating conditions.Trajectory errors are mapped to control outputs, and weights are updated when unseen trajectories appear.
- 5.2.2 Control of unknown system dynamics: DFNN controllers provide faster response times and effectively reduce tracking errors in real-time UAV applications.The controller combines deep neural networks with fuzzy logic and adapts to unknown dynamics using expert knowledge encoded as fuzzy rules.
- 5.2.3 Classification: Single-pass McSIT2RFNN learning from scratch uses fewer computing and memory resources for real-time non-stationary data streams.Its recurrent memory handles temporal dynamics, while a single fuzzy-rule layer adds rules for novel concepts and removes inactive ones.
- 5.2.4 Deep Reinforcement Learning for Navigation and Exploration: ED-DQN combines raw images with positional maps and successfully adapts UAV search-and-rescue behavior across unseen weather conditions.Images identify cluttered areas while maps estimate target locations during partially observable exploration.
- 5.2.5 Visual tracking: Online CNN fine-tuning adapts visual tracking to changed target appearances and improves tracking performance.Locally connected layers learn robust features, while focal loss addresses hard examples and class imbalance.
5.3 Urban Robots
Urban-robot continual learning covers control, manipulation, open-world recognition, cross-modal perception, navigation, and personalized motion classification. The surveyed methods use incremental regression, fine-tuning, rehearsal, distillation, self-supervision, and shared task representations to adapt to changing environments.
- 5.3.1 Autonomous Underwater Vehicles Dynamic Control: Incremental SVR models AUV dynamics using only support vectors, resulting in lower computational complexity under varying payloads and disturbances.The method performs online adaptation by learning the input-output relationship of the AUV dynamics.
- 5.3.2 Vision-Based Manipulation: Fine-tuning with reinforcement learning substantially improves vision-based manipulation over novel tasks without requiring real-world interaction during policy learning.Training samples are drawn equally from base-task and new-task data after initializing the new policy from the pre-trained policy.
- 5.3.3 Web Aided Deep Open World Recognition(OWR): Deep OWR recognizes unseen objects incrementally by combining web mining with rehearsal and distillation-based knowledge preservation.Web mining supplies labels and additional training data, while the rehearsal buffer and distillation loss help maintain previous knowledge.
- 5.3.7 An Autonomous Self-Supervised Learning-Based Navigation System: Self-supervised navigation outperforms other navigation policies by generalizing to novel environments with a small amount of training data.The system uses off-policy real-world data and learns an efficient policy without human supervision.
- 5.3.8 Personalized Online Learning of Whole-Body Motion Classification: Personalized online learning outperforms the average user model while using fewer computations and attaining high accuracy.The model adapts to individual motion behavior in real time for wearable healthcare devices.
6 Other Applications
Other applications apply continual learning to anomaly detection, energy forecasting, stereo estimation, imbalanced fault diagnosis, event-based recognition, and IoT feature learning. Their mechanisms include error-driven updates, edge training, modular adaptation, active learning, network expansion, and regularization.
- 6 Other Applications: ISTL adapts anomaly detection to evolving normal behavior by combining spatio-temporal reconstruction error, fuzzy aggregation, and human feedback.Samples are selected for continual learning according to prediction error, with human feedback confirming anomalies.
- 6 Other Applications: Incremental LSTM forecasting learns new energy scenarios by updating from prediction errors when exact load values arrive.The LSTM retains temporal correlations while forecasting and incrementally learning incoming data.
- 6 Other Applications: MADNet reduces learning time and computation complexity by independently optimizing high-loss network modules with shorter back-propagation.The network is divided into non-overlapping horizontal segments, and each iteration trains one module using prediction loss.
- 6 Other Applications: Incremental fault diagnosis expands a hierarchical neural network using informative data selected through an active-learning feedback loop.The diagnosis module first generates balanced fault inputs before informative samples drive incremental learning.
- 6 Other Applications: Event-based incremental broad learning dynamically expands feature and enhancement nodes while SVD prevents redundant additions.The flat network is designed for high-speed learning as new data arrives.
7 Looking Ahead
The survey frames online continual learning as necessary for task-agnostic infinite data streams and connects algorithms with practical autonomous-system use cases. It recommends evaluating realistic scalability, resource constraints, adaptivity, and forgetting, while noting that robust real-world benchmarks remain underdeveloped.
- 7 Looking Ahead: Online continual learning addresses real-world streams by learning from infinite sequential data without task identity.The survey contrasts this setting with task-based learning that assumes known task boundaries and offline convergence training.
- 7 Looking Ahead: Many use cases rely on image-processing experiments or virtual environments, so their expected performance in realistic scenarios requires clarification.The survey identifies practical use cases as a basis for comparing approaches across real-world settings.
- 7 Looking Ahead: The survey recommends problem-agnostic, adaptive, online, resource-conscious frameworks with graceful forgetting and bounded network growth.It also recommends evaluating scalability across many tasks and conducting ablations of continual-learning components.
8 Conclusion
The survey reviews continual-learning techniques for real-world autonomous systems, including self-driving cars, unmanned aerial vehicles, and urban robots, while identifying unresolved evaluation and deployment challenges.
- The survey presents a comprehensive overview of continual-learning techniques for real-world autonomous systems and their practical use cases.
- It covers applications in self-driving cars, unmanned aerial vehicles, and urban robots to categorize and compare emerging approaches.
- Many proposed algorithms have rarely been assessed in real-world settings, motivating robust benchmarks that represent actual scenarios.
Declarations
The authors report no funding, no conflicts of interest, and no ethical-approval requirement; they also document author contributions.
- No funding was received for preparing the manuscript.
- The authors declare that they have no conflict of interest.
- The four authors contributed across conceptualization, writing, reviewing, editing, and manuscript revisions.
- The paper does not report research requiring ethical approval.