Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks

• Offline-trained models remain static and struggle with changing data, while repeating training after distribution shifts is computationally intensive for resource-constrained systems.Real-world autonomous agents encounter non-stationary data as new data or tasks become available over time.Source: §1 · p. 2
• Sequential task learning can cause catastrophic forgetting, and autonomous systems must balance plasticity for new knowledge against stability for retaining prior knowledge.Excessive plasticity risks forgetting prior information; excessive stability makes learning sequential tasks more difficult.Source: §1 · p. 2
• Many continual-learning algorithms use synaptic regularization, structural plasticity, or memory replay, but task-based methods often assume i.i.d. data and known task identifiers.Source: §1 · pp. 2–3
• Online continual learning addresses these assumptions by learning from infinite data streams without task identities, including streams with gradually varying distributions and sparse supervision.Source: §1 · p. 3
• The survey covers continual-learning applications in self-driving cars, unmanned aerial vehicles, and urban robots, including their frameworks, algorithm or network complexity, and computing resources [27] [36] [44] [48].Source: §1 · p. 3
• The survey reviews online methods’ adaptability, computational demands, supervision needs, strengths, and limitations, and assesses how methods developed on static datasets or simulations apply to real-world systems.It also identifies gaps in prior reviews concerning algorithm comparisons, experimental frameworks, training methods, network complexity, and computing resources [55].Source: §1 · p. 3

2.1.1 Rehearsal

• Rehearsal replays selected past-task samples to retain knowledge; iCARL selects representative exemplars but requires substantial storage, while DGR and FearNet [60] generate past examples.DGR interleaves generated past data with new data; FearNet uses short- and long-term memory with a module that selects which memory to use for prediction.Source: §2.1.1 · p. 4

2.1.2 Regularization

• Regularization protects important old-task weights with additional loss terms: EWC uses a Fisher matrix but can saturate after several tasks, while distillation, transfers teacher predictions to a smaller student.Distillation’s forward transfer can reduce the size of an extremely large network.Source: §2.1.2 · pp. 4–5

2.1.3 Architectural

• Architectural methods isolate task-specific parameters by adding modules while preserving old ones; PNN [64] learns lateral connections and DEN [65] expands capacity as needed, but network parameters can grow uncontrollably.DEN uses task relatedness to partially retrain old-task networks and determine when to increase capacity.Source: §2.1.3 · p. 5

2.2.1 Domain-incremental learning (Domain-IL)

• Domain-IL keeps the task and output space fixed while input distributions shift; task identity is unknown at test time, and one single-headed model must retain old-task and learn new-task performance.Real-world shifts can arise from new backgrounds, noise, or illumination.Source: §2.2.1 · p. 5

2.2.2 Class-incremental learning (Class-IL)

• Class-IL incrementally adds distinct classes without retaining prior class data; a single-headed model shares one label space and must infer the current and previously observed tasks and classes.The output layer has one node per total class.Source: §2.2.2 · pp. 5–6

2.2.3 Task-incremental learning (Task-IL)

• Task-IL assigns each task a distinct output space and uses a separate output head per task, requiring the task identifier at evaluation to select the corresponding head.Tasks may differ in output dimensions and semantic concepts, such as a five-class classification task followed by single-value regression.Source: §2.2.3 · p. 6

3.1 Task-free Memory Aware Synapses; 3.7 Uncertainty-Guided Continual Learning With Bayesian Neural Networks

• Task-free MAS estimates important weights during stable loss regions and updates at peaks, preventing forgetting without added memory; UCB lowers learning rates for more important parameters.UCB’s pruning variant uses binary masks and more memory, while the version without pruning needs no additional memory but allows slight forgetting.Source: §3.1 · p. 7, §3.7 · p. 12

3.2 Latent Replay

• Latent Replay stores intermediate-layer activations instead of raw images, reducing storage and computation for edge-device learning at the cost of a slight accuracy decrease.It keeps the latent representation stable by slowing training below that layer, concatenates stored activations with current inputs, and stops backpropagation before the latent layer; it supports class-incremental learning without forgetting.Source: §3.2 · p. 8

3.3 Continual Unsupervised Representation Learning

• CURL learns task-specific representations without task labels or boundaries, dynamically expands its task-inference head, and uses generative replay to address forgetting.Its Gaussian-mixture task-inference heads encode clusters and their variations, while a decoder reconstructs inputs; poorly modeled samples trigger new components after a buffer threshold.Source: §3.3 · pp. 8–9

3.4 Maximally Interfered Retrieval

• MIR retrieves maximally interfered samples rather than random buffer samples, aiming to reduce loss on new inputs without increasing loss on the most interfered past samples.It is evaluated with experience and generative replay, and proposes a CIFAR-10 hybrid using an offline-trained autoencoder to compress incoming samples when neither replay strategy is feasible.Source: §3.4 · p. 10

3.5 Gradient based sample selection

• Gradient-based sample selection populates a task-agnostic replay buffer with representative streaming samples, using gradient constraints to maximize sample diversity.It offers inter-quadratic programming and a lower-complexity greedy option that scores samples by gradient cosine similarity and replaces buffer samples accordingly.Source: §3.5 · pp. 10–11

3.6 Neural Dirichlet Process Mixture Model

• CN-DPM expands a pool of gated neural experts according to data variation without known task boundaries, using short-term memory to gather samples before training a new expert.The method controls model complexity during dynamic expansion.Source: §3.6 · pp. 11–12

3.8 Incremental Task-Agnostic Meta-learning

• iTAML meta-learns a task- and model-agnostic generalized solution that identifies task identity and supports fast adaptation to task-specific parameters.Its momentum-based training mitigates catastrophic forgetting and data imbalance, and the method is described as learning infinitely large-scale streams with high adaptation and inference speed.Source: §3.8 · pp. 12–13

3.9 Online Aware Meta-Learning

• OML separates representation learning from prediction: online task-specific updates modify the prediction network, while meta-objective updates preserve generalized representations.It seeks generic representations effective across distributions and able to accelerate future online learning without catastrophic interference.Source: §3.9 · pp. 13–14

4.1 Scope of Continual Learning for Real-world Autonomous Systems

• Many CL approaches focus on image-processing experiments or virtual environments rather than direct application to autonomous real-world systems.Agents also draw on sensor experiences, including images, sound, and depth, to form representations useful for CL.Source: §4.1 · pp. 14–15

4.2.1 Hardware

• Real-world CL experiments are constrained by fragile, compact hardware, limited memory and computation, and the need for energy autonomy.Learning failures can damage infrastructure and cause long delays in experiments; agents must sustain long experiments without manual recharging or power failure.Source: §4.2.1 · p. 15

4.2.2 Stability

• A corrupted experience can degrade memory and subsequent learning, potentially collapsing the system when earlier data is unavailable for recovery.CL retains experience knowledge through memory-handling techniques, making corrupted sequential data difficult to restore.Source: §4.2.2 · p. 15

4.2.3 Data Sampling

• Agents must gather useful data in unknown, unlimited environments [66], using RL-generated goals or curiosity and self-supervision [68][69], while avoiding redundant data.Data aggregation is a major lifelong-learning challenge because redundant data is undesirable for CL strategies.Source: §4.2.3 · p. 15

4.2.4 Data Labelling

• Online training labels are tedious to obtain; safety-critical settings use external expert supervision, while RL rewards and sensor-based self-supervision can provide learning signals.Self-supervision uses sensor data to define abstract rules about the environment.Source: §4.2.4 · pp. 15–16

4.3.1 Supervised Learning

• Supervised CL learns from sequential labeled streams to map inputs to labels across the sequence, and its use reduces learning-algorithm complexity.Supervised learning uses labeled training data containing input-output knowledge.Source: §4.3.1 · p. 16

4.3.2 Unsupervised Learning

• Unsupervised CL can update generative models after distribution shifts and may build increasingly powerful representations that are fine-tuned with environmental feedback.Its goal is to develop suitable surrogates and meaningful self-supervised signals for robust, adaptive representations; generative replay is a common use.Source: §4.3.2 · p. 16

4.3.3 Reinforcement Learning

• RL is a CL scenario because complex environments reveal data sequentially, and RL uses parallel agents, replay memory, and Fisher Matrix methods akin to CL strategies.TRPO's Fisher Matrix improves learning similarly to EWC; the section argues that improving CL can also improve RL performance.Source: §4.3.3 · p. 16

5.1 Self-Driving Cars

• Self-driving cars map onboard sensor data to control, but evolving environments and safety and comfort demands call for methods that detect and adapt to novel driving scenarios on the fly.The section surveys continual-learning frameworks for self-driving cars and summarizes them in Table 2.Source: §5.1 · p. 17

5.1.1 Lane keeping

• For visual lane keeping, online learning adapts camera-based steering control to diverse road conditions absent from training, with a few minutes of real-time learning for a new road.The system takes road images from an in-vehicle camera and outputs a steering control value.Source: §5.1.1 · p. 17
• CNN-based steering control learns unseen road conditions from human driving, using mini-batch Adam to address noise and maneuver-divided buffers to reduce bias from successive similar data.The buffers separately store left-turn, right-turn, and straight-driving data; the divided-buffer method is described as producing stable, fast, efficient real-time learning.Source: §5.1.1 · pp. 17, 20, Figure 10 · p. 20, Figure 11 · p. 20
• For trajectory prediction, a parameter-sharing generative adversarial learning model adapts online [39]; Monte Carlo dropout estimates output-control uncertainty to detect and aggregate the most useful data at suboptimal states.Source: §5.1.1 · p. 18
• Symbiotic learning combines learning by human demonstration, instantaneous reinforcement, and self-reinforcement from predictive coding and human reward after the vehicle switches to autonomous mode.Its self-reinforcement relies on a visual constancy assumption about the road to generate performance feedback.Source: §5.1.1 · p. 20
• For lane following, model-free deep reinforcement learning [26] formulates driving as an MDP and uses DDPG actor-critic learning to maximize distance traveled without infractions on a full-sized vehicle.The critic estimates cumulative reward for state-action pairs, and the actor estimates the optimal policy; learning the continuous state-action space is performed in real time with less computational complexity.Source: §5.1.1 · pp. 20–21
• Hierarchical deep reinforcement learning [27] builds a new task’s meta-policy from previously learned policies, using an MLP for fully observable MDPs and an RNN for partially observable ones.The meta-policy requires less exploration and converges within a few iterations, attaining higher reward than policy networks learned by conventional RL methods.Source: §5.1.1 · pp. 21–22
• A deep-RL and DFSM method restricts available actions according to the driving situation, avoiding value calculations for all actions to reduce computation and accelerate training.The DFSM navigator restricts the state-action pair supplied to the deep-RL network, addressing difficult convergence in realistic driving environments.Source: §5.1.1 · pp. 18, 22

5.1.2 Scene Recognition

• Interpretable incremental learning aims to detect and learn unseen driving scenes without external supervision [29].Source: §5.1.2 · p. 22
• DRB uses self-evolving fuzzy IF..THEN rules and class-specific density prototypes to classify scenes under changing lighting conditions.Source: §5.1.2 · pp. 22–23
• Its density-based prototypes reduce computation and convergence time versus conventional deep networks, while tolerating incorrect semi-supervised pseudo-labels [29].Source: §5.1.2 · p. 23

5.1.3 Abnormality detection

• Because real-world data is non-stationary, self-aware systems need dynamic models that detect and incrementally learn novel states; DBN complexity control is a key challenge.Source: §5.1.3 · p. 23
• Free energy minimization uses GNG clusters to predict sensory observations, detects anomalies from prediction error, and creates a new DBN to incorporate novelty.Source: §5.1.3 · p. 24
• GP regression segments vehicle positions into spatial zones modeled by Kalman filters; prediction error identifies abnormalities, and new filter banks enable incremental learning.Source: §5.1.3 · p. 25

5.1.4 Controllers

• LWPR2 combines pseudo-data with online data to update vehicle-dynamics mappings, handling changing input-output relationships at reduced computational cost for MPC.Source: §5.1.4 · pp. 25–26
• PPO-based adaptive path tracking learns controller weights from buffered state-reward transitions, balancing tracking error and smoothness and outperforming fixed-weight control.Source: §5.1.4 · p. 26

5.1.5 Vehicle Tracking

• Because driver trajectories are uncertain and generic offline models need target-specific tailoring, BRNN adapts its policy online with particle filtering to predict feasible long-horizon trajectories.The adapted BRNN has a higher trajectory likelihood than the offline-trained model.Source: §5.1.5 · pp. 26–27
• PS-GAIL models interactive driving by averaging behaviors, then adapts driver-specific output-layer parameters with RLS-PAA while keeping offline features fixed for low-computation one-step learning.Its recurrent feedback loop also demonstrates multi-step prediction.Source: §5.1.5 · pp. 27–28

5.1.6 Pedestrian Tracking

• SILA [36] predicts urban-intersection pedestrian motion as transferable motion primitives linked by Gaussian-process transitions, merging similar graphs and adding novel primitives to constrain model growth.This approach addresses pedestrian motion prediction where behavioral rules are less clear or may be violated.Source: §5.1.6 · p. 28

5.1.7 Connected Vehicle

• V2V active learning integrates vehicle-generated labels by voting or weighted averaging; sample and data modes achieve high accuracy but require high network load, helping when ego sensors age.Vehicles can exchange labels only, data only, or both data and labels.Source: §5.1.7 · pp. 29–30

5.1.8 Driver-monitoring

• Driver monitoring requires incremental anomaly learning and driver-specific adaptation; its cloud framework retrains after performance drops and offloads computational complexity from the vehicle.The vehicle temporarily stores live data before offloading it to the cloud, while the system monitors driver behavior in real time and adapts to novel scenarios.Source: §5.1.8 · pp. 30–31

5.1.9 Uncertainity Detection and Data Aggregation

• UAIL attains a high success rate by anticipating infractions many steps ahead and using human intervention to prevent sequences of bad actions.It estimates output uncertainty with Monte Carlo Dropout and, at sub-optimal states, hands control to the driver while collecting unknown training data.Source: §5.1.9 · pp. 31–32
• CBIL collects uncertain mistakes online and retrains offline to limit forgetting, targeting rare adverse scenarios with little data in fleet-wide periodic development schedules.It concentrates on examples sensitive to the decision boundary, improving mean time to failure and scaling better than supervised learning by using less training data.Source: §5.1.9 · p. 32

5.2.1 Unseen Trajectory Tracking

• For unseen trajectories, an online DNN controller updates its weights using target information from an expert fuzzy logic system.The controller is first trained offline on PID-controller data for limited trajectories, then controls seen and unseen trajectories in real time; expert-defined fuzzy rules provide low-computation online targets.Source: §5.2.1 · p. 33

5.2.2 Control of unknown system dynamics

• DFNN-based control adapts to changing UAV dynamics and achieves faster response time while reducing tracking errors.It estimates the system’s inverse dynamics using three parallel networks, with offline training on conventional-controller data followed by online adaptation guided by expert fuzzy rules and FLS corrections.Source: §5.2.2 · pp. 33, 35

5.2.3 Online System Identification from datastreams

• McSIT2RFNN identifies non-stationary data streams online from scratch, using single-pass learning that requires fewer computing and memory resources.Its recurrent connections model temporal dynamics, while rules are added for novel concepts and inactive rules are removed; it also selects informative, uncertain samples.Source: §5.2.3 · p. 36

5.2.4 Deep Reinforcement Learning for Navigation and Exploration

• ED-DQN [44] combines raw images and positional maps to navigate partially observable environments, adapting successfully to unseen weather during search-and-rescue missions.The map estimates the target location, while images identify cluttered areas; evaluation covered light and heavy snow, dust, and fog.Source: §5.2.4 · p. 36

5.2.5 Object Tracking

• Online CNN tracking fine-tunes on target-centered samples to adapt to changing appearances, improving tracking performance and benefiting large-scale classification.The method incorporates locally connected layers for robust features and uses focal loss to learn hard examples and address class imbalance.Source: §5.2.5 · pp. 36–37

5.3.1 Control Systems:

• Urban robot controllers adapt online: IncSVR [46] uses support vectors to reduce computation and memory, while an adaptive neural controller updates weights from tracking error.IncSVR filters redundant and outlier samples to reduce forgetting while maintaining accuracy; the flexible-joint controller updates internal weights more slowly than output weights for faster tracking-error convergence with fewer computations.Source: §5.3.1 · pp. 38, 40

5.3.2 Vision-Based Manipulation

• Vision-based manipulation adapts to changes in backgrounds, lighting, gripper shape, and objects by fine-tuning a pre-trained policy with reinforcement learning [48].The policy gathers target-task data, then is fine-tuned offline on base- and new-task examples sampled with equal probability.Source: §5.3.2 · p. 40

5.3.3 Web Aided Deep Open World Recognition(OWR)

• Deep OWR enables open-world recognition of unknown objects and incrementally learns them using web mining and human-provided labels.A rehearsal buffer and distillation regularization help retain prior knowledge and avoid catastrophic forgetting.Source: §5.3.3 · p. 41

5.3.4 Visual-Tactile Cross-Modal Learning for Robotic Material Perception

• Lifelong cross-modal learning combines visual and tactile data to learn material properties in unfamiliar environments while addressing catastrophic forgetting.Task-shared representations learn inter- and intra-task correlations, using adversarial learning for intra-task alignment and distillation loss for inter-task retention.Source: §5.3.4 · pp. 41–42

5.3.5 Localization of human sound source in Complex Indoor Environment

• A sound-localization system uses HARAM to rank candidate rooms and incrementally learn from exploration without prior training.Human pose detection verifies the predicted room rank, while exploration supplies positive and negative examples for subsequent learning.Source: §5.3.5 · p. 43

5.3.6 Online Object and Task Learning via Human Robot Interaction

• A human-robot interface [52] lets users add object classes, associate objects with 3D motions, and teach the robot to perform those motions.The interface supports real-time learning of novel objects and their associated motions, providing reliable and diverse training data.Source: §5.3.6 · pp. 43–44

5.3.7 An Autonomous Self-Supervised Learning-Based Navigation System

• The self-supervised navigation method generalizes to novel environments and improves performance with a small amount of training data.It learns from real-world off-policy exploration, using LIDAR and IMU observations to label collision, bumpiness, and position scenarios.Source: §5.3.1 · p. 39, §5.3.7 · p. 44

5.3.8 Personalized Online Learning of Whole-Body Motion Classification

• The personalized whole-body motion model [54] outperforms the average-user model while using fewer computations and attaining high accuracy.It updates the average model online using samples with high testing error, with IMU observations and retrospectively annotated data.Source: §5.3.8 · p. 45

6.1 Anomaly Detection Using Deep Learning Real-Time Video Surveillance

• For real-time video surveillance, ISTL [73] detects anomalies from spatio-temporal auto-encoder reconstruction error and adapts evolving normal behavior using prediction-error-based active learning and human feedback.Fuzzy aggregation selects samples for continual learning, while human feedback confirms whether an anomaly is present.Source: §6.1 · pp. 45–46

6.2 Online deep learning-based energy forecasting

• For time-sensitive energy prediction, an edge-based online method incrementally trains a pre-trained DNN on small data subsets and uses JANET, an LSTM variant designed to avoid high online computation.Training occurs on data acquired at fixed time intervals, addressing the network-resource use and congestion associated with uploading the full dataset to a cloud datacenter.Source: §6.2 · p. 46

6.3 Incremental Continuous Load Prediction in Energy Management Systems

• For energy load management, an LSTM method retains temporal correlations and learns novel forecasting situations by updating from prediction errors once exact load values arrive.The model first forecasts the load and then computes the error when the actual value becomes available.Source: §6.3 · pp. 46–47

6.4 Real-time adaption for deep stereo

• MADNet adapts deep stereo online by independently training network modules and using shorter back-propagation to reduce learning time and computation complexity.It divides the network into non-overlapping horizontal segments and optimizes one module at a time, propagating through layers with high output loss.Source: §6.4 · p. 47

6.5 Incremental Learning for Chemical Fault Diagnosis

• Incremental-IMDNN addresses imbalanced chemical fault streams by balancing input fault data, selecting informative diagnosis data through active learning, and incrementally expanding a hierarchical neural network.The passage describes the balancing and active-learning feedback loop across two paragraphs.Source: §6.5 · pp. 47–48

6.6 Event-Based Incremental Broad Learning System for Object Classification

• For event-based object recognition, incremental broad learning dynamically expands feature and enhancement nodes as new data arrives, using SVD to avoid redundant nodes.The system uses a flat neural network structure for high-speed learning.Source: §6.6 · p. 48

6.7 Incremental Learning for Wireless Big Data Feature Learning

• For real-time IoT big-data feature learning, a tensor-based deep computation auto-encoder combines regularization and dynamic expansion with higher-order back-propagation to achieve high learning speed.Source: §6.7 · p. 49

7 Looking Ahead

• Online continual learning addresses real-world streams by learning continuously without task identity, unlike task-based training that assumes known boundaries and repeated passes over batches.The survey defines task-based learning as training on each task’s data until convergence, whereas real-world settings receive infinite data streams in a task-agnostic manner.Source: §7 · p. 49
• Autonomous-system use cases can guide approach selection, but image-processing experiments and virtual environments require clarification of expected performance in realistic scenarios.Source: §7 · p. 49
• Real-world frameworks should be problem agnostic, adapt to unlabeled and user data, selectively forget trivial information, and acquire new information online without large-batch or task-separated offline training.Source: §7 · p. 50
• Dynamic architectural changes should keep model complexity below linear parameter growth, while realistic systems must respect memory limits and evaluate scalability across many tasks.The survey also recommends ablations of components such as initialization, optimization methods, and hyperparameters.Source: §7 · p. 50
• Continual-learning evaluations should use complex, realistic, higher-resolution benchmarks and formally describe how algorithms handle distribution shifts across tasks and between batches.Source: §7 · p. 50

8 Conclusion

• The survey reviews current continual-learning techniques for autonomous systems, including approaches to alleviate catastrophic forgetting under limited supervision and computing resources.It also summarizes limitations and challenges of existing techniques for real-world systems.Source: §8 · p. 50
• It explores practical continual-learning use cases in self-driving cars, unmanned aerial vehicles, and urban robots, aiming to help categorize and compare emerging approaches.Source: §8 · p. 51
• Future work should develop fully autonomous, secure, and reliable learning frameworks with better performance and reduced supervision, training time, and resource use.The article also calls for evaluating real-world applicability more often and investigating benchmarks that represent actual real-world scenarios.Source: §8 · p. 51

Funding

• No funding was received to assist with preparing the manuscript.Source: Funding · p. 51

Conflicts of interest

• The authors declare that they have no conflict of interest.Source: Conflicts of interest · p. 51

Authors’ Contributions

• The paper’s four authors contributed through conceptualization, drafting, review, writing, editing, and revision assistance.Khadija Shaheen handled conceptualization and the original draft; Muhammad Abdullah Hanif wrote, reviewed, and edited the paper; Dr. Osman Hasan and Dr. Muhammad Shafique edited and helped with revisions.Source: Authors’ Contributions · p. 51

Ethics approval

• The paper does not report research requiring ethical approval.Source: Ethics approval · p. 51