Source-linked AI summary
Open-environment Machine Learning
Zhi-Hua Zhou
TL;DR
Open ML addresses the challenge of learning when factors assumed invariant in conventional machine learning change, especially as data accumulate over time. The article surveys strategies for emerging classes, changing features and distributions, varied objectives, and related theory. It presents Open ML as a research direction spanning these threads, while noting constraints including catastrophic forgetting and the costs of storing and repeatedly retraining on all data.
Problem
Conventional machine learning assumes invariant learning factors, whereas practical open-environment tasks involve changing factors and time-accumulated stream data.
Method
The article briefly surveys strategies for emerging new classes, decremental/incremental features, changing data distributions, varied learning objectives, and theoretical issues.
Results
The article presents Open ML as a research direction covering four changing-factor threads and associated theoretical issues.
Takeaways & Limitations
Open ML requires learning strategies that address changes in classes, features, distributions, and objectives rather than relying only on close-environment assumptions.
Takeaways & Limitations
Updating models with only new data can cause catastrophic forgetting, while retraining on all received data can impose huge computational and storage costs.
Abstract
from arXiv · showhide
Conventional machine learning studies generally assume close-environment scenarios where important factors of the learning process hold invariant. With the great success of machine learning, nowadays, more and more practical tasks, particularly those involving open-environment scenarios where important factors are subject to change, called open-environment machine learning (Open ML) in this article, are present to the community. Evidently it is a grand challenge for machine learning turning from close environment to open environment. It becomes even more challenging since, in various big data tasks, data are usually accumulated with time, like streams, while it is hard to train the machine learning model after collecting all data as in conventional studies. This article briefly introduces some advances in this line of research, focusing on techniques concerning emerging new classes, decremental/incremental features, changing data distributions, varied learning objectives, and discusses some theoretical issues.
1. Introduction
Open-environment machine learning addresses practical tasks where factors fixed in close-environment studies can change. The article surveys techniques for these changes, including emerging classes, changing features and distributions, varied objectives, and theoretical issues.
- Close-environment assumptions: Close-environment machine learning assumes invariant labels, features, data distributions, and learning objectives.These assumptions simplify learning tasks and supported the development of machine learning techniques.
- Open-environment machine learning: Open ML handles changes in important factors of the learning process, including previously unknown classes and changing data conditions.The article calls this setting Open-environment Machine Learning, or Open Learning and Open ML.
- Challenges: Artificially generating examples of possible changes is applicable only when users can estimate what changes will occur.The passage states that this condition is absent in most practical tasks.
- Challenges: Streaming data make offline training after collecting all examples impractical, motivating models that can be refined or updated as new data arrive.Continual learning aims to resist forgetting, but large-batch scanning and offline training raise computational and storage concerns.
- Challenges: Updating deep networks with only new data can cause catastrophic forgetting, while storing all received data for retraining can impose huge computational and storage costs.These constraints make open-environment learning especially challenging for big stream data.
- Article scope: The article surveys advances on emerging new classes, decremental/incremental features, changing data distributions, varied learning objectives, and theoretical issues.Its discussion focuses on research efforts in Open ML rather than a single learning algorithm.
2. Emerging New Classes
Emerging new-class learning addresses streams where unknown classes must be identified and incorporated without retraining on all accumulated data. The main challenges are distinguishing NewClass instances from known-class anomalies and updating models without sacrificing known-class performance.
- The broader Open ML setting also includes delayed refinement, few-example updates, and multi-label learning with emerging classes.
- Unknown classes are difficult to anticipate because their identities cannot be enumerated in advance, while accommodating every possible class is infeasible or prohibitively expensive.
- Emerging new-class learning handles streams in which models must recognize known classes, report unknown classes, and later accommodate them as known classes.The desired process avoids retraining from stored historical data.
- Unlike zero-shot learning, emerging-new-class learning does not assume external knowledge describing unseen classes, which are unknown rather than merely absent from training.
- A general solution uses anomaly detection for NewClass identification, but NewClass data may be indistinguishable from anomalies of known classes.Kernel mapping or representation learning can seek a feature space where NewClass instances are more abnormal, after which stream anomaly detection can be applied.
- Updating a model requires accommodating NewClass without degrading known-class performance, yet deep networks may require all-data retraining to avoid catastrophic forgetting and its computational and storage costs.Local refinement, such as modifying only affected tree leaves, is presented as an alternative.
3. Decremental/Incremental Features
Decremental and incremental features create streaming settings where models must continue predicting as sensors disappear or new features arrive. Shared features, cross-phase mappings, and model combinations can preserve useful earlier experience, although feature loss can seriously reduce performance.
- Feature-evolution learning handles streams with disappearing and newly arriving features while seeking refinement without storing all historical data for retraining.
- When phase feature spaces are entirely different, the earlier model is helpless for the later phase, requiring a new model trained from scratch.
- When feature increment precedes decrement, a small set of data containing both feature sets can learn a mapping that lets the phase-1 model remain useful despite disjoint later features.Predictions can combine the mapped earlier model with a model trained on the later feature space.
- Weighted model combination has cumulative loss comparable to the minimum loss of the two models, while weighted selection is comparable to optimal selection.
- Online learning supports these strategies on streams and permits combinations of models from multiple feature spaces, but feature decrement can seriously downgrade performance.
4. Changing Data Distributions
Open ML must address data distributions that change during streams, rather than assuming fixed i.i.d. training and testing data. Key approaches exploit recent data, forgetting, windows, or adaptive ensembles, but repeated scans and unbounded storage remain challenges.
- Close-environment learning assumes training and testing data are independent samples from an identical, unchanging distribution.
- Ignoring distribution change may seriously harm performance, including when models trained on seasonal sensor data must operate across seasons.
- Data distribution changes can occur throughout a stream, making distribution adaptation an important Open ML requirement.
- Recent observations can help because current and nearby stream instances are often drawn from similar or identical distributions.
- General adaptation mechanisms include sliding windows, age-based forgetting, and ensembles that add, remove, or reweight component learners.
- Many sliding-window and ensemble methods require multiple scans, whereas big-data streams ideally need one scan and storage independent of total data volume.
- A forgetting-based approach needs no prior knowledge of the change, discards instances after scanning, and has an estimate error that decreases until convergence.
5. Varied Learning Objectives
Open ML must support objectives that change over time or are pursued simultaneously, because a model optimal for one measure may not be optimal for others. The section discusses objective switching, Pareto optimization, evolutionary methods, and preference-based implicit objectives.
- Close-environment learning typically assumes the performance measure is invariant and known in advance, but practical objectives can change during operation.
- Open ML seeks models that switch between objectives without recollecting data and retraining an entirely new model.
- Simultaneous objectives can be handled through Pareto optimization, which seeks solutions not inferior to another solution on every objective.
- Evolutionary algorithms are commonly used for Pareto optimization but are criticized for appearing to be purely heuristic.
- Theoretical advances in evolutionary learning can guide new algorithms with better approximation guarantees than conventional algorithms.
- Implicit objectives can be expressed through user preferences between models, and effective models have been obtained by exploiting techniques such as bag of words.
6. Theoretical Issues
Open ML raises theoretical problems because changes are unknown, data arrive online, and familiar assumptions or statistics may fail. The section surveys challenges involving new classes, heavy-tailed data, non-stationarity, noise, and multiple objectives.
- Open ML remains theoretically underdeveloped, particularly for varied learning objectives, although changing distributions have received more theoretical study.
- Unknown or frequent changes can invalidate prior exploration, while existing reinforcement-learning adaptations generally assume changes are infrequent or non-continuous.
- Open ML requires model updates after only a few post-change instances, while previously unseen or minority examples remain important alongside majority performance.
- Heavy-tailed data invalidate some familiar tools: the Central Limit Theorem may fail, sample statistics can mislead, and empirical risk may poorly approximate risk.
- When input and output spaces are heavy-tailed, empirical risk minimization becomes invalid, posing problems even for simple L1-regression.
- Non-stationary online settings motivate dynamic regret, which compares cumulative learner loss with sequences of comparators rather than one constant comparator.
- Smoothing can suppress important signals or rare events in Open ML because signals treated as noise may carry essential information.
7. Conclusion
The article briefly surveys Open ML advances while emphasizing general principles and strategies over specific algorithms. It presents Open ML as relevant to robust AI and learnware, but not as a comprehensive review.
- The article introduces research advances in open-environment machine learning and discusses general principles and strategies rather than specific algorithms.
- The survey is not comprehensive and mainly summarizes exploration by the author and colleagues.
- Its strategies can be realized with various learning techniques, whose different strengths remain for future investigation.
- The article discusses varied issues separately even though they often occur simultaneously in real practice.
- Open ML is presented as crucial for robust artificial intelligence and as carrying desired properties of learnware.