Source-linked AI summary
modAL: A modular active learning framework for Python
Tivadar Danka, Peter Horvath
TL;DR
When labeling is costly, modAL offers a Python framework for building active learning workflows from modular components compatible with scikit-learn. The framework provides algorithms across several active learning settings and is designed to support prototyping and novel strategy development.
Problem
Labeling real-life data can consume significant time and money, making large training sets infeasible in some applications.
Source: §1 · p. 1Method
modAL represents workflows with learner and query-strategy components, built around scikit-learn-compatible models and interfaces.
Source: §2 · pp. 1–2Results
modAL provides active learning algorithms for pool- and stream-based scenarios, multiclass and multilabel classification, density weighting, regression, and Bayesian optimization.
Source: §2 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
modAL is a modular active learning framework for Python, aimed to make active learning research and practice simpler. Its distinguishing features are (i) clear and modular object oriented design (ii) full compatibility with scikit-learn models and workflows. These features make fast prototyping and easy extensibility possible, aiding the development of real-life active learning pipelines and novel algorithms as well. modAL is fully open source, hosted on GitHub at https://github.com/cosmic-cortex/modAL. To assure code quality, extensive unit tests are provided and continuous integration is applied. In addition, a detailed documentation with several tutorials are also available for ease of use. The framework is available in PyPI and distributed under the MIT license.
1. Introduction
modAL addresses costly labeling by providing a Python active learning framework designed for modularity, flexibility, and extensibility. It builds on scikit-learn to support rapid prototyping and implementation of novel strategies.
1. Introduction
- In real-life applications, labeling can consume significant time and money, making large training sets infeasible.The passage gives text sentiment annotation and costly or derivative-free black-box optimization as examples.Source: §1 · p. 1
- Active learning can query labels for the most informative instances when obtaining labels is costly.Source: §1 · p. 1
- modAL is built on scikit-learn and designed to support rapid prototyping and testing novel active learning strategies with minimal effort.Source: §1 · p. 1
2. Design principles and features
modAL aims to make complex active learning workflows simple and intuitive by combining modularity, extensibility, and compatibility with scikit-learn. It also provides algorithms for multiple active learning settings.
2. Design principles and features
- Learning algorithms and query strategies can be combined in any arrangement through the ActiveLearner class, enabling rapid prototyping.Source: §2 · p. 2
- Custom query strategies can be supplied as functions without understanding modAL internals, making it easier to test and compare novel strategies.Source: §2 · p. 2
- modAL accepts scikit-learn classifiers and regressors, and its API-compatible objects can be inserted into existing workflows.Source: §2 · p. 2
- The framework includes algorithms for pool- and stream-based active learning, multiclass and multilabel classification, density weighting, regression, and Bayesian optimization.The passage names strategies including uncertainty sampling, query by committee, expected error reduction, and probability of improvement.Source: §2 · p. 2
3. Classes and interfaces
modAL represents active learning workflows with classes that pair a learner and query strategy. Its query functionality can be separated into utility calculation and instance selection, while its interface supports ordinary learner training and querying.
3. Classes and interfaces
- ActiveLearner and BayesianOptimizer accept a learner implementing the scikit-learn API, while Committee and CommitteeRegressor accept lists of ActiveLearner instances.Source: §3 · p. 2
- A query function can be split into utility calculation for each instance and selection of instances based on their utility scores.The passage links this modular design to extensibility and interaction with other libraries.Source: §3 · p. 3
- The example trains the learner with fit, queries a pool for instances, obtains labels from an oracle, and supplies those labels with teach.Source: §3 · p. 3
4. Comparison with other libraries
The paper compares modAL with other active learning libraries across supported algorithms, design, and support, and reports a runtime comparison for three methods across Python libraries.
4. Comparison with other libraries
- The comparison covers modAL, Python libraries acton2, alp3, and libact, and Java library JCLAL.Source: §4 · p. 3
- Libraries are compared on supported algorithms, design, and support.Source: §4 · p. 3
- Runtime comparisons cover least confident sampling, query by committee, and expected error reduction, averaging 10 runs of 10 queries per algorithm.Source: §4 · p. 3
5. Availability
modAL is open source and hosted on GitHub, with documentation, tutorials, downloadable examples, and distribution through PyPI. The section also presents comparisons of libraries’ algorithms, design and support, and runtime.
5. Availability
- modAL is open source and hosted on GitHub, with detailed documentation and examples and tutorials available on the project website.Tutorials and examples can be downloaded as Jupyter notebooks.Source: §5 · pp. 3–4
- The section compares libraries on supported algorithms, design and support, and runtime.The runtime comparison covers Python libraries; each run used 10 queries and was repeated 10 times.Source: Table 1 · p. 4, Table 2 · p. 4, Table 3 · p. 4
- The project reports 98% code coverage from extensive unit tests and uses continuous integration with Travis-CI.modAL is also available from PyPI.Source: §5 · p. 4