Source-linked AI summary
A scikit-based Python environment for performing multi-label classification
Piotr Szymański, Tomasz Kajdanowicz
TL;DR
Multi-label classification lacked a high-quality solution integrated with Python’s scientific ecosystem. Scikit-multilearn addresses this with sparse-matrix-based methods, label-space techniques, embeddings, interoperability, and dataset tools, while benchmarks report lower memory use and frequent speed advantages over Java libraries.
Problem
Python’s machine-learning ecosystem lacked a high-quality multi-label classification solution.
Method
Scikit-multilearn builds a scikit-learn-compatible library using sparse matrices, native multi-label methods, label-space division, embeddings, wrappers, stratification, and dataset-management tools.
Results
Scikit-multilearn uses less memory than MEKA or MULAN and operates faster than them in most cases, while Binary Relevance is the most efficient choice on every dataset tested.
Takeaways & Limitations
The library provides an extensive Python-compatible multi-label classification stack that integrates with scikit-learn, Keras, PyTorch, and Java-based methods.
Takeaways & Limitations
Scikit-multilearn does not support multi-label regression or multi-output prediction, and extreme multi-label classification is not its main focus.
Abstract
from arXiv · showhide
scikit-multilearn is a Python library for performing multi-label classification. The library is compatible with the scikit/scipy ecosystem and uses sparse matrices for all internal operations. It provides native Python implementations of popular multi-label classification methods alongside a novel framework for label space partitioning and division. It includes modern algorithm adaptation methods, network-based label space division approaches, which extracts label dependency information and multi-label embedding classifiers. It provides python wrapped access to the extensive multi-label method stack from Java libraries and makes it possible to extend deep learning single-label methods for multi-label tasks. The library allows multi-label stratification and data set management. The implementation is more efficient in problem transformation than other established libraries, has good test coverage and follows PEP8. Source code and documentation can be downloaded from http://scikit.ml and also via pip. The library follows BSD licensing scheme.
1. Introduction
Python offered a strong scientific ecosystem but lacked a high-quality multi-label classification solution. The paper introduces scikit-multilearn as a tested, compatible library intended to fill that gap.
- Python’s machine-learning ecosystem lacked a high-quality solution for multi-label classification.
- Scikit-multilearn is a well-tested, multi-platform, Python 3-compatible, BSD-licensed library for multi-label problems.
- The library integrates with Python’s scientific and machine-learning ecosystem while providing efficient implementations, data stratification, and dataset management.
- The paper evaluates scikit-multilearn against libraries implemented in other languages and discusses conclusions and future ideas.
2. Multi-label classification
Multi-label classification assigns subsets of labels to observations and is addressed through method adaptation, problem transformation, ensembles, and embedding techniques. Extreme multi-label classification extends this area but differs in scale, benchmarks, and evaluation measures.
- Multi-label classification assigns a subset of available labels to each observation across domains such as text, multimedia, music, and genomics.
- Method adaptation modifies single-label algorithms to account for multiple labels, while problem transformation converts the task into one or more single-label problems.
- These approaches face overfitting, underfitting, label imbalance, and numerical issues related to label ordering in Bayesian models.
- Ensemble methods address multi-label learning by combining classifiers trained on label subspaces or observation subsets.
- Embedding methods reduce or transform large label spaces using techniques including PCA, canonical correlation, matrix decomposition, clustering, multidimensional scaling, and label-network embeddings.
- Extreme multi-label classification uses specialized deep-learning, tree-based, and embedding approaches but differs from classical multi-label classification in expectations, datasets, and quality measures.
3. Related work
Related work spans Java, Python, R, and Matlab/Octave libraries, with scikit-multilearn positioned as a specialized Python complement to scikit-learn. Its niche combines broad multi-label methods, sparse support, interoperability, and access to stream, imbalance, and deep-learning tools.
- MULAN and MEKA provide large Java-based multi-label stacks built heavily on WEKA’s single-label methods.
- Python’s ecosystem favors interoperable specialized libraries built on shared APIs rather than a few large monolithic libraries.
- Scikit-learn offers several multi-label algorithms but limited sparse label-matrix support and does not cover the full range of contemporary multi-label methods.
- Scikit-multilearn complements scikit-learn by providing specialized multi-label functionality within its API and ecosystem.
- Scikit-multiflow targets evolving multi-label streams, whereas scikit-multilearn targets the classic formulation with a broader state-of-the-art method offering.
- Imbalanced-learn can improve scikit-multilearn classifiers by resampling data after problem transformations produce multi-class tasks.
- Domain-specific and general deep-learning libraries support multi-label classification, but neural networks may require problem-specific architectures and can lack robustness.
- Scikit-multilearn supports Keras-compatible models across backends and PyTorch through skorch.
4. The scikit-multilearn library
Scikit-multilearn extends the scikit-learn ecosystem with a Python library for multi-label classification, combining advanced algorithms, label-space partitioning, embeddings, and supporting tooling. It emphasizes sparse internal computation, ecosystem compatibility, and access to methods beyond native Python implementations.
- Scikit-multilearn builds multi-label classification on the scipy stack and follows scikit-learn API principles and licensing.
- The library extends scikit-learn with advanced algorithm adaptation, problem transformation, and label-space division approaches.
- It implements multi-label embedding classifiers and provides a general framework supporting approaches such as CLEMS and LNEMLC.
- The library adds dataset manipulation, sparse multilabel representations, data stratification, Java-library wrappers, and Keras-compatible deep-learning problem transformations.
- Problem Transformation with Label Space Division: Label-space division supports clustering, network communities, random division, and expert-defined fixed partitions, with partitioning and voting ensemble strategies.
- Sparse internal matrices target multi-label outputs whose benchmark rows contain fewer than 5% labels on average, yielding a reported memory advantage over MULAN and MEKA.
- Scikit-multilearn does not currently support multi-label regression or multi-output prediction, while extreme multi-label classification is not its main focus.
5. Benchmark
The benchmark compares scikit-multilearn with other libraries on complete classification scenarios using Binary Relevance and Label Powerset. Scikit-multilearn is generally faster and uses less memory, with statistically significant advantages in Binary Relevance and competitive Label Powerset performance.
- Experimental setup: The benchmark evaluates MEKA, MULAN, and scikit-multilearn on 12 datasets using Binary Relevance and Label Powerset.Algorithm adaptation methods were excluded because they were not available in all three libraries.
- Experimental setup: Classification time includes dataset loading, error measurement, and the complete use-case scenario, with results normalized separately for each dataset.The worst median performance per dataset is set to 100%, and curves closer to zero indicate better performance.
- Experimental setup: All libraries were restricted to one CPU core, while user time and maximum resident set size measured runtime and memory usage.The measurements were obtained with the taskset and time -v commands.
- Overall performance: Scikit-multilearn uses less or equal memory than MEKA and MULAN in both classification schemes and is faster in most cases.The memory advantage is attributed to sparse matrix support.
- Binary Relevance: Binary Relevance makes scikit-multilearn the most efficient choice on every dataset and statistically significantly more efficient than the other libraries.The comparison uses the reported worst-case time and memory scenario and Quade testing with Bergmann-Hommel post hoc analysis.
- Label Powerset: With Label Powerset, scikit-multilearn and MLC Toolbox significantly outperform MULAN and MEKA; scikit-multilearn is slightly faster but uses slightly more memory than MLC Toolbox.Their speed and memory differences are not statistically significant, while scikit-multilearn outperforms MLC Toolbox on datasets with more label combinations.
6. Conclusions
Scikit-multilearn is presented as an extensive scikit-learn-compatible multi-label classification library. It combines sparse-matrix efficiency with broad algorithm coverage and integrations spanning Python, Java-based methods, and Keras models.
- Contributions: Scikit-multilearn provides popular algorithms, network-based label space division methods, sparse internal transformations, and optimized scikit-based classifiers.These design choices are presented as the basis for its speed and breadth.
- Integrations: The library integrates with the scipy and Python data science stack and wraps Keras, MEKA, WEKA, and parts of MULAN.These wrappers support using deep-learning and Java classification methods within the Python stack.