Source-linked AI summary
SLAQ: Quality-Driven Scheduling for Distributed Machine Learning
Haoyu Zhang, Logan Stafman, Andrew Or, Michael J. Freedman
TL;DR
Shared-cluster ML training creates a scheduling problem because fair resource allocation ignores differing opportunities for quality improvement across iterative jobs. SLAQ addresses this by predicting each job’s resource-dependent progress and allocating CPUs toward the largest expected loss reduction. Across diverse ML workloads, it improves overall quality and reaches target loss reductions faster, while its prediction approach remains limited for nonconvex optimization.
Problem
Fair schedulers are agnostic to model quality and resource efficiency, allocating equal resources even when jobs have different potential for improvement.
Method
SLAQ normalizes loss changes, predicts online quality improvement from convergence behavior, and incrementally allocates CPUs to maximize system-wide loss reduction.
Results
SLAQ improves overall quality and executes ML jobs faster under resource contention across a large evaluation of ML training workloads.
Takeaways & Limitations
Directing resources toward jobs with greater quality-improvement potential provides a quality-driven alternative to fair scheduling for exploratory ML training.
Takeaways & Limitations
SLAQ’s prediction mechanism is limited for nonconvex optimization because convergence may lack an analytical model and need not be monotonic.
Abstract
from arXiv · showhide
Training machine learning (ML) models with large datasets can incur significant resource contention on shared clusters. This training typically involves many iterations that continually improve the quality of the model. Yet in exploratory settings, better models can be obtained faster by directing resources to jobs with the most potential for improvement. We describe SLAQ, a cluster scheduling system for approximate ML training jobs that aims to maximize the overall job quality. When allocating cluster resources, SLAQ explores the quality-runtime trade-offs across multiple jobs to maximize system-wide quality improvement. To do so, SLAQ leverages the iterative nature of ML training algorithms, by collecting quality and resource usage information from concurrent jobs, and then generating highly-tailored quality-improvement predictions for future iterations. Experiments show that SLAQ achieves an average quality improvement of up to 73% and an average delay reduction of up to 44% on a large set of ML training jobs, compared to resource fairness schedulers.
1 Background and Motivation
Large-scale ML training is iterative, approximate, and increasingly constrained by growing model and dataset sizes. Because quality gains diminish over iterations and training is exploratory, shared-cluster scheduling should favor jobs with greater potential for improvement rather than allocate resources solely fairly.
- Model complexity and data volume are growing faster than hardware speed, increasing the need for efficient cluster-resource management.
- ML training iteratively optimizes model parameters, but quality improvement diminishes as more iterations are completed.
- Exploratory ML workloads repeatedly retrain models to test features, tune hyperparameters, and adjust models.
- Fair schedulers allocate equal resources to jobs regardless of whether they are early-stage or nearly converged.
- SLAQ’s key scheduling intuition is to allocate more resources to jobs with the greatest potential for quality improvement.
2 Design
SLAQ normalizes loss progress, predicts future quality improvement from online convergence behavior, and allocates CPUs incrementally to maximize system-wide loss reduction. Its design uses convergence-rate models and tailored resource decisions, while currently excluding some nonconvex algorithms and relying on assumptions for convergence analysis.
- SLAQ design: SLAQ maximizes overall job quality by normalizing quality metrics, predicting resource-dependent progress, and allocating CPUs for system-wide improvement.
- Normalizing quality metrics: SLAQ normalizes inter-iteration loss changes by the largest change seen so far, enabling progress comparison across jobs with diverse loss ranges.
- Scope and assumptions: The normalization approach does not currently support some nonconvex algorithms because their convergence cannot be modeled analytically.
- Predicting quality improvement: SLAQ predicts quality improvement online using loss-function convergence properties instead of repeatedly profiling changing computation and communication patterns.
- Convergence models: First-order algorithms commonly converge at O(1/k), while linear or superlinear methods follow O(µ^k), |µ| < 1.
- Convergence models: The predictor achieves less than 5% error when forecasting the next 10th iteration across the algorithms in Figure 2.
- Resource allocation: Starting with one CPU per job, SLAQ repeatedly gives the next CPU to the job with the highest predicted loss reduction until capacity is exhausted.
3 Evaluation
SLAQ was evaluated on diverse ML workloads and datasets against a work-conserving fair scheduler. It improved time to reach target loss reductions, shifted resources toward high-loss jobs, and remained scalable for large scheduling workloads.
- Setup: SLAQ was tested on 160 ML training jobs spanning classification, regression, and unsupervised learning across more than 200GB of diverse datasets.The evaluation used a 20-instance AWS cluster and compared SLAQ with a work-conserving fair scheduler.
- Scheduler Quality and Runtime Improvement: In exploratory training, the evaluated loss-reduction level was frequently sufficient, so SLAQ could substantially reduce users’ wait times.This consequence is stated for environments where users submit exploratory ML training jobs.
- Resource Allocation: SLAQ allocated 60% of resources to high-loss jobs and 22% to nearly converged jobs, explaining the improvements in quality and completion time.A fair scheduler would allocate resources proportionally to the number of jobs in each group.
- Scalability and efficiency: SLAQ made scheduling decisions in hundreds of milliseconds to a few seconds while scheduling 4,000 jobs across 16K worker cores.The experiment simulated tens of thousands of concurrent jobs on large clusters.
4 Conclusion and Future Work
SLAQ uses iterative ML behavior and tailored prediction to improve model quality faster under shared-cluster contention. Its main limitation is non-convex optimization, where unclear convergence can distort predicted loss reduction.
- Conclusion: SLAQ leverages iterative ML algorithms to produce tailored predictions and improve the overall quality of executing jobs faster under resource contention.The system is designed for large-scale ML training jobs in shared clusters.
- Future Work: For non-convex optimization, missing analytical convergence models can cause SLAQ to underestimate or overestimate potential loss reduction.The authors leave modeling convergence for these algorithms to future work.