Source-linked AI summary
Infrastructure for Usable Machine Learning: The Stanford DAWN Project
Peter Bailis, Kunle Olukotun, Christopher Re, Matei Zaharia
TL;DR
Machine-learning applications remain costly to build, especially where labeled data and production deployment require substantial effort. DAWN proposes end-to-end systems and tools for usable ML; its early results report 10–100× acceleration for ML applications. The project aims to make production-quality ML more accessible to domain experts.
Problem
Building ML applications remains expensive, requiring costly teams, substantial training data, and effort to deploy, operate, and monitor systems.
Source: §1 · p. 1Method
DAWN is a five-year project developing systems infrastructure and tools for end-to-end ML, including expert rule-based labeling with unsupervised denoising and supervised training.
Source: §1 · pp. 1–2, §2 · p. 2, DAWN Research Directions · p. 3Results
10–100× acceleration of ML applications is reported in early results from end-to-end pipeline optimization and exploiting algorithmic tolerance to inexact execution.
Source: §2 · p. 3Takeaways & Limitations
DAWN aims to let small teams of domain experts build high-quality ML applications and deploy production systems without large teams of ML specialists.
Source: §1 · pp. 1–2
Abstract
from arXiv · showhide
Despite incredible recent advances in machine learning, building machine learning applications remains prohibitively time-consuming and expensive for all but the best-trained, best-funded engineering organizations. This expense comes not from a need for new and improved statistical models but instead from a lack of systems and tools for supporting end-to-end machine learning application development, from data preparation and labeling to productionization and monitoring. In this document, we outline opportunities for infrastructure supporting usable, end-to-end machine learning applications in the context of the nascent DAWN (Data Analytics for What's Next) project at Stanford.
1 Introduction and DAWN Project Goals
Machine learning has advanced rapidly, but building and operating applications remains costly and demands specialized teams and difficult-to-obtain labeled data. DAWN proposes systems and tools for end-to-end development so domain experts can build production-quality applications without relying on large ML teams.
1 Introduction and DAWN Project Goals
- Recent advances in algorithms, large datasets, and fast parallel computing have brought machine learning into applications such as voice assistants, image recognition, and autonomous vehicles.The paper describes these successes as occurring over the preceding five years.Source: §1 · p. 1
- ML applications remain expensive to build and operate, with large teams and substantial labeled-data demands limiting their use across domains.The paper compares this stage of ML with early digital computers that required armies of technicians to keep a few machines operating.Source: §1 · p. 1
- DAWN aims to make production-quality ML applications attainable for small teams of domain experts, rather than improve algorithms described as good enough for many important applications.The project is a five-year effort to design systems infrastructure and tools for usable ML.Source: §1 · pp. 1–2
- The authors point to search libraries and relational databases as precedents for making complex technologies usable through simple interfaces and tuning options.They note that database abstractions reduced application-building effort and supported many database-backed applications.Source: §1 · p. 2
- The authors argue that most industrial ML effort goes to data preparation, feature selection and extraction, and productionization—not new models.These stages include acquiring and cleaning training data, identifying useful data characteristics, and deploying, monitoring, and debugging robust products.Source: §1 · p. 2
2 The DAWN Project and Systems Research in Usable ML
DAWN proposes infrastructure spanning the ML lifecycle and systems stack, aiming to let domain experts build production-quality applications. Its research directions target end-to-end workflows, usable interfaces, and coordinated software and hardware, with usability as the roadmap’s success measure.
2 The DAWN Project and Systems Research in Usable ML
- DAWN pairs lifecycle-wide tools with interfaces-to-hardware work, targeting end-to-end workflows and domain experts; early pipeline optimization reports 10–100× acceleration.The approach uses pipeline-wide optimization and algorithmic tolerance to inexact execution, with acceleration reported on current and emerging hardware.Source: §2 · pp. 2–3, Figure 1 · p. 3
DAWN Research Directions
- Interface research also targets human-interpretable prediction explanations and monitoring for model drift, with subsequent correction of deviations from expected behavior.User attributes can help contextualize predictions; monitoring deployed models and updating them are emphasized as phenomena evolve.Source: DAWN Research Directions · p. 4
- For data preparation, DAWN models source quality to prioritize enrichment and uses expert rules, denoising, and probabilistic labels to train models.The latter approach is called data programming; early Snorkel results produce high-quality models from low-quality rules.Source: DAWN Research Directions · pp. 3–4
- End-to-end systems target stream classification across domains and plug-and-play recommendations that tune, serve, monitor, and retrain incrementally.Stream operators combine feature transformation, classification, and aggregation; recommendation inputs can include user clicks or ratings.Source: DAWN Research Directions · p. 4
- DAWN explores integrating predictions with automated decisions, from alerts and notifications to physical actions, because inference and actuation usually run separately.The project identifies API integration and logic for reasoning about the combination as missing pieces.Source: DAWN Research Directions · pp. 4–5
- A unified engine could combine SQL, graph computations, and ML workloads by extending join-based execution techniques to linear algebra and sparse matrices.The described optimized join matches optimized SQL and graph engines when combined with SIMD execution.Source: DAWN Research Directions · p. 5
- Substrate work spans cross-library compilation, low-precision stochastic execution, programmable reconfigurable kernels, and distributed runtimes tuned across hardware and networks.Weld is reported to accelerate Apache Spark, Pandas, and TensorFlow by 10× and cross-library workloads by up to 30×.Source: DAWN Research Directions · pp. 5–6
Research Roadmap and Success Metrics
- The roadmap measures usability through specification time and cost, production execution time and cost, and end-user benefit, while integrating work across stack levels.The authors report production deployments of Snorkel, MacroBase, and DeepDive as confirming the project’s opportunity.Source: Research Roadmap and Success Metrics · p. 6