NSML: Meet the MLaaS platform with a real-world case study

1 INTRODUCTION

• Deep-learning complexity, model size, and practical collaboration needs make suitable tools and platforms necessary.The introduction names applications spanning image recognition, natural language processing, recommendation, question-answering, and biomedical work.Source: §1 · p. 1
• The two central challenges are setting up compatible training environments and coordinating shared data, models, and computing resources.Conflicting library versions can delay or prevent setup, while poorly managed shared components can impede enterprise-scale work.Source: §1 · p. 1
• Existing tools leave a gap: deep-learning libraries are not collaborative cluster managers, while some management platforms rely on public clouds rather than users’ private clusters.The introduction lists resource allocation, scheduling, sharing, monitoring, hyperparameter tuning, model comparison, and leaderboards among the desired capabilities.Source: §1 · p. 1
• NSML is proposed as a practical machine-learning platform, with efficient resource management, collaborative work, and model deployment as its key contributions.The paper also highlights real-world case studies, few-code insertion for running on NSML, and an intuitive UI/UX.Source: §1 · p. 2
• An intuitive interface for machine-learning developers and researchers is another stated contribution.The introduction presents this alongside real-world case studies and few-code insertion for running work on NSML.Source: §1 · p. 2

2.1 Platforms for machine learning tasks

• Existing machine-learning platforms can ease environment setup so users can focus on designing models.The section notes that setup difficulties may affect shareability and computational-resource utilization.Source: §2.1 · p. 2
• Common platform features include reproducibility, usability, visualization, and model deployment.Examples include Docker-based job packaging, model APIs, and tools for visualizing model performance or architecture; TensorFlow Extended’s framework dependency may limit availability.Source: §2.1 · p. 2

2.2 Cloud Platform

• Cloud platforms offer scalable, flexible virtual machines for deploying services and applying or developing machine-learning systems.The section says these platforms are not built solely for machine-learning tasks.Source: §2.2 · p. 2
• General cloud platforms may constrain machine-learning work through framework dependence, limited degrees of freedom, or usage expense.The section describes their systems as either too open to configure or too restrictive, and notes potential financial issues for researchers.Source: §2.2 · p. 2

3.1 System Overview

• NSML has resource-management and user-interaction modules, as shown in its high-level overview.The resource side comprises a scheduler and large-scale virtualized resources; the user side includes client tools and NSML.Source: §3.1 · p. 2, Figure 1 · p. 2
• After nodes register, NSML checks their resources and assigns each submitted machine-learning job to suitable computing resources.The scheduler operates across the registered computing nodes.Source: §3.1 · p. 2
• Command-line and web interfaces let users manage job pipelines and obtain trained models and experimental results; serving APIs support services using those models.The interface is described as supporting user experience and convenience.Source: §3.1 · pp. 2–3

3.2 Resources and Management

• Virtualizing pooled resources supports broader platform operation with limited resources and lets providers add capacity while the platform runs.NSML focuses its resource management on machine-learning workloads rather than complex production flows.Source: §3.2 · p. 3

3.2.1 Locality-Aware scheduling and residual resource defragmentation

• Locality-aware scheduling considers resident data and container images to mitigate setup overhead, while allocating GPU requests from nodes with fewer available GPUs first.This ordering can leave nodes with the largest GPU capacity available while other nodes are allocated.Source: §3.2.1 · p. 3

3.2.2 Scheduler fault tolerance

• A primary and warm-standby secondary scheduler provide robustness against primary-node failure, with possible additional computing-resource use.The scheduler design addresses the single-point-of-failure risk of a centralized system.Source: §3.2.2 · p. 3

3.2.3 Session and Resource Monitoring

• Resource monitors record CPU and GPU use for scheduling and user inspection, while session monitors track progress and report unexpected situations to users by email.The session monitor reports session status through a reporting chain so users can handle troubles.Source: §3.2.3 · p. 3

3.3 Datasets and Collaboration

• Datasets are copied to a node on demand and cached there, so later jobs requiring the same dataset can start without another copy.Public datasets are accessible to all users without limitation; identical code and data can make results easy to reproduce.Source: §3.3 · p. 3
• Figure 2 presents a dataset view in the NSML web application.The passage identifies the view but does not specify its contents.Source: Figure 2 · p. 4
• Teams can share sessions, results, models, and code, while private datasets can be limited to team members with permission.The dataset view exposes metadata such as size and latest access time.Source: §3.3 · p. 4

3.4.1 Command line interface

• The platform-independent CLI supports run and infer workflows and can launch existing code with a few additional lines.Its commands cover account management, session control, data analysis, and NSML services; some are administrator-only.Source: §3.4.1 · p. 4
• Administrators manage user permissions and cluster-use credits; exhausted credits can stop existing sessions and prevent new launches.Credits regulate monopolized cluster use under the credit policy.Source: §3.4.1 · p. 4
• Sessions retain code, logs, and metadata for reproduction or revision, while data-analysis commands collect progress and results for visualization, model retrieval, and session comparison.Resource requests may be rejected when available resources cannot meet their requirements.Source: §3.4.1 · p. 4

3.4.2 Web Interface and Visualization

• The web app supports session operations and simultaneous analysis, including scalar plots that compare sessions with different setups.The comparison panel distinguishes common and exclusive arguments, and the interface visualizes session ranks and resource usage.Source: §3.4.2 · p. 5

3.4.3 Serving API

• NSML can serve trained models through its client tool and RESTful APIs, with users submitting an inference procedure for web-based services.Users can test inference modules to confirm the submitted model's actions.Source: §3.4.3 · pp. 5–6

3.5 Hyperparameter tuning

• As the number of hyperparameters grows, the search space grows exponentially, making the best settings more difficult to find.The paper identifies hyperparameter optimization as one of the most time-consuming steps in building a machine-learning model.Source: §3.5 · p. 6
• NSML supports parallelized hyperparameter tuning.The passage does not specify a particular parallelization strategy.Source: §3.5 · p. 6
• Tuning is not limited to grid or random search; NSML can also apply algorithms such as population based training.The paper cites Jaderberg et al. (2017) for population based training.Source: §3.5 · p. 6

3.6 Implementations

• Containerized sessions manage memory and processors directly, supporting stable and reproducible machine-learning environments.NSML uses Docker as the virtualization layer and can stop or remove sessions.Source: §3.6 · p. 6
• NSML immediately kills a session that exceeds its allocated memory, protecting the host machine from damage.Source: §3.6 · p. 6
• The server-side system uses Python-compatible external components, allowing administrators to replace database choices; some CLI code uses Go for security and low latency.Source: §3.6 · p. 6
• NSML uses its own scheduler rather than a third-party library to manage host resources, including GPUs, at fine granularity.The scheduler runs in Docker for deployment and maintenance.Source: §3.6 · p. 6

4.1 Experimental Setup

• NSML reproduced prior MNIST, CIFAR-100, and ImageNet work, including 99.22% validation accuracy on MNIST.Training used one GPU for MNIST and CIFAR-100 and eight for ImageNet; the reported reproduction results appear in Table 2.Source: §4.1 · pp. 6–7, Table 2 · p. 7

4.2 ML Competitions via NSML

• In two of three competitions, the best models outperformed baselines used in real customer services, and were applied to enhance those services.Source: §4.2 · p. 7

4.2.1 Natural Language Tasks

• The NLP competition covered movie-review preference regression and duplicate-question recognition, using mean squared error and accuracy, respectively.It ran in three rounds: two online rounds with different dataset sizes, followed by a two-day offline final.Source: §4.2.1 · p. 7
• Most NLP participants were satisfied with NSML’s interface, usability, and visualization, but criticized robustness amid resource shortages and networking or database failures.Source: §4.2.1 · pp. 7–8
• Most NLP participants launched at least one session in phase one, while only a small proportion launched fewer than five models.The authors interpret these participant statistics as evidence of accessibility and user convenience.Source: Table 3 · p. 7, §4.2.1 · p. 8

4.2.2 Object-Rotating Angle Prediction

• In the internally run angle-prediction competition, slightly more than half used NSML fewer than five times, while the others trained frequently.The task predicted tilted-image angles, with accuracy measured using a one-degree error range.Source: §4.2.2 · p. 8

4.2.3 Keyboard-Typo Correction

• In keyboard-typo correction, over half of participants continued developing models on NSML after the competition midpoint.The internal competition ran for two and a half months and used F1 measure.Source: §4.2.3 · pp. 8–9

5.1 Resource Utilization

• Average GPU utilization exceeded 70%, reached about 100% at peak, and remained undiminished as the GPU count increased.The platform served hundreds of users; the authors highlight its use for experiments demanding massive computing resources.Source: §5.1 · p. 9
• After utilization visualization was introduced, GPUs above 80% utilization increased about 10%, while running GPUs increased less than 5%.The authors surmised users may have improved GPU-use strategies by referring to collected information.Source: §5.1 · p. 9

5.2 Limitations and Future Work

• NSML lacks dataset version control, making even minor revisions difficult to patch, and does not yet support multiple datasets in one task.The latter limitation follows from the current session representation; both features are identified as priorities for future work.Source: §5.2 · pp. 9–10
• Current visualization handles scalar plots but cannot readily express complex concepts, audio/video, embedding layers, or feature maps.Source: §5.2 · p. 10
• NSML is implementing distributed learning across multiple nodes for datasets or models difficult to load onto one machine.The authors expect this feature to improve overall utilization and relieve resource shortages caused by fragmented allocation.Source: §5.2 · p. 10

• NSML combines a machine-learning-as-a-service platform, collaborative tools, and a serverless foundation; competitions support its potential role in commercializing machine-learning models.Source: §6 · p. 10