PaperlayerView exact revision on arXiv

Source-linked AI summary

Unikraft: Fast, Specialized Unikernels the Easy Way

Simon Kuenzer, Vlad-Andrei Bădoiu, Hugo Lefeuvre, Sharan Santhanam, Alexander Jung, Gaulthier Gain, Cyril Soldani, Costin Lupu, Ştefan Teodorescu, Costi Răducanu, Cristian Banu, Laurent Mathy, Răzvan Deaconescu, Costin Raiciu, Felipe Huici

arXiv:2104.12721v1cs.OS

TL;DR

Unikernels can provide strong performance, but extracting it and porting applications can demand substantial engineering effort. Unikraft modularizes OS components and offers composable APIs for specialization; evaluations report improved performance over Linux guests alongside small images, low memory use, and fast boot times.

  • Problem

    Unikernels offer performance benefits, but extracting that performance and porting applications can require substantial engineering effort.

    Source: §1 · p. 1

  • Method

    Unikraft is a micro-library OS that modularizes OS primitives and provides composable performance-oriented APIs for selecting and replacing components.

    Source: §1 · p. 2

  • Results

    Unikraft applications achieve a 1.7x–2.7x performance improvement over Linux guests, with images around 1MB, under 10MB of RAM, and boot time around 1ms beyond VMM time.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The evaluation shows that off-the-shelf applications including nginx, SQLite, and Redis can run on Unikraft with improved performance and compact resource use.

    Source: §1 · p. 2

    Unikraft currently does not support processes and related functions such as fork() and exec().

    Source: §7 · p. 13

Abstract

from arXiv · show

Unikernels are famous for providing excellent performance in terms of boot times, throughput and memory consumption, to name a few metrics. However, they are infamous for making it hard and extremely time consuming to extract such performance, and for needing significant engineering effort in order to port applications to them. We introduce Unikraft, a novel micro-library OS that (1) fully modularizes OS primitives so that it is easy to customize the unikernel and include only relevant components and (2) exposes a set of composable, performance-oriented APIs in order to make it easy for developers to obtain high performance. Our evaluation using off-the-shelf applications such as nginx, SQLite, and Redis shows that running them on Unikraft results in a 1.7x-2.7x performance improvement compared to Linux guests. In addition, Unikraft images for these apps are around 1MB, require less than 10MB of RAM to run, and boot in around 1ms on top of the VMM time (total boot time 3ms-40ms). Unikraft is a Linux Foundation open source project and can be found at www.unikraft.org.

1 Introduction

Unikernels offer strong performance but demand substantial engineering to specialize and port applications. Unikraft combines modular OS components and composable performance-oriented APIs; its applications outperform Linux guests while retaining small images and low resource use.

1 Introduction

  • Unikernels can deliver strong throughput, memory, and boot-time benefits, but extracting performance often requires expert work repeated for each application and substantial porting.Existing projects may have intertwined APIs that require developers to understand kernel internals to optimize applications.Source: §1 · p. 1
  • Unikraft makes specialization easier through standalone OS-primitives micro-libraries and composable, performance-minded APIs that can be selected and replaced.The APIs are designed with performance features such as batching and minimality in mind.Source: §1 · p. 2
  • Musl and a syscall shim let applications use native build systems, while Unikraft also supports applications including SQLite, nginx, and Redis, plus several languages and VMMs.Supported languages and runtimes include C/C++, Go, Python, Ruby, Web Assembly, and Lua.Source: §1 · p. 2
  • 1.7x–2.7x performance improvement over Linux guests, with app images around 1MB, under 10MB of RAM, and roughly 1ms boot time beyond VMM time.Total boot time is reported as 2ms–40ms.Source: §1 · p. 2

2 Design Principles and Solution Space

The design problem is to make application-specific OS specialization practical without requiring extensive redevelopment or difficult application porting. Unikraft’s design choices respond to application-specific needs and the limits of existing OS and unikernel approaches.

2 Design Principles and Solution Space

  • Applications may not need protection-domain switches, multiple address spaces, schedulers, full network stacks, filesystem layers, or general-purpose allocators, making selective removal valuable but difficult in standard OSes.The examples include run-to-completion event loops for RPC-style servers and direct driver access for some UDP applications.Source: §2 · p. 2
  • The target is a specialized OS for each application that supports high performance while bounding OS-related development effort and enabling easy porting of existing applications.The proposed design targets single-application scenarios, modular components, static linking, POSIX support, and platform abstraction.Source: §2 · pp. 2–3
  • Existing approaches modify general-purpose OSes, reuse unikernel projects, bypass the OS, or build application-specific functionality from scratch, but retain tradeoffs in monolithic structure, dependencies, or repeated porting.The scratch-built approach can yield lean, fast images, but its porting effort is large and mostly repeated for each application or language.Source: §2 · pp. 3–4
  • Linux component dependencies are dense: the dependency analysis finds that removing or replacing one component can require understanding and fixing dependencies across other components.The analysis approximated components using kernel source subdirectories and recorded cross-component function calls.Source: §2 · p. 3
  • Because the reviewed projects were not designed around the desired principles, Unikraft adopts a clean-slate API design while reusing existing components where relevant..Source: §2 · p. 4

3 Unikraft Architecture and APIs

Unikraft uses modular components and well-defined APIs to let applications select system functionality and specialize critical paths. Its architecture extends that approach to networking, allocation, and scheduling interfaces.

3 Unikraft Architecture and APIs

  • Unikraft combines fine-grained components communicating through defined APIs with static linking, aiming to gain performance without short-circuiting API boundaries.Micro-libraries implement core APIs and are interchangeable; a Kconfig-based build system selects components and platforms and links the selected libraries into binaries.Source: §3 · p. 4
  • A minimal Hello World image is 200KB on KVM and 40KB on Xen, while the nginx image omits the block subsystem because it uses RamFS and has smaller components with fewer dependencies than Linux counterparts.The Hello World image uses platform bootstrapping code and nolibc; these examples illustrate adding and removing components.Source: §3 · p. 4
  • Applications can use POSIX compatibility or nolibc, select allocators, and provide custom boot code; lower-level APIs offer another route to specialize performance-critical paths.The architecture caption describes applications plugging into APIs at different levels and choosing among implementations.Source: §3 · p. 4, Figure 4 · p. 5

3.1 uknetdev API

  • uknetdev decouples device drivers from network stacks or applications, allowing driver reuse across platforms and letting applications select polling, interrupt-driven, or mixed operation.The API is designed not to restrict varied use cases or become a bottleneck for high-performance workloads.Source: §3.1 · p. 5
  • uknetdev leaves packet-buffer memory management to applications and supports multiple queues, zero-copy I/O, and batching, with application-selected driver properties and features.Applications can use preallocated buffer pools or allocate buffers from the standard heap.Source: §3.1 · p. 5
  • Transmit and receive burst calls process arrays of packet buffers; queue interrupts can activate when polling finds no more work, avoiding interrupt storms and allowing a transition back to polling under heavy load.The transmit count reports how many packets entered the send queue; receive works similarly.Source: §3.1 · pp. 5–6
  • ukalloc multiplexes allocation requests across separate allocator backends, and uk_malloc takes the chosen backend as an argument to avoid an additional allocation-path call.Allocators initialize early in boot and register with the interface; supported backends include buddy, TLSF, tinyalloc, Mimalloc, and Oscar.Source: §3.1 · p. 6
  • Scheduling is optional and can use multiple schedulers, including different policies on data-plane and control-plane virtual CPUs; uklock selects or compiles out synchronization primitives according to threading and multicore configuration.Unikraft supports cooperative and preemptive schedulers; multicore support is described as not yet available.Source: §3.1 · p. 6

4 Application Support and Porting

Unikraft links applications built with their native systems into the unikernel and uses musl compatibility to ease porting. The section assesses application syscall coverage and porting effort as the project matured.

4 Application Support and Porting

  • Runtime-translated syscalls in Unikraft are 2-3x faster than in Linux, but cost tenfold compared with function calls.The reported syscall comparison depends on whether KPTI and other mitigations are enabled.Source: §4 · p. 7
  • Native builds link statically compiled application objects into Unikraft; the syscall shim registers handlers and turns syscalls into inexpensive function calls.Source: §4 · p. 7

4.1 Application Compatibility

  • Most libraries build fully automatically with musl, and a glibc compatibility layer lets almost all libraries and applications compile and link.The comparison is reported in Table 2.Source: §4 · p. 7, Table 2 · p. 7
  • Unikraft implements 146 syscalls; more than half are unnecessary for popular applications, and most needed syscalls are already supported.The analysis compares application requirements with Unikraft’s current syscall support.Source: §4.1 · pp. 7–8
  • All 30 selected applications are close to full syscall support; some already run with missing calls stubbed or returning ENOSYS.Missing calls may be quickly stubbed, relatively easy to implement, or still in progress.Source: §4.1 · p. 8

4.2 Manual Porting

  • For applications without source code, Unikraft supports binary compatibility; projects using its supported languages and environments should work out of the box.Source: §4.1 · p. 8, §4.2 · pp. 8–9
  • When automation fails and performance is paramount, Unikraft uses manual porting; its common code base makes this less time-consuming than past projects.The authors cite a few lines of glue code for ports, compared with past projects that took months to assemble.Source: §4.2 · p. 8
  • A survey of about 70 developers found that time spent porting dependencies and implementing missing OS primitives decreased significantly from March 2019 to May 2020.Respondents estimated time spent on the application or library, its dependencies, and missing OS or build-system functionality.Source: §4.2 · p. 8, Figure 6 · p. 8

5 Base Evaluation

The base evaluation tests Unikraft’s resource efficiency and performance with standard and automatically ported applications, then explores allocator choices. It finds strong performance alongside small images, fast boots, and low memory requirements.

5.1 Resource Efficiency: Smaller is Better

  • Images for helloworld, nginx, Redis, and SQLite are all under 2MB; guest boot takes tens to hundreds of microseconds on QEMU and Solo5 and no more than 1ms on Firecracker.Total VM boot time ranges from 3ms with Solo5 and Firecracker to around 40ms with QEMU; 2-6MB suffices to boot Unikraft guests.Source: §5.1 · p. 9, Figure 8 · p. 10

5.2 Filesystem Performance

  • Unikraft's 9pfs has lower read and write latency than a Linux VM across different block sizes.The evaluation measures reads and writes using a host-resident 1GB filesystem containing random data.Source: §5.2 · p. 9, Figure 13 · p. 10

5.3 Application Throughput

  • Unikraft is 30%-80% faster than containers, 70%-170% faster than Linux VMs, and 10%-60% faster than native Linux for nginx and Redis.The comparison uses the same application configuration where possible; Unikraft measurements use Mimalloc.Source: §5.3 · p. 10, Figure 12 · p. 10, Figure 13 · p. 10

5.4 Performance of Automatically Ported Apps

  • Automatic SQLite porting is only 1.5% slower than the manually ported version and slightly faster than Linux bare metal.The automatically ported app is built with SQLite's build system and linked as a static library; only compilation flags are needed beyond that.Source: §5.4 · p. 11

5.5 Memory Allocators

  • Allocator choice changes boot and runtime performance: nginx boot ranges from 0.49ms with bootalloc to 3.07ms with buddy, while tinyalloc takes a 30% runtime performance hit.For SQLite, tinyalloc is fastest below 1,000 queries but becomes suboptimal at higher loads, where mimalloc provides a 20% boost; Redis results show workload-specific choices can boost performance by 2.5x.Source: §5.5 · p. 11, Figure 14 · p. 11, Figure 15 · p. 11

6 Specializing Applications

The specialization evaluation examines how changing boot, networking, filesystem, and application components affects performance. These experiments show that workload-specific choices can yield large gains, sometimes with trade-offs in resources or implementation effort.

6.1 Specialized Boot Code (scenario 5)

  • Preinitialized paging boots in 30us for 1GB; dynamic page tables are populated at boot and boot time increases with memory, while protected 32-bit mode can disable paging.A guest with a 32MB dynamic page table takes slightly longer to boot than one with a preinitialized 1GB page table.Source: §6.1 · p. 12, Figure 21 · p. 12

6.2 Networking Performance (scenario 7)

  • With vhost-user offloading, uknetdev achieves throughput similar to DPDK running in a Linux VM.The API is designed to support high-performance networking across virtualization technologies; vhost-user offloads the datapath at the cost of host polling.Source: §6.2 · p. 12, Figure 19 · p. 12

6.3 VFS and Filesystem Specialization (scenario 3)

  • For the web-cache specialization that removes Unikraft’s vfscore layer and connects the application directly to SHFS, performance improved 5–7x over nonspecialized Unikraft and even more over Linux.Source: §6.3 · pp. 12–13

6.4 Specializing a key-value store (Scenario 7)

  • Specializing the key-value store to use uknetdev matches DPDK performance with one core, a 1.4MB image, and an 80ms boot; the DPDK guest result uses two cores, one exclusively for DPDK.The Unikraft version removes lwip and the scheduler and uses uknetdev in polling mode; porting effort is similar to the DPDK version.Source: §6.4 · p. 13

7 Discussion

The discussion describes security and debugging facilities while noting a current process-support gap. Unikraft supports some standard security features and offers multiple ways to aid debugging, but does not currently support processes or related functions such as fork() and exec().

Security

  • Unikraft supports CFI, Address Sanitisation, and initial Intel MPK support, alongside a small Trusted Computing Base associated with specialization.The paper says good security should be possible while retaining high performance.Source: §7 · p. 13

Debugging

  • Debugging support includes configurable messages and assertions, trace points, disassembler integration, and a Linux user-space target for development with Linux tools.The paper notes that specialized operating systems and unikernels often have less extensive debugging tools than Linux.Source: §7 · p. 13

Processes (or lack thereof)

  • Unikraft currently lacks processes and functions such as fork() and exec(), though processes could be implemented by cloning VMs or emulating them with intra-address-space isolation.The paper notes that many modern applications do not depend on processes and that some provide configurable thread-based alternatives.Source: §7 · p. 13

8 Related Work

Related work spans unikernels, library OSes, software-stack specialization, and kernel-bypass systems. Unikraft is distinguished by supporting mainstream applications and performance-oriented customization.

Unikernels and Library OSes

  • Unlike many application- or language-specific unikernels, Unikraft supports a range of mainstream applications and runtimes efficiently.The comparison includes projects targeting JavaScript, C++, Haskell, Erlang, Python, networking, and caching.Source: §8 · p. 13
  • Several library OSes and unikernel systems trade customizability or compatibility for other goals; Rump, HermiTux, and OSv are described as monolithic or difficult to customize.The passage also notes HermiTux’s binary-compatibility performance costs and Rump’s compatibility for standard applications.Source: §8 · p. 14

Stack and API Specialization

  • Stack-specialization work includes Sandstorm’s hand-customized network stack, while the null-Kernel offers mixable abstraction levels but no implementation or application-compatibility details.Application-Specific Software Stacks and HALO are described as complementary optimization approaches.Source: §8 · p. 14

Kernel Bypass

  • Kernel-bypass systems target I/O-bound workloads, whereas Unikraft’s customizable APIs aim for high performance without bypassing the kernel.The related work includes systems providing direct or partial access to I/O devices.Source: §8 · p. 14

Bridging the Userspace / Kernel Divide

  • Unikraft contrasts with approaches that avoid user-kernel switches but do not specialize the software stack, and with Lupine, which the authors report underperforms it.The cited approaches include UML, LibOS, LKL, UKL, and Lupine.Source: §8 · p. 14

9 Conclusions

The conclusion presents Unikraft as a micro-library OS for easy specialization and reports benefits for unmodified applications. It identifies improved syscall compatibility and security as future work.

9 Conclusions

  • Unikraft targets high performance through full, easy specialization of unikernel images, and slight application changes to use its APIs result in even higher performance.The conclusion also notes faster boot times and lower memory consumption for unmodified applications.Source: §9 · p. 14
  • The authors identify better syscall compatibility as ongoing work to transparently support more mainstream applications.They also plan to explore memory-safe or statically verifiable micro-libraries and compartmentalization for security.Source: §9 · p. 14

Artifact Appendix

The artifact appendix describes experiment reproducibility, hardware and software preparation, execution, and access to further project resources.

Artifact Appendix

  • The appendix provides instructions for reproducing the experiments and estimates the time needed to run them.The Artifact Evaluation repository is identified as.Source: Artifact Appendix · p. 15

9.1 Hardware Requirements

  • The experiments use Linux hosts with KVM, a separate 10-Gbit/s link for DPDK experiments, and a Xen host for 9pfs; the DPDK setup requires two servers.The appendix also describes a roughly €800 Shuttle host with an Intel i7 9700K and 32 GB RAM.Source: §9.1 · p. 15

9.2 Software Requirements

  • The software setup uses Debian Buster and Linux 4.19, with host boot parameters disabling Hyper-Threading, isolating CPU cores, and disabling IOMMU and IPv6.Some experiments require additional kernel parameters, including for CPU-frequency scaling governors.Source: §9.2 · pp. 15–16
  • The appendix’s evaluation materials cover syscall support, minimum memory, Redis and NGINX performance, allocator boot time, and SQLite insertion time.The figure captions identify these as the respective comparison scopes.Source: Figure 5 · p. 15, Figure 11 · p. 15, Figure 12 · p. 15
  • Other evaluation materials describe 9pfs read/write latency, automated-porting outcomes with musl and newlib, and a specialized UDP key-value store comparison.The cited table caption specifies port success with and without the glibc compatibility layer.Source: Figure 20 · p. 15, Table 4 · p. 16

9.3 Getting Started

  • The run script prepares dependencies and experiments; users can prepare one experiment or all experiments before invoking the desired run.Preparing all experiments usually takes several hours.Source: §9.3 · p. 16

9.4 Notes

  • Experiments require root privileges, about 50 GB of disk space, and sequential rather than parallel execution because CPU-core affinities overlap.The appendix also notes that some experiments can print errors while still completing and producing plots.Source: §9.4 · p. 16

9.5 Beyond the Paper

  • The appendix points readers to Unikraft documentation and invites researchers to join its community through the Xen mailing list and GitHub.The artifact repository contains the performance evaluation, while the project documentation covers real-world use.Source: §9.5 · p. 16
Loading 2104.12721v1…