Source-linked AI summary
SGX-LKL: Securing the Host OS Interface for Trusted Execution
Christian Priebe, Divya Muthukumaran, Joshua Lind, Huanzhou Zhu, Shujie Cui, Vasily A. Sartakov, Peter Pietzuch
TL;DR
Sensitive cloud workloads need protection from a host OS that can observe or manipulate runtime calls. SGX-LKL runs Linux binaries inside SGX enclaves with a minimal, protected, and oblivious host interface; with emulated SGX, oblivious calls add 14%–21% overhead for single-machine TensorFlow training. Its protection scope has boundaries, including observable application exceptions.
Problem
TEE runtimes rely on an untrusted host OS for external resources, and its interface can expose application state or enable integrity attacks.
Source: §1 · p. 1, §2.2 · p. 2Method
SGX-LKL runs Linux binaries in SGX enclaves using an in-enclave library OS and a minimal interface that protects I/O data and makes host operations workload-independent.
Source: §1 · pp. 1–2, §3.2 · p. 5Results
With emulated SGX and oblivious calls, TensorFlow training incurs 14%–21% overhead on one machine and 2.7×–2.9× in distributed deployment.
Source: §1 · p. 2Takeaways & Limitations
Ignoring SGX paging effects, SGX-LKL's performance overhead is low even for complex applications such as TensorFlow.
Source: §8 · p. 12SGX-LKL hides which instruction caused an application exception but cannot hide the exception itself from an adversary.
Source: §5.2 · p. 10
Abstract
from arXiv · showhide
Hardware support for trusted execution in modern CPUs enables tenants to shield their data processing workloads in otherwise untrusted cloud environments. Runtime systems for the trusted execution must rely on an interface to the untrusted host OS to use external resources such as storage, network, and other functions. Attackers may exploit this interface to leak data or corrupt the computation. We describe SGX-LKL, a system for running Linux binaries inside of Intel SGX enclaves that only exposes a minimal, protected and oblivious host interface: the interface is (i) minimal because SGX-LKL uses a complete library OS inside the enclave, including file system and network stacks, which requires a host interface with only 7 calls; (ii) protected because SGX-LKL transparently encrypts and integrity-protects all data passed via low-level I/O operations; and (iii) oblivious because SGX-LKL performs host operations independently of the application workload. For oblivious disk I/O, SGX-LKL uses an encrypted ext4 file system with shuffled disk blocks. We show that SGX-LKL protects TensorFlow training with a 21% overhead.
1 Introduction
SGX-LKL is designed to run Linux binaries inside SGX enclaves while limiting what an untrusted host can learn or alter through the host interface. It combines an in-enclave library OS with protected I/O and workload-independent host operations.
1 Introduction
- Host interfaces can leak sensitive data or application state and can let a malicious host compromise computation integrity.Applications in a TEE rely on the untrusted host OS for external resources such as files and networking.Source: §1 · p. 1
- SGX-LKL targets a minimal, protected, and oblivious host interface for executing unmodified Linux binaries inside SGX enclaves.Its design aims to keep host calls independent of application state while encrypting and integrity-protecting data crossing the interface.Source: §1 · p. 1
- A complete in-enclave Linux-based library OS provides POSIX functionality, leaving the host interface with only seven low-level block- and packet-I/O calls.SGX-LKL uses LKL for its virtual file system and TCP/IP network stack.Source: §1 · pp. 1–2
- SGX-LKL protects file I/O with an encrypted ext4 image and network I/O with a VPN overlay, while fixed batches, shuffled blocks, and fixed-size traffic support obliviousness.The host interface uses indistinguishable dummy calls in fixed-size batches; disk blocks are randomized and shuffled, and network traffic is shaped to a constant bitrate between trusted nodes.Source: §1 · p. 2
- With emulated SGX and oblivious calls, TensorFlow training incurs 14%–21% overhead on one machine and 2.7×–2.9× in distributed deployment.With SGX hardware, PARSEC benchmarks that fit into SGX memory incur 3.1× overhead with oblivious calls and 1.5× without them.Source: §1 · p. 2
2 Host Interfaces for Trusted Execution
This section explains why TEE applications need host interfaces, how those interfaces create confidentiality and integrity risks, and how SGX-LKL's design principles narrow the interface and its parameters.
2.1 Trusted execution environments in clouds
- SGX enclaves isolate code and memory but lack I/O access, so lift-and-shift Linux applications need a runtime that relies on the untrusted host for external resources.The host interface is the interface between the TEE runtime system and the host.Source: §2.1 · p. 2
2.2 Security goals and threat model
- The threat model trusts the TEE but treats software outside it as adversarial, including a host that can observe calls or alter their parameters to attack confidentiality or integrity.Observed call patterns can reveal disk access and workload information; manipulated calls can enable data exposure or control-flow hijacking.Source: §2.2 · p. 2
2.3 Host interfaces of current TEE runtime systems
- Existing runtimes make different host-interface trade-offs, from forwarding libc calls to using an in-enclave library OS, while exposed buffers, identifiers, and semantic parameters create leakage or integrity risks.Some systems disclose buffer sizes or file sizes, and unprotected semantic parameters can reveal application state.Source: §2.3 · pp. 3–4
2.4 Designing a secure host interface
- Because host parameters are difficult to protect and validate, a secure design keeps the interface narrow and delegates only functionality that cannot be provided inside the enclave.SGX-LKL avoids address ranges and identifiers and uses only a few simple semantic parameters, all but time having fixed valid values or bounds checks.Source: §2.4 · p. 4
- Using low-level calls rather than host-provided POSIX abstractions simplifies parameter types and avoids delegating high-level resource management to the host.For example, relying on the host file system requires exposing impure identifiers such as file names.Source: §2.4 · p. 4
3 Minimising the host interface
SGX-LKL minimizes host interaction by keeping Linux OS functionality inside the enclave and exposing only seven host calls for low-level resources. Its in-enclave library OS supplies POSIX functionality while working around constraints of SGX version 1.
3.1 SGX-LKL host interface
- Only 7 host calls expose functionality that cannot be provided inside enclaves, covering low-level I/O, events, signals, and time.Disk and network calls operate on blocks and packets rather than POSIX file or socket operations.Source: §3 · p. 4, §3.1 · pp. 4–5
3.2 In-enclave OS functionality
- LKL provides an in-enclave library OS, with a modified musl library redirecting system calls to its Linux-based functionality.Its VFS layer supplies file systems, while LKL also makes kernel components such as the network stack available inside the enclave.Source: §3.2 · p. 5
- The in-enclave VFS needs only two disk I/O calls, hiding individual file accesses from the host, which sees disk-block offsets.The VFS also supports file systems such as the in-memory /tmp, /proc, and /dev.Source: §3.2 · pp. 5–6
- LKL’s in-enclave TCP/IP stack provides POSIX sockets while exposing a virtual network device for Ethernet frames, supporting transports such as UDP without extra host calls.The device backend uses net_poll(), net_read(), and net_write() to interact with the host.Source: §3.2 · p. 6
- SGX fixes enclave size at initialization, so SGX-LKL pre-allocates enclave memory and manages allocations inside the enclave.The heap is exposed through LKL and libc memory-management functions.Source: §3.2 · p. 6
- SGX version 1 fixes page permissions at enclave creation, so SGX-LKL uses an extra mem_protect() host call and host-controlled page-table permissions as a workaround.The paper identifies relying on the host to manage permissions as a security risk; SGX version 2 allows permission control from within the enclave.Source: §3.2 · p. 6
- A fixed number of host threads are assigned at startup; SGX-LKL uses in-enclave user-level threads and synchronization primitives to manage application threads.The maximum number of enclave threads must be set at enclave creation, and host-supported thread operations pose a security risk.Source: §3.2 · p. 6
- SGX version 1’s RDTSCP restriction makes clock_gettime() accuracy depend on vDSO update frequency; SGX-LKL also emulates RDTSC and caches CPUID information.The vDSO supplies time from shared memory, while RDTSC is executed outside the enclave and CPUID data is cached during setup.Source: §3.2 · p. 6
4 Protecting the host interface
SGX-LKL protects host-interface I/O with encryption and integrity mechanisms, and handles event and time inputs from the untrusted host. Its deployment workflow provisions an application, attests the enclave, and then provisions secrets.
4.1 Protecting disk I/O calls
- SGX-LKL combines device-mapper targets to protect disk I/O confidentiality and integrity according to application requirements.dm-crypt provides encryption; dm-verity provides read-only integrity; dm-integrity provides read/write integrity, and dm-crypt can be combined with dm-integrity for encrypted read/write integrity.Source: §4.1 · p. 7
4.2 Protecting network I/O calls
- SGX-LKL encrypts, authenticates, and integrity-protects all data sent through net_read() and net_write() using Wireguard.Its in-enclave TCP/IP stack enables transparent low-level network encryption, including for applications without TLS support or using UDP.Source: §4.2 · p. 7
4.3 Protecting event and time calls
- SGX-LKL checks monotonicity for CLOCK_MONOTONIC_* inputs and validates forwarded signals; invalid network events either cause a handled read failure or a packet rejected by Wireguard.The host may return untrusted timestamps, event masks, or signal descriptions, so SGX-LKL checks these inputs or relies on Wireguard integrity protection.Source: §4.3 · p. 7
4.4 Runtime attestation and secret provisioning
- Deployment proceeds through application provisioning, remote attestation, and secret provisioning among the service provider, cloud host, and Intel Attestation Service.After verification, the service provider establishes a secure channel and sends disk keys, a root hash, trusted-peer keys, and application configuration to the enclave.Source: §4.4 · p. 8
5 Making the host interface oblivious
SGX-LKL makes observable host calls independent of the application workload to prevent interface side channels. It uses batching, block shuffling, and constant-rate network traffic to obscure disk and network activity.
5 Making the host interface oblivious
- Host-call frequencies, sequences, and parameters can reveal execution behavior and file layout; SGX-LKL aims to make these observations workload independent.Repeated disk offsets can disclose the data layout.Source: §5 · p. 8
5.1 Disk I/O calls
- Disk calls run in fixed-interval batches and a deterministic order, with dummy calls padding underfilled batches and excess calls delayed.The interval t must be tuned: the experiments use 0.1 ms, which works well across a range of workloads.Source: §5.1 · p. 8
- Probabilistic re-encryption before writes hides buffer contents and prevents an adversary from distinguishing old data from new data.Source: §5.1 · p. 9
- SGX-LKL randomizes disk layout and shuffles blocks using a random permutation so an adversary cannot correlate block accesses across the shuffle.Its k-oblivious shuffle assumes k of n source blocks are in the private cache.Source: §5.1 · p. 9
- The ext4 shuffle maps source blocks to a new layout, using donor files to swap blocks and updating filesystem metadata.The algorithm uses a Fisher-Yates permutation; donor-file block locations are random and intermixed with regular file blocks.Source: §5.1 · p. 9
- Network calls use fixed-size packets and dummy traffic to maintain a constant rate between trusted nodes.SGX-LKL sends dummy packets only as needed to reach each peer's rate limit and pads packets to the MTU size.Source: §5.1 · pp. 9–10
- This network approach has overhead proportional to the number of trusted nodes; routing through one trusted proxy is an alternative for scaling and hiding potential recipients.Source: §5.1 · p. 10
5.2 Other host calls
- Application-caused signals still expose exceptions, while frequent scheduler reads of vDSO time obscure application-specific access patterns.SGX-LKL hides which instruction caused a signal but cannot hide the exception.Source: §5.2 · p. 10
6 Evaluation
SGX-LKL was evaluated with TensorFlow and PARSEC workloads, plus disk and network I/O benchmarks, in software and hardware modes. Results show strong TensorFlow performance without oblivious calls, while oblivious I/O and EPC paging impose workload-dependent costs.
6.1 Application performance
- Inference throughput exceeded native by 20–42% across the reported models, but with oblivious calls it was 15–79% lower than native.CPU-intensive networks were less affected than I/O-intensive networks.Source: Figure 4 · p. 10, §6.1 · pp. 10–11
- 28%, 33%, and 32% higher training throughput than native for ResNet-34, ResNet-50, and ResNet-101, respectively; disk encryption and integrity-checking overhead was negligible.The comparison uses SGX-LKL in software mode.Source: §6.1 · p. 10, Figure 3 · p. 10
- With oblivious calls and disk encryption, AlexNet throughput fell 24% and 37% versus software mode, or 14% and 21% versus native, while ResNet performance was almost equal to native.The authors attribute AlexNet's greater impact to its higher image-processing rate and resulting need for more shuffle operations.Source: §6.1 · p. 10
- With four workers, distributed training was 3% slower than native without network encryption, 52% with WireGuard, and 34% with oblivious network I/O at 200 Mbps per peer.The authors describe the security overhead as potentially compensable by scaling out.Source: Figure 5 · p. 10, §6.1 · p. 11
- With oblivious calls, PARSEC overhead reached 2× in software and 10× for fluidanimate and 21× for canneal in hardware, where EPC paging contributed to the slowdown.Without oblivious calls, software-mode execution was 1.05× slower than native overall; hardware-mode slowdowns also depended on working-set size.Source: Figure 6 · p. 11, §6.1 · p. 11
6.2 Disk I/O performance
- 510 MB/sec SSD throughput was achieved without protection; full disk encryption reduced it to 320 MB/sec (62% of native), and adding integrity protection reduced it to around 230 MB/sec (45%).The tests measured sequential reads of an uncached 1 GB file.Source: Figure 7 · p. 11, §6.2 · p. 11
6.3 Network I/O performance
- With 64 KB buffers, network throughput reached 8.4 Gbps in hardware and 8.8 Gbps in software, versus native saturation at about 9.4 Gbps with 4 KB buffers.With WireGuard, throughput rose to 2 Gbps for buffers larger than 4 KB, where decryption became the bottleneck.Source: Figure 8 · p. 11, §6.3 · pp. 11–12
7 Related Work
Prior TEE runtimes and ORAM systems address parts of the host-interface security problem, while related oblivious file systems lack POSIX semantics. Other work hides communication metadata, an approach SGX-LKL can leverage.
7 Related Work
- Existing TEE runtimes expose wider host interfaces than SGX-LKL and lack obliviousness; Ryoan makes data flow input-independent but not the host interface.Source: §7 · p. 12
- ORAM-based TEE systems hide access patterns, but hosts can still mount Iago attacks when returning data blocks to enclaves [39].Source: §7 · p. 12
- Oblivious file systems support file access or transactions, but the cited approaches do not provide a file-system abstraction with POSIX semantics.Source: §7 · p. 12
- Tor can face traffic analysis, while DC networks broadcast to all users at high overhead; PIR and differential privacy add noise, techniques SGX-LKL can leverage.Source: §7 · p. 12
8 Conclusion
SGX-LKL combines a minimal, oblivious host interface with Linux-provided POSIX abstractions inside SGX enclaves and shuffled disk blocks. Ignoring SGX paging effects, its performance overhead is low even for complex applications such as TensorFlow.
8 Conclusion
- Ignoring SGX paging effects, SGX-LKL has low performance overhead even for complex applications such as TensorFlow.Source: §8 · p. 12
- SGX-LKL executes Linux binaries by using the Linux kernel to provide POSIX abstractions inside SGX enclaves.Source: §8 · p. 12
- SGX-LKL shuffles disk blocks to hide access patterns from the host.Source: §8 · p. 12