PaperlayerView exact revision on arXiv

Source-linked AI summary

Effective Extensible Programming: Unleashing Julia on GPUs

Tim Besard, Christophe Foket, Bjorn De Sutter

arXiv:1712.03112v1cs.PLcs.DC

TL;DR

Accelerator programming is difficult because efficient device code is typically written with low-level toolchains, while high-level implementations may restrict language features or duplicate compiler functionality. The paper introduces extensible compiler interfaces, implements them in Julia, and uses them to support NVIDIA GPUs. The resulting toolchain supports high-level, flexible GPU programming with performance similar to CUDA C and improves reuse of existing Julia code and packages.

  • Problem

    High-level languages rarely integrate cleanly with accelerators because unsupported features, compiler duplication, and divergent implementations limit compatibility and productivity.

  • Method

    The paper adds fine-grained interfaces to Julia’s compiler so external device packages can adapt existing compilation infrastructure for NVIDIA GPUs.

  • Results

    The Julia GPU toolchain produces code with performance similar to CUDA C while enabling generic high-level GPU code and reuse of existing Julia packages.

  • Takeaways & Limitations

    Integrating GPU support with Julia’s existing compiler enables flexible high-level and dynamic GPU programming approaches within the Julia ecosystem.

  • Takeaways & Limitations

    The approach’s interfaces are currently designed around a single language and accelerator platform, and some existing host-library code remains opaque to the compiler for cross-call optimization.

Abstract

from arXiv · show

GPUs and other accelerators are popular devices for accelerating compute-intensive, parallelizable applications. However, programming these devices is a difficult task. Writing efficient device code is challenging, and is typically done in a low-level programming language. High-level languages are rarely supported, or do not integrate with the rest of the high-level language ecosystem. To overcome this, we propose compiler infrastructure to efficiently add support for new hardware or environments to an existing programming language. We evaluate our approach by adding support for NVIDIA GPUs to the Julia programming language. By integrating with the existing compiler, we significantly lower the cost to implement and maintain the new compiler, and facilitate reuse of existing application code. Moreover, use of the high-level Julia programming language enables new and dynamic approaches for GPU programming. This greatly improves programmer productivity, while maintaining application performance similar to that of the official NVIDIA CUDA toolkit.

1 INTRODUCTION

Accelerator programming offers performance advantages but is difficult to develop with low-level toolchains, while high-level-language implementations often diverge from their host languages. The paper proposes extensible compiler interfaces and demonstrates Julia GPU support that combines generic high-level code with CUDA-like performance.

  • Motivation: Accelerators improve performance for parallel workloads, but low-level toolchains such as CUDA and OpenCL trade developer productivity for control.Rapid hardware development also makes it difficult for developers to gain sufficient device-programming experience.
  • Motivation: High-level accelerator implementations commonly target restricted language subsets or embedded DSLs because features such as interpretation and managed runtimes are difficult to support on accelerators.These implementations can be incompatible with ordinary host-language code.
  • Problem: Derived-language implementations often use custom compilers, hindering maintainability as the host language changes and forcing users to manage semantic divergence.The passage identifies both long-term maintenance costs and differences between implementations.
  • Contribution: The paper introduces compiler interfaces, implements them in Julia, and uses them to support NVIDIA GPUs.The interfaces are intended to alter compilation while reusing the existing compiler.
  • Evaluation: Julia GPU code is shown to be generic and flexible without sacrificing performance.The benchmark analysis compares generated code with CUDA C compiled by NVIDIA’s reference compiler.

2 VISION

The proposed vision exposes fine-grained interfaces to a host language’s compiler so external device packages can adapt compilation without custom compiler duplication. The Julia and NVIDIA GPU instantiation emphasizes reuse, independent extension, and compatibility, while remaining scoped to one language and accelerator platform.

  • Compiler interfaces: Compiler interfaces expose fine-grained access to intermediate representations and the processes that generate and optimize them.This allows existing compiler functionality to be repurposed for new hardware or runtime environments.
  • External device packages: External device packages can use these interfaces to add hardware support without modifying the existing language implementation.Packages may reject incompatible features or replace them with compatible or optimized alternatives.
  • External device packages: Keeping device implementations external preserves the host language implementation’s stability and supports independent experimentation and vendor contributions.The paper connects this organization to the rapid development pace of accelerator hardware.
  • Reuse and compatibility: Reusing the existing compiler avoids duplicated functionality and improves compatibility with existing code across language implementations.The paper contrasts this with Numba’s duplicated compilation stages and version-specific maintenance.
  • Scope: The interfaces are instantiated for Julia and NVIDIA GPUs with CUDA, using Julia’s extensibility and LLVM-based compiler framework.The design is presented as conceptually general but currently developed around one language and accelerator platform.

3 BACKGROUND

GPUs offer substantial acceleration for parallel workloads but are difficult to program directly, while existing high-level approaches often sacrifice language integration or compiler reuse. Julia’s type specialization and extensibility provide a foundation for targeting accelerators, although its existing lower-level interfaces are insufficient.

  • 3.1 GPU Accelerators: GPUs accelerate compute-intensive applications but require separate memory spaces, host coordination, and suitable parallelism.These constraints make GPU programming relatively difficult.
  • 3.1 GPU Accelerators: Vendor GPU toolchains use low-level languages that expose hardware features and can avoid costly abstractions, but complicate programming.CUDA uses CUDA C, while AMD and Intel GPUs use OpenCL C.
  • 3.1 GPU Accelerators: Host libraries simplify accelerator access but prevent compilers from optimizing across calls and often provide coarse-grained abstractions that cannot compose with custom device code.Consequently, library-based programming can be unsuitable for some applications.
  • 3.1 GPU Accelerators: Existing high-level accelerator implementations use embedded DSLs or restricted language subsets, requiring code adaptation and duplicating compiler functionality.The passage cites PyGPU, Copperhead, and Numba as examples.
  • 3.2 Julia Programming Language: Julia combines high-level features with performance through type inference, multiple dispatch, and specialization that produce mostly statically-typed intermediate code.This design lets Julia reuse compiler frameworks such as LLVM without traditional tracing, speculative execution, or deoptimization.
  • 3.2 Julia Programming Language: Julia exposes reflection and metaprogramming interfaces for source code and high-level IR, but its LLVM interfaces mainly use string representations and do not suffice for accelerator targeting.The existing interfaces support the main compiler’s use cases but limit external device packages.

4 EFFECTIVE EXTENSIBLE PROGRAMMING

The paper proposes compiler extension interfaces that let external device packages alter code-generation processes while reusing the existing compiler. These interfaces support accelerator-specific code generation and higher-level LLVM interaction without requiring a custom compiler.

  • 4 EFFECTIVE EXTENSIBLE PROGRAMMING: Existing IR access alone cannot reuse the main compiler’s IR-generating components or support the broader code generation required for accelerators.The paper identifies both insufficient reuse of IR generation and limitations of string-based low-level IR interfaces.
  • 4 EFFECTIVE EXTENSIBLE PROGRAMMING: External language implementations need access to both generated IR and the processes that generate it, so incompatible runtime-dependent features can be avoided or replaced.Exceptions are an example because they rely on runtime support for stack unwinding and error reporting.
  • 4.1 Front-end IR Interfaces: Four new interfaces expose parameters and hooks that can reconfigure or replace lowering from ASTs to Julia IR and from Julia IR to LLVM IR.These controls improve compiler reusability while allowing accelerator-specific code generation.
  • 4.1 Front-end IR Interfaces: CodegenParam can disallow unsupported features, while CodegenHook can change generated code to avoid runtime-library dependencies.The GPU back end uses these mechanisms for exceptions, dynamic memory allocation, garbage collection, and other runtime-supported functionality.
  • 4.1 Front-end IR Interfaces: The interfaces currently target Julia IR and LLVM IR; parsing is considered generic enough not to require adjustment, while machine-code generation is too target-specific for substantial reuse.This defines the current scope of the extension design.
  • 4 EFFECTIVE EXTENSIBLE PROGRAMMING: Device-package developers can distribute and iterate accelerator implementations independently because no changes to the language compiler are required.The core language and compiler can remain stable while new implementations evolve through the package manager.
  • 4.2 Back-end IR Interfaces: LLVM.jl provides a high-level Julia wrapper around the LLVM C API for inspecting, modifying, and emitting LLVM IR.It improves usability of LLVM-level extension interfaces and supports reuse of back-end functionality.
  • 4.2 Back-end IR Interfaces: LLVM.jl avoids fragile string manipulation and makes accelerator-specific pointer-loading implementations more readable and adaptable across pointer types and memory optimizations.Without it, implementations would require string-heavy code and cases for every supported pointer type.

5 CUDA LANGUAGE IMPLEMENTATION

CUDAnative.jl adds NVIDIA GPU support to Julia through a package combining GPU functionality, compilation, and runtime management. Its compiler and metaprogramming interfaces promote code reuse, dynamic programming, and high-level abstractions without sacrificing performance.

  • Implementation: CUDAnative.jl implements an NVIDIA GPU version of Julia as a regular package without modifying the underlying Julia compiler.It supports an extensive Julia subset sufficient for real-life GPU applications and high-level abstractions.
  • Implementation: The device package combines a GPU-specific standard library, GPU compiler, and runtime system that together form a CUDA GPU JIT compiler.The runtime invokes and manages compilation together with the GPU hardware.
  • GPU programming support: Julia code provides many low-level GPU operations directly, including thread and block indexes, synchronization barriers, and shared-memory allocation.Julia’s expressiveness reduces reliance on compiler intrinsics for these definitions.
  • GPU programming support: Metaprogramming interfaces generate lower-level code for functionality such as atomics, while macros wrap LLVM IR snippets in user-friendly constructs.CUDAnative.jl uses these techniques with an LLVM API wrapper to implement GPU functionality.
  • Code reuse: Julia’s extensibility lets CUDAnative.jl extend or override standard-library behavior, including through type-based multiple dispatch.This improves compatibility or performance of existing language features.
  • Compiler extensions: The GPU compiler infers address spaces for pointers passed into kernels, addressing cases where host-side allocation sites are invisible.The optimization produces specialized memory operations for NVIDIA PTX state spaces.
  • Code reuse: Device code needs no explicit annotation or encapsulation, allowing compatible portions of the Julia standard library to be reused for GPU programming.This expands code-reuse opportunities across host and device contexts.
  • Compiler extensions: The address-space wrapper and inlining optimizations require less than 100 lines of Julia code, demonstrating reuse enabled by the tool-flow interfaces.The result relies on reusing existing compiler infrastructure rather than rebuilding it.

6 EVALUATION

The evaluation measures JIT compilation overhead, GPU runtime overhead, and high-level programming capabilities using CUDA-derived Julia benchmarks. It also tests whether high-level constructs can be used without sacrificing performance.

  • Evaluation design: The evaluation compares JIT compilation overhead with a static CUDA C toolchain and runtime performance using standardized benchmarks ported from CUDA C.The benchmark ports retain a low abstraction level for performance assessment.
  • Evaluation design: High-level Julia GPU kernels are evaluated to demonstrate whether typical Julia programming constructs can be used without sacrificing performance.
  • Evaluation design: Figure 4 compares first and subsequent CUDAnative.jl compilation times with reference NVRTC compilation times for various GPU kernels.

6.1 Experimental Set-up

The experiments use CUDA C with NVIDIA’s CUDA toolchain and Julia with specified compiler, package, driver, and operating-system versions.

  • Experimental environment: CUDA C uses NVIDIA CUDA compiler 8.0.61 with driver 375.66 on Debian Stretch Linux 4.9.0, while Julia measurements use Julia 0.6.Compilation-time measurements use a prerelease Julia 0.7, with specified versions of CUDAnative.jl, CUDAdrv.jl, and LLVM.jl.

6.2 JIT Compiler Performance

The JIT evaluation examines PTX generation time and kernel-launch overhead for interactive and static use. First compilation is costly, while subsequent compilation compares favorably with NVRTC but scales worse for more complex Julia source.

  • Metrics: The evaluation tracks PTX generation time and runtime launch overhead because CUDAnative.jl checks before every kernel launch whether new device code is needed.
  • 6.2.1 Code Generation: Figure 4 recompiles unique empty, vector-addition, peakflops, and high-level reduction kernels without cache hits, using a single-threaded process.
  • 6.2.1 Code Generation: The first GPU compilation incurs a significant penalty because Julia JIT-compiles CUDAnative.jl and its host dependencies.The penalty increases with kernel complexity as advanced kernels trigger compilation of more CUDAnative.jl functionality.
  • 6.2.1 Code Generation: Subsequent compilation timings are much lower because host code generation, FFI abstractions, device-library loading, and intermediate results are optimized or cached.
  • 6.2.1 Code Generation: CUDAnative.jl compares favorably with NVRTC compilation but scales worse because Julia source code is more complex to analyze.The authors expect compilation times to improve with future Julia versions.
  • 6.2.2 Run-time overhead: Dynamic CUDAnative.jl kernel launches perform additional argument conversion, compilation, and CUDA module setup compared with statically compiled CUDA C.These paths are optimized to minimize runtime overhead.
  • 6.2.2 Run-time overhead: Empty-kernel measurements compare static CUDA driver code, equivalent CUDAdrv.jl operations, and dynamic CUDAnative.jl compilation and execution using GPU events and CPU wall-clock timers.CUDAdrv.jl alone shows no significant overhead in either timing measurement.

6.3 Low-level Kernel Programming

CUDAnative.jl ports Rodinia CUDA C kernels to Julia while preserving low-level kernel semantics, reducing code size and achieving performance close to CUDA C.

  • Benchmark setup: Several CUDA C Rodinia benchmarks were ported to CUDAnative.jl, focusing on the suite’s smallest benchmarks and supported GPU features.The ports preserved low-level kernel semantics to isolate differences attributable to Julia and its compiler.
  • Code size: 32% shorter code was achieved on average, with device code reduced by 8% and host code by 38% relative to CUDA C.The comparison normalizes host and device lines of code against each benchmark’s total CUDA C implementation.
  • Measurement: Kernel timings excluded memory transfers and host-side operations, estimating each benchmark’s total from fitted per-kernel execution-time distributions.The reported speedup uses the geometric mean, with error margins propagated across operations.
  • Kernel performance: 0.50% slowdown was measured on average against CUDA C kernels compiled with nvcc.The result supports the conclusion that low-level GPU kernel programming in Julia does not incur a substantial slowdown.

6.4 High-level GPU Programming

CuArrays.jl demonstrates how CUDAnative.jl supports idiomatic, high-level GPU programming through fused broadcasts and generic reductions while retaining specialized device code.

  • CuArrays.jl abstractions: CuArrays.jl exposes host-level operations on GPU-resident arrays, combining predefined abstractions with device execution through CUDAnative.jl.The package is presented as a high-level interface for data processing on the GPU.
  • Broadcast: A dotted Julia expression is fused into one broadcast operation and compiled with inlined calls to both the lambda and auxiliary function.The resulting operation is specialized by CUDAnative.jl for execution on the device.
  • Language and compiler support: Julia specialization produces statically typed GPU code without type checks, while higher-order functions can compile without indirect calls or runtime calls.For the example, generated PTX is identical to equivalent CUDA C PTX except for slightly different inlining decisions.
  • Practical impact: The abstractions substantially reduce source code and remove API interactions for most use cases while supporting GPU-optimized data-processing algorithms.CuArrays.jl is described as demonstrating high-level abstractions enabled by the CUDAnative.jl JIT compiler.
  • Reduction: The reduce implementation uses GPU shuffle instructions and specializes kernels on reduction arguments, including custom objects such as Point{Int64}.Generated LLVM IR produces sequences of 32-bit shuffle instructions to move 128-bit objects between threads.

7 RELATED WORK

The work differs from accelerator approaches based on hosted DSLs or separate compilation steps by extending the main compiler and reusing its infrastructure.

  • Hosted DSLs: Prior high-level accelerator systems commonly host restricted DSLs, such as Accelerate’s embedded array language or Copperhead’s functional data-parallel subset.These approaches avoid depending directly on a lower-level device-specific language.
  • Integrated compilation: The proposed infrastructure lets programmers write accelerator code directly in the high-level source language while remaining integrated with the main compiler and ecosystem.This contrasts with approaches requiring manual build-time post-processing or manually constructed execution task graphs.
  • Novelty: Extending the main compiler reduces support effort and avoids duplicated compiler functionality, addressing a gap not previously focused on targeting hardware and environments through compiler-process extensibility.The paper distinguishes its approach from Rust’s experimental NVIDIA GPU support, which reuses low-level LLVM infrastructure but lacks the described extensibility.

8 CONCLUSION AND FUTURE WORK

The paper presents extensible compiler interfaces, instantiates them for Julia on NVIDIA GPUs, and demonstrates compatible high-level GPU programming with performance similar to CUDA C.

  • Conclusion: The proposed interfaces repurpose an existing compiler to add support for new hardware or environments while maximizing reuse of compiler functionality.The approach is presented as a way to extend an existing programming language efficiently.
  • Evaluation: Julia’s CUDAnative.jl infrastructure supports NVIDIA GPUs, and Rodinia experiments show Julia GPU code performs similarly to CUDA C.Integrating with the existing compiler improves code compatibility and allows many existing Julia packages to run on the GPU without extra effort.
  • Programming model: CUDAnative.jl enables high-level principles such as dynamically typed kernels, interactive tools, and flexible abstractions through CuArrays.jl.CuArrays.jl combines host-level library convenience with the flexibility of manual device programming.
  • Adoption: The compiler interfaces became part of Julia 0.6 and were subsequently used to target platforms including AMD GPUs and WebAssembly.These developments use LLVM.jl and often follow the CUDAnative.jl design.
  • Future work: Future work targets broader GPU feature support and high-level abstractions that preserve the ability to express low-level behavior.Planned compiler improvements include contextual method dispatch based on run-time device properties.
Loading 1712.03112v1…