PaperlayerView exact revision on arXiv

Source-linked AI summary

An Efficient Out-of-Core Tomographic Imaging Framework for Edge Devices

Xuetao Chen, Cong Ma, Xiangyu Meng, Du Wu, Zhengyang Bai, Tao Luo, Zhaorui Zhang, Emmanuel Jeannot, Edgar Josafat Martinez Noriega, Xun Wang, Peng Chen, Amelie Chi Zhou, Mohamed Wahib

arXiv:2609.07249v1cs.DC

TL;DR

Edge CT reconstruction must contend with limited compute, memory, and energy, while high-resolution volumes can exceed device memory. edgeFBP combines out-of-core partitioning with Tensor Core-accelerated back-projection on Jetson; evaluations report throughput gains over RTK and higher energy efficiency than a DGX system.

  • Problem

    Edge CT reconstruction faces limited computational power, memory capacity, and energy, while high-resolution volumes can exceed available on-device memory.

    Source: §I · p. 1, §9 · p. 3

  • Method

    edgeFBP partitions projections and volumes for out-of-core Jetson reconstruction and uses mixed-precision Tensor Core matrix operations for back-projection.

    Source: §I · p. 2

  • Results

    edgeFBP averages 1.83× over RTK on Jetson Orin Nano, improves throughput by 2.56× on Jetson AGX Orin, and achieves up to 5∼48× higher energy efficiency than an 8×A100 DGX system.

    Source: §I · p. 2

  • Takeaways & Limitations

    The results demonstrate the feasibility of high-quality tomographic imaging on edge platforms for portable medical, industrial, and scientific imaging applications.

    Source: §VI · p. 11

Abstract

from arXiv · show

Computed Tomography (CT) is an essential 3D imaging technology widely used in medical diagnostics and scientific research. However, performing CT imaging on edge devices is challenging due to limitations in computational power, memory capacity, and energy budget. This paper presents an efficient CT reconstruction framework, called edgeFBP, designed for Nvidia Jetson System-on-Chip (SoC) devices. edgeFBP adopts an end-to-end pipeline design for efficient out-of-core image reconstruction under tight power and memory constraints. edgeFBP utilizes a mixed-precision strategy leveraging half-precision Tensor Cores (TCs) to accelerate the bottleneck back-projection (BP) kernel. edgeFBP achieves a 1.83x speedup over the widely used RTK library on Jetson Nano and a 2.56x speedup on Jetson AGX. Under a strict 25-Watt power budget, edgeFBP on Jetson Nano achieves up to 5-48x higher energy efficiency than an Nvidia DGX A100, enabling datacenter-scale imaging on constrained edge devices.

I. INTRODUCTION

CT reconstruction on edge devices must fit limited compute, memory, and energy budgets while supporting high-resolution imaging. edgeFBP combines an out-of-core Jetson pipeline with Tensor Core-accelerated back-projection and reports higher throughput and energy efficiency than the stated baselines.

I. INTRODUCTION

  • Edge reconstruction is constrained by FBP's compute- and memory-intensive filtering and back-projection kernels, limited Jetson resources, and high-resolution volumes that exceed on-device memory.The introduction motivates SoC platforms as a compact, lower-power alternative to high-end discrete GPUs for portable CT systems.Source: §I · pp. 1–2
  • edgeFBP uses CUDA and cuFFT for filtering, then reformulates back-projection coordinate projection as FP16 Tensor Core matrix operations with mixed-precision recovery of single-precision accuracy.Back-projection is identified as a frequent performance bottleneck in FBP.Source: §I · p. 2
  • Projection and volume blocks fit limited Jetson memory, while pipeline scheduling overlaps I/O with FBP computation for high-resolution out-of-core reconstruction.The framework leverages the heterogeneous CUDA platform.Source: §I · p. 2
  • 1.83× average speedup over RTK on Jetson Orin Nano, 2.56× throughput improvement on Jetson AGX Orin, and up to 5∼48× higher energy efficiency than an 8×A100 DGX system.The energy comparison is under a strict 25 W power budget.Source: §I · p. 2
  • The paper presents the first CT image reconstruction on the Jetson series and proposes an end-to-end framework for efficient out-of-core reconstruction on Jetson platforms.The framework is designed to exploit Jetson's heterogeneous architecture under edge-device memory constraints.Source: §I · p. 2

II. BACKGROUND & RELATED WORK

The section describes CBCT geometry and how FBP turns detector projections into a reconstructed 3D volume. Filtering prepares each projection, and back-projection maps and accumulates sampled projection values into voxels.

A. Cone-beam Computed Tomography (CBCT)

  • CBCT geometry defines detector dimensions, volume dimensions, and a projection matrix that maps each voxel’s spatial position onto the X-ray sensor.The matrix is formed from the imaging-system geometry.Source: A. Cone-beam Computed Tomography (CBCT) · p. 2

B. FBP Image Reconstruction for CBCT

  • FBP filters each projection row with a one-dimensional Ramp filter, implemented in the frequency domain using FFT to reduce filtering complexity.The filtering equation uses convolution with the ramp filter.Source: B. FBP Image Reconstruction for CBCT · pp. 2–3
  • Back-projection projects each voxel through the angle-specific matrix, samples the filtered projection by bilinear interpolation, weights it by 1/z^2, and accumulates it into the output volume.The stated complexity is O(N^4) across projections and voxels.Source: B. FBP Image Reconstruction for CBCT · p. 3

C. Related work

Prior CT reconstruction work spans CPU, GPU, cluster, and supercomputer implementations, while RTK provides standard CPU and GPU FBP. The section also outlines the RTK-style back-projection inputs and output.

C. Related work

  • Prior CT reconstruction work includes distributed and multi-GPU iterative methods, supercomputer-scale solvers, and GPU-based iterative frameworks such as TIGRE.RTK provides standard FBP implementations for CPU and GPU platforms.Source: C. Related work · p. 3
  • Existing approaches are often limited to small-volume reconstructions by hardware memory capacity.The passage also mentions cache-aware out-of-core systems, but its description of them is incomplete.Source: C. Related work · p. 3
  • The back-projection algorithm takes filtered projections Q and projection matrices M as input and produces the reconstructed volume I.The stated dimensions are Q[Pn][Pr][Pc], M[Pn], and I[Vz][Vy][Vx].Source: C. Related work · p. 3

9 Function interp2(J, x, y)

edgeFBP is presented as a full-stack Jetson CT reconstruction design addressing memory and compute constraints. It combines out-of-core partitioning, pipelined execution, GPU memory-access optimization, and Tensor Core acceleration.

9 Function interp2(J, x, y)

  • edgeFBP is presented as the first CT reconstruction implementation on Jetson, targeting efficient, scalable reconstruction for portable CT devices.The paper identifies few existing frameworks for high-resolution reconstruction on edge devices.Source: §9 · p. 3

III. PROPOSED IMAGING FRAMEWORK: EDG EFBP

  • The edgeFBP pipeline partitions projections, reconstructs batches with an optimized FBP engine, and aggregates sub-volumes into the final 3D volume.Its design codesigns data management, scheduling, and GPU kernels, including Tensor Core acceleration for back-projection.Source: §III · p. 4, Figure 2 · p. 4

A. Out-of-core Projection & Volume Partition Design

  • Dynamic two-level batching partitions outer batches along Vz and inner batches along Pn to fit limited device memory.Outer batching reduces memory use for projections and volume; inner batching further reduces projection-memory use, not volume-memory use.Source: A. Out-of-core Projection & Volume… · p. 4, Figure 3 · p. 4
  • Outer batching increases projection overlap between adjacent batches, adding I/O overhead that the framework targets with performance-model-guided pipeline optimization.The i-th outer batch requires additional detector rows for reconstruction.Source: A. Out-of-core Projection & Volume… · p. 4

B. End-to-end Pipeline Optimization

edgeFBP overlaps I/O and computation in its out-of-core pipeline, while an arithmetic-intensity model relates workload characteristics to I/O or compute bounds. The supplied pipeline result reports a runtime reduction on Jetson Nano.

B. End-to-end Pipeline Optimization

  • Double buffering overlaps input-projection loading with computation across batches, while output volumes use a single buffer and are stored after computation.The pipeline also overlaps storing one outer batch with loading the next; within an outer batch, computation overlaps subsequent inner-batch loading.Source: B. End-to-end Pipeline Optimization · p. 4
  • 7.6% lower runtime, from 15.9 s to 14.7 s, was reported for pipeline overlap on Tomo 30 with a 512^3 output volume on Jetson Nano.The pipeline timeline includes loading projections, filtering and back-projection, and storing volumes.Source: Figure 4 · p. 5

C. Model-guided I/O Optimization

  • Batch size and scheduling order matter because each batch’s loading, computation, and storing times vary and can overlap.The pipeline uses this overlap to guide end-to-end I/O optimization.Source: C. Model-guided I/O Optimization · p. 5

1) Arithmetic Intensity Analysis:

  • The arithmetic-intensity model compares floating-point work with data movement and uses ζ, the volume-to-projection size ratio, to indicate I/O- versus compute-bound work.Because FFT contributes only a minor runtime portion, the analysis retains back-projection FLOPs; batch size is a tuning parameter rather than the primary determinant.Source: 1) Arithmetic Intensity Analysis · p. 5

2) Batch Size Optimization:

Batch sizes are selected according to whether the workload is compute- or I/O-bound and the device’s memory limits. Center-outward S-shaped traversal reuses data between batches while supporting pipeline overlap.

2) Batch Size Optimization:

  • The inner batch size is selected after the outer batch and is constrained by device memory; it can be larger near the projection center, where overlap is reduced.The projection spacing becomes smaller near the center, allowing larger inner batches under the memory constraint.Source: 2) Batch Size Optimization · p. 5

3) Loop Reordering for Enhanced Data Locality:

  • Two S-shaped paths traverse outward from the center, reusing data between adjacent outer batches and exploiting pipeline overlap.Starting each inner batch at projection angle zero would prevent that reuse; outward traversal also avoids memory pressure when transitioning from larger to smaller batches.Source: Figure 5 · p. 5, 3) Loop Reordering for Enhanced Data… · pp. 5–6

2) Batch Size Optimization:

  • I/O-bound CBCT workloads favor larger outer batches to maximize arithmetic intensity, whereas compute-bound micro-CT workloads can use smaller batches to improve data locality.For micro-CT, pipeline overlap means the extra I/O loading from a smaller outer batch does not degrade overall performance.Source: 2) Batch Size Optimization · p. 5

IV. GPU-OPTIMIZED FBP KERNELS

The GPU-optimized kernels target back-projection memory access and coordinate computation, while cuFFT accelerates filtering. Memory reuse, thread mapping, and packed projections address data movement and locality.

IV. GPU-OPTIMIZED FBP KERNELS

  • The runtime kernels target two dominant back-projection bottlenecks: memory access at projected coordinates and computation of projection coordinates.The design reduces and rearranges memory accesses, then accelerates coordinate computation using Tensor Cores.Source: §IV · p. 6

A. cuFFT-Accelerated Filtering Computation

  • cuFFT performs projection filtering in the frequency domain, where convolution becomes element-wise multiplication.Jetson’s unified CPU-GPU memory makes memory bandwidth critical, so efficient FFT-based filtering helps minimize memory traffic and sustain throughput.Source: A. cuFFT-Accelerated Filtering… · p. 6

B. Optimizing GPU Memory Access Pattern for BP

  • The memory-access strategy minimizes expensive global transactions by improving locality and reuse through on-chip caches and shared memory.The access categories are global memory, L1 cache, and shared memory.Source: B. Optimizing GPU Memory Access… · p. 6

2) Optimal CUDA Thread/Block Configuration:

  • With a fixed voxel block size of 256, the (tx, ty, 1) block mapping reuses XY-plane coordinates while iterating along Z.The stated configurations are (16, 16, 1) and (1, 1, 256); the paper identifies regular locality along Z and irregular locality along XY.Source: Figure 7 · p. 6, 2) Optimal CUDA Thread/Block… · p. 6

1) GPU Memory Optimization Principles:

  • Because L1 cache and shared memory are private to each SM, inter-kernel reuse is unavailable before Hopper, making intra-block data reuse important.A global-memory access by any thread stalls the entire warp until its threads complete.Source: 1) GPU Memory Optimization Principles · p. 6

3) Lightweight Runtime Packing for Data Locality:

  • Filtered projections are packed during the filtering kernel’s write-back, avoiding a separate preprocessing kernel and reducing redundant global-memory traffic.The packed Cn→Cc→bc→bn→Pr layout groups columns for cache behavior, fixes bn at 8 for Tensor Core computation, and supports prefetching along Pr.Source: 3) Lightweight Runtime Packing for… · pp. 6–7, Figure 8 · p. 7

4) Shared Memory Cache Optimization:

The back-projection runtime kernels combine a shared-memory cache with Tensor Core acceleration. The algorithm accepts projection data and geometry parameters and produces the reconstructed volume.

4) Shared Memory Cache Optimization:

  • A shared-memory cache uses bounded coordinate indices to replace expensive global-memory accesses with two low-latency shared-memory operations.The cache updates a tag when data are fetched, then lets the thread check the tag and retrieve the value from shared memory.Source: 4) Shared Memory Cache Optimization · p. 7

C. Optimizing BP Computation using TCs

  • Tensor Core MMA instructions execute fused matrix multiply-accumulate operations, offering a theoretical 8× speedup over CUDA cores.The method targets projection-coordinate computation, identified as a major bottleneck in the GPU-optimized runtime kernels.Source: C. Optimizing BP Computation using TCs · p. 7
  • The BP algorithm maps its input projection data and geometry matrix to an output image volume.The algorithm integrates optimized memory-access patterns and Tensor Core acceleration.Source: C. Optimizing BP Computation using TCs · p. 7

1) MMA Primitive for BP Computation:

The BP implementation reduces coordinate-computation work through reuse and MMA operations, then adapts the matrices for FP16 Tensor Cores. High/low decomposition and scaling address precision and overflow concerns.

1) MMA Primitive for BP Computation:

  • Each thread reuses an XY-plane coordinate computation across Z iterations, reducing arithmetic cost before MMA instructions accelerate the coordinate computation.The implementation invokes MMA PTX instructions to retain control of the original CUDA logic while transitioning to Tensor Core acceleration.Source: 1) MMA Primitive for BP Computation · p. 7

2) Optimizing BP Computation with FP16 TCs:

  • Applying Tensor Cores to coordinate computation requires handling fragment-layout constraints, FP16 precision, and FP16’s narrower dynamic range.The paper notes potential overflow errors from FP16’s narrower range than FP32.Source: 2) Optimizing BP Computation with FP16… · p. 7
  • The m16n8k8 layout aligns with Ampere warp-level fragment sizes and supports higher instruction-level parallelism with minimal pipeline stalls.The paper contrasts it with m8n8k4, which cannot reach Ampere’s theoretical peak performance because of suboptimal scheduling.Source: 2) Optimizing BP Computation with FP16… · p. 8
  • The projection matrices are decomposed into high- and low-order components to maintain accuracy, while scaling before and after matrix operations keeps intermediate values within FP16 range.The low-order component is the residual between the original FP32 value and its FP16 high-order component; projection matrices are scaled by 1/LSD, and resulting coordinates by LSD/z.Source: 2) Optimizing BP Computation with FP16… · p. 8, Figure 9 · p. 8

V. EVALUATION

The evaluation tests edgeFBP’s reconstruction quality, runtime, and deployment scope on Jetson platforms using industrial and phantom datasets. It reports faster end-to-end reconstruction than RTK and improved BP-kernel performance.

V. EVALUATION

  • Reconstruction slices for the Shepp-Logan phantom and Tomo 30 at 2048^3 are reported to confirm that edgeFBP preserves reconstruction quality.Source: §V · p. 8

A. Datasets and Evaluation Environment

B. Throughput Performance Evaluations

A. Datasets and Evaluation Environment

  • RMSE remains below 1×10−4 for all voxels against RTK, which the authors state meets XCT imaging quality requirements.The paper relates this requirement to the typical 16-bit precision of XCT projections and reconstructed-volume dynamic range.Source: A. Datasets and Evaluation Environment · p. 9

B. Throughput Performance Evaluations

  • At 512^3 and 1024^3 resolutions on Tomo 30, the optimized BP kernel achieves 1.49×–2.15× speedup over the baseline.The roofline analysis uses the Nvidia Nsight Compute profiler.Source: Figure 13 · p. 9

C. Roofline Analysis of the BP Kernel

The evaluation reports compute-bound BP behavior, successful out-of-core reconstruction beyond device memory, and performance gains from partitioning, Tensor Cores, and pipeline optimizations.

C. Roofline Analysis of the BP Kernel

  • Across datasets, edgeFBP’s BP kernel achieves 1.49×–2.15× speedups over RTK, averaging 1.83×, with both kernels largely compute-bound.The roofline analysis reports arithmetic intensities above the bandwidth ceilings.Source: C. Roofline Analysis of the BP Kernel · p. 9

D. Out-of-core Reconstruction Capability

  • For 2048^3 and 3072^3 volumes, edgeFBP completes reconstruction while RTK fails with out-of-memory errors.The volume footprints are 32 GB and 108 GB, respectively, exceeding Jetson Nano and AGX memory capacities.Source: D. Out-of-core Reconstruction Capability · pp. 9–10

E. Evaluation of the Proposed Optimizations

  • The proposed optimization sequence combines partitioning, Tensor Core acceleration, model-guided I/O, pipeline optimization, and memory-access-pattern optimization.The step-wise evaluation compares these implementations against RTK.Source: E. Evaluation of the Proposed… · p. 10, Figure 14 · p. 10

1) Evaluation of the Projection & Volume Partition:

  • Partitioning alleviates out-of-memory issues for Tomo 29 at 1024^3 and yields 1.20× and 2.02× speedups for the other datasets on the two platforms.The authors attribute the capability to projection and volume partitioning, which also facilitates overlap of I/O and computation.Source: 1) Evaluation of the Projection &… · p. 10

4) Evaluation of Pipeline Optimization:

  • Pipeline overlap reduces runtime by 10.36%, from 5.9 s to 5.3 s, on Tomo 30 at 512^3 output on Jetson AGX.Source: Figure 15 · p. 10

2) Evaluation of TCs Optimization:

  • TC acceleration achieves around 1.40× and 2.41× over RTK on Nano and AGX, excluding Tomo 29 at 1024^3; its BP kernel is about 1.53× and 1.54× faster than edgeFBP’s CUDA-based BP kernel.Compared with Partition, TC acceleration achieves 1.17× and 1.22× on the two platforms.Source: 2) Evaluation of TCs Optimization · p. 10

3) Evaluation of Model-guided I/O Optimization:

The evaluation finds gains from batched I/O and pipeline and memory-access optimizations, alongside higher Jetson energy efficiency than a DGX A100 system. The conclusion frames edgeFBP as a route to high-quality reconstruction on compact, low-power platforms.

3) Evaluation of Model-guided I/O Optimization:

  • Batched I/O yields 1.64× speedup on Jetson Nano and 2.45× on AGX over RTK, reducing data movement and improving locality.Compared with Partition, it achieves 1.37× and 1.25× acceleration on the two platforms.Source: 3) Evaluation of Model-guided I/O… · p. 10

4) Evaluation of Pipeline Optimization:

  • Pipeline optimization achieves 1.77× speedup on Nano and 2.57× on AGX over the baseline, with I/O-computation overlap eliminating pipeline bubbles.Relative to the partition-based approach, it delivers additional speedups of 1.47× and 1.32×.Source: 4) Evaluation of Pipeline Optimization · p. 11

5) Evaluation of Memory-Access Pattern Optimization:

F. Power Efficiency Comparison: Jetson vs. DGX

G. Impact of edgeFBP on Real-World Applications

  • The authors describe Jetson-based edgeFBP as suitable for local CT reconstruction on compact, low-power portable, embedded, and robotic imaging platforms.They note that such platforms may not accommodate high-end GPUs and state a 10–40 W power budget for local reconstruction.Source: G. Impact of edgeFBP on Real-World… · p. 11

VI. CONCLUSION

  • EdgeFBP outperforms RTK by up to 1.83× on Jetson platforms and achieves up to 48× higher energy efficiency than DGX-class systems.The authors conclude that this demonstrates feasibility of high-quality tomographic imaging on edge platforms for portable medical, industrial, and scientific applications.Source: §VI · p. 11
Loading 2609.07249v1…