PaperlayerView exact revision on arXiv

Source-linked AI summary

DEX: Digit-Level Early Exit for Energy-Efficient MSDF Neural Network Inference

Yousef Sadegheih, Dorit Merhof, Muhammad Usman

arXiv:2610.11748v1cs.ARcs.AIcs.LG

TL;DR

U-Net brain-tumor inference involves billions of operations, while fixed-cost computation cannot stop when later stages discard part of an output. This paper applies per-output early termination to MSDF quantized inference, reducing digit cycles while retaining the reported segmentation accuracy.

  • Problem

    U-Net inference requires billions of multiply-accumulate operations, and fixed quantization or pruning cannot react to an individual output while it is being computed.

    Source: §1 · p. 1

  • Method

    A grouped two-stage MSDF processing element supports signed INT8 operands and in-stream bias, with four per-output policies that terminate digit streams when remaining digits are unnecessary.

    Source: §1 · p. 2

  • Results

    38.38% fewer digit cycles are reported at 80.58% mean Dice on the 73-case evaluation; exact policies alone reduce cycles by 18.79% without changing quantized output.

    Source: §1 · p. 2, §12 · p. 19

  • Takeaways & Limitations

    The policies provide output-dependent digit-cycle reduction without modifying stored weights, while calibrated policies keep accuracy loss within an offline-enforced constraint.

    Source: §1 · p. 2

    The projected system results are estimates rather than synthesized hardware, and PE estimates exclude memory, I/O, and several other system components.

    Source: §7 · p. 11

Abstract

from arXiv · show

U-Net inference for brain-tumor segmentation requires billions of multiply-accumulate operations, motivating hardware that can reduce computation dynamically rather than relying only on fixed precision or static model compression. Most-significant-digit-first (MSDF) arithmetic exposes the leading digits of a result during computation, enabling output-dependent decisions before the full value is generated. This paper presents an MSDF accelerator for quantized U-Net segmentation with a two-stage grouped processing element supporting signed INT8 operands and in-stream bias accumulation. Four runtime mechanisms operate directly on the output digit stream: exact early negative detection (END) in ReLU layers, exact sign-only decision making in the segmentation head, calibrated low-order-digit skipping, and calibrated pruning. The two approximate mechanisms are selected offline under an accuracy constraint, while execution requires only lightweight control and does not modify the stored weights. On a residual U-Net trained with nnU-Net for BraTS, the proposed mechanisms reduce digit cycles by 38.38\% while achieving a mean Dice score of 80.58\% on 73 held-out cases, compared with 81.20\% for the floating-point model; the exact mechanisms alone reduce cycles by 18.79\% without altering the quantized output. Synthesized in 45~nm, the processing element operates at 500~MHz, occupies 0.858~mm$^2$, and consumes 0.726~mJ per $192\times192$ patch under switching-activity-annotated power analysis. A projected eight-output accelerator with shared activation delivery achieves 16.6~ms latency and 1.67~mJ per patch.

1 Introduction

U-Net brain-tumor segmentation is computationally intensive, and conventional fixed-cost computation cannot stop when later operations discard part of an output. This paper uses MSDF arithmetic and per-output termination policies to reduce digit cycles during quantized inference.

Motivation

  • Quantization and pruning are fixed before inference, while U-Net outputs may discard negative ReLU values, low-order accumulator bits, or logit magnitudes beyond their sign.These output-dependent decisions require the hardware to observe results as they form.Source: §1 · p. 1

Approach

  • The signed-INT8 PE folds bias into the digit stream and uses two-stage grouped reduction; four policies terminate outputs when later digits are unnecessary.The policies are exact early negative detection for ReLU, exact sign-only head decisions, calibrated pruning, and calibrated low-order-digit skipping.Source: §1 · p. 2

Evaluation

  • The evaluation uses an 8-bit quantized residual U-Net trained with nnU-Net for the Medical Segmentation Decathlon brain-tumor task and scored on 73 cases.Dice is reported for whole tumor, tumor core, and enhancing tumor.Source: §1 · p. 2
  • The policies reduce digit cycles by 38.38% and PE energy by 38.43%, with 80.58% mean Dice on 73 evaluation cases.The exact policies alone reduce cycles by 18.79% without changing the quantized output.Source: §1 · p. 2

2 Background

MSDF arithmetic exposes leading result digits early, enabling output-dependent decisions before a full result is computed. Prior neural-network and segmentation hardware offers relevant approaches, but the cited designs retain limitations in reduction structure or completion of individual outputs.

2.1 Most-Significant-Digit-First (MSDF) Arithmetic

  • Redundant signed-digit MSDF emits result digits after a fixed online delay; its first nonzero digit determines sign, and enough leading digits can fix the quantized output.The merged multiply-add architecture combines multiplication and reduction in one recurrence with fixed online delay.Source: §2.1 · p. 2

2.2 Bit-Serial and MSDF Accelerators

  • Bit-serial accelerators adapt precision or activity, while cited MSDF neural-network accelerators use leading digits for adaptation or early termination but retain cascaded multiplier-and-adder trees.Prior work also studied early termination in isolation; adding bias after the stream prevents exact sign decisions on the complete affine result.Source: §2.2 · pp. 2–3

2.3 U-Net Hardware

  • Prior U-Net hardware optimizes dataflow, memory reuse, or model compression, but the cited designs compute every individual output to completion with conventional arithmetic.The paper compares its accelerator quantitatively with existing segmentation accelerators.Source: §2.3 · p. 3

3 Segmentation Model, Training, and Quantization

The study evaluates a lightweight residual U-Net for brain-tumor segmentation on held-out Medical Segmentation Decathlon cases. It trains the model with nnU-Net, quantizes it to INT8, and formulates integer accumulations so their sign includes the folded bias.

3.1 Task and Data

  • The Medical Segmentation Decathlon evaluation uses 73 held-out cases from 484 labeled multi-parametric MRI volumes, with Dice reported for WT, TC, and ET.The remaining cases serve for training and validation.Source: §3.1 · p. 3

3.2 Network Architecture

  • The 2-D residual U-Net has 18 convolutional layers and 2.96×10^6 parameters; one input requires 2,162,294,784 MACs.Its encoder reduces a 4 × 192 × 192 input to a 256 × 12 × 12 bottleneck, and decoder stages concatenate matching encoder features.Source: §3.2 · p. 3

3.3 Training

  • After 1,000-epoch nnU-Net training, the FP32 reference reaches 81.20% mean Dice on the 73 evaluation cases.Training uses 192 × 192 patches and a combined Dice and binary crossentropy loss over three outputs.Source: §3.3 · pp. 3–4

3.4 Post-Training Quantization

  • The 8-bit post-training-quantized model reaches 81.00% mean Dice, 0.20 points below the FP32 reference.Weights and activations use symmetric qint8 quantization; activation scales use histogram calibration on training slices.Source: §3.4 · p. 4

3.5 Integer Formulation

  • Symmetric INT8 activations and weights require no zero-point correction, so the integer pre-activation includes bias and its sign matches the ReLU input or head logit.The exact early-termination policies rely on this sign correspondence.Source: §3.5 · p. 5

4 Two-Stage MSDF Processing Element

The PE combines signed INT8 MSDF arithmetic with a two-stage grouped reduction, in-stream bias handling, and runtime configuration. Its bounded recurrence supports digit streaming, while the output function delivers digits for early termination.

4.1 Grouped Reduction Architecture

  • The two-stage grouped recurrence bounds logic depth for long dot products while preserving the digit-level pipeline.Stage 1 reduces lanes within each group and maintains a local residual; Stage 2 combines group outputs through a separate residual path before the output generation function.Source: §4.1 · p. 5

4.2 Signed Partial-Product Selection

  • Only the activation sign plane selects −wi; other active planes select +wi, enabling signed INT8 products without an extra conversion cycle.The negation is representable because the symmetric INT8 weights exclude −128.Source: §4.2 · p. 6

4.3 Stage-1 and Stage-2 Recurrent Reduction

  • Stage 1 adds each group residual to its reduction and sends upper bits to Stage 2 while retaining lower bits for the next digit cycle.The six logical tree levels are divided into three registered partitions; Stage 2 uses the same tree-plus-residual scheme before the output generation function.Source: §4.3 · p. 6

4.4 Output Generation Function

  • The output generation function converts Stage-2 offset words into one redundant signed digit per cycle, and its stream feeds the runtime controller.The recurrent reduction remains in offset-word form; only the output generation function emits the MSDF digit stream.Source: §4.4 · p. 6

4.5 Digit-Level Timing

  • 24 cycles serve a single 64-lane group and 30 cycles serve all 64 groups; early termination omits unneeded trailing digit cycles.The registered two-stage recurrence is seven levels deep rather than a flat 13-level tree over 4,097 inputs, enabling the reported 500 MHz clock.Source: §4.5 · p. 7

4.6 Runtime Configuration and Clock Gating

  • A layer descriptor configures active groups and lanes, arithmetic mode, early-termination settings, data sources, bias, and output coordinates.Groups not required by a layer are clock-gated, and unused lanes within active groups are masked, allowing narrow and wide layers to share the same PE.Source: §4.6 · pp. 7–8

4.7 Exact Bias Folding

  • The folded bias is decomposed into exact pseudo-products and appended to the operand stream so it passes through the same datapath as real products.If the bias-capacity condition is violated, END and the sign-only head decision are disabled and bias is added after accumulation; the evaluated network did not require this fallback.Source: §4.7 · p. 8

5 Mapping Quantized U-Net Inference onto the MSDF Processing Element

Convolution transactions flatten kernel taps, input channels, and bias pseudo-products into packed operands, while the PE configuration follows each layer’s fan-in. A small integer front end handles decoder scaling, concatenation, and upsampling outside the PE.

5.1 Convolution-to-PE Mapping

  • Only 21.1% of groups are active on average across transactions, while packing within enabled groups reaches 94.5%.The two widest layers enable 55 of 64 groups, while transaction counts are dominated by high-resolution decoder layers; the PE is provisioned for the widest layer.Source: Table 1 · p. 8, §5.1 · p. 9

5.2 Skip Connections, Upsampling, and Residual Additions

  • The front end requantizes and concatenates skip features, then bilinearly upsamples them using shift-and-add operations; residual additions run outside the PE.Across four decoder stages, the front end takes 357,136 cycles, or 0.714 ms at 500 MHz.Source: §5.2 · p. 9

6 Digit-Level Early Termination

Four policies terminate MSDF output streams when later digits are unnecessary: two make exact decisions, while pruning and digit skipping are calibrated offline under an accuracy constraint. Runtime control applies the selected thresholds without changing stored network parameters.

6 Digit-Level Early Termination

  • The four policies reduce output transactions before their full digit length, with exact policies preserving INT8 outputs and calibrated policies constrained offline for accuracy.None of the policies modifies stored weights or other network parameters.Source: §6 · p. 9, Figure 4 · p. 10

6.1 Exact Early Negative Detection (END)

  • END commits zero when the first nonzero digit is −1, certifying a negative ReLU pre-activation whose bias is already in the stream.END is enabled only when the bias-capacity condition is satisfied; a first nonzero +1 alone does not end the transaction.Source: §6.1 · p. 10

6.2 Sign-Only Head Decision

  • The head stops at the first nonzero digit and assigns each label from the logit's sign, matching its zero-logit decision threshold exactly.The 1 × 1 head has no ReLU, and this is the only policy active there.Source: §6.2 · p. 10

6.3 Calibrated Pruning

  • Pruning commits zero if no nonzero digit appears by the 2^p decision point, approximating values in the interval −2^p < ao < 2^p as zero.This includes small positive pre-activations that could otherwise yield a nonzero quantized output.Source: §6.3 · p. 10

6.4 Calibrated Skip Calculation

  • Skip calculation commits the generated prefix at the 2^s decision point after a positive digit appears, clearing the remaining s low-order accumulator bits.It targets positive outputs whose discarded low-order digits would not survive INT8 requantization; when p ≤ s, the combination behaves like skip calculation alone.Source: §6.4 · pp. 10–11

6.5 Priority and Control

  • A priority controller chooses among exact head decisions, END, pruning, skipping, and full completion on valid digit cycles.The winning event commits or forces zero, asserts done, and ends the transaction; termination signals reach datapath clock domains through integrated clock-gating cells.Source: §6.5 · p. 11

6.6 Offline Calibration

  • One exponent pair is selected per network by evaluating candidate thresholds on 73 evaluation cases and discarding pairs whose WT, TC, or ET Dice falls over two points below FP32.Among remaining pairs, calibration chooses the largest cycle reduction; runtime only compares the digit counter against descriptor thresholds.Source: §6.6 · p. 11

7 Evaluation Methodology

The evaluation combines software simulation of output-level digit termination with synthesis-based PE power analysis. Verification checks the signed arithmetic, selector, control paths, and decoder interpolation against reference behavior.

7 Evaluation Methodology

  • The software and hardware flows meet at patch level, charging each realized cycle the energy of its layer’s group mode.The software flow records termination decisions, realized digit cycles, and segmentation scores; the hardware flow synthesizes the PE and estimates power by group mode.Source: §7 · p. 11

7.1 Cycle-Level Simulation

  • The cycle-level simulator applies termination policies to each integer pre-activation and passes onward tensors matching PE outputs.It flattens each convolution window, appends bias pseudo-products, and averages realized cycles over 10,079 slices against full execution; savings are attributed in END, pruning, then skip-calculation order.Source: §7.1 · pp. 11–12

7.2 RTL Synthesis and Switching-Activity-Based Power Analysis

  • The PE is synthesized from a 64-group, 64-lane netlist, with SAIF switching activity back-annotated to estimate power at 500 MHz.Energy is calculated per layer using its realized cycle count and group-mode power, then summed for the patch.Source: §7.2 · p. 12

7.3 Verification

  • Verification checks signed products and selector vectors against reference behavior, tests bias and control paths, and compares decoder interpolation with F.interpolate.The decoder tests covered 389,376 scalars and 510 directed vectors without mismatch.Source: §7.3 · p. 12

8 Results

The results select s = 8, p = 9 under the two-point accuracy gate, with cycle savings coming from multiple early-termination policies. The PE implementation and patch-level energy estimates quantify the hardware cost at that operating point.

8.1 Accuracy–Cycle Trade-off of Early Termination

  • 38.38% fewer cycles at s = 8, p = 9, with mean Dice 80.58%, 0.42 points below INT8; the selected configuration meets the two-point gate.Twenty of 49 configurations meet the calibration constraint; the largest per-region loss at the selected point is 1.26 points in ET.Source: §8.1 · p. 12, Table 2 · p. 13, Figure 5 · p. 13

8.2 Qualitative Segmentation Analysis

  • The selected operating point closely matches the FP32 and INT8 delineations in the two shown cases, while beyond-gate configurations degrade by region and severity.At s = 10 edema is clearly under-segmented; at s = 11 predictions fragment with false positives along the skull.Source: §8.2 · pp. 12–13, Figure 6 · p. 14

8.3 Cycle Reduction per Early-Termination Policy

  • 38.38% total cycle savings comprise 16.48 points from END, 15.38 from skip calculation, 4.39 from pruning, and 2.14 from the sign-only head decision.Skip calculation contributes most in shallow high-resolution layers, while END dominates in deep layers; per fused convolution layer reductions range from 35% to 42%.Source: §8.3 · pp. 13–14

8.4 Processing Element Implementation Results

  • The synthesized PE occupies 0.858 mm2 and meets its 2.00 ns timing constraint with a 1.85 ns critical path.Dynamic power rises from 0.70 mW with one group enabled to 2.67 mW with all 64; total power ranges from 7.36 mW to 9.55 mW because static power remains present in every mode.Source: §8.4 · p. 14

8.5 PE Energy Reduction at Selected Operating Points

  • 38.43% less PE energy per 192 × 192 patch: the selected point uses 0.726 mJ versus 1.179 mJ for full-digit INT8 execution.The selected point takes 45.71 M cycles versus 74.19 M; high-resolution decoder layers conv7_0 and conv8_0 account for 22.5% and 21.6% of PE energy, respectively.Source: §8.5 · pp. 14–15

9 Projection of an Eight-Output Accelerator

The section projects an output-tiled accelerator by sharing activation delivery across replicated processing elements and combining the network schedule with PE and memory models. The eight-lane projection improves throughput and energy per patch, with physical implementation still needed to resolve integration questions.

9.1 Output Tiling with Shared Activation Delivery

  • At To=8, shared activation delivery cuts reads from 242.4 MiB to 30.7 MiB per patch (7.91× reuse), while 17 of 18 convolution layers fill all lanes.Output lanes terminate independently, so each tile finishes when its slowest lane completes; a lane-valid mask handles unused lanes in the final tile.Source: §9.1 · p. 15

9.2 System-Level Projection Model

  • The system projection combines synthesized PE measurements and schedule-derived traffic and buffers with 45 nm SRAM models and interconnect, banking, and integration overheads.The model accounts for controller and external-I/O energy as well as SRAM access energy, leakage, and density.Source: §9.2 · pp. 15–16

9.3 Output-Tiled Accelerator Projection

  • At eight lanes, projected throughput is 6.0× higher and energy per patch 49% lower than one lane, for 61% more area (20.9 versus 13.0 mm2).The projection also reduces convolution digit-stream latency from 91.4 to 12.8 ms and activation-delivery and SRAM-access latency from 5.4 to 0.8 ms.Source: §9.3 · p. 16, Figure 7 · p. 17, Table 5 · p. 18
  • At eight lanes, dynamic SRAM energy drops from 1.43 to 0.23 mJ and leakage from 0.66 to 0.12 mJ, while PE-array energy grows from 0.80 to 0.91 mJ.Reduced data movement and run time offset the higher instantaneous power of replicated PEs.Source: Figure 7 · p. 17, §9.3 · p. 18
  • At To=8, projected throughput reaches 69% of the 379 GOPS peak; wider tiles are limited because the narrowest fused layer has 16 output channels.The on-chip bandwidth demand is 2.45 GB/s, below the assumed 3.2 GB/s external-interface bandwidth.Source: §9.3 · p. 18
  • Physical implementation must still establish whether weight delivery, broadcaster timing at 500 MHz, and front-end and SRAM placement work together.These questions do not affect the mapping, early-termination policies, or segmentation accuracy.Source: §9.3 · p. 18

10 Comparison With Prior Hardware

The comparison places this work’s runtime early termination in a complete U-Net segmentation pipeline alongside prior bit- and digit-serial accelerators. The reported energy and efficiency figures have different tasks, technologies, and measurement boundaries, so they are not directly comparable.

10 Comparison With Prior Hardware

  • Prior entries in Table 6 target classification or recognition, whereas this work applies runtime early termination within a complete U-Net segmentation pipeline.The comparison covers bit-serial designs using compile-time precision or runtime prediction and digit-serial designs acting on leading digits.Source: §10 · p. 18
  • Table 7 energy and efficiency values are not directly comparable because rows differ in task, technology, and reporting boundary.This work’s PE row is a pre-layout SAIF-annotated estimate for the convolution datapath; its projection adds modeled memory, control, I/O, and integration overheads.Source: §10 · p. 18
  • For BraTS, Xiong et al. use INT8 and Modiboyina et al. use W4/A8, while this work retains W8/A8 and removes digit cycles at run time.Song et al. report low energy using a binary/INT4 segmentation ASIC.Source: §10 · p. 18

11 Discussion

The discussion distinguishes exact sign-based decisions from calibrated policies that trade digit cycles for Dice. It also identifies PE energy overheads and scope boundaries that qualify the reported evaluation.

11 Discussion

  • The first nonzero digit certifies sign-based decisions exactly, while calibrated pruning zeros pre-activations below 2p and skip calculation discards digits below 2s.An offline sweep over s and p selects the digit-cycle versus Dice trade-off.Source: §11 · pp. 18–19
  • Early termination lowers PE energy only by removing cycles; static power dominates every mode, and powered idle groups contribute overhead.Adaptive provisioning and power gating of idle groups are identified as ways to address this overhead.Source: §11 · p. 19
  • The selected mean Dice is not an independent estimate because s and p were chosen on the same 73 cases used for accuracy reporting.The evaluation also covers one dataset and a 2-D network; PE figures are pre-layout estimates, the eight-output design is modeled, and the sign-only head assumes independent binary decisions per region.Source: §11 · p. 19

12 Conclusion

The conclusion presents a grouped two-stage MSDF processing element and per-output early termination for quantized U-Net segmentation. Exact policies reduce digit cycles without changing quantized outputs, while calibrated policies increase cycle reduction with a reported Dice score; the eight-output accelerator remains projected.

12 Conclusion

  • The exact policies cut digit cycles by 18.79% without changing the quantized output; calibrated policies raise the reduction to 38.38% at a mean Dice of 80.58%.These results are reported on 73 BraTS evaluation cases.Source: §12 · p. 19
  • The grouped two-stage PE accepts signed INT8 operands and carries folded bias inside the digit stream so leading digits determine the complete pre-activation’s sign.The PE supports per-output early termination.Source: §12 · p. 19
  • The PE consumes 0.726 mJ per patch, and the projected eight-output accelerator completes a patch in 16.6 ms.Future work includes physical implementation with compiled memories and evaluation on further segmentation datasets using a calibration split disjoint from evaluation cases.Source: §12 · p. 19

Related papers

Loading 2610.11748v1…