PaperlayerView exact revision on arXiv

Source-linked AI summary

TernScope: Model-Hardware Co-Design for Efficient Ternary Endoscopic Segmentation

Zhuoyu Wu, Wenhui Ou, Qiawei Zheng, Xuhong Qiu, Wenqi Fang, Raphaël C. -W. Phan

arXiv:2610.29617v1eess.IVcs.ARcs.CV

TL;DR

Edge medical segmentation must balance compact models, accurate delineation, and prompt inference, while model size alone does not guarantee efficient hardware execution. TernScope fuses a compact network before progressive ternary quantization and maps its W2A8 operators to a ternary array. It reports lower model cost, improved segmentation relative to its FP32 reference, and reduced compute-array area relative to Gemmini.

  • Problem

    Edge segmentation must balance limited memory, accurate lesion delineation, and prompt predictions, while inference cost also depends on memory access, operator organization, and target architecture.

    Source: §I · p. 1

  • Method

    TernScope combines split-channel large-kernel blocks and branch fusion with TPQ on the fused graph, using ternary weights and 8-bit activations across its convolutions.

    Source: §I · pp. 1–2

  • Results

    0.2483 M parameters and 0.0949 GMACs accompany performance above the FP32 reference across all three datasets, while compute-array area is 88.46% lower than the original Gemmini.

    Source: §I · p. 2

  • Takeaways & Limitations

    Cross-dataset evaluation and matched synthesis connect low-bit learning to a smaller compute array for edge endoscopy.

    Source: §V · p. 4

Abstract

from arXiv · show

Edge medical segmentation must reconcile small model size, high accuracy, and fast inference. TernScope addresses this tradeoff through model--hardware co-design, learning a compact segmentation network whose low-bit operators also simplify hardware execution. Split-channel large-kernel blocks preserve spatial context with limited computation, while branch fusion removes redundant inference paths. Ternary Progressive Quantization-aware Training (TPQ) then adapts the fused network from deep to shallow layers, through the decoder, and finally to the input and prediction layers. All 25 convolutions use ternary weights and 8-bit activations, enabling packed storage and multiplier-free computation. The resulting model uses 0.2483 M parameters and 0.0949 GMACs while retaining competitive segmentation accuracy on in-domain and external datasets. Under the same 45 nm technology, TernScope reduces compute-array area by 88.46% compared with the original Gemmini, a systolic-array accelerator with 8-bit $\times$ 8-bit multipliers. These results demonstrate a path from accurate lightweight medical segmentation to compact low-bit hardware.

I. INTRODUCTION

TernScope treats segmentation quality and deployment cost as joint design objectives, combining a compact network with low-bit operators and hardware-aware execution. Its fully ternary-weight framework aims to retain accuracy while reducing model and compute-array costs.

I. INTRODUCTION

  • Edge segmentation must balance memory-constrained model size, accurate lesion delineation, and prompt predictions.Reducing width or spatial processing can restrict contextual information, so predictive quality and deployment cost are joint objectives.Source: §I · p. 1
  • Low-bit operands reduce tensor storage and simplify arithmetic, while structural re-parameterization merges expressive training branches into efficient inference operators.A compact network alone does not guarantee fast inference or a small accelerator because execution also depends on memory access, operator organization, and architecture.Source: §I · p. 1
  • TPQ adapts the fused deployment graph to ternary weights across every convolution while retaining 8-bit activations.SplitLK allocates large-kernel processing to a channel subset, and the narrow decoder fuses multi-scale features before branch fusion removes redundant inference paths.Source: §I · pp. 1–2, Figure 1 · p. 1
  • 0.2483 M parameters and 0.0949 GMACs accompany performance above the FP32 reference across all three datasets.TernScope reduces compute-array area, synthesized standard-cell area, and estimated logic power by 88.46%, 70.28%, and 62.82%, respectively, compared with the original Gemmini.Source: §I · p. 2
  • 88.46% lower compute-array area, 70.28% lower synthesized standard-cell area, and 62.82% lower estimated logic power than the original Gemmini.The framework uses ternary PEs and shared arithmetic resources.Source: §I · p. 2

A. SplitLK: Spatial Context at Low Cost

SplitLK limits large-kernel spatial processing to selected channels while preserving a bypass and pointwise mixing. The Nearest Unifier projects and combines four encoder scales through a narrow decoder.

A. SplitLK: Spatial Context at Low Cost

  • Processing 32 of 128 channels reduces the large-kernel spatial term to 1/16 of its full-width counterpart, while pointwise mixing remains.Spatial convolutions operate on 16/16/16/32 channels across the four encoder stages; kernels of size 5, 7, and 9 are used at 32 × 32 resolution or below.Source: A. SplitLK: Spatial Context at Low Cost · p. 2

B. Nearest Unifier: Narrow Multi-Scale Fusion

  • The Nearest Unifier projects all four encoder scales to 16 channels and uses nearest-neighbor resizing to align them for addition and refinement.Addition keeps the fused width at 16 channels rather than growing it through concatenation.Source: B. Nearest Unifier: Narrow Multi-Scale… · p. 2
  • Deeper features are combined and refined before fusion with the shallow feature, introducing semantic context before fine-detail recovery.Convolutions learn spatial refinement, and dense kernels and channel widths aligned to 16 suit a common array.Source: B. Nearest Unifier: Narrow Multi-Scale… · p. 2

C. Fuse First, Then Learn Ternary Execution

The deployment topology is fused before ternary quantization, then TPQ progressively adapts the merged graph to W2A8. The supplied setup specifies the training schedule and evaluation scope for the segmentation and transfer comparisons.

C. Fuse First, Then Learn Ternary Execution

  • Batch normalization is folded into each RepLK branch before the branches are merged into the deployment topology.Fusion preserves the linear operation, removes intermediate branch execution, and lets TPQ adapt merged kernels rather than independently quantizing branches that disappear at deployment.Source: C. Fuse First, Then Learn Ternary… · p. 2
  • The per-output-channel ternary quantizer clips rounded, scale-normalized weights to −1, 0, or +1 and multiplies by a positive channel scale.FP master weights and channel scales use fake quantization with straight-through gradients; activations remain A8, while bias, scaling, and accumulation use wider precision.Source: C. Fuse First, Then Learn Ternary… · p. 2
  • TPQ first adapts the fused model in W8A8 for 5%, then transitions weights to ternary from deep to shallow encoder stages and through the decoder, input convolution, and segmentation head.All 25 convolutions are ternary by 40%, with the remaining 60% training the W2A8 graph; validation Dice selects checkpoints only from the fully W2A8 phase.Source: C. Fuse First, Then Learn Ternary… · pp. 2–3
  • Evaluation trains only on Kvasir-SEG and tests external transfer to ClinicDB and ColonDB without target-domain fine-tuning.The table caption identifies segmentation quality and cross-dataset transfer after training on Kvasir-SEG; the supplied setup lists test sizes of 200, 612, and 380.Source: C. Fuse First, Then Learn Ternary… · p. 3, Table I · p. 3

III. MODEL-TO-HARDWARE MAPPING

The W2A8 deployment graph maps convolution to a ternary PE array and uses shared affine and quantization resources for post-processing and feature addition. Ternary weights replace general multiplication with sign selection and accumulation.

III. MODEL-TO-HARDWARE MAPPING

  • The 16×16 ternary PE array performs convolution, while the shared Affine & Quantization Engine handles post-processing and multi-scale feature addition.Weights and activations are loaded into on-chip buffers, alongside an accumulator buffer.Source: §III · p. 3, Figure 4 · p. 4
  • Each ternary weight selects −a, zero, or +a for accumulation, eliminating general multipliers from the PE product path.Wider partial sums preserve the convolution range before per-channel scaling and bias addition.Source: §III · p. 3
  • Packed W2 weights use one quarter of the W8 payload, while the shared engine produces 8-bit outputs using round-to-nearest-even and saturation.Operand selection feeds shared multipliers, followed by bias addition for convolution or summation of scaled features for two-source Add.Source: §III · p. 3

A. Evaluation Protocol

Evaluation uses Kvasir-SEG for training and validation, with ClinicDB and ColonDB reserved for transfer testing without fine-tuning. The protocol fixes image preparation, optimization, checkpoint selection, and metric calculation.

A. Evaluation Protocol

  • Training uses 720 Kvasir-SEG images, with 80 for validation and 200 for testing; ClinicDB and ColonDB evaluate transfer without fine-tuning.The external datasets contain 612 and 380 images, respectively.Source: A. Evaluation Protocol · p. 4
  • RGB images are resized to 256×256 and normalized to [−1, 1], with flips, right-angle rotations, color adjustment, and Gaussian blur for augmentation.Images and masks use bilinear and nearest-neighbor resizing, respectively.Source: A. Evaluation Protocol · p. 4
  • Validation Dice selects checkpoints, and per-image scores are averaged at threshold 0.5; baseline and TPQ training use distinct optimizers and batch sizes.Baselines use Adam with batch 16, while TPQ uses AdamW with batch 2; both train for 200 epochs.Source: A. Evaluation Protocol · p. 4

B. Segmentation Quality and Quantization Ablation

TernScope combines a favorable accuracy–cost tradeoff with low-bit adaptation that improves on direct QAT and the FP32 reference. Ablations distinguish the benefits of fusion from the need to train the fused graph for ternary weights.

B. Segmentation Quality and Quantization Ablation

  • TernScope uses approximately 1/125 of U-Net’s parameters and 1/508 of its MACs, with in-domain Dice within 0.51 points and external Dice higher by 5.00 and 5.85 points.It achieves the strongest cross-dataset Dice and IoU among evaluated models; EGE-UNet and Mobile-PolypNet have fewer parameters but lower overlap scores.Source: B. Segmentation Quality and… · p. 4
  • Fusion removes 23.21% of parameters and 10.78% of MACs while preserving FP32 overlap scores, but ternary PTQ collapses segmentation despite unchanged topology.QAT restores useful predictions.Source: B. Segmentation Quality and… · p. 4
  • TPQ improves external Dice over direct QAT by 2.62 and 3.09 points and exceeds the FP32 reference on all three datasets.These ablation results identify low-bit adaptation as essential to exploiting the fused topology.Source: B. Segmentation Quality and… · p. 4

C. From Low-Bit Features to Implementation Cost

Under matched synthesis conditions, the ternary implementation reduces area and increases throughput relative to Gemmini. The accelerator’s shared post-processing engine also reduces estimated logic energy for the evaluated workload.

C. From Low-Bit Features to Implementation Cost

  • 88.457% lower compute-array area and 70.275% lower synthesized standard-cell area than Gemmini were measured for the same 25-convolution workload and 16 × 16 PE array.The comparison uses Yosys/ABC and the Nangate45 45nm library; the reported reductions stem from ternary arithmetic, reduced accumulator width, and simplified control logic.Source: Table III · p. 4, C. From Low-Bit Features to… · p. 4
  • Both designs use TT, 1.1 V, and 25 °C with a matched Yosys/ABC flow; logic energy is estimated using OpenSTA vectorless power analysis.Source: C. From Low-Bit Features to… · p. 4
  • At 100 MHz, TernScope delivers 102.701 FPS versus Gemmini’s 11.902 FPS, an 8.629× throughput improvement.Ternary weights occupy one quarter of the packed INT8 weight footprint and replace general multiplications with sign selection and accumulation.Source: C. From Low-Bit Features to… · p. 4
  • Logic energy cost falls from 14.019 to 0.604 mJ per image, a 95.690% reduction for this workload.The shared Affine & Quantization Engine performs scaling, bias addition, multi-scale feature addition, and requantization, reducing CPU post-processing overhead in the evaluated Gemmini mapping.Source: C. From Low-Bit Features to… · p. 4

V. CONCLUSION

TernScope combines SplitLK, branch fusion, and TPQ in a compact W2A8 graph, with cross-dataset evaluation and matched synthesis connecting low-bit learning to a smaller compute array. The paper identifies sparsity-aware execution and compressed storage as next steps for exploiting zero-valued ternary weights.

V. CONCLUSION

  • TernScope couples SplitLK, branch fusion, and TPQ to retain segmentation quality in a compact W2A8 graph.Source: §V · p. 4
  • Cross-dataset evaluation and matched synthesis connect low-bit learning to a smaller compute array for edge endoscopy.Source: §V · p. 4
  • Sparsity-aware execution and compressed storage are proposed next steps for exploiting zero-valued ternary weights to target lower computation and data-movement costs.Source: §V · p. 4

Related papers

Loading 2610.29617v1…