Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs

Hao Luo; Yiting Yang; Wenyi Zhao; Man Jiang; Zhijun Lin; Ghulam Mohiuddin; Ting Jiang; Kunming Luo; Zihao Zhang; Qingsen Yan; Guoqing Wang; Wei Dong; Peng Wang

Source-linked AI summary

Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs

Hao Luo, Yiting Yang, Wenyi Zhao, Man Jiang, Zhijun Lin, Ghulam Mohiuddin, Ting Jiang, Kunming Luo, Zihao Zhang, Qingsen Yan, Guoqing Wang, Wei Dong, Peng Wang

arXiv:2608.26069v1Aug 26, 2026cs.LG

TL;DR

Large-kernel CNNs remain difficult to deploy on edge devices because pointwise convolutions dominate their parameter volume. The paper proposes SVD-inspired Channel Group-Shared low-rank approximation, reducing parameters by over 81% with under 4.2% accuracy drop while enabling mobile deployment.

Problem
Pointwise convolutions account for over 87% of parameters in large-kernel CNNs, creating the primary storage bottleneck for resource-constrained edge deployment.
Method
CGS decomposes pointwise convolutions into shared down/up-projection matrices and channel-group-specific scalable diagonal matrices using an SVD-isomorphic structure.
Results
CGS reduces parameters by >81% with <4.2% accuracy drop across ImageNet-1K, ADE20K, and COCO experiments, while achieving an empirically favorable storage–accuracy balance.
Takeaways & Limitations
CGS enables successful deployment of pre-trained large-kernel CNN backbones on resource-constrained mobile devices by reducing storage overhead and initialization memory load.
Takeaways & Limitations
Existing compression methods neglect pointwise convolutions, leaving storage demands, initialization bandwidth, and pointwise compression as persistent edge-deployment bottlenecks.

Abstract

from arXiv · show

Large-kernel Convolutional Neural Networks (CNNs) deliver remarkable performance in vision tasks by significantly expanding receptive fields, yet their quadratic parameter growth critically impedes storage-efficient edge deployment. While existing efficient architectures adopt parameter-efficient depthwise separable convolution backbones that leverage techniques like low-rank approximation and weight sharing to compress depthwise convolutions, we identify a critical oversight: pointwise convolutions dominate parameter volume (>87% in models like RepLKNet-31B) and constitute the primary deployment bottleneck on resource-constrained edge devices. This results in prohibitive storage costs and severe memory-loading constraints on resource-limited devices (e.g., smartphones with 4-12 GB Random Access Memory (RAM)). To overcome this, we propose Channel Group-Shared (CGS) low-rank approximation, a novel Singular Value Decomposition (SVD)-based parameter-sharing strategy. CGS constructs a structured low-rank paradigm isomorphic to SVD decomposition, comprising shared (high-parameter-cost) down/up-projection matrices across channel groups within a layer and channel-group-specific (low-parameter-cost) scalable diagonal matrices. This group-sharing design achieves significant parameter reduction. Extensive experiments demonstrate that large-kernel CNNs (RepLKNet, ConvNeXt, SLaK) enhanced with CGS strike an empirically favorable balance between competitive performance and substantially reduced storage costs. Crucially, by alleviating storage constraints, reducing memory bandwidth pressure during loading, and minimizing model loading latency, CGS enables the feasible deployment of pre-trained large-kernel CNN models on edge devices, thereby bridging the gap between high-performance vision models and practical edge deployment.

1 Introduction

Large-kernel CNNs achieve strong visual-recognition performance but face edge-deployment barriers because pointwise convolutions dominate parameters and create storage and memory-loading constraints. The proposed SVD-inspired Channel Group-Shared low-rank approximation shares projection matrices across channel groups while using group-specific scaling, reducing parameters and enabling mobile deployment.

Motivation: Large-kernel CNNs capture long-range dependencies through expanded receptive fields and perform strongly in image classification, object detection, and semantic segmentation.These models have emerged as a leading architecture for visual recognition tasks.
Motivation: Over 87% of parameters can reside in pointwise convolutions, leaving this component as the primary compression bottleneck after depthwise-convolution techniques are applied.Efficient large-kernel architectures commonly use depthwise separable convolutions, whose pointwise component combines output channels through linear projection.
Edge Deployment Challenge: Pointwise-dominated parameter counts exceed edge-device storage capacities, increase transmission overhead, and cause inefficient memory-bandwidth use and peak-memory risks during model loading.These constraints can create system instability on resource-limited devices.
Proposed Method: CGS low-rank approximation decomposes pointwise convolutions into shared high-cost down/up-projection matrices and channel-group-specific low-cost scalable diagonal matrices.The structured decomposition is grounded in SVD theory and isomorphic to the SVD-factorized form.
Results and Deployment: The method reduces parameters with lower capacity loss, balances storage and accuracy among compared compression methods, and enables successful deployment of pre-trained large-kernel CNN backbones on mobile platforms.The introduction reports empirical validation of this deployment effectiveness.

2 Background and Motivation

Large-kernel CNNs improve visual-task performance by using kernels up to 51 × 51 to expand receptive fields and capture long-range dependencies. However, existing efficiency efforts largely compress depthwise convolutions while neglecting parameter-dominant pointwise convolutions, leaving storage and mobile deployment bottlenecks.

Motivation: Large-kernel CNNs such as RepLKNet, ConvNeXt, and SLaK achieve breakthroughs across classification, detection, and segmentation using kernels from 7 × 7 to 51 × 51.Their large kernels expand receptive fields and capture long-range dependencies.
Motivation: Pointwise convolutions, rather than depthwise convolutions, constitute the dominant source of overall parameters in large-kernel CNNs.This skewed parameter distribution challenges storage-efficient deployment despite depthwise convolutions’ role in capturing long-range spatial dependencies.
Existing Efforts: Efficient architectures primarily use depthwise separable convolution backbones and compress depthwise convolutions through kernel factorization, parameter sharing, and low-rank approximation.Examples include SLaK’s 51×51 →51×5+5×51 factorization and PeLK’s focus-blur mechanism with exponential scaling grids.
Limitation: Existing methods universally neglect pointwise convolutions, creating an efficiency gap because depthwise compression delivers only marginal gains while pointwise operations remain expensive.The neglected pointwise component continues to dominate parameterization.
Limitation: Pointwise convolutions impose prohibitive storage demands, exceed mobile bandwidth capacities during initialization, and remain a large-kernel CNN compression bottleneck.These constraints directly hinder edge deployment.

3 Methodology

The methodology develops CGS from SVD dimensional consistency and orthogonal-matrix equivalence, sharing projection factors across channel groups while retaining group-specific diagonal transformations. It defines automatic channel grouping, compression variants, selective application to later stages, and a cloud-to-mobile deployment pipeline.

CGS foundation: CGS exploits identical SVD factor dimensions and orthogonal-matrix equivalence to represent multiple group factors with shared bases and group-specific invertible transformations.This dimensional matching underpins the group-sharing compression framework.
Channel grouping: Pointwise weights are partitioned into k_auto equal-sized channel groups, with k_auto determined automatically from the output-to-input channel ratio.The automatic count avoids manual tuning while balancing parameter reduction and representational capacity.
Channel grouping: The same dimensional factorization extends CGS to depthwise convolutions by partitioning kernels along channels and sharing factors across the resulting group kernels.This preserves architectural consistency between pointwise and depthwise implementations.
CGS variants: CGS reduces parameters by restricting transformation matrices to diagonal forms and offers Large, Base, and Small configurations with progressively different parameter-sharing structures.CGS-L retains the derived formulation, whereas CGS-B and CGS-S merge additional outer or complete matrix components.
Stage selection: CGS is applied selectively to Stages 3-4 because their pre-trained pointwise and depthwise kernels show pronounced low-rank, power-law singular-value structure, while Stages 1-2 are compression-sensitive.The observed structure motivates low-rank down/up-projection matrices reconstructed from limited basis vectors.
Deployment pipeline: The deployment pipeline pre-trains compressed models on cloud GPUs, maps weights explicitly to mobile CPUs, reconstructs the architecture programmatically, and integrates it into Android applications for validated on-device inference.Real-time measurements and accurate image classification on mobile devices empirically validate the deployment process.

4 Experiments

Experiments assess CGS across classification, detection, segmentation, ablations, multiple large-kernel backbones, and mobile deployment. Results show substantial parameter and latency reductions while preserving competitive accuracy and enabling efficient edge-to-cloud inference.

Experimental Setup: Evaluation spans ImageNet-1K, CIFAR-100, ADE20K, and COCO under established protocols, supplemented by ablations and mobile-platform experiments.The benchmark evaluates large-scale and fine-grained classification, scene parsing, and object-level recognition.
Classification: 81% parameter reduction versus RepLKNet-31B limits the accuracy drop to 4.2%, while CGS-B improves Top-1 accuracy by 6.5% with only a 1.5M parameters increase.CGS variants integrated into RepLKNet also outperform under comparable parameter budgets, and broader comparisons confirm competitive accuracy with fewer parameters.
Dense Prediction: 47% parameter reduction for RepLKNet-31B accompanies competitive detection accuracy, while ADE20K segmentation reduces parameters by 57% and maintains competitive single-scale mIoU.Detection comparisons include SLaK and ConvNeXt; segmentation uses UperNet with a RepLKNet backbone fine-tuned for 160K iterations.
Ablation and Generalization: Compressing only Stages 3-4 preserves accuracy, whereas compressing all stages causes a relative accuracy reduction of 1.6% for merely 1.3M parameters saved; CGS-B compresses parameters by >77% with accuracy drops below 6.5% across three backbones.The selective strategy reflects parameter concentration in later stages and sensitivity of early low-level feature extraction; CGS-B generalizes across RepLKNet, ConvNeXt, and SLaK.
On-Device Deployment: CGS-B cuts latency by 25% and energy by 43% over the INT8 baseline; CGS-B+INT8 reaches 135 ms latency and 7.8 mWh energy with a 0.5% relative accuracy drop.Despite slightly higher FLOPs, CGS-B achieves lower inference latency on three Android devices because mobile NPUs are primarily memory-bound.
Wireless Inference: Wireless communication contributes 54%–62% of total latency, while CGS reduces RepLKNet-B end-to-end latency by 27% on 5G and 25% on Wi-Fi 6.CGS compresses RepLKNet-B from 306.94 MB to 58.41 MB and outperforms ConvNeXt-T with 46.5% smaller model size and 13-14% lower latency.

5 Related Works

Large-kernel CNNs improve vision performance through expanded receptive fields but face quadratic parameter growth and deployment challenges. Related compression work emphasizes depthwise convolutions and parameter sharing, while CGS targets parameter-dominant pointwise convolutions for mobile deployment.

Large-Kernel CNNs: Kernels of 7 × 7 or larger, including 31 × 31 designs in ConvNeXt and RepLKNet, model long-range dependencies but incur quadratic parameter growth.This growth creates significant deployment challenges on resource-constrained devices.
Parameter Compression in Large-Kernel CNNs: Efficient large-kernel CNNs predominantly use depthwise separable convolution backbones, with substantial compression research focused on the depthwise component.SLaK combines kernel decomposition, such as 51 × 5 + 5 × 51, with dynamic sparsity.
Parameter Sharing Strategies: Parameter sharing compresses large-kernel CNNs by concentrating unique parameters in kernel centers and sharing parameters elsewhere, as exemplified by PeLK’s peripheral convolution framework.PeLK applies this strategy to depthwise convolution.
CGS and Mobile Deployment: >81% parameter reduction for RepLKNet-31B is achieved by CGS, which applies SVD-inspired group sharing directly to parameter-dominant pointwise convolutions.The method is designed to alleviate storage limits and memory bandwidth pressure during model loading.

6 Discussion

CGS combines globally shared projections with lightweight group-specific scaling to improve parameter efficiency, optimization stability, and hardware-friendly deployment. Future work includes extending CGS beyond pointwise convolutions, adapting it to parameter-intensive early layers, and evaluating more aggressive quantization.

CGS Design: CGS shares down/up-projection matrices across channel groups and adds a learnable diagonal scaling matrix per group to capture group-specific variations efficiently.The shared bases promote representational consistency and geometric uniformity, while scaling introduces only linear parameter overhead.
Optimization: CGS has stable optimization because its low-rank bottleneck limits basis redundancy, group-wise gradient averaging reduces directional noise, and projections adapt to dominant feature subspaces.These three intrinsic characteristics jointly explain the framework’s stable training behavior.
Deployment: INT8 quantization requires shared projections to be quantized once and reused across groups, while per-group scaling matrices mitigate quantization error and support efficient NPU weight reuse.This compatibility reduces quantization overhead and contributes to hardware-friendly deployment.
Future Work: Future research should extend group-shared low-rank approximation from pointwise convolutions to Transformer feedforward networks and attention projections.The proposed extension targets other matrix-based components beyond the large-kernel CNN setting evaluated in this work.
Future Work: Future work should adapt CGS to architectures with more parameter-intensive early layers and investigate INT4 or mixed-precision quantization beyond the current INT8 setting.These directions aim to broaden architectural applicability and potentially improve deployment efficiency.

7 Conclusions

The paper presents CGS low-rank approximation as an SVD-grounded compression strategy for addressing large-kernel CNN deployment constraints. It reduces storage and initialization memory overhead while improving mobile execution efficiency and enabling deployment on resource-constrained devices.

Conclusions: CGS is an SVD-grounded parameter-compression strategy designed to resolve deployment constraints in modern large-kernel CNNs.The conclusions identify CGS as the paper’s central compression method.
Conclusions: Over 87% of parameters in RepLKNet-31B and ConvNeXt-B reside in pointwise convolutions, creating a fundamental barrier to edge deployment.The conclusion attributes the parametric imbalance to the main deployment obstacle.
Conclusions: CGS significantly reduces storage overhead and peak memory load during model initialization while enhancing execution efficiency on mobile devices.These gains address both storage and runtime-memory limitations.
Conclusions: CGS enables successful deployment of large-kernel backbones on resource-constrained mobile devices.The conclusion presents this deployment outcome as overcoming storage and runtime-memory constraints.

A Training Configurations … A.4 CIFAR-100

The paper specifies training configurations for ImageNet-1K pretraining, downstream detection and segmentation fine-tuning, and CIFAR-100 ablations. These configurations use defined optimizers, schedules, augmentation, and task-specific training durations.

A.1 ImageNet-1K: ImageNet-1K pretraining used 8 NVIDIA A800 GPUs, AdamW, weight decay 0.05, learning rate 8e-3, 10epoch warmup, and cosine decay.Training ran for 120 or 300 epochs, with 120 epochs primarily used for main and ablation studies.
A.1 ImageNet-1K: ImageNet-1K augmentation combined RandAugment ‘rand-m9-mstd0.5-inc1’, label smoothing coefficient = 0.1, Mixup α= 0.8, CutMix α= 1.0, and random erasing p = 0.25.Layer scale was initialized to 1e-6; the provided passage truncates the EMA decay factor.
A Training Configurations: The overall training configurations followed RepLKNet [15] where specified, with task-specific downstream fine-tuning applied to the pre-trained CGS-B model.The MS-COCO and ADE20K configurations used the RepLKNet-pytorch Github repository hyperparameters.
A.2 MS-COCO: MS-COCO fine-tuned pre-trained CGS-B using Cascade Mask R-CNN in MMdetection for 36 epochs.The hyperparameters were consistent with the RepLKNet-pytorch Github repository.
A.3 ADE20K: ADE20K fine-tuned pre-trained CGS-B using UperNet in MMSegmentation for 160K iterations.The hyperparameters were consistent with the RepLKNet-pytorch Github repository.
A.4 CIFAR-100: CIFAR-100 retained the ImageNet-1K training configuration and used 200-epoch experiments for ablation analysis.The 200-epoch experiments were intended to observe critical patterns.

B Detailed Results on Bottleneck Dimensions

On CIFAR-100 with RepLKNet, bottleneck dimension 200 provides the best balance of accuracy and parameter efficiency across CGS variants. Larger dimensions add parameters while reducing accuracy, whereas CGS-L adds only a marginal cost over CGS-B at dimension 200.

Bottleneck-dimension evaluation: Bottleneck dimensions below 200 consistently degrade model efficacy by failing to provide sufficient representational capacity.The evaluation covers CGS-S/B/L down/up-projection matrices with RepLKNet on CIFAR-100.
Bottleneck-dimension evaluation: 1.4% accuracy reduction occurs for CGS-B when dimension rises from 200 to 300.The decline suggests over-parameterization or optimization challenges within the compressed structure.
Bottleneck-dimension evaluation: Dimension 200 is the singular configuration that maximizes accuracy while minimizing unnecessary parameter growth across the evaluated CGS variants.This conclusion is based on the CIFAR-100 comparison in Table 12.
Bottleneck-dimension evaluation: CGS-L requires merely +2.5M parameters versus CGS-B at bottleneck dimension 200.The marginal increase alleviates scalability concerns for the larger-scale configuration.

C Detailed Results on Structural and Quantization Synergy · D Mobile Processor Model

CGS-B provides a favorable balance of model size, latency, energy efficiency, and accuracy for resource-constrained deployment, while combining effectively with INT8 quantization. Experiments on mobile device processors show that hardware-aware structural redesign can support memory- and power-constrained platforms.

C Detailed Results on Structural and Quantization Synergy: Real-world RepLKNet-31B comparisons evaluate various compression methods for resource-constrained deployment, with energy consumption measured over multiple runs.These experiments assess practical deployment behavior rather than compression metrics alone.
C Detailed Results on Structural and Quantization Synergy: CGS-B achieves a superior balance of parameters, latency, and energy efficiency compared with standard INT8 quantization and ADMM-based pruning.Although CGS-B has higher FLOPs, its inference latency is significantly lower because of improved memory access patterns and hardware utilization.
C Detailed Results on Structural and Quantization Synergy: CGS improves deployment efficiency by alleviating memory-bandwidth bottlenecks common on mobile NPUs/GPUs.The reported latency gain is attributed to improved memory access patterns and better hardware utilization.
C Detailed Results on Structural and Quantization Synergy: Over 80%: CGS-B reduces model size versus the original backbone while maintaining competitive accuracy.Pruning can create irregular sparsity that limits practical speedups.
C Detailed Results on Structural and Quantization Synergy: Over 30%: combining CGS-B with INT8 quantization further reduces latency and energy with lower accuracy loss.This demonstrates compatibility between structural compression and precision reduction.
C Detailed Results on Structural and Quantization Synergy: CGS enables joint structural and numerical optimization for memory- and power-constrained mobile platforms.The results emphasize hardware-aware structural redesign beyond conventional compression methods.
D Mobile Processor Model: The deployment and inference experiments used mobile device processors listed in Table 13.The processor models are identified as the hardware platforms for the reported mobile evaluations.