PaperlayerView exact revision on arXiv

Source-linked AI summary

Input-Layer Starvation: Why Per-Layer Pruning Breaks IoT Intrusion Detectors

Md Anas Biswas

arXiv:2609.30729v1cs.CRcs.LG

TL;DR

Overall-accuracy checks can miss class-level failures in pruned IoT intrusion detectors. The paper investigates first-layer starvation and evaluates prevention and recalibration. Its reported findings connect this failure to misattribution and false alerts, while showing that allocation choices and recalibration can mitigate it.

  • Problem

    Overall accuracy is commonly used to judge pruning, but it misses severe class-level failures in IoT intrusion detectors.

    Source: §1 · p. 1

  • Method

    The paper tests pruning-allocation interventions and recalibrates normalization statistics using unlabelled training data to investigate and address first-layer starvation.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    At 80% sparsity, uniform pruning leaves 38 of the CNN’s 192 first-layer weights and disconnects almost half its filters, while matched-or-smaller MLP and transformer models do not collapse.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    Protecting the first layer or pruning globally prevents collapse, while recalibrating normalization statistics repairs a collapsed model without changing weights.

    Source: §1 · p. 2

    Whether the particular surviving weights matter more than random ones was not tested.

    Source: §5.6 · p. 14

Abstract

from arXiv · show

Intrusion detectors for small Internet-of-Things (IoT) devices are usually compressed by pruning and judged by overall accuracy. We show that this hides a severe class-level failure, find its cause, and give low-overhead prevention and repair. On CICIoT2023, a two-layer convolutional detector pruned with uniform layer-wise magnitude pruning at 80% sparsity loses 16 points of accuracy but half of its macro-F1, the mean per-class F1 (0.542 to 0.271 over five independently trained models); 17 of 34 classes are materially damaged. Remaining weight count does not explain it: a perceptron and a transformer pruned to the same or fewer weights lose at most 0.096. The first layer does. It has 192 weights; uniform pruning leaves 38, 46% of its 64 filters lose every input weight, and fine-tuning under that starvation leaves the running means of the first normalisation layer displaced by up to 0.8 standard deviations in a few surviving channels, on which the deployed model collapses. Protecting those 192 weights, or pruning globally at the same sparsity, prevents the collapse (loss 0.013); recomputing the normalisation statistics on unlabelled training data, with no weight changed, repairs it (loss 0.039) and returns the false-alert rate to 33% (dense 29%). Damage shows a strong increasing dose-response in first-layer sparsity, starving a perceptron's input layer reproduces the collapse, and the pattern holds on TON_IoT. The failure is misattribution and false alerts, not silent evasion: on validation-selected blind spots, uniformly pruned detectors misattribute 72% of the traffic, against 50% with the first layer protected and 47% for the dense model.

1. Introduction

The paper shows that overall accuracy misses severe class-level failures in pruned IoT intrusion detectors, and identifies starvation of a tiny first layer as a removable cause. It tests prevention and repair strategies and measures the resulting security consequence.

1. Introduction

  • Uniform pruning damages class-level performance: macro-F1 halves while attacks are still flagged as the wrong attack and benign traffic receives false alerts.The detector is a two-layer CNN trained to classify 34 CICIoT2023 traffic types; overall accuracy does not reveal which classes fail or where errors go.Source: §1 · p. 1
  • At 80% sparsity, the CNN’s 192-weight first layer retains 38 weights and disconnects almost half its filters, while matched-or-smaller MLP and transformer models do not collapse.The paper calls this input-layer starvation, a partial form of layer collapse.Source: §1 · pp. 1–2
  • Interventions establish first-layer starvation—not remaining weight count—as the cause, including protection, global pruning, a monotone dose-response, and input-layer starvation in another architecture.The evidence covers both one-shot and gradual pruning.Source: §1 · p. 2
  • Across first layers of 32, 64, and 128 filters, the fraction of connected filters describes damage better than equal surviving-weight counts.Uniform 80% pruning leaves about half the filters connected at each width; live filters mediate the effect in the CNN, but neither live units nor covered features alone do so in the MLP.Source: §1 · p. 2
  • After fine-tuning under starvation, a few first-layer normalisation channels have displaced running means, and their error tracks model degradation.Re-estimating statistics from unlabelled training data recovers most CNN loss, less in the perceptron and on TON_IoT, and does not change protected or globally pruned models.Source: §1 · p. 2
  • On validation-selected blind spots, uniform pruning raises misattribution 25 points above dense, versus 2 to 4 points for fixed recipes, and doubles false alerts without raising attack-to-benign rates.Protecting the first layer or pruning globally recovers 0.26 macro-F1 with size and latency unchanged; recalibration is a one-pass repair on unlabelled training data.Source: §1 · p. 2

2. Related work

Related work establishes that pruning allocation and class-level damage are known concerns, but the paper positions its contribution as a causal account connecting tiny-layer starvation to intrusion-detection security consequences.

2.1 Pruning and the allocation of sparsity across layers

  • Per-layer pruning removes the same fraction from each layer, whereas global pruning applies one threshold; prior work commonly protects small or end layers.Magnitude pruning keeps the largest absolute weights [3], and global allocation usually gives better accuracy at equal total sparsity [4].Source: §2.1 · p. 2

2.2 Pruning damages some classes more than others

  • Prior studies report disproportionate pruning damage to rare or hard classes [7] [8], with theoretical and empirical accounts [10]; this paper distinguishes a removable recipe-induced component in small tabular detectors.The compared literature uses large image networks, gradual pruning, and often leaves the first convolution dense.Source: §2.2 · p. 3

2.3 Layer collapse and conservation

  • Layer-collapse work describes whole-layer failure and related neuron-level risks [12], while prior normalization studies address activation collapse or misleading batch-normalisation estimates [14] [15].The paper’s contribution is to connect the allocation rule and starved layer to intrusion-detection costs.Source: §2.3 · p. 3

2.4 Compression in intrusion detection

  • IoT intrusion-detection studies commonly report aggregate accuracy, weighted F1, model size, or latency, and rarely report per-class results.In a small sample of recent pruning papers, none reports input-layer exemption or per-class results; non-reporting does not establish that exemption was not used.Source: §2.4 · p. 3

2.5 Security of compressed models

  • Prior compression-security work studies backdoors and adversarial inputs, which require model modification or input perturbation; this paper instead examines ordinary traffic misattribution.The stated attacker goal is attribution failure, not evasion.Source: §2.5 · p. 4

2.6 Position of this paper

  • The paper does not claim novelty for protecting first layers, class damage, or normalization-state repair [12] [14]; it claims a causal chain linking allocation, collapse, and security consequences.A related companion study is described as sharing infrastructure but not pruned models, interventions, or result tables.Source: §2.6 · p. 4

3. Materials and method

The study compares three detector architectures on CICIoT2023 and TON_IoT using fixed splits, multiple pruning recipes, and tests of input-layer mechanisms. It also specifies class-damage and security measures, deployment-cost measurements, and normalisation recalibration.

3.1 Datasets

  • CICIoT2023 supplies 3,661,696 records, 39 features, and 34 classes; TON_IoT supplies 93,644 records, 30 features, and 10 classes.TON_IoT is used to test whether the mechanism generalises.Source: §3.1 · p. 4

3.2 Splits and leakage discipline

  • CICIoT2023 is split by capture provenance order into 70/15/15 train, validation, and test partitions, while TON_IoT uses within-class file order.The authors note that TON_IoT's order is weaker than a true temporal split; the split is frozen across models and seeds.Source: §3.2 · p. 4

3.3 Architectures

  • The three models are a 39-feature CNN1D with 192 first-layer weights, an MLP with 9,984 input weights, and an FT-Transformer.All use batch or layer normalisation as appropriate and square-root inverse-frequency class weights.Source: §3.3 · p. 5, Table 1 · p. 5

3.4 Training, pruning and fine-tuning

  • Pruning uses unstructured magnitude masks on convolutional and linear weights, with either per-layer or global thresholds, followed by masked fine-tuning; a gradual schedule is also tested.Biases and normalisation parameters are not pruned; the MLP and transformer capacity sweeps hold their heads and the transformer's token embedding at 80%.Source: §3.4 · p. 5, Table 2 · p. 6

3.5 Paired independent seeds; 3.6 What counts as a damaged class

  • Compression results average five independently trained baseline-and-pruned pairs, and a class is damaged only when its recall drop meets both thresholds in at least three seeds.The drop must be at least 0.10 and more than twice the baseline validation-recall standard deviation.Source: §3.5 · p. 6, §3.6 · p. 6

3.7 Capacity axis and mechanism tests

  • Mechanism tests vary first-layer sparsity, starve the MLP input layer, alter surviving-weight distribution and CNN width, and probe representations or refit the head.Models are compared at equal remaining non-zero weights; representation probes use penultimate-layer features and test-set AUC.Source: §3.7 · pp. 6–7

3.8 Security semantics and threat model

  • Security evaluation separates attack misattribution, attack-to-benign evasion, cross-family errors, and benign-to-attack false alerts, using validation-selected blind spots tested on held-out records.The threat-model folds reuse the same five models, and the cross-family analysis is repeated under three alternative mappings.Source: §3.8 · p. 7

3.9 Deployment cost; 3.10 Statistics and pre-stated gates; 3.11 Normalisation statistics

  • The study measures serialized and estimated sparse model size, CPU latency and throughput, seed-level uncertainty, and normalisation-statistic recalibration without weight updates or labels.Recalibration uses 200 batches of 4,096 training records; inference relies on effect sizes and seed-level bootstrap intervals.Source: §3.11 · p. 8, §3.10 · p. 8, §3.9 · p. 8

4. The collapse, and what it is not

Uniform layer-wise pruning at 80% sparsity sharply damages the CNN's macro-F1, despite a smaller accuracy drop, while comparable or lower remaining weight counts do not produce the same loss in other architectures. The pruned CNN retains class information in its representations, and refitting its final layer restores performance.

4. The collapse, and what it is not

  • Macro-F1 falls from 0.542 +/- 0.006 to 0.271 +/- 0.164 after uniform 80% pruning, while accuracy falls from 0.708 to 0.552 and 17 of 34 classes are materially damaged.The damaged classes include severe recall losses, and the pruned outcome varies substantially across seeds.Source: §4 · pp. 8–9

4.2 It is not lost capacity

  • At 5,823 remaining weights, the CNN loses 0.271 macro-F1, while an MLP at 5,145 weights loses 0.013 and a transformer at 3,987 weights loses 0.096.The MLP has no materially damaged classes at 5,145 weights; Table 3 and Figure 1 frame the comparison by remaining capacity.Source: §4.2 · p. 9, Table 3 · p. 9, Figure 1 · p. 10

4.3 It is not lost information

  • For each collapsed class, a linear probe on the pruned CNN's penultimate features reaches test AUC above 0.908, and refitting only the final layer restores macro-F1 to 0.526.The results indicate that class information remains in the pruned representation even as the model output collapses.Source: §4.3 · p. 10

5. Mechanism: input-layer starvation

Uniform 80% layer-wise pruning starves the CNN’s 192-weight first layer; intervention and dose-response tests link that allocation to collapse, while recalibrating normalization statistics repairs much of the loss.

5.2 Protecting the layer, or pruning globally, prevents the collapse at the same total sparsity

  • 0.271 to 0.013 loss: protecting conv.0 reduces damaged classes from 17 to 2, while global pruning at 80% yields 0.014 loss and no damaged class.Protection retains 154 more weights than uniform pruning; both fixes sharply reduce seed-to-seed spread.Source: Table 4 · p. 11, §5.2 · p. 11

5.3 Dose-response; 5.4 Positive control: starving the MLP reproduces the collapse

  • At 38 surviving weights, CNN loss is 0.271 and MLP loss is 0.323; both architectures show increasing damage as input-layer sparsity rises.The MLP has 18 damaged classes at 38 weights; its loss remains below 0.01 at 1,997 or 998 weights.Source: §5.3 · p. 11, §5.4 · p. 12, Table 5 · p. 12

5.5 The onset is not an absolute weight count

  • At 38 survivors, losses are 0.047, 0.271, and 0.318 for 32-, 64-, and 128-filter CNNs; matched weight counts do not predict damage across widths.Within tested conditions, connected-filter fractions above about 85% coincide with low losses, while fractions below about 65% coincide with high losses.Source: §5.5 · p. 13, Table 6 · p. 13

5.6 What mediates the effect in the CNN: live filters

  • At 96 surviving weights, spread masks yield 0.018 loss and no damaged class, versus 0.212 loss and 12 damaged classes for concentrated masks.At 38 weights, fully spreading survivors across filters is not better than the uniform recipe within seed noise.Source: Table 7 · p. 14, §5.6 · p. 14

5.7 Is there one mediator? Two failed gates

  • The CNN’s damage tracks live filters, but neither live-unit nor covered-feature counts governs MLP damage; effective dimensionality predicts damage in the MLP, not the CNN.The reported Spearman correlations for effective dimensionality are -0.91 in the MLP and -0.47 in the CNN.Source: §5.7 · p. 15

5.8 Recipe-induced and intrinsic damage

  • At the 80% operating point, input-layer allocation accounts for about 95% of total loss; the protected recipe retains 0.013 loss and two damaged, confusable flood-attack classes.The residual damage is associated with poor separability between same-protocol DoS and DDoS pairs.Source: §5.8 · p. 15

5.9 The proximate failure: displaced normalisation statistics

  • Recomputing all running means and variances on 200 unlabeled training batches, without changing weights, raises macro-F1 from 0.271 to 0.502, recovering 85% of the loss.Recalibrating only the first normalisation layer recovers 98% of the gain; among connected channels, the worst displacement is 0.77 standard deviations.Source: §5.9 · p. 15

6. Generality: the same law on a second dataset

Input-layer starvation produces a similar collapse on TON_IoT, although damage from pruning the rest of the network is larger there than on CICIoT2023.

Generality: the same law on a second dataset

  • The TON_IoT evaluation uses 93,644 records, 30 features, and 10 classes, versus 3.7 million records on CICIoT2023.The authors characterize the TON_IoT split as weaker.Source: §6 · p. 16
  • At 38 surviving weights, CNN loss is 0.256 on TON_IoT versus 0.271 on CICIoT2023, and MLP loss is 0.353 versus 0.323.Both architectures’ loss rises monotonically with input-layer sparsity on TON_IoT.Source: §6 · p. 16, Table 9 · p. 17
  • At the collapse point, input-layer damage accounts for about 75% of total loss on TON_IoT, compared with about 95% on CICIoT2023.The protected CNN and ordinarily pruned MLP lose 0.065 and 0.085 on TON_IoT, respectively.Source: §6 · p. 16

7. Security consequence

Uniformly pruned detectors misattribute validation-selected blind-spot traffic more often than protected, globally pruned, or dense detectors, and also flag more benign traffic as an attack.

7.1 Threat model

  • The attacker knows the target uses uniform layer-wise pruning, but not its specific trained model, and sends ordinary traffic without perturbing features or accessing the model.The stated goal is attribution failure rather than invisibility.Source: §7.1 · p. 17

7.2 Blind spots are small, predictable, and transferable

  • Validation-selected blind spots comprise one to five classes and are evaluated on untouched test records from held-out deployed models.MITM-ArpSpoofing appears in all ten folds, and DoS-HTTP_Flood in seven.Source: §7.2 · p. 17

7.3 The advantage comes from the recipe, not from the attacker

  • 72.4% of identical blind-spot traffic is misattributed by uniformly pruned detectors, versus 49.6% with protection, 51.5% with global pruning, and 47.3% for dense detectors.The excess over dense is 25 points for uniform pruning, versus 2 for protection and 4 for global pruning.Source: §7.3 · p. 17, Table 10 · p. 18

7.4 The alert flood

  • 56.3% of benign records are flagged as attacks by uniform pruning, versus 28.6% for dense detectors; fixed recipes reduce this to 30.8% and 34.0%.Recalibrating the uniformly pruned detector reduces the rate to 33.3%; the authors did not measure operator behavior.Source: Table 11 · p. 18, §7.4 · p. 18

7.5 What the failure is, and is not

  • The measured security consequence is misattribution plus elevated false alerts, not silent evasion; attack-to-benign labeling is 18–21% across recipes, including dense.Choosing blind spots adds 19 percentage points over the uniformly pruned detector’s 53% misattribution of randomly chosen detectable attack types.Source: §7.3 · pp. 17–18, §7.5 · p. 19

8. Defence and its cost

Protecting the first layer or using global pruning prevents the observed collapse, while recalibration repairs an already-pruned model without changing its weights.

8.1 Two recipes that remove the collapse

  • The preventive edits either skip conv.0 in the per-layer pruning rule or apply a global magnitude threshold across the same layers; repair uses one forward pass of unlabeled training data.The recalibration pass runs the deployed model with normalization layers in training mode.Source: §8.1 · p. 19, §8.3 · p. 20

8.3 A deployment checklist

  • The deployment checklist calls for checking first-layer connectivity, recomputing normalization statistics, evaluating per-class attribution, and reporting seed-to-seed spread.In tested conditions, losses were small above about 85% connected filters and large below about 65%.Source: §8.1 · p. 19, §8.3 · p. 20

8.2 What the fixes cost

  • 0.529 and 0.527 macro-F1 under the two fixes compare with 0.271 under uniform pruning; serialized size is 125,683 bytes in all four deployment cases.Median CPU latency differences were smaller than the run-to-run spread at every tested batch size; the repair changes neither weights nor size.Source: §8.2 · pp. 19–20, Table 12 · p. 20

9. Limitations and scope

The proposed mechanism is scoped to small tabular detectors with tiny input layers under uniform pruning; it does not account for disparate impact in large image networks. The section also distinguishes measured associations from explanations and identifies benchmark and evaluation boundaries.

9.1 What the mechanism explains

  • The mechanism addresses small tabular detectors with tiny input layers under uniform pruning, not disparate-impact results in large image networks with dense first layers.The paper frames its account as a decomposition, not a replacement for explanations of the residual component.Source: §9.1 · pp. 20–21

9.2 What is correlational

  • Recalibration restores less loss in the perceptron and on TON_IoT than in the CICIoT2023 CNN, so starvation also removes accuracy that statistics cannot restore.The reported recovery is 44 to 65% for the perceptron, 38% on TON_IoT, and 83 to 92% for the CNN on CICIoT2023.Source: §9.2 · p. 21
  • The displaced-statistics account after fine-tuning is an explanation, not a measured process; effective dimensionality is a post-hoc association likely to be a symptom rather than a cause.The paper identifies tracking per-channel batch means during fine-tuning as a direct test.Source: §9.2 · p. 21

9.3 Pre-stated criteria that were not met

  • Eight headline hypothesis groups had unmet criteria, including threshold misses, mechanism surprises, and effects that met the predicted direction but not size.The criterion record contains 38 met, 34 unmet, and 10 reference-only rows; the authors say these narrow claims but do not change the mechanism.Source: §9.3 · p. 21

9.4 Data, baselines and untested settings

  • The benchmarks are single-network datasets with known labeling limitations, and the reported damage magnitudes are benchmark-specific.The width sweep covers three widths of one three-tap CNN family and leaves a gap between 65% and 85% connected filters; structured pruning and quantisation-aware training were not tested.Source: §9.4 · p. 21

10. Conclusion

Uniform layer-wise pruning can sharply damage class-level performance in the standard IoT intrusion detector because pruning starves its first layer. The paper reports prevention and repair approaches, including recalibration that changes no weights.

10. Conclusion

  • Uniform pruning halves macro-F1 and doubles the false-alert rate while accuracy falls by only 16 points in the standard IoT intrusion detector.A 192-weight first layer is left with 38 weights, and displaced running means in a few first-layer normalisation channels accompany the deployed model's collapse.Source: §10 · p. 21
  • Two other architectures at equal or lower remaining weight counts do not collapse, while increasing starvation, a perceptron positive control, and a second dataset support the reported pattern.The conclusion attributes the failure to where pruning lands rather than how much weight remains.Source: §10 · p. 21
  • Protecting the first layer or pruning globally prevents collapse at the same size and latency, while recalibration on unlabelled training data repairs 85% without changing weights.Protection brings selected-subset misattribution and benign false-alert rates toward dense-model values; recalibration recovers most macro-F1 loss and reduces false alerts.Source: §10 · p. 22
  • The conclusion recommends reporting class-wise attribution, seed-to-seed spread, and connected-filter fraction alongside overall accuracy, with a recalibration pass before deployment.These measures and the recalibration pass are presented as low-overhead checks for compressed detectors in security roles.Source: §10 · p. 22

Supplementary material

The supplementary material catalogs tables and figures covering per-class damage, dose-response, model comparisons, threat-model evaluations, and recalibration. Its captions identify the scope of those materials, while the supplied passages do not provide most underlying cell values.

Supplementary material

  • The supplement states that tables are generated from committed CSV files in the companion repository and that values are five-seed means unless a seed column is present.Source: Supplementary material · p. 24

S1. Per-class damage by recipe (CICIoT2023, CNN)

  • Table S1 reports mean recall loss against the dense baseline and the fraction of seeds with material class damage for three one-shot CNN pruning recipes.Source: Table S1 · p. 24

S2. Per-class dose-response and the transformer arm

  • The dose-response and transformer materials include per-class recall-loss tables for CNN, MLP, and TON_IoT, plus transformer validation, capacity, probe, and head-refit tables.The supplied captions identify these table scopes but do not show their cell values.Source: Table S2 · p. 24, Table S3 · p. 25, Table S4 · p. 26

S3. Alert-family mapping

S4. Live channels, the K x F grid and effective-rank correlations

S5. TON_IoT dose-response

  • The S5 table is titled as a TON_IoT dose-response by cell, and Figure S1 places the TON_IoT dose-response over faded CICIoT2023 curves.Source: Table S13 · p. 28, Figure S1 · p. 29

S6. Threat model

S7. Normalisation recalibration

S8. Deployment benchmark

The deployment benchmark supplement lists measurements for model size and CPU performance by pruning recipe, seed, and batch size. The supplied passages provide table scopes and benchmark conditions, but no measured values.

S8. Deployment benchmark

  • Table S26 compares parameter counts, non-zero parameters, serialized bytes, and sparse payload across recipes and seeds.Source: Table S26 · p. 34
  • Table S27 reports CPU latency and throughput by recipe, seed, and batch size, using one thread, 25 warm-ups, and 100 repeats.Source: Table S27 · p. 34

S9. Pre-stated gates and notebook map

The notebook map records pre-stated criteria and their verdicts, while the supplementary source map traces manuscript tables, figures, and sections to their numerical sources.

S9. Pre-stated gates and notebook map

Related papers

Loading 2609.30729v1…