PaperlayerView exact revision on arXiv

Source-linked AI summary

UnAct: Gradient-Free Unlearning via Targeted Activation Intervention

Saeed Abdul Muizz, Aayat Rafiq, Iqra Altaf Gillani, Janibul Bashir

arXiv:2610.04426v1cs.LGcs.CV

TL;DR

Models can retain training-example influence after deletion, motivating removal of designated data; this paper asks whether class unlearning can use only the available forget images. UnAct uses a forward-only, iterative edit, and on ResNet-18 its retain accuracy stays close to retraining across tested forget-set sizes.

  • Problem

    Models can retain training-example influence after deletion, motivating removal of designated data; the paper asks whether class unlearning can use only the available forget images.

    Source: §1 · p. 1, §3.1 · p. 4

  • Method

    UnAct ranks late-layer units by their forget signal to the classifier, then softly attenuates selected connections iteratively using forward passes over forget images.

    Source: §1 · p. 2, §5 · p. 9

  • Results

    Across tested forget-set sizes on ResNet-18, UnAct's retain accuracy stays within 2.45 points of retraining, whereas SSD and LFSSD lose up to 86 points on some classes.

    Source: §1 · p. 2

  • Takeaways & Limitations

    On ResNet-18, UnAct is the only method reported safe at every tested forget-set size, and is closest to retraining with at most ten images on CIFAR-10 and CIFAR-20.

    Source: §5 · p. 9

    UnAct suppresses rather than erases a class (fine-tuning on a few forget images restores much of it); its scoring assumes one class per request.

    Source: §5 · p. 9

Abstract

from arXiv · show

Machine unlearning seeks to remove the influence of designated training data from a trained model without retraining from scratch. Retrain-free methods such as Selective Synaptic Dampening (SSD) and its label-free variant LFSSD avoid full retraining but still require backpropagation and parameter importance computed over the entire dataset. We ask: what happens when a deletion request arrives with only a few images of the class to be forgotten? To answer this question, we introduce UnAct, a gradient-free class-unlearning method that needs only forward passes over the forget images. UnAct scores late-layer units by their responses, attenuates the most responsive connections, and repeats this for up to 20 rounds using no gradients, no labels, and no retained data. On ResNet-18 trained with CIFAR-10, CIFAR-20, and CIFAR-100, UnAct is competitive with SSD and LFSSD when forgetting entire classes and, unlike them, never collapses the network when forget data is scarce. On ResNet-18, across all tested sizes, UnAct's retain accuracy stays within 2.5 points of retraining, while SSD and LFSSD, at their full-class operating points, lose up to 86 points on some classes. With five forget images on CIFAR-10, UnAct's distance to retraining is 0.21 points, against 67 for LFSSD and 90 for SSD, and re-selecting SSD's threshold at each size with an oracle does not close the gap. In preliminary transfer to ViT-B/16, UnAct's distance to retraining is 11.5 against 33.7 for SSD, and a request is 19x faster than SSD when SSD computes its importance at request time. The code is available at https://github.com/abdulmuizz0903/UnAct

1 INTRODUCTION

UnAct targets post-hoc class unlearning when only forget images are available, using forward-only, rank-based edits instead of gradients or retained data. Across tested settings, it remains close to retraining when forget data is scarce, while SSD and LFSSD can lose substantial retain accuracy.

1 INTRODUCTION

  • A deletion request can leave sensitive training examples encoded in a model, motivating methods that remove their influence without costly retraining.The cited risks include membership inference and model inversion; the data may also be mislabeled, corrupted, outdated, or harmful.Source: §1 · p. 1
  • SSD and LFSSD avoid retraining but still require gradients and an importance estimate over the full training set.SSD uses Fisher information; LFSSD replaces it with the gradient of the squared output norm to avoid labels.Source: §1 · pp. 1–2
  • UnAct scores late units from forget-image forward passes, selects a fixed fraction by rank, and softly attenuates their weights with re-scoring.Its scoring uses no labels, gradients, or retained data; rank-based selection bounds edit size independently of the number of forget images.Source: §1 · p. 2
  • Across tested forget-set sizes, UnAct’s ResNet-18 retain accuracy stays within 2.45 points of retraining, while SSD and LFSSD lose up to 86 points on some classes.With five CIFAR-10 forget images, UnAct’s distance to retraining is 0.21 points, versus 67.07 for LFSSD and 89.85 for SSD.Source: §1 · p. 2
  • With full forget sets, UnAct reduces forget accuracy to 0% while keeping retain accuracy within 1.15 points of retraining on each ResNet-18 dataset.The comparison covers CIFAR-10, CIFAR-20, and CIFAR-100 and reports competitiveness with SSD and LFSSD.Source: §1 · p. 2
  • Iterative activation re-ranking widens unit coverage rather than repeatedly deepening the same edit, trading request time against closeness to retraining.The paper also reports that editing only the last residual block suffices to forget on ResNet-18, while no method removes the class from penultimate features.Source: §1 · p. 2

2 RELATED WORK

Related work spans retraining, post-training optimization, retrain-free parameter dampening, and unit-level edits. UnAct is positioned as a retrain-free unit-level method with a fixed-size edit and no need for retained data or fine-tuning.

Retraining-based unlearning

  • Retraining from scratch on retained data is the oracle for complete removal, but it is computationally costly and requires retained data for each request.SISA retrains only the shard containing deleted samples, while Amnesiac unlearning subtracts logged updates; both require setup and stored information.Source: §2 · p. 3

Optimization-based unlearning

  • Optimization-based methods use fine-tuning and may require labels, retained data, or both; UnAct is aimed at requests lacking labels and retained data.The paper makes no claim of superiority over optimization-based methods, which use more information.Source: §2 · p. 3

Retrain-free dampening

  • SSD uses diagonal Fisher importance over forget and full training data, while LFSSD removes labels but still uses gradients and full-dataset sensitivity.Both are retrain-free baselines; UnAct instead uses forget-image forward activations and is reported reliable with few forget images.Source: §2 · p. 4

Unit-level removal

  • Prior unit-level approaches prune or perturb class-discriminative units, often using retained data, fine-tuning, or a one-shot edit; UnAct uses forget-only forward scores and iterative re-scoring.UnAct fixes the fraction of units edited in advance and requires no fine-tuning afterward.Source: §2 · p. 4

Positioning of UnAct

  • UnAct combines retrain-free efficiency with a unit-level edit whose size is fixed in advance, but remains approximate and is evaluated against retrained models.The paper also examines whether the edit removes the class beyond accuracy-based metrics and simple membership attacks.Source: §2 · p. 4

3 METHOD

UnAct is a post-hoc class-level method that uses forward-only scores from forget images to select and attenuate units in a classifier-relative late-layer scope. It re-scores the edited model across rounds without labels, gradients, or retained data.

3 METHOD

  • UnAct repeatedly scores classifier-relative units on forget images, rank-selects them, and attenuates their incoming and outgoing parameters without removing them.The edited model is rescored after each round.Source: §3 · p. 4, Figure 1 · p. 5, §3.3 · p. 6

3.1 PROBLEM FORMULATION

  • Class-level unlearning seeks to remove a target class while preserving behavior on other classes; retraining on retained data is the ideal reference.UnAct uses only the available forget images and does not access retained data, read labels, optimize a loss, or run a backward pass.Source: §3.1 · p. 4

3.2 SCOPE AND UNITS

  • The editable scope is late layers whose units write to the representation read by the classifier, leaving earlier shared features untouched.Units are convolutional channels in ResNet-18 and MLP hidden neurons in ViT-B/16.Source: §3.2 · pp. 4–5

3.3 UNIT SCORES

  • For ResNet-18, each unit’s score is its mean post-ReLU channel activation over forget images, matching the feature received by the classifier.The channel output is averaged over spatial positions.Source: §3.3 · p. 5
  • For ViT-B/16, UnAct scores neurons by direct class-logit attribution because activation alone fails; this first-order measure ignores paths through later blocks.The class is inferred from the highest mean logit on the forget images under the original model, effectively assuming a single-class forget set.Source: §3.3 · pp. 5–6

3.4 SELECTION AND ITERATIVE ATTENUATION

  • Percentile selection bounds edit size independently of forget-set size; rescoring lets later rounds widen coverage, with (p, k) controlling breadth and γ strength.A request costs k forward passes over n forget images and uses no backward pass, optimizer state, or retained data.Source: §3.4 · p. 6

4 RESULTS

Across full-class and few-image requests, UnAct stays close to retraining on ResNet-18, while SSD and LFSSD can sharply damage retain accuracy when forget data is scarce. The experiments also examine sequential requests, request cost, and what the distance-to-retraining metric certifies.

4.1 EXPERIMENTAL SETUP

  • The experiments train ResNet-18 from scratch on CIFAR-10, CIFAR-20, and CIFAR-100, and fine-tune ImageNet-1k-pretrained ViT-B/16 on CIFAR-10.Each request removes one class; unless stated otherwise, the forget set contains all its training images, and results average five classes fixed before experiments.Source: §4.1 · pp. 6–7

4.2 FINDINGS

  • With full forget sets, all three methods reach 0.00% forget accuracy on ResNet-18; UnAct is closest on CIFAR-10, while LFSSD is closest on CIFAR-20 and CIFAR-100.On ViT-B/16, UnAct’s Δ is 11.49 against SSD’s 33.74, but its mean retain accuracy remains 6.5 points below retraining.Source: Table 2 · p. 7, §4.2 · p. 8

4.1 EXPERIMENTAL SETUP

  • The study compares SSD and LFSSD with UnAct at selected operating points, and evaluates full-class, few-image, sequential-request, and request-cost settings.Δ is the mean per-class distance to retraining in percentage points and accounts for both retain-accuracy loss and deviation on forgotten classes.Source: §4.1 · p. 7

4.2 FINDINGS

  • Across 125 few-image runs, UnAct’s retain accuracy is never more than 2.45 points below retraining; SSD and LFSSD can lose up to 85.34 and 86.09 points per class on CIFAR-10.Their mean retain accuracy falls to 29.60% and 11.26%, respectively, against UnAct’s lowest mean of 95.51%; UnAct’s forget accuracy is near zero from n = 5 on ResNet-18.Source: Figure 2 · p. 8, §4.2 · pp. 8–9
  • At n = 5, UnAct’s Δ is 0.21, 0.77, and 1.37 on the three ResNet-18 datasets, and 12.63 on ViT-B/16.On CIFAR-10 and CIFAR-20, it is the closest method at every n ≤10; oracle re-selection still leaves SSD’s Δ at 87.5, 64.6, and 46.1 at n = 5.Source: §4.2 · p. 9
  • Across five sequential requests, UnAct keeps requested classes at or below 2.7% forget accuracy on CIFAR-10 and CIFAR-20, while SSD exceeds 50% by the fifth request.UnAct’s CIFAR-20 retain accuracy falls to 70.1% after five requests, versus SSD’s 85.7%; SSD performs better on CIFAR-100.Source: §4.2 · p. 9
  • A full-class ViT-B/16 request takes UnAct 73 s versus SSD’s 1,373 s, or 19× faster from cold; five images take UnAct 0.1 s.On ResNet-18, the best one-round configuration removes a whole class in 0.58, 0.38, and 0.12 s across the three datasets.Source: §4.2 · p. 9
  • Δ measures prediction agreement with retraining, not complete removal: a linear probe recovers the forget class from penultimate features with AUROC of at least 0.98 after every method.A trivial class-logit mask reaches Δ = 0.09, 0.25, and 0.33 on ResNet-18, as close to retraining as any method.Source: §4.2 · p. 9

5 DISCUSSION AND CONCLUSION

UnAct uses forward-only, forget-image signals to edit classifier-facing units, and is especially robust when forget data is scarce. Its scope and evidence remain limited, including to single-class requests and CIFAR-scale experiments across two architectures and one seed.

5 DISCUSSION AND CONCLUSION

  • UnAct scores late-layer units by their forget signal to the classifier, rank-selects and softly attenuates a fixed fraction, then rescores without gradients, labels, or retained data.The method is designed for one class per request.Source: §5 · p. 9
  • With at most ten forget images, UnAct is the closest method to retraining on CIFAR-10 and CIFAR-20, and SSD threshold re-selection does not close the gap.As the forget set grows, LFSSD becomes comparable or closer on some datasets.Source: §5 · p. 9
  • UnAct suppresses rather than erases a class: a few fine-tuning steps on a handful of forget images restore much of it, as they do for SSD.On ViT-B/16, its retain accuracy remains below the retrained model; the score must also be adapted to how units reach the classifier.Source: §5 · pp. 9–10
  • Evidence covers CIFAR-scale data, two architectures, and one training seed, with all methods’ operating points selected on the test split.The authors report that code, experiment scripts, and results files are provided in the supplementary material and online.Source: §5 · p. 10, REPRODUCIBILITY STATEMENT · p. 10
  • The authors report using generative AI for code debugging, formatting, literature identification, and manuscript editing, but not to propose the method or generate or clean datasets.They state that AI-assisted work was reviewed and that they take responsibility for the final content.Source: AI USE STATEMENT · p. 10

A METHOD DETAILS

UnAct's residual-block edits account for identity shortcuts and frozen BatchNorm statistics, while its rank-based selection has defined behavior for distinct and tied scores.

A.1 THE IDENTITY PATH IN RESIDUAL BLOCKS

  • On ResNet-18, scoring uses the residual-block output including the identity shortcut, but the update edits only the residual branch, so one round does not fully suppress a selected channel.Several rounds help for this reason; fully connected layers have no such path.Source: §A.1 · p. 12

A.2 WHAT ATTENUATION DOES ON RESNET-18

  • UnAct scales each selected channel’s conv2 filter and BatchNorm scale and shift, while frozen running statistics make the resulting pre-activation differ from a uniform channel rescaling.The classifier column is scaled exactly, while the classifier bias is not edited.Source: §A.2 · p. 12

A.3 SELECTION COUNT AND TIES

  • With distinct scores, percentile selection chooses N −⌈(N − 1) p/100⌉ units, slightly more than N(1−p/100); ties cause all tied units to be selected.For ViT class-token scores, ties are broken by index after selecting the same number of units by rank.Source: §A.3 · p. 13

B TRAINING AND EVALUATION DETAILS

The experiments use separately specified ResNet-18 and ViT-B/16 training setups and report evaluation and sensitivity procedures. Selected hyperparameters vary by dataset, while UnAct's attenuation factor has a threshold below which results are insensitive.

B TRAINING AND EVALUATION DETAILS

  • ResNet-18 uses a CIFAR stem and 100 epochs of SGD with momentum 0.9, weight decay 5×10−4, batch size 256, warm-up, cosine decay, crops, and flips.Retrained models use the same recipe on the retain set alone.Source: B TRAINING AND EVALUATION DETAILS · p. 13
  • ViT-B/16 is fine-tuned from torchvision ImageNet-1k weights for eight epochs; its pretraining differs from the SSD paper’s ImageNet-21k ViT, so these numbers are not a reproduction of those ViT results.The five retrained ViT models use the same recipe on the retain set.Source: B TRAINING AND EVALUATION DETAILS · p. 13
  • For forget class 3 on each dataset, label replacement produced bit-identical UnAct and LFSSD weights, while SSD weights differed in 10.9–11.0 million parameters.SSD and LFSSD importance is computed over the full training set once per model; the LFSSD importance depends on batch size because within-batch gradients can cancel.Source: B TRAINING AND EVALUATION DETAILS · p. 13
  • Membership inference is measured using a logistic regression on prediction entropy to distinguish retain-set training images from test images; the reported MIA is the fraction of forget-set training images labeled members.Source: B TRAINING AND EVALUATION DETAILS · p. 13

C HYPERPARAMETER GRIDS AND SENSITIVITY

  • Selected settings vary by dataset: SSD’s α is 8, 15, and 50; LFSSD’s is 6, 10, and 30; UnAct’s (p, γ, k) settings are (99, 0.01, 20), (99, 0.1, 20), and (90, 0.3, 1).The sequences correspond to CIFAR-10, CIFAR-20, and CIFAR-100, respectively.Source: C HYPERPARAMETER GRIDS AND SENSITIVITY · p. 14
  • UnAct is insensitive to γ below a threshold, and its p and round-count k trade off through coverage Γk.The supplied SSD and LFSSD grid captions specify rows of α and columns of importance batch size; UnAct’s grid uses rows of p and columns of rounds k.Source: C HYPERPARAMETER GRIDS AND SENSITIVITY · p. 14, Table 3 · p. 14, Table 4 · p. 14

D SELECTION WITHOUT A RETRAINED MODEL

A retrain-free selection rule can choose operating points without a retrained model, although it still uses a labeled retain evaluation set. UnAct's configurations transfer across datasets more reliably than the SSD and LFSSD configurations described here.

D SELECTION WITHOUT A RETRAINED MODEL

  • The retrain-free rule requires forget accuracy 0 and maximizes retain accuracy; it selects the Δ-optimal configuration in 8 of 9 cases, with UnAct on CIFAR-10 only 0.03 worse.It still uses a labeled retain evaluation set, here the test split; a practitioner would use held-out validation, so the result shows retraining is unnecessary, not that selection is held out.Source: D SELECTION WITHOUT A RETRAINED MODEL · p. 14
  • A shared UnAct configuration, (99, 0.01, 20), keeps Δ≤3.46 across datasets, while transferred SSD and LFSSD configurations reach Δ up to 32.42 and 93.72.The authors attribute the difference to importance ratios varying in meaning across models, unlike a percentile and attenuation factor.Source: D SELECTION WITHOUT A RETRAINED MODEL · p. 14
  • UnAct’s complete grid reports Δ averaged over five forget classes by p and rounds k at selected γ; its usable region trades higher p against more rounds at roughly constant coverage.Figure 3 also shows SSD and LFSSD grids at selected λ and marks each method’s selected configuration.Source: Table 5 · p. 15, Figure 3 · p. 15

E HELD-OUT CLASSES

  • On three held-out classes, all methods forgot the classes and stayed close to retraining at their selected configurations, indicating the baselines were not weakened by the selection.At SSD’s published default α = 10, λ = 1, its reported failure reproduced at some importance batch sizes.Source: E HELD-OUT CLASSES · p. 16

F FULL-CLASS RESULTS

The section reports the parameter share modified at selected configurations and evaluates how operating-point selection transfers to held-out classes. The supplied evidence does not include the full-class benchmark table.

F FULL-CLASS RESULTS

  • UnAct modifies 4.80%, 4.95% and 2.18% of parameters on CIFAR-10, CIFAR-20 and CIFAR-100, respectively.SSD modifies 3.63%, 0.60% and 0.23%, while LFSSD modifies 3.09%, 0.79% and 0.12%.Source: F FULL-CLASS RESULTS · p. 16
  • The retrain-free selection rule keeps configurations with mean forget accuracy 0 and chooses the one with highest retain accuracy, without a retrained model.It uses a labelled retain evaluation set; transfer selects the configuration with the lowest mean ∆ on the other two datasets.Source: Table 6 · p. 17
  • On held-out classes chosen from reported SSD failures, the table evaluates each method at its configuration selected on the five main forget classes and also reports an oracle best-grid result.At SSD’s published default, retain accuracy on baby ranges from 72.8 to 2.0 across reported importance batch sizes; lamp ranges from 73.8 to 34.2.Source: Table 7 · p. 17

G WHAT DISTANCE TO RETRAINING SHOWS, AND WHAT IT MISSES

The section clarifies that ∆ measures output accuracy, not whether the class has been removed from the model. Additional checks show that forgotten images may be sent to different classes than under retraining, while the class remains linearly decodable from penultimate features.

G WHAT DISTANCE TO RETRAINING SHOWS, AND WHAT IT MISSES

  • ∆=0.09, 0.25 and 0.33 for a logit mask that changes only the forget-class output, comparable to SSD and LFSSD and lower than UnAct on CIFAR-20 and CIFAR-100.The mask needs the class index but no images; ∆ indicates the class is no longer predicted and retain accuracy is preserved, not that the class has been removed.Source: G WHAT DISTANCE TO RETRAINING SHOWS,… · p. 17, Table 9 · p. 18
  • Only the combination of classifier-column and incoming-weight edits forgets the class on every dataset; either edit alone leaves substantial forget accuracy on CIFAR-20 and CIFAR-100.Classifier columns alone leave 20.2% and 41.4% forget accuracy, while incoming weights alone leave 20.6–51.7%.Source: G WHAT DISTANCE TO RETRAINING SHOWS,… · p. 17
  • A linear probe recovers the forget class from penultimate features with test AUROC of at least 0.98 after every method, and at least 0.96 after retraining.The probe is refit on each model’s features, so it measures linear decodability rather than whether the original readout still works.Source: G WHAT DISTANCE TO RETRAINING SHOWS,… · pp. 17–19
  • On CIFAR-10, 31.4% of UnAct’s forgotten images go to the class predicted by retraining, versus 65.5% for SSD, 62.5% for LFSSD and 62.4% for the logit mask.The passage reports the same pattern on CIFAR-20 and CIFAR-100 and notes that ∆ does not show this difference.Source: G WHAT DISTANCE TO RETRAINING SHOWS,… · p. 18
  • Figure 5 shows SSD and LFSSD are bimodal at small forget-set sizes: some classes’ retain accuracy collapses alongside forget accuracy, while others change little.The figure plots mean ∆ over five forget classes and faint markers for individual classes, with ∆ on a log scale.Source: Figure 5 · p. 19

H FORGET-SET SIZE

The forget-set-size analysis compares how much evidence each method needs and tests whether SSD’s small-sample results reflect its fixed operating point. SSD improves with per-size oracle tuning at some sizes, but its gap remains at very small sizes.

H FORGET-SET SIZE

  • UnAct’s n⋆ is 25, 5 and 5 images on CIFAR-10, CIFAR-20 and CIFAR-100; LFSSD’s is 100, 25 and 5, while SSD’s is the whole class on all three datasets.n⋆ is the smallest size within a factor of two of each method’s own best ∆ and should be read alongside absolute ∆ values.Source: H FORGET-SET SIZE · p. 19, Table 10 · p. 20
  • With SSD’s α re-selected at each size, ∆ improves at n=500 to 8.91 versus 51.39 on CIFAR-10 and 1.83 versus 18.02 on CIFAR-20.UnAct runs at its full-class configuration and receives no per-size tuning; SSD’s per-size oracle requires a retrained model at each size.Source: H FORGET-SET SIZE · p. 20, Table 11 · p. 20
  • At n≤10, SSD’s ∆ remains at 25 or above on every dataset, while UnAct’s single configuration stays near its full-class value.The authors conclude that SSD’s small-n failure within the published range is not only a calibration artefact; the oracle comparison is available only for SSD.Source: H FORGET-SET SIZE · pp. 19–20

I SEVERAL CLASSES

Sequential requests show that performance depends on dataset and request pattern: UnAct maintains forgetting on CIFAR-10 and CIFAR-20 but loses retain accuracy over repeated CIFAR-20 requests, while SSD performs better on CIFAR-100. Simultaneous multi-class removal at single-class settings fails for both methods.

I SEVERAL CLASSES

  • Across CIFAR-10 and CIFAR-20 class orders, UnAct’s forget accuracy on requested classes never exceeds 2.7%, while SSD exceeds 50% by the fifth request in every order.The sequence applies five one-class requests to the already-edited model without retuning; SSD recomputes importance at each request, and LFSSD is not included.Source: I SEVERAL CLASSES · pp. 20–21, Figure 6 · p. 21
  • On CIFAR-20, UnAct retain accuracy falls to 70.1% after the fifth request, with a worst order at 59.6%; SSD stays at 85.7%.UnAct retain accuracy is 85.1% to 83.7% over about the first three requests before eroding.Source: I SEVERAL CLASSES · p. 21
  • On CIFAR-100, SSD keeps retain accuracy at 77.1% and average forget accuracy at most 0.5%, while UnAct retain accuracy falls from 76.4% to 70.8%.The authors attribute the trade-off in part to UnAct’s accumulated per-request retain-accuracy costs and its modifying roughly ten times more parameters than SSD.Source: I SEVERAL CLASSES · p. 21
  • At single-class settings, simultaneous removal of five classes fails for both methods: ∆ is 84.76, 60.51 and 43.66 for UnAct versus 94.54, 89.97 and 81.29 for SSD.Neither method was tuned for simultaneous multi-class removal; UnAct is designed for one class per request.Source: I SEVERAL CLASSES · p. 21, Table 13 · p. 22

J CLASS SELECTIVITY

The analyses examine UnAct’s class selectivity, cost, ViT scoring, relearning behavior, and edit coverage. They report how these properties vary across configurations and settings.

J CLASS SELECTIVITY

  • At p = 75, the selected channels’ forget-to-retain activation mass ratio fell from 4.38 to 3.19 to 2.54 as more classes shared the 512 channels.The channels carried less forget activation and more retain activation as the number of classes sharing them increased.Source: J CLASS SELECTIVITY · p. 22, Table 14 · p. 23, Figure 7 · p. 23

K COST

  • At k = 5 on every dataset, and at selected k = 20 on CIFAR-10 and CIFAR-20, UnAct was slower than SSD with precomputed importance but faster than SSD computing importance at request time.The timing comparison uses paired sessions; SSD’s cold setting computes its full-training-set importance at request time.Source: K COST · p. 23, Figure 8 · p. 24, Table 15 · p. 24

L VISION TRANSFORMER DETAILS

  • For ViT-B/16, restricting UnAct to the last two blocks raised ∆ to 44.24, while scoring neurons by positive class-token activation yielded ∆ = 95.41.The activation score was the closest analogue to the ResNet score; the reported ViT variant instead scores MLP neurons in the last four blocks using a class-token score.Source: L VISION TRANSFORMER DETAILS · p. 23, Table 17 · p. 26
  • UnAct’s ViT score needs no backward pass, but it reads outgoing weights and the classifier row rather than relying on activations alone.The score is computed in the forward pass.Source: L VISION TRANSFORMER DETAILS · p. 23

M RELEARNING

  • After five relearning steps on ViT-B/16, UnAct’s forget accuracy was 87.3%, versus 33.6% for SSD and 0.0% for the retrained model.The result is averaged over five forget classes at m = 5.Source: M RELEARNING · p. 24, Figure 11 · p. 27

N COVERAGE

  • Measured coverage stayed close to the union bound for small k; the bound held in every run, and usable configurations occupied lower coverage bands as class count grew.The naive min(1, k(1 − p/100)) bound was exceeded in 489 of 1,500 runs because percentile selection kept slightly more units; the km/N bound was never exceeded.Source: N COVERAGE · p. 25, Figure 12 · p. 27

M RELEARNING

  • On ResNet-18, both unlearned models regained much of the forgotten class within 10–20 relearning steps, while the retrained model remained near zero at step 10.At m = 1, the probe also degraded the retrained model, so the authors draw no conclusion from that setting.Source: M RELEARNING · pp. 23–24, Figure 10 · p. 26
Loading 2610.04426v1…