Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders

• Protein annotation faces a growing label space, while prior protein-encoder work largely assumes fixed labels and vision CIL transfer to frozen protein representations remains untested.Source: §1 · pp. 1–2
• The controlled benchmark compares nine CIL methods across five frozen protein encoders, two enzyme datasets, and four phase-construction protocols.Source: §1 · p. 2
• No existing CIL method transfers reliably across encoders and dataset scales, with instability primarily representation-dependent rather than phase-dependent.Source: §1 · p. 2
• The instability is linked to weak gradient-based stability signals, heterogeneous embedding geometry, and covariance-capacity needs that vary with dataset scale and encoder.Source: §1 · p. 2

• A frozen encoder maps proteins to embeddings that remain fixed as learners encounter disjoint new-class phases and classify across all classes seen so far.Source: §2 · p. 3
• The benchmark caches embeddings from five sequence, structure, and sequence–structure encoders and fixes each method’s hyperparameters across encoder-dataset configurations.Source: §2 · p. 3
• Four phase protocols vary alignment with the EC hierarchy, while keeping phase sizes comparable and each seed’s incremental stream consistent across methods.Source: §2 · p. 3
• Nine CIL methods span regularization, replay, prototype or covariance, and projection or subspace approaches, with Naive fine-tuning and Joint training as reference points.Source: §2 · p. 3
• After each phase, evaluation covers all classes observed so far; average accuracy, forgetting, and plasticity summarize retained performance, degradation, and new-class acquisition.Source: §2 · p. 3

3.1 Overall results.

• No method dominates all five encoders on either dataset; method rankings vary, and replay, prototype, and projection methods show distinct failure patterns.Replay methods retain plasticity but incur substantial forgetting; FeTrIL forgets severely, FeCAM loses plasticity on EC Number, and RanPAC deteriorates on GearNet-Edge.Source: §3.1 · p. 4, Table 1 · p. 4
• Figure 2 tracks average incremental accuracy and forgetting across phases, then final accuracy across top-level EC families, where performance varies.The phase trajectories show how aggregate differences emerge as classes are introduced.Source: §3.1 · pp. 4–5, Figure 2 · p. 5

3.2 Robustness Across Phase Construction Protocols

• Alternative phase constructions preserve the broad performance pattern, while encoder-dependent variability remains substantial.RanPAC remains strongest on EC Number and RanPAC and FeCAM are consistently strong on Enzyme Reaction; cross-encoder standard deviations for RanPAC remain approximately ±20–28%.Source: §3.2 · p. 5, Table 2 · p. 5

3.3 Diagnostic Analysis

• Near-zero inter-phase conflict and concentrated Fisher information indicate weak gradient-based stability signals, while covariance condition numbers span roughly five orders across encoders.Intrinsic dimensionality and class separability also vary by encoder and dataset, indicating heterogeneous class geometries.Source: §3.3 · p. 6, Table 3 · p. 6
• Under a shared covariance, transferase–hydrolase confusions are especially prominent for ESM2-150M and much weaker for SaProt.Finer covariance estimates can alleviate the mismatch for several encoders, but finer granularity is not universally preferable.Source: §3.3 · p. 7, Table 4 · p. 7
• Full shared covariance is best for all five encoders on Enzyme Reaction, whereas per-class covariance is best for four of five on EC Number.GearNet is the exception: full shared covariance improves accuracy from 37.1% to 47.2% on Enzyme Reaction and from 58.5% to 72.4% on EC Number.Source: §3.3 · p. 7, Table 5 · p. 7

4 Discussion and Conclusion

• Across five encoders and two dataset scales, no CIL method is consistently reliable, and the same strategy can shift from near-best to poor performance when the encoder changes.The reported diagnostics identify weak gradient-based stability signals, heterogeneous embedding geometry, and covariance capacity that varies with dataset scale and encoder.Source: §4 · p. 8
• Changing phase construction alters absolute performance only modestly, while encoder-dependent patterns persist, indicating representation-dependent rather than phase-driven failures.RanPAC is competitive on several encoders but deteriorates sharply on GearNet; FeCAM preserves old classes but loses plasticity under its restrictive shared-covariance assumption.Source: §4 · p. 8
• Full shared covariance performs best across all five encoders on Enzyme Reaction, while per-class covariance performs best for four of five encoders on EC Number; GearNet favors shared covariance there.Shared covariance is more stable but can be too coarse; per-class covariance is more flexible but estimates rely on fewer samples. The authors propose adapting capacity between these extremes.Source: §4 · p. 8
• The authors recommend encoder-aware continual learning that adapts statistical capacity to representation geometry rather than applying vision-derived recipes unchanged.They identify covariance granularity, projection dimension, distance metrics, regularization strength, and memory allocation as potential adaptation targets.Source: §4 · p. 8
• The conclusions are scoped to frozen protein representations and enzyme function classification under the EC hierarchy; adapted encoders and other protein tasks remain untested.The fixed-hyperparameter protocol evaluates plug-and-play transfer, not the best performance obtainable through encoder-specific tuning.Source: §4 · p. 8

Declaration of LLM Usage

• Language models were used to polish prose and improve clarity during preparation of the work.Source: Declaration of LLM Usage · p. 12
• Language models assisted with routine coding tasks, including debugging, result formatting, and figure generation.Source: Declaration of LLM Usage · p. 12
• The authors determined and verified the experimental design, analyses, results, and conclusions.Source: Declaration of LLM Usage · p. 12

Ethics Statement

• The work evaluates continual-learning methods on publicly available protein sequence and function datasets.Source: Ethics Statement · p. 12
• The study involves no human subjects, personally identifiable information, or newly collected biological samples.Source: Ethics Statement · p. 12
• The authors characterize the work as methodological and diagnostic and report no foreseeable harmful applications arising directly from it.Source: Ethics Statement · p. 12

Reproducibility Statement

• Appendix A documents dataset construction, preprocessing, phase protocols, and encoder details; Appendix B provides experimental settings and baseline configurations.Source: Reproducibility Statement · p. 12
• All protein encoders remain frozen, and experiments use consistent evaluation protocols across representations.Source: Reproducibility Statement · p. 12
• Results are averaged over three random seeds unless otherwise specified, and the datasets and pretrained encoders are publicly available and cited to their original sources.Source: Reproducibility Statement · p. 12

A.1 Datasets and Preprocessing

• Enzyme Reaction annotations are reduced to one label by argmax and its training and validation splits are merged; EC Number retains only the first annotation.AlphaFold v4 structures are used for EC Number, while Enzyme Reaction structures come from TorchDrug.Source: §A.1 · p. 12
• Samples without matched structures are excluded for GearNet-Edge and SaProt but retained for sequence-only encoders.SaProt requires per-residue 3Di tokens generated from PDB files with Foldseek.Source: §A.1 · p. 12

A.2 Encoders and Embedding Extraction

• Each frozen encoder is applied once per protein, and its cached embeddings are reused across all CIL methods.Pooling and structure handling vary by encoder; GearNet-Edge embeddings are L2-normalized, and overlength sequences are truncated.Source: §A.2 · p. 12, Table 7 · p. 13

A.3 Baselines and Hyperparameters

• All MLP-based methods use a two-layer, 1024-hidden-unit classifier trained with Adam at 10^-3 for 100 epochs per phase.Replay, iCaRL, DER++, and RER use a 2,000-sample reservoir; DER++ sets α=0.5 and β=0.5, and EWC uses λ=2000.Source: §A.3 · p. 13
• FeCAM uses global shared covariance with ridge regularization (λ=10^-4), while RanPAC uses Gaussian projection D=12800 and ridge λ=1.0.FeTrIL translates new-class embeddings by old/new class-mean differences to generate old-class pseudo-features; EASE trains a rank-256 adapter per phase.Source: §A.3 · p. 13
• Hyperparameters are fixed across encoder–dataset configurations, with no setting-specific tuning.The method configurations are listed in Table 8.Source: §A.3 · p. 13, Table 8 · p. 13

B.1 Per-Encoder Breakdown

• Across encoders, alternative phase constructions produce modest within-encoder variation, while substantial encoder differences persist.On EC Number, replay- and regularization-based methods generally retain high plasticity but incur substantial forgetting; FeCAM and EASE have lower forgetting with reduced plasticity.Source: §B.1 · p. 14, Table 9 · p. 14, Table 10 · p. 15

B.2 Cross-family confusion patterns.

• Transferases and hydrolases are the dominant cross-family errors on both datasets, but confusion magnitude varies across encoders and datasets.This variation supports the finding that shared-covariance effectiveness depends on representation geometry; Table 11 reports frequent cross-family misclassifications and representative EC-level confusions.Source: §B.2 · p. 16, Table 11 · p. 16
• Figures 4 and 5 show per-encoder average incremental accuracy, forgetting, and final accuracy by reaction type or EC class.Figure 4 covers Enzyme Reaction and Figure 5 covers EC Number.Source: Figure 4 · p. 17, Figure 5 · p. 18
• Figure 6 compares phase-wise accuracy matrices for FeTrIL, RanPAC, and FeCAM across encoder–dataset configurations.Rows are training phases and columns are evaluated phases.Source: Figure 6 · p. 19