PaperlayerView exact revision on arXiv

Source-linked AI summary

dIon: Fragmentation-Based Invariance for Self-Supervised Learning of Tandem Mass Spectra

Alfred Nilsson, Joel Lapin, Samuel H. Payne, Mathias Wilhelm, Lukas Käll

arXiv:2610.06282v1cs.LG

TL;DR

Peptide MS/MS spectra are incomplete and variable, and only a minority have peptide labels. dIon uses precursor–fragment physics in two latent-prediction tasks to learn representations without sequence labels. Its initialization improves de novo peptide precision over training from scratch and supports strong peptide-similarity representations.

  • Problem

    Peptide MS/MS spectra are incomplete and variable, while only a minority of spectra carry peptide labels for supervised training.

    Source: §1 · p. 2

  • Method

    dIon adapts DINO with a precursor-conditioned mixture task and a precursor-withheld partial-view task that predict clean teacher representations without peptide sequence labels.

    Source: §1 · p. 3, §3.3 · p. 5

  • Results

    dIon initialization improves de novo peptide precision over training from scratch by 2.3–8.4 percentage points across both test sets and both supervised training corpora, and surpasses fully supervised state-of-the-art models on Kingdoms.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    Without peptide labels, dIon learns strong native peptide-similarity geometry compared with other learned models; with limited peptide-supervised adaptation, it achieves the best retrieval and pair-discrimination performance across representation benchmarks.

    Source: §1 · p. 3

    Evaluation against search-engine-assigned peptide labels inherits their errors and selection bias, and pretraining at this scale is costly.

    Source: §5 · p. 10

Abstract

from arXiv · show

We introduce a novel invariance for peptide tandem mass spectrometry data, unlocking self-supervised representation learning that improves de novo sequencing of peptides. This invariance exploits the physical relationship between precursor properties (mass and charge) and fragment-ion evidence, without requiring peptide sequence labels. We introduce dIon, which adapts the DINO framework with two latent prediction tasks, both recovering a clean teacher representation: one from a spectrum mixture, using the precursor as a selection query, and one from a partial spectrum with the precursor withheld. The first associates precursor information with fragment-ion evidence; the second prevents representational collapse onto that information alone. Mechanistic probes support both effects, and ablations show that the full objective performs best. Under identical end-to-end training, dIon initialization improves de novo peptide precision over training from scratch by 5.5 and 8.4 percentage points on the held-out MassIVE-KB and Kingdoms test sets, and by 2.3 and 4.8 percentage points with a larger supervised training corpus. The resulting models surpass fully supervised state-of-the-art de novo sequencing models on the diverse, multi-species Kingdoms corpus under the same greedy-decoding protocol. Without peptide labels, dIon learns strong native peptide-similarity geometry compared with other learned models; with limited peptide-supervised adaptation, it achieves the best retrieval and pair-discrimination performance across all representation benchmarks.

1 INTRODUCTION

Peptide MS/MS spectra are precise but incomplete and variable, and only a minority carry peptide labels. dIon adapts DINO with fragmentation-based views that use precursor–fragment physics to learn without sequence labels.

1 INTRODUCTION

  • View construction defines the invariance learned by view-based self-supervision, so image-based crops do not directly transfer to unordered peptide spectra.DINO matches student and EMA-teacher representations across augmented image views, while the self-distillation machinery itself is modality agnostic.Source: §1 · p. 2
  • Only a minority of public MS/MS spectra carry peptide labels, leaving supervised training to use a noisy, non-randomly selected subset while self-supervision can use every spectrum.Informative fragments may be missing, noise peaks may be present, and fragment intensities can vary substantially between measurements.Source: §1 · p. 2
  • dIon introduces fragmentation-based invariance: it uses precursor properties and fragment-ion evidence without sequence labels while preserving sensitivity to peptide-distinguishing fragmentation patterns.The method adapts DINO with two complementary latent-prediction tasks.Source: §1 · p. 3
  • The dual construction combines a precursor-conditioned mixture task with partial views that withhold precursor information, and dIon initialization improves de novo precision over training from scratch by 2.3–8.4 percentage points.The contribution passage also reports that dIon surpasses fully supervised Casanovo and InstaNovo across all 70 Kingdoms test-set species.Source: §1 · p. 3

2 RELATED WORK

Prior work spans view-based self-supervision, mixture training, and peptide-MS/MS representation learning, but the paper distinguishes dIon through its unblended teacher target and precursor-conditioned tasks.

2 RELATED WORK

  • Self-supervised methods include contrastive view learning and non-contrastive teacher-target prediction, with local objectives such as iBOT and DINOv2 also studied for dense tasks.Examples include SimCLR-style methods and data2vec, I-JEPA, and masked Siamese Networks.Source: §2 · p. 3
  • Unlike mixing methods whose targets reflect both sources, dIon uses conditioning information from one source to select its clean target and withholds that information in partial views.The related work cites Lee et al. (2021), Shen et al. (2022), and Alex et al. (2025) for mixing-based methods.Source: §2 · p. 3
  • Peptide-MS/MS self-supervision has included masked-peak or masked-view methods, while prior peptide representations largely used peptide supervision or identified peptide-spectrum matches.InstaNovo-FM reports faster convergence but no measurable de novo advantage over training the same architecture directly; dIon instead predicts teacher targets and improves sequencing over matched training from scratch.Source: §2 · p. 3
  • Modern de novo sequencers encode spectra into contextualized peak representations for decoder attention, the interface dIon’s dense representations must support.Casanovo established this formulation; later examples include InstaNovo, InstaNovo+, and Pairwise Attention.Source: §2 · p. 4

3 METHODS

dIon encodes peak and precursor information, then applies DINO-style self-distillation to views derived from fragmentation physics. Its dual latent-prediction objective trains pooled and dense representations for downstream transfer.

3.1 SPECTRUM REPRESENTATION AND ENCODER

  • The spectrum encoder represents an unordered set of fragment-ion observations alongside precursor mass and charge, retaining up to 200 intense peaks with m/z ≤2500.Peak tokens encode m/z and intensity, while a precursor token combines mass and charge information; the 15-block Transformer returns contextualized token states pooled into a global representation.Source: §3.1 · p. 4

3.2 SELF-DISTILLATION

  • DINO-style self-distillation trains a student’s assignment from one view to match an EMA teacher’s assignment from another, using Sinkhorn–Knopp-balanced teacher outputs.The teacher assignments are balanced across the batch, and the student is trained with cross-entropy between teacher and student assignments.Source: §3.2 · p. 4

3.3 FRAGMENTATION-BASED INVARIANCE

  • Views exploit the physical link between precursor properties and fragment evidence: clean views retain 95% of anchor peaks, mixtures add a competing spectrum, and partial views retain 60% while replacing the precursor with a null embedding.The anchor precursor selects which peptide’s representation the mixture student should recover; withholding it in partial views prevents relying on precursor information alone.Source: §3.3 · p. 5

3.4 DUAL LATENT-PREDICTION OBJECTIVE

  • The mixture student recovers the clean teacher target associated with its precursor, while each precursor-free partial view predicts every clean target; the two tasks receive equal weight.The objective is cross-entropy between teacher and student assignments and is applied to the pooled representation, thereby also updating dense peak-token representations without a separate dense-token loss.Source: §3.4 · pp. 5–6

3.5 DOWNSTREAM TRANSFER

  • De novo sequencing uses dense peak-token representations as decoder memory, whereas spectrum-level tasks and peptide-supervised metric learning use the pooled global representation.The de novo decoder is a freshly initialized autoregressive Transformer trained jointly with the encoder; metric learning uses a freshly initialized projection head.Source: §3.5 · p. 6

4 EXPERIMENTS AND RESULTS

dIon initialization improves de novo sequencing across the reported benchmarks, and limited-label adaptation yields strong peptide-representation results. Mechanistic probes and objective ablations support the dual objective’s intended behaviors and its advantage over single-task controls.

4.1 PRETRAINING AND DEVELOPMENT PROTOCOL

  • dIon is pretrained without peptide labels on 9,234,199 bacterial spectra, with a run-disjoint development subset of 467,723 spectra.The 127M-parameter encoder is trained for 284 epochs, and test splits are reserved for final evaluation.Source: §4.1 · p. 6

4.2 DE NOVO SEQUENCE TRANSFER

  • On Kingdoms, DNL-v1 dIon exceeds InstaNovo and Casanovo across all 70 species; dIon initialization also beats scratch on MSKB-final.With a 1,000-peak cap, DNL-v1 reaches 0.607 on Kingdoms and 0.856 on MSKB-final.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3 PEPTIDE-REPRESENTATION GEOMETRY

  • With 10% peptide labels, dIon initialization is best on every benchmark and metric, and with 1% it exceeds GLEAMS on Bacterial and Kingdoms.Frozen unlabeled dIon exceeds InstaNovo-FM on every benchmark and metric and Casanovo 4.0 on four of six; GLEAMS scores highest among those models.Source: §4.3 · pp. 7–8, Table 2 · p. 8

4.4 MECHANISTIC PROBES

  • Anchor-precursor conditioning selects the anchor almost always; the competing precursor selects its own source in the great majority of trials, though less often.The competing precursor is capped at the anchor’s peak count and is never itself a target.Source: Table 3 · p. 8, §4.4 · p. 8

4.5 OBJECTIVE ABLATIONS

  • The dual objective leads frozen-encoder validation peptide precision at 0.675, ahead of pure mixture (0.638), mixture-free (0.610), and random initialization (0.582).The pure mixture objective has the highest broad retrieval mAP but weakest strict pair discrimination, while mixture-free avoids precursor-information collapse and the dual objective pulls ahead on strict pairs later in pretraining.Source: §4.5 · pp. 8–9, Figure 2 · p. 9

5 DISCUSSION

dIon’s dual-view objective addresses precursor shortcuts while preserving fragment-pattern information, and its representations support strong similarity and downstream-task results. Evaluation remains tied mainly to search-engine-assigned peptide labels.

5 DISCUSSION

  • The precursor-withheld partial-view branch denies a shortcut that could otherwise let the model rely on conditioning information instead of fragment evidence.Neither branch alone reproduces the full result (Figure 2).Source: §5 · p. 9
  • Although the objective acts on pooled global representations, learned-query pooling and partial views leave dense peak-token representations informative for decoder adaptation.Gram anchoring to a 30% partial-view checkpoint improves frozen, but not fine-tuned, de novo transfer.Source: §5 · p. 9
  • With 10% of labels, dIon initialization outperforms every method on every benchmark and metric; frozen binned spectral angle beats learned representations not trained for peptide similarity, including Casanovo.The frozen comparison includes the peptide-supervised Casanovo encoder.Source: §5 · p. 9
  • Fine-tuning from dIon exceeds scratch training on all three auxiliary tasks, but frozen representations trail Casanovo and InstaNovo-FM on those tasks and are only negligibly behind on spectrum quality.Against other evaluated representations, fine-tuning also exceeds them on spectrum quality and methionine oxidation.Source: §5 · pp. 9–10
  • Most evaluations score predictions against search-engine-assigned peptide labels, inheriting their errors and selection bias; yeast instead checks proteome occurrence.Only pretraining is free of peptide labels.Source: §5 · p. 10
  • Label-free pretraining can use acquired spectra that search engines leave unassigned, including the majority of spectra.The paper notes that pretraining at this scale is costly.Source: §5 · p. 10

6 CONCLUSION

dIon transfers self-distillation to peptide tandem mass spectra through a measurement-derived relation, with improvements over scratch training and strong representation results. The closing material also documents AI use, data provenance, corpus construction, and reproducibility resources.

6 CONCLUSION

  • 2.3–8.4 percentage points: under identical end-to-end training, dIon initialization improves de novo precision across both test sets and supervised training corpora, and surpasses supervised state of the art on Kingdoms.Without labels, dIon learns strong native peptide-similarity geometry; with 10% of labels, initialization outperforms every compared method in peptide-supervised metric learning.Source: §6 · p. 10

AI USE STATEMENT

  • The authors conceived the invariance and objective; AI tools assisted auxiliary-task design, code, writing, and some related-work searches, with author review and no AI-produced results or AI authorship.The authors state that every submitted sentence was read and approved.Source: AI USE STATEMENT · p. 11

ETHICS STATEMENT

  • The work uses only publicly released spectra from prior studies, adds no new samples, and makes no attempt to recover donor identity; the authors report no misuse pathway specific to it.Pretraining used 64 accelerators for approximately three days, a cost the authors say is amortized over transfer experiments and later reuse.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • Code, pretrained encoder, and fine-tuned de novo checkpoints are available; appendices document training, datasets, evaluation, and external-model handling.The passage provides repository links for the code and model artifacts.Source: REPRODUCIBILITY STATEMENT · p. 11

A.1 OVERVIEW OF DATASETS AND ROLES

  • Table 4 lists each dataset’s role, split sizes, and split criterion.Source: §A.1 · p. 16

A.2 SSL PRETRAINING AND DEVELOPMENT CORPUS

  • 9,234,199 charge-1–10 training spectra: dIon pretraining uses the PXD010000 bacterial corpus, while peptide sequences, identities, PSM assignments, and fragment annotations are withheld from the objective.A held-out 467,723-spectrum subset is used for development; the corpus covers 51 bacterial organisms.Source: §A.2 · p. 16

A.3 NATIVE-GEOMETRY EVALUATION DATASETS

This section defines the evaluation corpora and the dIon pretraining setup, from spectrum encoding and view construction through self-distillation and optimization.

A.3 NATIVE-GEOMETRY EVALUATION DATASETS

  • PXD010613, NineSpecies V2, and Kingdoms provide held-out retrieval populations with distinct species and split designs; comparisons also use charge- and mass-matched spectra.The Kingdoms retrieval population spans 70 species, and matching requires equal precursor charge and mass within 10 ppm.Source: §A.3 · pp. 16–17, Table 5 · p. 17

A.4 SUPERVISED METRIC-LEARNING DATA

  • Metric-learning label budgets are sampled by peptide identity, with all eligible spectra retained and evaluation conditions fixed across scratch and dIon initialization.The 1% label subset is nested within the 10% subset.Source: §A.4 · p. 18

A.5 DE NOVO DATASETS AND SPLIT CONSTRUCTION

  • MSKB-final and DNL-v1 provide supervised de novo corpora, while Casanovo 4.0 serves as a separate representation reference.DNL-v1 contains 5,791,021 labeled spectra from eight sources and no Kingdoms data.Source: §A.5 · pp. 18–19

B.1 MODEL AND SSL CONFIGURATION

  • The dIon encoder is a 15-block, approximately 127.3M-parameter Transformer that exports contextualized peak tokens and a pooled global representation.The global representation uses learned-query cross-attention over non-padding tokens.Source: §B.1 · p. 19, §B.1.2 · p. 20

B.1.1 PEAK PREPROCESSING

  • Peak preprocessing retains at most 200 intense peaks, while peak tokens combine sinusoidal m/z and normalized-intensity encodings with precursor mass and charge information.The m/z components form a 1024-dimensional encoding, combined with a separate 256-dimensional intensity encoding before projection.Source: §B.1.1 · pp. 19–20, §B.1.2 · p. 20

B.1.3 VIEW CONSTRUCTION

  • Each anchor yields clean, precursor-conditioned mixture, and precursor-withheld partial views; competing peaks are size-matched to the anchor crop.At mixing strength 1.0, the competing contribution uses one peak per anchor-crop peak when enough peaks are available.Source: §B.1.3 · p. 20, Table 8 · p. 21

B.2.1 PROTOTYPE ASSIGNMENTS AND SELF-DISTILLATION TARGETS

  • The student predicts temperature-scaled prototype assignments, while batch-balanced, detached teacher targets use Sinkhorn–Knopp normalization and an exponential-moving-average teacher.The selected objective uses neither token masking nor the iBOT and KoLeo losses.Source: §B.2.1 · pp. 21–22

B.2.2 OPTIMIZATION

  • Pretraining uses a global batch of 8,192 spectra, with a 40,000-step warmup followed by 450,000 steps of cosine learning-rate decay.The canonical run uses 64 GH200 accelerators.Source: §B.2.2 · p. 22, Table 9 · p. 22

B.3 COMPUTE AND TRAINING EFFICIENCY

The canonical pretraining run used 64 GH200 accelerators for 284 epochs. Checkpoint selection relied on validation probes, with de novo settings fixed in advance and test data reserved for final evaluation.

B.3 COMPUTE AND TRAINING EFFICIENCY

  • 284 pretraining epochs took about 75 hours, or approximately 4,800 accelerator-hours, on 64 GH200 96 GB accelerators.These figures describe the canonical run; ablations used shorter horizons and separate configurations.Source: §B.3 · p. 22

B.4 MODEL SELECTION

  • The epoch-284 teacher checkpoint was selected by inspecting online validation probes, while temperature and momentum schedules were chosen using collapse diagnostics.De novo checkpoints were selected by maximum validation peptide precision.Source: §B.4 · p. 22
  • De novo training and decoding settings, including greedy decoding, were fixed in advance, and test splits were used only for final evaluation.The settings were not tuned in this work.Source: §B.4 · pp. 22–23

C.1 OBJECTIVE ABLATIONS

The dual objective performs best among the tested ablations and leads both development probes during the second half of pretraining. Mechanistic tests support precursor-conditioned source selection, while the diagnostic scope of the broader geometry measures is explicitly limited.

C.1 OBJECTIVE ABLATIONS

  • The dual objective combines precursor-conditioned mixture views with precursor-withheld partial views, whereas the controls omit one of those ingredients or the distractor peaks.The ablations use matched approximately 100-epoch runs and a shared compact validation probe suite.Source: §C.1 · p. 23
  • End-to-end fine-tuning preserves the ablation ordering, every objective improves on scratch, and the dual objective leads on fragment-sensitive measures.The dual objective also leads both probes in the second half of pretraining; Table 10 reports de novo transfer on MSKB-final validation.Source: §C.1 · pp. 23–24, Table 10 · p. 24

C.2 MECHANISTIC VALIDATION OF THE DUAL LATENT-PREDICTION VIEWS

  • On measured co-isolated spectra, conditioning on each true precursor mass selected its corresponding peptide in the illustrated case; across 45 pairs, margins changed sign in six.Three of those six pairs relied on a single reference scan for one peptide, and none of the sign-changing pairs was closer than 0.2 Da.Source: §C.2 · p. 25, Figure 5 · p. 26

C.3 COMPLETE REPRESENTATION-GEOMETRY DIAGNOSTICS

  • Unfiltered global retrieval and static-pair average precision are reported as diagnostics, since global retrieval can exploit precursor-mass and charge shortcuts.The complete native-geometry table covers all precursor charges and matches comparisons by charge and precursor mass.Source: §C.3 · p. 26, Table 13 · p. 26

D.1 AUXILIARY SPECTRUM-LEVEL TASK DEFINITIONS

The section describes spectrum-level binary tasks, supervised peptide metric learning, and the decoder and training setup used for de novo sequencing.

D.1 AUXILIARY SPECTRUM-LEVEL TASK DEFINITIONS

  • Spectrum-quality, chimericity, and oxidized-methionine tasks use a shared binary prediction head on the pooled global representation.Table 20 reports held-out AUROC for frozen representations and end-to-end adaptation as separate regimes.Source: §D.1 · p. 26, Table 20 · p. 28

D.2 SUPERVISED PEPTIDE METRIC LEARNING

  • Supervised metric learning trains unit-norm 128-dimensional embeddings with a supervised contrastive objective, using same-peptide examples as positives and other peptides as negatives.The loss is averaged over anchors with at least one positive.Source: §D.2 · p. 28

E.1 DE NOVO ENCODER–DECODER ARCHITECTURE

  • The freshly initialized Casanovo-style decoder attends causally over peptide prefixes and cross-attends to the encoder’s peak-token memory.Its target vocabulary contains 45 tokens, including 43 residue and N-terminal-modification tokens.Source: §E.1 · p. 29, Table 21 · p. 29

E.2 AUTOREGRESSIVE TRAINING AND FINE-TUNING

  • Supervised de novo training minimizes teacher-forced categorical cross-entropy over peptide tokens, using settings held fixed across scratch and dIon-initialized conditions.The objective uses no label smoothing.Source: §E.2 · p. 29, Table 22 · p. 29

E.3 UNLABELED YEAST EVALUATION

The unlabeled yeast benchmark evaluates three models on the same 39,460 scans using greedy decoding, then checks predictions against the yeast proteome and decoy references.

E.3 UNLABELED YEAST EVALUATION

  • All three models receive the same 39,460 unlabeled yeast spectra and decode greedily with a single beam.No ground-truth peptide enters the pipeline.Source: §E.3 · p. 29
  • Predictions are matched as contiguous substrings of the yeast proteome after stripping modifications and treating isoleucine and leucine as identical.The same matching procedure is applied to reversed-protein and composition-preserving shuffled decoys to bound matches attainable without real signal.Source: §E.3 · pp. 29–30
  • Casanovo declines are retained as incorrect, and match fractions are reported over both emitted predictions and all submitted spectra.Table 23 also reports coverage, tryptic-restricted matches, decoy matches, and modified calls.Source: §E.3 · p. 30, Table 23 · p. 30

E.4 KINGDOMS EVALUATION DETAILS

These evaluation details cover Kingdoms and MSKB-final comparisons, an oracle-precursor DIA pilot, shared sequence-scoring procedures, and diagnostic probes and refinement.

E.4 KINGDOMS EVALUATION DETAILS

  • The Kingdoms fast-evaluation mode repeats eight Akkermansia muciniphila records, but the maximum precision effect is 0.03 percentage points for that species.The repeated records bound the overall effect at 1.6 × 10^-4 percentage points; scores are reported unchanged.Source: §E.4 · pp. 30–31
  • With a 1,000-peak cap, dIon initialization gains in both Kingdoms training regimes, while scratch gains with MSKB-final but not DNL-v1.dIon initialization also has the highest full-coverage peptide precision of four models on every species, with the margin rounding to zero only for Rattus norvegicus.Source: §E.4 · pp. 30–31, Table 24 · p. 31, Table 25 · p. 32

E.5 MSKB-FINAL EVALUATION DETAILS

  • The complete MSKB-final results include amino-acid metrics, 1,000-peak-cap dIon models, and InstaNovo results omitted from the main text.Scratch and dIon initialization use identical end-to-end training protocols within each training regime.Source: §E.5 · p. 31, Table 26 · p. 33

E.6 ORACLE-PRECURSOR DECODING OF DIA SPECTRA

  • The DIA pilot tests oracle-guided recovery of target peptides from real wide-window mixtures using a frozen Gram-refined encoder and a decoder trained only on synthetic DDA mixtures.The reported precision is low and no baseline was evaluated; the proposed decisive test uses observed MS1 precursor features and narrower isolation windows.Source: §E.6 · pp. 31, 33, Table 27 · p. 33

E.7 DECODING AND SEQUENCE EVALUATION

  • All models use reverse-order, single-beam decoding under a precursor-mass constraint and one sequence-matching implementation.Residues must match within 0.1 Da and cumulative mass within 0.5 Da; peptide precision requires every aligned position to match, and confidence ranks predictions only within each model.Source: Table 28 · p. 33, §E.7 · pp. 33–34

E.8 ONLINE MINI-DE-NOVO PROBE

  • Retaining 30% rather than 60% of peaks strengthens the mini-de-novo probe but lowers strict pair ROC-AUC; the probe is a development diagnostic, not the de novo evaluation.Its mini-de-novo performance peaks mid-training and then declines.Source: §E.8 · p. 34

E.9 GRAM-ANCHORED REFINEMENT

  • Gram anchoring preserves pairwise peak-token similarity patterns, improving frozen de novo transfer but not end-to-end fine-tuning, while lowering strict pair ROC-AUC.The refined encoder was fine-tuned end to end only on DNL-v1; the MSKB-final entry is therefore empty, and the main results use the unrefined checkpoint.Source: §E.9 · p. 35

F ADDITIONAL TRAINING DIAGNOSTICS

Training diagnostics track collapse beyond the loss alone and compare temperature and momentum schedules; a masked-token screen underperformed the canonical run.

F.1 COLLAPSE MONITORING

  • Collapse monitoring tracks teacher entropy, teacher–student divergence, and the largest across-batch variance of student output probabilities.A collapsed model drives divergence and variance to zero, while entropy alone can remain moderate.Source: §F.1 · p. 36, Figure 9 · p. 36
  • A 10-epoch warmup to teacher temperature 0.07 with momentum rising from 0.998 to 1.0 kept all three diagnostics away from degenerate values.With the canonical schedules and Sinkhorn–Knopp balancing, the 100-epoch dual-objective ablation run's teacher entropy stayed between collapsed extremes.Source: §F.1 · p. 36, Figure 9 · p. 36

F.2 MASKED-TOKEN OBJECTIVE SCREEN

  • Adding an iBOT-style masked-token loss produced lower strict pair ROC-AUC and mini-de-novo precision than the canonical run at matched epochs.The screen stopped at epoch 85 of a planned 300, and the loss was not carried forward.Source: §F.2 · p. 37, Figure 10 · p. 37
Loading 2610.06282v1…