Source-linked AI summary
When Edit Flows are Edit Jumps: replicating Edit Flows and EvoFlows
Gabriel Bénédict, Melanie Buechler, Gerard Riera-Solà, Chloé de Ancos, Yves Gaetan Nana Teukam, Moritz Freidank
TL;DR
Antibody lead optimization needs controlled edits that balance novelty and naturalness, but existing Edit Flows and EvoFlows lacked code and complete training specifications. The paper formalizes them as pure-jump processes and releases EditJumps, an open implementation with a generalist antibody editor. Its replication analysis identifies clock normalization as governing realized edit counts and finds evaluation rankings sensitive to reference sample size.
Problem
Lead optimization needs models that choose a small number of edits while balancing novelty and naturalness, but Edit Flows and EvoFlows lacked released code and complete training specifications.
Source: §1 · pp. 1–2Method
Results
Changing the reference alignment from 200 to 800 homologs alters the normalized spectrum MMD ratio by 50% on Ty1 and 26% on HER2VH, enough to invert method rankings.
Source: §5 · p. 7Takeaways & Limitations
Clock normalization governs realized edit scale, and standardized reference sets are needed because internal normalization cannot prevent sample-size effects from reversing rankings.
Source: §5 · p. 7, §7 · p. 8Clock normalization has no closed-form mapping to realized edits, so achieving a specific edit budget requires empirical calibration.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Antibody lead optimization calls for a small, bounded set of edits to an existing candidate: substitutions, but also insertions and deletions. Edit-based generative models are the only ones that allocate such an edit budget without fixing the edit positions, the edit count, or the output length in advance. However, the existing approaches Edit Flows and EvoFlows did not release code or complete training specifications. Here, we show that both methods follow the same underlying process -- edits firing one at a time, at learned rates, in continuous time -- the pure-jump case of generator matching over finite sequences. With EditJumps we introduce the first open implementation of this framework, with a single generalist antibody editor trained on 1.66M Observed Antibody Space homolog pairs to propose homolog-like variants of a seed sequence, editing unseen leads zero-shot, without the per-family retraining original approaches require. Replicating this system from scratch exposes why open code is essential for generative biology: reconciling published edit distributions required reverse-engineering an undocumented rate-scaling hyperparameter that dictates realized mutation counts. Moreover, we show that published evaluation metrics are highly sensitive to reference sample size, frequently flipping method rankings. We release our full codebase, automated test suite, and configurations at: https://github.com/VisiumCH/editjumps
1 Introduction
Antibody lead optimization needs models that choose a small number of edits while balancing novelty with naturalness and allowing sequence length to change. This paper formalizes Edit Flows and EvoFlows as pure-jump processes, implements them in EditJumps, and reports that an undocumented clock normalization governs realized edit counts.
1 Introduction
- Lead optimization seeks small controlled edits because mutations interact non-linearly, making useful changes difficult to predict or enumerate by inspection.The model must choose edit positions and types while allowing sequence length to vary as in natural homologs.Source: §1 · p. 1
- Existing approaches leave important parts of this task or its reproducibility unresolved: common architectures constrain positions, edit counts, or length, while Edit Flows and EvoFlows lacked released code and complete training specifications.Edit-based NLP models place edits without fixing positions but use fixed refinement procedures without edit rates.Source: §1 · pp. 1–2
- At matched budgets of 4.1–4.6 edits per sequence, EditJumps produces greater sequence diversity than evotuning and EvoDiff-MSA with comparable distributional fidelity.It matches unnormalized MMD with standard evotuning on Ty1 and falls within 3% on HER2-VH.Source: Figure 1 · p. 2, §1 · p. 2
- Edit Flows and EvoFlows are pure-jump instances of Generator Matching, requiring non-explosion safeguards and length-dependent clock normalization for variable-length sequences.Edit Flows uses a hard length cap, whereas EvoFlows leaves the sequence space unconstrained; its clock normalization is unspecified in the publication.Source: §1 · pp. 2–3
- Reproducing published edit statistics required identifying clock normalization as the parameter governing realized edit counts, while six of ten EvoFlows metrics cannot be deterministically computed from the text alone.The spectrum MMD estimator is also sensitive to reference-set size.Source: §1 · pp. 2–3
- EditJumps provides an open implementation of both methods and trains one generalist editor on 1.66M paired homologs to edit held-out targets without per-target retraining.The benchmark targets are single-domain constructs, unlike the paired heavy-light sequences used for pretraining.Source: §1 · p. 3
2 Background
The background frames Edit Flows and EvoFlows as continuous-time processes whose sequences change through discrete, rate-governed edits. It then specifies a jump-process formulation, structured insertion, deletion, and substitution operations, and conditions for well-defined variable-length dynamics.
2 Background
- The paper treats sequence editing by Edit Flows and EvoFlows as the pure-jump case of Generator Matching, where discrete sequences change through instantaneous events rather than continuous flows.The process is defined on a countable space of finite sequences and is constant between jumps.Source: §2 · p. 3, §2.1 · p. 3
2.1 Sequence evolution as a jump process
- The process stays in a sequence between jumps, with transition rates governing when it changes and a jump kernel governing the destination sequence.The neural network estimates marginal transition rates by minimizing a Bregman divergence whose minimizer is the conditional expectation of the corresponding rate.Source: §2.1 · pp. 3–4
2.2 Sequence edits as structured jumps
- Structured jumps represent edits as single-token insertions, deletions, or substitutions at specified positions, avoiding direct parameterization over the intractable sequence space.Insertion and substitution rates can be factored into position-specific edit rates and conditional distributions over the resulting token; deletion needs no token distribution.Source: §2.2 · p. 4
2.3 Non-Explosion and Clock Normalization
- Variable-length dynamics require non-explosion safeguards because insertion opportunities make total transition rates grow with sequence length.Edit Flows limits sequence length with a hard cap, whereas EvoFlows leaves the sequence space nominally unbounded.Source: §2.3 · pp. 4–5
- Length-dependent clock normalization is needed to prevent longer sequences from accumulating proportionally more edits per unit time under identical per-position rates.The published EvoFlows paper notes normalization but omits its form and value; empirical calibration is needed because it governs realized edit scale.Source: §2.3 · p. 5
3 Replication Setup
EditJumps uses a single generalist editor trained across OAS, while the replication reconstructs EvoFlows’ training and evaluation setup despite missing original code and complete specifications.
3 Replication Setup
- The authors train one generalist antibody editor across OAS to edit novel leads zero-shot, avoiding per-family retraining.The original Edit Flows and EvoFlows releases lacked reference code or complete training-data specifications.Source: §3 · p. 5
- Training pairs come from clustered paired heavy- and light-chain OAS sequences; evaluation is limited to Ty1 and HER2-VH, with 20 templates and 200 held-out homologs each.Non-antibody targets lack OAS homologs, and anti-EphA2 could not be recovered from public repositories.Source: §3 · p. 5
- The editor uses an unfrozen pretrained ESM-2 encoder and time-conditioned rate and token heads; its 35M-parameter trunk matches the 650M trunk’s realized edit scale and distributional fidelity.The model is fine-tuned for 20,000 steps; the reported edit scales are 0.180 versus 0.182.Source: §3 · p. 5
- The replication follows Equation 23 for aligned gaps, while identifying a reference-code indexing bug that supervises adjacent-residue deletions across 49.2% of training pairs.Other departures include linear rate heads, a linear time schedule, frozen holding-interval rates, and a 0.7 seed-identity threshold for homolog retrieval.Source: §3 · p. 5
- Table 1 reports six reimplemented EvoFlows baselines by family, with each family’s rows compared against its own random-pairing floor and random-mutation ceiling.The Ty1 and HER2 blocks are not comparable to one another.Source: Table 1 · p. 6
- Evotuning is a fixed-length, substitution-only baseline that requires a separately retrained 650M trunk for each target family.The forced variant prevents the MLM from restoring masked residues, separating generative editing from non-intervention.Source: §3 · p. 6
4 Results
EditJumps outperforms forced evotuning across distributional metrics, while results vary by family and do not reach the natural-homology floor. The replication also identifies clock-rate calibration and metric-specification challenges.
4 Results
- All learned models outperform uniform random mutation but remain above the natural-homology floor (0.98 MMD versus 0.60).EditJumps beats forced evotuning across distributional metrics and matches standard evotuning on k-mer MMD, despite higher composition divergence and about 4% more edits.Source: §4 · p. 6
- A clock rate of approximately 95 replicates the published edit ratio of 0.43 across both families and three seeds.The clock normalization hyperparameter guides the realized edit budget.Source: §4 · p. 6
- Six of EvoFlows’ ten evaluation metrics cannot be deterministically recomputed from the paper text alone.Four are well-specified, one with an unstated MMD kernel parameter; several others lack scalar reductions or admit incompatible reductions.Source: §4 · p. 6
5 Discussion and Limitations
The discussion cautions that sequence metrics and cross-study comparisons depend on their reference data and evaluation protocol. It also identifies limits in clock calibration, CDR fidelity, and the training-to-evaluation domain match.
5 Discussion and Limitations
- A clock rate around 80 minimizes spectrum MMD, corresponding to about 8.5 edits despite the evaluation scenario allowing about five.Figure 2 compares distance to held-out natural homologs across Ty1 and HER2-VH and three random seeds.Source: Figure 2 · p. 7
- Covariance recovery shifts from 83.6% to 85.1% when reference partitions change, while an exact sequence recovery rate of 0.21 is 300-fold above chance at matched edit budgets.The authors warn that ungrounded metrics risk rewarding sequence conservatism rather than genuine biophysical fitness.Source: §5 · p. 7
- Expanding the reference alignment from 200 to 800 homologs changes normalized spectrum-MMD ratios by 50% on Ty1 and 26% on HER2-VH, enough to invert method rankings.The authors conclude that internal normalization cannot replace standardized reference sets; homolog-pair enumeration itself replicated without discrepancy.Source: §5 · p. 7
- Clock normalization has no closed-form mapping to realized edits, and the model closes less than 25% of the random-to-homolog fidelity gap in CDR loops.Training on concatenated variable domains also creates domain shift when evaluating isolated heavy chains and nanobodies across two target families.Source: §5 · p. 7
6 Related Work
Related work spans discrete generative processes and sequence-editing models. Edit Flows and EvoFlows combine edit operations with flow-matching training, which this paper reframes as a pure-jump process.
6 Related Work
- Discrete diffusion and flow-matching approaches cited here operate on fixed-length sequences and change positions by substitution, without insertions, deletions, or length variation.Discrete diffusion reverses a fixed corruption process, whereas flow matching regresses directly onto a conditional probability path.Source: §6 · pp. 7–8
- Earlier edit-based models use iterative policies and fixed numbers of passes, without assigning time or rates to individual edits.The Insertion Transformer inserts tokens, the Levenshtein Transformer adds deletion, and DiffusER uses insertion and deletion in denoising.Source: §6 · p. 8
- Edit Flows [14] and EvoFlows [6] combine insertion, deletion, and substitution rates with conditional-path training in a continuous-time Markov chain over sequences.The rate scale sets the expected number of edits.Source: §6 · p. 8
- Generator matching [17] organizes diffusion, flow matching, and pure-jump processes by generator form; on token sequences, the combined construction is pure jump.It inherits flow matching’s training path, not its dynamics.Source: §6 · p. 8
- The editing formulation produces bounded residue-level edits to a given protein, unlike protein generative models that produce whole sequences or structures.Discrete Walk-Jump Sampling [9] requires fixed masks for infilling and cannot calibrate an edit budget.Source: §6 · p. 8
7 Conclusions
The paper formalizes Edit Flows and EvoFlows as pure-jump sequence-editing processes and presents EditJumps as their shared implementation. Its empirical investigation identifies clock-rate normalization as important to realized edit counts and finds jump models outperform some baselines while remaining below natural homology floors.
7 Conclusions
- Edit Flows and EvoFlows are formalized as pure-jump cases of Generator Matching, with EditJumps implementing their shared sequence-editing framework.The framework represents insertions, deletions, and substitutions as jumps with continuous holding times and operation-specific rates.Source: §7 · p. 8
- Continuous-time jump models outperform uniform random mutation and forced evotuning, but remain separated from natural homology floors.The authors also identify balancing sequence diversification with structural preservation as an ongoing challenge.Source: §7 · p. 8
- The authors recommend parameterizing clock rates or adaptive stopping, standardizing distributional metrics, and evaluating baselines in both unforced and forced regimes.These recommendations address manual rate calibration, sample-size artifacts in metrics such as MMD, and separating active design from wild-type conservation.Source: §7 · p. 8
AI Use Statement
The authors report using LLM assistance in code development, data analysis, and manuscript preparation, with direct author supervision. Implementations, workflows, metrics, and empirical claims were audited or independently validated by the authors.
AI Use Statement
- LLM assistance supported code development, data analysis, and manuscript preparation under direct author supervision.Source: AI Use Statement · p. 9
- LLM coding assistance contributed to training and evaluation pipelines, analysis scripts, and figure-generation routines; authors audited and executed the work.Source: AI Use Statement · p. 9
- LLM assistance helped draft text, format comparison tables, and cross-reference citations; authors independently validated empirical claims and data points against raw artifacts.The authors take full intellectual and editorial responsibility for the manuscript.Source: AI Use Statement · p. 9
A Extended results on all metrics
The appendix describes the evotuned baseline and its sampling protocol, and notes that the original EvoFlows methodology reports 10 metrics. The baseline uses family-specific adaptation and entropy-guided iterative infilling, but remains limited to substitutions at fixed length.
A Extended results on all metrics
- The appendix reports all 10 metrics from the original EvoFlows methodology and says it interprets them in the text and code.Source: A Extended results on all metrics · p. 12
- Evotuning adapts an ESM-2 t33 650M UR50D trunk separately for each family using its training partition.The stated setup uses 36,417 Ty1 sequences and 40,489 HER2-VH sequences for 2,000 steps, with batch size 16 and learning rate 5 × 10−5.Source: A Extended results on all metrics · p. 12
- The baseline samples edit positions using column-wise Shannon entropy from family alignments, exhausting positive-entropy positions before conserved zero-entropy columns.It masks a sampled budget of positions and iteratively infills them in random order at temperature T = 1.0.Source: A Extended results on all metrics · p. 12
- Budget matching tops up masks for as many as eight rounds to reach the target substitutions; the forced regime masks template residues from output logits to guarantee the budget in one pass.Source: A Extended results on all metrics · p. 12
- Evotuning is limited to substitutions and cannot model sequence-length variability or indels.Source: A Extended results on all metrics · p. 12
B Sampling Algorithms and Discretization Analysis
EditJumps samples trajectories over t = 0 to 1 using either Euler τ-leaping or exact Gillespie simulation. In the reported synthetic-rate-field test, Euler's over-firing is negligible at the 50-step setting used in experiments, while clock normalization determines the direction of discretization error.
B Sampling Algorithms and Discretization Analysis
- Euler τ-leaping evaluates rates at uniform steps and fires candidate edits independently with probability λt(e | xt)h; multiple non-commuting edits can occasionally occur in one step.The method is first-order accurate in h and computationally efficient.Source: B Sampling Algorithms and… · p. 13
- Exact Gillespie simulation samples event-to-event without a time grid, using an integrated-rate condition for the next jump and then sampling the transition.The implementation freezes rates over the holding interval, yielding a first-order continuous-time sampler.Source: B Sampling Algorithms and… · p. 13
- At N = 50 steps, Euler over-fires by 0.08 ± 0.21 edits, less than 0.3% relative error against the exact Gillespie mean of 71.29 ± 0.15 edits.The benchmark used 2,000 trajectories starting from sequences of length 120; this was the setting used in all experimental runs.Source: B Sampling Algorithms and… · p. 13
- Clock normalization makes Euler slightly over-fire as sequence length expands, whereas unnormalized rates make it under-fire by −4.03 ± 0.42 edits at N = 2.The authors conclude sampler discretization does not account for the observed empirical discrepancies.Source: B Sampling Algorithms and… · p. 13
C Model Architectures, Training Paths, and Algorithmic Specifications
This section specifies how aligned, variable-length sequence pairs define edit training paths and how the EvoFlows model parameterizes edit rates and token predictions. It also audits implementation choices, training data, and reproducibility details.
C Model Architectures, Training Paths, and Algorithmic Specifications
- Aligned source–target pairs lift sequences with gap tokens, making each aligned column an insertion, deletion, or substitution transition.The conditional path mixes source and target tokens with schedule κt; marginal rates are trained using cross-entropy and Bregman divergence objectives.Source: C Model Architectures, Training Paths,… · p. 13
- EvoFlows uses a pretrained, jointly fine-tuned ESM-2 trunk, time-conditioned through sinusoidal embeddings and shared FiLM modulation, with separate rate and token-prediction heads.The three position-specific rate heads use shallow MLPs; token heads can use MLPs or adapted ESM-2 language-model heads.Source: C Model Architectures, Training Paths,… · pp. 13–14
- Head parameterization shifts novelty by 0.52–1.24, exceeding the 0.15 shift from corpus choice; pretrained ESM-2 heads yield more conservative edits.On OAS, novelty is 0.45 vs. 0.61; on stock, 0.50 vs. 0.89, for pretrained versus lightweight heads.Source: C Model Architectures, Training Paths,… · p. 14, Figure 3 · p. 14
- The reproduction corrected deletion supervision that the reference code assigns to the preceding surviving residue, and used the linear schedule specified by the released code.Although linear and cubic schedules have equal total cross-entropy mass, expected active edit targets were 32.7 versus 50.1 across 400 homolog pairs.Source: C Model Architectures, Training Paths,… · p. 14
- Needleman–Wunsch alignment parameters are unspecified in the published method, leaving training labels partly determined by third-party library defaults.Substitution-matrix choices and gap opening and extension penalties directly determine intermediate corruption targets.Source: C Model Architectures, Training Paths,… · p. 14
D Dataset Curation, Training Protocol, and Verification
- OAS homolog clustering capped families at 20 pairs and produced 1,660,105 symmetric pairs; audits found second-sequence higher scores in 44.9% and 44.3% of sampled pairs.Clustering used 0.5 minimum identity and 0.8 coverage thresholds.Source: D Dataset Curation, Training Protocol,… · p. 15
- Training ran for 20,000 steps on one NVIDIA L4 GPU, but validation loss was still improving at the final checkpoint.The final four evaluations improved by 2.3 units over preceding checkpoints; released artifacts include code, weights, and evaluation pipelines under an MIT license.Source: D Dataset Curation, Training Protocol,… · p. 15
E Evaluation Methodology, Metric Audits, and Ceilings
The evaluation audits metric definitions, reference-set sensitivity, and alignment-frame choices that affect comparisons with published results. It also reports reproduction checks and antibody-specific biophysical outcomes.
E Evaluation Methodology, Metric Audits, and Ceilings
- Only four of ten published evaluation metrics are fully defined in the text; three are named without mathematical definitions, and two lack scalar reduction functions.The pooled-versus-per-template reduction is also ambiguous for one metric.Source: E Evaluation Methodology, Metric… · p. 15, Table 3 · p. 16
- Covariance scores for real homologs rise from 0.790 at N = 20 to 0.921 at N = 100 and 0.985 at N = 800, making raw scores across reference sizes non-comparable.An exact sequence recovery rate of 0.21 represents 300-fold enrichment over chance at matched edit budgets.Source: E Evaluation Methodology, Metric… · p. 16
- Disjoint reference sets shift spectrum MMD by −36.7% for Ty1 and −12.6% for HER2-VH, while covariance agreement increases by 12–17 points.The disjoint subset is more homogeneous, with Levenshtein distance 23.00 versus 24.90.Source: E Evaluation Methodology, Metric… · p. 16
- A mismatched alignment frame shifts MIP agreement ceilings by +2.3 percentage points, so ceiling comparisons require identical coordinate frames.The ceilings are computed on held-out natural homologs.Source: E Evaluation Methodology, Metric… · p. 16
- Scaling ESM-2 from 35M to 650M shifts the edit ratio from 0.180 to 0.182, within seed variance of ±0.01.Published Appendix A heads are more conservative than lightweight linear heads; training-rate choices also move the edit budget.Source: E Evaluation Methodology, Metric… · pp. 16–17
- The reproduced evaluation arms and baseline anchors fall entirely within the ranges spanned by the six published series across their six datasets.The published series ranges were recovered from Figure 3.Source: Figure 4 · p. 17
- EditJumps preserves canonical IMGT disulfides in 100% of numberable generations on both targets, versus 89.0% and 93.8% under random mutation.In CDRs, it closes 18.7% of the composition KL distance for Ty1 and 23.9% for HER2-VH between random mutation and natural homologs.Source: E Evaluation Methodology, Metric… · p. 17