EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models

• Prior non-autoregressive editing methods can reflect target attributes, but their editing-degree control is absent or limited to a narrow range.Source: §1 · p. 1
• Diffusion-based text editing remains underexplored despite embedding diffusion models' potential to adapt techniques from continuous diffusion.Source: §1 · p. 1
• EdiText combines SDEdit-based coarse editing with self-conditioning-based fine editing to adjust text at different scales.The integration is intended to provide precise adjustments within the broader range of coarse-level editing.Source: §1 · pp. 1–2
• Across various tasks, EdiText achieves superior editing performance to the baseline and offers a wider, more nuanced range of editing control.Source: §1 · p. 2

• Diffusion language models have shown controllability in constrained generation, while their potential for text editing remains underexplored.Source: §2 · p. 2
• Text editing modifies reference text toward user intent, unlike controlled generation, which produces text satisfying constraints.Source: §2 · p. 2
• Reference-based editing and controlled paraphrase approaches generally lack mechanisms for controlling the degree of editing.Source: §2 · p. 2
• EBM and ParaGuide methods adjust editing intensity, but their control remains limited to micro-level changes; EdiText integrates strategies for multiple scales.Source: §2 · pp. 2–3

3 EdiText

• EdiText combines latent diffusion with SDEdit-based coarse editing and self-conditioning-based fine editing to control text edits at different scales.The framework uses LD4LG as its embedding-diffusion backbone.Source: §3 · p. 3

3.1 Latent Diffusion for Language Generation

• LD4LG compresses discrete text into fixed-length continuous latents and predicts clean latents from noisy inputs, supporting variable-length decoded text.Its encoder-decoder backbone makes continuous-domain editing methods applicable to text.Source: §3.1 · pp. 3–4

3.2 SDEdit-Based Coarse-Level Editing

• SDEdit controls coarse editing by perturbing the reference to timestep tCE: larger timesteps permit greater changes, while smaller ones preserve more structure.EdiText adapts this approach to text, enabling edits from slight changes to substantial divergence from the reference.Source: §3.2 · p. 5

3.3 Self-Conditioning-Based Fine-Level Editing

• Because small tCE changes can cause disproportionate editing changes, EdiText adds self-conditioning for finer control over edit extent.It uses the reference representation as a condition during initial sampling, then combines fine adjustments with coarse editing.Source: §3.3 · pp. 5–6

4 Experiments

• EdiText is evaluated on toxifying and detoxifying text and on conversions between negative and positive sentiment.The toxicity experiments use explicit-toxic and non-toxic responses from Implicit Toxicity; sentiment models use sentiment pseudo-labels for Enron emails.Source: §4 · p. 6, §4.1 · p. 6

4.1 Datasets

• The toxicity data provide 12,953 training samples and 1,000 evaluation samples; sentiment experiments use the Enron Email Corpus with classifier-assigned labels.The Implicit Toxicity subset contains 13,953 response pairs; Enron contains 76,551 unlabeled emails.Source: §4.1 · p. 6

4.2 Implementation Details

• The autoencoder and diffusion models follow Lovelace et al. (2023), with 50K training iterations for autoencoders and 50K or 100K for diffusion models.Diffusion training uses 50K iterations for toxicity and 100K for sentiment; evaluation uses T = 250, L = 32, and D = 64.Source: §4.2 · p. 7

4.3 Models

• ParaGuide is the non-autoregressive baseline, while Qwen2.5-0.5B-Instruct is the autoregressive baseline tested with standard and amplified prompts.ParaGuide uses classifier guidance; its toxicity checkpoint is trained on Implicit Toxicity, and Qwen2.5 is a smaller autoregressive model with a larger parameter count than the other baselines.Source: §4.3 · p. 7

4.4 Evaluation Metrics

• Evaluation measures reference retention with Hamming Distance, SacreBLEU, and BERTScore, and target reflection with toxicity APIs or sentiment classification accuracy.Toxicity uses Moderation and Perspective APIs; sentiment uses a sentiment classifier.Source: §4.4 · pp. 7–8

5 Results

• The experiments compare coarse and fine editing across toxicity and sentiment tasks, with additional human evaluations, qualitative samples, and data results in the appendix.Toxicity results are in Tables 1 and 2, sentiment results in Table 3, and toxicity curves in Figure 3.Source: §5 · p. 8

5.1 Model Comparison

• EdiText-CE adjusts editing over a wider range than ParaGuide or Qwen2.5, and performs on par with or better than baselines at similar retention rates across toxicity and sentiment metrics.Amplifying Qwen2.5 prompts can worsen toxicity reflection and only marginally improve sentiment reflection.Source: §5.1 · p. 8
• EdiText-FE provides more granular control than EdiText-CE, with better target-attribute reflection at retention rates similar to the baselines.Its variation in retention and reflection rates is narrower than EdiText-CE's.Source: §5.1 · p. 8

5.2 Integrated Method with EdiText

• Integrated EdiText has a more favorable toxicity trade-off than other baselines, including EdiText-CE, for both detoxifying and toxifying tasks.Figure 3 plots Moderation Toxicity against Hamming Distance; lower toxicity is preferred for detoxifying and higher toxicity for toxifying at a given distance.Source: Figure 3 · p. 8, §5 · p. 8, §5.2 · p. 9
• Integrated editing makes Hamming distances between 10 and 25 more reachable, a range that EdiText-CE rarely achieves.The fine-level adjustment fills gaps between the coarse method's wider intervals.Source: §5.2 · p. 9

6 Conclusion

• EdiText supports text modifications at both coarse and fine scales.The conclusion presents the framework as offering users options for selecting desired editing levels.Source: §6 · p. 9
• SDEdit-based editing supplies coarse-level control, while self-conditioning-based editing supplies fine-level control.The conclusion describes these as methods for editing at various scales.Source: §6 · p. 9
• Integrating the techniques yields precise modifications across a broad range of control.The paper concludes that EdiText produces the intended results and offers users options for choosing editing levels.Source: §6 · p. 9

Limitations

• Edited-sample quality depends on the backbone’s generative capability, with label-conditional generation assumed for LD4LG.The framework is built on generative text models trained with attribute-specific text data.Source: Limitations · p. 9
• Transferring pretrained generation to editing without an explicit editing objective could yield lower performance than methods trained for editing objectives.The authors describe this as a potential outcome, not a demonstrated failure.Source: Limitations · p. 9
• EdiText applies to backbones using continuous-embedding diffusion and self-conditioning, and editing itself requires no training.The authors characterize this dual nature as both a limitation of the study and a direction for future research.Source: Limitations · p. 9

• EdiText modifies reference text according to user-specified attributes.The framework is described as versatile, with examples including removing harmful content and unnecessary nuances to improve clarity and appropriateness.Source: Ethics Statement · p. 9
• The stated intended uses include removing harmful content and unnecessary nuances from whole sentences.Source: Ethics Statement · pp. 9–10
• The authors identify possible misuse in generating hateful content, jailbreaking human-aligned text models, and producing toxic responses.Source: Ethics Statement · p. 10

• The integrated algorithm takes a diffusion model, noise schedule, encoder and decoder, total timestep, reference text, coarse and fine editing timesteps, and target attribute.Source: A Appendix · p. 13
• The algorithm’s stated output is edited text we.Source: A Appendix · p. 13
• The fragment sets λt = 1 − αt and gives xhat0,t,SC as xθ(xt, t, xhat0,t, c), a model prediction based on xt, t, xhat0,t, and c.Source: A Appendix · p. 13

• The diffusion model takes the noisy sample, target attribute, and self-conditioning vector as inputs.Source: §A.1 · p. 13
• The target attribute c labels the desired editing direction, such as toxifying or detoxifying, and its learned embedding conditions class-conditional denoising.Source: §A.1 · p. 13
• The self-conditioning vector is either null or a previous-timestep prediction used to refine subsequent predictions.Source: §A.1 · p. 13
• EdiText handles self-conditioning as a separate context from timestep and target-attribute embeddings.Source: §A.1 · p. 13
• Coarse editing perturbs the encoded reference before editing, while fine editing conditions denoising on the unperturbed reference encoding.Source: §A.1 · p. 13
• Integrated editing applies coarse editing first, then adjusts tFE for fine control while adding noise according to tCE.Source: §A.1 · p. 13

A.2 Editing Prompt Template for Qwen2.5-0.5B-Instruct

• Qwen2.5-0.5B-Instruct uses an editing prompt that varies by task; its amplified version appends “as much as possible” to intensify editing.The prompt template is illustrated in Figure 5.Source: §A.2 · p. 13, Figure 5 · p. 16

A.3 Additional Results

• Mix-and-Match is an additional NAR baseline using classifier scores, with DISC emphasizing discriminator score and HAM emphasizing Hamming distance.The appendix describes editing with RoBERTa-large and the ParaGuide discriminator.Source: §A.3 · pp. 13–14
• EdiText ranked highest for semantic preservation, followed by Qwen, M&M DISC, and ParaGuide, in a human evaluation of negative-style conversion.Participants rated preservation on a 5-point MOS after outputs were matched for comparable negative-attribute shifts.Source: §A.3 · p. 14, Table 4 · p. 14
• EdiText-CE showed the largest variation in retention rate and formality score, while EdiText had comparable or better reflection rates at similar retention rates.Source: §A.3 · p. 14, Table 5 · p. 16
• In qualitative toxifying examples, coarse editing changed rapidly with tCE, fine editing retained more reference text, and integration enabled control using both tCE and tFE.The examples use non-toxic reference text; ParaGuide remained similar to the reference as λ varied.Source: §A.3 · pp. 14–15, Table 6 · p. 17

A.4 Extension to Context-Conditional EdiText

• Question-conditional EdiText adds the question as conditioning and shows a pattern similar to class-label-only EdiText; its Figure 4 separates detoxifying and toxifying curves.The appendix also provides quantitative task tables and qualitative examples for this variant.Source: Figure 4 · p. 14, §A.4 · p. 15, Table 7 · p. 17

A.5 Usage of AI Assistant

• The authors state that ChatGPT assisted with paraphrasing and spell-checking their original writing.Source: §A.5 · p. 15