Source-linked AI summary
The Tatoxa System for Text Detoxification in Low-Resource Languages: The Case of Tatar
Ilseyar Alimova, Bogdan Monogov, Artyom Mazur, Daniil Antonov, Vsevolod Karimov, Vitaliy Egorov, Bulat Khakimov, Alexander Panchenko
TL;DR
Low-resource Tatar has been understudied for text detoxification, while multilingual systems often perform poorly on the language. This paper introduces Tatoxa and a Tatar dataset, finding that Tatoxa outperforms compared baselines and proprietary LLMs, whereas cross-lingual transfer performs worse than native-language training.
Problem
Automatic text detoxification systems perform poorly on low-resource languages such as Tatar, where cultural knowledge also affects reliability.
Method
Tatoxa translates Russian resources into Tatar, trains a detoxification model, and ranks multiple generated candidates, alongside manually annotated Tatar examples.
Results
Tatoxa consistently outperforms compared baselines and proprietary LLMs, while cross-lingual transfer performs worse than training on native Tatar data.
Takeaways & Limitations
Automatically translated parallel corpora can substantially improve detoxification quality and may help mitigate shortages of native annotated Tatar data.
Takeaways & Limitations
Fine-tuning only target modules containing about 1% of the baseline model may have limited performance improvements.
Abstract
from arXiv · showhide
Text detoxification, the automated detection and mitigation of abusive and harmful content, is essential for ensuring the safety of online communities and protecting users. However, low resource languages such as Tatar have received little research attention. In this paper we present Tatoxa, a novel state-of-the-art system for text detoxification in the Tatar language. Comparative experiments show that the proposed approach outperforms existing open source and proprietary commercial LLMs on key quality metrics. We also introduce a new dataset for text detoxification in Tatar, designed for fine tuning and evaluation in low resource settings. Finally, cross lingual transfer experiments indicate that transfer from other languages, including the culturally close Russian, performs significantly worse than training on native Tatar data even when a large Russian corpus is available.
1 Introduction
Text detoxification rewrites toxic content into neutral text while preserving meaning, but current systems remain far below human performance, especially in low-resource and culturally specific settings. This paper addresses the Tatar gap with Tatoxa, expanded native-language data, and cross-lingual transfer experiments.
- Motivation: Text detoxification rewrites insults, profanity, and aggression into neutral text while preserving the original meaning for safer online interaction.The approach offers an alternative to deleting toxic or obscene posts.
- Motivation: Automatic detoxification quality remains far below the human baseline, particularly for low-resource languages, partly because shared multilingual LLMs underperform across languages.Cultural knowledge and pragmatic cues also affect whether toxic content is identified and rewritten reliably.
- Problem and approach: Tatar systems achieved the lowest scores among evaluated languages in the CLEF-2025 TextDetox shared task, motivating a Tatar-specific approach.The paper presents Tatoxa and evaluates state-of-the-art methods for Tatar.
- Contributions: The paper introduces Tatoxa as a new state-of-the-art Tatar detoxification method and extends existing datasets with toxic–non-toxic Tatar text pairs.The corpus is manually augmented with 701 annotated examples for Tatar-only training and cross-lingual transfer experiments.
- Contributions: Experiments evaluate cross-lingual data transfer to assess how portable text detoxification methods are across languages.The study reports improvements in task-specific metrics.
2 Related Work
Prior work shows that language-specific parallel supervision and in-domain fine-tuning are central to effective detoxification in low-resource languages. Despite progress, Tatar and other Turkic languages remain challenging, motivating dedicated datasets and methods.
- 2 Related Work: Multilingual detoxification coverage remains uneven because many languages lack resources, despite progress for high-resource languages and scalable collection frameworks for additional languages.MultiParaDetox extends the ParaDetox pipeline beyond English through crowdsourced parallel-data collection.
- 2 Related Work: Language-specific data-centric pipelines improve detoxification, including interpretable TF–IDF and rule-based methods for African languages and fine-tuning gains for Italian.Detoxify-IT reports improvements over zero-shot LLM prompting and generic multilingual baselines with limited language-specific fine-tuning.
- 2 Related Work: Synthetic parallel data can provide scalable supervision: SynthDetoxM finds that models fine-tuned on synthetic corpora outperform zero-shot prompting and comparably sized human-annotated datasets.Modern open-source LLMs serve as few-shot annotators for multilingual parallel detoxification corpora.
- 2 Related Work: For Ukrainian toxicity classification, fine-tuning on human-annotated, language-specific data performs best, whereas cross-lingual transfer alone provides weaker baselines.Evaluated approaches include back translation, adapter training, and LLM prompting.
- 2 Related Work: Tatar detoxification remains challenging, although the CLEF-2025 shared task introduced its first fully human-annotated parallel dataset for systematic low-resource evaluation.Competition results indicate that fine-tuned and hybrid multilingual systems achieve the strongest overall performance across languages.
- 2 Related Work: Prior work consistently identifies high-quality parallel supervision and in-domain fine-tuning as primary drivers, while hybrid and rule-guided approaches remain competitive under severe resource and orthographic constraints.Supervision may be human-annotated or carefully generated with LLM assistance; zero-shot prompting and cross-lingual transfer alone are less reliable.
3 Tatoxa
Tatoxa addresses the lack of Tatar detoxification data with a four-stage translation-based pipeline, then trains a robust detoxification model with multi-candidate generation and ranking. The system filters synthetic pairs for semantic consistency and uses an ensemble of LoRA adapters on an mT0-XL backbone.
- Pipeline: Tatoxa follows four stages: Russian→Tatar NMT fine-tuning, dataset translation, detoxification-model training, and ranked multi-candidate inference.The resulting model is applied to toxicity mitigation in Tatar texts.
- Synthetic data: The adapted NLLB-200 model learns both Tatar→Russian and Russian→Tatar translation from aligned bilingual sentence pairs.Each aligned pair produces two supervised training instances.
- Synthetic data: Synthetic Tatar detoxification pairs are created by translating both sentences in Russian detoxification pairs because parallel Tatar detoxification data is scarce.The synthetic corpus draws on public Russian detoxification datasets, including Russian ParaDetox and the Russian part of Multilingual ParaDetox.
- Synthetic data: Examples are retained only when LaBSE cosine similarity reaches at least 0.7 for both toxic and neutral Russian–Tatar sentence pairs.This filtering step mitigates noise introduced by imperfect machine translation.
- Detoxification model: Tatoxa fine-tunes mT0-XL with a 3-fold ensemble of LoRA adapters to improve robustness on the automatically translated training pairs.Each fold trains an adapter on its training partition and selects its best checkpoint using held-out validation loss; the mT0-XL backbone remains frozen.
- Inference: The inference strategy generates multiple detoxified candidates and ranks them by neutrality and semantic similarity to the original sentence.This addresses both residual toxicity from insufficient detoxification and semantic drift from excessive detoxification.
4 Dataset
The Tatar detoxification dataset expands CLEF-Tatar with 701 curated examples and native-speaker annotations focused on minimal meaning-preserving edits. Compared with CLEF, it contains 101 more, slightly shorter texts, predominantly uses rewriting, and includes substantial high-to-moderate toxicity coverage.
- 701 examples were added to the relatively small Tatar CLEF-2025 dataset, with curation aimed at consistency with the original data.
- Two native-speaker annotators and an NLP-qualified moderator labeled texts according to CLEF-2025 guidelines, prioritizing minimal detoxification while preserving meaning.
- The dataset also records toxicity on a two-point scale: 1 for moderately toxic texts and 2 for highly toxic texts.
- For variants containing only Russian letters, detoxified outputs were provided both with Russian-only spelling and with correct Tatar-alphabet spelling.
- Compared with CLEF, the corpus contains 101 more samples and slightly shorter texts; rewriting is most common, while deletion and combined methods are rarer.
- Approximately 57% of samples are highly toxic with explicit profanity or overt offensiveness, whereas 43% are moderately toxic and mainly implicit or indirect.
5 Experiments
The experiments compare lexicon-based, mT0-based, and proprietary LLM baselines using CLEF-2025 evaluation metrics, while also testing cross-lingual transfer and training-data size effects. Cross-lingual experiments use multilingual and Tatar data, and ablations use large Russian and English datasets.
- Training setup: The experiments include specified model configurations and training settings for NMT and detoxification models, including LoRA adaptation, cross-entropy training, learning rates, batch sizes, and regularization parameters.The reported settings include two training epochs and LoRA applied to attention projections.
- Baselines: The evaluation compares lexicon-based methods, multilingual mT0 variants, and proprietary LLMs including Claude, Gemini, GPT-5.3, and DeepSeek.mT0 variants include different prompting languages and a hybrid mT0-plus-lexicon workflow.
- Evaluation metrics: The CLEF-2025 evaluation uses Style Transfer Accuracy, content preservation, and fluency, combined as J = STA × SIM × FL and averaged across samples.STA uses an xlm-roberta-large toxicity classifier, SIM uses LaBSE cosine similarity, and FL uses xCOMET similarity to references.
- Cross-lingual transfer: Cross-lingual experiments train on 15 paired language datasets, their combined non-Tatar data, and the Tatar competition subset to assess in-language training.Each paired dataset contains 400 samples.
- Ablation studies: The ablation studies evaluate training-data size using Russian and English detoxification datasets with 12,206 and 19,744 samples, respectively.The hyperparameters and evaluation setups match those used in the cross-lingual experiments.
6 Results
Tatoxa achieved the strongest automatic-system results on key detoxification metrics, while simple toxic-word deletion remained a strong baseline and proprietary or open LLM approaches showed limitations. Cross-lingual transfer generally underperformed native Tatar fine-tuning, with French an exception, and larger training sets eventually yielded diminishing or unstable gains.
- Automatic-system comparisons: Tatoxa achieved the highest overall J scores, 69.5% and 68.0%, and STA scores, 98.2% and 97.0%, across the two datasets.The human baseline outperformed all automatic systems, so subsequent analysis focused on automatic methods.
- Baselines and prompts: Simple toxic-word deletion outperformed mT0-based detoxification on the authors’ dataset and was on par with mT0 on CLEF-Tatar.The English prompt produced the highest reported overall performance: 58.0% on CLEF-Tatar and 53.7% on the other dataset.
- LLM limitations: Proprietary LLMs remained ineffective overall, while high STA combined with low SIM and FL suggested excessive alteration of the original text.The authors attribute this limitation to models’ insufficient familiarity with low-resource Tatar, including its slang and toxic-expression semantics.
- Cross-lingual transfer: Native Tatar fine-tuning attained the highest cross-lingual-transfer J-score, while French performed nearly identically and English, Spanish, and Russian failed to surpass the baseline.The unexpectedly strong French results and weaker Russian results may reflect language-distribution biases from pretraining and fine-tuning.
- Training-set size: Performance improved with training-set size initially, then plateaued or declined after approximately 5k Russian samples and 7k English samples.Russian results showed greater instability and variance than English, plausibly because multiple neutral variants corresponded to the same toxic source sentence.
7 Conclusion
The paper presents Tatoxa for automatic detoxification of Tatar texts and reports that it outperforms compared baselines and proprietary LLMs on key quality metrics. Cross-lingual transfer, including from culturally close Russian, performs significantly worse than training on target-language data, while automatically translated parallel corpora can substantially improve results.
- Tatoxa is a method for automatic detoxification of texts in the Tatar language.
- Tatoxa consistently outperforms compared baselines and proprietary LLMs on key detoxification-quality and semantic-preservation metrics.
- Transfer from other languages, including culturally close Russian, falls significantly short of training on target-language data, even with a large Russian training set.
- Training on automatically translated parallel corpora can yield substantial improvements.
8 Limitations
The study could not fine-tune on other related Turkic languages, limiting cross-language comparison, and fine-tuned only a small subset of model parameters. These constraints may have limited performance improvements and motivate broader language coverage and larger parameter subsets in future work.
- Language coverage: The study did not fine-tune models on other Turkic languages related to Tatar, limiting cross-language comparison and assessment of linguistic proximity.Future work should include additional Turkic languages for a more comprehensive evaluation.
9 Appendix
The appendix specifies the Tatar text-detoxification prompt, directing an LLM to remove toxicity while preserving meaning, language, and style. It also provides Tatar examples illustrating selective rewriting and unchanged handling of non-toxic text.
- Prompt objective: The prompt instructs the system to neutralize insults, profanity, and hate speech in Tatar with minimal redaction while preserving the original meaning and language.It frames the model as a professional Tatar linguist and editor and says the explicit input is for analysis rather than blocking.
- Examples: Examples replace Tatar insults and demeaning terms while retaining surrounding content, but reproduce a non-toxic sentence unchanged.The examples include substitutions for an insult, profanity, and a demeaning animal comparison, alongside an unchanged positive sentence.
- Prompt procedure: The procedure requires rewriting all toxic parts, leaving non-toxic content unchanged, and preserving the sentence’s meaning, styling, and Cyrillic script when applicable.Sentences consisting only of swears should receive the shortest emotionally closest meaningful replacement.