Writerslogic at the CLEF 2026 SimpleText Track: Multi-Candidate LLM Simplification and Stacked Complexity Spotting

1. Introduction

• Biomedical writing’s specialized vocabulary, complex sentences, and dense statistics can impede patients, caregivers, journalists, and policymakers seeking to understand research findings.The SimpleText shared task addresses this challenge through automatic simplification and complexity spotting in machine-generated simplifications.Source: §1 · p. 1
• Task 1 simplifies Cochrane-review sentences or abstracts while preserving key facts and using accessible vocabulary; SARI evaluates simplification operations against human references.The task includes both sentence-level and document-level simplification.Source: §1 · p. 1
• Task 2 identifies sentences with overgeneration errors and classifies them among five error types, with evaluation at sentence and document levels.The errors include leaked instructions, ungrounded injection, generation failure, repetitive content, and grounded overgeneration.Source: §1 · pp. 1–2
• Writerslogic generates multiple simplification candidates at varying temperatures and selects one with a reference-free proxy score; its Task 2 approach combines engineered, neural, and sequential features.The sequential features target the tendency of overgeneration errors to propagate within a document.Source: §1 · p. 2

2. Related Work

• Text simplification progressed from rule-based lexical and syntactic changes to neural sequence-to-sequence models and more recent zero-shot and few-shot large language models.Paragraph-level simplification’s benefits from document context motivate distinguishing sentence- and document-level tasks.Source: §2 · p. 2
• Cochrane Plain Language Summaries replace jargon, remove parenthetical statistics, and retain important counts, providing domain knowledge for the generation pipeline.The authors incorporate this knowledge through few-shot examples and vocabulary priors.Source: §2 · p. 2
• SARI scores additions, keeping, and deletions of n-grams; the candidate-selection heuristic approximates its keep and delete components without reference simplifications.Unlike BLEU, SARI rewards appropriate deletions and penalizes unnecessary retention of complex content.Source: §2 · p. 2
• Prior unfaithful-generation detection includes NLI entailment checking, question-answering consistency, and error classification; this system uses cross-encoder NLI scores to assess source-sentence consistency.LightGBM and logistic-regression stacking provide related approaches for feature-engineered classification and shared-task settings.Source: §2 · p. 2
• LightGBM is described as competitive for feature-engineered classification when training data is limited and interpretability matters, while stacking ensembles have shown gains over single-stage classifiers.The cited examples concern shared-task settings.Source: §2 · p. 2

3.1. Multi-Candidate Generation with SARI-Proxy Reranking

• The primary approach generates five GPT-4o-mini simplification candidates per input at temperatures 0.0, 0.3, 0.5, 0.7, and 1.0, then scores them using four weighted criteria.The score weights compression 0.25 (target ratio 0.55–0.85), source-token retention 0.40, curated Cochrane PLS vocabulary 0.15, and word-length simplicity 0.20. The prompt requests minimal, targeted changes; English inputs receive seven curated examples, while non-English inputs receive none and an instruction to keep the same language.Source: §3.1 · pp. 2–3

3.2. Local Ensemble Approach

• The local Qwen3-8B alternative uses three prompting strategies and scores candidates for readability, word length, compression, and source overlap.It trades generation quality for cost efficiency and reproducibility through local inference.Source: §3.2 · p. 3

3.3. Claude Sonnet 4 Submission

• SARI 47.43 and BLEU 14.21: the Claude Sonnet 4 submission ranked highest among sentence-level systems on both metrics and third on the combined Task 1 leaderboard.Two document-level systems posted higher overall SARI scores of 48.85 and 47.57; the sentence-level system’s compression ratio was 0.73.Source: §3.3 · p. 4

3.4. Document-Level Simplification

• Document-level simplification uses Llama 3.3 70B with prompts allowing sentence merging, splitting, and reordering while preserving key findings and accessible language.Its token limit increases from 300 for sentence-level work to 1,500 for full-abstract simplification.Source: §3.4 · p. 4

4.1. Problem Formulation

• Task 2.1 identifies any overgeneration error versus None, while Task 2.2 classifies errors into six categories; both are evaluated at sentence level and aggregated to document level.Documents contain a source and a sequence of generated sentences.Source: §4.1 · p. 4

4.2. DeBERTa Classifier for Binary Identification (Task 2.1)

• For binary identification, the team fine-tuned DeBERTa-v3-base on concatenated source text and generated sentences, and also describes fine-tuning a DeBERTa-v3-large model.The base model was trained for five epochs; the large model was initialized from a checkpoint and trained on the team’s examples.Source: §4.2 · p. 4
• 0.8081 document-level macro F1: the DeBERTa system ranked first in the identification track and second among teams on this metric; its best ensemble reached 0.8085.For multi-class classification, the best submission achieved 0.804 accuracy, ranking second among unique teams; the standalone LightGBM ensemble scored 0.6868.Source: §4.2 · p. 5

4.3. Feature Engineering

• Feature engineering includes source alignment, sentence position, and document context; sequential features track novelty patterns across preceding and following sentences.These signals let the classifier learn transition patterns associated with overgeneration continuing across a document, without explicit sequence modeling.Source: §4.3 · pp. 5–6

4.4. Neural Features

• Embedding similarity and NLI entailment and contradiction probabilities provide semantic grounding signals beyond lexical overlap.The NLI cross-encoder scores each source-sentence pair; low similarity or low entailment can signal divergence from source content.Source: §4.4 · p. 6

4.5. Classification Architecture

• Two five-model LightGBM ensembles provide binary and multi-class predictions; a logistic-regression meta-learner combines out-of-fold predictions with document-context features.Threshold and per-class offset tuning optimize validation performance, including macro F1 for rare error categories.Source: §4.5 · pp. 6–7

4.6. Training Protocol

• The training pipeline computes neural and handcrafted features, produces grouped cross-validation out-of-fold predictions, trains meta-learners, tunes thresholds, and retrains base models before test prediction.The dev set is used for early stopping in the final training stage.Source: §4.6 · p. 7

5.1. Task 1: Text Simplification

• SARI 47.43 and BLEU 14.21 made Claude Sonnet 4 the top sentence-level system and 3rd on the combined Task 1 board.The document-level system ranked 50th overall and 20th among document-level entries, with a 0.80 deletion ratio.Source: Table 1 · p. 7, §5.1 · p. 7

5.2. Task 2: Complexity Spotting

• Document F1 0.8081 (0.8085 in the best ensemble) ranked Task 2.1 first in identification and 2nd among teams; Task 2.2 accuracy 0.804 ranked 2nd among unique teams.AIIR Lab scored 0.8197 on Task 2.1 and 0.827 on Task 2.2.Source: Table 2 · p. 8
• Five-fold GroupKFold cross-validation produced out-of-fold binary F1 consistent with official test performance, although detailed scores were omitted.Source: §5.2 · p. 8
• Sequential features, including tail novel mean and high-novelty streak, ranked among the multi-class model’s top 15 features.The top features also included novel token ratio, token overlap, relative sentence position, embedding similarity to source, NLI entailment, and longest source match.Source: §5.2 · p. 8
• Table 3 reports Task 2 component ablations using development-set document-level macro F1.Source: Table 3 · p. 8, §5.2 · p. 8

• Generating five temperature-varied candidates and selecting by a reference-free proxy balances compression, source preservation, and vocabulary simplicity without requiring references at inference.The approach is presented as applicable to any input domain.Source: §6 · p. 8
• Proxy weights of 0.40 for keep, 0.25 for compression, 0.20 for simplicity, and 0.15 for PLS vocabulary favor conservative simplification.The stated rationale is to preserve content and make targeted lexical substitutions rather than aggressively rewrite Cochrane text.Source: §6 · p. 8
• Overgeneration errors may propagate across document sentences, appearing as rising novel-token ratios, increasing prompt-pattern counts, and growing source-embedding divergence.Source: §6 · p. 9
• Novel-ratio changes, tail statistics, high-novelty streaks, and distance from first divergence let LightGBM learn transition patterns without explicit sequence modeling.The approach is described as computationally cheaper than CRF or BiLSTM-CRF methods.Source: §6 · p. 9
• A logistic-regression meta-learner recalibrates LightGBM probabilities and combines model confidence with document-level context features.The passage notes that base probabilities may be poorly calibrated for rare classes.Source: §6 · p. 9
• Greedy per-class offset optimization targets macro F1 and improves rare-class recall at a controlled cost to overall accuracy.It addresses under-prediction of rare Task 2.2 categories under standard argmax classification.Source: §6 · p. 9
• Task 1 relies on commercial APIs, limiting reproducibility and adding cost; Task 2 requires all document sentences at prediction, GPU resources, and incurs NLI and embedding latency.The sequential features preclude streaming or single-sentence inference.Source: §6 · p. 9

7. Conclusion

• For Task 1, multi-candidate generation with SARI-proxy reranking offers simplification without reference texts at inference.Cochrane PLS vocabulary and few-shot examples ground the system in domain-appropriate patterns.Source: §7 · p. 9
• For Task 2, the DeBERTa system combines feature engineering, neural embedding and NLI signals, and sequential features that capture overgeneration propagation.A two-stage stacking architecture with per-class threshold optimization addresses severe class imbalance in multi-class error detection.Source: §7 · p. 9
• Future work includes fine-tuned simplification models trained on Cochrane PLS data, joint sentence-and-document optimization for Task 2, and LLM-based judges.The proposed judges would support candidate reranking and overgeneration detection.Source: §7 · p. 9
• The team’s code is available on GitHub.https://github.com/dcondrey/simpletext-clef2026Source: §7 · p. 9

Declaration on Generative AI

• Claude Code was used to write and debug code, orchestrate experiments, and search literature.Source: Declaration on Generative AI · p. 10
• GPT-4o-mini generated simplified text candidates for Task 1 and classified Task 2 test samples; Claude Sonnet 4 also generated Task 1 candidates.Source: Declaration on Generative AI · p. 10
• The author reviewed and edited the content, takes responsibility for it, and states that the scientific content, experimental design, analysis, and conclusions are their own.Source: Declaration on Generative AI · p. 10