ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusion

• CXR report generation could reduce radiologists’ workload, but autoregressive VLMs incur high latency from sequential decoding.Source: §1 · pp. 1–2
• One-step diffusion can maximize throughput but suffers incoherent outputs because token-factorized denoisers cannot represent joint token dependencies.Source: §1 · p. 2, Figure 1 · p. 2
• DCD constructs unfactorized supervision from on-policy teacher trajectories conditioned on committed high-confidence context.Source: §1 · p. 2
• ECHO converts an enhanced autoregressive CXR model into block diffusion with RAD, then distills it into a one-step counterpart using DCD.Source: §1 · pp. 2–3
• 64.33% and 60.58% improvements in RaTE and SemScore, respectively, accompany up to 8× theoretical and 5.1× practical inference speedups over autoregressive baselines.Source: §1 · p. 3
• DCD reaches up to 390% speedup over the corresponding multi-step baseline at block size L=8 with marginal quality degradation, while RAD reduces theoretical training FLOPs by 72.3%.Source: §1 · p. 3

• Discrete diffusion language models progressively mask tokens and recover them through a learned reverse process over a discrete vocabulary.Source: §2.2 · p. 3
• dVLMs extend discrete diffusion to image-conditioned parallel generation through visual encoder alignment and instruction fine-tuning.Source: §2.2 · p. 3
• Existing acceleration methods reduce computation per step or distill multi-step denoising, but retain token-factorized prediction targets.Source: §2.3 · p. 3
• ECHO’s DCD supports one-step denoising for dLLMs and dVLMs, applied here to block diffusion.Source: §2.3 · p. 4

• dLLMs define a forward process that progressively masks tokens and a reverse process that recovers them.Source: §3 · p. 4

Mean-field parameterization

• Mean-field parameterization approximates the joint posterior with independent per-token factors because the full joint requires exponentially many parameters.Source: §3 · p. 4
• Per-position cross-entropy training matches each token’s conditional marginal but cannot capture cross-positional correlations in the true posterior.Source: §3 · p. 4
• Mean-field bias quantifies irreducible joint dependence structure that no factorized distribution can represent.Source: §3 · p. 4

Why multi-step sampling mitigates the bias

• Mean-field bias grows monotonically with masked-token count, reaching its maximum when all tokens are masked.Source: §3 · p. 4

Pipeline overview

• ECHO uses three successive stages: continued pre-training creates ECHOAR, RAD creates ECHOBase, and DCD produces the final ECHO model.The final model performs one-step-per-block decoding.Source: §4.1 · pp. 4–5, Figure 2 · p. 5

Response-Asymmetric Diffusion

• RAD converts ECHOAR into a block diffusion model by duplicating only response tokens and letting noisy response blocks attend to context and prior decoded blocks.This avoids duplicating long vision-token sequences in a two-stage conversion.Source: §4.2 · p. 5, Figure 2 · p. 5

Direct Conditional Distillation

• DCD collects high-confidence distributions along the teacher’s multi-step trajectory and stitches them into a joint target that captures cross-token dependencies.This non-factorized supervision is designed to reduce the mean-field limitation of one-step prediction.Source: §4.2 · pp. 5–6
• DCD aligns the student’s one-step block prediction with the teacher target using forward KL divergence and reweights positions according to their unmasking steps.Positions exposed to stronger mean-field bias receive stronger supervision.Source: §4.2 · p. 6

Termination and hallucination

• An additional cross-entropy loss targets <eos> tokens because their confidence is lower and more variable than content-token confidence, especially at larger block sizes.This addresses difficulty terminating generation and the associated repetition loops.Source: §4.3 · p. 7

Inference optimization

• Fused block KV caching removes the dedicated cache-update pass by combining the preceding block’s update with the current block’s denoising forward.The optimization targets the overhead that is especially costly for one-step decoding.Source: §4.4 · p. 8, Figure 4 · p. 8

Evaluation setup

• Experiments compare ECHO with proprietary, autoregressive, and diffusion-based models across linguistic, clinical, structural, and efficiency dimensions.The evaluation uses average metrics across MIMIC-CXR, CheXpert-Plus, and ReXGradient.Source: §5.1.4 · p. 9, §5.2 · p. 9, Table 1 · p. 10

Overall performance

• 8× theoretical speedup incurs only 3–7% quality degradation for ECHOblk8, while ECHOblk4 achieves 4× speedup with clinical-fidelity degradation within 3% on average.These configurations outperform the reported quality–speed trade-offs of T3D and dParallel.Source: §5.2 · pp. 9–10

DCD ablation

• SW reduces PPL from 23.72 to 21.07 and raises CheXpert-Plus RaTEScore from 54.87 to 56.30, whereas RKL lowers MIMIC-CXR CIDEr from 3.65 to 3.48.The authors associate SW with stronger inter-token conditioning and prefer forward KL for preserving plausible clinical findings.Source: §5.3 · pp. 10–11
• CE supervision on <eos> produces the largest ablation gain, raising ROUGE-L from 52.44 to 56.14 and CIDEr from 3.65 to 4.05.PPL falls to 18.83, supporting reliable termination as important for avoiding repetition loops.Source: §5.3 · p. 10

RAD adaptation

• Quality reaches or surpasses the ECHOAR baseline within about 60 RAD steps, while throughput rises from 1.62 to 2.17, a 33.95% increase.Quality saturates at roughly 2.2% of the full RAD training data, whereas throughput continues improving later.Source: §5.4 · p. 11

DCD ablation

• DCD improves ECHOblk4 on CheXpert-Plus by 36% in ROUGE-L and 44% in CIDEr over native one-step decoding, with SemScore gains up to 29% on MIMIC-CXR.The improvements remain consistent across both block sizes and all three benchmarks.Source: §5.5 · p. 12

Report normalization

• Normalized reports raise CheXpert-Plus ROUGE-L from 23.82 to 56.89 and RaTEScore from 45.59 to 59.18 during Stage I.Without normalization, distillation later reduces CheXpert-Plus ROUGE-L to 18.79, while normalized training keeps SemScore above 45.0.Source: §5.6 · p. 12

Conclusion

• ECHO demonstrates that high clinical accuracy and extreme decoding efficiency are simultaneously achievable in automated chest X-ray reporting.The conclusion positions this combination as the paper’s central outcome.Source: §6 · p. 12
• ECHO reduces training cost, eliminates multi-step denoising, and outperforms autoregressive and diffusion-based state-of-the-art methods across standard benchmarks.These claims summarize the paper’s reported system-level advantages.Source: §6 · p. 12

Supplementary analyses

• The supplementary material includes qualitative examples, experimental details, baseline implementations, detailed per-dataset results, and fused block KV-cache analysis.The KV-cache analysis formally shows preserved total FLOPs with half as many forward passes for one-step-per-block decoding.Source: Appendix · p. 17

Qualitative evidence

• Qualitative comparisons show severe disorder and repetition in undistilled native one-step decoding, while DCD largely removes these artifacts in ECHOblk8 and ECHOblk4.Additional examples report accurate detection and clinical description of multiple abnormalities.Source: A Qualitative examples · p. 17

Corpus construction

• The unified corpus aggregates reports from MIMIC-CXR, CheXpert-Plus, ReXGradient, and IU-Xray.A five-stage preprocessing pipeline is applied across sources.Source: §B.1 · p. 17

Preprocessing

• Modality filtering removes non-CXR studies and samples with incomplete clinical descriptions using clinical entity extraction.This is the first stage of corpus preparation.Source: §B.1 · p. 17
• Prompt-based rewriting standardizes terminology, enforces a structured Findings–Impression JSON format, and explicitly enumerates negative findings.The rewriting pipeline uses BaichuanM2-32B.Source: §B.1 · p. 17
• Semantic deduplication leaves approximately 260k MIMIC-CXR, 250k CheXpert-Plus, 190k ReXGradient, and 30k IU-Xray reports.Near-duplicates are removed using Qwen3-Embedding-8B and a cosine-similarity threshold.Source: §B.1 · p. 17
• Bilingual augmentation translates a random 50% of original reports from English into Chinese while preserving medical terminology and report structure.The pipeline also incorporates LLaVA-ReCap-558k to preserve general vision-language grounding.Source: §B.1 · pp. 17–18

Corpus construction

• The resulting corpus is large-scale, standardized, and bilingual for CXR report generation.The corpus combines the preceding preprocessing stages into a unified training resource.Source: §B.1 · p. 18

Training configuration

• All stages enforce a maximum image size of 2,250,000 pixels to balance high-resolution information retention with GPU memory consumption.Training is conducted on Alibaba PPUs.Source: §B.2 · p. 18

Stage 1: AR continual pretraining

• Stage 1 performs full-parameter autoregressive SFT on Lingshu using AdamW with a 1 × 10−5 learning rate and 0.01 weight decay.A linear warmup covers the first 0.03 epochs.Source: §B.2 · p. 18

Stage 2: Diffusion adaptation

• Stage 2 converts the model to diffusion using approximately 2.2% of Stage 1 data, freezing the vision encoder and projector while training only the LLM backbone.Neat-packing addresses the memory overhead of extended high-resolution vision-token sequences.Source: §B.2 · p. 18

Stage 3: Distillation

• Stage 3 distills the diffusion model using a randomly sampled 2.3% of the SFT dataset while keeping the vision encoder and projector frozen.This stage also uses neat-packing-based training.Source: §B.2 · p. 18

Evaluation framework

• Five established metrics evaluate generated reports across quality and efficiency dimensions.Quality metrics include ROUGE-L, CIDEr, Modified RaTEScore, SemScore, and Perplexity; TPF and TPS assess decoding efficiency.Source: §B.3 · pp. 18–19

Quality metrics

• ROUGE-L measures surface overlap through longest-common-subsequence structure and word-order similarity.Source: §B.3 · p. 18
• CIDEr emphasizes rare clinical terms using cosine similarity between TF-IDF-weighted n-grams.Source: §B.3 · pp. 18–19
• Modified RaTEScore measures clinical entity-level accuracy using NER, medical embeddings, and a weighted F1 clinical-importance matrix.Source: §B.3 · p. 19

Metric adaptation

• Negative-related entity types receive zero importance weights because the normalized test set explicitly enumerates negative findings.This adaptation focuses evaluation on correctly identified positive pathologies.Source: §B.3 · p. 19

Quality and efficiency metrics

• SemScore measures semantic similarity with cosine similarity between sentence-transformer embeddings, while Perplexity measures generated-text fluency.TPF divides decoded tokens by model forward passes, and TPS divides generated tokens by elapsed decoding time.Source: §B.3 · p. 19

Autoregressive baselines

• Autoregressive baselines use greedy decoding, a 512-token limit, KV caching, and FlashAttention.Source: C Implementation of baselines · p. 20

Diffusion baselines

• LLaDA-MedV uses low-confidence remasking with response length L = 256, block length B = 64, and Z = 256 sampling steps.Its default fast_dllm configuration further accelerates inference.Source: C Implementation of baselines · p. 20

Distillation baselines

• Additional distillation baselines share the ECHO-Baseblk8 model and training data and use semi-autoregressive decoding aligned with BD3LM.They are grouped into trajectory-, distribution-, and optimization-based methods.Source: C Implementation of baselines · pp. 20–21

Report preprocessing

• The standardization prompt separates Findings and Impression content and requires standardized reports in JSON format.The template addresses incomplete findings and misplaced interpretive statements, while preserving original findings and impressions.Source: C Implementation of baselines · p. 21

Distillation baselines

• Distribution-based distillation aligns predictions across noise levels with CE and KL losses, whereas optimization-based methods correct mean-field bias using teacher pseudo-labels.Source: C Implementation of baselines · p. 21

Per-dataset results

• ECHOBase achieves state-of-the-art performance across all datasets, while ECHO maintains performance with minimal quality degradation and maximum inference speedup.Source: D Detailed results · p. 21

FLOPs equivalence

• Fused Block KV Cache preserves total FLOPs while halving the number of forward passes, directly lowering inference latency.The equality holds for every n ≥ 1.Source: E Fused Block KV Cache Analysis · pp. 22–23

Vanilla Block KV Cache

• Vanilla Block KV Cache performs separate denoise and KV-update passes for each block, yielding 2N forward passes.Source: E Fused Block KV Cache Analysis · p. 22

Fused Block KV Cache

• Fused Block KV Cache defers block 0’s update and merges each later update with the next block’s denoising, yielding N forward passes.Source: E Fused Block KV Cache Analysis · p. 22

FLOPs equivalence

• The fused operation preserves the corresponding attention contexts for decoded and masked tokens under the block-causal mask.Source: E Fused Block KV Cache Analysis · p. 23