Source-linked AI summary
Style-Driven Data Synthesis and Degradation-Aware Enhancement for Ultrasound Image Restoration
Yu-Kai Wang, Chun-Xin Tan, Manh-Hung Nguyen, Ching-Chun Huang
TL;DR
Handheld-ultrasound enhancement lacks pixel-aligned LQ–HQ pairs because real scans differ in probe angle and tissue deformation. The paper synthesizes aligned pairs from unaligned clinical data and trains a degradation-conditioned enhancer, reporting a 16.7% FID reduction over the strongest baseline.
Problem
Supervised LQ-to-HQ enhancement requires pixel-aligned pairs, but scans of the same patient differ in probe angle and tissue deformation, leaving real LQ–HQ pairs unaligned.
Source: §I · p. 1Method
A two-stage framework learns HQ-to-LQ degradation from unaligned clinical pairs to synthesize aligned data, then trains a PiSA-SR-based enhancer with degradation-conditioned DDG-LoRA.
Source: §I · pp. 1–2, §V · p. 5Results
Takeaways & Limitations
The synthesized pairs retain real, unmodified HQ targets, which the paper says avoids hallucination risk on the supervision side.
Source: §I · p. 2
Abstract
from arXiv · showhide
Low-cost handheld ultrasound devices can be widely deployed compared to professional hospital ultrasound machines. However, their images suffer from compound degradation that can mislead clinical judgment. Motivated by this observation, mapping handheld low-quality (LQ) to hospital high-quality (HQ) images has been considered a valuable research question. Conventionally, the mapping requires pixel-aligned LQ-HQ pairs. This requirement is unsatisfactory in practical scenarios because real scans at different times are never pixel-aligned. This paper addresses the challenge with a two-stage framework. The first stage generates pixel-aligned LQ-HQ datasets, and the second stage trains an enhancement model that improves LQ images. The first stage trains a cycle-consistent style-transfer model on unaligned real LQ-HQ pairs to learn a HQ-to-LQ model. Then, the model transforms real HQ images into pixel-aligned LQ images. Based on the dataset generated by the first stage, the second stage uses the Dual Degradation-Guided (DDG) Low-Rank Adaptation (LoRA) method to fine-tune an LQ-to-HQ model based on aligned pairs. In this stage, the model is based on the well known PiSA-SR framework but inserts a degradation-conditioned correction matrix. Experimental results on the USenhance2023 dataset show that the FID metric is improved by 16.7% over the strongest baseline while other metrics indicate that our enhanced outputs are well aligned with the real HQ distribution. The source code of our method is available at https://github.com/Jason0411202/DDG_LoRA.
I. INTRODUCTION
The introduction frames unaligned clinical scans as a barrier to supervised handheld-ultrasound enhancement and proposes synthesizing aligned pairs before training an enhancer. Its two-stage approach keeps real HQ targets unchanged while learning degradation from real scans, and reports a 16.7% FID improvement over the strongest baseline.
I. INTRODUCTION
- Supervised enhancement requires pixel-aligned LQ–HQ pairs, but scans of the same patient differ in probe angle and tissue deformation.The paper states that unaligned supervision is inconsistent and leads to poor convergence.Source: §I · p. 1
- Hand-crafted degradation can preserve alignment but is too idealized, while accurate acoustic simulation is prohibitively complex.The introduction also notes that unpaired CycleGAN-style enhancement can fail on medical images when it generates the HQ side without real HQ supervision.Source: §I · p. 1
- The pipeline first learns HQ-to-LQ degradation from unaligned real pairs to synthesize pixel-aligned pairs, then trains an LQ-to-HQ enhancer on them.The real HQ images remain unchanged as supervision targets, avoiding the hallucination risk of methods that generate the HQ side directly.Source: §I · pp. 1–2
- DDG-LoRA extends PiSA-SR with a degradation-conditioned correction matrix to adapt enhancement to complex, spatially varying handheld-ultrasound speckle.PiSA-SR’s dual-LoRA branches separate low-level pixel fidelity from high-level semantic plausibility.Source: §I · p. 2
- 16.7% better FID than the strongest baseline is reported for the method on the real-world USenhance2023 dataset.FID measures distributional distance between enhanced outputs and authentic HQ scans.Source: §I · p. 2
II. RELATED WORK
The related-work section traces image restoration from CNN-based and real-world super-resolution methods to diffusion models, emphasizing that performance depends on whether the assumed degradation matches real inputs. It also notes that ultrasound-specific methods continue to train on synthetic degradation.
A. Image Super-Resolution and Degradation Modeling
- Real-world restoration methods use distinct degradation assumptions: Real-ESRGAN uses high-order degradation, DARSR performs test-time adaptation, and PDM adopts a probabilistic model.OSEDiff distills a latent diffusion model into a one-step restorer, while PiSA-SR introduces dual-LoRA branches for fidelity–perception trade-offs.Source: A. Image Super-Resolution and… · p. 2
- Across restoration paradigms, real-input performance hinges on matching the assumed degradation; architectural refinement alone cannot close the gap when it does not match.Source: A. Image Super-Resolution and… · p. 2
B. Ultrasound-Specific Image Enhancement
Prior ultrasound-enhancement methods differ in architecture but still train on synthetically degraded inputs. The paper instead uses unpaired translation as HQ-to-LQ degradation simulation, leaving real HQ targets untouched.
B. Ultrasound-Specific Image Enhancement
- Ultrasound enhancement methods include GAN architectures, wavelet-decomposition degradation, and self-supervised physics-guided degradation [14], yet all train on synthetic LQ images.The paper says this limits their generalization to real handheld output.Source: B. Ultrasound-Specific Image Enhancement · p. 2
C. Unpaired Translation: Enhancement vs. Degradation
- An unconstrained CycleGAN enhancer can create plausible-looking anatomy that is not present, a hallucination problem the paper calls unacceptable in medical imaging.Huang et al. used CycleGAN for crosscenter ultrasound domain normalization.Source: C. Unpaired Translation: Enhancement… · p. 2
- Rather than use unpaired translation to generate enhanced anatomy, the paper uses it in the HQ-to-LQ direction to simulate quality loss from unaligned data.The HQ training targets remain real and untouched, removing hallucination risk on the HQ side while retaining the generator’s ability to learn complex, device-specific degradation.Source: C. Unpaired Translation: Enhancement… · p. 2
III. PROPOSED METHOD
The proposed method separates aligned-data synthesis from enhancement training. CycleDiff-based style transfer learns degradation from unaligned clinical pairs, while DDG-LoRA uses staged, degradation-conditioned updates for enhancement.
A. Style-Driven Data Synthesis Pipeline
- CycleDiff-based style transfer learns HQ-to-LQ translation from unaligned clinical pairs, producing synthetic LQ images paired with their source HQ images for the second stage.The denoising network predicts a clean image component from the noisy mixture, which is used in the subsequent translation process; cycle consistency is also used to train style transfer.Source: A. Style-Driven Data Synthesis Pipeline · p. 2
- CycleDiff models forward diffusion in domain S, with image attenuation reaching zero at t = 1.The text describes the attenuation gradient and says the denoiser predicts the clean signal content isolated from the noisy mixture.Source: A. Style-Driven Data Synthesis Pipeline · p. 2
III. PROPOSED METHOD
- DDG-LoRA trains a pixel-stage update first, then freezes it while training a semantic-stage update; both stages use degradation conditioning from an LQ-image descriptor.The semantic stage drops the CSD loss.Source: Figure 2 · p. 3
B. DDG-LoRA Enhancer
The enhancer builds on PiSA-SR’s dual-LoRA design, adding input-dependent degradation guidance and adapting the training objective for ultrasound.
B. DDG-LoRA Enhancer
- PiSA-SR trains a pixel-level LoRA for fidelity, then a semantic-level LoRA while freezing the first; both are active at inference to trade off fidelity and perceptual quality.The base UNet and VAE weights remain frozen while the LoRA factorizations are trained.Source: B. DDG-LoRA Enhancer · p. 3
- A degradation estimator predicts noise and blur levels from each LQ input, and a correction matrix inserted between the LoRA factors makes the weight update input-dependent.Fourier features and a layer-specific block embedding let each UNet layer interpret the descriptor; pixel and semantic branches receive independent corrections.Source: B. DDG-LoRA Enhancer · p. 3
- Because the natural-image text-prompt extractor yields unhelpful out-of-domain concepts for ultrasound, DDG-LoRA disables LCSD and text conditioning, leaving LPIPS as the Stage-2 objective.Source: B. DDG-LoRA Enhancer · p. 3
A. Experimental Settings
Experiments construct aligned training pairs from unaligned clinical scans, train DDG-LoRA on them, and evaluate enhancement on the held-out real-world test split using no-reference metrics.
A. Experimental Settings
- The 1,050-pair USenhance2023 collection is split into 840 training and 210 test pairs, and CycleDiff transforms the training HQ images into aligned LQ counterparts while leaving HQ targets unchanged.The collection covers five organs, with both scans in each pair at 256×256 resolution.Source: A. Experimental Settings · p. 3
- DDG-LoRA uses SD-2.1-base, PiSA-SR dual-LoRA checkpoints, and single-step diffusion with residual learning for 12,500 training steps.The synthesized LQ inputs are downsampled 4× to 64×64; training first optimizes pixel LoRA for 4,000 steps, then semantic LoRA.Source: A. Experimental Settings · pp. 3–4
- Evaluation uses the 210 unaligned real-world test pairs and five metrics that do not require pixel-level correspondence, with FID comparing outputs against the authentic HQ distribution.FID is emphasized because it can reveal over-sharpened, structurally hallucinated artifacts that non-reference metrics may miss.Source: A. Experimental Settings · p. 4
B. Impact of Training Data on Enhancement Performance
Across five training-data sources, the style-driven aligned set ranks first on four reported metrics and second on NIQE, with a lower FID than the strongest hand-crafted baseline.
B. Impact of Training Data on Enhancement Performance
- The five training sources span general-purpose and ultrasound-specific hand-crafted degradations alongside the proposed style-driven set learned from real handheld LQ scans.The alternatives include realesrgan_deg, physics_guided_deg, usbsr_deg, and ultrasound_deg.Source: B. Impact of Training Data on… · p. 4
- FID falls from 127.01 for the lowest-FID hand-crafted baseline, realesrgan_deg, to 99.00 for USenhance2023-Aligned, which ranks first on FID, PI, Tenengrad, and Entropy and second on NIQE.The comparison evaluates DDG-LoRA trained on different LQ data sources against the real-world USenhance2023 test set.Source: Table I · p. 4, B. Impact of Training Data on… · p. 4
- The authors suggest that learning degradation from real handheld scans captures statistical characteristics that hand-crafted pipelines cannot reproduce.The reported FID indicates that outputs from USenhance2023-Aligned lie closer to the authentic HQ-scan distribution.Source: B. Impact of Training Data on… · p. 4
C. Comparison with State-of-the-Art Methods
DDG-LoRA outperforms seven representative baselines on all five reported metrics, with the largest highlighted comparison in FID. In a real-world sample, its enhanced output most closely matches the reference in grayscale intensity, contrast, and texture detail.
C. Comparison with State-of-the-Art Methods
- The seven baselines span CNN-based, cycle-consistent, degradation-adaptive, probabilistic, ultrasound-specific, and diffusion-based approaches.The set includes EDSR, CycleSR, DARSR, PDM, USBSR, OSEDiff, and PiSA-SR.Source: C. Comparison with State-of-the-Art… · p. 4
- 16.7% lower FID (99.00 vs. 118.87) than the strongest baseline; DDG-LoRA also leads all baselines on the other four metrics.Its reported scores are NIQE 5.619, PI 4.764, Tenengrad 0.1043, and Entropy 6.861.Source: C. Comparison with State-of-the-Art… · pp. 4–5
- DDG-LoRA most closely matches the reference in grayscale intensity, contrast, and texture detail in the real-world sample.The comparison shows the LQ input in (a), method outputs in (b)–(i), and GT in (j); the real scans are not pixel-aligned.Source: C. Comparison with State-of-the-Art… · p. 5, Figure 3 · p. 5
D. Ablation Study: Effect of Degradation Conditioning
The ablation compares full DDG-LoRA with a version using standard LoRA under identical training settings. Degradation conditioning improves three metrics, while FID is essentially unchanged and NIQE is slightly worse.
D. Ablation Study: Effect of Degradation Conditioning
- Adding DDG improves PI, Tenengrad, and Entropy; FID is 99.00 versus 98.98, while NIQE is 5.619 versus 5.589.The comparison uses models trained on USenhance2023-Aligned under identical settings; the table marks the better result for each metric.Source: D. Ablation Study: Effect of… · p. 5, Table III · p. 5
- The authors conclude that the gains outweigh the small loss, supporting a contribution from the degradation-conditioning module.The module is the DDG component tested against standard LoRA.Source: D. Ablation Study: Effect of… · p. 5
V. CONCLUSION
The conclusion presents a two-stage approach for training an enhancer from unpaired LQ–HQ data, combining synthetic aligned-pair generation with DDG-LoRA. On USenhance2023, the combined system ranks first across five no-reference metrics, with a reported 16.7% FID reduction over the strongest baseline.
V. CONCLUSION
- Stage one uses CycleDiff-based style transfer on unaligned clinical data to synthesize pixel-aligned LQ–HQ pairs while leaving HQ images unchanged.This preserves real clinical images as the supervision target.Source: §V · p. 5
- Stage two extends PiSA-SR with a degradation-conditioned correction matrix in each LoRA branch to adapt enhancement to input-specific speckle and blur levels.The correction matrix C(d) is inserted between A and B.Source: §V · p. 5
- 16.7% lower FID than the strongest baseline, and the combined system ranks first on all five no-reference metrics against seven representative baselines.The ablation reports a further 3.7% Tenengrad gain from DDG-LoRA.Source: §V · p. 5