PaperlayerView exact revision on arXiv

Source-linked AI summary

Spatially Gated Diffusion for Localized Counterfactual Chest Radiograph Editing

Kamran Ullah Afaq, Basit Raza

arXiv:2610.00805v1eess.IVcs.CV

TL;DR

Chest-radiograph editing must make a requested change while preserving unrelated content. The paper combines recurrent mixing of source and edit trajectories with separate image-space composition, and reports higher joint validity than a matched composition-only control in the assessed MIMIC-derived cohort.

  • Problem

    Chest-radiograph editing must satisfy a requested change while retaining unrelated structures, since target completion or preservation alone is insufficient.

    Source: §1 · p. 1

  • Method

    A learned gate mixes source and instruction-conditioned trajectory candidates, while a separate applied mask composites the decoded proposal with the source.

    Source: §1 · p. 2, §4.4 · p. 8

  • Results

    93.5% joint validity for the full editor exceeded composition only by 1.7 percentage points (paired patient-cluster 95% interval, 0.6–2.8) on 2,400 requests.

    Source: §7 · p. 13

  • Takeaways & Limitations

    The intervention results distinguish learned spatial mixing's influence on the denoising proposal from the additional preservation route provided by final composition.

    Source: §6 · p. 13

    The evidence is confined to the assessed MIMIC-derived cohort; independent-site and clinical validation remain necessary for broader use.

    Source: §6 · p. 13, §7 · p. 13

Abstract

from arXiv · show

Editing a chest radiograph requires completing the requested change while preserving unrelated content. We study a latent diffusion editor with an instruction-independent source trajectory and an instruction-conditioned editing trajectory. A learned gate mixes their post-sampler candidates at each executed step, and a separate image-space mask composites the decoded proposal with the source. On 2,400 MIMIC-derived requests, 2,244 outputs met the joint target, preservation, quality, and coverage rubric (93.5%), and target completion was 97.4%. Joint validity exceeded a matched composition-only control by 1.7 percentage points (paired patient-cluster 95% interval, 0.6--2.8). At a fixed learned mask, the learned-gate proposal improved joint validity by 1.6 points (0.8--2.4); at a fixed learned proposal, the learned mask improved it by 2.6 points (1.7--3.5). Across three training seeds, mean joint validity was 93.5% with a 0.3-point sample standard deviation. A blinded 240-request assessment yielded adjudicated joint validity of 93.3% for the full editor and 91.7% for composition only. Protected-region mean absolute error decreased from 0.0190 in the raw proposal to 0.0075 after composition. These findings distinguish recurrent-gating effects on the proposal from preservation through final composition in the assessed cohort.

1 Introduction

Chest-radiograph editing must complete a requested change while preserving unrelated content, since either success or preservation alone is insufficient. The paper separates recurrent latent gating from final image composition and evaluates their contributions with paired interventions.

1 Introduction

  • An edit must both satisfy the request and preserve unrelated structures; target completion or source copying alone is insufficient.The paper frames counterfactual editing as conditional visual editing, not inference about a patient's physiological response.Source: §1 · pp. 1–2
  • Spatial maps may control attention, denoising computation, or final blending, and a plausible-looking map alone does not show that it controls the proposal.Source: §1 · p. 1
  • The editor couples an edit-independent factual source trajectory with an instruction-conditioned trajectory, then composes decoded proposal y with source image x using a separate mask.Source: §1 · pp. 1–2
  • Paired interventions vary the learned or replacement gate and applied mask separately, distinguishing proposal-route effects from gains due to copying source pixels.Source: §1 · p. 2

2 Related Work

Related work distinguishes recurrent spatial control from masking and source preservation, and motivates evaluation that separates functional sensitivity from localization agreement and broader validity.

2.1 Spatial Control in Diffusion Editing

  • Prior editing methods use masking or source preservation, while this work studies recurrent mixing of candidate next states with a learned source-dependent gate.Its intervention design separates that computation from final image composition.Source: §2.1 · p. 2

2.2 Medical Counterfactual Generation

  • Medical editors span controlled biomedical shifts, region-specific instruction editing, disease and device editing, longitudinal instruction learning, reward-guided optimization, and causal formulations.Longitudinal image pairs do not identify patient-specific causal outcomes.Source: §2.2 · p. 2

2.3 Interpretation and Evaluation

  • A gate intervention measures functional sensitivity, while spatial overlap measures agreement with an independent annotation; neither alone establishes clinical correctness.The section also motivates jointly measuring editing accuracy, context preservation, and visual quality.Source: §2.3 · p. 3

3 Spatially Gated Editor

The editor maintains separate source and edit trajectories, mixes their post-sampler candidates with a learned gate, and composites the decoded proposal with the source. Training and evaluation specify distinct masks, paired observational targets, and implementation controls.

3.1 Task and Notation

  • The task represents a frontal radiograph and an insert-or-remove instruction, returning a raw proposal, gate sequence, applied mask, and final composite.Eligibility requires a source finding state compatible with the requested operation.Source: §3.1 · p. 3

3.2 Independent Source and Conditional Trajectories

  • Both trajectories start from the same source-noised latent, but the source branch uses a null instruction while the edit branch is instruction-conditioned.At each executed step, the shared source-conditioned predictor and DDIM sampler produce separate candidates; the source recurrence is independent of the requested edit for fixed inputs and parameters.Source: §3.2 · pp. 3–4

3.3 Learned Gate and Recurrent Mixing

  • The gate mixes post-sampler source and edit candidates into the next editing state, controlling generation before decoding; later denoising can spread its effect beyond the gated location.For fixed candidates, zero selects the source candidate and one selects the edit candidate.Source: §3.3 · p. 5

3.4 Applied Mask and Final Composition

  • Final composition uses xcf = (1 − A) ⊙ x + A ⊙ y, retaining source values wherever the evaluation mask is zero.This compositing identity does not guarantee that the chosen region is correct; the training sigmoid generally has support outside the eventual evaluation region.Source: §3.4 · p. 5

3.5 Paired-Target Training and Objective

  • Training uses observational longitudinal pairs with matched projection and verified requested changes, but acquisition variation and undocumented findings prevent treating them as proof that all other visible content is unchanged.The objective applies protected-region losses to both raw proposal y and final composite xcf; annotations and auxiliary evaluators are absent from inference inputs.Source: §3.5 · p. 6

3.6 Implementation

  • The implementation adapts SDXL base 1.0 with LoRA, sharing parameters across separate factual and conditional paths.The paired intervention fixes the source, request, noise, weights, sampler grid, guidance, and mask policy; the composition-only control is separately trained with the same backbone, data, budget, and test requests.Source: §3.6 · pp. 6–7, Table 2 · p. 7

4 Data and Evaluation

The evaluation uses patient-separated MIMIC-derived requests and blinded image-level review, with outcomes defined by a joint validity rubric. A factorial intervention distinguishes proposal-route and composition comparisons, while paired patient-cluster intervals quantify uncertainty for the assessed requests.

4.1 Data Sources and Eligibility

  • MIMIC-CXR-JPG v2.1.0 and MS-CXR v1.1.0 were used in a patient-separated evaluation of 2,400 requests from 1,200 patients across six findings, covering insertion and removal.MS-CXR provides 1,162 image–sentence annotation pairs with phrase-linked bounding boxes on a MIMIC-CXR subset and shares patient ancestry with MIMIC-CXR, rather than being an independent site or dense lesion-mask dataset. Eligibility excludes uncertainty and requires the requested operation to be compatible with the source; label uncertainty remains distinct from an explicit negative.Source: §4.1 · pp. 7–8

4.2 Evaluation Reliability

  • Two readers blinded to method identity assessed 240 held-out requests for target completion, unintended non-target change, structural and artifact quality, and joint validity.Disagreements were adjudicated; the assessment tests task compliance and visual plausibility, not physiological response.Source: §4.2 · p. 8

4.3 Joint and Component Outcomes

  • A valid edit requires a usable output, target completion, no unintended non-target change, and a pass on structural and artifact quality.All eligible requests, including generation failures, remain in the denominator; protected-region MAE and SSIM use independently specified non-target regions, not the applied mask's complement.Source: §4.3 · p. 8

4.4 Factorial Intervention

  • The factorial intervention compares learned and replacement gates at a fixed applied mask, and learned and replacement masks at a fixed proposal.Changing both routes cannot isolate the latent route; area matching does not guarantee matched gate shape, location, or image-space mask area, and conditional contrasts may interact.Source: §4.4 · pp. 8–9

4.5 Uncertainty and Scope of Inference

  • Paired patient bootstrap intervals retain each sampled patient’s requests, while three-seed sample standard deviations summarize a separate source of variation.The patient-cluster intervals describe paired sampling uncertainty for the assessed requests.Source: §4.5 · p. 9
  • 1.7 percentage points in JVR (paired requestweighted patient-cluster 95% interval, 0.6–2.8) separates the full editor from composition only.The contrasts are exploratory and are not adjusted for multiple comparisons.Source: §4.5 · p. 9

5 Results

The full editor achieved high joint validity across the assessed requests, with comparisons and interventions separating recurrent-gating and image-composition results. Blinded assessment, subgroup rates, ablations, and preservation accounting further characterize its performance.

5.1 Current Checkpoint and Local Comparators

  • 93.5% JVR (2,244/2,400) for the full editor exceeded composition-only at 91.8%; RadEdit’s 82.7% was a local rerun.All methods had complete output coverage; external-method scores are local evaluations, not published benchmark scores.Source: §5.1 · p. 9, Table 3 · p. 10

5.5 Separating Latent Editing from Composition

  • At fixed AG, the learned-gate proposal reached 93.5% JVR versus 91.9% for Y (Q), a 1.6-point difference (0.8–2.4).At fixed proposal, changing the applied mask from AQ to AG yielded 2.6 points (1.7–3.5); the complete contrast was 6.0 points.Source: Table 4 · p. 10, §5.5 · p. 12

5.2 Paired Contrasts and Training Runs

  • Across three seeds, full-editor JVR was 93.5% (SD 0.3 points) versus 91.8% (SD 0.3) for composition-only; paired differences ranged 1.6–1.7 points.Patient-cluster intervals quantify within-run sampling uncertainty, while across-seed dispersion measures training-run variation.Source: §5.2 · p. 9, Table 8 · p. 11

5.3 Blinded Reader Assessment

  • Adjudicated joint validity on 240 blinded requests was 93.3% for the full editor, 91.7% for composition-only, and 82.1% for RadEdit.The assessment also reported component judgments and pre-adjudication reader agreement.Source: Table 9 · p. 11, §5.3 · p. 11

5.4 Current-Checkpoint Findings and Operations

  • Insertion yielded 94.7% valid edits and removal 92.3%; pleural-effusion insertion was highest at 98.1%, while atelectasis removal was lowest at 87.5%, descriptively.The direction totals were 1,136/1,200 for insertion and 1,108/1,200 for removal.Source: §5.4 · p. 11, Table 10 · p. 12

5.6 Mechanism Ablations

  • Removing the target objective reduced JVR to 77.8%; time-invariant gating reached 90.7% versus 93.5%, and removing preservation raised final MAE to 0.0145.The full editor’s raw/final MAE was 0.0190/0.0075, compared with 0.0340/0.0084 for composition-only.Source: Table 5 · p. 10, §5.6 · p. 12

5.7 Preservation and Outcome Accounting

  • Protected-region MAE/SSIM shifted from 0.0190/0.963 for raw y to 0.0075/0.986 for xcf; device retention was 476/480 (99.2%).The priority-coded ledger recorded 62 target failures, 19 co-edit failures after target success, 75 quality failures, and 2,244 valid edits; the co-edit count is conditional, not the marginal rate.Source: Table 6 · p. 10, Table 7 · p. 11, §5.7 · pp. 12–13

6 Discussion

The discussion distinguishes recurrent spatial mixing, which influences the denoising proposal, from final composition, which adds a preservation route. It cautions that the evidence does not establish universal superiority or clinical validity.

6 Discussion

  • Learned spatial mixing influences the denoising proposal, while final composition adds a separate preservation route.Copying source pixels where A = 0 is an algebraic property of composition, not evidence of correct lesion localization; frozen-evaluator success need not mean a medically appropriate change.Source: §6 · p. 13
  • The local comparison does not establish universal superiority because the cited methods use different objectives, tasks, and metrics, and newer methods are not measured baselines.Spatial gating itself has substantial prior art; the closest methodological claim is combining learned recurrent mixing with a test that holds the final mask fixed.Source: §6 · p. 13
  • The grouped ablation controls do not isolate their component perturbations, and three training runs provide only a limited estimate of optimization variability.The main cohort and factorial results also depend on the study rubric and its evaluators.Source: §6 · p. 13
  • Because all requests derive from MIMIC, the evaluation does not measure transfer to an independent institution; longitudinal pairs may also differ in acquisition, positioning, treatment, or unrecorded findings.MS-CXR is a subset of MIMIC.Source: §6 · p. 13
  • The 240-request reader study assesses image-level task compliance, not patient-specific causal validity, and generated images are not patient-specific treatment outcomes.Agreement with the study rubric alone does not establish independent clinical validity.Source: §6 · p. 13

7 Conclusion

The conclusion reports that the full editor outperformed composition only on joint validity in the assessed 2,400-request cohort. The evidence remains confined to MIMIC-derived requests, with external-site and clinical validation still needed for broader use.

7 Conclusion

  • 93.5% joint validity on 2,400 requests exceeded composition only by 1.7 percentage points (paired patient-cluster 95% interval, 0.6–2.8).Fixed-mask and fixed-proposal interventions distinguished the contribution of each route.Source: §7 · p. 13
  • The reader assessment and three-seed evaluation complement the cohort findings.The passage does not report their numerical results here.Source: §7 · p. 13
  • The evidence is confined to the assessed MIMIC-derived cohort; external-site and clinical validation are required for broader use.Source: §7 · p. 13

Data Availability

Access to MIMIC-CXR-JPG and MS-CXR is governed by PhysioNet credentialing and data-use terms. Radiographs and redistributable derived material remain subject to the source datasets’ terms.

Data Availability

  • MIMIC-CXR-JPG and MS-CXR access requires PhysioNet credentialing and compliance with data-use terms; radiographs and redistributable derived material remain subject to source-dataset terms.Source: Data Availability · p. 14

Ethics and Intended Use

The study uses de-identified, access-controlled radiographs under the applicable PhysioNet data-use agreement. Generated images are research artifacts, not authentic future patient records or materials for patient care.

Ethics and Intended Use

  • Generated images do not depict observed future patient states and should not be presented as authentic clinical records or used to support patient care.Release of derived images or checkpoints requires assessment of memorization, re-identification, and redistribution constraints.Source: Ethics and Intended Use · p. 14

Related papers

Loading 2610.00805v1…