Source-linked AI summary

From Augmentation to Synthetic Evidence: Evaluating Generated Chest Radiographs at Dataset Intake

Shaikh Akib Shahriyar, Zinedine Sehili, Adam Arthur, Christopher Schwartz

arXiv:2610.29152v1cs.CVcs.AIcs.LG

TL;DR

Generated radiographs may be useful as declared augmentation but become unsupported evidence when offered as records of real examinations. The paper separates quality, predicted origin, and claim support, finding that selection improves quality passes but not passage through both pixel checks.

  • Problem

    Quality and origin checks do not establish that a generated radiograph records a real examination, leaving claim support as a separate intake question.

    Source: §1 · p. 1
  • Method

    The study separates quality, predicted origin, and claim support while testing selection, processing, source changes, and a registry that records source and intended use.

    Source: §1 · p. 2
  • Results

    Quality-based selection raises quality passes, but there is no evidence it increases passage through both receiver checks; processing can shift thresholds and reduce ranking quality.

    Source: §1 · p. 2, §8 · p. 9
  • Takeaways & Limitations

    Passing both pixel checks does not support a claimed acquisition, so source and intended use should be recorded and checked separately.

    Source: §1 · p. 2, §8 · p. 9

    The origin classifiers separate collections and may rely on acquisition or processing shortcuts, so the authors do not call them validated forgery detectors.

    Source: §7 · p. 8

Abstract

from arXiv · show

A generated chest radiograph can be useful as declared augmentation, but the same image becomes unsupported evidence when submitted as the record of a real examination. We study this problem by separating three questions: whether an image passes an automated quality check, whether an origin classifier flags it as generated, and whether a trusted record supports the claimed acquisition. In our experiments, Qwen selects up to 20 images from pools of 1,000 existing Cheff-identified or SynthCheX candidates. A receiver evaluates quality with MedGemma and predicted origin with a classifier on frozen DINOv3 features. Quality-based selection raises quality passes from 12 to 19 of 20 for Cheff-identified images and from 8 to 17 for SynthCheX. We find no evidence that selection improves passage through both checks: no selected Cheff-identified image passes both, while uniform, quality, and quality-constrained surrogate selection produce 2, 1, and 2 SynthCheX passes. On unprocessed test images, the origin classifier flags 4,677 of 5,000 SynthCheX images, but it also misclassify 110 of 2,297 NIH (real) images and 568 of 957 genuine images from another collection as fake. Our results further show that JPEG compression reduces SynthCheX-versus-NIH AUROC from 0.987 to 0.841. Even without processing, 152 SynthCheX images pass both pixel checks. These checks can filter submissions, but they cannot verify an acquisition claim. We strongly advocate that source and intended use must be recorded and checked separately for admission of chest radiographs in medical datasets.

1 Introduction

Automated quality and origin checks do not establish that a radiograph records a real examination. The paper separates those checks from claim support and tests how selection, processing, and source changes affect intake decisions.

1 Introduction

  • Quality-based selection raises quality passes, but the authors find no evidence that it increases images passing both receiver checks.The introduction frames this as the main selection result.Source: §1 · pp. 1–2

4.4 Processing experiment

  • Processing can shift classifier scores enough to invalidate a fixed threshold, while ranking loss can persist after recalibration.The paper distinguishes threshold drift from ranking loss.Source: §1 · p. 2, §4.4 · p. 4

3 Task and threat model

  • A pass on quality and predicted origin does not support the claimed acquisition; only a registry compares the claim with a source record.The experiment treats submitted images as lacking source records by design.Source: §3 · pp. 2–3

4.1 Data and splits

  • The study uses separate fit, development, calibration, and test splits, with NIH and generated-image collections; Open-I is added only for testing.The NIH threshold remains unchanged for the 957 Open-I images, and report groups do not establish unique patients.Source: §4.1 · p. 3

4.2 Models and their roles

  • Qwen ranks candidates, MedGemma checks quality, and frozen DINOv3 features support the main origin-classifier head.MedForge-Reasoner is used only in a separate transfer comparison.Source: §4.2 · p. 3, §4.3 · p. 3

4.3 Origin classifier and threshold

  • The raw CLS head is fit on NIH and Cheff-identified images, then calibrated on disjoint genuine NIH images at a nominal 5% false-alarm rate.SynthCheX is not used to fit that head, making its test a transfer to a second generator.Source: §4.3 · pp. 3–4

4.5 Quality check

  • The receiver’s quality check requires an assessable chest image, visibility of at least 3/4, artifact severity at most 1/4, and a passed flat-image guard.The submitter instead uses Qwen and the original rubric, so its rating differs in both model and rubric.Source: §4.5 · p. 5

4.6 Candidate selection

  • Selection submits at most 20 candidates from nested pools; uniform, quality, and quality-constrained surrogate strategies use only Qwen outputs.The surrogate rating stands in for the origin check, which the submitter cannot query.Source: §4.6 · p. 5

5 Results

Quality-based selection increased quality-check passes, but did not establish an increase in images passing both checks. Origin screening also produced false flags on genuine images, and processing and collection differences affected its results.

5.1 Quality selection increases quality passes

  • Quality selection raised passes from 12 to 19 of 20 Cheff-identified submissions and from 8 to 17 of 20 SynthCheX submissions.This result concerns MedGemma’s quality check alone; Qwen ranked candidates using a different model and rubric.Source: §5.1 · p. 5

5.2 Held-out evaluation of the quality rubric

  • The alternative rubric passed none of 60 invalid inputs without the flat-image guard, versus 5 for the original; held-out results were mixed and did not establish it as a better overall image-quality measure.For clean NIH images, it returned no usable judgment for 3/100 versus 0/100; it ranked undegraded images above degraded copies in 650/1,800 pairs versus 582/1,800, a 3.8-point gain (95% interval [−1.2, +8.8]). It also had more reversals (426 versus 228) and unavailable pairs (41 versus 2).Source: §5.2 · pp. 5–6

5.3 Origin-screen results

  • The origin classifier flagged 110 of 2,297 NIH images and rejected 4,677 of 5,000 SynthCheX images; the combined rule passed 152 SynthCheX images.The combined rule also passed 1,635 NIH images and no Cheff-identified images; passing both checks does not establish the claimed acquisition.Source: §5.3 · p. 6, Table 2 · p. 7

5.4 Selection and combined passage

  • 0 of 20 Cheff-identified images passed both checks under any strategy; SynthCheX had 2, 1, and 2 passes under uniform, quality, and quality-constrained surrogate selection.The resampling intervals overlapped, so the experiment did not resolve small differences among the SynthCheX strategies.Source: §5.4 · p. 6, Figure 3 · p. 7

5.5 Effects of image processing

  • Under JPEG 75, SynthCheX AUROC fell from 0.987 in the canonical condition to 0.841; recalibrating the threshold reduced NIH flags but also reduced SynthCheX rejections.For JPEG 75 with a matched threshold, the raw head flagged 65 of 1,000 NIH images and rejected 486 of 1,000 SynthCheX images.Source: §5.5 · p. 6

5.6 Cross Dataset Performance

  • At the unchanged NIH threshold, the raw classifier flagged 568 of 957 genuine Open-I images; MedForge-Reasoner labeled 178 of 300 genuine NIH images as fake.Open-I contained no generated images, and MedForge-Reasoner provided categorical verdicts without a calibrated 5% threshold.Source: §5.6 · pp. 6–7

6 Source, purpose, and reuse

The registry keeps quality, predicted origin, and claim support separate from the image pixels. In the policy experiment, adding pixel checks reduced acceptance of supported genuine acquisitions while declared augmentation remained allowed.

6 Source, purpose, and reuse

  • The registry stores quality, origin, and claim-support results separately, and paired cases reuse identical pixels while changing the claimed purpose or trusted-source record.The experiment used 8 selected base images, 24 claim-and-purpose variants, and 72 policy replays; variants reused images and were not population estimates.Source: §6 · pp. 7–8
  • A registry-only policy rejected unsupported generated acquisition claims; adding pixel checks reduced accepted supported genuine acquisitions from 4 of 4 to 2 of 4 while declared augmentation stayed allowed.A permissive importer accepted unsupported claims whether verification was retained or removed; a safe importer rehashed received bytes and left missing or invalid verification unknown.Source: §6 · pp. 7–8

7 Limitations

Quality-based selection raises quality-check passes but does not increase passage through both receiver checks; the study also identifies boundaries in data provenance, automated assessment, and reproducibility.

7 Limitations

  • Unknown Cheff provenance and unspecified SynthCheX filtering prevent claims about the Cheff model or Qwen’s independence from the data.The study also makes no claim about memorization, patient uniqueness, or freedom from pretraining exposure.Source: §7 · p. 8
  • Automated Qwen and MedGemma assessments were not judged by clinicians, and the origin classifiers are not validated forgery detectors.The models are not fully independent, and classifier false-alarm rates can differ after collection or processing changes.Source: §7 · p. 8
  • Processing changes geometry, intensity, and frequency together, while unrecoverable historical inputs prevent byte-for-byte reconstruction of some selection reports.The study also did not measure clinical deception, downstream harm, or deployed-system precision.Source: §7 · pp. 8–9

8 Conclusion

  • Quality-based selection increases MedGemma quality passes, but the study finds no evidence that it increases passage through both receiver checks.The paper concludes that passing both pixel checks does not support the claimed acquisition; source and intended use should be recorded and checked separately.Source: §8 · p. 9

S2 Shared definitions and notation

The supplement defines the study’s collections, roles, checks, processing conditions, classifiers, and selection terminology. It also specifies the split and statistical conventions used to report results.

Collections

  • NIH and Open-I are genuine collections, while Cheff-identified and SynthCheX are generated-image collections; “source” refers to a submission’s claimed source.The conditioning of a generated image is the generator input, such as a prompt or label.Source: S2 Shared definitions and notation · p. 12

Roles and checks

  • The submitter selects images, the receiver applies quality and origin checks, and a registry checks claimed source and declared purpose.The origin classifier’s threshold flags images as generated; a generated-image flag is a correct rejection, while a genuine-image flag is a false alarm.Source: S2 Shared definitions and notation · p. 12

Processing and heads

  • Canonical processing converts images to luminance, resizes with aspect-preserving Lanczos resampling, pads to a square, and copies to RGB without histogram equalization.The raw head is fitted without processing augmentation; an augmented head uses five processed views, and thresholds may be canonical, matched, or global.Source: S2 Shared definitions and notation · p. 13

Quality rubrics and selection

  • The original quality rubric combines technical quality, anatomical plausibility, and artifact severity; selection strategies include uniform, quality, and quality-constrained surrogate.The surrogate rating is Qwen’s judgment of how much an image looks like a real acquisition.Source: S2 Shared definitions and notation · p. 13

Splits and statistics

  • The fit, development, calibration, and test splits serve distinct purposes, and AUROC is the probability that a random generated image scores above a random genuine image.Partial AUROC covers false-positive rates up to 0.10 and is reported in standardized form.Source: S2 Shared definitions and notation · p. 13

S3 Dataset and split construction

The study constructs test panels from NIH, legacy Cheff-identified, SynthCheX, and Open-I images, with fixed identifiers, duplicate checks, and stated provenance boundaries.

NIH images

  • After excluding 500 pilot patients, the NIH test split contains 2,297 patients, with at most one image per patient.The authors accepted the smaller split rather than reach a target size using repeated images of one patient.Source: S3 Dataset and split construction · p. 13

Cheff-identified images

  • The Cheff-identified test split uses 1,000 unused images, but the generation checkpoint, conditioning groups, and completeness of raw attempts remain unresolved.The legacy listing differs from the selected pool, and old score files do not establish independent generation runs.Source: S3 Dataset and split construction · p. 13

SynthCheX images

  • SynthCheX splits draw 6,500 rows at random from a 75,649-row release that HealthGPT filtered before the experiment.The filtering checkpoint and conditioning mapping are unspecified, so independence from the Qwen family is unverified.Source: S3 Dataset and split construction · pp. 13–14

Open-I images

  • The Open-I test panel contains 957 frontal report groups and tests a change of genuine collection with NIH calibration unchanged.The panel was not used for threshold fitting and is not a clinical validation.Source: S3 Dataset and split construction · p. 14

Checks on selected images

  • The study fixed reserved identifiers before retrieval, withheld test labels during method development, and checked exact duplicates and recoverable shared-input groups.The extra low-resolution pair comparison cannot establish that images were not memorized or that conditioning was not shared.Source: S3 Dataset and split construction · p. 14

S4 Model and checkpoint details

The study uses pinned model checkpoints and frozen image features, specifies decoding settings, and defines a limited transfer comparison using MedForge-Reasoner.

S4 Model and checkpoint details

  • The study pins Qwen, MedGemma, and DINOv3 checkpoints and runs frozen backbones in BF16 without quantization or inference-parameter offloading.Qwen’s maximum-pixel setting limits its token budget and does not mean every canonical image is upsampled to that area.Source: S4 Model and checkpoint details · p. 14

S4.1 Image features

  • DINOv3 features use either the CLS token or mean patch tokens, while the medical representation averages MedGemma vision-tower patch features before its projector and decoder.The stored feature vector is linked to the exact pixels used to compute it.Source: S4.1 Image features · p. 14

S4.2 Decoding settings

  • Qwen and MedGemma use greedy decoding with token ceilings; development checks supported Qwen’s non-thinking setting but did not establish clinical or causal grounding.The development rubrics produced valid final replies with bounded token use.Source: S4.2 Decoding settings · p. 14

S4.3 MedForge-Reasoner protocol

  • MedForge-Reasoner is a Qwen-family model trained for localized medical-image manipulations, so whole-image generated radiographs test transfer rather than reproduce its benchmark.The final scope was 900 canonical images plus up to 580 unique images from fixed Qwen submissions, with scope choices based on runtime rather than performance.Source: S4.3 MedForge-Reasoner protocol · pp. 14–15
  • Only a unique explicit final verdict maps to a MedForge-Reasoner category; unavailable outputs are excluded from verdict interpretation, and no calibrated false-alarm curve is reported.The protocol does not convert a rationale or self-reported confidence into a probability.Source: S4.3 MedForge-Reasoner protocol · p. 15

S5 Quality prompts and response parsing

The section specifies how Qwen and MedGemma assess image quality, how replies are parsed, and how rubrics are evaluated on development and held-out data. It distinguishes valid unassessable judgments from unavailable replies and defines separate quality-pass and degradation-ordering criteria.

S5.1 How the prompts are sent

  • Qwen and MedGemma receive the image or ordered image pair before the task text, without filenames, labels, reports, claimed origin, or source metadata.The official processor and chat template format the query; native thought text is retained but does not replace final JSON, and responses are not repaired in a favorable direction.Source: S5.1 How the prompts are sent · p. 15

S5.2 Original rubric (Qwen and MedGemma)

  • Qwen applies the rubric to rate every candidate, and MedGemma applies it for comparison; it assesses chest-radiograph pixels for technical quality, visible anatomical plausibility, and conspicuous artifacts.Source: S5.2 Original rubric (Qwen and MedGemma) · p. 15

S5.3 Alternative rubric (primary MedGemma check)

  • The alternative rubric asks whether recognizable chest content is assessable, rates visibility and structural artifacts separately, and does not certify acquisition or origin.It assigns null artifact severity when an image is unassessable and treats uncertain anatomy that could reflect disease or projection as uncertainty rather than automatically as artifact.Source: S5.3 Alternative rubric (primary… · pp. 16–17

S5.4 Reading the replies

  • A reply is unavailable if parsing or runtime fails; unavailable outcomes remain in the denominator and never count as quality passes.The original rubric passes valid, assessable replies at Q ≥75; the alternative requires an assessable chest image, visibility of at least 3, and artifact severity at most 1, with a flat-image guard reported separately.Source: S5.4 Reading the replies · p. 17

S5.5 Ordering rules for degradation tests

  • Degradation ordering compares each undegraded image with its degraded copy, classifying outcomes as strict, tied, reversed, or unavailable.The alternative rubric compares visibility first and artifact severity second; valid unassessable, non-chest, or null-artifact replies are placed at the bottom.Source: S5.5 Ordering rules for degradation… · p. 17

S5.6 Choosing the alternative rubric on development data

  • The alternative rubric plus flat-image guard was selected after passing invalid-pattern and clean-NIH checks and improving pooled ordering, although its SynthCheX ordering count fell.It passed 0 of 60 distinct invalid patterns without the guard, and 1 of 100 clean NIH images lacked an assessable judgment; strict ordering rose to 1,132 of 2,700 pairs from 938, while SynthCheX ordering fell from 323 to 277 of 900.Source: S5.6 Choosing the alternative rubric… · p. 18

S5.7 Held-out validity test and quality under processing

  • On held-out data, the validity panel uses new degradation settings and control seeds, while a separate processing panel assesses quality under JPEG, resizing, and native-resolution conditions.The held-out validity panel includes 1,800 image pairs and 70 non-image exposures covering 60 patterns; the processing panel uses 300 reserved images per collection across four conditions.Source: S5.7 Held-out validity test and… · p. 18

S6 Origin classifier construction and calibration

The classifier heads use frozen features and calibrated thresholds, with separate reference-set qualifications for secondary probes. Development comparisons found limited gains from augmentation and no qualifying gain from the response-feature ablation.

S6.1 Fitting the heads

  • The primary head is fitted on NIH and Cheff-identified images, while secondary heads use separate genuine/generated budgets; logistic-regression regularization is selected on development data.SynthCheX outcomes are excluded from fitting and regularization choice for the primary head, and deterministic solver seeds do not represent independent fits.Source: S6.1 Fitting the heads · p. 18

S6.2 Threshold calibration

  • With 3,000 genuine calibration images, the conformal p-value measures score typicality among genuine images, not acquisition probability, and has no guarantee under arbitrary collection shift.An image is flagged when its calibrated value is at most α; a score equal to the threshold is not flagged.Source: S6.2 Threshold calibration · p. 19

S6.3 Reference sets for the secondary checks

  • Secondary checks use distinct reference sets and qualifications: the rubric-field probe excludes failed replies and lacks the full calibration guarantee, while the pixel-statistic head tests collection differences rather than validated forgery detection.The rubric-field probe’s 1% and 5% thresholds coincide because of ties, and its reference is not independent of earlier rubric development; the pixel statistics are scale-dependent.Source: S6.3 Reference sets for the secondary… · p. 20

S6.4 Augmented head and the response-feature ablation

  • Augmentation raised Cheff-identified rejection by 4.62 percentage points (95% paired interval [3.95, 5.26]) and genuine-image flags by 0.12 points ([−0.58, 0.84]), short of the five-point target.The prespecified comparison was still evaluated, with no further method search or test-split tuning.Source: S6.4 Augmented head and the… · p. 20
  • The response-feature head rejected 500 of 500 Cheff-identified development images versus 495 of 500 for its three-view comparator, while both flagged 49 of 1,000 genuine images.The one-point rejection gain failed the five-point adoption criterion, so no final response arm was adopted.Source: S6.4 Augmented head and the… · p. 20

S7 Selection procedures

Selection strategies rank generated candidates using quality or surrogate judgments, while a bounded pairwise procedure refines only top-score ties. The pairwise development checks met the criteria for adopting that limited refinement.

S7.1 Strategies and denominators

  • Uniform selection samples randomly, quality selection submits the highest original quality scores, and quality-constrained surrogate selection ranks eligible candidates by surrogate rating.Each strategy submits at most K images; when fewer candidates qualify, both actual submissions and the full candidate budget are reported.Source: S7.1 Strategies and denominators · pp. 20–21

S7.2 Surrogate rating prompt

  • The surrogate rating is an uncalibrated ranking of visible acquisition-like appearance, not a verified acquisition record or probability.It considers structural coherence and conspicuous rendering cues while excluding pathology absence, devices, position, filenames, and labels as origin certification.Source: S7.2 Surrogate rating prompt · p. 21

S7.3 Pairwise refinement

  • Pairwise refinement compares images in order and uses pairwise wins to score candidates, but applies only to top-score ties in the 100-candidate, K = 20 condition.It is an automated ranking judgment, not an expert judgment; other budgets retain the ordinal baseline, and a failed or incomparable required query returns that generator’s condition to baseline.Source: S7.3 Pairwise refinement · pp. 21–22
  • All 288 swapped development pairs were comparable, and outcomes stayed the same after swapping in 261 of 288 pairs overall, supporting adoption of the bounded refinement.The projected 1,600-query generation time was 6,116.070 seconds, below the 7,200-second cap; repeated pools were not queried pairwise.Source: S7.3 Pairwise refinement · p. 22

S8 Processing conditions

The processing evaluation defines 13 image conditions and compares threshold policies using fixed calibration attempts and test-image counts. Outputs marked unavailable are neither passes nor detections.

S8 Processing conditions

  • The 13 conditions include JPEG quality levels, resampling, native 512 processing, border and central masks, and padding, with distinct input paths for native 512 images.The severe border mask alters 36.5% of the square and cannot isolate text; the DINOv3 and MedGemma processors remain at 224 and 896 pixels, respectively.Source: S8 Processing conditions · p. 22
  • Four threshold policies compare raw and augmented heads with canonical, matched, or global thresholds, using the same 3,000 genuine calibration attempts and 1,000 test images per collection.The global-threshold guarantee applies only to its mixture of five assigned conditions, not to each condition or an unknown processing shift.Source: S8 Processing conditions · pp. 22–23
  • An unavailable test output is neither a genuine pass nor a detection, and newly fitted heads require all 3,000 calibration scores to be finite.The rubric-field probe is an exception, using only valid replies.Source: S8 Processing conditions · p. 23

S9 Statistical procedures

The section specifies uncertainty and resampling procedures, representation comparisons, and dependence limits for the reported analyses.

  • Rates count attempted images, including zeros, with Wilson 95% intervals; a fixed-list interval does not capture selection variability.Source: S9 Statistical procedures · p. 23
  • Bootstrap intervals resample base images while keeping related views and claim variants together; selection intervals also resample candidate groups and repeat selection.The 100 repeated pool draws assess sensitivity to pool membership, not independent datasets; intervals condition on fixed fitted models and thresholds.Source: S9 Statistical procedures · p. 23
  • MMD uses an unbiased cubic-kernel estimate on 500-image panels, with a disjoint NIH–NIH reference; values are comparable only within encoder and normalization.Negative estimates are retained, and angular sensitivity analysis is secondary.Source: S9 Statistical procedures · p. 23
  • NIH–NIH interval spread is descriptive only and does not test whether the two NIH panels are equal.Source: S9 Statistical procedures · p. 23
  • Nearest-neighbor examples compare queries with NIH fit images and show representation association, not clinical similarity or proof that memorization is absent.The limited reference cannot establish that memorization is absent elsewhere.Source: S9 Statistical procedures · p. 23
  • Origin-score correlations with pixel statistics are observational, while projection-standardized rates use fixed development weights and report observational strata.The analysis does not pool collections into a causal explanation; report-derived finding labels are not independent labels of normality or quality.Source: S9 Statistical procedures · p. 23
  • Views, repeated pool draws, claim variants, and repeated heads on the same images are not independent observations.Tables retain them so every denominator is visible.Source: S9 Statistical procedures · p. 23

S10 Extended results and diagnostics

The extended results examine how quality and origin diagnostics vary across image strata and with processing, threshold, encoder, and representation choices. They characterize these checks without establishing a claimed acquisition or causal explanation.

S10.1 Quality on the test split

  • All three evaluators rated generated images lower on average; 53%–92% passed each evaluator, yet fewer than half of generated images reached the top score under Qwen or the alternative rubric.Qwen-selected lists had higher MedGemma pass rates than uniform lists under both rubrics, while SynthCheX images with original-rubric score 100 spanned nearly the full origin-score range of 0–1.Source: S10.1 Quality on the test split · p. 24, Table S2 · p. 25, Table S3 · p. 25

S10.2 Held-out quality validity and quality under processing

  • With the flat-image guard, both rubrics passed none of 60 invalid patterns; without it, only the alternative rubric passed none, while the original passed 5.The alternative rubric returned assessable judgments for 97 of 100 clean NIH images versus 100 of 100, and its strict ordering improved for NIH and Cheff-identified pairs but fell for SynthCheX pairs.Source: S10.2 Held-out quality validity and… · p. 24, Table S7 · p. 29, Table S6 · p. 29

S10.3 Origin check: thresholds, ranking, and transfer

  • Canonical origin heads achieved SynthCheX AUROC of 0.9699–0.9891, but the primary head flagged 110 of 2,297 NIH images at 5% and learned heads flagged 15.9%–59.4% of genuine Open-I images.At 1% and 10%, the primary head flagged 26 and 200 NIH images and rejected 4,339 and 4,780 SynthCheX images. Heads fitted on one generator passed more images of the other than heads fitted on both, but the test generators also contributed to the latter’s fitting data.Source: S10.3 Origin check: thresholds,… · p. 24, Table S11 · p. 31, Table S13 · p. 33

S10.4 MedForge comparison

  • MedForge labeled 178 of 300 genuine NIH images fake and passed 127 of 300 Cheff-identified and 50 of 300 SynthCheX images.On fixed selected lists, it passed 4, 4, and 9 of 20 Cheff-identified images and 1, 0, and 5 of 20 SynthCheX images for surrogate, quality, and uniform selection; these counts are not comparable to main-paper origin checks because MedForge lacks a matched false-alarm target.Source: S10.4 MedForge comparison · p. 37, Table S22 · p. 39, Table S23 · p. 39

S10.5 Processing and selection tables

  • Across 13 conditions, the full policy change lowered NIH flags by 35.33 points and SynthCheX rejection by 4.00, while threshold-only recalibration lowered them by 34.01 and 19.45 points.At JPEG 50, matched thresholds reduced SynthCheX rejection from 998 to 355 of 1,000; the augmented head retained 845 rejections, with 52 NIH flags against 68. At B = 1,000 and K = 20, no DINOv3 or medical head passed both checks for any Cheff-identified image or more than 2 of 20 SynthCheX images; probe baselines passed many more.Source: S10.5 Processing and selection tables · p. 37, Table S24 · p. 40, Table S25 · p. 41

S10.6 Recorded strata and score shifts

  • Projection and findings are recorded metadata, and submitter-declared projection is not used as a shortcut to acceptance.The section describes strata, paired score shifts, and within-collection pixel-statistic correlations, but the supplied passages do not report their results.Source: S10.6 Recorded strata and score shifts · pp. 37, 41
  • Under JPEG 75, the raw head’s median origin score shifted 0.200 (95% interval 0.179 to 0.221), and under resize 128 it shifted 0.295 (0.251 to 0.345), versus a canonical interquartile range of 0.00604.Border, padding, and central-mask conditions changed the median score by 0.001 or less.Source: S10.6 Recorded strata and score shifts · p. 41
  • Quality pass rates were 47.50% for AP versus 83.14% for PA, and 83.44% for No Finding versus 55.05% for other findings; these observational strata do not show causation.Source: S10.6 Recorded strata and score shifts · p. 41
  • DINOv3 heads flagged 3.58%–5.97% of NIH images across strata, the medical head’s rate ranged from 2.38% to 9.82%, and the largest pixel-statistic/origin-score correlation was 0.415 in absolute value.Source: S10.6 Recorded strata and score shifts · p. 41

S10.7 Representation proximity and screening examples

  • DINOv3 CLS placed Cheff-identified images closer to NIH than SynthCheX under both normalizations, but the raw medical representation reversed the ranking and three other paired contrasts included zero.No encoder gave a universal ranking of the two generators.Source: S10.7 Representation proximity and… · p. 41

S11 Registry experiment

The registry experiment separates pixel checks from source support, then tests how policies treat paired claims and trusted records.

S11.1 Design

  • Paired cases keep pixels identical while changing purpose, claim, or trusted-record availability; quality, origin, and claim support remain separate fields.An unsupported acquisition claim cannot obtain a source record by passing pixel checks.Source: S11.1 Design · pp. 41, 51
  • Tables S26–S27 report both-check passes for fixed lists at B = 1,000 and K = 20 across seven heads and three strategies; surrogate selection uses a different quality check than MedGemma.Each head uses its own nominal 5% threshold, and unavailable replies remain in denominators.Source: Table S26 · p. 42, Table S27 · p. 42
  • Supplementary analyses cover NIH origin flags and quality passes by projection and recorded findings, paired score shifts under processing, and observational score–pixel-statistic correlations.The strata are observational, and the correlations do not establish a causal mechanism.Source: Table S28 · p. 43, Table S29 · p. 43, Table S30 · p. 44
  • Tables S31–S33 are parts 2–4 of Table S30; Table S34 reports unbiased squared MMD for fixed 500-image panels, comparing generated collections with NIH using raw and angular-normalized encoder features.S34 values are comparable only within one encoder and normalization, and may be negative; in S35, a negative Cheff-minus-SynthCheX difference means Cheff-identified images are closer to NIH.Source: Table S31 · p. 45, Table S32 · p. 46, Table S33 · p. 47
  • Nearest-neighbor examples use NIH fit images only, and the four-outcome examples show model outcomes rather than clinical assessments.Patient-disjoint splits do not make representation proximity proof that memorization is absent.Source: Figure S5 · p. 49, Figure S6 · p. 50
  • The registry binds authenticated source and derived-file records to files by hash, while the safe importer rehashes received bytes and keeps missing verification unknown.Registration does not certify clinical history or implement C2PA; a permissive importer trusts unsupported claims.Source: S11.1 Design · p. 51

S11.2 Results

  • The registry-only policy rejects unsupported generated observation claims; adding pixel requirements lowers accepted supported genuine observations from four of four to two of four, while declared augmentation stays allowed.The 72 replays use 8 selected base images and 24 claim-and-purpose variants, so their counts are controls rather than population estimates.Source: S11.2 Results · p. 51

S12 Reproduction and execution records

S12 records execution events, reproducibility boundaries, and measured costs, distinguishing their effects on runs from evidence about assessor validity.

S12.1 Execution records

  • A shared GPU process may have affected timing in the completed original-rubric development stress run.GPU dispatch checks ownership and waits for free devices, with 300-second monitoring and safe pauses between owned batches.Source: S12.1 Execution records · p. 51
  • The first eight alternative-rubric replies preceded the fresh-GPU-ownership check and were preserved without rescoring; later loaders enforced that check before model construction.The records describe hardware use, not assessor validity.Source: S12.1 Execution records · p. 51
  • Earlier pilot-head outputs used a 3,000-patient reference overlapping fitting and selection, so reported pilot-head evaluations instead use the 2,914-patient reference.The overlapping outputs remain descriptive records only.Source: S12.1 Execution records · p. 51
  • The execution record includes repaired development controls, revised parsing of four saved MedForge replies, and a read-only adapter for empty checkpoint device-map metadata.The controls repair added ten patterns without changing the original inputs or cached outputs; the adapter did not alter files or categorical results.Source: S12.1 Execution records · p. 51
  • The surrogate allocation’s total wall time is unavailable, and interrupted runs without reliable end times have no reported total wall time.The surrogate attempts combine a preserved prefix of 1,748 with a completed suffix of 4,252.Source: S12.1 Execution records · pp. 51–52
  • The exact earlier metadata file for primary raw DINOv3 CLS and mean heads was unrecoverable, preventing byte-for-byte reconstruction despite verified checkpoints and data identities.The fit, development, and calibration identities were verified.Source: S12.1 Execution records · p. 52

S12.2 Computing costs

  • Measured costs cover selected stages, not a project-wide wall-time or GPU-hour estimate, and tasks may reuse images.The inference policy requires standard GPU-resident BF16 weights without quantization or parameter offloading; data preparation, tokenizer replay, and cached-feature statistics run separately.Source: S12.2 Computing costs · p. 52, Table S36 · p. 53
Loading 2610.29152v1…