Source-linked AI summary
Towards Autonomous and Auditable Medical Imaging Model Development
Shengyuan Liu, Jia-Xuan Jiang, Boyun Zheng, Cheng Wang, Zipei Wang, Wentao Pan, Hongtao Wu, Houwen Peng, Yu Gu, Lichao Sun, Yixuan Yuan
TL;DR
Medical imaging model development requires domain-conditioned experimentation and strict validation and artifact requirements that generic agentic MLE does not directly address. AMID uses data-conditioned method planning and verification-controlled optimization to produce auditable model artifacts, improving over the strongest listed baseline on 19 of 20 challenges and tying the remaining task.
Problem
Medical imaging model development requires modality- and task-specific engineering because preprocessing, models, validation, metrics, and prediction artifacts depend on clinical and data constraints.
Method
AMID performs data-conditioned planning and verification-controlled search over executable method lanes, producing model packages with validated evidence and submission-ready prediction artifacts.
Results
AMID produced valid accepted results on all 20 challenges, improved over the strongest listed baseline on 19, and tied it on the remaining task.
Takeaways & Limitations
The results suggest that domain-conditioned planning and artifact-level verification are important ingredients for autonomous, auditable medical imaging model development.
Takeaways & Limitations
The main limitation is cost, and the experiments do not exhaustively evaluate every challenge under all latest model-runtime combinations.
Abstract
from arXiv · showhide
Large language model (LLM) agents are beginning to automate machine learning engineering (MLE) by coupling planning, code execution, debugging, and empirical feedback. Translating this capability to medical imaging remains difficult because each task imposes modality-specific experimentation and strict requirements for validation protocols and prediction artifacts. Here we introduce AMID, an autonomous multi-agent framework for medical imaging model development. AMID first proposes Data-Conditioned Method Planning, which refines coarse task-level search spaces into executable, parallelizable method lanes grounded in task-specific data analysis and runnable medical-imaging resources. It then develops Verification-Guided Two-Stage Optimization, moving from broad early exploration of diverse method lanes to selective exploitation of promising candidates while enforcing strict verification of validation protocols, metric computation, and prediction artifacts throughout the optimization. Across 20 medical imaging challenge tasks spanning diverse modalities and prediction types, AMID outperformed evaluated general-purpose MLE systems and, on several tasks, approached or matched strong human-designed challenge solutions. These results suggest that AMID can turn task-specific medical imaging model development from bespoke manual engineering into an agentic workflow for producing high-performing and auditable model artifacts across heterogeneous tasks.
1 Introduction
Medical imaging model development requires agents to account for modality-, clinical-, data-, and submission-specific constraints beyond generic experimentation loops. AMID addresses these challenges through data-conditioned method planning and verification-guided optimization that produce auditable, executable model artifacts.
- Medical imaging development couples preprocessing, model choice, validation, metrics, and prediction artifacts to modality, clinical objective, annotation process, and patient-level data organization.
- Generic autonomous MLE systems construct search spaces that are too coarse for heterogeneous tasks because suitable methods depend on modality, anatomy, acquisition, annotations, clinical targets, and submission constraints.
- AMID’s Data-Conditioned Method Planning refines coarse task-level spaces into executable, parallelizable method lanes grounded in task-specific data analysis and relevant medical-imaging resources.
- Verification-Guided Two-Stage Optimization progresses from broad exploration of diverse method lanes to selective exploitation of promising candidates.
- Reviewer checks enforce validation protocols, metric computation, and prediction or submission artifacts before experiments drive promotion or final selection.
- AMID produces auditable model packages containing executable code, validation evidence, and submission-ready prediction artifacts across heterogeneous tasks and modalities.
2 System Overview
AMID converts a medical-imaging dataset and task definition into an auditable model package through data profiling, executable method planning, staged optimization, and artifact verification. Its workflow searches data- and resource-grounded method lanes while enforcing task-contract compliance before promotion or final selection.
- System workflow: AMID records the modeling target, evaluation metric, prediction schema, and output contract, then profiles modality, geometry, supervision, organization, labels, and risks.The input may include diverse 2D, 3D, pathology, detection, segmentation, classification, graph-label, and image-quality tasks.
- Two-Stage Optimization: Two-Stage Optimization progresses from broad exploration across diverse method lanes to selective exploitation of promising candidates.The portfolio can include conservative baselines, foundation-model adaptation, task-specific architectures, preprocessing or post-processing lanes, and ensembles.
- Verification and auditability: Reviewer checks verify validation protocols, metric computation, and prediction or submission artifacts before attempts affect promotion, repair, or final selection.A high local score is insufficient unless the score is comparable, the split is legal, the prediction format matches the evaluator, and model lineage is inspectable.
- Data-Conditioned Method Planning: Data-Conditioned Method Planning converts task and data evidence plus executable resources into parallelizable method lanes tailored to observed constraints.Diagnostics become search predicates, while resources include local toolkits and curated foundation-model or external-resource registries.
- Data-Conditioned Method Planning: Each method lane specifies its hypothesis, motivating conditions, resources, implementation requirements, validation obligations, budget, artifacts, and likely failure modes.This shifts workers from open-ended brainstorming toward executable routes that are medically plausible and operationally runnable.
3 Experiments
Across 20 heterogeneous medical-imaging challenges, AMID produced valid accepted results for every task and exceeded the strongest listed autonomous MLE baseline on 19, tying it on one. Controlled comparisons and case analyses indicate that its advantage comes from task-specific method search, adaptation, and artifact verification, while performance remains weaker on several structurally difficult tasks.
- Evaluation setup: 20 medical-imaging tasks spanning diverse modalities and prediction types form the ReX-MLE evaluation suite for comparing AMID with three autonomous MLE systems.The suite includes segmentation, detection, classification, image-quality assessment, and enhancement across panoramic X-ray, CT, MRI, CTA, MRA, histopathology, ultrasound, and microscopy.
- Full-suite results: 19 of 20 tasks improved over the strongest listed baseline, with the remaining TopCoW-CTA-Cls task tied; segmentation gains reached +0.89 Dice on SEG.A and +0.67 Dice on ISLES’22.AMID’s gains were most pronounced on segmentation and detection, where completing the medical-imaging pipeline often bottlenecks general-purpose agents.
- Controlled comparison: 0.08 AP versus 0.49 AP on DENTEX, 0.52 Dice versus 0.64 Dice on PUMA-T1-Seg, and 0.39 accuracy versus 0.50 accuracy on TopCoW-MRA-Cls favored AMID under unified GPT-5.5 baselines.The comparison holds the underlying model backend fixed for AIDE, ML-Master, and R&D-Agent, while comparing their best baseline scores with the best AMID row.
- Strong cases: Task-matched search and adaptation produced strong solutions, including DENTEX detector-plus-classifier pipelines and pathology models using foundation encoders, lightweight heads, and classical classifier ensembles.PUMA nuclei detection reached macro-F1 scores of 0.54359 for the three-class task and 0.28422 for the ten-class task.
- Failure study: USenhance remained weak despite valid outputs, with LNCC 0.18658, SSIM 0.38495, and PSNR 17.12416, while TopCoW classification reached only 0.46296 anterior accuracy and 0.16667 posterior accuracy.The failures involved unstable adversarial training and a coupled vessel-segmentation, topology-extraction, and graph-classification bottleneck for small anatomical structures.
4 Discussion
AMID frames medical-imaging model development as a search over executable, verifiable candidates, with verification shaping optimization rather than merely checking final outputs. Its challenge-benchmark design extends to broader deployment artifacts, while long-horizon experimentation remains costly.
- Medical model development as a search problem: AMID treats medical-imaging model development as a search over executable and verifiable solution candidates, not free-form idea generation or one prompt-to-code task.Candidates must account for task-specific factors including modality, anatomy, annotation semantics, data organization, metrics, scale, and compute budget.
- Verification as the control layer: Verification links validation scores to committed code, logs, fold or protocol records, metric computation, prediction artifacts, and lane attribution.Reviewer checks assess metric direction, legal data access, fold provenance, complete outputs, prediction schema, and traceability to final packages.
- Verification as the control layer: Reviewer checks change optimization itself: candidates failing protocol requirements are routed to diagnosis or removed rather than treated as valid evidence.Verification therefore functions as an internal validity layer throughout both optimization stages, not only as a final safety check.
- From fine-grained prompting to agent-loop orchestration: AMID shifts long-horizon agent engineering from finely designed prompts for individual steps toward orchestration using coding-agent runtimes with persistent workspaces, terminals, version control, tool use, and iterative repair.The remaining design problem is coordinating the agent loop rather than hand-writing every individual step.
- Scope beyond challenge submissions: AMID is not tied to challenge participation: deliverables can include submission files, trained model packages, Dockerized inference pipelines, prediction services, or evidence bundles for human review.Fixed-time-budget evaluation is one reproducible deployment mode, while open-ended use can allow additional operation beyond challenge settings.
- Limitations and next steps: The main limitation is cost, because long-horizon medical MLE consumes GPU time and coding-agent tokens while exploring method lanes, running reviewer checks, and preserving evidence across workers.Current experiments do not exhaustively evaluate every challenge under all latest model-runtime combinations; full-suite results use the primary backend.
5 Related Work
Related work spans autonomous empirical software agents, end-to-end autonomous research systems, and medical autoresearch systems. AMID addresses medical model-development optimization and verification through executable evidence and artifact auditing.
- Autonomous empirical software and MLE agents: Autonomous empirical software agents use execution-feedback loops in which code, logs, tests, and validation scores provide evidence for progress.The passage situates expert-level empirical software writing, code-space exploration, and AI-driven systems research within this paradigm.
- Autonomous empirical software and MLE agents: ML-Master, AutoMLGen, MLEvolve, and Arbor extend autonomous MLE through competing branches, graph or tree search, specialized agents, and stored hypotheses and evidence.These systems support branch maintenance, cross-branch propagation, specialized draft and debugging roles, and evidence storage.
- Autonomous research systems: Autonomous research systems generalize empirical software loops into scientific workflows covering hypothesis generation, literature grounding, experiment design, execution, interpretation, drafting, and review.Research-agent benchmarks have operationalized this broader workflow.
- Medical autoresearch systems: Medical autoresearch systems aim to help clinicians translate clinical or imaging questions into usable models, analyses, and manuscript-ready evidence through runnable workflows and auditable artifacts.The passage conditions this benefit on producing runnable development workflows and auditable artifacts.
- Medical autoresearch systems: AMID focuses on optimization and verification by auditing artifacts and promoting, rejecting, or accepting model packages based on executable evidence rather than clinician steering alone.This positions AMID within medical autoresearch while emphasizing evidence-based package decisions.
6 Conclusion
AMID frames medical imaging model building as a data-conditioned, verification-controlled search process that produces auditable outputs. The report presents the framework and initial benchmark evidence while emphasizing that it is an initial technical report, not a definitive finished system.
- Framework contribution: AMID combines task contracting, medical-imaging data profiling, executable method-lane construction, staged search, and reviewer-checked acceptance of final artifacts.The framework coordinates broad exploration with selective exploitation and accepts outputs only when required protocols and artifacts are satisfied.
- Scope and limitations: The manuscript establishes the framework and first benchmark evidence while identifying design choices intended to make agentic medical model development auditable.It is explicitly positioned as an initial technical report rather than a definitive claim about a finished system.