Source-linked AI summary
Towards Fully Automated Medical Imaging Code Generation via Validation-based Context Engineering
Zixiao Zhao, Jing Sun, Zhe Hou, Cheng-Hao Cai, Qian Liu, Mengze Li, Zijian Zhang, Jin Song Dong
TL;DR
Complex medical imaging code generation is limited by insufficient domain knowledge, validation, and context continuity, often requiring human intervention. AutoMedImg combines specialised multi-agent planning and coding with validation-based context engineering and adaptive pipeline synthesis, achieving fully automated generation across diverse datasets. The framework achieves zero human intervention while maintaining competitive performance.
Problem
LLMs lack the domain knowledge, robust validation, and cross-project context continuity needed for reliable complex medical imaging code generation.
Method
AutoMedImg uses specialised agents in Planning and Coding phases, integrating domain knowledge, shared memory, validation feedback, static and execution checks, and adaptive synthesis.
Results
AutoMedImg achieves fully automated code generation across diverse medical imaging tasks with zero human intervention while maintaining competitive performance.
Takeaways & Limitations
Validated pipeline accumulation and retrieval supports cross-project knowledge reuse for future medical imaging generation tasks.
Takeaways & Limitations
AutoMedImg is specifically designed for medical image processing, requiring domain-specific knowledge bases and agent configurations for adaptation to other domains.
Abstract
from arXiv · showhide
Large language models (LLMs) have demonstrated considerable promise in program generation for small-scale and conventional application development; however, they remain limited when applied to complex, domain-specific tasks such as medical image processing. General-purpose models lack explicit domain knowledge and robust validation mechanisms to ensure correctness, often requiring substantial human intervention to produce reliable processing pipelines. To address these limitations, we propose AutoMedImg, a multi-agent framework for fully automated medical image processing code generation. AutoMedImg orchestrates specialised agents across two phases: a Planning Phase that performs dataset analysis and architecture design with semantic and formal verification, and a Coding Phase that generates modules in parallel with static checking, execution testing, and assembly validation. This multi-stage validation mitigates error propagation throughout generation, while comprehensive auto-context engineering combining domain-specific knowledge bases, shared memory, and validation feedback automates context construction without manual prompting. A cross-project adaptive pipeline synthesis mechanism further accumulates validated pipelines and retrieves proven components for new tasks based on project similarity, enhancing generation efficiency through cross-project learning. Extensive evaluation across six diverse and well-established medical imaging datasets with five backbone LLMs demonstrates that AutoMedImg achieves zero human intervention, with Dice scores of up to 0.90 for segmentation tasks and 99% accuracy for classification.
1 Introduction
LLMs perform strongly on conventional code-generation benchmarks but remain unreliable for complex medical imaging tasks because they lack domain knowledge, validation, and cross-project continuity. AutoMedImg addresses these limitations with validation-based context engineering and adaptive pipeline synthesis, achieving fully automated generation across six datasets.
- 97.6% accuracy on HumanEval and 92.7% on MBPP demonstrate strong LLM performance on small-scale code-generation benchmarks.
- Complex medical imaging code generation requires detailed prompts, human intervention, and mechanisms for preserving context and memory across projects.Medical images involve multimodal data, 2D/3D structures, varied intensities, and specialised formats such as NIfTI and DICOM.
- Existing code-generation, AutoML, multi-agent, and medical-imaging automation approaches remain insufficient because they lack domain-specific integration, systematic validation, or full-pipeline coverage.Several approaches are restricted to predefined datasets, partial pipelines, or specific task types.
- AutoMedImg achieves fully automated medical image processing code generation across six diverse datasets with zero human intervention and competitive model performance.
- AutoMedImg integrates medical imaging knowledge, shared memory, and multi-stage validation to construct and maintain generation context without manual prompting.
- Adaptive pipeline synthesis transfers validated components across projects to support knowledge reuse while maintaining pipeline quality.
2 Related Work
Prior work improves code generation and medical imaging automation, but existing methods typically depend on human-provided tests, predefined search spaces, manual setup, restricted tasks, or prepared datasets. AutoMedImg is presented as a two-phase workflow for broader automated medical imaging pipeline generation.
- Self-debugging and execution-feedback methods refine generated code using runtime errors but rely on human-provided test cases.
- NAS discovers network structures within predefined search spaces, while nnU-Net automates segmentation configuration but requires manual environment setup and remains segmentation-specific.
- AutoMedImg’s Figure 1 depicts a two-phase workflow consisting of Planning and Coding.
- M3Builder automates medical imaging workflows through multi-agent coordination but restricts generation to predefined datasets and accompanying code templates.
3 Method: AutoMedImg Framework
AutoMedImg is a multi-agent framework that generates medical image processing pipelines through specialised planning and coding agents, shared validated context, and independent validation. Its validation-based context engineering iteratively refines outputs, while adaptive synthesis reuses validated components across projects.
- Framework workflow: AutoMedImg divides generation into Planning and Coding phases coordinated by routers and validation agents.Planning analyses datasets and designs architectures; Coding generates modules in parallel and validates final assembly.
- Multi-agent architecture: Specialised agents distribute dataset analysis, architecture design, and module generation across focused contexts rather than one agent handling multiple domains.Shared Memory stores validated intermediate results, while an independent Validation Layer reduces self-validation bias.
- Context engineering: Agent context combines task prompts, shared validated memory, domain knowledge, adaptive synthesis knowledge, skills, and validation feedback.These components support role-specific expertise and context construction without relying on a single accumulating context window.
- Validation-based propagation: Only outputs that pass validation propagate to subsequent agents; failed outputs receive diagnostic feedback and are regenerated with enriched context.Planning validates dataset analyses and architectures, while Coding validates modules and complete assembly.
- Cross-project synthesis: Cross-project adaptive synthesis accumulates validated pipelines and retrieves components for similar projects before validating their adaptations.This constrains new solutions to validated neighbourhoods and supports supervised knowledge reuse without human intervention.
4 Evaluation
The evaluation examines existing code-generation limitations, AutoMedImg’s comparative performance, component contributions, and the effect of multi-agent coordination. Experiments include baseline comparisons and comprehensive ablation studies.
- Research questions: RQ1 assesses the fundamental limitations of existing code-generation approaches on medical image processing tasks.The question targets limitations of current approaches when applied to these specialised tasks.
- Research questions: RQ2 compares AutoMedImg with advanced multi-agent and automated ML frameworks to examine how it addresses those limitations.The comparison focuses on the framework’s response to the identified shortcomings.
- Research questions: RQ3 investigates how validation and context-engineering components reduce LLM errors and support code correctness.This question motivates component-level analysis of the proposed framework.
- Research questions: RQ4 evaluates whether the multi-agent architecture improves code-generation quality over a single-agent system using the same context-engineering techniques.The evaluation isolates the contribution of multi-agent coordination.
- Evaluation structure: The experimental results are organised by research question and include comprehensive ablation studies.The evaluation section follows dataset, configuration, and metric descriptions.
4.1 Benchmarking datasets
AutoMedImg is evaluated on six publicly available, widely adopted datasets spanning segmentation and classification. The datasets cover a difficulty gradient from supervised classification to complex 3D semi-supervised segmentation.
- Benchmark scope: The benchmark suite contains six datasets covering four medical image processing task categories.The selected tasks include the two fundamental categories of segmentation and classification.
- Dataset diversity: The datasets span diverse modalities, dimensions, supervision settings, and clinical structures.Examples include semi-supervised 3D abdominal-organ segmentation in CT and supervised 3D cardiac-structure segmentation in MRI.
- Benchmark scope: The benchmark difficulty ranges from simple supervised classification to complex 3D semi-supervised segmentation.This gradient enables evaluation across varied medical imaging scenarios.
- Selection rationale: Datasets were selected because they are publicly available, widely adopted, and collectively support robust evaluation across diverse scenarios.Their characteristics include limited supervision and challenging volumetric structures.
4.2 Evaluation settings & criteria
The evaluation gives baselines equivalent task structure, tool access, execution permissions, and filesystem access where supported. Methods are assessed for autonomy using human intervention counts alongside their execution and refinement capabilities.
- Baseline configuration: Baseline methods receive prompts with equivalent task structure and necessary tool access for the same four-stage pipeline attempted by AutoMedImg.The stages include dataset analysis and architecture design before implementation and assembly.
- Baseline configuration: Baselines may execute scripts, read and write files, install dependencies, and refine code from execution feedback where their architectures permit.This configuration is intended to expose errors and limitations without AutoMedImg’s validation-based context engineering and coordination.
- Performance evaluation: Figure 4 compares code-generation tools and base LLMs on medical image processing tasks.The figure is the evaluation’s performance-comparison view across the tested approaches.
- Experimental environment: Experiments run on an Ubuntu 22.04 server with a 64-core CPU, 252GB RAM, and two RTX A6000 GPUs.Methods share a unified Python environment with pre-installed medical imaging libraries and may install additional dependencies.
- Evaluation criteria: Human intervention counts the times people correct errors or provide guidance that systems cannot resolve autonomously.Interventions include direct prompts, templates or skeleton code, and occasional manual script or configuration updates.
4.3 Experimental results
Existing tools and frameworks remain dependent on human input for specialised medical imaging code generation, whereas AutoMedImg combines domain knowledge, validation, memory, and adaptive synthesis to achieve zero-intervention generation. Ablations show that these components, especially execution testing and domain knowledge, materially affect autonomy, performance, or efficiency.
- Existing approaches: 5–17 human interventions are required by proprietary IDE tools across tasks, mainly for domain-specific design, preprocessing, volumetric-data handling, and execution-error correction.Antigravity performs more competitively with fewer than 10 interventions, associated with partial memory retention across sessions.
- Existing approaches: Existing code generation tools and LLMs consistently fail on medical imaging tasks because they lack domain knowledge and effective validation.Antigravity’s partial memory retention suggests cross-project knowledge accumulation as a promising direction.
- AutoMedImg comparison: AutoMedImg achieves zero human interventions across all task types, including semi-supervised segmentation, while maintaining competitive or superior performance against the evaluated frameworks.The framework uses domain-specific knowledge bases, multi-stage validation, and adaptive pipeline synthesis to address manual input dependencies and fixed templates.
- Coding-phase validation: 3.50 higher HI and 4.5% performance degradation follow removal of execution testing, making it the most critical validation for automated generation.Without runtime verification, tensor mismatches and incorrect function calls can propagate and require substantial human debugging.
- Memory and knowledge components: 13.7% lower performance follows removal of the knowledge base, while adaptive synthesis removal causes 8.4% lower performance and more than 37 minutes of additional generation time.The knowledge base guides domain-appropriate designs, whereas adaptive synthesis reuses prior projects instead of regenerating solutions from zero.
- Multi-agent ablation: 10.2% lower performance, 1.33 additional interventions, and 10 extra minutes result when AutoMedImg uses a single agent instead of its multi-agent architecture.Role separation, phase-separated context management, and concurrent execution address self-validation bias, context compaction, and lost parallelism.
5 Discussion
AutoMedImg extends automated medical imaging code generation through complete pipeline generation, cross-project knowledge reuse, systematic validation, and full automation. Its main trade-off is higher generation cost and runtime than direct generation.
- IDE tools and vanilla LLMs provide strong code generation but lack domain-specific conventions and built-in validation, increasing pipeline-design and correction challenges.
- Validated solutions can be retrieved and adapted for future tasks through cross-project knowledge accumulation, a capability absent in prior work.
- AutoMedImg is the only compared approach supporting complete pipeline generation, task-specific design, multi-task support, cross-project knowledge reuse, systematic multi-stage validation, and full automation.
- AutoMedImg incurs higher API costs and generation time than direct generation because multi-agent coordination uses iterative validation.Complete pipeline generation costs approximately $20–40 and averages 32 minutes, compared with manual development requiring 4–8 hours at $200–600.
6 Conclusion
AutoMedImg is a validation-based, multi-agent framework for fully automated medical image processing code generation. Evaluation reports zero human intervention with competitive performance, while future work targets additional imaging tasks and algorithmic improvements.
- AutoMedImg combines multi-agent coordination, multi-stage validation, domain knowledge, shared memory, validation feedback, and adaptive pipeline synthesis.
- AutoMedImg achieves fully automated code generation across diverse medical imaging tasks with zero human intervention and competitive performance.
- Future work will extend AutoMedImg to image denoising and registration and improve algorithmic performance beyond code generation.