Source-linked AI summary
Candidate-Expanding Routing with Permutation-Stabilized Experts for Mixed-Format Medical VQA
Hai-Dang Nguyen, Huy-Hieu Pham
TL;DR
Mixed-format medical VQA must stabilize multiple-choice option selection while preserving machine-readable open-answer serialization. The paper separates candidate formation from selection, adding expert top-2 routing to memory and expert top-1; this yields 39 net correct cases and 96.15% oracle coverage, while strict open-path generation produces 475/475 schema-valid outputs.
Problem
Mixed-format medical VQA combines unstable multiple-choice option selection with open-answer failures caused by malformed machine-readable fields.
Method
The paper separates candidate formation from selection using answer-text memory, ranked vision-language candidates, and an explicit serialization contract for open answers.
Results
39 net correct cases from expert top-2 expansion raise oracle coverage to 96.15% from 90.31%, while the final retrospective result reaches 92.23%.
Takeaways & Limitations
Candidate expansion supplies the principal controlled routing gain, while open-path evidence establishes output-contract validity rather than clinical correctness or deployment readiness.
Takeaways & Limitations
Evidence remains retrospective and is limited by answer regularity, image or template overlap, record-level OOF, unavailable hidden results, and absent clinician review.
Abstract
from arXiv · showhide
Mixed-format medical visual question answering (VQA) requires stable option selection and machine-readable free-text output. The two formats fail differently: multiple-choice predictions can change with option symbols or positions, while clinically plausible open answers can fail automated evaluation when serialization is malformed. We address both challenges with an answer-text memory, a permutation-stabilized vision--language expert, and a sparse candidate- expanding router. The cyclic schedule follows prior work; our contribution is to make expert top-2 a routable candidate alongside memory and expert top-1. On a 1,403-case retrospective internal analysis, this expansion improves a matched binary router from 88.95% to 91.73% (+2.78 percentage points; 95% CI 1.57--3.99), with 56 rescued errors and 17 regressions. Oracle coverage rises from 90.31% to 96.15%, and the final submitted configuration reaches 92.23% on the same retrospective split. For open questions, strict generation and deterministic guards produce 475/475 schema- valid participant-facing outputs without repair, retry, or hard-gate failure. Visual ablations reveal substantial textual dependence. Candidate expansion supplies the principal controlled routing gain; open-path evidence establishes output-contract validity rather than clinical correctness in medical use or deployment.
1 Introduction
Mixed-format medical VQA requires both stable selection among case-defined MCQ options and machine-readable open-answer serialization. The paper separates candidate formation from selection, expanding MCQ candidates while using explicit output guards for open questions.
- MCQs can fail through unstable option identifiers, while clinically plausible open answers can become unscorable through malformed fields.
- The system keeps memory and VLM outputs as independent candidates, merging them only at selection.The MCQ path forms one memory candidate and two ranked VLM candidates.
- The approach separates candidate coverage from router regret, distinguishing whether any candidate is correct from errors after a correct candidate is available.
- Open-output guards address parsing and serialization failures only, without establishing clinical correctness or rationale faithfulness.
- Adding expert top-2 expands routing beyond memory and expert top-1, raising matched accuracy by 2.78 percentage points and oracle coverage by 5.84 points.The expansion retains ranked semantic alternatives so a correct second expert candidate remains available to the selector.
2 Method
The method routes a shared image-question input through separate MCQ and open-answer paths. MCQs combine answer-text retrieval with permutation-stabilized VLM scoring and sparse selection, while open questions use retrieved context and guarded JSON generation.
- 2.1 System Overview: The system dispatches a shared image and question by task type, producing a case-defined MCQ label or validated open-answer JSON.MCQ retrieval and VLM scoring meet only at the router; the open branch has separate retrieval and generation.
- 2.2 Heterogeneous MCQ Candidate Generation: Answer-text memory retrieves 50 neighbors, retains five, matches answer text to current options, and similarity-votes for cR.The memory contains 11,592 released fit MCQs and uses cosine similarity over normalized, IDF-weighted unigram features.
- 2.2 Heterogeneous MCQ Candidate Generation: Balanced cyclic shifts place every semantic option in every symbol-position slot before scores are mapped back and aggregated.Only slot assignment changes; image, question, and option content remain fixed.
- 2.2 Heterogeneous MCQ Candidate Generation: The permutation-stabilized expert ranks the geometric mean after within-shift normalization, with its two highest scores defining c1 and c2.Balanced exposure equalizes the mean slot term but cannot remove content-slot interactions.
- 2.3 Candidate-Expanding Router: A one-vs-rest L1 logistic router selects among cR, c1, and c2 using inference features covering agreement, retrieval, confidence, input, metadata, keywords, and shift consistency.References, correctness, IDs, and oracle switches are excluded; five-fold record-level out-of-fold predictions select class multipliers.
- 2.4 Open-Output Contract: Open retrieval supplies up to three contexts to one greedy JSON prompt, followed by parsing, schema guards, serialization-only repair, and one retry.Persistent failure aborts, and both answer and reasoning_trace fields are generated once.
3 Experiments
Experiments evaluate the proposed routing system on retrospective internal medical VQA, including matched baselines, controlled ablations, mechanistic analyses, qualitative cases, robustness controls, and open-output validity. Candidate expansion improves routing performance, while the open path demonstrates serialization validity rather than clinical correctness.
- Data and protocol: 1,403 retrospective internal MCQs formed the analysis split, while the unscored participant set contained 475 open cases.The internal split was retrospective development evidence rather than independent evaluation.
- Main baselines: 91.73% matched binary routing accuracy followed candidate expansion from 88.95%, while the final retrospective result reached 92.23%.The reviewed same-split baselines report the matched routing comparison and final configuration separately.
- Controlled ablation: 56 binary-router errors were rescued and 17 regressed, yielding 39 net correct cases and a +2.78-point gain with 95% CI [1.57, 3.99].The matched comparison used pHash-cluster bootstrap confidence intervals and Holm-adjusted exact McNemar testing.
- Mechanistic analysis: Top-2 candidates expanded oracle coverage from 90.31% to 96.15% and matched routing from 88.95% to 91.73%.Mechanistic analysis separates candidate availability from errors made after a correct candidate is reachable.
- Qualitative evidence across formats: Qualitative CT cases show top-2 exposing the reference answer and an open output preserving both reference findings in valid JSON.These fixed, fit-disjoint examples illustrate mechanisms rather than aggregate validity.
- Robustness and sensitivity: All 475 participant-facing open outputs were schema-valid without repair, retry, or hard-gate failure, establishing output-contract validity only.Seven review warnings remained, and organizer pre-evaluation was reported without an available evaluator.
4 Discussion and Conclusion
Expert top-2 candidate expansion supplies the main controlled routing gain and supports the final retrospective result. The evidence remains limited by retrospective evaluation and insufficient validation of clinical grounding and deployment relevance.
- 39 net correct cases and 96.15% oracle coverage demonstrate the principal gain from adding expert top-2.Oracle coverage increased from 90.31%, indicating more cases had a correct candidate available.
- The final configuration reaches 92.23% on the same retrospective split, while Figure 3 illustrates mechanism rather than aggregate validity.
- Retrospective evidence lacks hidden results and clinician review, limiting assessment of clinical validity and deployment performance.The analysis also lacks patient/study IDs and uses record-level retrospective evidence.
- Future evaluation should use blind patient-disjoint testing, set-valued targets, and clinician-supported comparisons of guarded and raw answers.