Source-linked AI summary
A doctrine-grounded visual question answering dataset for Tactical Combat Casualty Care
Junseob Kim, Jade Chng, Ayman Ali, Victor Moas, Yichun Lee, Po-Chun Chin, Sunil Hwang, Rishikesan Kamaleswaran
TL;DR
TC3 care requires linking visible injuries and interventions to clinical guidance, while visual and knowledge-grounding errors complicate that supervision. TC3-VQA constructs a traceable dataset from public footage and authoritative sources, with 1,860 questions across 581 items and 11 concepts.
Problem
Visual hallucinations, narration misalignment, language priors, and insufficiently supported generated citations complicate reliable grounding of scenes and clinical claims.
Source: §1 · p. 2Method
TC3-VQA links public TC3 footage and authoritative guidance through visual annotations, retrieved doctrine, verified answer spans, and cross-family review.
Source: §1 · p. 3, §2.4 · p. 6Results
1,860 questions cover 581 items from 190 videos and 11 concepts, including 431 answerable and 150 refusal items.
Source: §4 · p. 7Takeaways & Limitations
The dataset is intended to support VLM fine-tuning and evaluation of visual grounding, doctrine-based reasoning, and appropriate abstention in TC3.
Source: §1 · p. 3Instructional and field-training footage differs from real combat casualties, so performance on TC3-VQA does not establish deployment readiness.
Source: §6.6 · p. 15
Abstract
from arXiv · showhide
Tactical Combat Casualty Care (TC3) requires responders to connect visual observations of injuries and interventions with established clinical guidance. Developing vision-language models to support this process requires supervision that links visible evidence to traceable doctrine. We present TC3-VQA, a dataset constructed from public instructional and field TC3 videos and authoritative TC3 documents. It contains 581 items spanning 11 concepts, with 1,860 questions covering intervention recognition, doctrine, clinical reasoning, procedural guidance, and refusal when visual information is insufficient. Doctrine-based answers preserve verbatim source passages and character offsets. Construction combines visual annotation, passage retrieval, entailment checks, and verification across model families. Equipment boxes, anatomical labels, temporal segments, and source metadata accompany the question-answer pairs. Automated audits and ratings by two physicians and two medical students characterize annotation quality, with human ratings available for 88 retained items. The dataset provides a resource for adapting vision-language models to TC3, studying the connection between visual evidence and clinical knowledge, and evaluating recognition, doctrine recall, and abstention.
1 Background & Summary
TC3 responders must connect visual observations to clinical guidance, but existing visual resources and verification practices leave this link difficult to supervise reliably. TC3-VQA pairs public TC3 footage with authoritative doctrine and traceable answers.
Background
- Physiological decision-support systems do not directly show which equipment is present or how an intervention is performed, and comparable TC3 visual resources remain limited.The gap is especially relevant to questions linking visible battlefield-care procedures with clinical guidance.Source: §1 · pp. 1–2
- Visual hallucinations, narration misalignment, language priors, and unsupported generated citations complicate independent grounding of scenes and clinical claims.These risks motivate independently checking what the scene supports and what the clinical source substantiates.Source: §1 · pp. 2–3
Contribution
- TC3-VQA contains 1,860 questions on 581 items across 11 concepts, linking public TC3 footage to clinical guidance with traceable passages and visual annotations.Its question types cover intervention recognition, doctrine, clinical reasoning, procedural guidance, and answerability; the dataset is intended for VLM fine-tuning and evaluation.Source: §1 · p. 3
2 Methods
The methods assemble public video and doctrine sources, then combine visual annotation, retrieval, answer verification, and review to curate the released items.
2.1 Data collection and preprocessing
- The team collected 415 public videos, sampled and filtered frames, then retained windows with an active procedure or post-procedure result and usefulness score of at least 3.This yielded 8,012 windows from 32,987 frames; the doctrine corpus comprised 9,320 chunks from 11 sources.Source: §2.1 · pp. 3–4
2.2 Visual annotation and concept assignment
- Visual concept labels were assigned and independently reviewed; equipment, anatomy, and answer-region annotations supplied additional evidence for retained labels.Only clearly or partly visible candidates continued, and low-confidence windows were reserved for refusal construction.Source: §2.2 · pp. 4–5
2.3 Doctrine retrieval and question construction
- Doctrine retrieval preceded question writing, and retrieved passages supplied verbatim answers for doctrine, reasoning, and procedural-how questions.Four answerable question types covered recognition, doctrine, reasoning, and how-to guidance; refusal questions asked for information not established by the frames.Source: §2.3 · p. 5
2.4 Verification and release curation
- Curation checked answer provenance and entailment, reviewed items for visual and question quality, and retained answerable items only after cross-family consensus.At least three of six VLMs had to endorse a concept; human ratings characterized annotation quality but did not modify released labels.Source: §2.4 · pp. 5–6
2.5 Ethics statement
- The study used public videos and government or CoTCCC documents, enrolled no patients, and performed no clinical intervention.Two physicians and two medical students rated public video material as research-team members.Source: §2.5 · p. 6
3 Data Records
The release is organized into linked item records, doctrine sources, visual annotations, provenance, and supporting evaluation materials.
Data records
- Version 1.1 separates item records, doctrine chunks, equipment detections, temporal segments, and frame features, with metadata and evaluation materials in companion directories.The release also includes human adjudication and a doctrine multiple-choice extension.Source: §3 · p. 6
- Each item links questions and answers to source video, segment, timestamps, and doctrine-span offsets, alongside equipment boxes, answer regions, and observed body regions.Review fields record concept, visibility, audit, safety, and review-priority information; refusal records identify missing-information categories.Source: §3 · pp. 6–7
- The example frames span five MARCH categories and show released annotations from the equipment ontology.The displayed categories are massive hemorrhage, airway, respiration, circulation, and hypothermia.Source: Figure 3 · p. 7
4 Data Overview
The overview describes the released dataset’s size, question mix, source materials, visual context, and annotation composition.
Data overview
- 1,860 questions cover 581 items from 190 videos and 11 concepts, including 431 answerable and 150 refusal items.The answerable set contains 431 recognition, 427 doctrine, 425 reasoning, and 427 how questions; all four types appear for 418 items.Source: §4 · p. 7
- The sampled context has a 10-second median duration; 309 items use one frame and 272 use two to four frames.Items contribute a median of three per video, with a maximum of eight including refusals.Source: §4 · p. 7
- ATP 4-02.11 supplies approximately half of cited answers and 73% of procedural answers; no refusal-information category exceeds 26% of refusal items.The CoTCCC and Joint Trauma System guidelines supply most of the remaining cited answers.Source: §4 · p. 7
- The release includes 1,565 equipment boxes across 743 of 903 frames, covers 16 of 17 classes, and most often depicts extremities followed by the chest.Figure 4 organizes composition by refusal request, source document, frequent equipment-box counts, and body region.Source: §4 · p. 7, Figure 4 · p. 8
5 Technical Validation
Technical validation combines model audits, expert ratings, annotation checks, and baseline comparisons. It finds that pipeline-generated pairs outperform direct generation on joint concept consistency and doctrine support, while visibility and frame context remain important constraints.
5.1 Concept labels
- Among 431 answerable items, physicians accepted 82% of clearly visible labels versus 44% of partly visible labels; six-model mean agreement was 4.8 of 6.Unanimous model endorsement occurred for 102 of 245 clearly visible items and 45 of 186 partly visible items.Source: §5.1 · p. 8
5.2 Question and answer validity
- Majorities judged 98.7% of refusal questions unanswerable (10.8% of controls) and 94.6% of how questions paired-frame-anchored (11.0% on substituted frames).The reasoning audit accepted 91.5% of released answers versus 33.0% of within-concept controls, but also accepted 35 of 38 answers flagged by a physician; MedGemma-27B marked 89.2% of substituted-frame controls anchored.Source: §5.2 · pp. 9–10, Table 2 · p. 9
5.3 Equipment annotations
- Box review verified object presence and class assignment, not localization accuracy; YOLOv8s achieved mAP50 = 0.49 and mAP50-95 = 0.35.Because evaluation was split by frame rather than source video, the detector results characterize consistency within the collection rather than generalization to unseen footage.Source: §5.3 · p. 10
5.4 Human ratings
- Both physicians rated 69% of recognition labels correct (AC1 0.80), with uncertainty concentrated in partly visible or ambiguous procedural scenes.Across quoted-answer types, outright incorrect ratings ranged from 0 to 11%; physician severity agreement was lower (AC1 0.27).Source: Table 3 · p. 10, §5.4 · p. 11
5.5 Baseline performance and image dependence
- Recognition accuracy ranged from 0.67 to 0.85, but Qwen2-VL-7B’s 0.85 came with 26% refusal accuracy and RWHR 0.92.Frame substitution reduced accuracy below the 0.20 uniform-choice reference for four models; using only the first frame lowered accuracy 7.8–12.7 percentage points for four models.Source: §5.5 · pp. 11–12, Figure 5 · p. 12, Table 5 · p. 12
5.6 Comparison with direct question-answer generation
- Pipeline-generated pairs met both concept-consistency and doctrine-support criteria for 95–99% of questions, versus 72–78% for direct Qwen2.5-VL-72B generation.Among concept-consistent questions, at least 92% of answers in both groups satisfied both criteria, so the difference primarily reflected concept misalignment.Source: Figure 6 · p. 13, §5.6 · pp. 13–14
6 Usage Notes
The usage notes recommend preserving visual, source, and audit metadata; separating evaluation by video and, for doctrine transfer, by source passage; and reporting task-specific metrics with their scope and caveats. They also describe annotation limits that constrain what benchmark performance can establish.
6.1 Model adaptation and data partitioning
- Partition data by source video, keep each answerable item with its derived refusals, and separate source passages when testing transfer to unseen doctrine.Report dataset version, question types, and human-rating use; 1,279 quoted answers cite 313 passages from 49 chunks across six documents.Source: §6.1 · p. 14
6.2 Recognition and abstention
- Report top-1 recognition accuracy, macro accuracy across six MARCH categories, and refusal rate separately; stratify results by visibility and other item characteristics.Safety-critical flags for cited passages are distinct from concept-level severity weights used by RWHR.Source: §6.2 · p. 14
6.3 Doctrine scoring
- Score free-form doctrine answers by entailment and multiple-choice answers by accuracy, reporting both because they test different response requirements.Sentence-level and whole-answer entailment are reported; unsupported-content rates warrant caution because the classifier may miss valid paraphrases. Distractors contrast facets from other concepts or the same concept, with near-duplicates filtered and options randomized.Source: §6.3 · pp. 14–15
6.4 Visual grounding annotations
- Use equipment boxes and answer and body regions for detection and evidence-localization analyses, but treat missing boxes as unannotated rather than negative.Boxes cover a 17-class ontology, and detection experiments should split by source video.Source: §6.4 · p. 15
6.5 Risk-weighted error and coverage
- Report RWHR alongside coverage, unweighted error, and the risk–coverage curve because it is undefined at zero coverage and uses dataset-specific weights.The weights are 3 for hemorrhage, airway, and respiration; 2 for IV/IO access, tranexamic acid, and hypothermia prevention; and 1 for documentation. RWHR can exceed one and should not replace unweighted results.Source: §6.5 · p. 15
6.6 Limitations
- Automated labels and the 88 retained adjudicated items do not establish clinical accuracy for every record or deployment readiness.Coverage reflects available footage and doctrine; medication identity, dose, route, and timing may not be reliably resolved from sampled frames, and repeated doctrine can permit answers from general knowledge. Related model families and frame-level detector splits may affect evaluation; instructional and field-training footage differs from real casualties.Source: §6.6 · p. 15
7 Data Availability
TC3-VQA version 1.1 is deposited at Zenodo with data, annotations, ratings, and evaluation materials. The package supports access to all 581 items through permitted frame files, source timestamps, or frozen image features.
7 Data Availability
- Version 1.1 is deposited at Zenodo and includes item records, doctrine chunks, visual annotations, source and license manifests, temporal segments, human ratings, and evaluation materials.Construction artifacts are retained by the authors and available on request.Source: §7 · pp. 15–16
- Image files are included for 115 frames from 25 sources with redistribution permission; remaining frames can be regenerated from source identifiers and timestamps.For items whose pixels cannot be redistributed, frozen image-encoder features of every frame keep all 581 items usable without source videos.Source: §7 · p. 16
- As of 17 September 2026, 189 of 190 videos were reachable, while annotations use CC BY 4.0 and scripts use the MIT license.One removed video covered two items.Source: §7 · p. 16
8 Code Availability
The authors provide construction and evaluation code, including frame reconstruction, annotation, retrieval, verification, and scoring. They also report the models and hardware used for visual auditing and related tasks.
8 Code Availability
- Construction and evaluation code is available on GitHub and includes frame reconstruction, annotation, retrieval, verification, and scoring.Source: §8 · p. 16
- Qwen2.5-VL-72B-Instruct was served with vLLM 0.15 on four NVIDIA H200 GPUs, while Claude Opus 4.8 performed visual auditing, doctrine-facet mining, and review.Images were resized to a longest side of at most 896 pixels.Source: §8 · p. 16