DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis

1 Introduction

• Automated vision methods extract quantitative gait metrics, but existing approaches largely classify gait patterns without explaining the pathological features underlying them.The lack of interpretability hinders clinical adoption, where understanding diagnostic reasons matters for disease management and treatment planning.Source: §1 · p. 2
• VLMs can generate explanations but struggle to measure subtle 3D gait deviations, such as distinguishing ankle dorsiflexion deficits from increased pelvic tilt.The paper attributes numerical hallucinations in exact kinematic and temporal measurements to visual encoders that compress spatial coordinates into discrete tokens.Source: §1 · p. 2
• DrGait assigns high-level task planning to the VLM and delegates numerical measurements from pose, mesh, and video to specialized biomechanical tools.The tool library includes joint-kinematics extractors, gait-event detectors, and pose-derived symmetry analyzers.Source: §1 · p. 2
• DrGait follows a Triage-Verification-Synthesis loop in which candidate hypotheses guide tool calls, and contradictory evidence prompts updated hypotheses and further verification.Synthesis cross-references measurements with clinical guidelines until the final diagnosis has no unresolved verification conflicts.Source: §1 · p. 2
• Clinicians preferred DrGait reports in 73.2% of GAVD cases and 69.6% of ProGait cases, while classification accuracy improved by up to 31%.The evaluation used open-ended clinical categories and assessed both diagnostic accuracy and semantic correctness of reasoning.Source: §1 · pp. 2–3

Video-based Gait Analysis

• Traditional CNN- and transformer-based gait systems classify patterns as black boxes, while motion-language work focuses on gross action recognition rather than pathological deviations.Fine-tuned rationale generation can be expensive and relies on parametric memory, leaving it prone to hallucinations; DrGait instead uses pre-trained VLMs to retrieve verifiable physical data.Source: §2 · p. 3

VLMs in Medicine

• Medical VLM encoders optimized for semantic understanding compress precise spatial measurements, making raw-video diagnosis prone to hallucinated kinematic angles or temporal events.DrGait shifts the VLM from direct visual reasoning to clinical planning and delegates low-level measurements to deterministic mathematical tools.Source: §2 · p. 3

Agentic AI and Tool-Use

• Existing agentic reasoning structures are largely predefined and static, whereas clinical gait assessment requires workflows to adapt to patient-specific observations.DrGait adapts the ReAct paradigm [32] into a dynamic Triage-Verification-Synthesis loop that updates its reasoning trajectory from tool outputs.Source: §2 · p. 3

Biomechanical Tools for Gait Analysis

• DrGait integrates deterministic event detectors and inverse-kinematics tools as callable functions whose outputs provide auditable biomechanical evidence.The cited tools support calculations of joint angles and temporal milestones from reconstructed human meshes.Source: §2 · p. 3

3 Method

• DrGait uses an interactive, tool-augmented Triage-Verification-Synthesis loop to adapt clinical gait reasoning across domains and tasks.The framework varies domain-specific inputs, knowledge bases, and tool schemas while retaining the same reasoning structure.Source: §3 · p. 3

3.1 Multi-Modal Input Context

• The starting context combines raw video with deterministic pose and geometry, derived spatiotemporal metrics, and task-specific gait knowledge.Metrics initialize reasoning as a quantitative lab report, while the knowledge base supplies labels and expected biomechanical patterns.Source: §3.1 · pp. 3–4

3.2 Task Formalization via Heuristic Triage

• Triage matches initial context to the injected knowledge base to form ranked hypotheses and explicit verification objectives, while gating normal, abnormal, or uncertain cases.For uncertain cases, the agent chooses a safe hypothesis path and uses verification objectives to resolve uncertainty rather than declaring a diagnosis prematurely.Source: §3.2 · pp. 4–5

3.3 Agentic Verification and Computation Tools

• Verification selects schema-constrained tool calls to gather targeted evidence across gait events, joint motion, posture, foot alignment, timing, and symmetry.Calls include a rationale and are limited to one bounded round per iteration; 2D outputs are image-plane proxies, not calibrated 3D measurements.Source: §3.3 · pp. 5–6

3.4 Recursive Synthesis and Belief Refinement

• Synthesis labels objectives and hypotheses with evidence-grounded statuses, recursively revises context for later tool calls, and preserves unresolved conflicts when stopping criteria are unmet.The loop stops when a hypothesis is confirmed and no inconclusive objective targets it; otherwise, the final report retains unresolved conflicts or anomalous tool feedback.Source: §3.4 · p. 6

4.1 Datasets, Baselines, and Evaluation Protocol

• The curated GAVD subset contains 134 videos spanning normal and five abnormal categories; ProGait evaluation groups paired views into 138 trial-level items.GAVD predictions use an expanded 11-category dictionary, while training-based baselines use a closed six-class task and are not directly comparable with open-dictionary VLM settings.Source: §4.1 · pp. 6–7

4.2 Gait Classification Accuracy

• On GAVD, DrGait with Gemini-3-Flash achieves 92.4% normality-recognition accuracy and 51.4% overall abnormal-subtype accuracy, improving over Gemini-3-Flash by 29.0% and Gemini-3.1-Pro by 12.1%.The largest gains occur in Parkinson’s (94.1%), Stroke (88.5%), and Antalgic gait (54.6%).Source: Table 2 · p. 7, §4.2 · p. 8
• On ProGait, Gemini-3.1-Pro leads top-1 general-category accuracy (18.11%), while DrGait leads top-3 accuracy and semantic agreement with human annotations.The authors suggest tool-guided reasoning helps reports capture relevant deviations even when the exact broad category is not ranked first.Source: Table 3 · p. 8, §4.2 · p. 8

4.3 Human Evaluations on Interpretability

• On GAVE, DrGait’s overall report score is 3.67 versus 2.57 for Gemini-3-Flash and 2.88 for Gemini-3.1-Pro; it is selected as best overall in 73.22% of evaluations.On ProGait, DrGait achieves the highest aggregate overall score (2.71) and best-overall selection rate (49.64%).Source: Table 4 · p. 8, §4.3 · p. 8

4.4 Efficiency and Cost Analysis

• Average end-to-end latency is 79.9 seconds with Gemini-3-Flash and 107.0 seconds with Gemini-3.1-Pro, versus roughly 11.7–14.1 seconds for direct VLM evaluation.DrGait executes 1.06–1.23 TVS loops per video and 5.15–5.74 tool calls per round; the authors characterize sub-two-minute processing as acceptable for asynchronous diagnosis.Source: Table 5 · p. 9, Table 6 · p. 9, §4.4 · p. 9

4.5 Ablation Studies

• Removing verification tools leaves GAVD normality accuracy near unchanged (91.8% versus 92.4%) but lowers abnormal-subtype accuracy from 51.4% to 44.9%, ProGait top-3 accuracy from 32.6% to 13.1%, and its LLM-judge score from 1.78 to 1.30.The authors identify verification tools as important for clinically specific differential diagnosis and report grounding.Source: Table 7 · p. 9, §4.5 · p. 9
• On GAVD, recursion raises normality accuracy from 91.0% to 92.4% and subtype accuracy from 43.9% to 51.4%; on ProGait, one-pass verification scores 34.1% top-3 accuracy and 1.84 on the LLM judge.The authors suggest many prosthetic cases are resolved in one targeted round, while further refinement can sometimes introduce secondary-deviation evidence into synthesis.Source: Table 7 · p. 9, §4.5 · p. 10
• Removing the normality gate lowers GAVD normality accuracy to 85.1% and abnormal-subtype accuracy to 33.6%, but increases ProGait top-3 category retrieval.The authors conclude that the gate helps when normality is part of the decision, while verification tools matter across tasks.Source: Table 7 · p. 9, §4.5 · p. 10

5 Conclusion

• DrGait is a training-free framework for clinically interpretable gait analysis.It pairs general-purpose VLMs with auditable geometric tools.Source: §5 · p. 10
• Its Triage-Verification-Synthesis loop forms clinically motivated hypotheses, verifies them with biomechanically grounded tools, and produces an evidence-backed report.Source: §5 · p. 10
• DrGait significantly improves VLM gait-analysis capability by pairing general-purpose VLMs with auditable geometric tools.Source: §5 · p. 10

A TVS Loop Algorithm

• The loop prepares model visual content, then builds a schema-constrained triage context from the visual input, metrics, dictionary, and prior context.The triage response includes ranked hypotheses and verification objectives.Source: A TVS Loop Algorithm · p. 13
• Verification executes at most 10 selected tool calls on pose, metrics, and visual input, storing sanitized results and any returned event frames.The stored results include why each tool was used, its arguments, and a compact summary.Source: A TVS Loop Algorithm · p. 13
• Differential synthesis uses the triage response, tool outputs, verification images, dictionary, and prior context to label objectives supported, refuted, or inconclusive.Source: A TVS Loop Algorithm · p. 13
• The loop checks for convergence and, if no synthesis is set, uses the last completed synthesis before producing the final report and saving the reasoning trace.Source: A TVS Loop Algorithm · p. 13

B GAVD Example Case

• DrGait initially favored Parkinsonian gait for rapid, short steps and reduced arm swing, but full hip/knee extension and lack of stooped posture refuted it; wide base and rhythmic coronal trunk sway supported Waddling/Myopathic gait.Step lengths were < 0.25 m and cadence was 134 steps/min; bilateral arm-swing amplitude was ~5–7°. Step width was 0.342 m, with 9.56° lateral sway and SI < 6%.Source: B GAVD Example Case · pp. 14, 20
• Biomechanical checks found 9.56° coronal trunk sway, a trunk angle 7.13° from vertical, and hip extension to 180° during stance, challenging Parkinsonian rigidity.The synthesis interpreted the 9.56° coronal sway as relatively high for rigid Parkinsonian gait, while the 4.21° sagittal sway suggested limited anterior-posterior movement.Source: B GAVD Example Case · pp. 19–20
• The final report found an abnormal gait with rapid, very short steps and a widened base, but no definitive diagnosis; it recommended neurological and musculoskeletal evaluation.Reduced arm swing was Parkinsonian-like, while the lack of stooped posture and significant lateral sway also suggested possible proximal weakness or myopathic influence.Source: B GAVD Example Case · p. 21

• Waddling gait was confirmed (confidence 0.7) on 9.56 deg coronal sway, 0.342m step width and stance SI < 15%, despite cadence of 134 steps/min and flat-foot strike.Source: B GAVD Example Case · pp. 26–27
• Parkinsonian (Festinating) gait was indicated by high cadence (134 steps/min), extremely short steps (<0.25m), flat-foot strike, and bilateral reduction in arm-swing amplitude (approx 5-7 deg).Source: B GAVD Example Case · p. 27
• The report recommends assessing pelvic-girdle strength and physical therapy focused on core and hip stabilization.Source: B GAVD Example Case · pp. 27–28
• The final synthesis identifies Waddling (Myopathic) Gait, marked by a 0.34m wide base and 9.56 deg lateral trunk sway, with full extension and low variability against alternatives.Source: B GAVD Example Case · pp. 27–28

C ProGait Example Case

• Tool verification reversed the initial prosthetic swing interpretation from Lateral Whip to Medial Whip before the final abducted-gait diagnosis.Source: C ProGait Example Case · p. 28
• Verification confirmed abducted gait with a 0.65-shoulder-width lateral-deviation proxy and confirmed 11.39% stance-time asymmetry favoring the sound side.Source: C ProGait Example Case · pp. 34–35
• Trunk-lean status remained inconclusive: sway amplitude was 13.9 degrees, but belt assistance prevented reliable isolation of voluntary lean.Source: C ProGait Example Case · p. 34
• The final report identifies Abducted Gait / Increased Stride Width and Decreased Prosthetic Stance Time / Short Prosthetic Step, with a 0.65-shoulder-width base and ~11% less right-prosthesis loading.Suggested actions include reviewing socket fit and limb length and considering socket-position adjustments.Source: C ProGait Example Case · pp. 35–36

• Normal gait is smooth and symmetric, while antalgic gait shortens stance on the painful limb and hemiparetic gait combines asymmetry with signs such as circumduction or cane use.The entries distinguish pain-avoidance timing from unilateral motor-weakness patterns.Source: D Gait Knowledge Dictionary · pp. 36–37
• Parkinsonian gait features shuffling, festination, stooped posture, and reduced arm swing, whereas myopathic gait has rhythmic side-to-side sway, a wide base, and largely symmetric timing.The myopathic entry excludes classification when a unilateral cane is used.Source: D Gait Knowledge Dictionary · p. 37
• Cerebellar ataxia is marked by variable step placement and irregular sway, neuropathic gait by high stepping and foot slap, and Trendelenburg gait by pelvic drop or compensatory trunk lean.Trendelenburg is differentiated from antalgic gait by the absence of rushed, pain-avoidance stepping.Source: D Gait Knowledge Dictionary · pp. 37–38
• Other listed patterns include scissoring with stiff knees and toe-walking, sensory ataxia with stomping and visual dependence, and crouch gait with persistent hip and knee flexion.Crouch gait is typically bilateral and symmetric; excessive ankle dorsiflexion is also noted.Source: D Gait Knowledge Dictionary · pp. 38–39
• Prosthetic deviation entries describe pelvic drop, lateral-arc circumduction, hip hiking, and sound-side vaulting, linking these patterns to possible prosthetic length or clearance issues.The circumduction entry also lists possible causes such as limited knee flexion, abduction contracture, and weak hip flexors.Source: D Gait Knowledge Dictionary · pp. 39–41
• Additional prosthetic entries cover widened-base gait, toe drag, premature heel rise, foot slap, inadequate heel off, and reduced prosthetic stance time with a short contralateral step.Source: D Gait Knowledge Dictionary · pp. 41–44

Step Length and Timing Issues

• Uneven timing or a long prosthetic step is linked to hesitation from prosthetic mid-stance to sound-side heel strike, causing an unusually long sound-side step.Listed causes include hip flexion contracture, pain on the sound-limb side, excessive plantarflexion, insufficient socket flexion, and an excessively stiff toe lever.Source: D Gait Knowledge Dictionary · p. 44

Knee Instability and Malalignment

• Knee sagittal-plane entries describe excessive prosthetic-side flexion with buckling in early stance and excessive knee extension during stance.The flexion entry lists knee flexion contracture, quadriceps weakness, and weak hip extensors; the extension entry lists limited knee flexion or an extension contracture and compensations for weakness or hip contracture.Source: D Gait Knowledge Dictionary · p. 45
• Frontal-plane knee entries describe medial thrust during mid-stance for valgus and lateral thrust for varus, with causes including poor socket stabilization, ligamentous instability, weak hip abductors, and inadequate balance.Prosthetic-related causes listed include lateral pylon or foot translation, socket alignment or fit issues, excessive foot inset, and loss of frontal-plane containment.Source: D Gait Knowledge Dictionary · p. 46

Socket Fit and Stability

• Trunk entries describe lateral lean over the prosthesis during stance, increased lumbar arch during stance, and significant forward lean during stance.Associated causes include weak hip abductors, pain or balance issues, tight hip flexors, weak hip extensors or core muscles, hip flexion contracture, and compensation for knee instability.Source: D Gait Knowledge Dictionary · p. 47

Rotational Deviations

• Rotational entries describe external foot rotation at heel strike and abrupt medial or lateral prosthetic-heel movement during swing, associating these patterns with rotational alignment or control issues.Causes include how the socket is donned, gait habits, insufficient muscle activation, and prosthetic component rotation.Source: D Gait Knowledge Dictionary · pp. 48–49

Knee Instability and Malalignment

• Other knee entries describe a terminal-swing extension slam, excessive heel rise at pre-swing or toe-off, and a prosthetic knee that remains stiff at toe-off.Listed causes include swing-control or extension-assist settings, strong hip-flexor use, insufficient stance control or knee friction, and inadequate toe loading or confidence in signaling the stance-to-swing transition.Source: D Gait Knowledge Dictionary · pp. 49–50

Rotational Deviations

• Toe-in entries describe excessive internal rotation of the prosthetic foot relative to the sound side during stance or swing, linked to habitual hip rotation or internally rotated prosthetic components.The dictionary also lists excessive toe-in alignment built into the foot, shank, knee, or socket.Source: D Gait Knowledge Dictionary · p. 51

Deviation patterns

• The dictionary links toe-out asymmetry, narrow base of support, pylon lean, asymmetric step length, early foot flat, and knee varus to characteristic gait patterns.For example, it describes excessive toe-out relative to the sound side, feet placed too close to the midline, and early foot-flat after initial contact.Source: D Gait Knowledge Dictionary · pp. 52–54

Rotational deviations

• Rotational entries cover excessive foot toe-out, an externally rotated prosthetic knee, and incongruent knee and ankle axes, with rotation adjustments recommended.The listed actions include rotating the foot externally or internally, internally rotating the knee, and correcting knee-foot rotational alignment.Source: D Gait Knowledge Dictionary · pp. 52, 55, 58

Step and base-of-support deviations

• Step-width and frontal-plane entries associate narrow support or pylon lean with alignment issues and recommend socket, knee, or foot-position changes.Actions include changing knee valgus, shifting or adducting the socket, and reducing valgus or translating the socket laterally.Source: D Gait Knowledge Dictionary · pp. 52–53

Step length, timing, and rollover

• Timing and progression entries describe asymmetric steps, early foot-flat, and incomplete rollover, linking them to progression or loading limitations and alignment adjustments.The recommended actions include changing socket flexion or translation and reducing plantarflexion.Source: D Gait Knowledge Dictionary · pp. 54, 57

Prosthetic length issues

• Length entries distinguish a functionally short limb associated with pelvic drop from a long limb associated with clearance compensations, recommending lengthening or shortening accordingly.Examples of long-limb compensations include vaulting, circumduction, hip hike, or toe drag.Source: D Gait Knowledge Dictionary · p. 56

Socket fit, stability, and foot/ankle deviations

• Fit and foot/ankle entries include excessive mediolateral socket width, lateral instability, and excessive plantarflexion, with fit correction or alignment changes recommended.The dictionary also associates instability with inadequate lateral containment and describes plantarflexion during stance or initial contact.Source: D Gait Knowledge Dictionary · pp. 57–59

D Gait Knowledge Dictionary

• The dictionary distinguishes excessive plantarflexion from dorsiflexion, linking these foot settings to gait effects and recommending plantarflexion adjustments at the ankle component.For excessive dorsiflexion, it specifies loosening the anterior set screw and tightening the posterior set screw.Source: D Gait Knowledge Dictionary · p. 59
• Foot eversion or pronation may cause medial-border contact, whereas inversion or supination may cause lateral-border contact; recommended corrections adjust foot or ankle alignment.The listed actions include inverting the foot by loosening the lateral and tightening the medial set screw, or inducing eversion with the reverse adjustment.Source: D Gait Knowledge Dictionary · pp. 60, 62, 63
• The dictionary distinguishes normal gait requiring no action from unsuitable inputs and inconclusive frontal-plane assessments that require a direct frontal view for reevaluation.The inconclusive category covers assessment of alignment changes such as varus or valgus when the available view is insufficient.Source: D Gait Knowledge Dictionary · pp. 60–61
• Excessive socket adduction is associated with valgus gait, medial pylon lean or foot contact, and abnormal base width, with socket or foot alignment adjustments recommended.Suggested actions include abducting the socket or adjusting foot eversion or dorsiflexion.Source: D Gait Knowledge Dictionary · p. 62
• The entries also cover asymmetric knee flexion and prosthetic-knee abduction with external rotation, pairing these patterns with resistance or alignment corrections.Recommendations include lowering swing flexion resistance, internally rotating the knee component, and shifting the knee medially.Source: D Gait Knowledge Dictionary · pp. 63–64

• Normal gait maps to Healthy (Normal) Gait.Source: E Gait Category Mapping for GAVD · p. 64
• Antalgic and Stroke map respectively to Antalgic Gait and Hemiparetic (Hemiplegic) Gait.Source: E Gait Category Mapping for GAVD · p. 64
• Parkinson’s maps to Parkinsonian (Festinating) Gait, while Myopathic maps to Waddling or Trendelenburg Gait.Source: E Gait Category Mapping for GAVD · p. 64
• Cerebral Palsy maps to Spastic Diplegic (Scissors), Crouch, Spastic Paraparetic, or Hemiparetic (Hemiplegic) Gait.Source: E Gait Category Mapping for GAVD · p. 64

TVS workflow

• The TVS workflow assigns triage to normality assessment and hypothesis proposals, verification to deterministic tools, and synthesis to integrating verified evidence.Source: F DrGait Prompt Templates · p. 64

Global policy

• The prompts prohibit invented measurements and require quantitative claims to be grounded in provided metrics or tool outputs.The policy also says not to rule out a hypothesis from a single piece of evidence.Source: F DrGait Prompt Templates · p. 64

Output policy

• Each stage must follow its requested output schema, with triage applying the normality gate before differential diagnosis and returning JSON only.Source: F DrGait Prompt Templates · p. 65

Triage

• Triage requires a normality gate before proposing dictionary-based hypotheses and biomechanical verification objectives.The prompt specifies how to handle normal, abnormal, or uncertain cases and cautions against using step-length variation alone to confirm or rule out a hypothesis.Source: F DrGait Prompt Templates · p. 65

Verification

• Verification prioritizes high-priority objectives, uses declared tools purposefully, and permits only one tool-calling round.Source: F DrGait Prompt Templates · pp. 65, 67

Synthesis

• Synthesis must ground evidence in prior tool results, identify triage contradictions, preserve existing hypotheses and objectives, and report a conclusion with a suggested next step.Source: F DrGait Prompt Templates · p. 67

Additional gait measures

• The declarations also provide complete-cycle vertical COM displacement normalized by body height and bilateral symmetry indices for step length, stance time, or swing time.Foot progression is estimated from heel-to-toe landmarks during a complete-cycle phase and is an image-plane proxy in 2D contexts for toe-in/toe-out comparison.Source: G Tool Declarations · pp. 70, 71, 74

Event evidence

• Gait-event frames return up to five raw RGB frames from complete gait cycles, with optional view selection for segment-specific evidence in 2D contexts.Each returned entry includes a JPEG frame and metadata.Source: G Tool Declarations · pp. 71–72

Joint-angle measurements

• Joint-angle tools measure angles at specified gait events or max/min extrema within a gait phase, including cross-side combinations.In image-based 2D contexts, results are image-plane angles or extrema rather than calibrated 3D rotations; ankle angles may average toe and little-toe landmarks when explicit foot joints are unavailable.Source: G Tool Declarations · p. 72

Posture and sway

• Postural-angle measurements assess pelvis or trunk alignment at one frame, whereas postural-sway amplitude measures dynamic range over complete gait cycles.The declarations warn against using a single-frame posture angle to measure rhythmic sway and identify sway amplitude as relevant to waddling, ataxic instability, or Trendelenburg compensations.Source: G Tool Declarations · pp. 71, 73

Limb-motion measures

• Other cycle-based tools measure signed arm-swing extrema, body-height-normalized minimum foot clearance, and shoulder-width-normalized lateral deviation of a sided lower-limb joint.In image-based 2D contexts, foot clearance and lateral deviation are image-plane proxies; arm swing uses the view best representing sagittal-plane motion when available.Source: G Tool Declarations · pp. 73–74

Tool Declarations

• The knee–ankle alignment tool estimates frontal-plane axis alignment over a complete-cycle phase for varus/valgus, pylon lean, and axis-incongruity hypotheses.Source: G Tool Declarations · p. 75
• The vertical center-of-mass tool measures total vertical body-center displacement normalized by body height over complete gait cycles.In image-based 2D contexts, its result is an image-plane proxy.Source: G Tool Declarations · p. 75
• The postural-sway tool measures the trunk or pelvis dynamic range of motion, defined as maximum minus minimum angle over complete gait cycles.For segmented 2D contexts, the view should best represent the requested anatomical plane.Source: G Tool Declarations · p. 76
• The symmetry tool calculates a bilateral symmetry index in percent for step length, stance time, or swing time.In multiview 2D contexts, available left/right apparent step-length and timing metrics are averaged.Source: G Tool Declarations · p. 76