Source-linked AI summary
Recognition-Conditioned Reasoning: A Training-Free Multimodal-LLM Pipeline for Fine-Grained Micro-Action Understanding
Fengshun Wang, Jin'ang Han, Zhigang Tu
TL;DR
Micro-action understanding must identify subtle movements while producing faithful spatial, temporal, and causal explanations, yet frozen MLLMs often confuse neighbouring fine categories. The paper presents a prompt-only pipeline that routes eight sub-tasks to task-specialized frozen MLLMs and conditions generation on discriminative predictions. It wins MA-Bench, with open-ended performance averaging 2.68 versus 1.44 for the runner-up.
Problem
Micro-action understanding requires fine-grained labels and faithful descriptions or reasoning, while frozen MLLMs confuse neighbouring categories and may produce fluent explanations with wrong labels.
Method
The training-free pipeline routes each of eight sub-tasks to the empirically strongest frozen MLLM and injects discriminative coarse and fine predictions into generative reasoning prompts.
Results
2.68 versus 1.44: the pipeline achieves the higher open-ended average and wins first place on MA-Bench.
Takeaways & Limitations
Recognition-conditioned reasoning separates discrimination from explanation and localises the remaining reasoning bottleneck to label accuracy.
Abstract
from arXiv · showhide
Micro-actions are subtle, short, low-amplitude body movements, such as a fidgeting hand or a slight head tilt, that humans perform with little conscious intent yet that reliably leak emotional and psychological state. Understanding them goes beyond assigning a label: a model must also describe which body parts move and reason, faithfully, about why a clip warrants a particular fine-grained category. We present the training-free, prompt-only system that won first place in the fine-grained understanding track (MA-Bench) of the MAC~2026 Micro-Action Challenge, where both fine-tuning and ground-truth supervision are disallowed. Built entirely upon frozen multimodal large language models (MLLMs), the system dynamically routes each of the eight sub-tasks to the MLLM empirically best suited for that task: a discriminative MLLM for closed-ended recognition tasks and a generative MLLM for open-ended description and reasoning tasks. This architecture achieves a statistically significant performance advantage on open-ended tasks, attaining an average score of 2.68 (on a five-point scale) compared to 1.44 for the second-best approach.
1 Introduction
Micro-action understanding requires fine-grained recognition, spatial-temporal description, and faithful reasoning under a no-training constraint. The paper addresses these challenges by routing tasks across frozen MLLMs and conditioning explanations on predicted labels.
- Micro-actions are brief, subtle, low-amplitude movements that can convey emotional and psychological state.
- MA-Bench spans eight sub-tasks, requiring category recognition, spatial-temporal judgements, body-part descriptions, and causal reasoning.
- Frozen MLLMs often confuse neighbouring fine categories and can produce fluent explanations that assert incorrect labels.
- The system routes closed-ended recognition to a discriminative MLLM and open-ended description and reasoning to a generative MLLM.
- 23 and 28 points are gained in coarse and fine label accuracy when explanations are conditioned on predicted labels.
- The training-free pipeline wins first place, with an open-ended average of 2.68 versus 1.44 for the runner-up.
2 Related Work
Prior work positions micro-actions between conventional action recognition and micro-expression analysis, while frozen video MLLMs shift task-specific capacity toward prompting. This paper extends that direction through task-adaptive routing and recognition-conditioned generation.
- Micro-Action and Nonverbal Behavior Understanding: Micro-actions are whole-body, low-amplitude behaviors distinct from large intentional activities and facial micro-expressions.
- Video Foundation Models: Frozen video models use sampled frames and inference-time prompting instead of task-specific training, making frame selection a central design lever.
- Multimodal LLMs for Video Understanding: Video MLLMs perform well on coarse semantics but remain weak on fine-grained and temporally precise judgements.
- Prompting and LLM-based Evaluation: Prompting and chain-of-thought can improve reasoning, but multimodal reasoning may harm perception-heavy tasks and explanation quality can diverge from label correctness.
- Prompting and LLM-based Evaluation: The paper realizes recognition-conditioned reasoning as a training-free cascade from a discriminative frozen MLLM to a generative frozen MLLM.
3 Methodology
The methodology formulates eight MA-Bench sub-tasks and solves them with frozen MLLMs selected and prompted per task. Recognition-conditioned reasoning separates label prediction from explanation by injecting discriminative predictions into generative prompts.
- 3.1 Task Formulation and Overview: MA-Bench contains two multiple-choice recognition tasks, four yes/no spatial-temporal judgements, and two open-ended generation tasks.
- 3.1 Task Formulation and Overview: Closed-ended tasks use a discriminative MLLM, while open-ended description and reasoning use a generative MLLM selected through development-split benchmarking.
- 3.1 Task Formulation and Overview: The pipeline samples frames, fixes output formats through per-task prompts, and assigns decoding budgets according to answer type.
- 3.1 Task Formulation and Overview: Temporal-order tasks receive a larger frame budget because short samples otherwise provide insufficient ordering cues.
- 3.3 Recognition-Conditioned Reasoning: A single MLLM can narrate motion fluently while choosing a neighbouring fine label, motivating separation of discrimination and explanation.
- 3.3 Recognition-Conditioned Reasoning: The open-ended MADU prompt explicitly covers dominant action, spatial detail, and temporal structure for judging.
- 3.3 Recognition-Conditioned Reasoning: The discriminative prediction is injected into the generative reasoning prompt as a strong prior that can be justified or overridden when evidence contradicts it.
- 3.3 Recognition-Conditioned Reasoning: Generated reasoning is evaluated with judge-free label-mention accuracy using cross-task gold labels and synonym-normalised agreement.
4 Experiments
Experiments evaluate model selection, prompt and frame-sampling choices, recognition-conditioned reasoning, and final MA-Bench performance. The pipeline ranks first because strong open-ended description and reasoning outweigh slightly weaker closed-ended recognition.
- Dataset and Evaluation: 12,000 questions over eight sub-tasks are evaluated using exact-match closed-ended accuracy and language-judge scores for open-ended answers.The benchmark comprises 1,000 short single-person clips, while open-ended dimensions are scored from 0 to 5 and scaled by 20 for the overall score.
- Per-task model selection: gpt-5.5 is selected for closed-ended tasks, while qwen3.7-plus is selected for open-ended description and reasoning.On the development split, gpt-5.5 reaches a closed-task struct-average of 0.61, while qwen3.7-plus reaches mean open-ended judge quality of 2.66 versus 2.62 for gpt-5.5.
- Frame sampling and image fidelity: 64 frames improve temporal tasks but hurt static judgements, so the pipeline uses 64 frames for MAS and MAD and 8 frames for the remaining tasks.On gpt-5.5, MAS rises from 0.58 to 0.64 and MAD from 0.54 to 0.58, while SAD falls from 0.72 to 0.62 and PPR from 0.62 to 0.48 when all sampled frames replace 8.
- Main Result: First Place on MA-Bench: 57.14 final score places the pipeline first, despite a lower closed-ended average of 60.73 versus GNC’s 63.78, because its open-ended average is 2.68 versus 1.44 for the runner-up.The open-ended route describes spatial paths and temporal phases and rules out nearby alternatives before reaching the same labels in the qualitative case study.
- Recognition-Conditioned Reasoning: Recognition-conditioned reasoning raises coarse label accuracy from 28% to 51% and fine label accuracy from 7% to 35% on development reasoning items.The oracle with correct injected labels reaches 79% coarse and 66% fine accuracy, indicating remaining headroom from recognition errors.
5 Conclusion
The paper presents a training-free, frozen-MLLM pipeline that routes each MA-Bench sub-task to the best-suited model and achieves first place through strong open-ended performance. Recognition-conditioned reasoning separates discrimination from explanation, while a judge-free metric addresses fluency bias and identifies label accuracy as the binding constraint.
- The pipeline routes each of the eight MA-Bench sub-tasks to the empirically best-suited frozen MLLM without fine-tuning or ground-truth supervision.
- Recognition-conditioned reasoning conditions a generative MLLM’s explanation on an external discriminative prediction, decoupling label discrimination from explanation.
- A judge-free, gold-grounded label-mention metric sidesteps LLM-judge fluency bias in evaluating open-ended micro-action reasoning.
- The system wins first place at MAC 2026, with its advantage concentrated on open-ended description and reasoning tasks.
- Conditioning improves development-split performance and localises the binding constraint on micro-action reasoning to label accuracy.