Source-linked AI summary
MA-Bench: Towards Fine-grained Micro-Action Understanding
Kun Li, Jihao Gu, Fei Wang, Zhiliang Wu, Hehe Fan, Dan Guo
TL;DR
Fine-grained micro-action understanding lacks dedicated MLLM benchmarks despite its relevance to emotional and human-behavior analysis. This paper introduces MA-Bench and MA-Bench-Train, then evaluates current models and fine-tunes Qwen3-VL-8B with structured supervision. The benchmark exposes substantial difficulty with subtle temporal dynamics, while training on MA-Bench-Train substantially improves performance, including reasoning-heavy evaluation.
Problem
Dedicated datasets for evaluating MLLMs on fine-grained micro-action understanding are lacking despite micro-actions’ importance for emotion analysis and affective computing.
Method
The paper constructs MA-Bench with 1,000 videos and a three-tier evaluation architecture, alongside MA-Bench-Train with structured micro-action captions for model training.
Results
Evaluations of 23 MLLMs reveal substantial difficulty with subtle temporal dynamics, while MA-Bench-Train substantially improves micro-action understanding performance.
Takeaways & Limitations
MA-Bench and MA-Bench-Train provide a foundation for evaluating and advancing fine-grained multimodal reasoning about subtle human motions.
Abstract
from arXiv · showhide
With the rapid development of Multimodal Large Language Models (MLLMs), their potential in Micro-Action understanding, a vital role in human emotion analysis, remains unexplored due to the absence of specialized benchmarks. To tackle this issue, we present MA-Bench, a benchmark comprising 1,000 videos and a three-tier evaluation architecture that progressively examines micro-action perception, relational comprehension, and interpretive reasoning. MA-Bench contains 12,000 structured question-answer pairs, enabling systematic assessment of both recognition accuracy and action interpretation. The results of 23 representative MLLMs reveal that there are significant challenges in capturing motion granularity and fine-grained body-part dynamics. To address these challenges, we further construct MA-Bench-Train, a large-scale training corpus with 20.5K videos annotated with structured micro-action captions for fine-tuning MLLMs. The results of Qwen3-VL-8B fine-tuned on MA-Bench-Train show clear performance improvements across micro-action reasoning and explanation tasks. Our work aims to establish a foundation benchmark for advancing MLLMs in understanding subtle micro-action and human-related behaviors. Project Page: https://MA-Bench.github.io
1. Introduction
Micro-action analysis is important for interpreting emotional and social behavior, yet existing datasets and MLLM benchmarks leave fine-grained micro-action understanding underexplored. MA-Bench and MA-Bench-Train address this gap through structured motion descriptors, tiered evaluation, and training supervision.
- Micro-actions are spontaneous body movements associated with emotional changes and can support human communication and emotional-state analysis.
- Existing micro-action datasets mainly emphasize upper-limb actions, limiting coverage of broader fine-grained body dynamics.
- Dedicated datasets for evaluating MLLMs on fine-grained micro-action understanding remain lacking despite micro-actions’ relevance to emotion analysis and affective computing.
- MA-Bench and MA-Bench-Train use body-part motion descriptors, structured captions, and three evaluation categories: perceptual recognition, relational comprehension, and interpretive reasoning.
- MA-Bench-Train contains 20.5K videos with structured micro-action captions, while evaluations of 23 MLLMs reveal difficulty capturing subtle motion patterns and fine-grained temporal dynamics.
2. Related Work
Prior micro-action research has developed datasets and specialized recognition methods, while MLLM video benchmarks cover broader perception and reasoning settings. Fine-grained micro-action understanding remains a distinct benchmarking need.
- Datasets: Existing micro-action datasets include iMiGUE, SMG, and BBSI, spanning identity-free gestures, stress-related clips, and multi-person bodily behaviors.
- Methods: Prior methods address representation learning, subtle temporal cues, semantic alignment, and ambiguity through unsupervised, hypergraph-convolution, joint-embedding, prototype-based, and motion-guided models.
- MLLM benchmarks: MLLM video benchmarks evaluate capabilities including perception, temporal and causal understanding, diverse domains, long videos, ego activities, and fine-grained motion.
- MLLM benchmarks: These existing resources provide broad coverage but do not specifically establish a benchmark for fine-grained micro-action understanding by MLLMs.
3. MA-Bench
MA-Bench is a 1,000-video benchmark with 12,000 QA pairs organized around progressively harder perception, relational comprehension, and interpretive reasoning tasks. Its construction combines body-part motion descriptors with semi-automatic caption and QA generation.
- 3.1. Overview: MA-Bench contains 1,000 videos covering 52 micro-action categories and 12,000 QA pairs across eight fine-grained understanding tasks.Videos average 2.12 seconds, with most micro-actions lasting 1–3 seconds.
- 3.3. Task Definition: The Perception–Comprehension–Reasoning architecture progresses from basic recognition to spatial-temporal relational reasoning and interpretive explanation.
- 3.3. Task Definition: Perceptual Recognition separates coarse-grained body-level categorization from fine-grained action-level discrimination using nuanced spatial-temporal cues.
- 3.3. Task Definition: Relational Comprehension examines how body parts interact and coordinate through YES/NO questions targeting structural or temporal reasoning.
- 3.4. Question-Answer Generation: The annotation pipeline converts optical flow and skeleton information into body-part motion descriptors, then uses LLM generation, reflection verification, and manual rectification to create QA pairs.
- 3.4. Question-Answer Generation: Each video receives an average of 2.9 multiple-choice questions, while Table 2 reports Acc. and AVG across evaluated MLLMs.
4. Experiments
The experiments evaluate 23 proprietary and open-source MLLMs on MA-Bench using hierarchical scoring for body-level labels, action-level labels, and causal reasoning. Fine-tuned Qwen3-VL-8B achieves the strongest reported performance across closed- and open-ended evaluations.
- Experimental Setup: 23 MLLMs spanning proprietary and open-source models are evaluated in a zero-shot setting on MA-Bench.The evaluation covers varied model sizes, architectures, and training methodologies.
- Evaluation Protocol: L1 evaluates coarse-grained body-level labels, L2 evaluates fine-grained action-level labels, and L3 evaluates causal reasoning-chain consistency on a 0–5 scale.Both MADU and MARE use this hierarchical scoring structure.
- Closed-ended Evaluation: Gemini-2.5-Flash achieves 50.70% average accuracy, while fine-tuned Qwen3-VL-8B reaches 50.68% in closed-ended evaluation.The fine-tuned Qwen3-VL-8B result is reported as the best overall model in the cited comparison, despite the displayed Gemini figure being slightly higher.
- Open-ended Evaluation: Fine-tuned Qwen3-VL-8B achieves the highest open-ended average score of 1.69, compared with 0.76 for Gemini-2.5-Flash.The result indicates a large advantage for the fine-tuned model on reasoning-heavy evaluation.
- Performance Analysis: Domain-adaptive supervision improves both visual discrimination and interpretative and descriptive ability for subtle human motions.The reported gains span both closed- and open-ended tracks.
5. Further Analysis
Further analyses examine frame sampling, fine-tuning components, and qualitative reasoning behavior. Eight frames provide the best reported frame-based result, while MA-Bench-Train improves quantitative scores and produces more coherent micro-action explanations.
- The Influence of Video Frames: 8 frames yield 46.97% closed-ended accuracy and 0.75 open-ended performance for Qwen3-VL-8B, outperforming the tested frame settings.Increasing frames from 4 to 8 improves most tasks, whereas 16 frames provide no additional gains and slightly reduce performance.
- Qualitative Analysis: The MARE prompt asks models to infer coarse- and fine-grained micro-action labels through a detailed reasoning chain.The example concerns body-level and action-level prediction from observed actions.
- Qualitative Analysis: The ground-truth reasoning identifies shaking head from dominant head amplitude, continuous left-right oscillation, midline-spanning motion, and coordinated head–torso sway.Secondary body-part motions are described as low-amplitude and non-conflicting.
- Fine-tuning Analysis: Fine-tuning the whole module reaches average scores of 50.68 for closed-ended tasks and 1.69 for open-ended tasks.The cited ablation reports further improvements after fine-tuning both components.
- Qualitative Analysis: Fine-tuning on MA-Bench-Train produces more accurate and coherent reasoning that correctly identifies the Head and Shaking head actions.The qualitative analysis contrasts this behavior with models that were not fine-tuned on MA-Bench-Train.
6. Conclusion
The paper introduces MA-Bench and MA-Bench-Train to benchmark and train MLLMs for fine-grained micro-action understanding. Evaluations reveal substantial difficulty with subtle temporal dynamics, while the training corpus substantially improves performance.
- Conclusion: MA-Bench contains 1,000 videos, 10,000 multiple-choice QA pairs across six tasks, and 2,000 additional closed-ended QAs.The benchmark evaluates fine-grained micro-action understanding in both open-ended and closed-ended formats.
- Conclusion: Evaluation of 23 modern MLLMs reveals substantial limitations in capturing the subtle, fine-grained temporal dynamics characteristic of micro-actions.The conclusion frames this difficulty as a central benchmark finding.
- Conclusion: MA-Bench-Train comprises 20,510 annotated videos with detailed structured micro-action captions and leads to significant improvements on MA-Bench.The corpus is presented as the paper’s training resource for addressing the observed understanding gap.