MA-Bench: Towards Fine-grained Micro-Action Understanding

Introduction

• Earlier datasets span upper-limb-focused iMiGUE [43], SMG, and BBSI and full-body Micro-Action-52 [19], but MLLM video benchmarks lack dedicated micro-action coverage.The paper frames this gap against video benchmarks for descriptive QA, causal-temporal interactions, and other scenarios [65] [60] [14] [26] [44] [33] [54] [57] [46] [47] [21] [49].Source: §1 · pp. 1–2

Contributions

• MA-Bench evaluates fine-grained micro-action understanding through perceptual recognition, relational comprehension, and interpretive reasoning.Source: §1 · p. 2, Figure 1 · p. 2
• Its construction pipeline extracts body-part motion descriptors, uses them with prompts and MLLMs to generate structured captions, then uses those captions to create the benchmark and training corpus.Source: §1 · p. 2, Figure 1 · p. 2
• The annotation strategy integrates optical-flow and skeleton information to construct motion descriptors for each body part.Source: §1 · p. 2
• Evaluations of 23 MLLMs reveal that current models struggle to capture subtle motion patterns and fine-grained temporal dynamics.The paper identifies these limitations in models evaluated with MA-Bench.Source: §1 · p. 3
• MA-Bench-Train provides 20.5K videos with structured micro-action captions as supervision for model training and fine-grained motion understanding.Source: §1 · p. 3

2.1. Micro-Action Understanding

• Earlier datasets emphasized upper-limb actions, while MA-52 expanded coverage to 22K samples across 52 action-level and 7 body-level categories.iMiGUE [43] covers 32 micro-gesture categories from athlete press conferences; SMG supports recognition and detection with gesture and emotional-state annotations; the social-interaction dataset captures 15 body movements. MA-52 [19] was collected from professional psychological interviews.Source: §2.1 · p. 3
• Micro-action methods use representation learning, ensemble architectures, joint-embedding losses, prototype calibration, and skeleton-guided modulation to capture subtle cues and resolve ambiguity.Liu et al. [43] used KL divergence for intrinsic representations; Huang et al. developed hypergraph and multi-scale heterogeneous ensembles; Guo et al. [19] used joint-embedding loss; Li et al. [34] calibrated ambiguous samples against prototypes; Gu et al. [17] used motion-guided modulation.Source: §2.1 · p. 3

2.2. MLLMs for Video Understanding

• Existing video MLLM benchmarks and methods are not designed for micro-action challenges, motivating MA-Bench and MA-Bench-Train for systematic evaluation and advancement.Benchmarks include MVBench [33] for perception-to-reasoning capabilities and Video-MME [14] across domains, durations, and modalities. Methods include joint audio–visual encoding in Video-LLaMA [67], align-before-project in Video-LLaVA [40], progressive instruction tuning in VideoChat2 [33], and reinforcement learning in Video-R1 [13] and VideoRFT [53].Source: §2.2 · p. 3

3.1. Overview

• MA-Bench contains 1,000 videos across 52 categories and 12,000 QA pairs covering eight tasks; videos average 2.12 seconds, and categories have long-tailed distributions.Most videos last 1–3 seconds, and the benchmark emphasizes fine-grained hand and directional movements, temporal segmentation, and reasoning-oriented language.Source: §3.1 · pp. 3–4

3.2. Data Curation

• The benchmark samples MA-52 videos from 20 participants, while MA-Bench-Train provides 20,510 captioned videos from 166 participants with non-overlapping subjects.MA-52 contains 22,422 videos from 205 participants; the cross-subject setting separates training and benchmark participants.Source: §3.2 · p. 4

3.3. Task Definition

• The Perception–Comprehension–Reasoning architecture assesses micro-action understanding from hierarchical recognition through spatial-temporal relations to natural-language interpretation.Perceptual Recognition separates coarse-grained body-level motion patterns from fine-grained action-level distinctions; relational tasks assess action details, sequences, multiple-action dependencies, and part proximity.Source: §3.3 · p. 4
• Interpretive Reasoning includes objective movement description and reasoning-based explanation linking evidence to coarse- and fine-grained labels.Micro-Action Descriptive Understanding describes observed body parts and motions without inference, while Micro-Action Reasoning and Explanation provides a reasoning chain for labeling decisions.Source: §3.3 · pp. 4–5

3.4. Question-Answer Generation

• The caption pipeline combines CoTracker3 optical flow, YOLOv8x-Seg human-region segmentation, and skeleton data to produce body-part motion descriptors and structured captions.LLMs summarize individual body-part motion patterns, and LMMs synthesize them into fine-grained captions used to build MA-Bench-Train.Source: §3.4 · p. 5
• MA-Bench QA generation uses automatic question creation, LLM reflection verification, and manual rectification, yielding an average of 2.9 multiple-choice questions per video.Questions are generated from motion captions in task-specific formats, checked for alignment and logical consistency, then reviewed for factual accuracy, clarity, and unambiguous answers grounded in video evidence.Source: §3.4 · pp. 5–6

4.1. Experimental Setup

• The evaluation covers 23 released MLLMs in zero-shot settings, using eight sampled frames per video and accuracy for closed-ended tasks or GPT-4o [25] judging for open-ended tasks.MADU and MARE are scored from 0–5 across three dimensions each: action semantics, spatial relations, and temporal structure for MADU; coarse body-level labels, fine action-level labels, and causal reasoning for MARE.Source: §4.1 · pp. 6–7

4.2. Performance Analysis

• 50.70% average accuracy for Gemini-2.5-Flash [9] and 50.68% for MA-Bench-Train fine-tuned Qwen3-VL-8B [64] are the reported closed-ended results; the text calls Qwen best overall.The same passage reports VideoChat-Flash-Qwen2-7B [39] at 49.87% and InternVideo2.5-8B [56] at 44.09%; its claim that 50.68% surpasses 50.70% is numerically inconsistent.Source: §4.2 · p. 7
• 1.69 is the highest reported average open-ended score for fine-tuned Qwen3-VL-8B, compared with 0.76 for Gemini-2.5-Flash.The passage says Qwen outperforms all open-source counterparts and proprietary models, and attributes the reasoning gains to domain-adaptive supervision.Source: §4.2 · p. 7
• The authors conclude that domain-specific fine-tuning can improve visual discrimination and interpretative and descriptive ability for subtle human motions, allowing open-source models to match or exceed proprietary models.They base this conclusion on improvements reported across both closed- and open-ended tracks.Source: §4.2 · p. 7

5. Further Analysis

• Sampling 8 frames yields 46.97% average accuracy on closed-ended tasks and 0.75 on open-ended tasks, outperforming 4-frame sampling on most tasks without gains from 16 frames.The passage attributes the slight drop at 16 frames to possible visual redundancy and distraction of temporal attention; 8 frames is described as a trade-off between temporal coverage and computational efficiency.Source: §5 · p. 7
• Fine-tuning both the vision and language modules achieves the best averages: 50.68 on closed-ended tasks and 1.69 on open-ended tasks.Fine-tuning either module alone also yields average gains on both task types compared with the original model.Source: §5 · pp. 7–8

5.1. Qualitative Analysis

• Fine-tuned Qwen3-VL-8B identifies both “Head” and “Shaking head” actions with coherent reasoning, whereas zero-shot models often miss subtle cues or produce inconsistent explanations.The passage says the zero-shot models frequently confuse head- and limb-related micro-actions, and that current MLLMs still struggle with subtle cues and fine-grained temporal dynamics.Source: §5.1 · p. 8

• MA-Bench is the first comprehensive benchmark for fine-grained micro-action understanding, comprising 1,000 videos with 10,000 multiple-choice QA pairs across six tasks and 2,000 closed-ended QAs.Source: §6 · p. 8
• Evaluation of 23 modern MLLMs reveals substantial limitations in capturing the subtle, fine-grained temporal dynamics characteristic of micro-actions.Source: §6 · p. 8
• MA-Bench-Train comprises 20,510 annotated videos with detailed, structured micro-action captions and leads to significant improvements in micro-action understanding performance on MA-Bench.Source: §6 · p. 8